Radio Galaxy Zoo Talk

Anyone know how to read files in HEASARC's tdat format?

  • JeanTate by JeanTate

    I have the IRAS Point Source Catalog, downloaded from this HEASARC website, as an unzipped ~64MB .tdat file (heasarc_iraspsc.tdat). This webpage describes the format (tdat? "Transportable Database Aggregate Table (TDAT) Format"), which is nice, but all I want is the data, plus some names for the columns. I could probably write Python code to read it (probably take me a day or two), but I thought I'd ask here, to see if anyone has already encountered this problem, and has a convenient solution.

    I'm also going to post this in GZ Talk, as well as the new Zooniverse Talk ...

    Posted

  • JeanTate by JeanTate in response to JeanTate's comment.

    BTW, the same data/catalog is also available from IRSA (in .tbl format) and Vizier/CDS (in .dat format), when unzipped. I have had no more success with either of these (I'm trying to get the catalog into TOPCAT).

    ETA: If I could get a FITS version of the catalog, that'd be wonderful.

    Posted

  • c_cld by c_cld

    You could use the query service http://heasarc.gsfc.nasa.gov/db-perl/W3Browse/w3table.pl?tablehead=name%3Diraspsc&Action=More+Options and choose the output format

    Posted

  • JeanTate by JeanTate in response to C_cld's comment.

    Thanks C_cld.

    All three of the sources offer similar functionality, and at least two also offer source extraction via file upload (with limits!). The file upload/batch method is one (of two) approaches I'm now considering ... I have, for now, just ~nk sources/positions that I want data on/from (n<3), which should be manageable; however, I'd prefer to have the whole catalog on my own machine, in a TOPCAT-readable form, so I'm leaning towards the solution Brooke suggests (in the Z Talk thread), which is to write code to strip the header lines out (and add one with column names). That would should be a whole lot easier and quicker.

    Posted

  • JeanTate by JeanTate in response to JeanTate's comment.

    Just posted this comment in the Z Talk thread I started on this:

    Update: I ended up taking the 'hacking' approach Brooke suggested, on the HEASARC .tdat file, using what I expected to be a most unlikely tool, Win7's Notepad! 😮

    It was rather more difficult than I had expected, for a quite (for me) unexpected reason: the file header - which contains detailed info on the fields (among other vital things) - contains an error. And not a trivial one*. Some possible implications of this are pretty mind-blowing: no astronomer (or other scientist) has found this error before? No astronomer (or other scientist) has downloaded the IRAS data from HEASARC (or at least the 'main' data file, in .tdat format)? (The file seems to have been last modified in Feb 2007!)

    Anyway, as I now have the data I want, in a format I can use (i.e. I can read it into TOPCAT) - albeit in a less-than-perfect form - I will not be following up with Vizier/CDS' .dat format file, nor IRSA's .tbl one.

    *I'll explain what it is, in some detail, if anyone's interested. Oh, and the error is also mirrored in HEASARC's traditional Table browser interface, for IRAS

    Posted