Subject: Re: Deprecation of Random Groups

From: pence at tetra.gsfc.nasa.gov (William Pence)
Date: Fri, 1 May 1998 13:46:55 -0400

Putting aside the issue of 'deprecation' for a moment, what would it
take for the radio interferometry community to switch from using random
groups to using binary tables for future data collection?   For
example, would it be possible for the major players in this field to
form a working group to work out the details for a binary table format
and on a time table for implementation (say by 1 Jan 2000, like the
Y2000 agreement)?

-Bill Pence

---

From: Bill Cotton <bcotton at nrao.edu>
Date: Fri, 1 May 1998 14:15:20 -0400 (EDT)

> and on a time table for implementation (say by 1 Jan 2000, like the

   This was attempted several years ago and basically failed for
reasons that had nothing to do with the merits of the different
formats.  A binary tables format for radio interferometry was
developed and widely circulated for comments.  The implementation of
this format was only partially complete and is limited to some VLBI
correlator outputs.
   The state of software politics in the radio astronomical community
is such that a change of formats is currently completely impossible. 

   From a purely technical point of view there is little to be gained
from switching from a Random Groups format to binary tables; the
difference is merely the file headers - the actual bit patterns could
be unchanged.  I'm afraid that we're stuck with the current situation
for the forseeable future. 

-Bill Cotton

---

From: pence at tetra.gsfc.nasa.gov (William Pence)
Date: Fri, 1 May 1998 16:43:27 -0400

Bill Cotton wrote:
> be unchanged.

That's true, and in fact the CFITSIO library
(http://heasarc.gsfc.nasa.gov/fitsio) internally interpretes random
groups as binary tables:  each group is a separate row in the virtual
table, with the group parameters in column 1 and the data array in
column 2 (in general both columns are vectors).  While CFITSIO does
contain  specialized routines for reading and writing group parameters
and grouped arrays, these just call the more general CFITSIO routines
which read and write data in binary table columns.  CFITSIO also treats
primary arrays and image extensions as binary tables in the same way,
but in these cases the 1st column (containing the group parameters) has
a vector repeat count = 0, and there is only one row in the virtual
table.  So really all FIT HDUs are binary tables as far as CFITSIO is
concerned.

Thus, it is possible for software that is based on CFITSIO to read and
write grouped data as though they were binary tables, even without
changing the current format of the files (e.g., to read the 4th group
parameter in the 2nd group, call the CFITSIO binary table reading
routine to read the 4th pixel element of column 1 in the 2nd row of the
virtual binary table).

-Bill Pence