Subject: RESULT: WFC endorses BINTABLE/IMAGE/Blocking

From: dwells at fits.CV.NRAO.EDU (Don Wells)
Date: Thu, 3 Jun 1993 17:01:59 GMT

-=- WFC has approved the BINTABLE/IMAGE/Blocking proposals -=-
			Don Wells (WFC Chair)
			     June 1, 1993

The WFC [WGAS FITS Committee] has voted unanimously, 20-0, to approve
the BINTABLE, IMAGE and Blocking proposals which are currently being
considered by the regional FITS committees for adoption as additions
to the FITS standards. The WFC is the regional FITS committee for
North America. WGAS is the Working Group for Astronomical Software of
the American Astronomical Society. The voting membership of the WFC is
representative of North American observational astronomy, both
ground-based and space-based, at all observed frequencies.

These three proposals were approved by the European FITS Committee
[EFC] in April 1992, and the EFC reaffirmed their approval at their
annual meeting in Garching in April 1993. The Japanese FITS Committee
will vote on the proposals in the near future. If, as expected, they
also approve them, the proposals will be considered by the IAU
[International Astronomical Union] FITS Working Group, which controls
the FITS standards. The Chairs of the FITS committees hope, and
expect, that it will be possible to obtain IAU final approval of these
proposals in time to have a FITS resolution passed by the General
Assembly of the IAU at its triennial meeting at the Hague in August
1994.

		  -=- Anonymous-FTP Information -=-

Drafts of the three proposals are available at fits.cv.nrao.edu
[192.33.115.8] in directory /fits/documents/proposals/ as the files

     145877 May 18 16:29 bintable2.ps
      38591 May 18 16:27 bintable2.tex
       2183 Oct  1  1991 blocking90.txt
      94980 Jun  4  1992 image.ps
      12973 Mar 12  1992 image.tex

Information about the current FITS standards is in the directory
/fits/documents/standards/ as the files

       7895 Oct 26  1992 fp89.txt
     431252 Dec 29  1991 standard.ps
       8325 Dec 29  1991 standard_read.ME
       2710 Jan 24  1992 xtension.txt

	       -=- The BINTABLE extension proposal -=-

The 1979 Basic FITS Agreement specified an architecture for encoding
an n-dimensional matrix with a self-documenting syntax. The 1979
design allowed for the possibility that arbitrary object types might
be appended to the matrix. The FITS community has traditionally called
such appended objects "extensions". The 1984 Generalized Extensions
Agreement specified a set of meta-rules for the syntax of approved
extension types. The first extension conforming to the Agreement was
type TABLE, a self-describing syntax for tables in ASCII form which
was also negotiated in 1984. The BINTABLE (BINary TABLE) extension is
very similar to TABLE, but the data are encoded in binary rather than
ASCII. In addition to the obvious advantages of compactness and
processing efficiency, BINTABLE has a special superiority over TABLE:
fields in the table are allowed to be *vectors*, not just scalars.
This rule enables BINTABLE to encode sets of n-dimensional data
matricies in the table. In fact, arrays of arbitrary *structures* can
be encoded in BINTABLE, and can thereby gain the advantage of
portability in a self-documenting form. The requirement to append
extensions to the primary FITS matrix is not really a limitation on
the use of TABLE and BINTABLE (and other extensions), because the
rules of Basic FITS permit the primary matrix to have dimensionality
zero.

The prototype for BINTABLE was the experimental extension A3DTABLE,
which was developed by Bill Cotton for the VLBA [Very Long Baseline
Array] project in the mid-80s. The VLBA and the VLA [Very Large Array]
had several years of production experience with A3DTABLE in the 80s.
In the late 80s features were added, the wording of the draft document
was refined, and the FITS committees agreed to use the name BINTABLE
for this extension.  Interoperability trials occurred circa 1989.
More recently BINTABLE was selected as the interchange format for
photon event data in the high energy astrophysics community. The
committee votes on BINTABLE are acknowledging that BINTABLE is the de
facto interchange and archive standard for a number of critical
applications in astronomy.

The FITS Committees consider the basic BINTABLE architecture to be the
foundation for a variety of conventions to be negotiated in the
future.  For example, in 1991 Doug Tody and Bill Cotton proposed a
simple trick that enables BINTABLE to encode variable length data
structures in a backward compatible fashion (see Appendix A of the
bintable2 document mentioned above for the details). More recently
there has been extensive discussion of proposed conventions for
encoding matricies in fields (Appendix B) and for arrays of character
strings (Appendix C). All of these conventions will be layered on top
of BINTABLE. Discussions of these ideas during the past three years
has led to significant improvements in the BINTABLE draft. The
committee votes on BINTABLE are acknowledging that BINTABLE now
appears to be strong enough and flexible enough to support the layered
conventions that astronomy will surely need in the future.

		 -=- The IMAGE extension proposal -=-

The IMAGE extension also conforms to the Generalized Extensions
Agreement. It encodes an n-dimensional matrix. This extension is not
intended to supercede the matrix capability of Basic FITS, but rather
to augment it. IMACE enables datasystems to append things like PSFs
and validity masks to matricies, thereby keeping the related objects
together in one file. IMAGE was proposed by designers associated with
the IUE [International Ultraviolet Explorer] satellite.

	       -=- The proposed Blocking convention -=-

FITS was described as a "tape" format in the original FITS papers, but
it was redefined as a bitstream format in the 1984 Generalized
Extensions Agreement.  Blocking conventions must be negotiated for
each new type of medium which may be used to convey FITS bitstreams in
interchange.  The 1990 Blocking proposal codifies de facto blocking
conventions which have been in widespread use in the FITS community in
recent years.
--

  Donald C. Wells           Associate Scientist       dwells at nrao.edu
  National Radio Astronomy Observatory                +1-804-296-0277
  520 Edgemont Road                              Fax= +1-804-296-0278
  Charlottesville, Virginia 22903-2475 USA         78:31.1W, 38:02.2N