Subject: Binary Table Structure for Spectra

From: [email-masked] (Don Wells)
Date: Mon, 16 Oct 89 17:03:26 EDT

Dear Friends-of-FITS:

The paper below is a proposal to the single-dish radio community, 
which could adopt it even if the rest of astronomy doesn't want to do so.
I, of course, hope that it may be possible to forge an alliance across
all of astronomical spectroscopy.

My actual proposal is only the first 9 pages. The remaining 23 pages are
appendices containing the text of the Floating Point Agreement, the 
Grosbol/Wells Hierarchical Keyword draft proposal and a detailed 
description of AIPS support for single-dish random-scan data and
Cotton's 3-D Binary Table extension format.

Donald C. Wells, Associate Scientist | NSFnet: [email-masked] [192.33.115.2]
National Radio Astronomy Observatory | SPAN:   NRAO::DWELLS    [6654::]
Edgemont Road                        | BITnet: DWELLS@NRAO
Charlottesville, VA 22903-2475 USA   | UUCP:   ...!uunet!nrao.edu!dwells
+1-804-296-0277  (38:02.2N/78:31.1W) | TWX=510-587-5482, Fax=+1-804-296-0278

------------------- LaTeX, 32 pages, cut here -------------------------
\documentstyle[11pt,twoside]{article}
\pagestyle{headings}
\raggedbottom
% 
\begin{document}

\title{A Binary Table Structure for Spectra}

\author{Donald C. Wells\\
{\small \raisebox{0.1cm}{\rule{0cm}{0.3cm}}NRAO, Charlottesville, VA}\\
{\small 804-296-0277, {\tt [email-masked]}}}
\date{{\em DRAFT} \today}
\maketitle

\begin{abstract}
The 3-D binary table FITS extension is proposed as an efficient format
to transmit groups of astronomical spectra and associated parameters.
Syntactic conventions for keywords and table column labels are
suggested.
\end{abstract}

\tableofcontents
\clearpage

\section{The Problem}

The basic FITS (Flexible Image Transport System) Agreement of 1979
defined a syntax for transmitting N-dimensional matricies. It was (and
still is) perfectly capable of transmitting spectra, for which N=1, as
vectors of binary data described by ASCII headers using the
keyword=value notation. However, such usage is inefficient: in most
cases more bytes are needed for the header than for the binary vector!
Spectroscopy observations generally need more ancillary parameters,
more keyword=value pairs, than do imaging observations and this only
exaggerates the disproportion between header and data for
spectroscopy. The basic FITS format is simply not efficient for
spectroscopy applications.

Spectroscopy observations often involve large numbers of short
integrations, which implies that large numbers of ASCII headers must
be written, transmitted, archived and processed. Although many people
have wished that the vectors as well as the headers could be
transmitted in ASCII form so that {\em everything} would be
human-readable, this is simply not a practical option for our new data
systems in which more and more feeds are analyzed by spectrometers
with higher and higher resolution.  Binary notation is essential for
the spectra, and we must strive to reduce the overhead of the headers.
This means that even the headers must be in binary, and redundancy
must be minimized.

\section{The Solution}

We need binary data structures which include the ancillary parameters
in binary form, bound to the binary vectors.  These structures must
be described in FITS style in order to make them {\em flexible.} The
ASCII description will require almost as many bytes as would a basic
FITS header. To minimize this overhead we need to be able to transmit
{\em arrays} of the structures. In this concept only one copy of the
ASCII description is needed for an arbitrarily large number of binary
spectra plus parameters, all grouped together in one FITS file.

There are two existing FITS formats that have the desired property:
the random groups format\footnote{Greisen, E.W., Harten, R.H.: 1981,
Astron. Astrophys. Suppl. Ser. 44, 371, "An Extension of FITS for
Groups of Small Arrays of Data".} and the 3-D binary tables
format\footnote{Cotton, circa 1986, coded in AIPS; see
Appendix~\ref{app:C3DBTF} and Section~\ref{sect:C3DBTF}.}. There is no
question that random groups are a feasible solution, but binary 3-D
tables are preferable, for two reasons. First, experience shows that
many people have difficulty understanding the random groups concept
but everyone claims that they understand tables intuitively. Second,
the random groups format requires that the same data type be used for
both spectra and parameters but the 3-D binary tables allow mixed data
types, a great convenience and valuable notational power. I propose
that Cotton's 3-D binary table format be adopted for astronomical
spectroscopy data interchange.

\section{Conventions}

The basic concept that I am proposing is that a set of spectroscopic
observations be regarded as a {\em table}, with the spectral vectors
appearing as one of the columns of the table and the ancillary
parameters appearing in other columns. The following subsections
propose certain conventions for the organization of the table.

\subsection{Minimizing Redundancy}\label{sect:redundant}

Many of the parameters associated with a set of spectroscopic
observations are constant for the whole set. If each such parameter
must appear in a  separate column of the table the same bytes will
appear over and over. I propose that such parameters be moved to the
extension header and encoded as keyword=value, and that they be
regarded as a {\em virtual column.}

\subsection{Multiple Feeds}

Many spectroscopic instruments today observe several positions on the
sky simultaneously. There are two possible encoding conventions.
First, we could encode such observations as one structure containing
several binary vectors, with many parameters shared between the
spectra and many other parameters appearing once for each of the
spectra. Alternatively, we could encode the several spectra as
separate structures, with the parameters which are identical being
duplicated but without the notational complexity of the multiple
instances of spectra and parameters in one structure. I believe that
the latter case, the separate structure scheme, is easier to design
and to implement, and so I propose that it be adopted.

Some spectrometers measure more than one band simultaneously. This
case is analogous to multiple feeds and the separate structure
approach handles it with no new notation.

In some cases datasets contain spectra with different numbers of
channels. I propose that the number of channels be specified in a
column. If all spectra have the same length the column need not be
carried with the spectra in the structures --- it can be given once in
the extension header (see section \ref{sect:redundant}).

\subsection{Hierarchical Table Labels}

Table column labels pose the same problems of potential name conflict
as do keywords.  It appears that the only feasible solution will be
some kind of hierarchical classification scheme. The two draft
proposals available currently are the Hierarchical Keyword Agreement
(the ``domain-name-system'' proposal) of Grosb{\o}l and Wells, whose
text is given below in Appendix~\ref{app:HKA}, and the
``push-down-stack'' proposal of Tody and Hanisch, as discussed in
Hanisch's DISHFITS-exploder message of 05~October. In the remainder of
this paper I will discuss the Grosb{\o}l/Wells notation because it
leads to a natural generalization for table labels. I have not yet
decided whether the Tody/Hanisch proposal offers an equally suitable
solution for this application.

The FITS ASCII Tables standard\footnote{Harten, R.H., Grosb{\o}l, P.,
Greisen, E.W., Wells, D.C.: 1988, Astron.  Astrophys. Suppl. Ser. 73,
365, "The FITS Tables Extension".} does not specify any limit
on the length of column label strings. This is yet another example of
the continuing need for interpretation of the rules of the standards.
Let us assume that the string-value rule of the Basic FITS
Agreement\footnote{Wells, D.C., Greisen, E.W., Harten, R.H.: 1981,
Astron. Astrophys. Suppl.  Ser. 44, 363, "FITS: A Flexible Image
Transport System".  See p.~364-365.} applies: string values may be of
any length which fits on one card (80 characters), but should be
unique/useful if truncated to 8 characters.  I therefore propose that
hierarchical classification codes be used to obtain uniqueness in the
labels, {\em but that the order be reversed so that the last
substring, the keyword, will be in the first 8 characters.} If the
same code string is used as a keyword in the extension header, as
proposed in Section~\ref{sect:redundant} above, it will appear in
normal order, with the {\tt HIERARCH} code in the first 8 columns and
the keyword at the end of the string, just before the equals sign.

\subsection{Units}

I strongly recommend that only SI units be used in data interchange in
astronomy, as specified in the Basic FITS Agreement.

\subsection{Spectral Format Table Labels}

I propose that column names and parameter values should be the keyword
and axis-type names used previously in FITS designs, wherever
possible. By this I mean that in situations where in other proposals
keywords are used in FITS headers that those same names be used in
table column labels for tables of spectra, and that where various
conventions are specified for the value fields of those keywords that
those same conventions be used for the value fields in those columns.
Thus, I am proposing that the parameter columns be regarded as {\em
virtual header cards}.

\section{Status of Relevant Draft Standards}

\subsection{Generalized Extensions, ASCII Tables}

The two FITS standards were published in Astron.~Astrophys.~Suppl. in
1988, and were approved by IAU Commission~5 at the Baltimore meeting
in August 1988.\footnote{IAU Resolution B1, p.~10,
IAU Information Bulletin 61, January 1989.}

\subsection{The Floating Point Agreement} The FPA has been discussed
for at least four years in committees. The prototype implementations
were done circa 1985-6. The final text was adopted 28~February~1989,
and is given in Appendix~\ref{app:FPA}. The FPA was endorsed by the
European Working Group for Coordination of Astronomical Software in
May 1989, and by the AAS Working Group for Astronomical Software in
June 1989. The Agreement is now under consideration by the IAU FITS
Working Group, which is awaiting the final interchange and
interoperability proof for the 64-bit data type.  Activation is
anticipated for 1990.

\subsection{Binary 3-D Table Format}\label{sect:C3DBTF}
Bill Cotton designed his table format as an extension of the standard
ASCII table format in order to represent certain complicated
calibration table structures needed for the VLBA analysis software.
The only description currently available is from section 14.5.3 of
``Going AIPS'', the AIPS programmer's reference manual. This text has
been extracted from the AIPS directories and is given below in
Appendix~\ref{app:C3DBTF}.  This extension was implemented in AIPS
circa 1986, and code supporting it has been operational on several
hundred computers for several years now. The extension was implemented
in MIDAS circa 1987, and AIPS-MIDAS interoperability trials have been
made. As of 1989 the extension is being implemented as a format for
photon lists in high energy astronomy, first for the ROSAT data
analysis software.  The 3-D Binary Table design is generally believed
by the FITS standards committees to be the draft proposal ``most
likely to succeed''. Only one or two trivial changes are anticipated
when it finally passes the IAU sometime in the future.

\subsection{Hierarchical Keyword Agreement} The original idea was 
implemented by Eric Greisen in the first FITS software for the
NRAO-Charlottesville IBM synthesis mapping package in 1979. A more
recent prototype circa 1986-7 is the SNGLDSH format at NRAO. The
concept has been discussed in the FITS committees since 1987. A
current draft proposal by Grosb{\o}l and Wells dates from
28~February~1989; it is reproduced below in Appendix~\ref{app:HKA}.
Alternative concepts are still being offered. At present no clear
consensus exists favoring any one proposal.

\section{An Example Showing Packing Efficiency}

The preceding discussion has proposed Cotton's FITS extension as a
solution appropriate for all spectroscopic applications in astronomy.
At present the main astronomy discipline considering this problem is
single-dish radio astronomy. There has been much preliminary work
involving keywords, syntax and even semantics for this application,
and it need not be repeated here. Indeed, even the header syntax for
the 3-D extension is well documented in an appendix to this paper, and
so I choose not to give a full example.  The important point to be
addressed is the {\em packing efficiency} which is possible for the
proposed design..

Consider a radio spectrometer with 1024 channels in floating point
form. A data file will start with a basic FITS header which has no
data matrix ({\tt NAXIS=0}). This will be followed by the extension
header, which will specify {\tt XTENSION='A3DTABLE'} (once the IAU
approves this extension the value will probably be {\tt '3DTABLE'}).
Each spectrum will be encoded as a binary data structure which is
described with the notational machinery of the 3-D tables. A grossly
simplified data structure which has an example of each data type might
be:
\begin{center}
\begin{tabular}{|l|l|l|l|l|l|l|}
\hline
label:	&{\tt SCAN}	&{\tt SOURCE}	&{\tt BASEFREQ}	
&{\tt DELTFREQ}	&{\tt NCHAN}	&{\tt SPECTRUM}	\\
\hline
format:	&{\tt 1J}	&{\tt 20A}	&{\tt 1D}	
&{\tt 1F}	&{\tt 1I}	&{\tt 1024F} \\
\hline
bytes:	&4		& 20		& 8
& 4		& 2		& 4096 \\
\hline
\end{tabular}
\end{center}
The {\tt SCAN} number is a 32-bit integer, the {\tt SOURCE} name is a
string, the frequency of the reference channel (in Hertz, of course)
is a double precision number and the frequency increment is single
precision. The spectrum itself is single precision floating point and
the number of channels {\tt NCHAN} is specified by a 16-bit integer
(note that the {\em maximum} number of channels is given by the format
of the {\tt SPECTRUM} field; {\tt NCHAN} might be less than this
number). {\em The column labels used here are not to be regarded as a
part of my proposal; rather, if my 3-D table concept is accepted for
astronomical spectra an ad hoc task force should negotiate these
details.} The binary data types can be flexible: one site might encode
{\tt SCAN} numbers as 16-bit integers while another used 32-bits.
Likewise, some implementations may choose to encode spectra using
16-bit integers rather than 32-bit reals in order to save bytes in the
files (the FITS machinery for integer scaling permits this to work
even for inherently non-integer values).

For this example there will be 2880 bytes in the basic header, 5760
bytes in the extension header, 4134 bytes per spectrum and one
tapemark.  It is true that a real spectral observation will probably
need about 50 parameters, not 5, but even at 4 bytes per parameter
this will add only 200 more bytes per spectrum. Therefore, it will
still be true that for one spectrum the header overhead will dominate.
Indeed, 50 more columns in the table will require several more logical
records of extension header to document, making the imbalance even
worse!  However, additional spectra will add no more overhead. If the
table contains as many as 20 spectra the efficiency will exceed 90\%.

\section{Unresolved Issues, Other Ideas}

\begin{itemize}

\item{\bf Hierarchical Keywords:} There is no consensus yet on the 
syntax, or even on the necessity of a hierarchical data space.

\item{\bf Vocabulary, Semantics, Units:} This proposal is only 
concerned with the syntax for transmitting spectra. Each discipline of
astronomy must agree on the vocabulary and on the semantics and units
for its special language.

\item{\bf World Coordinates:} We must agree on conventions for 
describing the mapping between pixel number and frequency in the
spectra. I recommend that we utilize notations similar to those
devised by Eric Greisen for imagery. This means that we need to
identify how many types of mappings occur in astronomical instruments
and assign names to them.  We must analyze their mathematical forms to
determine how many parameters each has and attempt to devise a uniform
convention for the notation.  An important question will be whether
all of astronomy can agree to use frequency as the independent
variable for spectroscopic data interchange; if not, we will have both
wavelength and frequency (wavenumbers are just frequency by another
name, of course). All of these world coordinates issues are beyond the
scope of this paper.

\item{\bf Higher-Level Organization of Datasets:} Multiple named table 
extensions may be grouped together in one file --- this provides a
higher level organizational tool. The Generalized Extensions standard
also specifies keywords and conventions for a hierarchical structure
of extensions, which is yet another higher level organizational tool.

\item{\bf Auxilliary Tables:} The FITS table extensions could be 
used to transmit calibrations, source lists, etc., in addition to the
spectra --- this could permit the elimination of even more redundancy
in the sense of ``normalizing'' a relational DBMS schema.

\end{itemize}

\appendix

% ==================================================================
\clearpage
\section{Floating Point Proposal for FITS}\label{app:FPA}

\begin{center}Don Wells, Preben Grosb{\o}l\end{center}

\begin{center}28 February 1989\footnote{This 
text is very slightly revised from the version that was endorsed by
the European FITS Committee in April 1989 and by the AAS Working Group
for Astronomical Software in June 1989. It will now be reviewed by the
IAU FITS Working Group for adoption as a part of the FITS standards,
and will probably be activated in 1990.}\end{center}

\subsection{The Proposal}

The Basic FITS, Random Groups and Generalized Extensions Agreements
are revised to add IEEE-754 32- and 64-bit floating point numbers to
the original set of FITS data types.  BITPIX=-32 and BITPIX=-64
signify 32- and 64-bit IEEE floating point numbers; the absolute value
of BITPIX is used for computing the sizes of data structures. The full
IEEE set of number forms are allowed for FITS interchange, including
all special values (e.g., the 'not-a-number' cases). The order of the
bytes will be sign and exponent first, followed by the mantissa bytes
in order of decreasing significance.  The BLANK keyword will be
ignored by FITS readers when BITPIX=-32 or -64.

\subsection{Discussion}

The dynamic ranges of astronomical data have increased steadily in the
years since the original FITS Agreement (March 1979) which defined 8-, 16-
and 32-bit integers as the only data formats. All integer formats are
limited by an absolute error, whereas floating point formats can span a
much wider numerical range with only a relative error. Floating point
formats are desirable for the applications that need increased dynamic
range. Many astronomical data systems now use floating point in data
processing, and conversion to and from FITS integer formats sacrifice
accuracy and dynamic range and consume significant computer time.  Almost
all of the new computers designed since about 1981 use the IEEE format
(see references below). We are proposing that FITS be revised to allow
transmission of 32- and 64-bit floating point data within the FITS format
using the IEEE standard. This Floating Point Agreement will also apply to
the Random Groups Extension and to those Generalized Extensions for which
BITPIX is not explicitly restricted (e.g., BITPIX=8 for XTENSION='TABLE').

In order to transmit floating point data in the main data matrix we must
add some new convention to the main FITS header. We propose to add two new
allowed values for the basic keyword BITPIX. The set of values specified
in the original FITS Agreement was 8, 16 and 32; we propose to add -32 and
-64 to indicate IEEE single- and double-precision floating point data.
Thus, the number of bits per pixel would be the absolute value of BITPIX.
This implies that the number of 2880-byte logical records used by the main
data matrix must be computed using the absolute value of BITPIX (this rule
was specified in the Generalized Extensions Agreement (Grosb{\o}l etal.
1988)).

The IEEE standard specifies a variety of special exponent and mantissa
values in order to support the concepts of plus and minus infinity, plus
and minus zero, 'denormalized' numbers and 'not-a-number' (NaN). We propose
that all of these special cases should be fully accepted for FITS
interchange.

We propose that the order of the bytes should be sign and exponent first,
followed by the mantissa bytes in order of decreasing significance (i.e.,
the standard non-byte-swapped order).

The BLANK keyword of the original FITS Agreement should be ignored by FITS
readers when BITPIX=-32 or -64 (the NaNs of the IEEE format will act as the
blank). FITS writers should not write the BLANK keyword if BITPIX=-32 or
-64.

The BSCALE and BZERO values should be applied by FITS readers if they
differ from 1.0 and 0.0. However, such usage be regarded as bad practice,
and strongly discouraged, because of the risk of generating overflows and
underflows in FITS readers.

\subsection{Bibliography}

"An Implementation Guide to a Proposed Standard for Floating-Point
Arithmetic", COMPUTER magazine, Vol. 13, No.  1, pp.68-79, January 1980.

COMPUTER, Vol. 14, No.  3, March 1981.

ANSI/IEEE  754-1985, "IEEE Standard for Binary Floating Point Arithmetic".

ISDN-xxxxx?

Wells, D.C., Greisen, E.W., Harten, R.H.: 1981, Astron. Astrophys. Suppl.
Ser. 44, 363, "FITS: A Flexible Image Transport System".

Greisen, E.W., Harten, R.H.: 1981, Astron. Astrophys. Suppl. Ser. 44, 371,
"An Extension of FITS for Groups of Small Arrays of Data".

Grosb{\o}l, P., Harten, R.H., Greisen, E.W., Wells, D.C.: 1988, Astron.
Astrophys. Suppl. Ser.  73, 359-364, "Generalized Extensions and Blocking
Factors for FITS".

Harten, R.H., Grosb{\o}l, P., Greisen, E.W., Wells, D.C.: 1988, Astron.
Astrophys. Suppl. Ser. 73, 365, "The FITS Tables Extension".

\subsection{Implementation Notes}

Note-1: Many data analysis systems today use floating point internally and
so conversion costs for FITS input and output will be reduced by this
proposal. For machines with IEEE hardware the conversion cost will be
nearly zero, especially when BZERO=0 and BSCALE=1 and the reader has two
DO-loops (one with and one without scaling).

Note-2: The interpretation of the IEEE sign, exponent and mantissa bit
fields is precisely specified by the five rules of Section 3.2.1 of
ANSI/IEEE Standard 754-1985. Algorithms for transformation between IEEE and
non-IEEE formats can be derived from these rules. The FITS Committees will
provide a central archive for conversion software for various
architectures.

FITS readers on non-IEEE systems should test the IEEE exponents for
the special values 0 and 255/2047 (for 32/64-bit numbers). Exponent
zero should be mapped to the local value for floating zero (exponent
zero is used by IEEE for 'denormalized' numbers as well as for both
plus and minus zero); exponent 255/2047 should be mapped to the local
value used for blank pixels (all IEEE infinity and NaN cases have
exponent 255/2047). The IEEE exponent field can be selected from the
32/64-bit values by masking with hexadecimal byte string
7F,80,00,00/7F,F0,00,00,00,00,00,00 (i.e., the constant {\tt
7F800000}/{\tt 7FF0000000000000} hexadecimal on computers with the
normal word ordering; for VAXes use constant {\tt 00007F80}/{\tt
0000000000007FF0} instead). The two tests are made by comparing the
result against zero and against the mask.

FITS writers executing on non-IEEE systems should map their local
value for blank pixels to an IEEE NaN value; we suggest that all bits
set (minus one in 32- and 64-bit two-s complement integers) is a good
choice. The FITS writer should also map local floating point absolute
values which exceed the largest value of IEEE (approximately 10**38
for 32-bit, 10**304 for 64-bit) to IEEE-Infinity.  We recommend that
absolute values smaller than the smallest normalized IEEE value
(approximately 10**-38 or 10**-304) be mapped to zero. These two tests
are particularly important on machines which use 'hexadecimal'
normalization, with an exponent range of approximately 10**75 in
single-precision.

The DEC VAX architecture is an important special case. VAX systems use
the DEC-proprietary 'F' format for 32-bit floating data and their byte
order is 'swapped'. Once the swap has been corrected the F-format
differs from the IEEE 32-bit format only in that the exponent bias
differs by one and the (hidden) binary point is shifted by one bit.
The two differences add, and the total effect is that the two number
systems differ by a factor of 4.0 (VAXF = 4.0 * IEEE32, IEEE32 = 0.25
* VAXF). This floating point multiplication operation is little, if
any, more expensive than the conversion operation between integer and
floating point which is required by the original FITS Agreement. VAX
systems should test the exponent field on input to detect the infinity
and NaN cases and map them to the local blank code using the test
specified previously. The minus zero and denormalized cases may also
need to be trapped. Initial prototype implementations of this
Agreement have disclosed that some IEEE hardware produces -0.0 results
and that reserved operand errors occur in VAXes when these values are
used as operands. The 32-bit operand should be compared against 32768
to detect this case, and the result 0.0 be given rather than
multiplication by 4.0. 

The DEC VAX architecture supports two different double precision
formats, 'D' and 'G'. The 'G' format has the same relationship to IEEE
64-bit format that the DEC 'F' format has to IEEE 32-bit (VAXG = 4.0 *
IEEE64). The 'D' format, however, requires a complicated
transformation to convert to/from IEEE 64-bit; programmers should
consult archives of FITS code to obtain suitable algorithms and
implementations for this case.

% ===================================================================
\clearpage
\section{Hierarchical Keywords in FITS Headers}\label{app:HKA}

\begin{center}P.~Grosb{\o}l and D.~Wells\end{center}
\begin{center}28 February 1989 (Draft-1)\end{center}

\subsection{The Proposal}

A new standard keyword HIERARCH\footnote{This name was derived by the
traditional FITS rule that names longer than 8 characters should be
truncated. It is, in fact, an English word in its own right: the {\bf
Oxford English Dictionary} says (definition B.2) that a {\em hierarch}
is the ``...commander of the celestial hierarchy''!} is introduced for
all FITS headers, both Basic FITS headers and Generalized Extension
headers. The remaining 72-characters of an 80-character HIERARCH line
will contain one or more keyword classification codes followed a
single keyword=value pair.  Code and keyword names will be 1-8
characters (uppercase alphanumeric plus the minus sign), just as for
standard keywords.  Column 9 will be blank; the first code will begin
in column 10 and additional codes and the keyword will be delimited by
one or more blanks.  Value fields will be decoded according to the
rules of Basic FITS.  The intent of this convention is to provide a
hierarchical classification of keywords in order to reduce the
possibility of conflicts of keyword names, especially 'proprietary'
keywords.  The set of approved top-level classification codes will be
controlled by the FITS Working Group of IAU Commission 5.

\subsection{Example}

\begin{verbatim}
         1         2         3         4         5         6
123456789012345678901234567890123456789012345678901234567890
HIERARCH topcode level2 code3 keyword = value
\end{verbatim}

\subsection{Discussion}

The IAU FITS Working Group will need to decide whether top-level
classification codes should be assigned for individual institutions
and projects, or for disciplines of astronomy.  Name-spaces assigned
to institutions can be managed privately by those institutions.
Name-spaces assigned to disciplines will require the creation of
international committees.  The discipline groups could assign
institutional codes under their top-level codes as well as agreeing on
discipline-specific generic codes and keywords.  The latter approach
can lead to a greater degree of standardization among special interest
groups in astronomy, without burdening the IAU Working Group.  This
is, in principal, very desirable.  However, it is unclear whether or
not such interest groups can be organized in practice.

\subsection{Notes}

\begin{enumerate}

\item The Basic-FITS rule that equal signs must be followed by one or
more blanks is relaxed for HIERARCH lines because the rules for
value-field syntax prevent ambiguities.

\item The maximum number of classification codes is naturally limited by
the finite length of a HIERARCH line; however, implementors should
probably limit HIERARCH codes to no more than five levels.

\item Note that Basic FITS does not specify whether keywords are limited
to the set of uppercase alphanumeric characters plus the minus sign,
but those are the only characters occurring in the examples given in
the 1981 paper.

\item Lines 2/4 and 2/5 of Figure 1 on page 370 of the Basic FITS paper
(Astron. Astrophys. Suppl. Ser. 44, 363-370, 1981) show an example of
a hierarchical keyword notation which is similar to the one proposed
here. It is regretable that this notation was not recommended more
clearly in the Basic FITS paper, as much confusion would have been
prevented over the years.

\end{enumerate}

% ===================================================================
\clearpage
\section{3-D Binary Table Extension Format}\label{app:C3DBTF}

% document translated from DEC RUNOFF to LaTeX format
% by program RNOTOTEX version CVF02B at 13-APR-1989 10:52:05.03
% Source file: CHAP14.RNO
%\chapter{FITS Tapes}

{\bf Note:} The text of this appendix is taken from Chapter~14 of {\bf
Going AIPS}, the AIPS programmer's reference manual. The chapter
describes how AIPS uses the FITS agreements. It gives an overview of
FITS from the AIPS perspective, and then proceeds to discuss each of
the FITS formats used by AIPS. Section~\ref{app:C3DBTF2} below (14.5.3
in {\bf Going AIPS}) describes the 3-D binary table format, which Bill
Cotton designed circa 1986 to meet the special requirements of the
calibration databases for the VLBA.

Subsections of the {\bf Going AIPS} chapter which are titled ``Image
Files'', ``Random Group (UV data) Files'', ``Older AIPS Tables'',
``AIPS FITS INCLUDEs'', and ``AIPS FITS Parsing Routines'' have been
deleted because they are not relevant to this paper.

The subsection titled ``Single Dish Data'' has been retained because
of its special interest for single-dish radio astronomers; it
describes the support provided by AIPS for forming N-dimensional
images out of signals (continuum or multi-spectral) sampled at {\em
random} locations by various scanning instruments.  This section shows
the random-groups approach to the spectroscopic format problem. It is
important to realize that any data structure which can be realized
within the random groups concept can also be realized within the 3-D
binary tables concept. Indeed, at the bit stream level the two formats
are essentially identical. The binary tables design does support
several data types which are not possible in random groups (the most
important case is character strings); this is a fundamental advantage
for the table approach.

\subsection{Overview }
The principal route for getting data and images into and out of AIPS
is by FITS (Flexible Image Transport System) format tape files. FITS
is an internationally adopted medium of exchange of astronomical data
and allows easy interchange of data between observatories and image
processing systems.  FITS also has the advantages that it is a
self-defining format and that the actual bit pattern on the tape is
independent of the machine on which the tape was written.  The purpose
of this chapter is to describe the general features of FITS and the
details of the AIPS implementation.  This chapter is not intended to
be a rigorous description of the FITS standards.  See the chapter on
devices for information on read and writing tapes in AIPS.

The fundamental definition of the FITS system\index{FITS format} is
given in Wells,
Greisen, and Harten (1981), with an extension described in Greisen and
Harten (1981).  A proposed further extension is given in Harten,
Grosb{\o}l, Tritton, Greisen and Wells, (1984).  FITS has been adopted as
the recommended medium of exchange of astronomical data by the IAU,
the Working Group on Astronomical Software (WGAS) of the AAS, and
comparable working groups in Europe.  AIPS now also supports the
proposal of these working groups for the writing of blocked FITS
tapes.\index{blocked tapes}

Because of the great flexibility of the FITS system, many of its
features have been adopted for the internal data storage format in
AIPS.  See the chapter on the catalog header for more details on the
AIPS internal storage format.

There are three main portions of a FITS file (1) the main header, (2)
the main data, and (3) any number of records containing auxiliary
information.  In addition, an extension of the original definition of
the FITS structure allows storage of ungridded visibility data.  Each
of these is discussed in detail in the following sections.

\subsection{Philosophy }
FITS is a philosophy as much as a data format.  The underlying
philosophy is to provide a standardized, simple, and flexible means to
transport data between computers or image processing systems. FITS is
standardized in the sense that any FITS reader should be able to read
any FITS image, at least to the degree that the array read is of the
correct dimensions and the pixel values have at least the correct
relative scaling.  In addition, any FITS reader should be able to cope
with any FITS format tape and, at least, skip over portions, or ignore
keywords, that it doesn't understand.

\index{FITS format}
The requirement of simplicity means that the implementation of FITS
reading and writing should be fairly straightforward on any computer
used for astronomical image processing.  Simple also implies that the
structure of the file should be self-defining and, to a large degree,
self-documenting.

The main advantage of FITS is its flexibility.  Due to the
self-defining nature of the files, a large range of data transport
needs are fulfilled. The introduction of new keywords gives the
ability to add new pieces of information as needed, and the use of
generalized extension files allows almost unlimited flexibility in the
type of information to be stored.  Thus, FITS can grow with the needs
of the Astronomical community.

The great flexibility of FITS is a potential weakness as well as a
strength.  There is a great temptation to proliferate keywords and new
extension file types. This should be done with great caution. Since
FITS is a worldwide medium of data exchange, there needs to be
coordination of keywords and extension files to prevent duplication
and inconsistencies in usage.

The most fundamental philosophical ideal of FITS is that no change in
the system should render old tapes illegal or unreadable. This
philosophy is reflected in the AIPS implementation of FITS in that all
obsolete implementations (e.g., old CLEAN component or antenna
extension files) are trapped and processed in the most accurate manner
possible.

\subsection{Single Dish Data }
     Observations made with filled aperature instruments are
frequently made at essentially random positions on the sky, possibly
using a number of offset feeds or detectors.  This type
of data may be convienently described using the random groups (UV
FITS) format.  The FITS form of this data is the same as visibility
data except that the number and meaning of the random parameters are
different.  The celestial coordinates may be either Right Ascension
and declination or projected coordinates about a specified tangent
point.

     A logical record consists of all data recorded from a given
beam on the sky at a given time.  A dummy AN table is optional.

\subsubsection{Single dish random parameters }
The single dish random parameter types (PTYPEn)
are described in the following:
\begin{enumerate} % list nest 1
\item 'RA' and 'DEC':  These random parameters are the Right
Ascension and Declination of the observation in degrees.  If the
coordinates have been projected onto the tangent plane then the RA and
Declination types become 'RA--xxxx' and 'DEC-xxxx' where xxxx is the
projection code.  See the chapter on AIPS catalogue headers and/or
AIPS memoes 27 and 46 for details of the projection codes.  These
random parameter these are required but the order is arbitrary.
\item 'DATE': The time tags for the data are kept in the form of Julian date
in days. This random parameter is required but the order is optional.
\item 'BEAM': This random parameter gives the beam number + 256. This random
parameter is optional.  The beam offset makes the data look more like
uv data and more of the the AIPS uv data tasks will work for this data.
\item 'SCAN': This random parameter gives the scan number. This random
parameter is optional.
\item 'SAMPLE': This random parameter gives the sample number in the scan.
This random parameter is optional.

\end{enumerate} % - list nest 1
\subsubsection{Single dish regular axis coordinates }
The units of the regular axis coordinates are defined by convention;
the conventions used by AIPS for the regular axis types (CTYPEn)
are the following:
\index{uv FITS format}
\begin{enumerate} % list nest 1
\item 'COMPLEX': the complex axis consists of the real, imaginary and
(optional) weight.  Magic value blanking is supported.  The imaginary
part may be used to carry any baseline values which have been
subtracted.  This axis is required.
\item 'STOKES': this axis is used to describe which Stokes' parameters are
given; the conventions are the same as used internally in AIPS.  These
conventions are discussed in the chapter on disk I/O.  This axis is
required.
\item 'FREQ': the frequency axis coordinates are in Hz.  This axis is required.
\item 'IF': The IF axis is a construct which allows irregularly spaced groups
of frequency channels.  The IF number specifies an entry in the ('AIPS
CH') table which must follow the data if this axis is present. This
table gives the offsets from the reference frequency specified by the
FREQ axis.  This axis is optional but if it is present then a CH table
must be present.
\item 'RA' and 'DEC': the celestial coordinates are given in degrees.  The
values associated with these axes are irrelevant (although they should
be present) for unprojected data.  For data with projected coordinates
the coordinate values of these axes should be the tangent point, i.e.
the position on the sky at which the plane onto which the coordinates
are projected is tangent to the celestial sphere and these axes should
become 'RA---ccc' and 'DEC--ccc' where ccc is the projection code.
These axes are required.

\end{enumerate} % - list nest 1
     Weights and flagging are handled the same as for visibility data.
Sort order is the same as for visibility data except that the sort
codes for sorting by u and v become:
\begin{verbatim}
          U => ordered by RA
          V => ordered by Declination
          X => descending ABS (RA)
          Y => descending ABS (Declination)
          Z => ascending ABS (RA)
          M => ascending ABS (Declination)

\end{verbatim}
\subsubsection{Example Single Dish Data File Header }
The following is a FITS header for a single dish data file
containing 16 frequency channels and a single IF and using unprojected
coordinated.  The first two lines are not part of the FITS file.
\index{single dish FITS format}

\begin{verbatim}
000000000111111111122222222223333333333444444444455555555556666666666
123456789012345678901234567890123456789012345678901234567890123456789
SIMPLE  =                    T /
BITPIX  =                   16 /
NAXIS   =                    6 /
NAXIS1  =                    0 /NO STANDARD IMAGE JUST GROUP
NAXIS2  =                    3 /
NAXIS3  =                    1 /
NAXIS4  =                   16 /
NAXIS5  =                    1 /
NAXIS6  =                    1 /
EXTEND  =                    T /TABLES FOLLOWING MAIN IMAGE
BLOCKED =                    T /TAPE MAY BE BLOCKED
OBJECT  =  'ALL SKY '          /SOURCE  NAME
TELESCOP=  '300 FT  '          /
INSTRUME=  '6 CM    '          /
OBSERVER=  'PHANTOM '          /
DATE-OBS=  '02/12/86'          /OBSERVATION START DATE DD/MM/YY
DATE-MAP=  '09/11/87'          /DATE OF LAST PROCESSING DD/MM/YY
BSCALE  =    4.98494155534E-05 /REAL = TAPE * BSCALE + BZERO
BZERO   =    0.00000000000E+00 /
BUNIT   =  'JY      '          /UNITS OF FLUX
EPOCH   =      1.950000000E+03 /EPOCH OF RA DEC
BLANK   =               -32768 /TAPE VALUE OF BLANK PIXEL
CTYPE2  =  'COMPLEX '          /
CRVAL2  =    1.00000000000E+00 /
CDELT2  =      1.000000000E+00 /
CRPIX2  =      1.000000000E+00 /
CROTA2  =      0.000000000E+00 /
CTYPE3  =  'STOKES  '          /
CRVAL3  =   -1.00000000000E+00 /
CDELT3  =     -1.000000000E+00 /
CRPIX3  =      1.000000000E+00 /
CROTA3  =      0.000000000E+00 /
CTYPE4  =  'FREQ    '          /
CRVAL4  =    1.66499989984E+09 /
CDELT4  =      5.000000000E+07 /
CRPIX4  =      1.000000000E+00 /
CROTA4  =      0.000000000E+00 /
CTYPE5  =  'RA      '          /
CRVAL5  =    0.00000000000E+00 /
CDELT5  =      1.000000000E+00 /
CRPIX5  =      1.000000000E+00 /
CROTA5  =      0.000000000E+00 /
CTYPE6  =  'DEC     '          /
CRVAL6  =    0.00000000000E+00 /
CDELT6  =      1.000000000E+00 /
CRPIX6  =      1.000000000E+00 /
CROTA6  =      0.000000000E+00 /
GROUPS  =                    T /
GCOUNT  =               14655. /
PCOUNT  =                    5 /
PTYPE1  =  'RA      '          /  RA in degrees
PSCAL1  =    1.09890000000E-02 /
PZERO1  =    0.00000000000E+00 /
PTYPE2  =  'DEC     '          /  Declination in degrees
PSCAL1  =    1.09890000000E-02 /
PZERO2  =    0.00000000000E+00 /
PTYPE3  =  'BEAM    '          /  Beam number
PSCAL3  =    3.66412235782E-10 /
PZERO3  =    0.00000000000E+00 /
PTYPE4  =  'DATE    '          /  Date + time as Julian date (days)
PSCAL4  =    2.50000000000E-01 /
PZERO4  =    2.44603650000E+06 /
PTYPE5  =  'DATE    '          /
PSCAL5  =    1.52587890600E-05 /
PZERO5  =    0.00000000000E+00 /
ORIGIN  = 'AIPSNRAO CVAX            15APR87'    /
DATE    = '13/11/87 '                    / FILE WRITTEN ON DD/MM/YY
HISTORY AIPS WTSCAL =  1.83759533423E+00 / CMPLX WTS=WTSCAL*
                                           (TAPE*BSCALE+BZERO)
END

\end{verbatim}

\subsection{Extension Files }
There is frequently auxiliary information associated with an image or
data set which needs to be saved in the same tape file. Examples of
this in AIPS are the Antenna files and CLEAN component files.  There
is currently a draft proposal to the IAU (Harten {\it et. al.} 1984)
defining a standard format for the invention of extension files to be
written after the main data records (if any) and defining a ``Tables''
type extension file.  The Tables extension files will be able to carry
information which can be expressed in the form of a table. AIPS also
makes use of a 3-D Table extension which is similar to tables but
allows arrays as table entries.  The following section will describe
the proposed standards which are being incorporated into AIPS.
\index{FITS format}
\index{FITS tables}

\subsection{Standard Extension }
The standard, generalized extension file is not a true tape file in
the sense that it is separated by tape EOF marks, but is a number of
records inside a FITS tape file which contains information of
relevance to the file.  Each standard extension ``file'' will have a
header which is very similar to the main FITS header.  This header
consists of one or more 2880 8-bit byte ``logical'' records each
containing 36 80-byte ``card images'' in the form:

\begin{verbatim}
     keyword = value    / comment

\end{verbatim}
The extension file header begins in the first record following the
last record of main data (if any) or following the last record of the
previous extension file.  The format of the generalized extension
``file'' header is such that a given FITS reader can decide if it wants
(or understands) a given extension file type and can skip over the
extension file if the reader decides it doesn't.

Most of the standards concerning data types and bit orders for the
main FITS data records also apply to extension files.  One difference
is that 8-bit pixel values can be used to indicate ASCII code.

The use of the generalized extension ``files'' requires the use of a
single additional keyword in the main header:
\begin{enumerate} % list nest 1
\item EXTEND (logical) if true (T) indicates that there {\it may} be extension
files following the data records and, if there are, that they conform
to the generalized extension file header standards.
\end{enumerate} % - list nest 1
\index{FITS format}

\index{FITS tables}
The required keywords in an extension file header record are, in
order:
\begin{enumerate} % list nest 1
\item XTENSION (character) indicates the type of extension file, this must
be the first keyword in the header.
\item BITPIX (integer) gives the number of bits per ``pixel'' value.  The
types defined for the main data records plus 8-bit ASCII are allowed.
\item NAXIS (integer) gives the number of ``axes''; a value of zero is allowed
which indicates that no data records follow the header.
\item NAXIS1 (integer) is the number of ``pixels'' along the first axis (if
any).
\item NAXISn (integer) is the number of ``pixels'' along the last axis.
\item PCOUNT (integer) is the number of ``random'' parameters before each
group.  This is similar to the definition of random group main data
records.  The value may be zero.
\item GCOUNT (integer) is the number of groups of data defined as for the
random group main data records. If an image-like file (e.g., a table
file) is being written this will be 1.
\item END is always the last keyword in a header.  The remainder of the
record following the END keyword is blank filled.
\end{enumerate} % - list nest 1
\index{FITS format}

\index{FITS tables}
There are three optional standard keywords for extension file header
records.  The order, between the required keywords and the END
keyword, is arbitrary.
\begin{enumerate} % list nest 1
\item EXTNAME (character) can be used to give a name to the extension file
to distinguish it from other similar files.  The name may have a
hierarchal structure giving its relation to other files (e.g.,
``map1.cleancomp'')
\item EXTVER (integer) is a version number which can be used with EXTNAME to
identify a file.
\item EXTLEVEL (integer) specifies the level of the extension file in a
hierarchal structure.  The default value for EXTLEVEL should be 1.
\end{enumerate} % - list nest 1
\index{FITS format}

\index{FITS tables}
The number of bits in an extension file (excluding the header) should
be given by the formula:

\begin{verbatim}
NBITS = BITPIX * GCOUNT * (PCOUNT + NAXIS1 * NAXIS2 * ... * NAXISn)

\end{verbatim}
The number of data records following the header record are then given
by:

\begin{verbatim}
     NRECORDS = INT ((NBITS + 23039) / 23040)

\end{verbatim}
It is important that the above formulas accurately predict the number
of data records in an extension ``file'' so that readers can skip over
these ``files''.  The data begins in the first record following the last
record of the header.

Extreme caution must be exercised when inventing new types of
extension files.  In particular, duplication of types, or several
types with the same function, must be avoided.  This means that when a
new extension file type is invented, it should be as general as
possible so that it may be used for other similar problems.

\subsection{Tables Extension }
A very common type of extension file is one containing data that can
be expressed in the form of a table.  That is, a number of entries
which are all identical in form.  A general, self-defining table
extension file type is proposed by Harten {\it et. al.} (1984). The
following sections describe the proposed format.
\index{FITS format}

\index{FITS tables}
The table extension file uses ASCII records to carry the tabular
information.  Each table entry will contain a fixed number of entries
(although the number can vary between different extension files). For
each entry is given (1) a label (optional), (2) the beginning column,
(3) an undefined value (optional), (4) a Fortran format to decode the
entry, (5) scaling and offset information (optional), and (6) the
units (optional).

\subsubsection{Tables Header Record }
The keywords for tables extension file headers are given in the
following:
\begin{enumerate} % list nest 1
\item XTENSION (character) is required to be the first keyword and has a
value 'TABLE   ' for table extension files.
\item BITPIX (integer) is a required keyword which must have a value of 8
indicating printable ASCII characters.
\item NAXIS (integer) is a required keyword which must have a value of 2 for
tables extension files.
\item NAXIS1 (integer) is a required keyword which gives the number of
characters in a table entry (i.e., ``row'').
\item NAXIS2 (integer) is a required keyword which gives the number of
entries in the table (i.e., number of rows). A value of 1 is allowed.
\item PCOUNT (integer) is a required keyword which must have the value of 0
for tables extension files.
\item GCOUNT (integer) is a required keyword which must have the value of 1
for tables extension files.
\item TFIELDS (integer) is a required keyword which must follow the GCOUNT
keyword.  TFIELDS gives the number of fields in each table entry.
\item EXTNAME (character) is the name of the table.
\item EXTVER (integer) is the version number of the table.
\item EXTLEVEL (integer) is the hierarchal level number of the table, 1 is
recommended. (optional)
\item TBCOLnnn (integer) the pixel number of the first character in the
nnn'th field.
\item TFORMnnn (character) the Fortran format of field nnn (I,A,E,D)
\item TTYPEnnnn (character) the label for field nnn. (optional, order
arbitrary)
\item TUNITnnn (character) the physical units of field nnn. (optional, order
arbitrary)
\item TSCALnnn (floating) the scale factor for field nnn. True\_value =
tape\_value $\ast$ TSCAL + TZERO.  Note: TSCALnnn and TZEROnnn are not
relevant to A-format fields. Default value is 1.0 (optional, order
arbitrary)
\item TZEROnnn (floating) the offset for field nnn. (See TSCALnnn.) Default
value is 0.0 (optional, order arbitrary)
\item TNULLnnn (character) the (tape) value of an undefined value. Note: an
exact left-justified match to the field width as specified by TFORMnnn
is required. (optional, order arbitrary)
\item AUTHOR (character) the name of the author or creator of the table.
(optional, order arbitrary)
\item REFERENC (character) the reference for the table. (optional, order
arbitrary)
\item END must always be the last keyword and the remainder of the record
must be blank filled.
\end{enumerate} % - list nest 1
\index{FITS format}

\index{FITS tables}
The TFORMnnn keywords should specify the width of the field and are of
the form Iww, Aww, Eww.dd, or Dww.dd (integers, characters, single
precision and double precision).  If -0 is ever to be distinguished
from +0 (e.g., degrees of declination), the sign field should be
declared to be a separate character field.

\subsubsection{Table Data Records }
The table file data records begin with the next record following the
last header record and each contains 2880 ASCII characters in the
order defined by the header.  Table entries do not necessarily begin
at the beginning of a new record.  The last record should be blank
filled past the end of the valid data.

\subsubsection{Example Table Header and Data }
The first two lines of numbers are only present to show card columns
and are not part of the extension file.
\index{FITS format}
\index{FITS tables}

\begin{verbatim}
         1         2         3         4         5         6         7
12345678901234567890123456789012345678901234567890123456789012345678901
XTENSION= 'TABLE   '           / EXTENSION TYPE
BITPIX  =                    8 / PRINTABLE ASCII CODES
NAXIS   =                    2 / TABLE IS A MATRIX
NAXIS1  =                   60 / WIDTH OF TABLE IN CHARACTERS
NAXIS2  =                  449 / NUMBER OF ENTRIES IN TABLE
PCOUNT  =                    0 / RANDOM PARAMETER COUNT
GCOUNT  =                    1 / GROUP COUNT
TFIELDS =                    3 / NUMBER OF FIELDS IN EACH ROW
EXTNAME = 'AIPS CC '           / AIPS CLEAN COMPONENTS
EXTVER  =                    1 / VERSION NUMBER OF TABLE
TBCOL1  =                    1 / STARTING CHAR. POS. OF FIELD N
TFORM1  = 'E15.6   '           / FORTRAN FORMAT OF FIELD N
TTYPE1  = 'FLUX    '           / TYPE (HEADING) OF FIELD N
TUNIT1  = 'JY      '           / PHYSICAL UNITS OF FIELD N
TSCAL1  =                  1.0 / SCALE FACTOR FOR FIELD N
TZERO1  =                  0.0 / ZERO POINT FOR FIELD N
TBCOL2  =                   17 / STARTING CHAR. POS. OF FIELD N
TFORM2  = 'E15.6   '           / FORTRAN FORMAT OF FIELD N
TTYPE2  = 'DELTAX  '           / TYPE (HEADING) OF FIELD N
TUNIT2  = 'DEGREES '           / PHYSICAL UNITS OF FIELD N
TSCAL2  =                  1.0 / SCALE FACTOR FOR FIELD N
TZERO2  =                  0.0 / ZERO POINT FOR FIELD N
TBCOL3  =                   33 / STARTING CHAR. POS. OF FIELD N
TFORM3  = 'E15.6   '           / FORTRAN FORMAT OF FIELD N
TTYPE3  = 'DELTAY  '           / TYPE (HEADING) OF FIELD N
TUNIT3  = 'DEGREES '           / PHYSICAL UNITS OF FIELD N
TSCAL3  =                  1.0 / SCALE FACTOR FOR FIELD N
TZERO3  =                  0.0 / ZERO POINT FOR FIELD N
END

\end{verbatim}
The rest of the header block is blank filled.  The data cards start on
the next block boundary.

\begin{verbatim}
   0.183387E+00   -0.138889E-03    0.694444E-04
   0.146710E+00   -0.138889E-03    0.694444E-04
   0.117368E+00   -0.138889E-03    0.694444E-04
   0.938941E-01   -0.138889E-03    0.694444E-04
   0.183387E+00   -0.138889E-03    0.694444E-04

                      .
                      .
                      .

\end{verbatim}

\subsection{3-D Tables Extension }\label{app:C3DBTF2}
There are several types of extension files in AIPS which do not fit in
a natural way into the FITS Tables format (e.g., gain and calibration
tables).  These files tend to be large, and conversion to and from
ASCII can be expensive.  A ``3-D'' extension to the FITS tables format,
in which an entry can be a 1-dimensional array, gives the needed
flexibility.  Use of binary data, including IEEE floating point
formats, allows efficient implementation of readers and writers.
\index{FITS format}

\index{FITS tables}
Each row in a 3-D table contains a fixed number of entries
(although the
number can vary between different extension files).  The header is a
standard FITS extension header; for each table entry is given (1) the
size and data type of the entry, (2) a label (optional), (3) the units
(optional), and (4) an undefined value (optional).  An entry may be
omitted from the table, but still defined in the header, by using a
zero repeat count in the TFORMn entry.

\subsubsection{3-D Tables Header Record }
The keywords for 3-D tables extension file headers are given in the
following:
\begin{enumerate} % list nest 1
\item XTENSION (character) is required to be the first keyword and has a
value 'A3DTABLE' for table extension files.
\item BITPIX (integer) is a required keyword which must have a value of 8.
\item NAXIS (integer) is a required keyword which must have a value of 2 for
3-D Tables extension files.
\item NAXIS1 (integer) is a required keyword which gives the number of 8-bit
bytes in a table row.
\item NAXIS2 (integer) is a required keyword which gives the number of rows
in the table.  A value of 1 is allowed.
\item PCOUNT (integer) is a required keyword which must have the value of 0
for tables extension files.
\item GCOUNT (integer) is a required keyword which must have the value of 1
for tables extension files.
\item TFIELDS (integer) is a required keyword which must follow the GCOUNT
keyword.  TFIELDS gives the number of fields in each table entry.
\item EXTNAME (character) is the name of the table.
\item EXTVER (integer) is the version number of the table.
\item EXTLEVEL (integer) is the hierarchal level number of the table, 1 is
recommended. (optional)
\item TFORMnnn (character) is the size and data type of field nnn. If repeat
count is absent, it is assumed to be 1.  A value of zero is allowed.
(required, order arbitrary)
\item TTYPEnnnn (character) the label for field nnn.  (optional, order
arbitrary)
\item TUNITnnn (character) the physical units of field nnn.  (optional,
order arbitrary)
\item TNULLnnn (character) the value of an undefined value.  These are only
for integer data types; for floating data types, the IEEE nan (not a
number) values are used.
\item AUTHOR (character) the name of the author or creator of the table.
(optional, order arbitrary)
\item REFERENC (character) the reference for the table.  (optional, order
arbitrary)
\item END must always be the last keyword and the remainder of the record
must be blank filled.
\end{enumerate} % - list nest 1
\index{FITS format}

\index{FITS tables }
The TFORMnnn keywords specify the width of the field and are of the
form rL, rX, rI, rJ, rA, rE, or rD (logical, bit, 16-bit integers,
32-bit integers, characters, single precision and double precision,
r=repeat count).  If -0 is ever to be distinguished from +0 (e.g.,
degrees of declination), the sign field should be declared to be a
separate character field.

\subsubsection{Table Data Records }
The 3-D table file binary data logical records begin with the next
record following the last header record and each contains 2880 8-bit
bytes in the order defined by the header.  Table entries do not
necessarily begin at the beginning of a new record.  The last record
should be zero filled past the end of the valid data.
\index{FITS format}

\index{FITS tables}
The data types are defined in the following list (r is the repeat
count):
\begin{enumerate} % list nest 1
\item {\it rL.} A logical value consists of an 8-bit ASCII ``T'' indicating true
and ``F'' indicating false.
\item {\it rX.} A bit array will start in the most significant bit of the byte
and the following bits in the order of decreasing significance in the
byte.  A bit array entry consists of an integral number of bytes with
trailing bits zero.
\item {\it rI.} A 16-bit integer is a two's complement integer with the bits
in decreasing order of significance.
\item {\it rJ.} A 32-bit integer is a two's complement integer with the bits
in decreasing order of significance.
\item {\it rA.} Character strings are 8-bit ASCII characters in their natural
order.
\item {\it rE.} Single precision floating point values are in IEEE 32-bit
precision format with the bits in the following order:
\begin{verbatim}
                  1         2         3
        01234567890123456789012345678901
        seeeeeeeemmmmmmmmmmmmmmmmmmmmmmm

sign = -1 $\ast\ast$ s, exponent = eee..., mantissa = 1.mmmmm... .
The value is given by:

     value = sign * 2 **(exponent-127) * mantissa

\end{verbatim}
Note:  these values have a ``hidden'' bit and must always be normalized.
The IEEE nan (not a number) values are used to indicate an invalid
number; a value with all bits set is recognized as a nan.  The IEEE
special values (-0., +/- Infinity) are not recognized.  A
multiplication by a factor of 4.0 converts between VAX F and IEEE
32-bit formats.

\item {\it rD.}  Double precision floating point values are in IEEE 64-bit
precision format with the bits in the following order:

\begin{verbatim}
               1         2         3
     01234567890123456789012345678901
     seeeeeeeeeeemmmmmmmmmmmmmmmmmmmm

     3       4         5         6
     23456789012345678901234567890123
     mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm

sign = -1 ** s, exponent = eee..., mantissa = 1.mmmmm... . The value
is given by:

     value = sign * 2 **(exponent-1023) * mantissa

\end{verbatim}
Note: these values have a ``hidden'' bit and must always be normalized.
The IEEE nan (not a number) values are used to indicate an invalid
number; a value with all bits set is recognized as a nan. The IEEE
special values (-0., +/- Infinity) are not recognized. A
multiplication by a factor of 4.0 converts between VAX G and IEEE
64-bit formats.
\end{enumerate} % - list nest 1
\index{FITS format}

\index{FITS tables}
\subsubsection{Example 3- D Table Header }
The first two lines of numbers are only present to show card columns
and are not part of the extension file.
\index{FITS format}
\index{FITS tables}

\begin{verbatim}
         1         2         3         4         5         6
1234567890123456789012345678901234567890123456789012345678901234
XTENSION= 'A3DTABLE'           / EXTENSION TYPE
BITPIX  =                    8 / BINARY DATA
NAXIS   =                    2 / TABLE IS A MATRIX
NAXIS1  =                   12 / WIDTH OF TABLE IN BYTES
NAXIS2  =                  449 / NUMBER OF ENTRIES IN TABLE
PCOUNT  =                    0 / RANDOM PARAMETER COUNT
GCOUNT  =                    1 / GROUP COUNT
TFIELDS =                    3 / NUMBER OF FIELDS IN EACH ROW
EXTNAME = 'AIPS CC '           / AIPS CLEAN COMPONENTS
EXTVER  =                    1 / VERSION NUMBER OF TABLE
TFORM1  = '1E      '           / COUNT AND DATA TYPE OF FIELD N
TTYPE1  = 'FLUX    '           / TYPE (HEADING) OF FIELD N
TUNIT1  = 'JY      '           / PHYSICAL UNITS OF FIELD N
TFORM2  = '1E      '           / COUNT AND DATA TYPE OF FIELD N
TTYPE2  = 'DELTAX  '           / TYPE (HEADING) OF FIELD N
TUNIT2  = 'DEGREES '           / PHYSICAL UNITS OF FIELD N
TFORM3  = '1E      '           / COUNT AND DATA TYPE OF FIELD N
TTYPE3  = 'DELTAY  '           / TYPE (HEADING) OF FIELD N
TUNIT3  = 'DEGREES '           / PHYSICAL UNITS OF FIELD N
END

\end{verbatim}
The rest of the header block is blank filled. The binary data starts
on the next logical block boundary. The last record of table data is
zero filled past the end of the valid data.

\subsection{References }
\hspace*{  0.00cm } Wells, Greisen, and Harten 1981, {\it Astronomy and Astrophysics}
Supplement series, vol. 44, pp 363 - 370.
\smallskip
\hspace*{  0.00cm } Greisen and Harten, 1981, {\it Astronomy and Astrophysics}  Supplement
Series, vol. 44, pp 371 - 374.
\smallskip
\hspace*{  0.00cm } Harten, Grosb{\o}l, Tritton, Greisen and Wells 1984, draft reproduced in
the IAU Commission 9 {\it Astronomical Image Processing Circular.}
Also appeared in {\it Mem. S. A. It.}, 56, 437 (1985).

\end{document}