Subject: Re: A question on keywords for 3-D images

From: valdes at tucana.tuc.noao.edu (Frank Valdes)
Date: 30 Oct 1998 16:25:38 GMT

Hello Bob,

Of your two options, 3D image with indexed keywords and image extensions, the
latter is the one to use.  Indexed keywords are a hassle and there is
no standard way to do this.  You would be unlikely to find software which
used this.  Multiextension FITS is actually quite commonly used in
applications such as yours.  Examples of this are the NOAO CCD Mosaic
format (8 CCDs recorded in a file with 8 image extensions) and
NICMOS/STIS.  Because this is a very useful format it is well supported
in IRAF.  Any IRAF task that operates on a 2D image can access any
of the extensions with a simple syntax; for example

    cl> display sirtif01[im7]

would access the extension with EXTNAME of "im7".  The syntax also allows
a positional selection such as sirtif01[3] to get the third extension in
the file.  The important advantage regard keywords is that if a program
understands a particular set of keywords, such as SIGMA, then it will
work with any extension and you don't need all the indexing stuff.

Another aspect of the multiextension format, adopted as a convention by
HST and NOAO, is the use of a global header.  The primary FITS unit consists
only of keywords.  These keywords are common to all the image extensions
in the file.  Thus the image extension headers can contain just the
keywords which are different.  The IRAF software supports this concept
of keyword inheritance.  Any task that access an image extension, as in
the above example, sees a single header which is the merger of the global
keywords and the separate extension keywords.  It does not need to know
about the inheritance and two headers as this is taken care of by the
I/O routines.

So the summary is use multiextension FITS, possibly with the inheritance
convention, which is a solution adopted by a number of projects and is
well supported at least in IRAF.

Cheers,
Frank Valdes
NOAO/IRAF Group

---

From: hodge at bowline.stsci.edu (Phil Hodge)
Date: 30 Oct 1998 17:37:56 GMT

>  Each extension would contain a 2D image and its own set of
>  keywords.  Going to this form loses the 3-D nature of the data with
>  each 2-D image looking like an independent image rather than being
>  part of a 3-D cube.  Also, there is a concern is that many FITS
>  readers and standard software may not be able to deal with files in
>  this form.  It may not be supported well in IRAF, CFITSIO, etc. for
>  3-D operations.

Yes, you would lose the 3-D nature of the data by using either rows
or columns of 2-D table entries or multiple 2-D image extensions.

Phil

---

From: Arnold Rots <arots at head-cfa.harvard.edu>
Date: Thu, 5 Nov 1998 10:14:51 -0500 (EST)

I would strongly warn against the use of this convention.  If all you
want is to use your images in IRAF and if you know others will do the
same, in perpetuity, it's fine.  But most other systems explicitly
assume the opposite: each HDU stands by itself and there is no
inheritance.  Previous discussions on the subject concluded that the
notion of inheritance is not supported by the FITS standard and may
not be assumed.

  - Arnold Rots

Frank Valdes wrote:
> ...

--------------------------------------------------------------------------
Arnold H. Rots                                         AXAF Science Center
Smithsonian Astrophysical Observatory                tel:  +1 617 496 7701
60 Garden Street, MS 81                              fax:  +1 617 495 7356
Cambridge, MA 02138                             arots at head-cfa.harvard.edu
USA                                     http://hea-www.harvard.edu/~arots/
--------------------------------------------------------------------------

---

From: valdes at noao.edu (Frank Valdes)
Date: Thu, 5 Nov 98 10:09:44 MST

Hello Arnold,

I appreciate your concern.  However, the usage of this is larger than you
may think.  The inheritance approach began, I believe, with the needs at
HST for the new instruments (NICMOS/STIS).  Years of software development
has gone into this and it has been discussed and status papers have been
presented at ADASS for many years.  The IRAF FITS kernel support for
inheritance began at ST.  I expect HST to continue to use this
approach.  NOAO adopted this because it made sense for mosaic instruments
but it is a smaller effort.  One thing I did not say in my posting is that
for the NOAO Mosaic the header design is such that critical keywords are
included in all extensions even though they are the same in all extensions.
For example, exposure time appears in all extensions.  Thus any software
which may not merge the global keywords will still have a useful minimal
header for interpreting the data.

I purposely phrased my comment that this was a "convention" since, as
you say, it is not a FITS standard.  However the idea of conventions is
a known practice in the community.  There is no "assumption" that this
is supported by all FITS readers (though all readers should be able to
ingest both the primary (global) and extension headers.  For the
specific formats, NICMOS, STIS, NOAO/MOSAIC, other similar mosaics,
people will be advised of the format.  It is not very hard to merge
headers if needed.

Frank Valdes

My role in this has been in the design of the NOAO Mosaic format and I had
little to do with the development of the convention.

> From arots at head-cfa.harvard.edu Thu Nov  5 08:25:03 1998
>

---

From: Arnold Rots <arots at head-cfa.harvard.edu>
Date: Thu, 5 Nov 1998 12:37:00 -0500 (EST)

I agree that adopting the convention in the context of HST is
reasonably safe, but would argue that its adoption outside that
context is dangerous.  I certainly appreciated your use of the word
"convention", but although most conventions are fairly harmless if not
understood, this one can bite the unwary user: software will either
support this convention or its opposite - there is no "neutral" position.

Hence, the safe thing is not to assume inheritance and repeat all
relevant keywords in all headers.

  - Arnold