Subject: HDUCLASn keywords

From: pence at tetra.gsfc.nasa.gov (William Pence)
Date: Wed, 1 Sep 93 18:09:37 EDT

As was posted here previously, the HEASARC plans to use a hierarchy of
HDUCLAS1, HDUCLAS2, ... FITS keywords to classify each type of
FITS extension (Header and Data Unit) that it creates.  An example of this
would be HDUCLAS1 = 'GTI' (For Good Time Intervals extension) and
HDUCLAS2 = 'STANDARD' or HDUCLAS2 = 'ALL' depending on whether it contains
all the good time intervals, or only the standard set of intervals.

Before implementing this scheme, however, a debate arose here about 
whether it would be useful to reserve the top level 'HDUCLAS1' keyword
as an identifier for different diciplines, or oranizations that
had defined the lower level hierarchy of HDUCLASn keywords. For example,
we could have:

HDUCLAS1 = 'OGIP' or 'XRAY'
HDUCLAS2 = 'GTI'
HDUCLAS3 = 'STANDARD'

This would immediately tell software and users that this file follows
some standard defined by the OGIP (or is an XRAY type file), as opposed
to some FITS file which may have been defined by NRAO for instance for
radio data.  This could be useful if different groups choose the same
class name, such as 'SPECTRUM' for files which have entirely different
format and keywords (an radio FITS spectral file will probably look
very different from an Xray FITS spectral file).

It was argued that without this extra hierarchical layer, others in the
community would be reluctant to use the HDUCLASn keywords and if they did,
the lack of standardization on permitted HDUCLAS1 keyword values would
cause confusion.

There were a number of objections to this suggestion, however:

- This use of the HDUCLAS1 keyword as a 'family' label is rather
different from the intended use of providing a hierarchical
classification of the scientific content of file.  Thus, it may be
better to use a separate distinct keyword to define this if it is
really necessary.

- In practice, there are many other keywords or combinations of
keywords in the header (TELESCOP, INSTRUME, AUTHOR), which at least
implicitly identify the source or origin of the file.

- in most cases the software will simply have to skip over the HDUCLAS1
keyword before looking at the HDUCLAS2 and higher keywords to determine
which type of FITS extension it is.  This wastes header space and CPU
cycles.

- this may lead to geo-political copyright disputes over which group
first defines certain new classes of extensions.  For instance, SAO and
the HEASARC may well define similar formats to store xray data.  Does
one institution then get all the credit in the header, by having
HDUCLAS1 = 'SAO' in all those types of FITS files.  Or if the HDUCLAS1
keyword has a broad definition e.g., 'XRAY' then it may seem
presumptuous for one group to define a new class and expect that all
other XRAY groups should recognize and adopt it.

- just because the OGIP decides not to use the HDUCLAS1 keyword in this
way does not prevent other groups from doing so.  For instance NRAO
could uniquely identify all it's FITS files with HDUCLAS1= 'NRAO' if it
wanted to.

Given that we must make some decisions on the usage of HDUCLASn
keywords in the next few days, we would like some input from other
groups.  Without strong opinions to the contrary, we will most likely
not use HDUCLAS1 in the way described here.

Do any other groups see a practical need for the HDUCLASn keywords, and
do they think they might use them in their FITS files?  If so do you
have an opinion on whether or not the HDUCLAS1 keyword should be
reserved as a generic family/institution identifier?

-Bill Pence

---

From: Lucio Chiappetti <lucio at IFCTR.MI.CNR.IT>
Date: Thu, 2 Sep 1993 10:16:42 +0200 (MET DST)

On Wed, 1 Sep 1993, William Pence wrote:

> HDUCLAS1, HDUCLAS2, ... FITS keywords to classify each type of [...]
> 
> had defined the lower level hierarchy of HDUCLASn keywords. [...]

> the lack of standardization on permitted HDUCLAS1 keyword values would

  Yes
  Or at least I think it is wise to have a "signature" in the file which
  tells what convention is used (this could be also useful for other
  "quick classification" of the files, sort of (although not completely)
  the discussion going over on sci.data.formats  ... I'm thinking here
  of some sort of "magic number" to be used by the "file" command.

  One already has SIMPLE=T for FITS, could have XTENSION=something for
  extensions, e.g. to tell a bintable, just one more keyword can identify
  the convention used and then convention dependent keywords, so one
  could tell a file is a

      i    ii       iii   iv        ...

      FITS BINTABLE OGIP  SPECTRUM  ...   or a
      FITS BINTABLE MIDAS Photonlist      or a
      XAS  TABLE          SPECTRUM        (non FITS file) or a
      FITS BINTABLE XAS   SPECTRUM        (same stored in FITS) etc.

> implicitly identify the source or origin of the file.
   I would not rely too much on "implicit" things, better have some
   explicit thing. It is true that this could use some existing keyword

   one possibility is EXTNAME (the dread EXTNAME ?)
   another one is ORIGIN if this is used not to identify the organization
   or observatory, but the software package who wrote it, e.g.

   ORIGIN  = 'ESO-MIDAS'          / Written by MIDAS

   or a combination

   ORIGIN  = 'XAS     '           /
   EXTNAME = 'TIME PROFILE'       / name of this binary table extension

   However currently I have the problem that, if I import a file, and then
   copy/manipulate it I tend to preserve the ... original ORIGIN keyword 

> which type of FITS extension it is.  This wastes header space and CPU
  80 bytes and how many nanoseconds ?

> first defines certain new classes of extensions.  For instance, SAO and

  That's why I suggested an identifier of the software package ...  

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
IN CONCLUSION
I do not care whether this keyword is called HDUCLAS1, ORIGIN, EXTNAME or
whatever, but I'd strongly like to be able to identify clearly what a 
file is.  
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

For instance I have a "fromfits" program which converts from
FITS to internal format. Can easily tell an IMAGE from a BINTABLE, but if
EXTNAME is not one of the recognized ones, it cannot classify the bintable
and flags it as "generic table". I also have a separate prototype "fromogip"
program to handle EXTNAME.EQ.'SPECRESP MATRIX', EXTNAME.EQ.'MATRIX' and
EXTNAME.EQ.'SPECTRUM'. I have "manually" to know which program to use,
while having a definite convention, I could merge the two programs, or
have one call the other.

-----------------------------------------------------------------------------   
Lucio Chiappetti - IFCTR/CNR Milano        |         U N I C U I Q U E          
via Bassini 15 - I-20133 Milano - Italy    |          System  Manager           
Internet: LUCIO at IFCTR.MI.CNR.IT            |           FORTUNAE SUAE            
Decnet:   IFCTR::LUCIO (39610::LUCIO)      |                                    
Bitnet:   <please do not use any more>     |                                    
-----------------------------------------------------------------------------