Subject: Re: Some FITS tables suggestions

From: forveill at gag.observ-gr.fr (Thierry Forveille)
Date: Wed, 7 Oct 92 16:03:42 +0100

Lucio Chiappetti (LUCIO at IFCTR.MI.CNR.IT) writes

> YES, conventions are needed to avoid those incompatibilities, which can
> be very annoying.
> And I would say those conventions are needed beyond the area of databases,
> but, for instance, in the area of keywords describing the content of a
> photon list.
> At the moment I am using provisional, local conventions for the designation
> of columns for a photon list : for instance I call THETA and PHI the angular
> cooordinates of a photon (from the optical axis), X and Y the coordinates in
> micron on the focal plane, and XPOSITN YPOSITN the coordinates in detector
> pixels ...or I call ENERGY true energy in keV, but PHA energy in channels,
> and I ignore whether they are consistent e.g. with EXSAS or PROS - which
> I am told are not compatible among them)

I certainly agree that conventions for keywords names should be agreed upon 
in the various subfields of astronomy, but they should at least be compatible
with the existing standard. Specifically, A VALID FITS FILE MUST USE SI
UNITS. This tends to be repeatedly forgotten, even though it was already 
explicitly specified in the original FITS paper.
X and Y should therefore be stored in metres, not microns, ENERGY in Joules
not kev (or cm**-1, or Kelvin, or GHz),  and so on...
Also, there are some well established conventions to store projected angular
coordinates (RA---GLS or GLAT-ATF or...) though I am not sure they are part
of the standard yet.

	Thierry Forveille
	Observatoire de Grenoble
	forveill at gag.observ-gr.fr

---

From: Lucio Chiappetti <lucio at ifctr.mi.cnr.it>
Date: Wed, 07 Oct 92 16:19:26 ITA

In reply to a posting of mine (which in turn replied to some questions
by Bill Pence) :

On Mon, 5 Oct 1992 18:01:00 GMT thompson at stars.gsfc.nasa.gov said:
>>
>>>A common example of this problem is the Modified Julian Date
>>>which has a value something like 48043.8797453703700742
>>
>>  I am inclined to think this is somehow related to the need of expressing
>>  time quantities like this.
>
>        (He goes on to describe suggested definitions of the quantities ANGLE,
>        HANGLE, DATE, DATETIME, and TIME.  Definitions omitted to save space.)
>
>I strongly suggest looking into the document "Time Code Formats" from the
>Consultative Committee for Space Data Systems, CCSDS 301.0-B-2, April 1990.

  I knew about this standard from a previous posting (may be by the
  same author, I have not looked it up) in fitsbits or WGAS, which I
  have noted down
  (perhaps for the disastrous state of our postal system unfortunately I tend
   to consider any document NOT available via network as not existing)
>
>discussed there in some detail.  The suggested format for ASCII encoding is
>briefly outlined below for the following example (5 Oct 1992 at 1:23:45 UT)
>
>                           1992-10-05T01:23:45.000Z
>

   However perhaps I gave the WRONG impression that the topic of my mail
   was to establish conventions for ASCII encoding |
   That was not my primary aim | I intended to solicit a discussion about
   NAMES and CONTENTS of keywords from a semantic and logical point of view
   (the fact that those keywords need an ASCII encoding is specific to a
   FITS implementation, while my concern goes beyond FITS into the native
   representations used by analysis systems ... which MAY be non FITS but
   SHOULD be FITS-compatible)
   I am ready to comply with whatever ASCII encoding for date-times will
   be agreed for FITS in addition to the one currently used in DATE and
   DATE-OBS.
   My interim solution was not so different from the CCSDS scheme and
   had that in mind since it is very easy to map it ;

                        1992-10-05-01:23:45.00

   actually it is the notation used by VMS. I find more LEGIBLE to have
   an hyphen instead of a T, and no Z at the end (which makes me think
   of "Zulu" timezone notation), but it is really trivial to switch to
   the other notation.

   See more below on standards ....

>not kev (or cm**-1, or Kelvin, or GHz),  and so on...

   Again I gave the WRONG impression that my concern was on units of
   measurement, while I was more concerned about the fact the file be
   self-documenting about the units used, and about establishing conventions
   for names, contents and "binary" representation.

   Concerning SI units and alike, when I was a student I was an enthusiast
   of the SI, and I swore never to use cgs and similar things.
   When I later started my research life I realized one needs to be more
   flexible. I am not scandalized to use Angstroms or keVs (actually I
   cannot THINK of an X-ray spectrum if not in photon/cm2/s/keV vs keV,
   and of an UV spectrum if not in erg/cm2/s/A vs A  ... and I believe
   this is what everybody does ... I have never seen a paper on Ap.J. with
   photons/m2/s/J or J/m2/s/m
   Similarly the examples I made in my posting were just examples (although
   I will not find very useful to give photon energies in Joules or to
   use metres instead of microns to measure positions on a CCD plate: the
   latter example is particularly specific to Bill Pence's posting, since
   it avoids an unnecessary factor 10**6 and therefore 6 digits of -
   fictitious - precision)

   Concerning standards, therefore they also are subject to a silent
   judgement from the community, based on comfort and legibility.
   There is a nice paper by Arnold Toynbee, the historian, explaining why
   the French Revolution Calendar was not able to supplant the traditional
   calendar.
   Everybody uses Internet domains for e-mail addressing, which are legible,
   instead of the awful X.400 notations.
   Some years ago there was some proposed standard to express ALL decimal
   numbers in exponential notation in power of 1000, so a date in year
   1992 had to expressed as 1.992d3 or something like that : it fell in
   the place where everything is forgotten.

On the contrary other standards, of which FITS looks a working example,
meet the general agreement and ARE used.

I would however just reiterate my concept, that I regard the two points
treated above (ASCII-encoding and units) as completely minor in my line
of thought, while I would once again solicit a discussion on names of
keywords (and their content).

I feel that this would be beneficial AT LEAST to the X-ray community which
is now working separately at many separate missions, in fields like :

   list of photons :

   should we call photon detector-pixel positions X and Y, or XPOSITN and
   YPOSITN (8-char FITS names) or whatever ?
   should we call photon energy expressed as a PHA channel (not keV) with
   the name of ENERGY (I would say no and reserve it for keV, Joule or
   whatever physical unit) or PHA or what else ?

   time profiles (this perhaps also beyond X-ray astronomy) :

   it is important to agree a convention on the format of such files as
   binary tables, on the names of the columns and on the additional
   keywords needed to specify time of each bin

I hope to listen to some discussion on the latter topics in the near
future.

-----------------------------------------------------------------------------
       A member of  G.ASS : Group for Astronomical Software Support
-----------------------------------------------------------------------------
Lucio Chiappetti - IFCTR/CNR Milano        |         U N I C U I Q U E
via Bassini 15 - I-20133 Milano - Italy    |          System  Manager
Internet: LUCIO at IFCTR.MI.CNR.IT            |           FORTUNAE SUAE
Decnet:   IFCTR::LUCIO (39610::LUCIO)      |
Bitnet:   EXOSAT at IMISIAM                   |
-----------------------------------------------------------------------------

---

From: thompson at stars.gsfc.nasa.gov (William Thompson, code 682.1, x2040)
Date: Wed, 7 Oct 1992 22:02:00 GMT

In article <9210071555.AA26547 at fits.cv.nrao.edu>, Lucio Chiappetti <lucio at ifctr.mi.cnr.it> writes...

	(stuff deleted)

>   keywords needed to specify time of each bin

I would just like to make two comments on this general topic.  No, three
comments--the zeroth comment being that it's a very difficult thing to do.

One thing to avoid is the mistake I've been told the Planetary Data System made
with their Object Descriptor Language (although I haven't had any experience
with it directly).  As I understand it, basic names got frozen in at an early
stage, so that for example the keyword "FLUX" could only be used with particle
fluxes, and light measurements could only be referred to as a "BRIGHTNESS"
(which is a different thing).

Of course, you might say that brightness is the quantity that astronomers are
really interested in, but I argue that one does not want to limit oneself to
specific "most-common" observation modes and data types.  I can think of
situations where flux would be the fundamental quantity of measurements, e.g.
solar UV luminosity monitoring, or the flux distribution as a function of
height in a planetary atmosphere from a planetary probe.  Flux is the
fundamental quantity when concerned with energy transport in a plane-parallel
atmosphere, and a standard file format should be able to handle data from
theoretical calculations as well as from actual observations.

Secondly, I would just like to emphasize that such standardization should
proceed outside of the formalization process of the FITS standard itself (which
I'm sure was Lucio Chiappetti's intent as well).  Any such effort could only
consider a subset of the possible types of observations and the forms the data
could take.  Granted, it could possibly cover a large fraction of the
astronomical data actually taken, but there will always be data products and
observations which don't fall into the mold.

It may be best for this effort to proceed along disciplinary lines, rather than
to try to encompass all possible astronomical data into one grand scheme.  For
instance, I'm aware of a standardization process of this sort for radio
interferometry data, I believe organized by the Green Bank group.  Simple
image data already has some standardization applied to it, since that's what
FITS was originally designed for.  Just in general, I think more progress will
be made if the goals remain more focused.

Bill Thompson