Subject: Crab et al., a warning Re: wcs.ps

From: Lucio Chiappetti <lucio at ifctr.mi.cnr.it>
Date: Fri, 2 Apr 1999 09:50:32 +0200 (MET DST)

On Thu, 1 Apr 1999, Eric Greisen wrote:

> Arnold Rots writes:
>  > I noticed that your list was missing the unit that is dear to many HEA
>  > types' hearts: the "Crab".  

>      if the Crab is as well defined as the Solar units, we could add
> it.  What is the definition?

About the definition of "1 Crab" I guess it could be good to agree a standard
definition. I believe everybody has one's own variant ...

In principle it should be a "flux" unit (ANY flux, from erg/cm2/s to ph/cm2/s
or even cts/s for a particular mission instrument) which results from the
comparison of the integrated flux (in a "given" energy band, see below) of
a Crab-like spectrum, i.e. the spectrum of the synchrotron component in the
Crab nebula   dN/dE = 9.8 * E**(-2.1) * absorption(NH=3.10E21).

At least that's what I use. Little variations in norm, spectral index and NH
are possible according to your favourite reference.

However what is not well defined is the band in which the integral is taken.
In principle you can integrate the above spectrum in ANY band, and compare the
flux of your source in the same band to the Crab-like spectrum.

In practice the definition was born in 2-10 (or 2-6) keV X-ray astronomy, and
it would be of less use and ease in other bands ... of course a, say, 1 mCrab
source in 2-10 keV would'nt be a 1 mCrab in 100-300 keV if the spectrum is
steeper or flatter than 2.1 !

                           BUT .. BUT ... BUT ...

One moment. Are we talking here of standardization of unit indication in
generic FITS files (tables) or just of WCS ?
And does not WCS apply "primarily" (or "originally") to IMAGES and
"secondarily" (or "more recently") to SPECTRA ?

I would expect it would be very difficult that the "spatial" axes of an image,
or the "dispersion" axis of a spectrum, would be measured in Crab units (or
in Ohm, if that matters).
So aren't we discussing about "the sex of angels" ?

                            AND MORE BUT'S ...

Even if we would be discussing standardization of units in generic tables, I
reiterate my provocation, is it really worth while to go in extreme detail ?
For what are essentially "plot labels" ?
I'd expect humans (astronomers) be able to read a column heading and know what
it is (and convert to their favourite customary units) anyhow, while software
will just copy and use labels as they are, as "semantic-less" strings.
Do we really expect to have an all-encompassing universal routine library
which will convert any unit to any unit ?

I heard a saying "build a system than any fool can use, and only a fool will
want to use it". But it could also be reversed "build a standard that is too
clever, and nobody will be clever enough to want to use it".

I'm thinking of a recent discussion about a largish database private to a
smallish consortium (less than 10 institutes), of course to me it was obvious
to say FITS had to be used for interchange and not ASCII ... but it was
pointed to me that the programming competences of the "mean quadratic
astronomer" were quite below that mark ...

And if he/she will have difficulty in using age-old consolidated good old
plain FITS, is it likely it will adhere, or even check, a "marginal"
convention ?

Even when major agencies (funded also by my taxes) involved in major missions
produce and distribute tools which measure fluxes in eV/mm2/s (!) instead of
the more usual erg/cm2/s ?  :-) [sorry, I could not resist]

----------------------------------------------------------------------------
Lucio Chiappetti - IFCTR/CNR - via Bassini 15 - I-20133 Milano (Italy)      
For more info : http://www.ifctr.mi.cnr.it/~lucio/personal.html             
----------------------------------------------------------------------------

---

From: Eric Greisen <egreisen at valen.cv.nrao.edu>
Date: Fri, 2 Apr 1999 10:40:22 -0500 (EST)

> definition. I believe everybody has one's own variant ...

      Arnold's earlier reply that "we don't want to go there" about
Crabs is well explained by your description of them.  I have no
intention of putting them in the paper, until the community gets a
much better definition.

 > "secondarily" (or "more recently") to SPECTRA ?

     WCS applies to any observation of a physical quantity.  The issue
is most pressing when we attempt to associate coordinates with the
voxels in an image, but images as well as single pixels can and do
occur in tables and elsewhere.

 > So aren't we discussing about "the sex of angels" ?

     Derived quantities may also be sampled into images or tables and
expressed against whatever physical coordinates seem appropriate to
the Physics of the observation.  I have removed trigonometric
functions from wcs.ps but am concerned about matters that are indeed
functions of e.g. cos(latitude).  Maybe it is wrong to remove them.
I agree that Ohm is unlikely in astronomical data and so we are free
to change its spelling, but Crabs and Angstroms are not unlikely.

 > For what are essentially "plot labels" ?

     WCS are not plot labels.  They have that function, but they have
a great deal more as well.  If units are expressed properly, then they
may be parsed in software and converted to other comparable units,
after which vast tables may be searched for matches, sub-set
selection, etc etc etc  This is now done with tabular data a great
deal with for example the NVSS survey here at NRAO and, judging by
Francois Ochsenbein's remarks, many of the tables at the Strasbourg
data center.  WCS for imagery is often used to regrid one image to the
coordinates of another so that the physics of the two may be compared
visually and numerically.  WCS is not a plot tool, it is a Physics
tool.

 > it is (and convert to their favorite customary units) anyhow, while software

      Yes - I do not have such a thing.  But I am told that numerous
Object Oriented tool sets (e.g. aips++) do have exactly that.  Current
data sets, tables, and images are already awfully large to depend on
astronomers to read them visually.  And as you point out below, the
working astronomer may not be the best equipped to understand
technical details of what you would force him to read.

 > clever, and nobody will be clever enough to want to use it".

      FITS as a whole was claimed when we started to be too hard
because we allowed and required BINARY data.  Where would we be today
if we had done only character-form data?  Where are ASCII tables
today?  I agree that we can get too fancy and I think that some
hierarchal relational data-base proposals that I have heard are just
that.  However, we do need also to get things right.  To leave things
inexact and prone to error and misinterpretation simply because "it is
too hard" is wrong.  The whole subject of WCS is hard and that is why
we have had to leave it until later in the FITS process.  It was
needed at the beginning (hence CDELTA, CTYPE etc in the first FITS
paper) and has been used in detail in some software from the early
1980s (e.g. aips).

 > plain FITS, is it likely it will adhere, or even check, a "marginal"

     The initial software may not be the best that it could be.  But
with a good standard and the 3-sigma astronomer it can get better with
time if there is a standard to which it may aspire.  FITS tapes are
usually written even today with little or no WCS info.  They are
useful, but would be very much more useful if they had better WCS,
especially WCS that software would recognize.

 > the more usual erg/cm2/s ?  :-) [sorry, I could not resist]

     But there is no problem with either of these units if they are
described properly.  The point of wcs.ps is to allow even misguided
folks to express their misguided opinions in such a fashion that other
misguided folks may correctly convert the data to their particular
religion.

     I knew that the units section would get people to talk about the
paper.  I just hope you are all also reading the rest of it and Paper
III (and Paper II when it becomes available).  The spectroscopy paper
is new and important.  FITS itself has guided how astronomers think of
their data and how software systems think of data for their users.
These WCS papers will, if adopted, affect how you will see data and
how you will receive them from your colleagues.

Eric Greisen

---

From: Lucio Chiappetti <lucio at ifctr.mi.cnr.it>
Date: Fri, 2 Apr 1999 18:52:21 +0200 (MET DST)

> intention of putting them in the paper, until the community gets a

  Ok for letting the Crab out

> occur in tables and elsewhere.

  hmm ... tables within tables, or images within tables, look to me what
  you'd call "fancy" stuff.

  I do appreciate that an image can be other than a sky image (see one
  example further below), but most of the WCS image paper is for astrometry,
  or anyhow sky coordinate handling (and that's not particularly my field).
  
  Also a spectrum is in generally something quite defined (flux vs energy)
  although there can be peculiarities (and the WCS spectral papeer addresses
  many of them from optical astronomy, again things I'm not particularly
  familiar with) particularly for "raw" or "nearly raw" data.

  I could imagine the need of some standard for flux conversions of "reduced"
  spectra. I mainly think of the conversion of dN/dE vs E from/to Fnu vs nu,
  Flambda vs lambda, nuFnu (SED) etc.  with proper handling of error bars,
  and clear distinction of the case the single points in a spectrum are
  monochromatic or broad band fluxes (in which case proper conversion need
  to know the form of the underlying continuum !)

  But "any other thing" is just an "histogram" or "plot" of some y quantity
  vs some x quantity. Is it worthwhile to commit to a very hard enterprise
  of a fully general handler ?

  Or is it better to consider only the cases of images and spectra, and
  at most light curves and other timing analysis data structures (thinking of 
  x-axis which could be in time, frequency, or phase) ...

> to change its spelling, but Crabs and Angstroms are not unlikely.

Angstrom are extremely likely on the X (dispersion, or energy) axis of an UV
or optical spectrum. Crabs (a flux unit) are totally unlikely (probably
meaningless) as units of the axes of an image or a spectrum, and even if they
are flux units, quite unlikely also on the "Z axis" (BUNIT), or the Y axis of
a spectrum. They are likely to occur only in pure tables (catalog entries).

> a great deal more as well.

I've never told that WCS were plot labels, my reference to plot labels was to
the physical unit convention (i.e. the part of the Appendices which were not
immediately related to WCS). It was even farther from my mind to criticise or
diminish the WCS effort ! I do appreciate at least some usages of "original"
WCS for astrometry, and of course also the ones in the spectroscopy paper.

My doubts were on the insertion of the ENTIRE business of units into the WCS
approval process, particularly for what concerned "exotic" units, or anyhow
units outside the ra-dec-energy-flux (astrometry/spectroscopy) context.

After all the important thing in WCS are "internal transformation" (like sky
projections going beyond the original CRPIX CRVAl CDELT, or intrinsic usage of
"log(units)"   ... I tend myself to keep chi-square grid vs spectral index and
column density NH in form of images ... and of course NH has a log range ...).

But once there is a clear convention (-> WCS) to know that the value at pixel
coordinates i,j correspond a given log(NH),gamma, or instead to
log(pears),apples ... NH and gamma, or pears and apples, become plot labels.