Subject: Identification of a FITS byte stream

From: warnock at Hypatia.gsfc.nasa.gov (Archie Warnock)
Date: 18 Aug 93 15:18:00 GMT

The recent discussion on suitability of formats for archiving has
reminded me of the following small issue I'd like to raise here.  How
does someone who knows nothing about FITS identify an arbitrary byte
stream as FITS?

Note - if we _know_ about FITS, we can tell it's FITS by the presence of
the SIMPLE keyword at the start of the byte stream.  But there's nothing
required in the byte stream that says "I'm a FITS file and here's where
you go to find out about me."  Should such a comment be mandatory?
--
_______________________________________________________________________
-- Archie Warnock              Internet:  warnock at hypatia.gsfc.nasa.gov
-- Hughes STX                  "Unix --- JCL For The 90s"
-- NASA/GSFC                   Project STELAR: WAIS to do science

---

From: pence at tetra.gsfc.nasa.gov (William Pence)
Date: Wed, 18 Aug 93 13:16:19 EDT

>you go to find out about me."  Should such a comment be mandatory?

I think it would be a good idea to 'strongly recommend' that all
FITS writers should add a standard comment citing a reference to a FITS 
definition document. This cannot be made manditory however without
invalidating all the existing FITS files which don't have such a 
comment, unless there is a grandfather clause that exempts all 
existing FITS files.

What is the best reference to cite?  The old A&A Suppl. papers, which
are somewhat out of date but are at least widely available in any
Astronomical library?  Or the new NOST FITS definition document 
which is much more complete, but may not be as easily accessable
50 years from now when someone is trying to decifer a dusty FITS tape?
Or both?

Also, should it be recommended that this comment be located as near as
possible to the beginning of the FITS file, to make it easy to spot?
Presumable this comment would only be needed in the primary array, and
there would be little reason to repeat it in every extension.

-Bill Pence

---

From: warnock at Hypatia.gsfc.nasa.gov (Archie Warnock)
Date: 19 Aug 93 20:54:47 GMT

>comment, unless there is a grandfather clause that exempts all 

Agreed - "strongly recommend" is good.

>50 years from now when someone is trying to decifer a dusty FITS tape?

Yeah - that's a problem isn't it.  Forward referencing in the literature
is hard.  I think that the original A&A paper is probably enough of a
lef-up on the problem for starters (since it tells you something about
parsing the initial header), but I think more is probably better.  The
ultimate solution would be to get the NOST document into the refereed
literature somehow.

>there would be little reason to repeat it in every extension.

I don't think position is critical - we're doing this primarily for
machine parsing, rather for a human reading a dump.  And once is
probably sufficient.  Hmmm...

---

From: gsh7w at fermi.clas.Virginia.EDU (Greg Hennessy)
Date: Fri, 20 Aug 1993 15:21:10 GMT

Archie Warnock  writes:
#I don't think position is critical - we're doing this primarily for
#machine parsing, rather for a human reading a dump.  And once is
#probably sufficient.  Hmmm...

Err, I may have missed something, but I find it hard to imagine that a
computer program is going to care about the reference of the FITS
file. A grad student trying to figure out what is on the tape 10 years
later when the spectrometer that took the data no longer exists, and
the professor who took the data was not the observer, but the
collaborator of the observer *IS* going to find that information
useful. 

And yes, I actually had to do that task. Got it to work, although I
had to figure out that the file was not *really* fits, since it was
written in byte-swaped format to the tapes. VAXEN drive me crazy at
times. 

--
-Greg Hennessy, University of Virginia
 USPS Mail:     Astronomy Department, Charlottesville, VA 22903-2475 USA
 Internet:      gsh7w at virginia.edu  
 UUCP:		...!uunet!virginia!gsh7w

---

From: warnock at Hypatia.gsfc.nasa.gov (Archie Warnock)
Date: 21 Aug 93 20:29:54 GMT

>file. A grad student trying to figure out what is on the tape 10 years

Urp...  That's what I meant.  No really.  Would you believe it's an
obscure form of dyslexia?  No?  How about temporary insanity?  Maybe
it's not so temporary...

Anyway, what I _meant_ is that we're _NOT_ talking about machine
parsing.  Sigh...  I need another vacation to recover from my
vacation... 

>written in byte-swaped format to the tapes. VAXEN drive me crazy at

You just wouldn't believe the bizarre stuff we got submitted to
International Halley Watch with the claim that it was FITS.  It's not
nearly as universally used as we'd like to believe.

I got a standard VMS ASCII file (yep - variable length records, the
whole works) from one submitter claiming it was FITS - well, he could
type it out and it _looked_ like the samples in the papers...
--
_______________________________________________________________________
-- Archie Warnock              Internet:  warnock at hypatia.gsfc.nasa.gov
-- Hughes STX                  "Unix --- JCL For The 90s"
-- NASA/GSFC                   Project STELAR: WAIS to do science