Subject: Identification of a FITS byte stream

From: [email-masked] (Archie Warnock)
Date: Wed, 18 Aug 93 12:45:47 -0400

The recent discussion on suitability of formats for archiving has
reminded me of the following small issue I'd like to raise here.  How
does someone who knows nothing about FITS identify an arbitrary byte
stream as FITS?

Note - if we _know_ about FITS, we can tell it's FITS by the presence of
the SIMPLE keyword at the start of the byte stream.  But there's nothing
required in the byte stream that says "I'm a FITS file and here's where
you go to find out about me."  Should such a comment be mandatory?
--
-- 
_______________________________________________________________________
-- Archie Warnock              Internet:  [email-masked]
-- Hughes STX                  "Unix --- JCL For The 90s"
-- NASA/GSFC                   Project STELAR: WAIS to do science

---

From: [email-masked] (Pat Murphy)
Date: Wed, 18 Aug 93 15:58:15 -0400

On Wed, 18 Aug 93 12:45:19 -0400, [email-masked] (Archie
Warnock) said to WGAS-L:

AW> Note - if we _know_ about FITS, we can tell it's FITS by the presence of
AW> the SIMPLE keyword at the start of the byte stream.  But there's nothing
AW> required in the byte stream that says "I'm a FITS file and here's where
AW> you go to find out about me."  Should such a comment be mandatory?

Certainly it should not be mandatory for FITS readers to look for and
barf if they don't find it, but I don't think that's what you meant.  I
see nothing wrong with having existing FITS writers add something like:

COMMENT   The FITS format is documented in the following references:
COMMENT   Wells, D.C., Greisen, E.W., Harten, R.H.: 1980, Astronomy
COMMENT     and Astrophysics Supplement 44, 363.

and so on.  It's too bad something like this couldn't have been added
from day one, but then hindsight usually is 20/20.

Just MHO.
				- Pat
--
==========================================================================
| Patrick P. Murphy, Ph.D.                Scientific Programming Analyst |
| National Radio Astronomy Observatory    Net:       [email-masked]    |
| 520 Edgemont Road                       Phone:     (804) 296-0372      |
| Charlottesville, VA 22903-2475          VoiceMail: (804) 980-5889      |
|      "I don't believe in the no-win scenario"  --- James T. Kirk       |
==========================================================================

---

From: [email-masked] (Archie Warnock)
Date: Wed, 18 Aug 93 16:21:11 -0400

> see nothing wrong with having existing FITS writers add something like:

That's not what I intended, though in any event I'd expect a FITS reader
to be more robust than that.  It's people who don't have FITS readers
and don't even know what FITS is that I'm worried about.

> COMMENT     and Astrophysics Supplement 44, 363.

That's exactly what I had in mind - we have to tell the world that "this
is a FITS file and here's where to get more information".

> from day one, but then hindsight usually is 20/20.

Ain't it the truth.  Now, the real question is, at what level to we
implement this?  Recommendation?  Strongly recommended?  Required?  How
serious are we about identifying the files?

Archie

---

From: [email-masked] (Hans-Martin Adorf)
Date: Thu, 19 Aug 93 02:49:54 -0400

>you go to find out about me."  Should such a comment be mandatory?

I support the view that there should be some mechanism by which a FITS file
identifies itself. Recently I found that the byte stream transmitted by the
ZModem protocol (used for transferring files between a host and a personal
computer) identifies itself and may trigger appropriate actions (e.g. a
protocol selection for further transmission) on the recipient's side.

I'd like to see something similar happen for FITS files, particularly in
view of the rapid spreading of wide-area information services on the
Internet. At least there should be an agreeement about the default file
extension (some people use .FTS others use .FITS).

Hans-Martin Adorf
------------------------------------------------------------------------------
                      **** note the new postal code ****
Hans-Martin Adorf                                     [email-masked] (Internet)
Space Telescope - European Coordinating Facility         [email-masked] (UUCP)
European Southern Observatory                     [email-masked] (EARN)
Karl-Schwarzschild-Str. 2                                    ESO::ADORF (SPAN)
D-85748 Garching b. Muenchen                         +49-89-320 06-261 (Phone)
F.R. Germany                                           +49-89-320 06-480 (FAX)
----------------------------------------------------------------------------
--

---

From: [email-masked] (Mike Newberry)
Date: Mon, 23 Aug 93 18:49:40 -0400

>protocol selection for further transmission) on the recipient's side.

	There is such an identifier already in place: the SIMPLE
	keyword.

>extension (some people use .FTS others use .FITS).

	For those of us who use PC's running DOS, the use of a filename
	extension such as .FTS uses 3 of the precious 11 characters
	that we can use for file names.

Mike Newberry
Steward Observatory
University of Arizona

---

From: [email-masked] (Hans-Martin Adorf)
Date: Tue, 24 Aug 93 01:04:14 -0400

>        There is such an identifier already in place: the SIMPLE

That is, as far as I can see, insufficient for self-identifying a
byte-stream as being a FITS file. It only specifies a particular
FITS-format.

>        that we can use for file names.

Fair enough. I could live with .FTS or .FIT (also used). My only other
comment would be: anyone who wants to do serious astronomy at the end of
the 20th century, should get hold of a professional computer.

Hans-Martin Adorf

---

From: [email-masked] (Pat Murphy)
Date: Tue, 24 Aug 93 09:58:28 -0400

On Tue, 24 Aug 93 01:04:14 -0400, [email-masked] (Hans-Martin Adorf)
said to wgas-l: 

(quoting someone who quoted someone else as saying:)
>>identifies itself. Recently I found that the byte stream transmitted by the
 ( ... )
>        There is such an identifier already in place: the SIMPLE keyword

HA> That is, as far as I can see, insufficient for self-identifying a
HA> byte-stream as being a FITS file. It only specifies a particular
HA> FITS-format.

I suspect what the previous poster was thinking of might have been
something analogous to the "file" command on unix systems.  This command
identifies what it thinks is in a file based on a set of rules, and it
can be (and has been) extended to deal with, e.g. gif, jpg, etc files.
It would not take much to extend it further to check if the first 10
bytes of the file are "SIMPLE = " (did I get the = in the right place?).

Agreed that this is not what the original gist of the debate was, i.e.
an identifier for human eyes that leads to references in a library or
elsewhere that fully describe the FITS format.  And whether such
additional COMMENT keywords should be mandatory, strongly recommended, or
what.  I tend to lean towards strongly recommended (and that we should
lean on fits programmers to put something into their fits writers once we
can agree on what to put in).

I won't comment on the rest of the article.

				- Pat

---

From: [email-masked] (William L. Sebok)
Date: Tue, 24 Aug 93 10:15:44 -0400

> bytes of the file are "SIMPLE = " (did I get the = in the right place?).

Indeed we have the line:

0	string		SIMPLE\040\040=	FITS file

in /etc/magic, the file that controls the behavior of the "file" command.

Bill Sebok      Univ. of Maryland, Astronomy    Internet: [email-masked]

---

From: [email-masked] (Doug Mink)
Date: Tue, 24 Aug 93 10:39:55 -0400

A minor flaw in the use of "SIMPLE  =" to detect FITS files is that it
flags IRAF STF format header files as FITS files, too.
-Doug Mink ([email-masked])
 Telescope Data Center
 Harvard-Smithsonian Center for Astrophysics

---

From: Lucio Chiappetti <[email-masked]>
Date: Tue, 24 Aug 93 10:50:24 -0400

> flags IRAF STF format header files as FITS files, too.

  What about "SIMPLE  =                    T" ?

---

From: [email-masked] (Mike Newberry)
Date: Thu, 26 Aug 93 19:40:45 -0400

>From [email-masked] Mon Aug 23 22:04:46 1993
Date: Tue, 24 Aug 93 01:02:57 -0400
Comment: AAS Working Group for Astronomical Software
Originator: [email-masked]
Reply-To: <[email-masked]>
Sender: [email-masked]
Version: 5.41 -- Copyright (c) 1991/92, Anastasios Kotsikonas
From: [email-masked] (Hans-Martin Adorf)
To: Multiple recipients of list <[email-masked]>
Subject: Re: Identification of a FITS byte stream
Status: R

>byte-stream as being a FITS file. It only specifies a particular

	But simply reading the word "SIMPLE" as the first 6 bytes of
	the FITS file has to be as good as identifying a TIFF format file
	by reading the number "42" in ASCII as bytes 3 and 4 of the
	TIFF file, right?

>the 20th century, should get hold of a professional computer.

	But MIRA running on a 486/33 or 486/66 runs a lot faster and more
	user friendly that IRAF running on a Sun Sparcstation.  MIRA runs
	in 32-bit mode on a DOS-based system and can process enourmous images
	rapidly.  I don't know if that's professional enough or not.  MIRA
	is available from Axiom Research, Inc. here in Tucson, AZ.  See
	advertisement in August 1993 Sky & Telescope magazine.

---

From: [email-masked] (Archie Warnock)
Date: Fri, 27 Aug 93 08:15:51 -0400

> 	by reading the number "42" in ASCII as bytes 3 and 4 of the

Right - and neither one is sufficient to identify the format to someone
not already familiar with the format.
_______________________________________________________________________
-- Archie Warnock              Internet:  [email-masked]
-- Hughes STX                  "Unix --- JCL For The 90s"
-- NASA/GSFC                   Project STELAR: WAIS to do science