Subject: Floating point NaN values as Keywords

From: Bob Garwood <bgarwood at nrao.edu>
Date: Mon, 11 Aug 1997 16:35:13 -0400

Just as it is useful to store NaN (not a number) values in
binary tables to indicate an invalid number, it would be equally
useful to have some convention for indicating a NaN as
a keyword value.

FORTRAN-77 doesn't say anything about NaNs so this would
appear to be not allowed.

Nevertheless, it would be useful.

The only alternative when a certain keyword is expected
to be there is to invent a value that is a lie (which leaves
more than just a bad taste in my mouth) or to not write
that keyword (which leaves someone down the road wondering
if the keyword was left out because it was unknown or some
writer was just sloppy and lost that particular keyword, and
could be a particular problem if some convention requires
that that keyword be present).

In the Single Dish FITS convention that has been firming
up for years (often called the Green Bank convention, at least
in part), columns in a binary table which have a constant
value and which have a name with 8 characters or less can
be represented as keywords (called virtual columns).
Without the ability to write a NaN as a keyword, columns which
are constant NaNs could not be made virtual in this convention.

One data point for consideration:

   Pense's cfitsio library will write out "NaN" when given a
keyword value that is a NaN, at least under Linux and Solaris.

What does anyone else feel about NaN values and keywords?

Cheers,
   Bob Garwood
   bgarwood at nrao.edu

---

From: seaman at noao.edu (Rob Seaman)
Date: 12 Aug 1997 04:34:14 GMT

Bob Garwood <bgarwood at nrao.edu> writes:

> Pense's cfitsio library will write out "NaN" when given a keyword
> value that is a NaN, at least under Linux and Solaris.

Is this formatted as a string in this case, or do the NaN "digits"
appear as a real valued keyword?  The latter would be illegal.

Any viable solution will likely have to agree with the original FITS
FORTRAN-derived format rules.  The golden "once FITS, always FITS" rule
doesn't actually forbid expanding the definition of a real number by
adding newly allowed values - but this seems like a mighty big change.
Note that restricting (as opposed to "deprecating") previously allowed
values would be forbidden, on the other hand.)

> What does anyone else feel about NaN values and keywords?

Is NaN the only IEEE feature that needs to be considered?  The mapping
between a binary IEEE data representation and an ASCII FITS keyword is
unlikely to ever be perfect.

I see no easy way to support NaN for any keywords that are required to
be type real by the standard (e.g., BSCALE/BZERO).  FITS "user" keywords
are not explicitly typed in general, on the other hand, and an otherwise
real keyword could be retyped as a string to express a NaN.

This presents a bit of a complication for software to handle, but in
practice shouldn't be too big a deal.  Many packages will likely parse
a real/string value correctly already - assuming they can handle NaN's,
that is.  Only applications that need to read that particular keyword
will care in any event.

Rob Seaman
-- 
seaman at noao.edu, http://iraf.noao.edu/~seaman
NOAO, 950 N Cherry Ave, Tucson AZ 85719, 520-318-8248
PGP: 98 8D 8B 49 74 9A 41 88  3A 43 87 54 51 BF 30 4B

---

From: Bob Garwood <bgarwood at nrao.edu>
Date: Tue, 12 Aug 1997 12:20:42 -0400

> appear as a real valued keyword?  The latter would be illegal.

It apears as a real valued keyword, so, I agree, this seems illegal.

Someone here just point out to me that the FITS definition allows
for the following construct as valid FITS:

K       =                      / 

i.e. no value given for the keyword.  Apparently in the FORTRAN-77
list-directed read this would be interpretted as leaving the
value of K unchanged.   For my purposes, where the desire is to
provide a required keyword for which, in some specific cases, the
value of this keyword is unknown, the above is suficient, I think
to convey that.  It doesn't provide a bogus value (i.e. it isn't
a lie) and it also doesn't omit the keyword (leaving the reader to
wonder if that was an intentional omission or not).

-Bob Garwood

---

From: Arnold Rots <arots at xebec.gsfc.nasa.gov>
Date: Tue, 12 Aug 1997 14:00:01 -0400 (EDT)

> values would be forbidden, on the other hand.)

However, if one wants to be able to distinguish a NaN unambiguously
from the string value "'NaN'", then one would have to extend the
definition of a float to include the "digits" NaN (case significant?).
I understand that this is a major change that may have to get the
IAU FWG blessing, but if one does this, one might as well do it right.

> unlikely to ever be perfect.

Indefs and Infins?

> that is.  Only applications that need to read that particular keyword

To complicate matters further, there is the precedent of arrays
of numbers encoded as comma-separated lists in a string ...
Retyping a float keyword as a string could work, but requires
two things: the string should not represent a number (such as '42')
and only NaNs could be thus encoded.

> Rob Seaman

  - Arnold Rots

---

From: bschlesinger at nssdca.gsfc.nasa.gov (Barry M. Schlesinger)
Date: 13 Aug 1997 09:36 EDT

In article <33EF7781.1DC11185 at nrao.edu>, Bob Garwood <bgarwood at nrao.edu> writes...
>to be there is to invent a value that is a lie ...  or to not write
>that keyword ... 

>In the Single Dish FITS convention... 
>that has been firming
>...

As NOST 100-1.1 puts it, "The value shall be the ASCII text
representation of a string or constant."   What is the ASCII
representation of a NaN? 

				Barry Schlesinger
				FITS Support Office
				GSFC/ADF

---

From: Chris Flatters <cflatter at nrao.edu>
Date: Wed, 13 Aug 1997 10:11:18 -0600

> representation of a string or constant."   What is the ASCII

There is no standard ASCII representation of a NaN nor even a
requirement that there be one.  The relevent paragraph of
IEEE-754 states

	If decimal to binary conversion over/underflows,
	the response is as specified in Section 7 [trap
	or +/- infinity depending on mode].  Over/underflow
	and NaNs and infinities encountered during binary
	to decimal conversion should be indicated to the
	user by appropriate strings. NaNs encoded in decimal
	strings are not specified in this standard.
				[Section 5.6]

Summarizing: there a recommendation (not a requirement ---
note the use of "should" rather than "shall") that NaNs be
represented be some "appropriate string" but there is no
specification for the form of that string; if this
recommendation is followed there is no requirement that
such a string should be recognized if encountered when
reading a floating-point number.

In practice, textual representations of NaNs show significant
variability including NaN, -NaN, and QNaN as well as longer
representations.

Without a specific agreement on the format to be used for
indeterminate values of FITS keywords it would not be advisable
to use such values.

	Chris Flatters
	cflatter at nrao.edu

---

From: pence at tetra.gsfc.nasa.gov (William Pence)
Date: Tue, 19 Aug 1997 15:15:52 -0400

Bob Garwood wrote:

>    Pence's cfitsio library will write out "NaN" when given a

This behavior of CFITSIO is not intentional, and is just an accidental
result of a 'non-standard' feature in these C compilers.  I believe
these keywords are illegal according to the FITS standard, so CFITSIO
should return an error if one attempts to write a NaN value to a
keyword.  I'll try to trap these cases in the next release of CFITSIO.

As to the question of how to represent an undefined keyword value
(which is what a NaN represents) I would support the form illustrated
in Garwood's follow-up message:

> value of K unchanged. 

This usage has apparently always been legal in FITS (since it follows
the rules for list-directed Fortran-77 reads) but has not been widely
known.  (It is mentioned in the draft of the next version of the FITS
Standard, however).

The latest versions of CFITSIO and FITSIO support this usage.

-Bill Pence
 HEASARC/GSFC