Subject: Re: VAX R*8 to FITS/IEEE R*8 conversion prob

From: cflatter at cv3.CV.NRAO.EDU (Chris Flatters)
Date: Thu, 17 Dec 1992 18:09:12 GMT

In article AA05949 at tetra.gsfc.nasa.gov, pence at tetra.gsfc.nasa.gov (William Pence) writes:
>As an aside, I'm sure you are aware that the VAX has 2 different R*8 data 
>representations.  The default D_floating representation which you are probably using
>reserves 8 bits for the exponent and 55 bits for the mantissa.  The G_floating
>representation on the otherhand is similar to the IEEE R*8  representation: both
>have 11 bits for the exponent and 52 bits for the mantissa.  The G_floating
>and IEEE values are still not identical though because the VAX interprets the
>exponent and mantissa numbers as normalized, ("hidden bit normalization") which
>effectively gives you another bit of precision compared to the IEEE value.
>If you were to use G_floating on the VAX, you probably would not notice the
>loss of precision in converting the data back and forth from FITS.

IEEE values are also normalized as long as they are greater than or equal
to 2 ** E_min, where E_min is the minimum exponent.  An IEEE-754 double
has a precision of 53 base-2 digits (identical to VAX G-floating) as long
as E >= E_min (E_min = -1022).  VAX G-floating and IEEE-754 doubles probably
differ in the bias of the exponent (this is the difference between normalized
IEEE singles and VAX F-floats).

Incidentally, if you think about it you will realize that the exponent can
not be normalized.

	Chris Flatters
	cflatter at nrao.edu