Subject: The Year-2000 DATExxxx Proposal

From: dwells at nrao.edu (Don Wells)
Date: 23 Jul 1997 14:47:44 GMT

Dear friends of FITS,

I append the most recent draft of the year-2000 proposal. This version
of the text has received unanimous approval by all three regional FITS
committees (European FITS Committee, WGAS [North American] FITS
Committee, Japanese FITS Committee). I have therefore submitted it to
the IAU FITS Working Group [IAU-FWG] for consideration as an addition
to the FITS agreements/standards.

Please examine this proposal and post suggested modifications to
sci.astro.fits/fitsbits (or send them privately to me at
<dwells at nrao.edu>) before 1997-07-30 (one week from today), when I
expect to issue the call-for-votes to the IAU-FWG.

I am aware of one large data analysis software package which has
already made the necessary code changes to implement this proposed
agreement. I would like to be told of other implementations.

Regards,
Don Wells (Chair, IAU FITS Working Group)

------- start of forwarded message (RFC 934 encapsulation) -------
From: Arnold Rots <arots at xebec.gsfc.nasa.gov>
Sender: owner-wfc at marmoset.cv.nrao.edu
To: wfc at cv3.cv.nrao.edu
Subject: Year 2000 Proposal
Date: Thu, 3 Jul 1997 17:07:26 -0400 (EDT)

This file contains a proposal for how to revise the definition of the
DATExxxx keywords to provide a 4-digit year.  It has been developed as a
distillation of discussion on the sci.astro.fits newsgroup, electronic
mail interchanges, discussion with Preben Grosbol, and discussions within
the WGAS (North American) FITS Committee [WFC]. The previous version of
the proposal was submitted to the FITS committees by Peter Bunclark of
the Royal Greenwich Observatory, and was endorsed by the European FITS
Committee [EFC].

This newly amended version of the DATExxxx proposal is consistent with
the Bunclark/EFC version, with the following exceptions, additions, and
deletions:

1. The use of the UTC time scale is mandatory and implied for data
   that originated since 1972, unless an alternative, valid time
   scale is specified.  UT is adopted as the pre-1972 equivalent of UTC.

2. The appendix suggests an implementation of the time scale specification.
   Time scales that will be initially recognized are: UTC, TAI, ET,
   TT, TDT, TDB, TCG, TCB.

3. Support for use of the "Z" qualifier is deleted since it would be
   either superfluous or create an inconsistency.

4. The default interpretation of all DATExxxx keywords shall use the
   Gregorian Calendar.

5. The ISO 8601 appendix is removed and all relevant information included
   in the body of the proposal.

6. The exception for 19th century plates is restricted to existing files.

For a convenient summary of time scales, see:
   http://tycho.usno.navy.mil/systime.html

To summarize our concerns:
a. UTC is not defined before 1972.
b. FITS files should, internally, not mix time scales; there are files
   around that use TT, TDB, TAI, in addition to UTC.
c. Not all code needs to know about all time scales; in almost all
   cases it would not hurt if UTC were assumed.

I would suggest that WFC forward this proposal to the IAU FITS Working
Group.

  - Arnold

- --------------------------------%<----------------------------------------
 
                  Precise re-definition of DATE-OBS Keyword
                         encompassing the millennium
 
                         Peter Bunclark 1996 Nov 19
                      Amended: Arnold Rots 1997-06-27
 
                             1) Introduction
Although this document formally defines the format of the value field of
the DATE-OBS keyword, the same format is recommended for all other keywords
specifying date and time which we shall refer to collectively as DATExxxx.
The original DATExxxx keywords, including in particular DATE-OBS,  have
several shortcomings which make it desirable to alter the definition:
1.1) The year is expressed in only 2-digits;  currently, digitized astronomical
      data spans more than a century, and further, the implied most-significant
      two digits of the year will change from 19 to 20 shortly.
1.2) The timescale of DATExxxx is not defined.
1.3) The relation of DATE-OBS to the start, middle or end of an observation
     is not defined.
1.4) The order of day, month, year is least-significant first, so lists
     of dates cannot be sorted simply on the ASCII collating sequence.
 
                               2) Scope
There are three main issues addressed;
2.1) the format of date strings to be used in any DATExxxx keyword.
2.2) the future of the DATE-OBS keyword itself
2.3) the specification of the time scales (time systems) used

                    3) The Date-String format Proposal
 
3.1) A DATExxxx field in the old format of 'DD/MM/YY' will explicitly refer
   to a year 1900-1999.  The very few instances of digitized nineteenth-century
   plates represented as FITS files (only files created before this proposal
   went into effect) must be handled as special cases.
 
3.2) The new recommended format is a restricted subset of
   ISO-8601, being one of two options:
   a) 'CCYY-MM-DD'
   b) 'CCYY-MM-DDThh:mm:ss[.sss...]'

   <CCYY> represents a calendar year, <MM> the ordinal number of a calendar
   month within the calendar year, and <DD> the ordinal number of a day
   within the calendar month.  <hh> represents the hour in the day, <mm>
   the minutes, <ss[.s...]> the seconds.  The literal 'T' is the ISO 8601
   time designator.

   In the short form, there may not be any additional terminator/separator
   (such as T).  In the long form, there must be a 'T' time designator
   between the date and the time.
   The decimal point character is an ASCII full-stop (hexadecimal value 0x2E).
   The number of decimal places in the `seconds' field may be arbitrarily 
   long, up to the FITS header-card limitations.
 
3.3) Only fully-specified date or date/time strings are permitted. No fields
   may be defaulted, no leading zeroes omitted.  The decimal part of
   the seconds field is optional.
 
                    4) Use of the DATE-OBS keyword
 
4.1) The name of the keyword shall remain DATE-OBS.
 
4.2) It is suggested that the DATE-OBS refer to the start of an observation.
   This relationship should be clarified in the comment field.

4.3) The default interpretation of all DATExxxx keywords shall use the
   Gregorian Calendar for the date portion.

4.4) The value of the DATExxxx keywords shall be expressed in the principal
   time scale or time system of the HDU to which they belong.  The default
   interpretation shall use UTC (for dates since 1972) or UT (for dates
   before 1972).
 
4.5) It is recommended that the time scale or time system be specified
   explicitly.  However, the error resulting from ignoring the time scale
   specification and making the default assumption, will not exceed
   1000 s for the period 1001-01-01 through 3000-12-31.
 
                          5) Examples
Three legal representations of the date of the first formal draft of this
proposal are possible:
 
DATE-OBS= '14/10/96'           / Original format, means 1996 Oct 14.

DATE-OBS= '1996-10-14'         / Date of start of observation, by default UTC.

DATE-OBS= '1996-10-14T10:14:36.123' / Date and time of start of obs. in UTC.
 
                          6) Transition
 
FITS readers must continue to interpret the old format, as a twentieth
century date, forever.  Readers should be altered as soon as possible
to cope with the new format.  In order to give adequate time for the
major package writers to revise their software, FITS writers should commence
writing the new format between 1998-01-01T00:00:00 and 2000-01-01T00:00:00.
 
                    7) Appendix: Suggested time scale specification

       [Note: this appendix is not part of the formal DATExxxx agreement]

7.1) Use of the keyword TIMESYS is suggested as an implementation of the time
   scale specification; it is currently in use as such in the RXTE mission
   archive.  It sets the dominant time system for time-related keywords
   and data in the HDU (i.e., it does not preclude the addition of keywords
   or data columns that provide information for transformations to other
   time scales, such as sidereal times or barycenter corrections).
   Initially, officially allowed values are:
      UTC  Coordinated Universal Time; defined since 1972.
      UT   Universal Time, equal to Greenwich Mean Time (GMT) since 1925;
            the UTC equivalent before 1972;
            see: Explanatory Supplement, p. 76.
      TAI  International Atomic Time; "UTC without the leap seconds";
            31 s ahead of UTC on 1997-07-01.
      IAT  International Atomic Time; deprecated synonym of TAI.
      ET   Ephemeris Time, the predecessor of TT; valid until 1984.
      TT   Terrestrial Time, the IAU standard time scale since 1984;
            continuous with ET and synchronous with (32.184 s ahead of) TAI.
      TDT  Terrestrial Dynamical Time; =TT.
      TDB  Barycentric Dynamical Time.
      TCG  Geocentric Coordinate Time; runs ahead of TT since 1977-01-01
            at a rate of approximately 22 ms/year.
      TCB  Barycentric Coordinate Time; runs ahead of TDB since 1977-01-01
            at a rate of approximately 0.5 s/year.
   For reference, see:
      Explanatory Supplement to the Astronomical Almanac, P. K. Seidelmann,
      ed., University Science Books, 1992, ISBN 0-935702-68-7.
   Use of GPS time (19 s behind TAI) is deprecated.

7.2) Times will be deemed to be as measured at the detector (or in practical
   cases, at the observatory) unless a different location is implied by the
   time system specified.  This is the case for coordinate times (such as
   TCG and TCB) and TDB which are tied to an unambiguous coordinate origin
   In those cases the meaning of time values will be: time as if the
   observation had taken place at the origin of the coordinate time system.
   This implies that path length differences have been corrected for.  Note
   that the difference TDB-UTC is approximately sinusoidal, with period
   one year and amplitude up to 500 s, depending on source position.  Also,
   note that when the location is not unambiguous (such as in the case of
   an interferometer) precise specification of the location is strongly
   encouraged in, for instance, geocentric Cartesian coordinates.

7.3) Note that "TT" is the IAU preferred standard.  It may be considered
   equivalent to "TDT" and "ET", though "ET" should not be used for data
   taken after 1984.  For reference, see: Explanatory Supplement, pp. 40-48.

7.4) If the TIMESYS keyword is absent or has an unrecognized value,
   the value "UTC" will be assumed for dates since 1972, and "UT" for
   pre-1972 data.

7.5) Examples
   The three legal representations of the date of the first formal draft of
   this proposal would become:
 
DATE-OBS= '14/10/96'           / Original format, means 1996 Oct 14.

TIMESYS = 'UTC     '           / Explicit time scale specification: UTC.
DATE-OBS= '1996-10-14'         / Date of start of observation in UTC.

DATE-OBS= '1996-10-14'         / Date of start of observation, also in UTC.

TIMESYS = 'TT      '           / Explicit time scale specification: TT.
DATE-OBS= '1996-10-14T10:14:36.123' / Date and time of start of obs. in TT.

7.6) The convention suggested in this Appendix is part of the mission-specific
   FITS conventions adopted for, and used in, the RXTE archive, building on
   existing High Energy Astrophysics FITS conventions.  See:
      http://legacy.gsfc.nasa.gov/docs/xte/abc/time_tutorial.html
      http://heasarc.gsfc.nasa.gov/docs/xte/abc/time.html
   The VLBA project has adopted a convention where the keyword TIMSYS, rather
   than TIMESYS, is used, currently allowing the values UTC and IAT.
   See p.9 and p.16 of:
      http://www.cv.nrao.edu/fits/documents/drafts/vlba_format.ps
------- end -------

--
  Donald C. Wells         Associate Scientist         dwells at nrao.edu
                    http://fits.cv.nrao.edu/~dwells
  National Radio Astronomy Observatory                +1-804-296-0277
  520 Edgemont Road,   Charlottesville, Virginia       22903-2475 USA