From bschlesinger@nssdca.gsfc.nasa.gov Fri Jan 27 15:02:42 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!portal.gmu.edu!europa.eng.gtefsd.com!gatech!howland.reston.ans.net!agate!ames!newsfeed.gsfc.nasa.gov!nssdca.gsfc.nasa.gov!bschlesinger From: bschlesinger@nssdca.gsfc.nasa.gov (Barry M. Schlesinger) Newsgroups: sci.astro.fits,sci.data.formats Subject: FITS basics and information (periodic posting) - 1/2 Followup-To: poster Date: 27 Jan 1995 11:19 EDT Organization: NASA Goddard Space Flight Center -- Greenbelt, Maryland USA Lines: 271 Distribution: world Expires: 23 February 1995 19:00 GMT Message-ID: <27JAN199511191422@nssdca.gsfc.nasa.gov> Reply-To: fits@nssdca.gsfc.nasa.gov NNTP-Posting-Host: nssdca.gsfc.nasa.gov Summary: This post provides a brief introduction to FITS and discusses available documentation. News-Software: VAX/VMS VNEWS 1.41 Xref: solitaire.cv.nrao.edu sci.astro.fits:1013 sci.data.formats:773 FITS basics and information, Part 1 Preface This basic Flexible Image Transport System (FITS) information is posted and updated periodically by the NASA/Science Office of Standards and Technology (NOST) FITS Support Office, under the overall supervision of Donald M. Sawyer (GSFC/NSSDC). It provides a brief description of FITS and information on software and documentation, discusses some topics that have appeared on sci.astro.fits, and answers some questions on FITS frequently received by the FITS Support Office. Changes from Previous Month o Imminence of action on units revision to NOST Definition o New draft of BINTABLE; where to find it o New section on proposed conventions o Discussion of grouping proposal o Discussion of checksum proposal o New version of FITSIO discussed o Updated FITSview information o Relocation of material from hypatia.gsfc.nasa.gov to nssdc.gsfc.nasa.gov Table of Contents Part 1 Preface Changes from Previous Month Table of Contents 1 Introduction 1.1 What FITS Is 1.2 How FITS Evolves 1.3 What FITS Is Not 2 FITS Documents 2.1 The FITS Papers 2.2 Binary Tables 2.3 User's Guide 2.4 NOST Definition of FITS 2.4.1 Version 1.0 2.4.2 Proposed Revision on Units Specification 2.5 Floating Point Agreement 2.6 World Coordinates 2.7 Proposed Conventions Part 2 3 Software and Sample Data 3.1 NOST 3.1.1 FITS Product Conformance Tester with Instructions 3.1.2 Header Lister 3.1.3 Error Test Files 3.2 HEASARC 3.2.1 FITSIO 3.2.2 FTOOLS 3.2.3 VERIFITS 3.3 ADC FITS Tables Browser 3.4 Converting FITS Files to Image Format 3.4.1 Major Astronomical Imaging Packages 3.4.2 pbm+ 3.4.3 IMDISP (IBM/PC) 3.4.4 Applications with xv 3.0 3.4.5 ViewFITS (OS/2) 3.4.6 FITS and the Macintosh 3.4.7 FITSview (Windows) 3.5 World Coordinates 4 On-line Information Sources 4.1 NOST 4.2 HEASARC 4.3 NRAO 4.4 HEAFITS exploder 5. Contributors (non-exhaustive list) 6. NOST services 1 Introduction 1.1 What FITS Is FITS (Flexible Image Transport System) is a data format designed to provide a means for convenient exchange of astronomical data between installations whose standard internal formats and hardware differ. A FITS file is composed of a sequence of Header Data Units (HDUs). The header consists of keyword=value statements, which describe the format and organization of the data in the HDU and may also provide additional information, for example, about instrument status or the history of the data. The data follow, structured as the header specifies. The data section of the HDU may contain a digital image, but, except for the first, *it doesn't have to*. Other possible formats include tables and multidimensional matrices that are not images. The first HDU must contain a multidimensional matrix or no data at all; the data in subsequent HDUs, called extensions, may be of any type, consistent with certain rules. The "Image" in the name comes from the original use of the format to transport digital images, but it's not just for images any more. FITS supports 5 data types in primary or IMAGE data arrays: 8-bit unsigned binary integers, 16-bit twos-complement signed binary integers, 32-bit twos-complement signed binary integers, 32-bit IEEE-754 standard floating point numbers, and 64-bit IEEE-754 floating point numbers. For signed integers, the byte that includes the sign bit is first and the byte that has the 1-bit as its least significant bit is last. FITS does not support the 16-bit unsigned integer data type generated by many analog/digital converters. Conforming FITS files can be produced from such data by subtracting 32768_decimal from the converter output before writing to the file, while setting the BZERO keyword in the FITS header equal to 32768 and the BSCALE keyword equal to 1. A FITS reader will then add 32768 to the value in the file, restoring the original value, before interpreting it. Whether a 16-bit unsigned data type should be added, and if so, how, is controversial and under discussion, especially in sci.astro.fits. 1.2 How FITS Evolves The international authority for FITS is the International Astronomical Union (IAU) FITS Working Group (IAUFWG), which was given authority over FITS matters by the 1988 IAU General Assembly. This Group is associated with the Working Group on Astronomical Data. The current chair is D. Wells (NRAO) and the vice-chair is E. Raimond (NFRA). When the developer of a data structure finds that it does not fit well into an existing standard FITS format, a new design may be developed. No change can be made that would cause existing FITS files to be out of conformance -- the "once FITS, always FITS" rule. A unique name for any new extension type must be registered with the IAU FITS Working Group, optionally through the NOST FITS Support Office. After community discussion, most of which will be electronic, a formal proposal is distributed. This proposal is discussed by the community and may be further modified. Tests are run using the new format to confirm that it can be practically used for data transport. If the astronomical community reaches a consensus that the proposal should be adopted as standard FITS, and if successful data transfer using the proposed extension can be demonstrated, it is submitted for ratification to the regional committees--the European FITS Committee, the Japanese FITS Committee, and the American Astronomical Society Working Group on Astronomical Software (WGAS) FITS Committee. Following approval by the regional committees, it is submitted to the IAU FITS Working Group. Approval by the Working Group establishes it as a standard extension. 1.3 What FITS Is Not FITS is not principally a graphics format designed for the transfer of pictures; it does not incorporate "FITS viewers", packages for decoding the data into an image. Users must develop or obtain separate software to convert the data from the FITS file into a form that can be readily displayed. There is no standard package for all applications; section 3.4 discusses some possibilities. 2 FITS Documents 2.1 Published Papers The fundamental references on FITS are the following five papers. The first four have often been referred to collectively as the "Four FITS Papers". These papers, along with the Floating Point Agreement (section 2.5) and the binary tables definition (section 2.2), are the formal standard for FITS, endorsed by the IAU. Wells, D. C., Greisen, E. W., and Harten, R. H., "FITS: a flexible image transport system," Astronomy and Astrophysics Supplement Series, 44, 363-370, 1981. Greisen, E. W. and Harten, R. H., "An extension of FITS for small arrays of data," Astronomy and Astrophysics Supplement Series, 44, 371-374, 1981. (NOTE: The format described in this paper has been used almost exclusively to transport radio interferometry and is likely to be replaced by other formats in the future. Writing data other than radio interferometry data using this format is not recommended.) Grosbol, P., Harten, R. H., Greisen, E. W., and Wells, D. C., "Generalized extensions and blocking factors for FITS," Astronomy and Astrophysics Supplement Series, 73, 359-364, 1988. Harten, R. H., Grosbol. P., Greisen, E. W., and Wells, D. C., "The FITS tables extension, Astronomy and Astrophysics Supplement Series, 73, 365-372, 1988. Ponz, J. D., Thompson, R. W., and Munoz, J. R., "The FITS Image Extension," Astronomy and Astrophysics Supplement Series, 105, 53-55, 1994. 2.2 Binary Tables On June 15, 1994, the IAU FITS Working Group announced the acceptance of BINTABLE, the binary table extension, as a standard extension. A draft of the description of the extension, to be submitted for publication to Astronomy and Astrophysics, is available at the NRAO site (see section 4.3, in part 2). 2.3 User's Guide A User's Guide for FITS, commissioned by NASA Headquarters, is maintained by the NOST FITS Support Office. This Guide is intended to be a tutorial for new FITS users. In addition to presenting the rules of FITS, it provides some of the history and reasoning behind the choice of the rules, adds recommendations on good practices, and discusses current developments in FITS. The current version, 3.1, was issued in May 1994. 2.4 NOST Definition of FITS 2.4.1 Version 1.0 The NOST has codified FITS as endorsed by the IAU into a formal standard, eliminating some contradictions and ambiguities in the original FITS papers. This Definition of FITS, version 1.0, was developed by a Technical Panel chaired by Dr. Robert J. Hanisch (STScI), with review by the astronomical community. On June 18, 1993, it was approved as a NOST Standard by an Accreditation Panel consisting of the NOST Executive Board and an astronomical community representative; this review was to confirm that the community had been given a satisfactory opportunity to review the standard and that the Technical Panel had properly considered and responded to all comments. The NOST standard has been submitted to the IAUFWG for endorsement as the international FITS standard, to replace the endorsed standard -- originally the four FITS papers, to which the Floating Point Agreement (section 2.4), and now the IMAGE and binary table extensions and the physical blocking conventions have been added. While oversights in non-controversial areas may be rectified as a result of the review by the IAUFWG, significant changes are unlikely, because members of this committee were active in the process of reviewing the standard and their comments were given significant weight in the deliberations of the Technical Panel. 2.4.2 Proposed Revision on Units Specification A new Technical Panel has been formed, primarily to draft a revised NOST definition of FITS incorporating IMAGE, BINTABLE, and the blocking agreement, but also to rectify any oversights or omissions that may be brought to its attention by the community. Dr. Hanisch, the chair of the panel that developed version 1.0 of the Definition of FITS, is chairing this new panel. In response to community comments on the version 1.0 treatment, the panel developed and released text covering requirements and recommendations on units to be used in FITS files. The Technical Panel is now reviewing community comments received over a one-month review period. The draft text is available from the NOST FTP site (section 4.1) in the file units_revision.txt, in flat ASCII text form, with an approximate length of two printed pages. The NOST Librarian can also provide printed copies and electronic copies for those without ftp access. When the community review process is complete, the text finally adopted by the Technical Panel will be incorporated in the NOST Definition of FITS, producing version 1.1. Action is expected shortly. 2.5 Floating Point Agreement Originally, FITS permitted only integers in the data array following the first, or primary header. The IAU has since endorsed the Floating Point Agreement, which specifies the use of IEEE-754 floating point and describes its use in FITS. The basic agreement appears verbatim in the User's Guide, and the substance is incorporated in the NOST standard. 2.6 World Coordinates A draft text of conventions for World Coordinates is currently under community review. It proposes rules for relating a FITS data array to the physical quantities the numbers represent, with detailed discussion of projections from the celestial sphere to the array plane. It is available electronically from the NRAO site (4.3). 2.7 Proposed Conventions Rob Seamon and Bill Pence have proposed a scheme for embedding a checksum within a FITS header. This checksum could be used to verify that the data in a file were transported without errors. A copy is available by anonymous ftp from iraf.noao.edu, in the /misc/checksum directory. IAU FITS Working Group Chair D. Wells has recommended that this proposal be considered by the regional FITS committees. D. Jennings, W. Pence, M. Folk, and B. Schlesinger have proposed a convention for logically grouping together FITS HDUs that are physically separated in a given file or are located in different files. One of the features of this proposal is that it will facilitate HDU-FITS conversion. Versions in TeX/PostScript are available by anonymous ftp at ssvs.gsfc.nasa.gov in the pub/convert directory; an html version may be viewed at http://acadia.gsfc.nasa.gov/convert/group.html .