From fitsbits-request Thu Dec 1 17:53:17 1994 X-VM-Message-Order: (1 3 2 4 5 6 7 10 11 8 9 12 13 14) X-VM-Summary-Format: "%n %*%a %-17.17F %-3.3m %2d %4l/%-5c %I\"%s\"\n" X-VM-Labels: nil X-VM-VHeader: ("Resent-" "From:" "Sender:" "To:" "Apparently-To:" "Cc:" "Subject:" "Date:") nil X-VM-Bookmark: 14 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["417" "" "30" "November" "1994" "15:22:26" "GMT" "Astronomy Ireland" "ai at iol.ie" "<3bi5bi$8nd at barnacle.iol.ie>" "10" "TIFF spec" "^From:" nil nil "11" "1994113015:22:26" "TIFF spec" (number " " mark " Astronomy Ireland Nov 30 10/417 " thread-indent "\"TIFF spec\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA17123; Thu, 1 Dec 94 17:53:17 EST Return-Path: Message-Id: <3bi5bi$8nd at barnacle.iol.ie> Organization: Ireland On-Line Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!portal.gmu.edu!europa.eng.gtefsd.com!news.mathworks.com!uhog.mit.edu!bloom-beacon.mit.edu!gatech!howland.reston.ans.net!Germany.EU.net!EU.net!ieunet!iol!ai Newsgroups: sci.astro.fits From: ai at iol.ie (Astronomy Ireland) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: TIFF spec Date: 30 Nov 1994 15:22:26 GMT Does anyone have the technical specifications for TIFF image format? Or know where I might get them? -- David Moore BSc FRAS, Editor of "Astronomy & Space" magazine. (ai at iol.ie) Chairman, Astronomy Ireland, P.O.Box 2888, Dublin 1. Tel: +353-1-459 8883. Fax: +353-1-459 9933. ________Irish News: 1550-111-442 (calls cost upto 58p/minute)________ _____U.K.: 0891-88-1950 (39p/min cheap, 49p/min all other times)_____ From fitsbits-request Thu Dec 1 18:27:52 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["2853" "" " 1" "December" "1994" "23:27:41" "GMT" "Don Wells" "dwells at nrao.edu" "" "68" "Re: TIFF spec" "^From:" nil nil "12" "1994120123:27:41" "TIFF spec" (number " " mark " Don Wells Dec 1 68/2853 " thread-indent "\"Re: TIFF spec\"\n") "<3bi5bi$8nd at barnacle.iol.ie>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA17266; Thu, 1 Dec 94 18:27:52 EST Return-Path: Message-Id: Organization: nrao Path: solitaire.cv.nrao.edu!news.cv.nrao.edu!dwells References: <3bi5bi$8nd at barnacle.iol.ie> Newsgroups: sci.astro.fits From: dwells at nrao.edu (Don Wells) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: TIFF spec Date: 01 Dec 1994 23:27:41 GMT In article <3bi5bi$8nd at barnacle.iol.ie> ai at iol.ie (Astronomy Ireland) writes: "Does anyone have the technical specifications for TIFF image format? Or know where I might get them?" The traffic archive at ftp://fits.cv.nrao.edu/fits/traffic/ contains a wide variety of information about data formats which I judge might be of potential interest to astronomy-related people. The subdirectories are named for the newsgroups and mailing lists which I monitor and archive: iaufwg, scidataformats, altgraphicspixutils, metadata, sciimageprocessing, compcompression, sciastro, wfc, sciastrofits, wgas, heafits, sciastroresearch. This archive is indexed, and the WAIS server wais://fits.cv.nrao.edu/nrao-fits will search the index for you. A WAIS search on the string 'tiff' currently finds 129(!) items. Here are some sample titles: gary at maple Re: Re: TIFF library at Uni. of California, Berkeley tgl+ at cs.cm Re: JPEG image compression: Frequently Asked Questions lip at s1.gov Re: FITS, GIF, TIFF, etc... ipe at utu.fi Re: Re: TIFF File format eric at fuji1 Re: Re: TIFF image file question A quick look at these messages found this one, a response to some variation on your question: -------------------------------------------------- Newsgroups: sci.data.formats From: grisebac at isoit109.BBN.HP.COM (#Eberhard Grisebach) Subject: Re: TIFF File format Date: Wed, 28 Jul 1993 06:01:12 GMT The description of TIFF Revision 6.0 is about 120 pages!!! What exactly do you want? ---------------------------------------------------- Searching further down in the list of 129 items found this message: ------------------------------------------- Newsgroups: alt.graphics.pixutils From: feck at informatik.uni-kl.de (Christoph Feck IRZ) Subject: Re: TIFF specification... Date: Thu, 30 Jun 1994 12:37:25 GMT ffeschet at ens-lyon.fr (Fabien Feschet) wrote: > Tag Image File Format Rev 4.0 > April 31, 1987 Ouch! This is old. Rev 6.0 is the current. You can find it at ftp.sgi.com:/graphics/tiff/TIFF6.ps.Z (P*stScript file). ----------------------------------------------------------- The last line above is probably the answer to your question. -*- O'Reilly & Associates have announced a new book: Encyclopedia of Graphics File Formats James D. Murray & William vanRyper 1st Edition July 1994 928 pages, plus CD-ROM ISBM 1-56592-058-9, $59.95 They say "..we managed to describe nearly 100 formats and to include specs, images, and/or code examples for most of them on the CD-ROM.." Of course, both FITS and TIFF are in the list. I intend to order a copy for myself RSN. -- Donald C. Wells Associate Scientist dwells at nrao.edu http://fits.cv.nrao.edu/~dwells National Radio Astronomy Observatory +1-804-296-0277 520 Edgemont Road, Charlottesville, Virginia 22903-2475 USA From fitsbits-request Thu Dec 1 19:02:00 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["499" "" "30" "November" "1994" "12:38:02" "-0700" "Mike Fitzpatrick" "fitz at noao.edu" "<3bikaq$70g at tucana.tuc.noao.edu>" "12" "Re: TIFF spec" "^From:" nil nil "11" "1994113019:38:02" "TIFF spec" (number " " mark " Mike Fitzpatrick Nov 30 12/499 " thread-indent "\"Re: TIFF spec\"\n") "<3bi5bi$8nd at barnacle.iol.ie>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA17369; Thu, 1 Dec 94 19:02:00 EST Return-Path: Message-Id: <3bikaq$70g at tucana.tuc.noao.edu> Organization: IRAF Project, National Optical Astronomy Observatories Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!portal.gmu.edu!europa.eng.gtefsd.com!news.mathworks.com!newshost.marcam.com!charnel.ecst.csuchico.edu!nic-nac.CSU.net!news.Cerritos.edu!news.Arizona.EDU!CS.Arizona.EDU!noao!iraf!not-for-mail References: <3bi5bi$8nd at barnacle.iol.ie> Newsgroups: sci.astro.fits From: fitz at noao.edu (Mike Fitzpatrick) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: TIFF spec Date: 30 Nov 1994 12:38:02 -0700 >From article <3bi5bi$8nd at barnacle.iol.ie>, by ai at iol.ie (Astronomy Ireland): > Does anyone have the technical specifications for TIFF image format? > Or know where I might get them? Try ftp://zamenhof.cs.rice.edu/pub/graphics.formats ftp://ftp.cosy.sbg.ac.at/ub/mirror/file-formats/graphic ftp://ftp.ncsa.uiuc.edu//misc/file.formats/graphics.formats TIFF software is available via anonftp from ftp://ucbvax.berkeley.edu/pub/tiff/*.tar.Z or ftp://ftp.uu.net/graphics/tiff.tar.Z From fitsbits-request Fri Dec 2 05:16:47 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1283" "" " 1" "December" "1994" "18:07:48" "-0500" "Lee E. Brotzman" "leb at clark.net" "<3bll04$dg4 at explorer.clark.net>" "25" "Re: FITS Sorting Convention" "^From:" nil nil "12" "1994120123:07:48" "FITS Sorting Convention" (number " " mark " Lee E. Brotzman Dec 1 25/1283 " thread-indent "\"Re: FITS Sorting Convention\"\n") "<9411291435.AA21256 at SIMBAD.u-strasbg.fr>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA18038; Fri, 2 Dec 94 05:16:47 EST Return-Path: Message-Id: <3bll04$dg4 at explorer.clark.net> Organization: Clark Internet Services, Inc., Ellicott City, MD USA Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!howland.reston.ans.net!news.sprintlink.net!news.clark.net!news.clark.net!not-for-mail References: <9411291435.AA21256 at SIMBAD.u-strasbg.fr> Newsgroups: sci.astro.fits From: leb at clark.net (Lee E. Brotzman) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS Sorting Convention Date: 1 Dec 1994 18:07:48 -0500 francois at SIMBAD.u-strasbg.fr writes: >Just a few comments concerning the TSORTKEY convention proposed by OGIP: > >1) The proposed definition implies additional non-standard FITS requirements: > unique names, no possibility of names starting with the minus sign. > Since FITS headers are (fortunately!) to be read only by programs, > wouldn't it be so much simpler and less ambiguous to use only the > TFIELD number, i.e. definitions like > TSORTKEY = '1,-2,12,-14' I usually don't like to post "me too" messages, but I have to concur. It is "prettier" to use column names, however, since column names are not required, but column numbers (TBCOL) are, then the only acceptable value would be the numbers. Plus, negative column numbers are unambiguous, whereas minus signs before column names can be. Also, as a programmer, having the numbers saves me code, as opposed to names which require some type of table lookup. So, all in all, using column numbers is a win all the way around. -- -- Lee E. Brotzman E-mail: leb at clark.net -- Advanced Data Solutions Phone : 301-776-0324 -- "Sometimes I think the surest sign that intelligent life exists elsewhere in -- the Universe is that none of it has tried to contact us." --- Calvin, 1990 From fitsbits-request Fri Dec 2 15:09:44 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["713" "" " 2" "December" "1994" "13:54:55" "-0000" "C. G. Page" "cgp at le.ac.uk" "<3bn8vf$alo at hawk.le.ac.uk>" "14" "Re: FITS Sorting Convention" "^From:" nil nil "12" "1994120213:54:55" "FITS Sorting Convention" (number " " mark " C. G. Page Dec 2 14/713 " thread-indent "\"Re: FITS Sorting Convention\"\n") "<9411290830.AA03446 at ns2.hq.eso.org>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA21035; Fri, 2 Dec 94 15:09:44 EST Return-Path: Message-Id: <3bn8vf$alo at hawk.le.ac.uk> Organization: University of Leicester, UK Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!caen!newsxfer.itd.umich.edu!gatech!swrinde!pipex!warwick!leicester!leicester!not-for-mail References: <9411290830.AA03446 at ns2.hq.eso.org> Newsgroups: sci.astro.fits From: cgp at le.ac.uk (Dr C.G. Page) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS Sorting Convention Date: 2 Dec 1994 13:54:55 -0000 I appreciate the comments of Preben Grosbol that in theory it is possible to construct FITS tables which have no column names, or ones which start with minus-signs, or with duplicated names. It seems to me, however, that such FITS files are very rare and I think that everything should be done to discourage their use. No one is required to adopt this convention for indicating how a table is sorted, I don't think it matters that the TSORTKEY proposal will not work on such pathological cases. -- ------------------------------------------------------------------------- Clive Page, Internet: cgp at star.le.ac.uk Dept of Physics & Astronomy, University of Leicester. From fitsbits-request Mon Dec 5 17:39:53 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["3133" "Mon" " 5" "December" "1994" "17:39:48" "-0500" "William Pence" "pence at tetra.gsfc.nasa.gov" "<199412052239.RAA19568 at tetra.gsfc.nasa.gov>" "63" "re: TSORTKEY convention" "^From:" nil nil "12" "1994120522:39:48" "TSORTKEY convention" (number " " mark " William Pence Dec 5 63/3133 " thread-indent "\"re: TSORTKEY convention\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA02109; Mon, 5 Dec 94 17:39:53 EST Return-Path: Message-Id: <199412052239.RAA19568 at tetra.gsfc.nasa.gov> From: William Pence Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: re: TSORTKEY convention Date: Mon, 5 Dec 1994 17:39:48 -0500 Regarding the TSORTKEY convention posted here last week, several people questioned the use of the column name rather than the column index to specify the sort column(s). Our FITS working group did consider this issue but in the end decided to use the column label (as given by the TTYPEn keyword) for several reasons: - It is easier for humans to read and decode (contrary to Francois Ochsenbein's assertion that only software reads FITS headers we find that users and programmers also frequently need to read the FITS headers). - It is more in the 'object oriented' spirit to refer to a column by its name rather than absolute position within the table. This helps to insulate the creator and user of the TSORTKEY keyword from having to know about the internal structure of the table. This is similar to relational database systems that generally refer to table columns by name rather than position. - If the columns in the table are rearranged, or if columns are added or deleted in the table, then the TSORTKEY does not have to be modified. The main concern about using the column name seems to be "what does one do if the table does not have a TTYPEn keyword for the sort column?". There is a simple solution: just make up a default name for the column and add the appropriate TTYPEn keyword along with the TSORTKEY keyword. For instance if one wants to indicate the table is sorted on the 3rd column then add the following 2 keywords: TTYPE3 = 'COL_3' TSORTKEY= 'COL_3' The other concern is what to do if the sort column name begins with a minus sign or the column name is not unique. The best anwser is probably that the user should modify the FITS file to give the columns sensible names. If this is not possible though, then one is simply out of luck and cannot use the TSORTKEY convention, which after all, is not required. If someone is going to ignore the NOST recommendations on the choice of column names (i.e., use only alphanumeric characters and the underscore) then they should not be surprised to encounter some difficulties later in using that FITS file in certain software systems. Another way to state this is that the TSORTKEY convention is layered on top of another well established convention on the choice of column names, and this was spelled out in section 5 of the definition of the TSORTKEY convention. As an aside, the OGIP is currently drafting some specific guidelines for column names in all the FITS tables that we produce. This will follow the NOST recommendations plus possibly include additional restrictions, such as that the names must be unique within the first 8 characters to be compatible with IRAF. Finally, Francois Ochsenbein asked: > 2) NULL values: does the definition state that, if a TNULL value is defined > as "-999.99", these values will come AFTER a valid value like "999." ? Yes. The particular value used to indicate a null pixel is not really relevant. All that matters is that these elements are undefined, and thus will appear at the bottom of the sorted table. -The OGIP FITS Working Group From fitsbits-request Tue Dec 6 06:28:57 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["783" "Tue" " 6" "December" "1994" "12:28:36" "+0100" "Preben Grosbol" "pgrosbol at eso.org" "<9412061128.AA06125 at ns2.hq.eso.org>" "17" "Re: TSORTKEY convention " "^From:" nil nil "12" "1994120611:28:36" "TSORTKEY convention" (number " " mark " Preben Grosbol Dec 6 17/783 " thread-indent "\"Re: TSORTKEY convention \"\n") "<199412052239.RAA19568 at tetra.gsfc.nasa.gov>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA04491; Tue, 6 Dec 94 06:28:57 EST Return-Path: Message-Id: <9412061128.AA06125 at ns2.hq.eso.org> In-Reply-To: Your message of "Mon, 05 Dec 94 17:39:48 EST." <199412052239.RAA19568 at tetra.gsfc.nasa.gov> From: Preben Grosbol Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: TSORTKEY convention Date: Tue, 06 Dec 1994 12:28:36 +0100 I fully agree that the usage of column labels is nice and more readable. The disturbing point is to have a convention which may or may not work. Just to add one more marginal issue, FITS recommend that string values are unique to 8 character (due to old outdated readers). Now take column labels which only differ at the 8th character. By adding the minus sign they can not any longer be distinguished within the 8 first characters. As the usage of names in the TSORTKEY is better that numbers but has its problem, one could consider to allow both i.e. demand that label names used with the TSORTKEY convention do not start with a '-' or a digit. If in the TSORTKEY field a label is found starting with either a digit or -digit it is interpreted as a field number. Preben Grosbol From fitsbits-request Tue Dec 6 07:49:27 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1197" "" " 6" "December" "1994" "10:11:13" "GMT" "David Allan" "dja at star.ac.bham.ac.uk" "<3c1dc1$a68 at sun4.bham.ac.uk>" "25" "FITSIO on PC's" "^From:" nil nil "12" "1994120610:11:13" "FITSIO on PC's" (number " " mark " David Allan Dec 6 25/1197 " thread-indent "\"FITSIO on PC's\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA04565; Tue, 6 Dec 94 07:49:27 EST Return-Path: Message-Id: <3c1dc1$a68 at sun4.bham.ac.uk> Organization: The University of Birmingham, UK. Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!howland.reston.ans.net!pipex!sunsite.doc.ic.ac.uk!lyra.csx.cam.ac.uk!warwick!bham!xun9!dja Newsgroups: sci.astro.fits From: dja at star.ac.bham.ac.uk (David Allan) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: FITSIO on PC's Date: 6 Dec 1994 10:11:13 GMT I am trying to use FITSIO on an PC. I can build simple programs which fit into the 550K conventional memory available on my machine. What I would like to do is build Windows applications using the Microsoft Fortran QUICKWIN interface, or link C programs to the Fortran using a DLL. I have no problems constructing either alternative, but the former fails to run the test1 program with an error which disappears immediately before I can read it, and the latter fails with a GP fault when the DLL is loaded. I feel the first solution is most likely to work as the test2 program appears to start up ok. The DLL approach seems doomed to failure as Micro soft place restrictions on the use of assumed length character strings in Fortran modules inside DLLs. My question: As anyone built a complex FITSIO based program on a PC, with an executable size bigger than 640K. If so, how did you do it (link & compile options etc). David Allan David J. Allan | School of Physics and Space Research dja at star.sr.bham.ac.uk | University of Birmingham Tel : 0121-414-3655 | Edgbaston FAX : 0121-414 3722 | Birmingham B15 5TT From fitsbits-request Thu Dec 8 01:19:21 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["509" "" " 7" "December" "1994" "17:06" "EDT" "Barry M. Schlesinger" "bschlesinger at nssdca.gsfc.nasa.gov" "<7DEC199417064585 at nssdca.gsfc.nasa.gov>" "12" "Comments collected on units draft" "^From:" nil nil "12" "1994120721:06:00" "Comments collected on units draft" (number " " mark " Barry M. Schlesin Dec 7 12/509 " thread-indent "\"Comments collected on units draft\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA08550; Thu, 8 Dec 94 01:19:21 EST Return-Path: Message-Id: <7DEC199417064585 at nssdca.gsfc.nasa.gov> Organization: NASA Goddard Space Flight Center -- Greenbelt, Maryland USA Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!tulane!ames!newsfeed.gsfc.nasa.gov!nssdca.gsfc.nasa.gov!bschlesinger Newsgroups: sci.astro.fits From: bschlesinger at nssdca.gsfc.nasa.gov (Barry M. Schlesinger) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Comments collected on units draft Date: 7 Dec 1994 17:06 EDT Acknowledgements have been sent to all who provided comments on the draft revision to the NOST Definition of FITS covering units for angles and celestial coordinates. If you sent in comments and have not received an acknowledgement, please notify the NOST Librarian at nost at nssdca.gsfc.nasa.gov immediately. As before, apologies to those who may receive multiple copies, but we're trying to reach everyone quickly. Barry Schlesinger NSSDC/NOST FITS Support Office From fitsbits-request Thu Dec 8 14:18:21 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1740" "Thu" " 8" "December" "1994" "20:19:32" "+0100" "Thierry Forveille" "forveill at gag.observ-gr.fr" "<9412081918.AA11069 at fits.cv.nrao.edu>" "36" "re: TSORTKEY convention" "^From:" nil nil "12" "1994120819:19:32" "TSORTKEY convention" (number " " mark " Thierry Forveille Dec 8 36/1740 " thread-indent "\"re: TSORTKEY convention\"\n") "<199412052239.RAA19568 at tetra.gsfc.nasa.gov>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA11075; Thu, 8 Dec 94 14:18:21 EST Return-Path: Message-Id: <9412081918.AA11069 at fits.cv.nrao.edu> Posted-Date: Thu, 8 Dec 1994 20:19:32 +0100 In-Reply-To: <199412052239.RAA19568 at tetra.gsfc.nasa.gov> References: <199412052239.RAA19568 at tetra.gsfc.nasa.gov> From: forveill at gag.observ-gr.fr (Thierry Forveille) Sender: fitsbits-request at fits.CV.NRAO.EDU To: William Pence Cc: fitsbits at fits.CV.NRAO.EDU Subject: re: TSORTKEY convention Date: Thu, 8 Dec 1994 20:19:32 +0100 William Pence writes: > Regarding the TSORTKEY convention posted here last week, several people > questioned the use of the column name rather than the column index to > specify the sort column(s). Our FITS working group did consider this > issue but in the end decided to use the column label (as given by the > TTYPEn keyword) for several reasons: > > - It is easier for humans to read and decode (contrary > to Francois Ochsenbein's assertion that only software reads FITS > headers > we find that users and programmers also frequently need to read > the FITS headers). > It is (unfortunately) true that humans occasionaly have to look at the header, but human readable information is best conveyed through comments (ever tried to make sense of an RA in degrees, or remember the observing hour from a UT in seconds...). If human readibility is the main concern, an easy fix would be to use column numbers and strongly suggest to add the column name in the comment field. The points about the 'object oriented spirit' and rearrangements are however strong enough alone to shift my vote towards column names. One point I don't like in the present state of the proposal is its use of the 'sign' of the column name to convey the ordering. This strikes me as an adhoc fix to store two informations in the same field. Of course FITS contains a number of adhoc historical patches (such as BITPIX=-32 for floats), but why add one when we can avoid it. Storing the ordering as a separate keyword (say TSORTORDR) would be cleaner, and would eliminate potential problems with column names which are unique over 8 chars but not 7). Thierry Forveille Observatoire de Grenoble forveill at gag.observ-gr.fr From fitsbits-request Thu Dec 8 14:35:36 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1012" "Thu" " 8" "December" "1994" "14:35:30" "EST" "Jonathan McDowell" "jcm at urania.harvard.edu" "<9412081935.AA03388 at urania.harvard.edu>" "19" "Re: TSORTKEY convention " "^From:" nil nil "12" "1994120819:35:30" "TSORTKEY convention" (number " " mark " Jonathan McDowell Dec 8 19/1012 " thread-indent "\"Re: TSORTKEY convention \"\n") "<9412081918.AA11069 at fits.cv.nrao.edu>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA11118; Thu, 8 Dec 94 14:35:36 EST Return-Path: Message-Id: <9412081935.AA03388 at urania.harvard.edu> In-Reply-To: Message from forveill at gag.observ-gr.fr (Thierry Forveille) of "Thu, 08 Dec 94 20:19:32 +0100." <9412081918.AA11069 at fits.cv.nrao.edu> From: "Jonathan McDowell" Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: TSORTKEY convention Date: Thu, 08 Dec 94 14:35:30 EST > One point I don't like in the present state of the proposal is its > use of the 'sign' of the column name to convey the ordering. This > strikes me as an adhoc fix to store two informations in the same > field. Of course FITS contains a number of adhoc historical patches > (such as BITPIX=-32 for floats), but why add one when we can avoid > it. Storing the ordering as a separate keyword (say TSORTORDR) > would be cleaner, and would eliminate potential problems with column > names which are unique over 8 chars but not 7). This works for the single sort key case, but it'll get messy if you want to use secondary keys with a diferent ordering. I think Bill's proposal is pretty elegant, and not that difficult to parse. The minus sign does have a real arithmetical interpretation in this case - we have reversed the direction of the sort - unlike the BITPIX example where the minus sign is used to mean "bad bad not integer nasty horrible real" (at least that's how I read it :-)) - Jonathan McDowell From dwells Thu Dec 8 17:04:44 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["2406" "Thu" " 8" "December" "1994" "21:46:13" "+0100" "Francois Ochsenbein" "francois at simbad.u-strasbg.fr" "<9412082046.AA03921 at SIMBAD.u-strasbg.fr>" "46" "re- TSORTKEY convention" "^Resent-Message-Id:" nil nil "12" "1994120820:46:13" "re- TSORTKEY convention" (number " " mark " Francois Ochsenbe Dec 8 46/2406 " thread-indent "\"re- TSORTKEY convention\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA11501; Thu, 8 Dec 94 17:04:44 EST Return-Path: Message-Id: <9412082046.AA03921 at SIMBAD.u-strasbg.fr> Resent-Message-Id: <9412082204.AA11501 at fits.cv.nrao.edu> Resent-Date: Thu, 8 Dec 94 21:46:13 +0100 Resent-From: dwells (Don Wells) Resent-To: fitsbits From: francois at SIMBAD.u-strasbg.fr (Francois Ochsenbein) Sender: fitsbits-request at fits.CV.NRAO.EDU Apparently-To: fitsbits Subject: re- TSORTKEY convention Date: Thu, 8 Dec 94 21:46:13 +0100 I would like to see more comments about the TSORTKEY convention proposed by the OGIP group, because this convention is likely to be very useful for tables: we have here at CDS (and in other data centres) over thousand tables which can be translated into FITS, and for instance the search capabilities on FITS tables can benefit a lot from the knowledge of the sorting order. However, the requirements imposed by the OGIP proposal (unique TTYPEs names, usage of a restricted character set in TTYPEs names) do not match the present standard, while the TTYPE numbers (i.e. column numbers) do not impose further constraints. Regarding OGIP FITS answers: > - It is more in the 'object oriented' spirit to refer to a column by > its name rather than absolute position within the table. This > helps to insulate the creator and user of the TSORTKEY keyword > from having to know about the internal structure of the table. > This is similar to relational database systems that generally > refer to table columns by name rather than position. No. Look into your favorite DB system, you'll discover that the definitions of the indexes are made of numbers. The DBMS of course is able to map names into numbers and vice-versa. This mapping can even be performed by a human being without too much trouble! > - If the columns in the table are rearranged, or if columns are added > or deleted in the table, then the TSORTKEY does not have to be > modified. Such an operation implies a complete redefinition of the table definitions and if the table data, and is normally done by a piece of software, which could easily include the few lines of code to perform this mapping. Furthermore, I don't think it would be advisable to introduce dependencies between the values taken by keywords, like the TTYPE and the TSORTKEY: renaming a column (changing just the value of a TTYPEs keyword) would have an impact on the TSORTKEY contents. Concerning the position of NULL values in a sorted list (at the beginning or at the end of the list ?) the answer is not obvious. For instance in ASCII tables, the blanks are usually used to denote such values, and are "naturally" (in the ascii order) situated at the top of the list. The answer may be different for binary tables using IEEE floating-point NULL values. What's the general opinion ? Francois Ochsenbein / CDS Strasbourg From fitsbits-request Mon Dec 19 11:31:13 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["22701" "" "19" "December" "1994" "15:52:57" "GMT" "Don Jennings" "jennings at tcumsh.gsfc.nasa.gov" "" "430" "Reposting: Hierarchical Grouping Convention for FITS" "^From:" nil nil "12" "1994121915:52:57" "Reposting: Hierarchical Grouping Convention for FITS" (number " " mark " Don Jennings Dec 19 430/22701 " thread-indent "\"Reposting: Hierarchical Grouping Convention for FITS\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA22407; Mon, 19 Dec 94 11:31:13 EST Return-Path: Message-Id: Organization: NASA Goddard Space Flight Center -- Greenbelt, Maryland USA Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!caen!hookup!ames!newsfeed.gsfc.nasa.gov!paperboy.gsfc.nasa.gov!jennings Newsgroups: sci.astro.fits From: jennings at tcumsh.gsfc.nasa.gov (Don Jennings) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Reposting: Hierarchical Grouping Convention for FITS Date: 19 Dec 1994 15:52:57 GMT Hello all, I am reposting the draft of our FITS hierarchial grouping proposal. The previous posting seemed to be a missing a few lines of text. As before, you may get the tex/postsript version of this proposal via anonymous ftp at ssvs.gsfc.nasa.gov in the /pub/convert directory or view the html version at http://acadia.gsfc.nasa.gov/convert/group.html . Comments, suggestions, and general arguments for/against are welcome. ------------------------------------------------------------------------------ A Hierarchical Grouping Convention for FITS Donald G. Jennings, GSFC/HSTX William D. Pence, GSFC Michael Folk, NCSA Barry M. Schlesinger, GSFC/HSTX Proposal Draft, Revision 4 December 16, 1994 Abstract This paper describes a grouping convention for FITS that facilitates the construction of hierarchical associations of Header Data Units (HDUs). The grouping convention uses FITS table structures (ASCII or binary) to encapsulate pertinent information about the HDUs belonging to a group. Group members may reside in a single FITS file or be distributed in many FITS files; the FITS files themselves may reside on different computer systems. 1. Introduction The rules for generalized extensions in FITS (Grosbol et. al., 1988) provide for FITS formatted files containing more than one header data unit. By using combinations of ASCII tables (Harten et. al., 1988), binary tables (Cotton et. al., 1994) and image extensions (Ponz et. al., 1994) related data sets requiring different data structures may be stored in the same FITS file, each within its own HDU. Unfortunately, once the related data sets are segregated into separate HDUs the relationship between them is often lost. The FITS standard currently allows for simple hierarchical associations of HDUs within a single FITS file through use of the EXTLEVEL keyword. However, this mechanism has several major limitations. First, its use is not well defined. Different organizations may use EXTLEVEL for widely varying purposes and still not violate the FITS standard. Secondly, it does not specify a mechanism for defining distinct multiple groups of HDUs within a FITS file. Lastly, it cannot be used to associate HDUs residing in different FITS files. Except for very simple cases, FITS contains no mechanism for creating or preserving associations between HDUs or groups of HDUs. As the volume and complexity of FITS formatted data grows, the need for a recognized and versatile HDU grouping mechanism increases. Individuals can be overwhelmed trying to manage and analyze large data sets unless those sets are logically organized. Software tools also require data organization in order to access all necessary components of an observation, simulation or experimental data set. As an example of where grouping capabilities within FITS would be useful, consider the following. It is desirable to combine a set of observations from a given time period into a single FITS file for transport and archival purposes. For each observation there is an observation log, an event list, a derived image and a set of instrument calibration data; furthermore, several observations share a common set of calibration data. By using a grouping mechanism each [log, event list, image, calibration] set could be logically tagged as an associated observation group and the calibration data could be made a part of many different observation groups, thus eliminating the need to store it more than once. Software could retrieve all the information about a given observation simply by extracting those HDUs defined in the table that identifies members of the group. Also, observations of the same object from different observational periods could be combined into a group and accessed as a unit, even though the HDU sets comprising the different observations reside in separate FITS files. The following sections describe a scheme for implementing a hierarchical grouping of header data units within single and multiple FITS files. Section 2 discusses the content of table extensions used to define HDU groupings. Section 3 lists those keywords recommended for headers of group member extensions. Finally, Section 4 provides sample headers from FITS table extensions containing grouping structures. 2. Group Tables A group table, as defined in this convention, is a FITS table extension that contains a list of all the associated member HDUs in the group. Group tables may be represented by either FITS ASCII tables XTENSION= 'TABLE ' or binary tables XTENSION= 'BINTABLE', and are uniquely distinguished from other types of FITS tables by having the EXTNAME = 'GROUPING' keyword and value in the header. The other required or recommended keywords and columns in a group table are described in the following sections. There may be zero, one, or more group tables within a given FITS file. Each group table may reference any number of HDUs. The entire set of HDUs referenced in a group table, along with the group table itself, form a group . Individual HDUs referenced in a group table are said to be members of the group or group members. Groups can contain any type and mix of HDU. This includes all of the IAU-endorsed extensions as well as other extensions that conform to the requirements for generalized FITS extensions. Note that a group may also contain other groups as members, since a group table is itself a FITS extension. This feature allows for the construction of hierarchical structures of HDUs within a single FITS file or across many FITS files. 2.1 Group Member Identification Methods Group tables specify the names and locations of FITS files containing member HDUs as well as identifying members within their FITS files. The name and location of each FITS file is specified by using the World-Wide Web (Berners-Lee, 1994) Uniform Resource Identifiers, or URIs. All current and future forms of URIs, such as Uniform Resource Locators (URL) and the proposed Uniform Resource Names (URN), shall constitute valid names, although the group table must specify the type of URI being used. If the group member resides in a different FITS file but on the same computer system then partial URIs (specifically partial URLs) may be used instead of absolute URIs to specify the member's file location. If the group member resides in the same FITS file as the group table itself, then the URI field may be left blank. The location of member HDUs within FITS files may be specified in two different ways, either by reference or by absolute position. The reference identification method uses the values of the XTENSION, EXTNAME and EXTVER keywords to uniquely identify the member HDU within the FITS file. The position method uses the HDU order number to identify members, with the primary array having order value 0, the first extension order value 1, and so on. Users may choose either or both identification methods when constructing a group table. While the reference method is not invalidated by a reordering of HDU positions within FITS files, it does require that each member HDU have a unique set of (non-FITS-required) keyword values, Thus, this method may present problems for FITS files whose headers cannot be easily modified, such as FITS files on read-only media. The position identification method provides for quick ``random'' access to the member HDUs, since software does not have to sort though each extension looking for the correct set of keyword values, but will be affected if the order of member HDUs within their FITS files is changed (please note: there is nothing within the current FITS standard governing how or when HDUs may be reordered within their files). 2.2 Group Table Keywords In addition to the standard required FITS table extension keywords, the following keywords are required in the header of a group table: EXTNAME (character): This value of the FITS reserved keyword uniquely identifies that this FITS extension contains a group table. For group tables EXTNAME must have the value `GROUPING'. EXTVER (positive integer): The value of this FITS reserved keyword serves as a group ID number that uniquely distinguishes this group from any other groups that may be defined in the same FITS file. This group number may also be used in the header of each group member to identify the group(s) to which the member belongs (see section 3, GRPIDn keyword). The following keyword is strongly recommended for inclusion in the header of each group table: GRPNAME (character): This keyword contains the name associated with the group table. GRPNAME values are case-insensitive and should only contain letters, digits, and the underscore character (and not contain any embedded blank (ASCII 32) characters). 2.3 Group Table Columns The number of columns required in a group table depends on which method is used to identify the members (and recall that both methods may be used within the same group). If the members are identified by reference then the following columns are required: TTYPEn = 'MEMBER_XTENSION' -- character field: Contains the value of the XTENSION keyword from the group member's header. In the case of primary HDUs where there is no required XTENSION keyword, the value of `PRIMARY' will be used instead. Therefore, the current valid entries for this column are 'PRIMARY ', 'TABLE '+, 'BINTABLE', 'IMAGE ' or any other IAU FITS Working Group registered XTENSION value. This field may contain the FITS null value appropriate for this column type if the value is unknown (e.g., if the position identification method described below is used to identify the member location). TTYPEn = 'MEMBER_NAME' -- character field: Contains the value of the EXTNAME keyword from the group member's header. In the case of primary HDUs where the EXTNAME keyword is not defined or when the member extension has no EXTNAME keyword present, this field may contain the FITS null value appropriate for the column type. TTYPEn = 'MEMBER_VERSION' -- integer field: Contains the value of the EXTVER keyword from the group member's header. In the case of primary HDUs, or if the EXTVER keyword is not present in the member header then a value of 1 should be assumed. If members are identified by file position then the following column is required: TTYPEn = 'MEMBER_POSITION' -- integer field: Contains a group member's position within its FITS file. The file's primary header is given a position value of 0, the first extension is given a position value of 1, and so on. If for some reason a group member's `MEMBER_POSITION' value becomes invalid or undefined, then this column field should be filled with the FITS null value appropriate for the column format. If some or all of the group members reside in FITS files separate from the group table itself then the following two columns are also required: TTYPEn = 'MEMBER_LOCATION' -- character field: Contains the location of the group member's FITS file using Uniform Resource Identifiers. If the FITS file resides on the same computer system as the group table, then partial URIs may be used instead of absolute URIs, and if the group member resides in the same FITS file as the group table then the field may be filled with blanks. If the MEMBER_LOCATION value becomes unknown or invalid then this field may be filled with the FITS null value appropriate for the column type. TTYPEn = 'MEMBER_URI_TYPE' -- character field: Contains the mne-monic for the Uniform Resource Identifier type used in the corresponding MEMBER_LOCATION field. Recommended values for this column field are `URL' for the Uniform Resource Locator and `URN' for the Uniform Resource Name. As other URI types are defined their mnemonics will also become acceptable values for this field. In cases where the MEMBER_URI_TYPE is undefined (such as a null or blank MEMBER_LOCATION field value) this field may contain the FITS null value appropriate for the column type. Besides the table columns defined above, a group table may contain any number of user defined columns. Group table columns may appear in any order within the table and their TTYPEn values are not to be considered case-sensitive. 3. Keywords for Group Member Extensions No additional keywords are required for HDUs that are members of a group. This rule is to ensure that all currently existing FITS files and their constituent HDUs may all be part of this convention. There are, however, several grouping related keywords whose presence is strongly recommended in newly created headers. The description of these keywords follow. EXTNAME (character): This keyword is the FITS reserved keyword EXTNAME. Extensions that are part of groups should use this keyword to allow member identification by reference. EXTVER (integer): This keyword is the FITS reserved keyword EXTVER. Extensions that are part of groups should use this keyword in their headers to allow member identification by reference. GRPIDn (integer): A series of indexed keywords that denote the group(s) to which an HDU belongs. The value of GRPIDn is the EXTVER value of the nth group table that the HDU is a member of. In this sense, the EXTVER value of a group table defines a unique ID for the group within a FITS file. If the value of GRPIDn is negative, then the HDU is a member of a group defined in another file. In this case the absolute value of GRPIDn is the EXTVER value of the external group table, and the corresponding GRPLCn keyword holds the URI of the FITS file containing the group's table. The GRPIDn keywords (and their associated GRPLCn keywords) not only identify HDUs as members of groups, but also allow group members to ``point'' back to their group tables. Any software that might change the position or nature of the HDU would know that it was a member of a group and that the group table would require updating. GRPLCn (character): A series of indexed keywords that contain the Uniform Resource Identifiers corresponding to the GRPIDn keyword. The GRPLCn values follow the same syntax rules as those specified for the group table's MEMBER_LOCATION column (see section 2). It is unnecessary to have a GRPLCn keyword accompany a GRPIDn keyword when the value of the GRPIDn keyword is positive. 4. Example Group Table Headers The following are examples of valid group table headers that use different combinations of identification methods. Example 1: A group containing five members all of which reside in the same file as the group table. This group is itself a member of two other groups and both of those groups' tables reside in the same file as this extension. The member position identification method is used to locate member HDUs. XTENSION= 'BINTABLE' / This is a binary table BITPIX = 8 / Table contains 8-bit bytes NAXIS = 2 / Number of axis NAXIS1 = 4 / Width of table in bytes NAXIS2 = 5 / Number of member entries GCOUNT = 1 / Mandatory FITS keyword PCOUNT = 0 / Number of bytes in HEAP area TFIELDS = 1 / Number of columns in table EXTNAME = 'GROUPING' / This BINTABLE contains a group EXTVER = 3 / The ID number of this group GRPID1 = 1 / Part of group 1 GRPID2 = 2 / Part of group 2 TTYPE1 = 'MEMBER_POSITION' / Position of member within file TFORM1 = '1J' / Datatype descriptor END Example 2: A group containing 150 members, some of which reside in FITS files different from that of the group table. This group is not a member of any other group, although it is the seventh group table defined in the FITS file. All member identification methods are used. XTENSION= 'BINTABLE' / This is a binary table BITPIX = 8 / Table contains 8-bit bytes NAXIS = 2 / Number of axis NAXIS1 = 76 / Width of table in bytes NAXIS2 = 150 / Number of member entries GCOUNT = 1 / Mandatory FITS keyword PCOUNT = 0 / Number of bytes in HEAP area TFIELDS = 6 / Number of columns in table EXTNAME = 'GROUPING' / This BINTABLE contains a group EXTVER = 7 / The ID number of this group TTYPE1 = 'MEMBER_LOCATION' / URI of file containing member HDU TFORM1 = '30A ' / Datatype descriptor TTYPE2 = 'MEMBER_URI_TYPE' / URI type of MEMBER_LOCATION field TFORM2 = '3A ' / Datatype descriptor TTYPE3 = 'MEMBER_POSITION' / Position of member within file TFORM3 = '1J ' / Datatype descriptor TTYPE4 = 'MEMBER_XTENSION' / XTENSION keyword value of member TFORM4 = '8A ' / Datatype descriptor TTYPE5 = 'MEMBER_NAME' / EXTNAME keyword value of member TFORM5 = '30A ' / Datatype descriptor TTYPE6 = 'MEMBER_VERSION' / EXTVER keyword value of member TFORM6 = '1J ' / Datatype descriptor END Example 3: A group containing 17 members, some of which reside in FITS files different from that of the group table. This group is a member of six other groups, two of which are defined in FITS files on other computer systems and one that is defined in a FITS file on the same computer system. The member reference identification and member file location methods are used. Two user defined columns are also present. XTENSION= 'BINTABLE' / This is a binary table BITPIX = 8 / Table contains 8-bit bytes NAXIS = 2 / Number of axis NAXIS1 = 180 / Width of table in bytes NAXIS2 = 17 / Number of member entries GCOUNT = 1 / Mandatory FITS keyword PCOUNT = 0 / Number of bytes in HEAP area TFIELDS = 7 / Number of columns in table EXTNAME = 'GROUPING' / This BINTABLE contains a group EXTVER = 7 / The ID number of this group GRPID1 = 3 / Member of group 3 GRPID2 = 6 / Member of group 6 GRPID3 = 18 / Member of group 18 GRPID4 = -1 / Member of external group 1 GRPLC4 = 'http://fits.gsfc.nasa.gov/FITS/file1.fits' / location of COMMENT FITS file containing group GRPID5 = -5 / Member of external group 5 GRPLC5 = '/FITS/file5.fits' / Location of file containing group GRPID6 = -2 / Member of external group 2 GRPLC6 = 'http://www.noao.edu/irafdir/file2.fits' / location of COMMENT FITS file containing group TTYPE1 = 'USER_INFO_1' / A user supplied column TFORM1 = '25J ' / Datatype descriptor TTYPE2 = 'MEMBER_LOCATION' / URI of file containing member HDU TFORM2 = '30A ' / Datatype descriptor TTYPE3 = 'MEMBER_XTENSION' / XTENSION keyword value of member TFORM3 = '8A ' / Datatype descriptor TTYPE4 = 'MEMBER_NAME' / EXTNAME keyword value of member TFORM4 = '30A ' / Datatype descriptor TTYPE5 = 'USER_INFO_2' / A user supplied column TFORM5 = '5A ' / Datatype descriptor TTYPE6 = 'MEMBER_VERSION' / EXTVER keyword value of member TFORM6 = '1J ' / Datatype descriptor TTYPE7 = 'MEMBER_URI_TYPE' / URI type of MEMBER_LOCATION field TFORM7 = '3A ' / Datatype descriptor END Example 4: A group containing 82 members, some of which reside in FITS files different from that of the group table. This group is a member of three other groups, and makes use of the member position and member file location methods. One user defined column is present. Note that in this example an ASCII table (as opposed to a binary table) is used to define the group. XTENSION= 'TABLE ' / This is an ASCII table BITPIX = 8 / Table contains 8-bit ASCII characters NAXIS = 2 / Number of axis NAXIS1 = 46 / Width of table in bytes NAXIS2 = 82 / Number of member entries GCOUNT = 1 / Mandatory FITS keyword PCOUNT = 0 / Mandatory FITS keyword TFIELDS = 4 / Number of columns in table EXTNAME = 'GROUPING' / This TABLE contains a group EXTVER = 31 / The ID number of this group GRPID1 = 3 / Member of group 3 GRPID2 = 9 / Member of group 9 GRPID3 = 27 / Member of group 27 TTYPE1 = 'USER_INFO_1' / A user supplied column TFORM1 = 'E10.3 ' / Datatype descriptor TBCOL1 = 1 / Starting table column for field TTYPE2 = 'MEMBER_LOCATION' / URI of file containing member HDU TFORM2 = 'A30 ' / Datatype descriptor TBCOL2 = 11 / Starting table column for field TTYPE3 = 'MEMBER_URI_TYPE' / URI type of MEMBER_LOCATION field TFORM3 = 'A3 ' / Datatype descriptor TBCOL3 = 41 / Starting table column for field TTYPE4 = 'MEMBER_POSITION' / XTENSION keyword value of member TFORM4 = 'I3 ' / Datatype descriptor TBCOL4 = 44 / Starting table column for field END 5. Acknowledgments We gratefully acknowledge the support of the NASA Applied Information Systems Research Program, underwhich this effort is funded. 6. References Berners-Lee, Tim, 1994. ``World Wide Web Initiative'', CERN - European Particle Physics Lab. http://info.cern.ch/hypertext/WWW /TheProject.html . Cotton, W. D., Tody, D. and Pence W., 1994. ``Binary Table Extension to FITS: A Proposal'', version dated June 13, 1994, final form. NRAO FITS Archives. http://fits.cv.nrao.edu/fits/documents/standards/bintable3.ps . Grosbol, P., Harten, R. H., Greisen, E. W., and Wells, D. C., 1988. ``Generalized extensions and blocking factors for FITS,'' Astronomy and Astrophysics Suppl., 73, 359-364. Harten, R. H., Grosbol. P., Greisen, E. W., and Wells, D. C., 1988. ``The FITS tables extension'', Astronomy and Astrophysics Suppl., 73, 365-372. Ponz, J. D., Thompson, R. W., and Munoz, J. R., 1994. ``FITS Image Extension'' , Astronomy and Astrophysics Suppl., vol 105, pp 53-55. From fitsbits-request Tue Dec 20 19:36:43 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["3291" "" "20" "December" "1994" "23:55:49" "GMT" "Rob Seaman" "seaman at noao.edu" "<3d7qu5$6ol at noao.edu>" "101" "FITS checksum proposal" "^From:" nil nil "12" "1994122023:55:49" "FITS checksum proposal" (number " " mark " Rob Seaman Dec 20 101/3291 " thread-indent "\"FITS checksum proposal\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA27601; Tue, 20 Dec 94 19:36:43 EST Return-Path: Message-Id: <3d7qu5$6ol at noao.edu> Organization: National Optical Astronomy Observatories, Tucson, AZ, USA Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!caen!newsxfer.itd.umich.edu!europa.eng.gtefsd.com!library.ucla.edu!csulb.edu!nic-nac.CSU.net!news.Cerritos.edu!news.Arizona.EDU!CS.Arizona.EDU!noao!seaman Newsgroups: sci.astro.fits From: seaman at noao.edu (Rob Seaman) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: FITS checksum proposal Date: 20 Dec 1994 23:55:49 GMT FITS Checksum Proposal R.L. Seaman and W.D. Pence 20 December 1994 ABSTRACT This paper describes a simple means for embedding a checksum within a FITS header, providing a mechanism for later verifying the integrity of the FITS file. The method uses ASCII coded 32 bit 1's complement arithmetic to force the checksum of each FITS HDU to zero. This technique requires no extension of the FITS standard to be used by any given project, in fact the technique may be used to zero the checksum of any ASCII file. The purpose of this proposal is simply to reserve keywords that will allow projects and software packages to seamlessly verify and update each other's data. -------- A copy of the proposal is available via anonymous ftp from iraf.noao.edu in the /misc/checksum directory, where you will find the following files: README this file checksum.ps FITS Checksum Proposal, Seaman & Pence stb249851.fits FITS file w/ CHECKSUM and DATASUM keywords sum32.tar simple C program to calculate the 32 bit 1's complement checksum and to ASCII encode the checksum (w/ SunOS binary) FITS.sum.example pseudo FITS header with CHECKSUM keyword FITS.sum.zeroed same header with the checksum zeroed -------- The two files, FITS.sum.example and FITS.sum.zeroed, are an example of how the technique works. Before accumulating the checksum for a FITS file, set the value of the CHECKSUM keyword to all 0's: SIMPLE = T / BITPIX = 8 / NAXIS = 0 / to align the next keyword CHECKSUM= '0000000000000000' / ASCII 1's complement checksum DATASUM = '0000000000000000' / checksum of data records END (The comment for the NAXIS keyword shifts the alignment of the CHECKSUM keyword to impersonate real FITS - these are not 80 character cardimages.) For this example, calculate the checksum using the sum32 program: % sum32 -p FITS.sum.example checksum16: 23910 = IHKGIGIG complement: 41625 = VZWXVXVX checksum32: 3708584025 = Fh3PGg3PFg3PFg3P complement: 0586383270 = YAoRa1lOS8lOY8lO file size: 255 bytes (The -p tells the program to permute the ASCII encoding for the odd byte alignment required by placement of the FITS keyword value.) Edit the header to replace the 0's with the complement of the checksum (32 bit version): SIMPLE = T / BITPIX = 8 / NAXIS = 0 / to align the next keyword CHECKSUM= 'YAoRa1lOS8lOY8lO' / ASCII 1's complement checksum DATASUM = '0000000000000000' / checksum of data records END with the result that the checksum of the FITS header (or file, or extension) is set to zero: % sum32 FITS.sum.zeroed checksum16: 65535 = rroooooo complement: 00000 = 00000000 checksum32: 4294967295 = rrrroooooooooooo complement: 0000000000 = 0000000000000000 file size: 255 bytes To later verify the file, accumulate the checksum and see if the checksum is still zero. Rob Seaman seaman at noao.edu Bill Pence pence at tetra.gsfc.nasa.gov -- Rob Seaman: seaman at noao.edu/602-325-9248 National Optical Astronomy Observatories 950 North Cherry Avenue, Tucson AZ 85719