From fitsbits-request Fri Jul 1 03:57:13 1994 X-VM-Message-Order: (1 2 4 5 6 24 25 3 31 27 26 14 15 18 17 22 23 16 28 29 30 32 19 33 34 35 36 41 37 38 40 39 7 9 42 12 13 44 43 20 45 47 49 50 10 21 11 46 8 51 52 48 53 54 55 56 57 60 58 59) X-VM-Summary-Format: "%n %*%a %-17.17F %-3.3m %2d %4l/%-5c %I\"%s\"\n" X-VM-Labels: nil X-VM-VHeader: ("Resent-" "From:" "Sender:" "To:" "Apparently-To:" "Cc:" "Subject:" "Date:") nil X-VM-Bookmark: 60 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["499" "" "30" "June" "1994" "00:01:05" "-0400" "SteveS1825" "steves1825 at aol.com" nil "17" "Re: FITS on Mac via NIH Image Mods" "^From:" nil nil "6" nil nil (number " " mark " SteveS1825 Jun 30 17/499 " thread-indent "\"Re: FITS on Mac via NIH Image Mods\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA00758; Fri, 1 Jul 94 03:57:13 EDT Return-Path: Message-Id: <2utg21$cvd at search01.news.aol.com> Organization: America Online, Inc. (1-800-827-6364) Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!news.dfn.de!news.dfn.de!Germany.EU.net!EU.net!uunet!newstf01.cr1.aol.com!search01.news.aol.com!not-for-mail References: <28JUN199408331456 at nssdca.gsfc.nasa.gov> From: steves1825 at aol.com (SteveS1825) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS on Mac via NIH Image Mods Date: 30 Jun 1994 00:01:05 -0400 In article <28JUN199408331456 at nssdca.gsfc.nasa.gov>, bschlesinger at nssdca.gsfc.nasa.gov (BARRY M. SCHLESINGER) writes: Does it have limits on the values of NAXIS and BITPIX it will accept? Barry Schlesinger NOST FITS Support Office Barry, I don't know, I have yet to obtain access to FITS format images apart from the handful that came from the program, so I have yet to come near to testing it to its' limits. I would say that it was definitely worth taking a look at. From fitsbits-request Sat Jul 2 23:35:45 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1399" "Sun" " 3" "July" "1994" "03:34:19" "GMT" "Patrick P. Murphy" "pmurphy at nrao.edu" nil "33" "Re: FITS on Mac via NIH Image Mods" "^From:" nil nil "7" nil nil (number " " mark " Patrick P. Murphy Jul 3 33/1399 " thread-indent "\"Re: FITS on Mac via NIH Image Mods\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA04257; Sat, 2 Jul 94 23:35:45 EDT Return-Path: Message-Id: Organization: National Radio Astronomy Observatory Path: saips.cv.nrao.edu!news!pmurphy References: <28JUN199408331456 at nssdca.gsfc.nasa.gov> <2utg21$cvd at search01.news.aol.com> From: pmurphy at nrao.edu (Patrick P. Murphy) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS on Mac via NIH Image Mods Date: Sun, 3 Jul 1994 03:34:19 GMT (attribution corrected) In article <2utg21$cvd at search01.news.aol.com>, steves1825 at aol.com (SteveS1825) writes: > In article <28JUN199408331456 at nssdca.gsfc.nasa.gov>, > bschlesinger at nssdca.gsfc.nasa.gov (BARRY M. SCHLESINGER) writes: >> Does it have limits on the values of NAXIS and BITPIX it will accept? > I don't know, I have yet to obtain access to FITS format images > apart from the handful that came from the program, so I have yet to > come near to testing it to its' limits. If AOL gives you the ability to use ftp, try the following: fits.cv.nrao.edu (192.33.115.8) /fits/sampledata/image/ info.cv.nra.edu (192.33.115.103) /pub/aips/DDT/15JAN94/ These directories contain many samples of test fits files. If you explore the fits server, you'll find even more. - Pat -- ============================================================================= Patrick P. Murphy, Ph.D. Scientific Programming Analyst National Radio Astronomy Observatory pmurphy at nrao.edu 520 Edgemont Road Tel: (804) 296-0372 Charlottesville, VA 22903-2475 Fax: (804) 296-0278 web: http://orangutan.cv.nrao.edu/ "I don't believe in the no-win scenario" --- James T. Kirk ============================================================================= From fitsbits-request Mon Jul 4 20:11:27 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["894" "Mon" " 4" "July" "1994" "21:48:35" "GMT" "Stefan Mochnacki" "stefan at helios.physics.utoronto.ca" nil "18" "Need FITS --> uuencoded in < 64KB chunks utility." "^From:" nil nil "7" nil nil (number " " mark " Stefan Mochnacki Jul 4 18/894 " thread-indent "\"Need FITS --> uuencoded in < 64KB chunks utility.\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA08830; Mon, 4 Jul 94 20:11:27 EDT Return-Path: Message-Id: Organization: University of Toronto Physics/Astronomy/CITA Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!howland.reston.ans.net!cs.utexas.edu!utnut!helios.physics.utoronto.ca!stefan From: stefan at helios.physics.utoronto.ca (Stefan Mochnacki) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Need FITS --> uuencoded in < 64KB chunks utility. Date: Mon, 4 Jul 1994 21:48:35 GMT We are starting to get more requests to send occasional images via e-mail. This sounds silly, I know, but we have had some dreadful problems with people e-mailing giant files. Many machines don't have anonymous ftp servers, so rapid file transfer can be difficult. My question therefore is: can somebody point to a utility which takes a given file (preferably FITS) and breaks it up into uuencoded chunks which can be easily reassembled, the chunks being no more than 64KB each , preferably mailing the darn things as well ? (I don't particularly want a utility which people can use on Usenet images ...). -- Stefan W. Mochnacki INTERNET - stefan at centaur.astro.utoronto.ca Astronomy, U. of Toronto UUCP - {uunet,pyramid}!utai!helios.physics!stefan Ph. (905) 884-9562 LOCATION - David Dunlap Observatory FAX (905) 884-2672 Ph. (Mon,Wed) (416)-978-4165 (St.George Campus) From fitsbits-request Tue Jul 5 10:59:01 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["3750" "Tue" " 5" "July" "1994" "10:58:54" "-0400" "William Pence" "pence at tetra.gsfc.nasa.gov" nil "88" "FITS TSORTKEY keyword" "^From:" nil nil "7" nil nil (number " " mark " William Pence Jul 5 88/3750 " thread-indent "\"FITS TSORTKEY keyword\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA09435; Tue, 5 Jul 94 10:59:01 EDT Return-Path: Message-Id: <199407051458.KAA00475 at tetra.gsfc.nasa.gov> From: William Pence Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Cc: pence at tetra.gsfc.nasa.gov Subject: FITS TSORTKEY keyword Date: Tue, 5 Jul 1994 10:58:54 -0400 Last year there was some discussion on this newsgroup of a proposal for a FITS keyword to indicate the order in which a FITS table has been sorted. The OGIP FITS Working Group has recently readdressed this issue and is proposing that the 'TSORTKEY' keyword be used for this purpose as described below. Any comments or criticisms about this are welcome. Bill Pence Ofice of Guest Investigator Programs NASA/GSFC ----------------------------------------------------------------------- TSORTKEY: a FITS Keyword to Specify the Sort Order of a Table The TSORTKEY keyword is used to indicate the order in which the rows in a FITS TABLE or BINTABLE extension have been sorted. The value of the TSORTKEY keyword is a character string which lists the name (as given by the TTYPEn keyword) of the primary sort column, followed by the names of any (optional) secondary sort column(s). The rows in the table are sorted first by the value in the primary column; any rows which have the same value in the primary column are further sorted by the value in the secondary column and so on for all the specified sort columns. If more than one column is specified by TSORTKEY then the names must be separated by a comma. One or more spaces are also allowed between the comma and the following column name. By default, columns are sorted in ascending order, but a minus sign may precede the column name to indicate the rows are sorted in descending order. (Note that this implies that the names of any columns used to sort a table must not begin with a minus sign). Integer or floating point columns are sorted by numerical order and not by their internal ASCII representation in the case of TABLE extensions. Character columns are sorted by the ASCII collating sequence order of the entire string by default. If the table has been sorted on a substring within the character column, this may be indicated by including the range of the substring characters in parentheses after the column name (e.g., 'NAME(3:8)' indicates that the rows of the table are sorted based on the 3rd through 8th characters in the NAME character column). The sort order for logical format columns (L) in BINTABLE extensions is defined such that all false values (F) will precede the true values (T) when sorted in ascending order. The sort order of complex data type columns ('C' or 'M') is not defined by this convention. In the case of vector columns in binary tables, the TSORTKEY convention only supports sorting on the value of a particular element within the vector (except for character string vectors as described above). The vector element number that was used to sort the table is included in parentheses following the vector column name. If no element is explicitly specified it is assumed that the table is sorted on the first element in the vector. Any null/undefined elements in a sort column will appear after all the defined values when the table is sorted in ascending order. Undefined elements will precede all defined elements when sorted in descending order. Examples of TSORTKEY usage: TSORTKEY = 'X ' The table is sorted in ascending value of the X column. TSORTKEY = 'X,Y ' The table is sorted in ascending value of the X column. Rows which have the same value of X have been further sorted in ascending value of the Y column. TSORTKEY = '-TIME ' The table is sorted in descending value of the TIME column. TSORTKEY = '-OBJECT(2:4)' The table is sorted in reverse alphabetical order on the 2nd through 4th characters in the OBJECT character string column. TSORTKEY = 'ARRAY(10)' The table is sorted in ascending value of the 10th element of the ARRAY vector column. From fitsbits-request Tue Jul 5 12:45:57 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["2705" "Tue" " 5" "July" "1994" "12:45:51" "-0400" "William Pence" "pence at tetra.gsfc.nasa.gov" nil "57" "FITS continuation keywords" "^From:" nil nil "7" nil nil (number " " mark " William Pence Jul 5 57/2705 " thread-indent "\"FITS continuation keywords\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA09538; Tue, 5 Jul 94 12:45:57 EDT Return-Path: Message-Id: <199407051645.MAA00598 at tetra.gsfc.nasa.gov> From: William Pence Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Cc: pence at tetra.gsfc.nasa.gov Subject: FITS continuation keywords Date: Tue, 5 Jul 1994 12:45:51 -0400 Last summer there was considerable debate in this newgroup about a proposed convention for encoding very long character strings as the value of FITS keywords. We will soon need to begin using this convention (or some variant on it) for the FITS files generated by the XTE mission. The main purpose of this posting is to see if there is any concensus within the FITS communitiy as to which of 2 possible variations on this convention we should use. Our orginal (and still preferred) proposal is to use a backslash character (\) at the end of a string to indicate that is continued on the next keyword, and to have columns 1=10 of the following keyword(s) be filled with blanks. FITS readers that do not understand this continuation convention would just treat the continuation lines as comment keywords (since the keyword name is blank). For example: LONGSTR = 'This is a very long string keyword value that \' 'is continued over three\' 'keywords in the FITS header.' One objection that was raised to this convention was that some FITS readers apparently treat comment keywords differently from the other keywords that have a value, and the keywords might get separated, thus destroying the order of the keywords that is needed to reconstruct the long string value. One way to solve this particular problem (if it really is a problem) is to replace the blank keyword name with a reserved keyword such as 'CONTINUE' followd by an equal sign, as in: LONGSTR = 'This is a very long string keyword value that \' CONTINUE= 'is continued over three\' CONTINUE= 'keywords in the FITS header.' Is there any preference as to which of these 2 conventions we use? If not we will probably use the original proposal, with blank continuation keyword names. Incidentally, to preempt a common suggestion, it is not possible to use a numbered sequence of keywords for this convention, as in: LONGS1 = 'This is a very long string keyword value that \' LONGS2 = 'is continued over three\' LONGS3 = 'keywords in the FITS header.' because we need to use this convention in FITS tables where the keyword name already has a sequence number attached to it to indicate which column of the table it refers to. So the convention that we adopt must place no restrictions on the name of the long string keyword itself. Finally, we realize that this is a controversial proposal, and thus will restrict its use to rather mission-specific keywords that should minimize the impact on other data systems that don't understand this convention. In most cases other data systems would not know what to do with these long string values even if they could read them. -Bill Pence OGIP, NASA/GSFC From fitsbits-request Tue Jul 5 14:59:03 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1467" "Tue" " 5" "July" "1994" "14:55:06" "-0400" "Nikolai Piskunov" "piskunov at phobos.astro.uwo.ca" nil "45" "Re: FITS continuation keywords" "^Resent-Date:" nil nil "7" nil nil (number " " mark " Nikolai Piskunov Jul 5 45/1467 " thread-indent "\"Re: FITS continuation keywords\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA09725; Tue, 5 Jul 94 14:59:03 EDT Return-Path: Reply-To: Nikolai Piskunov In-Reply-To: <01HECXU36O9E9GVRIG at hylk.Helsinki.FI> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Resent-Date: Tue, 5 Jul 1994 14:55:06 -0400 (DST) Resent-From: fitsbits-request Resent-Message-Id: <9407051859.AA09725 at fits.cv.nrao.edu> Resent-Sender: Nikolai Piskunov From: Nikolai Piskunov Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: Tue, 5 Jul 1994 14:55:06 -0400 (DST) On Tue, 5 Jul 1994, William Pence wrote: > ... > Last summer there was considerable debate in this newgroup about a > proposed convention for encoding very long character strings as the > value of FITS keywords. > ... > For example: > > LONGSTR = 'This is a very long string keyword value that \' > 'is continued over three\' > 'keywords in the FITS header.' > > ... > or to replace the blank keyword name with a > reserved keyword such as 'CONTINUE' followd by an equal sign, as in: > > LONGSTR = 'This is a very long string keyword value that \' > CONTINUE= 'is continued over three\' > CONTINUE= 'keywords in the FITS header.' > > Is there any preference as to which of these 2 conventions we use? > I feel that the idea of using \' combination may cause problems for FITS readers written in C. It may also result in confusion, if the text of the comment is interpreted after extraction with a shell script. On the other hand second example in WP mail shows certain redundancy. So what if we keep the keyword CONTINUE and forget the back slash? In other words, the example above will look like: LONGSTR = 'This is a very long string keyword value that ' CONTINUE= 'is continued over three ' CONTINUE= 'keywords in the FITS header.' Dr. Nikolai Piskunov Department of Astronomy University of Western Ontario London, Ontario N6A 3K7, Canada From fitsbits-request Tue Jul 5 19:37:15 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1062" "Tue" " 5" "July" "1994" "20:37:51" "GMT" "Rob Seaman" "seaman at noao.edu" nil "27" "Re: FITS continuation keywords" "^From:" nil nil "7" nil nil (number " " mark " Rob Seaman Jul 5 27/1062 " thread-indent "\"Re: FITS continuation keywords\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA10018; Tue, 5 Jul 94 19:37:15 EDT Return-Path: Message-Id: <1994Jul5.203751.21438 at noao.edu> Organization: National Optical Astronomy Observatories, Tucson, AZ, USA Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!howland.reston.ans.net!gatech!ncar!noao!seaman References: From: seaman at noao.edu (Rob Seaman) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: Tue, 5 Jul 1994 20:37:51 GMT Nikolai Piskunov writes: > LONGSTR = 'This is a very long string keyword value that ' > CONTINUE= 'is continued over three ' > CONTINUE= 'keywords in the FITS header.' This is perhaps a better suggestion - or, at least, the alternative offered of using a backslash (\) in a literal string as a continuation character seems liable to break a lot of software. However, nowhere in the FITS standard is there a mandate that the order of header keywords must be preserved. All of the options offered so far assume that the order of the header is unchangeable. Even the simple addition of a new keyword, let alone a wholesale header reordering or sorting, could disrupt a continuation card sequence. Any continuation convention must either overtly preserve the continuation order (within the context of the individual continuation cards), or the FITS standard must be amended to require more stringent limitations on header keyword operations. Are continuation cards really needed? Rob Seaman NOAO/IRAF Group seaman at noao.edu From fitsbits-request Wed Jul 6 04:53:15 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["192" "" " 6" "July" "1994" "01:28:04" "-0400" "steves1825 at aol.com" "steves1825 at aol.com" "<2vdfd4$dep at search01.news.aol.com>" "6" "Re: FITS on Mac via NIH Image Mods" "^From:" nil nil "7" "1994070605:28:04" "FITS on Mac via NIH Image Mods" (number " " mark " steves1825 at aol.co Jul 6 6/192 " thread-indent "\"Re: FITS on Mac via NIH Image Mods\"\n") ""] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA10295; Wed, 6 Jul 94 04:53:15 EDT Return-Path: Message-Id: <2vdfd4$dep at search01.news.aol.com> Organization: America Online, Inc. (1-800-827-6364) Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!swrinde!pipex!sunic!trane.uninett.no!eunet.no!nuug!EU.net!uunet!newstf01.cr1.aol.com!search01.news.aol.com!not-for-mail References: From: steves1825 at aol.com (SteveS1825) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS on Mac via NIH Image Mods Date: 6 Jul 1994 01:28:04 -0400 In article , pmurphy at nrao.edu (Patrick P. Murphy) writes: Pat, thank you for that ftp information for FITS files! I hope I can find a way to ftp. From fitsbits-request Wed Jul 6 05:20:55 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1722" "Wed" " 6" "July" "1994" "09:07:00" "+0000" "Clive Page" "cgp at star.le.ac.uk" "" "34" "TSORTKEY and continuation proposals" "^From:" nil nil "7" "1994070609:07:00" "TSORTKEY and continuation proposals" (number " " mark " Clive Page Jul 6 34/1722 " thread-indent "\"TSORTKEY and continuation proposals\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA10894; Wed, 6 Jul 94 05:20:55 EDT Return-Path: Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 1721 From: Clive Page Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: TSORTKEY and continuation proposals Date: Wed, 6 Jul 1994 09:07:00 +0000 (GMT) (1) Sort order I agree that we need a way of specifying the sort order of FITS tables. Many existing tables are actually sorted on some column or columns; for example the astronomical catalogues distributed by NSSDC on CD-ROM mostly seem to be sorted on right ascension, but at present you have to check all through the table to discover this. Many applications will be able to access such tables much more efficiently if the sorting order is declared. The proposal by Bill Pence seems simple and effective and I'd like to support it. (2) Strings that are so long that they need to be continued. I'd prefer the second option, that is to have the continuation lines labelled with 'CONTINUE= '. This is because lines with a blank keyword may not be processed properly by current systems. It is true that the FITS Standard does not require the order of keywords to be preserved, so any system of continuations is liable to fail, but in practice lots of existing FITS files have a sequence of keywords containing COMMENTs that only makes sense if they can be read in sequence; all the systems that I know of seem to preserve their order in such cases. If so then maybe the backslash character at the end of the string being continued is not necessary. The backslash can be hard to handle in C programs, and many Fortran systems running on Unix also treat the backslash as an escape character. If we can do without it, so much the better. If we do need a continuation character then I'd prefer the ampersand "&" as this is consistent with Fortran 90. ! Clive Page, Dept of Physics & Astronomy, University of Leicester, UK. ! e-mail: cgp at star.le.ac.uk ! phone: +44 533 523551 Fax: +44 533 550182 From fitsbits-request Wed Jul 6 06:38:03 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1600" "" " 6" "July" "1994" "09:39:18" "GMT" "Martin Shepherd" "mcs at goblin.caltech.edu" "" "44" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070609:39:18" "FITS continuation keywords" (number " " mark " Martin Shepherd Jul 6 44/1600 " thread-indent "\"Re: FITS continuation keywords\"\n") ""] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA11447; Wed, 6 Jul 94 06:38:03 EDT Return-Path: Message-Id: Organization: California Institute of Technology. Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!gatech!swrinde!elroy.jpl.nasa.gov!netline-fddi.jpl.nasa.gov!nntp-server.caltech.edu!nntp-server.caltech.edu!mcs References: From: mcs at goblin.caltech.edu (Martin Shepherd) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 06 Jul 1994 09:39:18 GMT In article Nikolai Piskunov writes: :On Tue, 5 Jul 1994, William Pence wrote: :> ... :> For example: :> :> LONGSTR = 'This is a very long string keyword value that \' :> 'is continued over three\' :> 'keywords in the FITS header.' :> ... :> or to replace the blank keyword name with a :> reserved keyword such as 'CONTINUE' followd by an equal sign, as in: :> :> LONGSTR = 'This is a very long string keyword value that \' :> CONTINUE= 'is continued over three\' :> CONTINUE= 'keywords in the FITS header.' :> :> Is there any preference as to which of these 2 conventions we use? :> : :I feel that the idea of using \' combination may cause problems for :FITS readers written in C. No. Back slashes and quotes are not significant in C programs except when they appear in the text of a C file at compilation time. :... :On the other hand second example in WP mail shows certain redundancy. :So what if we keep the keyword CONTINUE and forget the back slash? :In other words, the example above will look like: : : LONGSTR = 'This is a very long string keyword value that ' : CONTINUE= 'is continued over three ' : CONTINUE= 'keywords in the FITS header.' This seems logical, but I can see the value of having some indication in the string being continued that the current keyword is incomplete, rather than having to look ahead at the next keyword. IMHO it would be better to side-step the whole issue of continuation lines in FITS headers, and place such strings in tables. From fitsbits-request Wed Jul 6 06:40:24 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["880" "" " 6" "July" "1994" "09:41:02" "GMT" "Guy Rixon" "gtr at mail.ast.cam.ac.uk" "<2vdu7e$3i0 at lyra.csx.cam.ac.uk>" "18" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070609:41:02" "FITS continuation keywords" (number " " mark " Guy Rixon Jul 6 18/880 " thread-indent "\"Re: FITS continuation keywords\"\n") "<1994Jul5.203751.21438 at noao.edu>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA11461; Wed, 6 Jul 94 06:40:24 EDT Return-Path: Message-Id: <2vdu7e$3i0 at lyra.csx.cam.ac.uk> Organization: Royal Greenwich Observatory Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!gatech!swrinde!pipex!lyra.csx.cam.ac.uk!cast0.ast.cam.ac.uk!gtr References: <1994Jul5.203751.21438 at noao.edu> From: gtr at mail.ast.cam.ac.uk (Guy Rixon) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 6 Jul 1994 09:41:02 GMT I for one would prefer the CONTINUE keyword to blank keywords in continuation lines. If it's done this way, the FITS reader can detect a continuation line without having to check the value field, which simplifies the code. I'd also prefer to keep the value field free of continuation symbols. How about putting a sort key in the comment field? LONGSTR1= 'This string is going to run ' / 0001 CONTINUE= 'on to two lines... ' / 0002 LONGSTR2= 'This one is too, although ' / 0003 CONTINUE= 'I'm forcing the case... ' / 0004 Since the sort keys are unique within a header the text can be reassembled however the descriptors have been shuffled. An unenlightened FITS reader will still get all the data, and will read meaningful text in the value field, but won't do the reassembly for you. Is this any help to you? Guy. From fitsbits-request Wed Jul 6 06:59:55 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["2288" "" " 6" "July" "1994" "10:05:30" "GMT" "David Robinson" "drtr at mail.ast.cam.ac.uk" "<2vdvla$4pn at lyra.csx.cam.ac.uk>" "50" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070610:05:30" "FITS continuation keywords" (number " " mark " David Robinson Jul 6 50/2288 " thread-indent "\"Re: FITS continuation keywords\"\n") "<199407051645.MAA00598 at tetra.gsfc.nasa.gov>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA11490; Wed, 6 Jul 94 06:59:55 EDT Return-Path: Message-Id: <2vdvla$4pn at lyra.csx.cam.ac.uk> Organization: Institute of Astronomy, Cambridge Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!swrinde!ihnp4.ucsd.edu!agate!doc.ic.ac.uk!lyra.csx.cam.ac.uk!drtr References: <199407051645.MAA00598 at tetra.gsfc.nasa.gov> From: drtr at mail.ast.cam.ac.uk (David Robinson) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 6 Jul 1994 10:05:30 GMT In article <199407051645.MAA00598 at tetra.gsfc.nasa.gov> William Pence writes: >... >The main purpose of this posting is to see if there is any concensus >within the FITS communitiy as to which of 2 possible variations on this >convention we should use. Our orginal (and still preferred) proposal >is to use a backslash character (\) at the end of a string to indicate >that is continued on the next keyword, and to have columns 1=10 of the >following keyword(s) be filled with blanks. FITS readers that do not >understand this continuation convention would just treat the >continuation lines as comment keywords (since the keyword name is >blank). For example: > >LONGSTR = 'This is a very long string keyword value that \' > 'is continued over three\' > 'keywords in the FITS header.' > >One objection that was raised to this convention was that some FITS >readers apparently treat comment keywords differently from the other >keywords that have a value, and the keywords might get separated, thus >destroying the order of the keywords that is needed to reconstruct the >long string value. One way to solve this particular problem (if it >really is a problem) is to replace the blank keyword name with a >reserved keyword such as 'CONTINUE' followd by an equal sign, as in: > >LONGSTR = 'This is a very long string keyword value that \' >CONTINUE= 'is continued over three\' >CONTINUE= 'keywords in the FITS header.' > >Is there any preference as to which of these 2 conventions we use? >If not we will probably use the original proposal, with blank continuation >keyword names. >... Unfortunately, a FITS file with multiple CONTINUE keywords does not conform to the NOST standard, and could be rejected by a fits reader. Your first suggestion is the only way of doing this, (apart from numbered keywords); however, I would suggest making your fits headers independent of the keyword order, e.g. LONGSTR = 'This is a very long string keyword value that \' LONGSTR 2 'is continued over three\' LONGSTR 3 'keywords in the FITS header.' The keyword name and a seqence number appear in every record. C programmers should have no problem with strings containinf '\'. David Robinson. (drtr at mail.ast.cam.ac.uk) From fitsbits-request Wed Jul 6 08:06:48 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1455" "Wed" " 6" "July" "1994" "14:12:19" "+0200" "Lucio Chiappetti" "lucio at ifctr.mi.cnr.it" "" "32" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070612:12:19" "FITS continuation keywords" (number " " mark " Lucio Chiappetti Jul 6 32/1455 " thread-indent "\"Re: FITS continuation keywords\"\n") "<2vdvla$4pn at lyra.csx.cam.ac.uk>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA11577; Wed, 6 Jul 94 08:06:48 EDT Return-Path: Organization: Istituto di Fisica Cosmica e Tecnologie Relative In-Reply-To: <2vdvla$4pn at lyra.csx.cam.ac.uk> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII From: Lucio Chiappetti Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: Wed, 6 Jul 1994 14:12:19 +0200 (MET DST) On 6 Jul 1994, David Robinson wrote: > > Unfortunately, a FITS file with multiple CONTINUE keywords does not conform > to the NOST standard, and could be rejected by a fits reader. > > Your first suggestion is the only way of doing this, (apart from numbered > keywords); however, I would suggest making your fits headers independent of > the keyword order, e.g. > > LONGSTR = 'This is a very long string keyword value that \' > LONGSTR 2 'is continued over three\' > LONGSTR 3 'keywords in the FITS header.' > > The keyword name and a seqence number appear in every record. > IF the need for long keyword really exists (which I feel questionable besides the case of HISTORY keywords), mainly for descriptive and mission specific purposes, why not just treat them as comments ?: > COMMENT = 'LONGSTR This is a very long string keyword value that \' > COMMENT = 'LONGSTR 2 is continued over three\' > COMMENT = 'LONGSTR 3 keywords in the FITS header.' ---------------------------------------------------------------------------- Lucio Chiappetti - IFCTR/CNR | Ma te' vugl' da' quost avis a ti' Orsign via Bassini 15 - I-20133 Milano | Buttet rabios intant te se' pisnign Internet: LUCIO at IFCTR.MI.CNR.IT | Decnet: IFCTR::LUCIO | (Rabisch, II 46, 119-120) ---------------------------------------------------------------------------- From fitsbits-request Wed Jul 6 10:01:51 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["3219" "" " 6" "July" "1994" "13:46:50" "GMT" "Paul Repacholi" "prep at yarrow.wt.uwa.edu.au" "" "75" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070613:46:50" "FITS continuation keywords" (number " " mark " Paul Repacholi Jul 6 75/3219 " thread-indent "\"Re: FITS continuation keywords\"\n") "<199407051645.MAA00598 at tetra.gsfc.nasa.gov>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA00270; Wed, 6 Jul 94 10:01:51 EDT Return-Path: Message-Id: Organization: Winthrop Technology Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!swrinde!ihnp4.ucsd.edu!munnari.oz.au!news.uwa.edu.au!news!prep References: <199407051645.MAA00598 at tetra.gsfc.nasa.gov> From: prep at yarrow.wt.uwa.edu.au (Paul Repacholi) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 06 Jul 1994 13:46:50 GMT In article <199407051645.MAA00598 at tetra.gsfc.nasa.gov> William Pence writes: > > Last summer there was considerable debate in this newgroup about a > proposed convention for encoding very long character strings as the > value of FITS keywords. We will soon need to begin using this ^^^^ > convention (or some variant on it) for the FITS files generated by the > XTE mission. > > The main purpose of this posting is to see if there is any concensus > within the FITS communitiy as to which of 2 possible variations on this > convention we should use. Our orginal (and still preferred) proposal > is to use a backslash character (\) at the end of a string to indicate > that is continued on the next keyword, and to have columns 1=10 of the > following keyword(s) be filled with blanks. FITS readers that do not > understand this continuation convention would just treat the > continuation lines as comment keywords (since the keyword name is > blank). For example: > > LONGSTR = 'This is a very long string keyword value that \' > 'is continued over three\' > 'keywords in the FITS header.' This means interpreting the value of 'LONGSTR' as part of inputting it. I would rather see LONGSTR = ".... that'\ LONGSTR = 'is cont...'\ LONGSTR = 'to a final un-escaped line' type syntax. Thus each line is a 'LONGSTR' to old reader, and the semantics are cleaned up, and could extend to other keywords as well. > > One objection that was raised to this convention was that some FITS > readers apparently treat comment keywords differently from the other > keywords that have a value, and the keywords might get separated, thus > destroying the order of the keywords that is needed to reconstruct the > long string value. One way to solve this particular problem (if it > really is a problem) is to replace the blank keyword name with a > reserved keyword such as 'CONTINUE' followd by an equal sign, as in: > > LONGSTR = 'This is a very long string keyword value that \' > CONTINUE= 'is continued over three\' > CONTINUE= 'keywords in the FITS header.' > > Is there any preference as to which of these 2 conventions we use? > If not we will probably use the original proposal, with blank continuation > keyword names. Hum try this for parsing... LONGSTR = 'This is a long string\' CONTINUE= ' with\' COMMENT = 'This is a comment with\' CONTINUE= 'MORE THAN ONE LINE!! We think' FITS seems to have a problem with lack of clear interfaces and 'levels' of function. Putting the continue char into the value is an example of this. If the '\' is the LAST char, the reader can strip it, the closing ' and everything upto the next value start, then append the input to the buffer. Higher level code need not ever know there has been a change. Except for the size of the strings it will now be fed of course. ;-) The word "need" is the real worry though... -- ~Paul +61 (09) 257-1001 prep at yarrow.wt.uwa.edu.au ( preferred ) 1 Crescent Rd, zrepachol at cc.curtin.edu.au Kalamunda, West Aust 6076 From fitsbits-request Wed Jul 6 10:56:32 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["5523" "Wed" " 6" "July" "1994" "15:54:54" "+0100" "cur at star.rl.ac.uk" "cur at star.rl.ac.uk" "<9407061456.AA00395 at fits.cv.nrao.edu>" "105" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070614:54:54" "FITS continuation keywords" (number " " mark " cur at star.rl.ac.uk Jul 6 105/5523 " thread-indent "\"Re: FITS continuation keywords\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA00401; Wed, 6 Jul 94 10:56:32 EDT Return-Path: Message-Id: <9407061456.AA00395 at fits.cv.nrao.edu> Reply-To: cur at star.rl.ac.uk X-Vms-To: SMTP%"fitsbits at fits.cv.nrao.edu" From: cur at star.rl.ac.uk Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: Wed, 6 Jul 1994 15:54:54 +0100 The first question to address is "Are long strings necessary?". Even if you are prepared to lose the comments (undesirable IMHO) you are limited to 68 characters, and 18 characters if you have a comment starting at the normal column. I can think of many instances where a value in our data structures might exceed 68 characters, albeit not a common occurrence, and where quite frequently 18 characters would be inadequate. If the OGIP people did not have this need also, I do not think they would have gone to the trouble of making the proposal. So let's take it for granted, we need a way of storing header values comprising up to a few hundred characters. I do not like having such medium-length header-type information tucked away in tables as Martin Shepherd suggests. It has to be dealt with separately, possibly being written to a separate file (what is it called?) or not at all by some readers; thus the information can all too readily become detached from the dataset it is describing. So the question boils down to which ad-hoc convention does the job with minimal disruption to existing software and does not limit further developments. Starting with the OGIP proposal. The continuation-character convention restricts the content of the string, albeit in a minimal way, in that you cannot use a nominated character in a literal sense at the end of a string. In practice this is unlikely to be a problem except perhaps for strings written by users ignorant of the convention or in values written in existing files. Since readers would have to look at the next keyword to see whether or not it is a blank string or CONTINUE, it can be handled. Should readers complain if there is a continuation character, but no continuation line, and treat the final character as part of the keyword's value? The above would suggest not, but the following damning point against the OGIP proposal RS> However, nowhere in the FITS standard is there a mandate that the order RS> of header keywords must be preserved. All of the options offered so RS> far assume that the order of the header is unchangeable. Even the RS> simple addition of a new keyword, let alone a wholesale header RS> reordering or sorting, could disrupt a continuation card sequence. argues the contrary. It is fine for systems written by observatories and satellite missions, but not for those where users can input or relocate new header cards. Given header-validation software how will it know if the continuation character was intended to be literal or not? I have probably laboured this point more than it merits from a practical sense. Whatever is decided the documentation of the convention should be clarified accordingly. WDP> LONGSTR = 'This is a very long string keyword value that \' WDP> CONTINUE= 'is continued over three\' WDP> CONTINUE= 'keywords in the FITS header.' The redundancy question raised by Nikolai occurred to me too. Why is there a continuation character and a CONTINUE keyword? Is it that the continuation character flags to the reader that it has to look at the next header card? The CONTINUE card may have been used already for another purpose. The fact that it is not part of the current standard, is less of a problem for me. We would have to modify the standard to allow multiple CONTINUE cards. Coming back to Rob Seaman's posting, RS> Any continuation convention must either overtly preserve the continuation RS> order (within the context of the individual continuation cards), or the RS> FITS standard must be amended to require more stringent limitations on RS> header keyword operations. there have been some suggestions for the former. I am not keen on numbering schemes and ones that require looking at the comments (shades of UNIX shell scripts) as Guy Rixon suggests. I should rather have the number in the string if it could be done unambiguously. The trouble with numbering is that you have to know which numbers already exist, or at least the highest index, before inserting a new card. That is not a big deal, just a little messy to code offering more opportunity for violations, and is less efficient. If you want to connect keywords why not have a linked list by keyword name rather than number? (-: I am probably prejudiced against David Robinson's suggestion because it reminds me too much of Isaac Newton Group's hierarchical keywords. Given an arbitrary blank keyword (or COMMENT in Lucio Chiappetti's proposal) you will not be able to have the first word being another keyword followed by a number. Probably not a frequent occurrence, but it is possible. The disadvantage of using COMMENT keywords is that certain readers may detach the information from the data the long string describes. I do not think it unreasonable that a dataset has some ancillary information longer than 68 characters. >From a selfish point of view in Starlink we can cope with either OGIP proposal as we store the FITS header verbatim in our standard data structures, and so order is preserved. Presumably FITSIO users will also be able to handle whatever is agreed without pain. Of the suggestions made so far, I would plump for WDP> LONGSTR = 'This is a very long string keyword value that \' WDP> 'is continued over three\' WDP> 'keywords in the FITS header.' being the least undesirable. However, I would be interested in other proposals that avoid a specific ordering of header cards to if we can improve on the OGIP proposal. Malcolm Currie Starlink Project From fitsbits-request Wed Jul 6 11:26:00 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["856" "Wed" " 6" "July" "1994" "17:22:49" "+0200" "Thierry Forveille" "forveill at gag.observ-gr.fr" "<9407061525.AA00470 at fits.cv.nrao.edu>" "15" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070615:22:49" "FITS continuation keywords" (number " " mark " Thierry Forveille Jul 6 15/856 " thread-indent "\"Re: FITS continuation keywords\"\n") ""] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA00476; Wed, 6 Jul 94 11:26:00 EDT Return-Path: Message-Id: <9407061525.AA00470 at fits.cv.nrao.edu> Posted-Date: Wed, 6 Jul 1994 17:22:49 +0200 In-Reply-To: References: <199407051645.MAA00598 at tetra.gsfc.nasa.gov> From: forveill at gag.observ-gr.fr (Thierry Forveille) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: Wed, 6 Jul 1994 17:22:49 +0200 Since all proposed continuation syntaxes violate the present FITS rule that the header ordering is irrelevant, I'd like to see a better case for the need of >68 characters in header values. Contrary to a statement in one of the follow-ups to the original posting, some FITS readers (including ours...) do use this feature of the present standard. The disruption would be relatively minor since OGIP intends to use continuation lines for mission specific information only (and our users will presumably not look at any XTE product anyway), but this opens the door to a more general use and potential problems. The modification will put a small but non-zero burden on a number of people, so its proposers need to convince the community through a worked out example that their purpose cannot be met otherwise. Thierry Forveille Observatoire de Grenoble From fitsbits-request Wed Jul 6 14:37:04 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1963" "Wed" " 6" "July" "1994" "17:32:09" "GMT" "Frank Valdes" "valdes at tucana.tuc.noao.edu" "<1994Jul6.173209.29646 at noao.edu>" "41" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070617:32:09" "FITS continuation keywords" (number " " mark " Frank Valdes Jul 6 41/1963 " thread-indent "\"Re: FITS continuation keywords\"\n") "<9407061525.AA00470 at fits.cv.nrao.edu>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA01211; Wed, 6 Jul 94 14:37:04 EDT Return-Path: Message-Id: <1994Jul6.173209.29646 at noao.edu> Organization: IRAF Project, National Optical Astronomy Observatories Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!newsxfer.itd.umich.edu!gatech!ncar!noao!tucana.tuc.noao.edu!valdes References: <199407051645.MAA00598 at tetra.gsfc.nasa.gov> <9407061525.AA00470 at fits.cv.nrao.edu> From: valdes at tucana.tuc.noao.edu (Frank Valdes) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: Wed, 6 Jul 1994 17:32:09 GMT My personal preference is numbered keywords. All parts of the string should be preserved by any reader/writer and the order can be recovered by software interested in it that knows the root keyword. This is also an existing practice. However, the statement has been made that this is not possible for a certain dataset (which was a design decision that might have been different). I would suggest that the same keyword be used for each part and the index be part of the value field (with some variants following). Before giving the argument an example would be: LONGSTR = 1 'First part of this keyword' LONGSTR = 3 'Third part of this keyword' ANOTHER = 2 ' very long keyword' LONGSTR = 2 'Second part of this keyword' ANOTHER = 1 'Here is another' The argument for using the same keyword is that all parts of a long value are associated. Use of blank, COMMENT, HISTORY, etc. can become confused. The index allows the order to be reconstructed if some reader/writer does not preserve the original order. Finally existing readers will likely take one of four actions; just save all keywords, save the first with the same name, save the last with the same name, give a warning or error. Also the value will either be preserved completely or just the first part, whether it is the string or the index. The main problem is having two parts to the value field. A variant would be: 'Second part of this keyword\2' (or some other delimiter). In this case, unlike some other proposals, the reader doesn't necessarily have to scan the value. If the reader is prepared for this syntax it would only need to scan the value for ordering when a second instance of a keyword is found. A caveat is that I don't know without researching it whether there is any mandate in the standard against multiple occurances of the same keyword though I know that blank and HISTORY were specifically allowed. So in a sense there is a precedent. Frank Valdes IRAF Group/NOAO From fitsbits-request Wed Jul 6 19:38:49 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1601" "" " 6" "July" "1994" "23:25:15" "GMT" "Allyn Tennant" "tennant at alph.msfc.nasa.gov" "<2vfegr$96l at hammer.msfc.nasa.gov>" "41" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070623:25:15" "FITS continuation keywords" (number " " mark " Allyn Tennant Jul 6 41/1601 " thread-indent "\"Re: FITS continuation keywords\"\n") "<199407051645.MAA00598 at tetra.gsfc.nasa.gov>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA01846; Wed, 6 Jul 94 19:38:49 EDT Return-Path: Message-Id: <2vfegr$96l at hammer.msfc.nasa.gov> Organization: NASA/MSFC Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!news.msfc.nasa.gov!usenet References: <199407051645.MAA00598 at tetra.gsfc.nasa.gov> Reply-To: tennant at alph.msfc.nasa.gov From: tennant at alph.msfc.nasa.gov (Allyn Tennant) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 6 Jul 1994 23:25:15 GMT In article <199407051645.MAA00598 at tetra.gsfc.nasa.gov> William Pence writes: > (Do you prefer:) > > LONGSTR = 'This is a very long string keyword value that \' > 'is continued over three\' > 'keywords in the FITS header.' > > (or) > > LONGSTR = 'This is a very long string keyword value that \' > CONTINUE= 'is continued over three\' > CONTINUE= 'keywords in the FITS header.' > > Is there any preference as to which of these 2 conventions we use? I vote for neither. I object to the use of the backslash to denote a continuation line. This is for two main reasons: 1) For hundreds of years publishers have used a trailing - (hyphen) to denote a continuation line. Why create a new symbol to do the same job? 2) In UNIX, C and many other parsers, backslash is an escape character which means that the OS is to pass the following character without interpretation. Thus in C the backslash at the end of the line means don't treat the following linefeed as an end of line marker but pass it through. In the two examples above the backslash precedes a single quote character. Thus someone who knows C et al. should naturally read the line to mean that the single quote is part of the string. I believe that using backslash in the proposed manner will make it very difficult to use backslash in it's more normal meaning as an escape character in the future. In other words, using backslash as a continuation character only makes sense if it is followed by a linefeed character, and this condition does not arise in FITS. Allyn From fitsbits-request Wed Jul 6 23:35:33 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["2858" "" " 6" "July" "1994" "21:51:48" "-0400" "Lee E. Brotzman" "leb at clark.net" "<2vfn3k$k8o at explorer.clark.net>" "54" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070701:51:48" "FITS continuation keywords" (number " " mark " Lee E. Brotzman Jul 6 54/2858 " thread-indent "\"Re: FITS continuation keywords\"\n") ""] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA01943; Wed, 6 Jul 94 23:35:33 EDT Return-Path: Message-Id: <2vfn3k$k8o at explorer.clark.net> Organization: Clark Internet Services, Inc., Ellicott City, MD USA Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!gatech!udel!news2.sprintlink.net!news.sprintlink.net!news.clark.net!news.clark.net!not-for-mail References: From: leb at clark.net (Lee E. Brotzman) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 6 Jul 1994 21:51:48 -0400 mcs at goblin.caltech.edu (Martin Shepherd) writes: >In article Nikolai Piskunov writes: >:I feel that the idea of using \' combination may cause problems for >:FITS readers written in C. >No. Back slashes and quotes are not significant in C programs except >when they appear in the text of a C file at compilation time. Exactly right. Parsing for backslashes poses no difficulty at all for C programs. Shells, which are written in C, do it all the time. I prefer a backslash because it does not appear in normal text (which can not be said for the amersand, &, suggested by a previous poster). >:On the other hand second example in WP mail shows certain redundancy. >:So what if we keep the keyword CONTINUE and forget the back slash? >:In other words, the example above will look like: >: >: LONGSTR = 'This is a very long string keyword value that ' >: CONTINUE= 'is continued over three ' >: CONTINUE= 'keywords in the FITS header.' >This seems logical, but I can see the value of having some indication >in the string being continued that the current keyword is incomplete, >rather than having to look ahead at the next keyword. I definitely DO NOT want a CONTINUE keyword. In order to generalize the parser I would have to change the algorithm to use look ahead for all keywords. Yuch. That is a very significant coding change. If continuations are needed at all (see below), then the keyword being continued must indicate that there is more information to follow. This can be coded much more simply as an exceptional, as opposed to a general, rule. >IMHO it would be better to side-step the whole issue of continuation >lines in FITS headers, and place such strings in tables. This I agree most strongly with, but in a different way. My impression is that these keywords would hold "one-time" descriptive information for an entire image or table column. Putting it into the table would introduce a lot of redundancy. However, I tend to doubt that there is a need here that can only be met by breaking the standard, or otherwise introducing new conventions to write code around. I have been pretty successful using conventions in standard COMMENT keywords to do exactly the same thing (kind of a mini-heiarchical keyword scheme... works great for me at least). Unless I can see a really compelling argument for this convention, I most certainly would vote against it (if I'm still on the E-mail voting body, that is, I haven't heard from Don in quite a while). -- -- Lee E. Brotzman E-mail: leb at clark.net -- Advanced Data Solutions Phone : 301-776-0324 -- "Sometimes I think the surest sign that intelligent life exists elsewhere in -- the Universe is that none of it has tried to contact us." --- Calvin, 1990 From fitsbits-request Thu Jul 7 10:46:26 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1393" "" " 7" "July" "1994" "14:27:48" "GMT" "Robert Hill" "bhill at trifle.gsfc.nasa.gov" "<2vh3d4$sm7 at paperboy.gsfc.nasa.gov>" "33" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070714:27:48" "FITS continuation keywords" (number " " mark " Robert Hill Jul 7 33/1393 " thread-indent "\"Re: FITS continuation keywords\"\n") "<2vfn3k$k8o at explorer.clark.net>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA03641; Thu, 7 Jul 94 10:46:26 EDT Return-Path: Message-Id: <2vh3d4$sm7 at paperboy.gsfc.nasa.gov> Organization: NASA Goddard Space Flight Center -- InterNetNews site Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!swrinde!ihnp4.ucsd.edu!ames!newsfeed.gsfc.nasa.gov!bhill References: <2vfn3k$k8o at explorer.clark.net> From: bhill at trifle.gsfc.nasa.gov (Robert Hill) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 7 Jul 1994 14:27:48 GMT I don't like the idea of continuations, myself. If I had this need in my project, I think I would try to find some way to embed the whole structure in history or comment cards, with some extra code to break it out of there. Alternatively, it seems to me that a long string of mission-specific information must have some substructure, so that it could be parsed into separate parameters before being written to the header. Of the schemes proposed, however, I prefer the backslash continuation onto 'blank' comment cards as having the minimum impact on FITS readers outside the project. I really dislike the CONTINUE= keyword. I think what really rankles me is the equals sign, which implies that CONTINUE is somehow a parameter of the data set, like NAXIS or CRPIX1. Really, CONTINUE is another keyword of the HISTORY/COMMENT/blank class, and the equals sign is there only because FITS allows us to invent new keywords, but not new history/comment indicators. The invention of such an indicator specifically for continuing strings would require the standard to be amended, and nobody wants to get into that. So, if you must, backslash + blank-comment card. -- Robert.S.Hill at gsfc.nasa.gov Not speaking for any of these: Hughes STX Corp., Code 681, NASA/GSFC, Greenbelt, MD 20771 "There's nothing you can't prove if your outlook is only sufficiently limited." -- Lord Peter Wimsey From fitsbits-request Thu Jul 7 17:48:41 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["3693" "" " 7" "July" "1994" "17:30" "EDT" "BARRY M. SCHLESINGER" "bschlesinger at nssdca.gsfc.nasa.gov" "<7JUL199417305565 at nssdca.gsfc.nasa.gov>" "72" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070721:30:00" "FITS continuation keywords" (number " " mark " BARRY M. SCHLESIN Jul 7 72/3693 " thread-indent "\"Re: FITS continuation keywords\"\n") "<2vh3d4$sm7 at paperboy.gsfc.nasa.gov>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA04817; Thu, 7 Jul 94 17:48:41 EDT Return-Path: Message-Id: <7JUL199417305565 at nssdca.gsfc.nasa.gov> Organization: NASA - Goddard Space Flight Center Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!news.umbc.edu!haven.umd.edu!ames!newsfeed.gsfc.nasa.gov!nssdca.gsfc.nasa.gov!bschlesinger References: <2vfn3k$k8o at explorer.clark.net> <2vh3d4$sm7 at paperboy.gsfc.nasa.gov> From: bschlesinger at nssdca.gsfc.nasa.gov (BARRY M. SCHLESINGER) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 7 Jul 1994 17:30 EDT I'm commenting on a number of points from different posts, and VNEWS doesn't have an obvious way of keeping the attribution straight from multiple non-sequential posts. On the question of repeated keywords... The NOST standard does not specifically address the general issue of keyword repetition. The order and placement of most mandatory keywords is prescribed. Presence in the header of more than one HISTORY, COMMENT, or [blank] is specifically permitted. But there is no specification on reserved or user-defined keywords. The issue of repeating this kind of keyword was left to the User's Guide for discussion, rather than being prescribed in the standard. The relevant text in the User's Guide: For keywords that have no value and are used to transfer text, such as HISTORY and COMMENT, the same keyword may be used repeatedly to indicate that the remainder of the card image contains text or comments. However, repetition of a keyword with conflicting values may cause confusion. There is no standard interpretation in this case as to which value is to be retained as the true value of the keyword. Different FITS readers may interpret the header differently. Do not repeat a keyword on different card images if the values conflict. So multiple CONTINUE keywords would not be a violation of the standard. On distinguishing keywords without values.... Except for COMMENT, HISTORY, and [blank], the contents of columns 9 and 10 distinguish whether or not the keyword has a value. Except for those three keywords, the keyword has a value if and only if columns 9 and 10 contain '= '. Otherwise, columns 9-80 are comment and there is no value. The reason for the exceptions in the cases of COMMENT, HISTORY and [blank] is historical; the original FITS papers did not forbid '= ' in columns 9 and 10, and this freedom must be preserved, following the "once FITS, always FITS" rule. The keyword name tells the reader that there is no value. For user-defined keywords, though some way of distinguishing whether there is a value is needed. One point to consider is that some readers may treat multiple occurrences of keywords without values differently from multiple occurrences of keywords with values. On the order of keywords in a header... To say that the order of optional keywords in a header is irrelevant is perhaps too strong a statement. The order is not specified. Because the order is not specified, FITS readers are not required to preserve it. Some don't. This potential for card shuffling is the reason that it is unwise to rely on card image order in a FITS header. Finally, whatever convention is accepted should not break or even bend FITS readers. An example of how to construct such a convention is the NRAO use of hidden keywords behind the HISTORY keyword -- also used in the HIERARCH convention at ESO -- illustrated in Example 1 of the User's Guide. A reader that is familiar with the convention will find the "keywords" that start in the later columns, and be able to process them. However, a reader that is unfamiliar with the convention still has no problem; it simply treats the card images in question as containing comments, and the information can be easily obtained by the human reader by reading a listing of the header. We must keep the basic purpose of FITS in mind -- to allow the community access to data without the necessity of constructing specialized software for each data set read. If we are to achieve this goal, we must avoid creating conventions that will require patches to a significant fraction of the FITS readers in use. Barry Schlesinger NOST FITS Support Office From fitsbits-request Fri Jul 8 18:01:47 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1441" "" " 8" "July" "1994" "18:43:39" "GMT" "Paul Repacholi" "prep at yarrow.wt.uwa.edu.au" "" "32" "Re: TSORTKEY and continuation proposals" "^From:" nil nil "7" "1994070818:43:39" "TSORTKEY and continuation proposals" (number " " mark " Paul Repacholi Jul 8 32/1441 " thread-indent "\"Re: TSORTKEY and continuation proposals\"\n") ""] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA07324; Fri, 8 Jul 94 18:01:47 EDT Return-Path: Message-Id: Organization: Winthrop Technology Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!caen!malgudi.oar.net!news.pipeline.com!news.intercon.com!howland.reston.ans.net!agate!ihnp4.ucsd.edu!munnari.oz.au!news.uwa.edu.au!news!prep References: From: prep at yarrow.wt.uwa.edu.au (Paul Repacholi) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: TSORTKEY and continuation proposals Date: 08 Jul 1994 18:43:39 GMT In article Clive Page writes: ... > may not be processed properly by current systems. It is true that the > FITS Standard does not require the order of keywords to be preserved, so > any system of continuations is liable to fail, but in practice lots of > existing FITS files have a sequence of keywords containing COMMENTs that > only makes sense if they can be read in sequence; all the systems that I > know of seem to preserve their order in such cases. In the interest of the golden rule "once FITS, always FITS" it seems that preserving order should become a REQUIRMENT in FITS. 1) No existing files will be affected. 2) Software packages will have two less free variables to worry about. 3) Any multi-line form of extention will reqire that ordering and grouping be preserved, so this will be needed as part of Bill's proposal. I am still not happy with a syntax form that embeds packing info into a value. The problem is not that it is \ or & or -, but that it apeares inside the value. So is the \ part of the string? Inband signaling is very seldom a good idea! -- ~Paul +61 (09) 257-1001 prep at yarrow.wt.uwa.edu.au ( preferred ) 1 Crescent Rd, zrepachol at cc.curtin.edu.au Kalamunda, West Aust 6076 From fitsbits-request Fri Jul 8 23:33:00 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["3511" "" " 8" "July" "1994" "23:06:51" "-0400" "Lee E. Brotzman" "leb at clark.net" "<2vl48b$k7k at explorer.clark.net>" "76" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070903:06:51" "FITS continuation keywords" (number " " mark " Lee E. Brotzman Jul 8 76/3511 " thread-indent "\"Re: FITS continuation keywords\"\n") ""] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA07463; Fri, 8 Jul 94 23:33:00 EDT Return-Path: Message-Id: <2vl48b$k7k at explorer.clark.net> Organization: Clark Internet Services, Inc., Ellicott City, MD USA Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!sundog.tiac.net!news3.sprintlink.net!news.sprintlink.net!news.clark.net!news.clark.net!not-for-mail References: <199407051645.MAA00598 at tetra.gsfc.nasa.gov> From: leb at clark.net (Lee E. Brotzman) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 8 Jul 1994 23:06:51 -0400 prep at yarrow.wt.uwa.edu.au (Paul Repacholi) writes: >In article <199407051645.MAA00598 at tetra.gsfc.nasa.gov> William Pence writes: >> following keyword(s) be filled with blanks. FITS readers that do not >> understand this continuation convention would just treat the >> continuation lines as comment keywords (since the keyword name is >> blank). For example: >> >> LONGSTR = 'This is a very long string keyword value that \' >> 'is continued over three\' >> 'keywords in the FITS header.' >This means interpreting the value of 'LONGSTR' as part of inputting >it. I would rather see >LONGSTR = ".... that'\ >LONGSTR = 'is cont...'\ >LONGSTR = 'to a final un-escaped line' >type syntax. >Thus each line is a 'LONGSTR' to old reader, and the semantics are cleaned >up, and could extend to other keywords as well. BINGO! This is it. Except the syntax should be: LONGSTR = 'This is the beginning of the string...' LONGSTR = 'followed by even more of this string...' LONGSTR = 'summed up by this string' When one defines a new keyword, one must also define the keyword's value, either integer, floating point, string, etc. Therefore, to have "long strings" one simply defines the keyword syntax as a string and that the keyword can appear more than once, like COMMENT. There is no specific prohibition against repeated occurences of a keyword, so as far as I can tell, this is a legal FITS solution to the "problem" (which, of course, I still do not believe really exists). It may even be better to define the keyword without the '=', thereby creating a new form of COMMENT/HISTORY. I'll leave that to Barry to see if it is legal (frankly, his message talking about 'no value' for keywords without '=' was a little confusing to me, just what does 'no value' mean?). As a FITS programmer, how would I deal with this? Basically, my code ignores any keyword it is not specifically looking for; as is normal for FITS. So to begin with, the 'LONGSTR' keywords would have no effect whatsoever. Continuation characters and blank keywords would, however, since my code scans for blank keywords followed by text and interprets them as comments. This has the effect of all the blank keywords *following* the 'LONGSTR' keyword being put into my comment structure, thereby rendering it incomplete and invalid. If I actually *want* to scan for the 'LONGSTR' keywords, and if they are repeating keywords, all I have to do is add something like this in my parser: Comment* LONGSTR; enum keywords { ... , longstr, ... } struct keyentry { ... , { "LONGSTR ", longstr}, ... ... switch (keytype) { ... longstr: ParseComment(value, LONGSTR); A recompile, a relink, and voila! The strings are read in and stored in total. This does not qualify, in my mind, as writing new code, since I would have to do the same thing for *any* new keyword I want to scan for. Maybe it isn't the prettiest solution, but it has worked quite well for me. So, to sum up. If indeed such a need exists (which I doubt) the best legal solution is repeating keywords based on COMMENT syntax. There. We can all go home now. No wait, I'm already there! Oh well, you know what I mean. -- -- Lee E. Brotzman E-mail: leb at clark.net -- Advanced Data Solutions Phone : 301-776-0324 -- "Sometimes I think the surest sign that intelligent life exists elsewhere in -- the Universe is that none of it has tried to contact us." --- Calvin, 1990 From fitsbits-request Sat Jul 9 03:49:46 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["3361" "Sat" " 9" "July" "1994" "05:47:26" "GMT" "Doug Tody" "tody at lepus.tuc.noao.edu" "<1994Jul9.054726.18859 at noao.edu>" "61" "Re: TSORTKEY and continuation proposals" "^From:" nil nil "7" "1994070905:47:26" "TSORTKEY and continuation proposals" (number " " mark " Doug Tody Jul 9 61/3361 " thread-indent "\"Re: TSORTKEY and continuation proposals\"\n") ""] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA07576; Sat, 9 Jul 94 03:49:46 EDT Return-Path: Message-Id: <1994Jul9.054726.18859 at noao.edu> Organization: National Optical Astronomy Observatories Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!ncar!noao!lepus.tuc.noao.edu!tody References: Reply-To: tody at noao.edu From: tody at lepus.tuc.noao.edu (Doug Tody) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: TSORTKEY and continuation proposals Date: Sat, 9 Jul 1994 05:47:26 GMT In article , prep at yarrow.wt.uwa.edu.au (Paul Repacholi) writes: > > In the interest of the golden rule "once FITS, always FITS" it seems > that preserving order should become a REQUIRMENT in FITS. > > 1) No existing files will be affected. > > 2) Software packages will have two less free variables to worry about. > > 3) Any multi-line form of extention will reqire that ordering and > grouping be preserved, so this will be needed as part of Bill's > proposal. This is easy to say but it is not that simple. The possibility of reordering header keywords occurs when the FITS header is edited. Often the header contains related groups of keywords. For example, the header may contain a world coordinate system (WCS) specification stored in a number of related keywords (any number of other examples are possible). If the WCS associated with the image changes it may be that the number of keywords required to describe the WCS changes. One must delete all the old WCS keywords and insert the new WCS keywords. It is possible to insert or append the new group of keywords as a sequential block, but there is no guarantee that all the original WCS keywords will be consecutive. Hence one deletes all the old WCS keywords wherever they may appear in the header, and inserts or appends the new keywords somewhere else, the logical place being near the beginning of the header since these are standard keywords. This may mean that the order of the keywords changes. Preserving the original order could be difficult or impossible. The situation can get even worse. It may be that the header is ambiguous, containing multiple copies of the same keyword, e.g. two WCS specifications. Which is correct? If the WCS is modified do we delete both sets? If one thinks of the FITS header as time ordered the last entry would take precedence, but the defacto FITS practice is to order the header with the "standard" keywords near the top, followed by an arbitrary number of application specific keywords, comments, history etc. in the latter part of the header. Hence to preserve the logical order of the FITS header one wants to consolidate the system keywords near the top. If the input header is poorly structured it may need to be reordered when written back out by the software which edits the header. Another example would be combining two images and writing a single new output image. How do we combine the two headers without reordering at least some of the keywords, and probably removing some duplicate keywords? Obviously one wants to preserve the original order of the header keywords where possible, but the point is that this is not always possible or desirable. Note that this problem only arises when using FITS as a runtime format where headers are subject to dynamic updates. Since FITS is intended as a static archival and exchange format issues such as dynamic updates or reordering keywords are not officially an issue. Nonetheless it is an issue for real data systems, since we all ingest FITS files, generate new images, and write out new FITS files, and one wants to preserve the information in the input headers so far as possible. -- Doug Tody National Optical Astronomy Observatories IRAF project tody at noao.edu P.O. Box 26732, Tucson, Arizona, 85726 iraf at noao.edu From fitsbits-request Sun Jul 10 11:14:44 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["3029" "Sun" "10" "July" "1994" "17:20:35" "+0200" "Lucio Chiappetti" "lucio at ifctr.mi.cnr.it" "" "64" "Re: TSORTKEY and continuation proposals" "^From:" nil nil "7" "1994071015:20:35" "TSORTKEY and continuation proposals" (number " " mark " Lucio Chiappetti Jul 10 64/3029 " thread-indent "\"Re: TSORTKEY and continuation proposals\"\n") "<1994Jul9.054726.18859 at noao.edu>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA11369; Sun, 10 Jul 94 11:14:44 EDT Return-Path: Organization: Istituto di Fisica Cosmica e Tecnologie Relative In-Reply-To: <1994Jul9.054726.18859 at noao.edu> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII From: Lucio Chiappetti Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: TSORTKEY and continuation proposals Date: Sun, 10 Jul 1994 17:20:35 +0200 (MET DST) On Sat, 9 Jul 1994, Doug Tody wrote: > In article , prep at yarrow.wt.uwa.edu.au (Paul Repacholi) writes: > > > > In the interest of the golden rule "once FITS, always FITS" it seems > > that preserving order should become a REQUIRMENT in FITS. > > > > This is easy to say but it is not that simple. The possibility of > reordering header keywords occurs when the FITS header is edited. Often the ... > keywords and insert the new WCS keywords. It is possible to insert or > append the new group of keywords as a sequential block, but there is no > guarantee that all the original WCS keywords will be consecutive. Hence one > deletes all the old WCS keywords wherever they may appear in the header, and > inserts or appends the new keywords somewhere else, the logical place being ... The point is that a FITS header is somewhat an awkward structure to be handled in memory, and on disk, since it is an almost fixed (once written) number of card images in a fixed number of 2880-blocks. Extending the header will require writing a new file (a reason not to use FITS as working format, see below). Hence the point is : either read all the header in memory, and REPLACE the values of changed kewyords, or EXTEND the header allocating more memory and finally FLUSH the header to disk or read "wanted" keywords by name from the input file and write them to the output file eventually with a modified value. What to do then with "unwanted" or "unexpected" keywords ? Ignore them ? or do a further pass and write out those not already written (which implies they are flagged) ? > The situation can get even worse. It may be that the header is ambiguous, > containing multiple copies of the same keyword, e.g. two WCS > specifications. Which is correct? If the WCS is modified do we delete both Is this a "legal" FITS file ? I'd assumed that, besides COMMENTS and HISTORY, duplication of keywords was not allowed or at least discouraged. > Another example would be combining two images and writing a single new > output image. How do we combine the two headers without reordering at least > some of the keywords, and probably removing some duplicate keywords? > Yes indeed > Note that this problem only arises when using FITS as a runtime format where > headers are subject to dynamic updates. Since FITS is intended as a static > archival and exchange format issues such as dynamic updates or reordering ^^^^^^^^^^^^^^^^^^^^^ YES INDEED ---------------------------------------------------------------------------- Lucio Chiappetti - IFCTR/CNR | Ma te' vugl' da' quost avis a ti' Orsign via Bassini 15 - I-20133 Milano | Buttet rabios intant te se' pisnign Internet: LUCIO at IFCTR.MI.CNR.IT | Decnet: IFCTR::LUCIO | (Rabisch, II 46, 119-120) ---------------------------------------------------------------------------- From fitsbits-request Sun Jul 10 16:50:38 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["424" "" "10" "July" "1994" "20:43:50" "GMT" "Andrew T. Chambers" "atc at waikato.ac.nz" "<2vpmi6$1nbb at thebes.waikato.ac.nz>" "16" "Visual Basic VBX and DLL file for FITS?" "^From:" nil nil "7" "1994071020:43:50" "Visual Basic VBX and DLL file for FITS?" (number " " mark " Andrew T. Chamber Jul 10 16/424 " thread-indent "\"Visual Basic VBX and DLL file for FITS?\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA11535; Sun, 10 Jul 94 16:50:38 EDT Return-Path: Message-Id: <2vpmi6$1nbb at thebes.waikato.ac.nz> Organization: The University of Waikato Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!newsxfer.itd.umich.edu!gumby!wupost!waikato!atc.ijk.waikato.ac.nz!atc From: Andrew T. Chambers Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Visual Basic VBX and DLL file for FITS? Date: 10 Jul 1994 20:43:50 GMT Does anyone know of an VBX or DLL files for Visual BASIC that are designed to help one implement a FITS viewer or image manipulation program? I woul like to design such a program but its practically impossible without extra VBX or DLL's specially written for the job. I have looked at a list of commercial packages but they don't support FITS. Any help appreciated Andrew Chambers The University of Waikato New Zealand From fitsbits-request Mon Jul 11 02:36:54 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["651" "" "11" "July" "1994" "06:26:04" "GMT" "Bruce Cogan" "cogan at merlin.anu.edu.au" "<2vqolt$me2 at manuel.anu.edu.au>" "17" "Handling many small images" "^From:" nil nil "7" "1994071106:26:04" "Handling many small images" (number " " mark " Bruce Cogan Jul 11 17/651 " thread-indent "\"Handling many small images\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA11836; Mon, 11 Jul 94 02:36:54 EDT Return-Path: Message-Id: <2vqolt$me2 at manuel.anu.edu.au> Organization: Mt. Stromlo and Siding Spring Observatories Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!swrinde!ihnp4.ucsd.edu!munnari.oz.au!newshost.anu.edu.au!merlin.anu.edu.au!cogan From: cogan at merlin.anu.edu.au (Bruce Cogan) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Handling many small images Date: 11 Jul 1994 06:26:04 GMT One of our astronomers has a project in which he will be generating a large number (100's) of small (~10x10 pixel) images. He would like to archive these in FITS format, but dosn't want the overhead of a separate file and its header for each image. I would appreciate suggestions on the best way of storing these in FITS files, such that they would be readily accessible by standard reduction packages. -- Bruce Cogan cogan at mso.anu.edu.au Mt Stromlo Observatory Tel. +61 6 249 0232 Australian National University Fax +61 6 249 0233 Private Bag Weston Creek P.O. , ACT 2611, Australia From fitsbits-request Mon Jul 11 06:22:58 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["2355" "Mon" "11" "July" "1994" "10:03:10" "+0000" "Clive Page" "cgp at star.le.ac.uk" "" "44" "FITS continuation problem" "^From:" nil nil "7" "1994071110:03:10" "FITS continuation problem" (number " " mark " Clive Page Jul 11 44/2355 " thread-indent "\"FITS continuation problem\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA13126; Mon, 11 Jul 94 06:22:58 EDT Return-Path: Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 2354 From: Clive Page Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: FITS continuation problem Date: Mon, 11 Jul 1994 10:03:10 +0000 (GMT) I take the point that in-band signalling is undesirable - as characters like \ & - might sometimes occur as normal contents of the value string. But what about putting the continuation marker and count in the comment field? Something like this: LONGSTRG= 'This is a very long string value which will need to' / &1 = ' be continued because it clearly will not fit in its' / &2 = ' entirety on a single line' The presence of an ampersand (or whatever character turns out to be most acceptible) as the first non-blank character of a comment, followed by a line counter, will signal to suitable FITS readers that a set of lines that form a single long character-string value is coming its way. This should not upset existing FITS readers (they will just lost the remainder of the long string). Of course it doesn't solve the problem, as pointed out by Doug Tody, that programs that read and then write information in FITS files may be forced to re-order keywords in some circumstances. One way of getting around this might be to use an even more elaborate comment field, for example: LONGSTRG= 'This is a very long string value which will need to' / &LONGSTRG1 = ' be continued because it clearly will not fit in its' /&LONGSTRG2 = ' entirety on a single line' /&LONGSTRG3 With this format a clever FITS reader could reconstruct the original long string even if somewhere in the processing chain the order of the lines had been randomly scrambled. Unfortunately it leaves even less space for the value string, so even more of them will need to be continued on to two or more lines. This is, of course, not a fully developed proposal, just an idea that would seem to serve the purpose without violating the FITS Standard. No doubt it could be improved to simplifiy FITS readers, e.g. by flagging the last line of the continuation sequence in some way. ------------------------------------------------------------------------- Clive Page, Internet: cgp at star.le.ac.uk Dept of Physics & Astronomy, University of Leicester, Until Apr 95: Phone +44 533 523551 Leicester, LE1 7RH, Fax +44 533 550182 UK. After Aug 94: Phone +44 116 252 3551 Fax: +44 116 255 0182 From fitsbits-request Mon Jul 11 12:36:24 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1904" "Mon" "11" "July" "1994" "12:36:11" "-0400" "William Pence" "pence at tetra.gsfc.nasa.gov" "<199407111636.MAA26424 at tetra.gsfc.nasa.gov>" "41" "Re: Handling many small images" "^From:" nil nil "7" "1994071116:36:11" "Handling many small images" (number " " mark " William Pence Jul 11 41/1904 " thread-indent "\"Re: Handling many small images\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA13641; Mon, 11 Jul 94 12:36:24 EDT Return-Path: Message-Id: <199407111636.MAA26424 at tetra.gsfc.nasa.gov> From: William Pence Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Cc: pence at tetra.gsfc.nasa.gov Subject: Re: Handling many small images Date: Mon, 11 Jul 1994 12:36:11 -0400 Bruce Cogan (cogan at mso.anu.edu.au) wrote: > One of our astronomers has a project in which he will be generating > a large number (100's) of small (~10x10 pixel) images. He would like > to archive these in FITS format, but dosn't want the overhead of > a separate file and its header for each image. > > I would appreciate suggestions on the best way of storing these in > FITS files, such that they would be readily accessible by standard > reduction packages. > The most accessible way to store the images is to just put each image into a separate FITS file with a 10x10 primary array. Every reduction package will be able to read them, and the total size of even a 1000 files, at 5760 bytes each (the minimum size for a FITS header + data unit), is less than 6 MBytes which is not much to worry about these days on most systems. But this is not very efficient in terms of disk space and in I/O time if a program has to open and read many different files. The most efficient way to store the images would be to store tham all in a single FITS binary table extension. (Or the logically related sets of images could be grouped together in a small number of binary tables). Each row of the table could hold one image. The column that stores the images would need to be defined as a vector (e.g., TFORMn = '100I' to hold a 10x10 array of 16 bit integers), and the TDIMn keyword convention would need to be used to specify the dimentionality of the array (e.g., TDIMn = '(10,10) '). You could define as many other columns in the table as desired to store other descriptive parameters about the image, such as the coordinates, or date, or filter. We use this format a lot for our X-ray data files, but many reduction packages cannot readily read these images without preprocessing the FITS files to convert them into a standard primary array image or IMAGE extension. Bill Pence NASA/GSFC From fitsbits-request Mon Jul 11 16:39:13 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["654" "" "11" "July" "1994" "18:44:16" "GMT" "Ron Watkins" "ron at argus.lpl.arizona.edu" "<2vs3u0$a7e at news.CCIT.Arizona.EDU>" "12" "Re: Handling many small images" "^From:" nil nil "7" "1994071118:44:16" "Handling many small images" (number " " mark " Ron Watkins Jul 11 12/654 " thread-indent "\"Re: Handling many small images\"\n") "<199407111636.MAA26424 at tetra.gsfc.nasa.gov>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA13993; Mon, 11 Jul 94 16:39:13 EDT Return-Path: Message-Id: <2vs3u0$a7e at news.CCIT.Arizona.EDU> Organization: Lunar and Planetary Lab, U of AZ Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!howland.reston.ans.net!europa.eng.gtefsd.com!MathWorks.Com!yeshua.marcam.com!charnel.ecst.csuchico.edu!nic-nac.CSU.net!news.Cerritos.edu!news.Arizona.EDU!argus.lpl.Arizona.EDU!ron References: <199407111636.MAA26424 at tetra.gsfc.nasa.gov> From: ron at argus.lpl.Arizona.EDU (Ron Watkins) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: Handling many small images Date: 11 Jul 1994 18:44:16 GMT I would have thought the most efficient way would be to use a 3 dimensional primary array. This has the advantage of having a single header and there wont be any padding of the data space for EVERY array, rather only one padding at the end of all the arrays. If the arrays are all the same size, then a 3-D array might be the best storage mechanism. Ron W. -- Ron Watkins [ron at argus.lpl.arizona.edu] / /~~~~) / 931 Gould-Simpson / /____/ / University of Arizona / / / Tucson AZ. 85721 -- (602) 621-8606 (____ unar & / lanetary (____ ab. From fitsbits-request Mon Jul 11 20:05:21 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1190" "Mon" "11" "July" "1994" "17:39:22" "GMT" "Doug Tody" "tody at lepus.tuc.noao.edu" "<1994Jul11.173922.10619 at noao.edu>" "26" "Re: Handling many small images" "^From:" nil nil "7" "1994071117:39:22" "Handling many small images" (number " " mark " Doug Tody Jul 11 26/1190 " thread-indent "\"Re: Handling many small images\"\n") "<2vqolt$me2 at manuel.anu.edu.au>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA14261; Mon, 11 Jul 94 20:05:21 EDT Return-Path: Message-Id: <1994Jul11.173922.10619 at noao.edu> Organization: National Optical Astronomy Observatories Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!ncar!noao!lepus.tuc.noao.edu!tody References: <2vqolt$me2 at manuel.anu.edu.au> Reply-To: tody at noao.edu From: tody at lepus.tuc.noao.edu (Doug Tody) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: Handling many small images Date: Mon, 11 Jul 1994 17:39:22 GMT In article <2vqolt$me2 at manuel.anu.edu.au>, cogan at merlin.anu.edu.au (Bruce Cogan) writes: > One of our astronomers has a project in which he will be generating > a large number (100's) of small (~10x10 pixel) images. He would like > to archive these in FITS format, but dosn't want the overhead of > a separate file and its header for each image. > > I would appreciate suggestions on the best way of storing these in > FITS files, such that they would be readily accessible by standard > reduction packages. Hi Bruce, My recommendation would be to keep it simple. A few hundred images is not that many - the separate file approach should not be ruled out even if it is all overhead for such a small image. If greater efficiency is desired I would consider either using the IMAGE extension (if each image requires a separate header) or storing the small images as bands of a 3D image (if all can share the same header). The most general and efficient approach is to use a table, but this is overkill for such a simple dataset. - Doug -- Doug Tody National Optical Astronomy Observatories IRAF project tody at noao.edu P.O. Box 26732, Tucson, Arizona, 85726 iraf at noao.edu From fitsbits-request Mon Jul 11 22:07:44 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["868" "" "11" "July" "1994" "19:56:22" "GMT" "Timothy Kimball" "kimball at stsci.edu" "<2vs856$7if at marvel.stsci.edu>" "18" "Re: Handling many small images" "^From:" nil nil "7" "1994071119:56:22" "Handling many small images" (number " " mark " Timothy Kimball Jul 11 18/868 " thread-indent "\"Re: Handling many small images\"\n") "<2vs3u0$a7e at news.CCIT.Arizona.EDU>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA14319; Mon, 11 Jul 94 22:07:44 EDT Return-Path: Message-Id: <2vs856$7if at marvel.stsci.edu> Organization: Space Telescope Science Institute Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!caen!sol.ctr.columbia.edu!newsxfer.itd.umich.edu!gatech!swrinde!ihnp4.ucsd.edu!ames!ncar!noao!stsci!kimball References: <199407111636.MAA26424 at tetra.gsfc.nasa.gov> <2vs3u0$a7e at news.CCIT.Arizona.EDU> From: kimball at stsci.edu (Timothy Kimball) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: Handling many small images Date: 11 Jul 1994 19:56:22 GMT In article <2vs3u0$a7e at news.CCIT.Arizona.EDU>, ron at argus.lpl.Arizona.EDU (Ron Watkins) writes: |> I would have thought the most efficient way would be to use a 3 dimensional |> primary array. This has the advantage of having a single header and there |> wont be any padding of the data space for EVERY array, rather only one padding |> at the end of all the arrays. If the arrays are all the same size, then a |> 3-D array might be the best storage mechanism. |> Ron W. Yes, except for the keyword values that will be different for each image. You would need to handle this with: 1) a whole lot of individual keywords in the common header, or 2) a table extension listing keyword values for each plane of the primary array. So it's not as strightforward as just putting another axis in the image, unless the keyword values are common to all the images. -- tdk From fitsbits-request Tue Jul 12 16:17:23 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["6867" "Tue" "12" "July" "1994" "16:17:16" "-0400" "William Pence" "pence at tetra.gsfc.nasa.gov" "<199407122017.QAA16849 at tetra.gsfc.nasa.gov>" "147" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994071220:17:16" "FITS continuation keywords" (number " " mark " William Pence Jul 12 147/6867 " thread-indent "\"Re: FITS continuation keywords\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA16383; Tue, 12 Jul 94 16:17:23 EDT Return-Path: Message-Id: <199407122017.QAA16849 at tetra.gsfc.nasa.gov> From: William Pence Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Cc: pence at tetra.gsfc.nasa.gov Subject: Re: FITS continuation keywords Date: Tue, 12 Jul 1994 16:17:16 -0400 We've been following the discussion of the continuation proposal with interest and we appreciate the comments. Many variations on our proposed convention have been suggested, (both recently, and during the prior discusion of this issue in May 1993) but it appears to us that there is still no consensus that any of them is significantly better than our original proposal, So, at this point we are inclined to stick with our simple convention using blank continuation lines (although we could be persuaded to use the CONTINUE= alternative; see point 3 below). To be specific, this convention for encoding long string values is: a. Insert a backslash character (\) immediately before the closing quote character in the string value that is to be continued. b. Continue the string value, enclosed in single quotes, on the next keyword in the header c. The continuation keyword must contain ASCII blanks in columns 1-10 (i.e., the keyword name must be blank and there must be no equal sign in column 9). FITS readers that do not understand this continuation convention will simply treat the continuation lines as COMMENT keywords. A somewhat frivolous example of this usage is: LONGSTR = 'This is a very long string keyword value that \' 'is continued over three\' ' keywords in the FITS header.' or to give a more realistic example of the 'array descriptor' syntax to be used in the XTE FITS files: TDDES2 = 'E(X1L|X1R|X2L|X2R|X3L|X3R) & C("TDDMAC1(0:14)") & \' 'T([" at TIME",1024,0.00390625])' We don't wish to needlessly prolong the debate on this, but would like to comment on a few issues that have been raised: 1. Is the order of keywords in a FITS header significant? The only statement made in the NOST FITS Standard about keyword order is: "Except where specifically stated otherwise in this standard, keywords may appear in any order." and the only places that the order is specifically stated is for the required keywords at the beginning of each Header-Data Unit. We interpret this to mean that the creators of a FITS file are free to decide on the most appropriate order for the keywords, but it does not mean, in our view, that readers may come along later and arbitrarily rearrange the order of the keywords without risking the loss of some information. Clearly, if comment keywords have been inserted in the header at strategic places to explain the meaning of the preceding or following keywords, or if history keywords are inserted to document the order of the data processing operations, then rearranging the header keywords will lead to loss of information. Also, keywords may be put in a specific order to make them easier for humans to understand, or to make it more efficient for software to read them. So we believe that our convention that requires that the order of the continuation lines be preserved is not unprecedented nor violates the FITS Standard. 2. On the use of backslash (\) as a continuation character. We prefer to use the backslash because it is a rarely used ASCII character and is thus unlikely to be confused with a literal backslash within the string value. One can still write string keyword values that end in a backslash, and these will not be confused with our continuation convention as long as one of the 3 conditions listed above is violated (e.g., insert a space character between the backslash and the closing quote character, or make sure that the following keyword does not have a blank name). Some people objected to having any sort of continuation character at the end of the line, but we think it is important to have some sort of indicator to software that it should check the following keyword to see if it is a continuation. Otherwise the software would have to always check the next keyword every time it reads a string valued keyword. As was stated by several other people, the fact that the backslash has a different meaning in other contexts (e.g., C source code and Unix shell scripts) we believe is not a real source of confusion here. Some people suggested putting the continuation character outside the value field in the comment area at the end of the keyword. But this would require imposing new conventions on the format of the comment field in keywords that we would rather avoid. Also, some FITS readers simply ignore the comment field entirely, so putting significant information there could cause difficulties for them. If the continuation character is put in the value field itself then every FITS reader will be able to get at it easily. 3. Should the continuation keywords have a blank name, or some other name. This in our minds is the biggest open issue about this proposal. The blank keyword names will simply be treated as comments by FITS readers that don't understand this convention which we felt was advantagous (i.e. is the least harmful way to deal with an unknown convention). But if certain data systems strip comments from the header and segregate them from the other keyword=value keywords, then this is not so desirable. This is why we suggested, as an alternative, using a special keyword name for the continuation keywords (such as CONTINUE=). While a number of people did to prefer this alternative, the votes were not overwhelming in favor. IF ANYONE ELSE HAS A STRONG PREFERENCE BETWEEN THESE 2 OPTIONS, PLEASE LET US KNOW. We, on the other hand, do not like the idea of repeating the original keyword name on all the continuation cards. This could cause unpredicable results for FITS readers that do not understand this convention (would they return the value of the first keyword, or of the last?), and could be confusing if a FITS reader tried to delete the keyword, but didn't delete all the continuations. Under our convention, if a FITS reader just deleted the first keyword of a continuation sequence then the header would just be left with some orphaned continuation lines that are interpreted as comments and do no harm. 4. Why not use a more complicated 'keyword referencing' scheme to point to the next keyword in the continued string? Some of the alternative proposals are completely independent of the keyword order and still allow unlimited continuation lines. We had suggested such a scheme last year: KEYNAMEA= 'This is a very long keyword string value which \KEYWORDB\' KEYWORDC= ' of the FITS header.' KEYWORDB= 'continues over several lines\KEYWORDC\' Some other variations have been offered in the past week. But no one we've talked to has much interest in implementing such a complicated scheme, and neither do we. Our convention is much simplier and still serves our needs, so why introduce an even more complicated convention into FITS that very few people are interested in using? Regards, Bill Pence Arnold Rots From fitsbits-request Wed Jul 13 03:05:38 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["2612" "Wed" "13" "July" "1994" "04:53:49" "GMT" "Doug Tody" "tody at lepus.tuc.noao.edu" "<1994Jul13.045349.5076 at noao.edu>" "48" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994071304:53:49" "FITS continuation keywords" (number " " mark " Doug Tody Jul 13 48/2612 " thread-indent "\"Re: FITS continuation keywords\"\n") "<199407122017.QAA16849 at tetra.gsfc.nasa.gov>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA16920; Wed, 13 Jul 94 03:05:38 EDT Return-Path: Message-Id: <1994Jul13.045349.5076 at noao.edu> Organization: National Optical Astronomy Observatories Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!inxs.concert.net!taco.cc.ncsu.edu!gatech!ncar!noao!lepus.tuc.noao.edu!tody References: <199407122017.QAA16849 at tetra.gsfc.nasa.gov> Reply-To: tody at noao.edu From: tody at lepus.tuc.noao.edu (Doug Tody) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: Wed, 13 Jul 1994 04:53:49 GMT In article <199407122017.QAA16849 at tetra.gsfc.nasa.gov>, William Pence writes: > 4. Why not use a more complicated 'keyword referencing' scheme to point > to the next keyword in the continued string? > > Some of the alternative proposals are completely independent of the > keyword order and still allow unlimited continuation lines. We had > suggested such a scheme last year: > > KEYNAMEA= 'This is a very long keyword string value which \KEYWORDB\' > KEYWORDC= ' of the FITS header.' > KEYWORDB= 'continues over several lines\KEYWORDC\' > > Some other variations have been offered in the past week. But no one > we've talked to has much interest in implementing such a complicated > scheme, and neither do we. Our convention is much simplier and still > serves our needs, so why introduce an even more complicated convention > into FITS that very few people are interested in using? Bill - The above is more or less the convention used in IRAF, except that we use numbers instead of A B C. It works fine and is independent of the keyword order. The only disadvantage is that it is hard to squeeze both the root keyword name and sequence number into an 8 character keyword. But we don't find we need to do this very often (there are only a few cases in all of IRAF), so it is not a big problem. Note that in IRAF, only certain applications are expected to understand this convention; generic FITS programs see only what appear to be separate keywords. Hence if we want to delete a "multiline keyword" we do something like delete keywords "wsv*" to delete all the wsvNNNN keywords with a generic image or FITS tool. The same operation applied to an image using your convention will delete only the initial header card, leaving ambiguous, nameless continuation cards in the header. Ditto for any other operation such as copying or printing a keyword. To properly deal with your headers, all general FITS tools would have to understand your special convention. This is not the case with the keyword based convention we use. You are free to use any convention you choose for your data so long as it is legal FITS - so long as you don't expect everyone else's FITS code to understand it. Anyhow, the point I wanted to make is that we don't agree that your convention is "much simpler", or that the numbered continuation approach is "something more complicated which very few people are interested in using". - Doug -- Doug Tody National Optical Astronomy Observatories IRAF project tody at noao.edu P.O. Box 26732, Tucson, Arizona, 85726 iraf at noao.edu From fitsbits-request Wed Jul 13 04:50:28 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["887" "Wed" "13" "July" "1994" "07:02:48" "GMT" "Nick Szabo" "szabo at netcom.com" "" "21" "Customized data compression needs" "^From:" nil nil "7" "1994071307:02:48" "Customized data compression needs" (number " " mark " Nick Szabo Jul 13 21/887 " thread-indent "\"Customized data compression needs\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA16976; Wed, 13 Jul 94 04:50:28 EDT Return-Path: Message-Id: Organization: NETCOM On-line Communication Services (408 261-4700 guest) Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!newsxfer.itd.umich.edu!zip.eecs.umich.edu!yeshua.marcam.com!charnel.ecst.csuchico.edu!csusac!csus.edu!netcom.com!szabo From: szabo at netcom.com (Nick Szabo) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Customized data compression needs Date: Wed, 13 Jul 1994 07:02:48 GMT I'm embarking on a project, one of the applications of which is developing data compression algorithmns optimized for specific kinds of data. By taking advantage of domain specific pattern knowledge (eg redundancy in the genetic code for DNA, patterns in astronomical phenomenon, etc.) customized compression algorithms can in some cases provide a substantial improvement over general-purpose algorithms like Lempel-Ziv. I'd like to find out what data formats in the scientific community would most benefit from customized compression. Specifically, * What formats account for the largest volume of data transmitted over networks? * What kinds of data have the greatest amount of redundancy (especially domain-specific redundancy that wouldn't be caught by a normal compression algorithm)???? much thanks, -- N1ck Szab0 szabo at netcom.com Big Brother is parsing you. From fitsbits-request Wed Jul 13 07:14:11 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["2014" "Wed" "13" "July" "1994" "07:14:03" "-0400" "Arnold Rots" "arots at xebec.gsfc.nasa.gov" "<199407131114.HAA09807 at xebec.gsfc.nasa.gov>" "40" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994071311:14:03" "FITS continuation keywords" (number " " mark " Arnold Rots Jul 13 40/2014 " thread-indent "\"Re: FITS continuation keywords\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA18221; Wed, 13 Jul 94 07:14:11 EDT Return-Path: Message-Id: <199407131114.HAA09807 at xebec.gsfc.nasa.gov> From: Arnold Rots Sender: fitsbits-request at fits.CV.NRAO.EDU To: tody at noao.edu Cc: fitsbits at NRAO.EDU Subject: Re: FITS continuation keywords Date: Wed, 13 Jul 1994 07:14:03 -0400 Doug Tody wrote: > In article <199407122017.QAA16849 at tetra.gsfc.nasa.gov>, William Pence writes: > > 4. Why not use a more complicated 'keyword referencing' scheme to point > > to the next keyword in the continued string? > > > > Some of the alternative proposals are completely independent of the > > keyword order and still allow unlimited continuation lines. We had > > suggested such a scheme last year: > > > > KEYNAMEA= 'This is a very long keyword string value which \KEYWORDB\' > > KEYWORDC= ' of the FITS header.' > > KEYWORDB= 'continues over several lines\KEYWORDC\' > > > > Some other variations have been offered in the past week. But no one > > we've talked to has much interest in implementing such a complicated > > scheme, and neither do we. Our convention is much simplier and still > > serves our needs, so why introduce an even more complicated convention > > into FITS that very few people are interested in using? > Bill - The above is more or less the convention used in IRAF, except that > we use numbers instead of A B C. It works fine and is independent of the > keyword order. The only disadvantage is that it is hard to squeeze both > the root keyword name and sequence number into an 8 character keyword. > But we don't find we need to do this very often (there are only a few cases > in all of IRAF), so it is not a big problem. The big problem with using numbers (and, hence, the general applicability of your scheme) is, as pointed out in an earlier message, that it does not work for column-related keywords - or any other keyword that *has* to end in a digit. This, by the way, is a situation where one really gets into a crunch: a mandatory/customary "T", three digits for the column number, at least one character for a sequence indicator - and there is very little left to indicate what the keyword is really about, especially if it follows some kind of convention that prescribes yet another mandatory character. - Arnold Rots From fitsbits-request Wed Jul 13 09:46:41 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1604" "Wed" "13" "July" "1994" "13:14:01" "GMT" "Doug Tody" "tody at lepus.tuc.noao.edu" "<1994Jul13.131401.18344 at noao.edu>" "30" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994071313:14:01" "FITS continuation keywords" (number " " mark " Doug Tody Jul 13 30/1604 " thread-indent "\"Re: FITS continuation keywords\"\n") "<199407131114.HAA09807 at xebec.gsfc.nasa.gov>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA18356; Wed, 13 Jul 94 09:46:41 EDT Return-Path: Message-Id: <1994Jul13.131401.18344 at noao.edu> Organization: National Optical Astronomy Observatories Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!newsxfer.itd.umich.edu!gatech!ncar!noao!lepus.tuc.noao.edu!tody References: <199407131114.HAA09807 at xebec.gsfc.nasa.gov> Reply-To: tody at noao.edu From: tody at lepus.tuc.noao.edu (Doug Tody) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: Wed, 13 Jul 1994 13:14:01 GMT In article <199407131114.HAA09807 at xebec.gsfc.nasa.gov>, Arnold Rots writes: > > The big problem with using numbers (and, hence, the general > applicability of your scheme) is, as pointed out in an earlier > message, that it does not work for column-related keywords - or any > other keyword that *has* to end in a digit. This, by the way, is a > situation where one really gets into a crunch: a mandatory/customary > "T", three digits for the column number, at least one character for a > sequence indicator - and there is very little left to indicate what > the keyword is really about, especially if it follows some kind of > convention that prescribes yet another mandatory character. > > - Arnold Rots Arnold, this is a good point. I was not proposing this as a fully general mechanism however. Extending FITS to allow any string keyword to be continued arbitrarily is a pretty major change and I was not proposing anything of the sort. We use this mechanism in IRAF only for a few special cases and in these cases the 8 character keyword limit, while a nusiance, is not a serious problem. Frankly, in the cases where we do this there can be a lot of data and we would be better off using a table - we use the multiple keyword technique only because we don't do this often enough to warrant the more complex and general approach (a table). For most normal header strings we live within the 68 character limit. - Doug Doug Tody National Optical Astronomy Observatories IRAF project tody at noao.edu P.O. Box 26732, Tucson, Arizona, 85726 iraf at noao.edu From fitsbits-request Wed Jul 13 10:33:29 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["5719" "Wed" "13" "July" "1994" "10:33:27" "-0400" "Ian M. George" "GEORGE at lheavx.gsfc.nasa.gov" "<940713103327.20207952 at lheavx.gsfc.nasa.gov>" "112" "OFWG Recommendation R11 - Exposure-time related keywords in OGIP FITS files" "^From:" nil nil "7" "1994071314:33:27" "OFWG Recommendation R11 - Exposure-time related keywords in OGIP FITS files" (number " " mark " Ian M. George Jul 13 112/5719 " thread-indent "\"OFWG Recommendation R11 - Exposure-time related keywords in OGIP FITS files\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA18425; Wed, 13 Jul 94 10:33:29 EDT Return-Path: Message-Id: <940713103327.20207952 at lheavx.gsfc.nasa.gov> From: "Ian M George, Code 668, NASA/GSFC, USA (Usque ad mortem bibendum)" Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Cc: GEORGE at lheavx.gsfc.nasa.gov Subject: OFWG Recommendation R11 - Exposure-time related keywords in OGIP FITS files Date: Wed, 13 Jul 1994 10:33:27 -0400 (EDT) At a meeting on 1994 Jul 06, the OGIP FITS Working Group (OFWG) raised its earlier preliminarily recommendation p20.1 regarding exposure-time related keywords for OGIP FITS files to the lofty status of a full recommendation: --------------------------------------------------------------------------- OFWG Recommendation R11 ------------------------------------------------------ | EXPOSURE-TIME RELATED KEYWORDS FOR OGIP FITS FILES | ------------------------------------------------------ Version: 1994 May 11 Below are two sets of FITS keywords & definitions: Table 1 lists the keywords which should be used in OGIP files to store various 'exposure times' associated with a FITS dataset; Table 2 lists the related keywords which should be used to store various correction factors to these times. This set of keywords & definitions was constructed from the most common usage within X- & Gamma-ray FITS files. The lack of a consistent naming convention most probably reflects the fact that most of these keywords were devised by different people in different places at different times. The OFWG strongly recommends that these keywords (only) be used to store the various quantities within OGIP FITS files. It should be noted that only those keywords/quantities considered necessary must be included in the header of a given FITS dataset. However the OFWG suggests that where possible all know quantities are included in the header. TABLE 1 - "Exposure-time" related keywords ****************************************** Keyword Meaning & Notes ------- --------------- TELAPSE is the time interval (in seconds) obtained as difference between the start and stop times of an observation. Any gaps due to Earth occultation, or high background counts and/or other anomalies, are included. ONTIME is the total "good" time (in seconds) on 'source'. If a 'Good Time Interval' (GTI) table is provided, ONTIME should be calculated as the sum of those intervals. Corrections for instrumental 'dead time' effect are NOT included. LIVETIME is the total time (in seconds) on source, corrected for the 'total' instrumental dead time effect. The ratio LIVETIME/ONTIME therefore gives the dead time correction value (which hence lies in the range 0.0 to 1.0). EXPOSURE is the total time (in seconds) on source, corrected for any relevant quantity used to calculate the corrected count rate. The value can include correction which are not directly related with time (e.g. collimation efficiency or vignetting). This keyword used in the header of FITS file, is a mean value when appropriate. TABLE 2 - Keywords for "Exposure-time" correction factors ********************************************************* Keyword Meaning & Notes ------- --------------- DEADC is the total correction factor for any dead time effect (ie LIVETIME/ONTIME), and lies in the range 0.0 to 1.0. Thus the multiplication of this value by the ONTIME value gives the 'effective' integration time or LIVETIME (TIMEDEL in the case of a light curve). Since the total dead time of a given dataset can be the result of a multitude of instrumental/processing effects (especially related to the particular experiment, and/or processing by an onboard computer, and/or spacecraft operations), it is recommended that as well as including the total correction factor in the DEADC keyword, instrument-specific keywords are used to keep a record of the original value for the individual correction factors. ERRCOR Corrections are usually applied to the stored counts and the error on those counts. However, in some cases the errors can need an extra correction, different to that applied to the counts. The ERRCOR keyword contains this extra value. The value is unitless. Es. If the dead time is related to the sampling of the on board computer, it has been found that different correction are needed for counts (count/s) and errors. This can be generalized to all cases in which the error needs to be treated differently compared to the counts. VIGNET is the correction factor for the collimator response or vignetting function (depending upon the instrument) such as to enable the scientific dataset (eg lightcurve) to be rescaled to that which would have been obtained had the source been observed at the angle of peak collimator response (for collimated instruments) or on-axis (for instruments employing imaging optics). In both cases the value of VIGNET should lie in the range 0.0 - 1.0, and thus be the factor by which the scientific dataset should be divided in order to obtain the equivalent on-axis value. In both cases (but especially in the case of imaging optics) the correction factor can be a function of energy, and thus can only be used if a useful mean value can be defined. In the case of imaging optics, VIGNET should contain the the total correction factor due to vignetting and any (energy-independent) reduction in collecting area resulting from obscuration by the near-side of the mirror, support structures etc. It should be noted that historically these correction factors have often [naturally] been referred to by different names, and sometimes as the reciprical of the value defined above. ----------------------------------- END ---------------------------------- Ian M George NASA/GSFC OFWG 1994 Jul 11 From fitsbits-request Wed Jul 13 10:31:00 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1243" "Wed" "13" "July" "1994" "10:31:00" "-0400" "Ian M. George" "GEORGE at lheavx.gsfc.nasa.gov" "<940713103100.20207952 at lheavx.gsfc.nasa.gov>" "28" "OFWG Recommendation R9 - The OGIP data quality flag convention" "^From:" nil nil "7" "1994071314:31:00" "OFWG Recommendation R9 - The OGIP data quality flag convention" (number " " mark " Ian M. George Jul 13 28/1243 " thread-indent "\"OFWG Recommendation R9 - The OGIP data quality flag convention\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA18413; Wed, 13 Jul 94 10:31:00 EDT Return-Path: Message-Id: <940713103100.20207952 at lheavx.gsfc.nasa.gov> From: "Ian M George, Code 668, NASA/GSFC, USA (Usque ad mortem bibendum)" Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Cc: GEORGE at lheavx.gsfc.nasa.gov Subject: OFWG Recommendation R9 - The OGIP data quality flag convention Date: Wed, 13 Jul 1994 10:31:00 -0400 (EDT) At a meeting on 1994 Jul 06, the OGIP FITS Working Group (OFWG) raised its earlier preliminarily recommendation p20.3 regarding the values used to flag good & bad data within FITS datasets produced by the OGIP to the lofty status of a full recommendation: --------------------------------------------------------------------------- OFWG Recommendation R9 ----------------------------------------- | THE OGIP DATA QUALITY FLAG CONVENTION | ----------------------------------------- Version: 1994 Jul 06 The meaning and use of quality flags is often specific to a given instrument and/or data type. However, when devising quality flags for new FITS files, the OFWG recommends that the integer value 0 (zero) be used to indicate that the data to which the flag refers is of GOOD quality, and is to be automatically used by downstream software. An instrument/dataset/application-specific scale of non-zero quality flag values can be used to indicate why the data is considered of bad quality, the degree of 'badness', and data flagged to be ignored in downstream applications by the user. ----------------------------------- END ---------------------------------- Ian M George NASA/GSFC OFWG 1994 Jul 11 From fitsbits-request Wed Jul 13 17:17:57 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1550" "" "13" "July" "1994" "16:59" "EDT" "BARRY M. SCHLESINGER" "bschlesinger at nssdca.gsfc.nasa.gov" "<13JUL199416595914 at nssdca.gsfc.nasa.gov>" "36" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994071320:59:00" "FITS continuation keywords" (number " " mark " BARRY M. SCHLESIN Jul 13 36/1550 " thread-indent "\"Re: FITS continuation keywords\"\n") "<2vl48b$k7k at explorer.clark.net>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA19292; Wed, 13 Jul 94 17:17:57 EDT Return-Path: Message-Id: <13JUL199416595914 at nssdca.gsfc.nasa.gov> Organization: NASA - Goddard Space Flight Center Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!news.umbc.edu!haven.umd.edu!ames!newsfeed.gsfc.nasa.gov!nssdca.gsfc.nasa.gov!bschlesinger References: <199407051645.MAA00598 at tetra.gsfc.nasa.gov> <2vl48b$k7k at explorer.clark.net> From: bschlesinger at nssdca.gsfc.nasa.gov (BARRY M. SCHLESINGER) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 13 Jul 1994 16:59 EDT In article <2vl48b$k7k at explorer.clark.net>, leb at clark.net (Lee E. Brotzman) writes... >When one defines a new keyword, one must also define the keyword's value, either >integer, floating point, string, etc. Therefore, to have "long strings" one >simply defines the keyword syntax as a string and that the keyword can appear >more than once, like COMMENT. There is no specific prohibition against >repeated occurences of a keyword, so as far as I can tell, this is a legal FITS >solution to the "problem" (which, of course, I still do not believe really >exists). > There is no specific prohibition against repeated occurrences of a keyword. >It may even be better to define the keyword without the '=', thereby creating a >new form of COMMENT/HISTORY. I'll leave that to Barry to see if it is legal It is permitted. The meaning is that columns 9-80 of the card image are comment. Users may define their own keywords that have only comment. The controversial HIERARCH is an example. >(frankly, his message talking about 'no value' for keywords without '=' was a >little confusing to me, just what does 'no value' mean?). > The operational definition is that a value field, following an "= " in columns 9-10 must be of one of the specified types and follow the format prescriptions for the type, for example, character strings enclosed in single right quotes. Comment fields are not limited. A conceptual definition has not been stated and would require some careful thought. Barry Schlesinger NOST FITS Support Office From fitsbits-request Wed Jul 13 18:24:44 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["151" "Wed" "13" "July" "1994" "13:20:28" "-0500" "Steve Mazuk" "Mazuk at courier2.aero.org" "" "4" "FTP Site for FAQ archive?" "^From:" nil nil "7" "1994071318:20:28" "FTP Site for FAQ archive?" (number " " mark " Steve Mazuk Jul 13 4/151 " thread-indent "\"FTP Site for FAQ archive?\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA19430; Wed, 13 Jul 94 18:24:44 EDT Return-Path: Message-Id: Organization: The Aerospace Corporation Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!howland.reston.ans.net!swrinde!elroy.jpl.nasa.gov!news.aero.org!news2.aero.org!sheila.aero.org!user From: Mazuk at Courier2.Aero.Org (Steve Mazuk) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: FTP Site for FAQ archive? Date: Wed, 13 Jul 1994 13:20:28 -0500 Where is the FTP archive for the FAQ for this group? My site doesn't have the FAQ and the SCI.ASTRO site doesn't appear to have this group's FAQ. From fitsbits-request Thu Jul 14 12:07:01 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["664" "Thu" "14" "July" "1994" "14:38:24" "GMT" "Rob Hewett" "hewett at cfabuh.harvard.edu" "" "24" "How to mount Digitized Sky Survey CDROM???" "^From:" nil nil "7" "1994071414:38:24" "How to mount Digitized Sky Survey CDROM???" (number " " mark " Rob Hewett Jul 14 24/664 " thread-indent "\"How to mount Digitized Sky Survey CDROM???\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA21365; Thu, 14 Jul 94 12:07:01 EDT Return-Path: Message-Id: Organization: Smithsonian Astrophysical Observatory, Cambridge, MA, USA Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!howland.reston.ans.net!europa.eng.gtefsd.com!MathWorks.Com!news.duke.edu!godot.cc.duq.edu!newsfeed.pitt.edu!ctc.com!news.cs.umb.edu!hsdndev!cfanews!cfabuh!hewett From: hewett at cfabuh.harvard.edu (Rob Hewett) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: How to mount Digitized Sky Survey CDROM??? Date: Thu, 14 Jul 1994 14:38:24 GMT This question is not a FITS question but I thought some s.a.f reader might know: How do you mount a CDROM from STSI's Digitized Sky Survey on a Sun running either SunOS 4 or 5? This did not work on OS 4: mount -rt hsfs /dev/sr0 /cdrom I got essentially the same result with a Volume Manager mounted CD on OS 5. That is that you can see the directories, but they all seem to be empty. And you don't see the copyright file, which is supposed to be in the base directory of each CD in the set. The CD claims to be ISO 9660. Thanks in advance for any help or ideas. Please send email. I'll post the answer if I ever get it :-) -Rob hewett at cfa.harvard.edu From fitsbits-request Thu Jul 14 15:35:23 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["4407" "Thu" "14" "July" "1994" "15:35:18" "-0400" "Arnold Rots" "arots at xebec.gsfc.nasa.gov" "<199407141935.PAA11649 at xebec.gsfc.nasa.gov>" "105" "The Registration of Time Bins in FITS Files" "^From:" nil nil "7" "1994071419:35:18" "The Registration of Time Bins in FITS Files" (number " " mark " Arnold Rots Jul 14 105/4407 " thread-indent "\"The Registration of Time Bins in FITS Files\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA00200; Thu, 14 Jul 94 15:35:23 EDT Return-Path: Message-Id: <199407141935.PAA11649 at xebec.gsfc.nasa.gov> From: Arnold Rots Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at NRAO.EDU Subject: The Registration of Time Bins in FITS Files Date: Thu, 14 Jul 1994 15:35:18 -0400 Relating Pixel Coordinates (Counting Indices) to Physical Coordinates ===================================================================== The Proposed FITS Convention ---------------------------- The NOST Standards document, in its section on the proposed conventions regarding World Coordinate Systems, states, correctly, that the common convention in FITS, and the one put forward by Greisen and Calabretta, is to regard the counting index (integer pixel coordinate) as referring to the center of each pixel. Though this deviates from practices common in other fields, it is felt that such a convention is the most intuitive in an astronomical context. No explanation is given, but the observation is, in our view, correct. Since there is no formal standard, though, the document encourages FITS writers to document the convention that is used in embedded comments. The Problem ----------- As stated above, we consider the observation that astronomers naturally refer to the center (rather than the edge) of a pixel as the point that coordinates refer to as correct and justified. However, there is one glaring exception to this rule: the time coordinate. When velocity bins, frequency bins, or spatial bins are tagged with a coordinate value, that value does usually refer to the center of that bin, but when time bins are tagged with a time stamp, that time stamp most commonly refers to the start of the bin, or the end of it, but seldom the middle. There are two main reasons for this. 1. What is directly measurable is either the start or the end of an integration, but not the halfway point. Alternatively, one might say that if one had to specify an integration period, one would use start and/or stop time, not the average value of (absolute) time. 2. In various instruments, a certain event (e.g., a pulse) may initiate a number of separate time series measurements, with different bin sizes: one quantity may be read out every second, while another is accumulated over 16 seconds, etc. If one collects these measurements (say, in a table), it makes eminent sense to stamp them with the start time of the collective integrations. It would be much more cumbersome to stamp everything with its halfway time. Consequently, if one looks in various FITS archives that hold time series data, there is no consistency in the definition of what location in the pixel (bin) the time stamps refer to. Some people have taken pains to comply with the informal FITS convention, while others, no doubt, did not even realize that there was a convention that might conceivably apply to this situation - they just did the "natural" thing. As a result, the convention in use has not even always been documented in comments in the FITS header. Even if it were, though, it would be unsatisfactory. It is of crucial importance for time series analysis software to know where the relative and the absolute time stamps fall (if there are different pixel sizes it is not even a matter of a constant offset). Essential information should not be buried in comment or history keywords; it should be provided in "real" keywords. Our Proposed Solution --------------------- Based on the considerations given above, we have a strong need for a keyword indicating the convention used for registering pixels along the TIME axis. We propose to use for this purpose the keyword: TIMEPIXR which indicates the (relative) registration of TIME pixels, through a value between: 0.0 <= TIMEPIXR <= 1.0 Actually, one should probably state that the value of TIMPIXR *usually* would be between 0.0 and 1.0. TIMEPIXR indicates the position in the pixel that corresponds to the coordinate values defined by the regular WCS-type keywords. For instance: TIMEPIXR= 0.0 / time stamps refer to the START of the pixel TIMEPIXR= 0.5 / time stamps refer to the CENTER of the pixel TIMEPIXR= 1.0 / time stamps refer to the END of the pixel If the keyword TIMEPIXR is absent, we shall assume a default value of 0.5, consistent with the convention that coordinate values usually refer to the center of the pixel, in FITS. We considered a generalized version of this keyword, but decided against that. As far as we can see, the problem only occurs for the TIME coordinate axis, and there seems no need to complicate matters by allowing the same mechanism to apply to other coordinates. - Arnold Rots From fitsbits-request Thu Jul 14 16:18:17 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["844" "Thu" "14" "July" "1994" "16:18:14" "EDT" "Eric Greisen" "egreisen at primate.cv.nrao.edu" "<9407142018.AA02951 at primate.cv.nrao.edu>" "17" "Location of coordinate in a pixel" "^From:" nil nil "7" "1994071420:18:14" "Location of coordinate in a pixel" (number " " mark " Eric Greisen Jul 14 17/844 " thread-indent "\"Location of coordinate in a pixel\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA00273; Thu, 14 Jul 94 16:18:17 EDT Return-Path: Message-Id: <9407142018.AA02951 at primate.cv.nrao.edu> From: egreisen at primate.CV.NRAO.EDU (Eric Greisen) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at primate.CV.NRAO.EDU Subject: Location of coordinate in a pixel Date: Thu, 14 Jul 94 16:18:14 EDT The reason to locate physical coordinates at the centers of voxels is to allow that pixel "location" to remain invariant under rotation, transposition, and the like. If one transposes the standard convention (where the edge encountered first as pixel number increases is the "location") then one must alter the coordinates of the pixel - truely a non-physical thing to do. I see no reason why we should do anything different with time pixels. It may be true that current FITS tapes have varying, usually undocumented, usages. But that is also true for other coordinate types. The usual measurements that is made is of both the start and stop time of a pixel, so averaging them to give its coordinate is not onerous. Using the average time also makes computing time smoothings and the like rather simpler and more intuitive. Eric Greisen From fitsbits-request Fri Jul 15 01:16:26 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["6196" "" "14" "July" "1994" "22:51:14" "-0400" "Lee E. Brotzman" "leb at clark.net" "<304tj2$pr9 at explorer.clark.net>" "141" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994071502:51:14" "FITS continuation keywords" (number " " mark " Lee E. Brotzman Jul 14 141/6196 " thread-indent "\"Re: FITS continuation keywords\"\n") "<199407122017.QAA16849 at tetra.gsfc.nasa.gov>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA00644; Fri, 15 Jul 94 01:16:26 EDT Return-Path: Message-Id: <304tj2$pr9 at explorer.clark.net> Organization: Clark Internet Services, Inc., Ellicott City, MD USA Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!howland.reston.ans.net!swrinde!news.dell.com!tadpole.com!uunet!news2.sprintlink.net!news.sprintlink.net!news.clark.net!news.clark.net!not-for-mail References: <199407122017.QAA16849 at tetra.gsfc.nasa.gov> From: leb at clark.net (Lee E. Brotzman) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 14 Jul 1994 22:51:14 -0400 William Pence writes: > ...it appears to us that >there is still no consensus that any of them is significantly better >than our original proposal, So, at this point we are inclined to stick >with our simple convention using blank continuation lines A lack of consensus should make you step back and rethink the situation, not go ahead with it. >FITS readers that do not understand this continuation convention will >simply treat the continuation lines as COMMENT keywords. Right, and corrupt their comment structures by introducing incomplete and confusing into them. This is not a Good Thing (TM). >or to give a more realistic example of the 'array descriptor' syntax >to be used in the XTE FITS files: >TDDES2 = 'E(X1L|X1R|X2L|X2R|X3L|X3R) & C("TDDMAC1(0:14)") & \' > 'T([" at TIME",1024,0.00390625])' This looks like it could factor into more than one keyword and avoid the problem altogether, but I won't belabor the point. >1. Is the order of keywords in a FITS header significant? > ... Clearly, if comment keywords have been inserted in the >header at strategic places to explain the meaning of the preceding or >following keywords, or if history keywords are inserted to document the >order of the data processing operations, then rearranging the header >keywords will lead to loss of information. The "order of the keywords" pertains more to the relative order of *all* keywords, I think, than the order of one kind of keyword (like COMMENT). Putting COMMENT cards next to a keyword to describe it without clearly labelling the COMMENT is asking for trouble later on, e.g. TTYPE1 = 'SpType ' COMMENT Types are arranged by color. is poor style. If a reader extracts information from this header, it may read comment cards into a single structure and then spew them out as one, like so: COMMENT Names are sorted alphabetically COMMENT Types are arranged by color COMMENT Velocities are relative to Sol What goes with what? The convention I follow at the ADC is this TTYPE1 = 'SpType ' COMMENT SpType: Types are arranged by color The name of the field is in the comment. The comment explicitly names the field it is describing. It doesn't matter (as much) if the comments get shuffled to the top or bottom of the header. This is a simple, non-binding convention that works real well for our data. My code will even detect this construction and put the comments back adjacent to the appropriate keyword. You may wish to look into something similar: COMMENT TDDES2: E(X1L|X1R|X2L|X2R|X3L|X3R) & C("TDDMAC1(0:14)") & COMMENT TDDES2: T([" at TIME",1024,0.00390625]) This is readable, parseable, and best of all it conforms to the current standard in every way. >in a specific order to make them easier for humans to understand, or to >make it more efficient for software to read them. This is an aside from the main topic, but does FITSIO still destroy the contents of inline comments following a slash character '/'? I actually use inline comments and it perturbs me to see FITSIO wipe out that information. It's important for making the header "easier for humans to understand". >2. On the use of backslash (\) as a continuation character. No real argument here, except that it is unnecessary and unhealthy (for reading software) to use your convention at all. >3. Should the continuation keywords have a blank name, or some other >name. >This in our minds is the biggest open issue about this proposal. The >blank keyword names will simply be treated as comments by FITS readers >that don't understand this convention which we felt was advantagous >(i.e. is the least harmful way to deal with an unknown convention). The reason I object to your convention is because it is the *most* harmful way to introduce the information to a FITS reader that does not understand the convention. You are partially using previously defined conventions, thereby confusing just about all readers. >We, on the other hand, do not like the idea of repeating the original >keyword name on all the continuation cards. This could cause >unpredicable results for FITS readers that do not understand this >convention (would they return the value of the first keyword, or of the >last?), and could be confusing if a FITS reader tried to delete the >keyword, but didn't delete all the continuations. A reader that doesn't understand the convention almost certainly won't read the keyword at all. You are not suggesting using this convention for reserved keywords like TTYPE are you? I didn't think so, but if you are *please don't*. The value returned by a reader that doesn't know the convention is a moot point. It will return incomplete (and supposedly useless) information in any case, following any of the suggested courses. That's why we've always been told to just ignore keywords we don't understand. I don't want to ignore the blank comment keyword, though, although under your system, I won't have a choice. > Under our >convention, if a FITS reader just deleted the first keyword of a >continuation sequence then the header would just be left with some >orphaned continuation lines that are interpreted as comments and do no >harm. I'm going to beat this horse into the ground. Orphaned, incomplete, nonsense comments most certainly do harm. They make headers more difficult to read and interpret correctly later on in the archival life of the data. >4. Why not use a more complicated 'keyword referencing' scheme to point >to the next keyword in the continued string? No argument here. To sum up. Please do not follow the convention you initially proposed with blank comment keywords to hold continuations. It will not be nearly as harmless as you think it is, and it defies the current standard. It is bad medicine. >Regards, >Bill Pence >Arnold Rots -- -- Lee E. Brotzman E-mail: leb at clark.net -- Advanced Data Solutions Phone : 301-776-0324 -- "Sometimes I think the surest sign that intelligent life exists elsewhere in -- the Universe is that none of it has tried to contact us." --- Calvin, 1990 From fitsbits-request Fri Jul 15 02:53:23 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1971" "" "14" "July" "1994" "23:04:55" "-0400" "Lee E. Brotzman" "leb at clark.net" "<304ucn$r91 at explorer.clark.net>" "39" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994071503:04:55" "FITS continuation keywords" (number " " mark " Lee E. Brotzman Jul 14 39/1971 " thread-indent "\"Re: FITS continuation keywords\"\n") "<13JUL199416595914 at nssdca.gsfc.nasa.gov>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA00716; Fri, 15 Jul 94 02:53:23 EDT Return-Path: Message-Id: <304ucn$r91 at explorer.clark.net> Organization: Clark Internet Services, Inc., Ellicott City, MD USA Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!swrinde!news.dell.com!tadpole.com!uunet!news2.sprintlink.net!news.sprintlink.net!news.clark.net!news.clark.net!not-for-mail References: <199407051645.MAA00598 at tetra.gsfc.nasa.gov> <2vl48b$k7k at explorer.clark.net> <13JUL199416595914 at nssdca.gsfc.nasa.gov> From: leb at clark.net (Lee E. Brotzman) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 14 Jul 1994 23:04:55 -0400 bschlesinger at nssdca.gsfc.nasa.gov (BARRY M. SCHLESINGER) writes: >In article <2vl48b$k7k at explorer.clark.net>, leb at clark.net (Lee E. Brotzman) writes... >>It may even be better to define the keyword without the '=', thereby creating a >>new form of COMMENT/HISTORY. I'll leave that to Barry to see if it is legal >It is permitted. The meaning is that columns 9-80 of the card image >are comment. Users may define their own keywords that have only >comment. The controversial HIERARCH is an example. >>(frankly, his message talking about 'no value' for keywords without '=' was a >>little confusing to me, just what does 'no value' mean?). >The operational definition is that a value field, following an "= " in >columns 9-10 must be of one of the specified types and follow the >format prescriptions for the type, for example, character strings >enclosed in single right quotes. Comment fields are not limited. Aahh. Now I get it. Thanks for the clarification. This confirms then my belief that using multiple instances of the same keyword and a comment (i.e. 'no value') format is the most appropriate form for "continuing" long strings. For example: TTDES1 This is the beginning of the structured TTDES1 field.. TTDES1 This is the end of the structured TTDES1 field. Bill, I hope that you and Arnold will reconsider this and move away from adopting a new interpretation convention for an established keyword (the "blank" comment keyword in this case.) The format above is indeed considered legal under the current standard and will provoke the least problem for readers that do not adhere to the convention. > Barry Schlesinger > NOST FITS Support Office -- -- Lee E. Brotzman E-mail: leb at clark.net -- Advanced Data Solutions Phone : 301-776-0324 -- "Sometimes I think the surest sign that intelligent life exists elsewhere in -- the Universe is that none of it has tried to contact us." --- Calvin, 1990 From fitsbits-request Fri Jul 15 15:12:40 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["2671" "" "15" "July" "1994" "15:47:50" "GMT" "Robert Hill" "bhill at trifle.gsfc.nasa.gov" "<306b36$ck5 at paperboy.gsfc.nasa.gov>" "62" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994071515:47:50" "FITS continuation keywords" (number " " mark " Robert Hill Jul 15 62/2671 " thread-indent "\"Re: FITS continuation keywords\"\n") "<304tj2$pr9 at explorer.clark.net>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA03890; Fri, 15 Jul 94 15:12:40 EDT Return-Path: Message-Id: <306b36$ck5 at paperboy.gsfc.nasa.gov> Organization: NASA Goddard Space Flight Center -- InterNetNews site Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!howland.reston.ans.net!agate!ames!newsfeed.gsfc.nasa.gov!bhill References: <199407122017.QAA16849 at tetra.gsfc.nasa.gov> <304tj2$pr9 at explorer.clark.net> From: bhill at trifle.gsfc.nasa.gov (Robert Hill) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 15 Jul 1994 15:47:50 GMT >William Pence writes: >> ...it appears to us that >>there is still no consensus that any of them is significantly better >>than our original proposal, So, at this point we are inclined to stick >>with our simple convention using blank continuation lines leb at clark.net (Lee E. Brotzman) writes: >A lack of consensus should make you step back and rethink the situation, not go >ahead with it. I second Lee's comment here. In an earlier post, I was willing to go along with the scheme of backslashes and blank comments, but I believe I said `if you must' in there somewhere. I still hope you don't. (Yet another alternative suggestion is made at the end of this posting.) >William Pence writes: >>TDDES2 = 'E(X1L|X1R|X2L|X2R|X3L|X3R) & C("TDDMAC1(0:14)") & \' >> 'T([" at TIME",1024,0.00390625])' leb at clark.net (Lee E. Brotzman) writes: >This looks like it could factor into more than one keyword and avoid the >problem altogether, but I won't belabor the point. On the face of it, I have to agree with this. Why can't the E(), C(), and T() constructs be made into separate strings? In this example, they aren't too long. Could they be? All right, suppose there is a compelling reason the value can't be pre-parsed like that. Now here's my concrete suggestion. It's really a combination of Lee's idea with something like IRAF-style keyword references: TDDES2 = 'COMMENT TDDES2:' / Value exists in comment COMMENT TDDES2: [the rest as Lee had it] COMMENT TDDES2: [etc.] COMMENT TDDES2: [etc.] Your design constraints appear to be these: You want the existence of the parameter to be indicated at the level of regular FITS keywords. But you are willing to do a little extra work to get at the actual string value. OK, this scheme lets you tell immediately that the value is present (because keyword TDDES2 exists). It tells you that it isn't in the keyword value itself (value starts with COMMENT), and it tells you which comments to look in. And just like the backslash + blank-comment scheme, it could be applied arbitrarily to any locally-defined string parameter. If the word COMMENT occurring at the beginning of a string value is a problem, you could use something else, of course. Best of all, it avoids the stylistic problem of values split across two different *kinds* of FITS records. -- Bob Hill -- Robert.S.Hill at gsfc.nasa.gov Not speaking for any of these: Hughes STX Corp., Code 681, NASA/GSFC, Greenbelt, MD 20771 "There's nothing you can't prove if your outlook is only sufficiently limited." -- Lord Peter Wimsey From fitsbits-request Fri Jul 15 19:30:33 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["977" "" "15" "July" "1994" "11:02" "EDT" "BARRY M. SCHLESINGER" "bschlesinger at nssdca.gsfc.nasa.gov" "<15JUL199411020326 at nssdca.gsfc.nasa.gov>" "22" "Re: FTP Site for FAQ archive?" "^From:" nil nil "7" "1994071515:02:00" "FTP Site for FAQ archive?" (number " " mark " BARRY M. SCHLESIN Jul 15 22/977 " thread-indent "\"Re: FTP Site for FAQ archive?\"\n") ""] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA04252; Fri, 15 Jul 94 19:30:33 EDT Return-Path: Message-Id: <15JUL199411020326 at nssdca.gsfc.nasa.gov> Organization: NASA - Goddard Space Flight Center Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!caen!malgudi.oar.net!sun!oucsace!att-out!pacbell.com!ames!newsfeed.gsfc.nasa.gov!nssdca.gsfc.nasa.gov!bschlesinger References: From: bschlesinger at nssdca.gsfc.nasa.gov (BARRY M. SCHLESINGER) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FTP Site for FAQ archive? Date: 15 Jul 1994 11:02 EDT In article , Mazuk at Courier2.Aero.Org (Steve Mazuk) writes... > >Where is the FTP archive for the FAQ for this group? My site doesn't have >the FAQ and the SCI.ASTRO site doesn't appear to have this group's FAQ. A modified copy of the most recent FITS basics and information proposal is available by anonymous ftp from nssdca.gsfc.nasa.gov in the directory FITS (it's a VAX; case is not important). The main difference from the posted material is that the basics and information file at nssdca.gsfc does not contain the material describing the contents of the NSSDC ftp directories; that information is, however, available in the AAREADME.DOC file in the same directory. Copies of the actual posting are automatically archived at ftp://rtfm.mit.edu/pub/usenet/sci.astro.fits/. The MIT site is the archive location for all Usenet periodic posting and is frequenty busy. Barry Schlesinger NSSDC/NOST FITS Support Office From fitsbits-request Sun Jul 17 05:32:07 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["6304" "" "15" "July" "1994" "16:33" "EDT" "BARRY M. SCHLESINGER" "bschlesinger at nssdca.gsfc.nasa.gov" "<15JUL199416332137 at nssdca.gsfc.nasa.gov>" "134" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994071520:33:00" "FITS continuation keywords" (number " " mark " BARRY M. SCHLESIN Jul 15 134/6304 " thread-indent "\"Re: FITS continuation keywords\"\n") "<199407122017.QAA16849 at tetra.gsfc.nasa.gov>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA08691; Sun, 17 Jul 94 05:32:07 EDT Return-Path: Message-Id: <15JUL199416332137 at nssdca.gsfc.nasa.gov> Organization: NASA - Goddard Space Flight Center Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!howland.reston.ans.net!agate!ames!newsfeed.gsfc.nasa.gov!nssdca.gsfc.nasa.gov!bschlesinger References: <199407122017.QAA16849 at tetra.gsfc.nasa.gov> From: bschlesinger at nssdca.gsfc.nasa.gov (BARRY M. SCHLESINGER) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 15 Jul 1994 16:33 EDT In article <199407122017.QAA16849 at tetra.gsfc.nasa.gov>, William Pence (and Arnold Rots) write... >... of the continuation proposal .... Many variations on our >proposed convention have been suggested, ... but it appears to us that >there is still no consensus that any of them is significantly better >than our original proposal, So, at this point we are inclined to stick >with our simple convention ... I concur with Lee Brotzman and Robert Hill. Failure to reach consensus is not a justification for moving ahead; on the contrary it is a reason for not taking any action and discussing the issue further. Consensus is the established process by which standards are adopted. The NOST standard underwent three review cycles, during which the community was given opportunity to comment. The Technical Panel considered all comments, either incorporating the substance into the standard or determining the reason the response should not be incorporated. NOST also has an Accreditation Panel, whose role is to be certain that in the development of any standard, the Technical Panel makes all reasonable effort to notify the affected community of the opportunity to review the standard, and that all reviewers comments were addressed, either through revision, or through explanation why there was no revision. Those who followed the development of Fortran-90 will remember the extensive review cycles, -- so extensive that the language originally started as Fortran 8x -- and the significant changes made during the process. Standards are effective only when observed by the great majority of people in the affected area. They will do so only if they believe that the standard is a good standard. Hence the requirement for general, consensus acceptance. If there is no consensus, the rules will not be followed. Proceeding to define a convention in the absence of consensus will not produce a standard, only a local convention that is at least not understood elsewhere, and, very likely, misinterpreted by FITS readers. > > c. The continuation keyword must contain ASCII blanks > in columns 1-10 (i.e., the keyword name must be blank > and there must be no equal sign in column 9). > > We ... would like >to comment on a few issues that have been raised: > >1. Is the order of keywords in a FITS header significant? > >The only statement made in the NOST FITS Standard about keyword order is: > > "Except where specifically stated otherwise in this standard, > keywords may appear in any order." > >and the only places that the order is specifically stated is for the >required keywords at the beginning of each Header-Data Unit. We >interpret this to mean that the creators of a FITS file are free to >decide on the most appropriate order for the keywords, Yes. >but it does not >mean, in our view, that readers may come along later and arbitrarily >rearrange the order of the keywords without risking the loss of some >information. But, as it does not forbid reorganizations, it does not mean that writers can rely on keyword order. >Clearly, if comment keywords have been inserted in the >header at strategic places to explain the meaning of the preceding or >following keywords, or if history keywords are inserted to document the >order of the data processing operations, then rearranging the header >keywords will lead to loss of information. Also, keywords may be put >in a specific order to make them easier for humans to understand, or to >make it more efficient for software to read them. So we believe that >our convention that requires that the order of the continuation lines >be preserved is not unprecedented nor violates the FITS Standard. > One reason for the failure of the KHBEGIN/KHEND hierarchy convention (discussed in the User's Guide) to fail to gain wide acceptance was the problem of keyword rearrangement. What we're dealing with here is a requirement for readers to recognize a connection between different keywords, as opposed to recognizing a comment block. In that sense the requirement is unprecedented. And not everything that is permitted by the rules of FITS is wise. >2. On the use of backslash (\) as a continuation character. > >We prefer to use the backslash because it is a rarely used ASCII >character and is thus unlikely to be confused with a literal backslash >within the string value. One can still write string keyword values >that end in a backslash, and these will not be confused with our >continuation convention as long as one of the 3 conditions listed above >is violated (e.g., insert a space character between the backslash and >the closing quote character, or make sure that the following keyword >does not have a blank name). > >Some people objected to having any sort of continuation character at >the end of the line, but we think it is important to have some sort of >indicator to software that it should check the following keyword to see >if it is a continuation. Otherwise the software would have to always >check the next keyword every time it reads a string valued keyword. > >As was stated by several other people, the fact that the backslash has >a different meaning in other contexts (e.g., C source code and Unix >shell scripts) we believe is not a real source of confusion here. > ANSI X3.4-1977 (code for information interchange) states, "Their [ISO and CCITT] recommendations, however, permit national standardization by the various countries in seven code table positions." One of these National Use positions is 5/12 (hexadecimal 5C), which, in the U. S. is \ (reverse slant or backslash). So the symbol is not necessarily internationally portable. >3. Should the continuation keywords have a blank name, or some other >name. > There have been some suggestions by others in this thread that the continuation have the first eight columns blank, followed by '= ' in 9-10. While such a practice is permitted, the reason is basically historical: it was allowed in the FITS papers. Except for the three original keywords, '= ' is not permitted in columns 9-10 of comment keywords, and using it in HISTORY, COMMENT, and [blank] is not encouraged. Barry Schlesinger NSSDC/NOST FITS Support Office From fitsbits-request Sun Jul 17 07:42:52 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["810" "Fri" "15" "July" "1994" "21:41:19" "GMT" "daniel fox" "foxd at silver.ucs.indiana.edu" "" "17" "PC FITS program" "^From:" nil nil "7" "1994071521:41:19" "PC FITS program" (number " " mark " daniel fox Jul 15 17/810 " thread-indent "\"PC FITS program\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA09059; Sun, 17 Jul 94 07:42:52 EDT Return-Path: Message-Id: Organization: Indiana University, Bloomington IN Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!howland.reston.ans.net!sol.ctr.columbia.edu!usenet.ucs.indiana.edu!silver.ucs.indiana.edu!foxd From: foxd at silver.ucs.indiana.edu (daniel fox) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: PC FITS program Date: Fri, 15 Jul 1994 21:41:19 GMT I have written a couple of programs for viewing some FITS files from a CDROM I obtained some time back. I would like to release the programs into the public domain. Where can they be uploaded to? I tried to design the programs to be easy to use. Dan ****************************************************************************** * Daniel B. Fox * All men are mortal. * * KF9ET * Aristotle is a man. * * FOXD at SILVER.UCS.INDIANA.EDU * Therefore: All men are Aristotle. * ****************************************************************************** * My opinions are my own. Mine you hear me!!! MINE!!! MINE!!! MINE!!!! * ****************************************************************************** From fitsbits-request Wed Jul 20 01:46:24 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["345" "" "19" "July" "1994" "20:36:20" "GMT" "Kenneth M. Smith" "kmsg7172 at uxa.cso.uiuc.edu" "<30hdg4$t5m at vixen.cso.uiuc.edu>" "22" "/*_____ HELP: Need to find FITS file converter ___*/" "^From:" nil nil "7" "1994071920:36:20" "/*_____ HELP: Need to find FITS file converter ___*/" (number " " mark " Kenneth M. Smith Jul 19 22/345 " thread-indent "\"/*_____ HELP: Need to find FITS file converter ___*/\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA18568; Wed, 20 Jul 94 01:46:24 EDT Return-Path: Message-Id: <30hdg4$t5m at vixen.cso.uiuc.edu> Organization: University of Illinois at Urbana Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!inxs.concert.net!taco.cc.ncsu.edu!gatech!howland.reston.ans.net!vixen.cso.uiuc.edu!uxa.cso.uiuc.edu!kmsg7172 From: kmsg7172 at uxa.cso.uiuc.edu (Kenneth M Smith) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: /*_____ HELP: Need to find FITS file converter ___*/ Date: 19 Jul 1994 20:36:20 GMT Hi, I need to find some 'C' code for converting 16-bit FITS files to either raw ASCII files or binary byte files. I have to imagine that somebody out there in net-land has done this before, and would really appreciate it if you could help me out. Please respond by e-mail to: kensmith at uiuc.edu Thanks in advance for your help! Ken From fitsbits-request Wed Jul 20 15:15:45 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1889" "Wed" "20" "July" "1994" "19:10:27" "GMT" "Pat Murphy" "pmurphy at nrao.edu" "" "40" "Re: How to mount Digitized Sky Survey CDROM???" "^From:" nil nil "7" "1994072019:10:27" "How to mount Digitized Sky Survey CDROM???" (number " " mark " Pat Murphy Jul 20 40/1889 " thread-indent "\"Re: How to mount Digitized Sky Survey CDROM???\"\n") ""] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA21150; Wed, 20 Jul 94 15:15:45 EDT Return-Path: Message-Id: Organization: National Radio Astronomy Observatory Path: saips.cv.nrao.edu!news!pmurphy References: From: pmurphy at nrao.edu (Pat Murphy) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: How to mount Digitized Sky Survey CDROM??? Date: Wed, 20 Jul 1994 19:10:27 GMT In article , hewett at cfabuh.harvard.edu (Rob Hewett) writes: RH> How do you mount a CDROM from STSI's Digitized Sky Survey RH> on a Sun running either SunOS 4 or 5? This did not work RH> on OS 4: RH> mount -rt hsfs /dev/sr0 /cdrom First off, you need a patch under SunOS 4; the included software installation notes for the "getimage" software on the CDrom set has instructions on how to do this for various SunOS versions. Then, you will probably want to get some third party software/freeware to enable joe user to mount /cdrom, unless you want people to run the "getimage" software as root (not recommended). Here at NRAO I've used a utility called "cdmount" which is part of a general package of mounting programs called "mntdisk" written by Mike J. Fuller (NASA Lewis Research Center, mikef at sarah.lerc.nasa.gov according to the manual page). I'm willing to make the mntdisk.tar.Z file available via anon ftp if enough people want it. Note that the binaries made from this package need to be made set-uid to root. There are some hacks needed to get cdmount to be used instead of the "mount" command; I can provide these to you if needed. >From your further description (omitted here) it sounds like you might need the patches. - Pat -- -- ============================================================================= Patrick P. Murphy, Ph.D. Scientific Programming Analyst National Radio Astronomy Observatory pmurphy at nrao.edu 520 Edgemont Road Tel: (804) 296-0372 Charlottesville, VA 22903-2475 Fax: (804) 296-0278 web: http://orangutan.cv.nrao.edu/ "I don't believe in the no-win scenario" --- James T. Kirk ============================================================================= From fitsbits-request Thu Jul 21 15:51:31 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["11828" "Thu" "21" "July" "1994" "15:50:31" "EDT" "Eric Greisen" "egreisen at primate.cv.nrao.edu" "<9407211950.AA08937 at primate.cv.nrao.edu>" "241" "Units in FITS and the NOST Document" "^From:" nil nil "7" "1994072119:50:31" "Units in FITS and the NOST Document" (number " " mark " Eric Greisen Jul 21 241/11828 " thread-indent "\"Units in FITS and the NOST Document\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA25540; Thu, 21 Jul 94 15:51:31 EDT Return-Path: Message-Id: <9407211950.AA08937 at primate.cv.nrao.edu> From: egreisen at primate.CV.NRAO.EDU (Eric Greisen) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU, fits at stsci.edu Subject: Units in FITS and the NOST Document Date: Thu, 21 Jul 94 15:50:31 EDT Units and the Proposed NOST Standard for FITS Background ---------- FITS was designed with the philosophy that data transport reliability is enhanced if the number of options describing a data set are kept to a reasonable minimum. This "Occam's razor" philosophy was applied, for example, in the choice of only 3 binary formats originally allowable under FITS (8-bit unsigned and 16-bit and 32-bit signed integer). It was also applied in another crucial area: data units. FITS deliberately adopted a minimalist approach to units to simplify the standard and allow greatly simplified data transport. The correctness of this approach is suggested by the wide acceptance of FITS in the scientific community. The purpose of this document is to consider how the proposed NOST standard deals with units, and to outline how the proposed NOST standard should be changed in this area to enhance the usability of the standard. This enhancement will at the same time minimize the costs to the astronomical community of using the NOST standard. In brief, the proposed NOST standard needs to be strengthened in the way it deals with units. If it is not, then it is in danger of promulgating a standard which imposes an excessive burden on data consuming sites, and could result in a standard which is significantly less useful than it otherwise might be. Philosophy of a Data Transport System ------------------------------------- The primary purpose of the FITS standard was to allow straightforward sharing of scientific data, attempting to communicate not only the data bits, but also the meaning of those bits. In several areas, restrictions were used in the standard to provide a balance between the ease of writing FITS data and the ease of reading FITS data. In the selection of supported numerical formats, for example, FITS chose not to do the "general" thing, which would be to allow any number of bits with some keyword to define signed/unsigned or one's versus two's complement data storage. Such generality is not NEEDED to transfer the image data. Instead, some burden is placed, _where it belongs_, on the data-producing site which understands fully its internal formats and can easily provide a unique translation to a well-defined external format. A fully general system removes any burden from the data producing site and places all the burden on the data consuming site. The latter would not know what to expect and hence would have to attempt to provide generalized FITS readers that anticipate all possible combinations of the allowed parameters. In effect, a fully general standard is no standard at all. The important point is that data communication will fail if the format allows so many combinations that they cannot be reasonably anticipated IN ADVANCE. If a proposed standard is too general, data consuming sites we will then be forced to write innumerable specialized FITS readers: one for IRAF, another for high-energy, a third for Midas, and so on. If this situation occurs, the advantages of the standard will largely have been lost. The problem of the communication of data from the data producing sites has increased substantially since FITS was first proposed. Any change in the standard forces all data consuming sites to change their FITS readers or to be "left behind." These sites are not just those fully serviced by a few standard packages such as AIPS, IRAF, and FITSIO, but also numerous other packages involving tens or hundreds of programmers distributed all over the world (and over the years as well). We should not add complexity to the format for the sake of generality and we should NEVER compromise our ability to transmit data simply to make FITS more appealing for use as a format internal to current software packages. We should only add complexities when they are TRULY needed to convey the data and their meaning. Coordinates and Units in FITS ----------------------------- The original FITS paper skirted the issue of coordinates to avoid controversy which could have caused the community to reject the entire format out of hand. There was also concern over making errors in specifying the coordinates. While these concerns may have been valid, the original FITS document should have provided more guidance on how to convey MEANING along with image bits. As a result of the lack of clear specification in the standard, many FITS data sets have been written with a diverse set of coordinates, even when the keywords from the original papers were used. This is unfortunate, because doing science with accurate image bits and no coordinate information is problematic at best: To be told that the coordinate is 10 with an increment of 2.345679 is of no value whatsoever without the information that these are in meters! (For simplicity, I will use distance in all axis comments.) To try and avoid this situation, the original FITS paper stated, unambiguously, " IV. Units A. Basic set 1. Consistent with the International System of Units 2. Add degrees for angles 3. Add Janskys for flux B. Examples 1. Distances in meters 2. Masses in kilograms 3. Times in seconds 4. Temperatures in Kelvins " Note that the SI system of units for distance is meters alone, not millimeters, Megameters, etc. The choice not to include prefixes was a simple application of Occam's razor - since we allow exponential formats (and usually scaling parameters), there is no NEED for centimeters when meters are available. The presence of a dozen or more prefixes makes any attempt to determine units fraught with potential for failure. Coordinates and Units in the Proposed NOST Standard --------------------------------------------------- The proposed NOST standard has committed a serious error by not addressing the subject of units and imposing sufficient standards on units which may be used. It does not grandfather the original units into the standard, nor does it provide a replacement. Instead, it states "units defined in the IAU Style Manual are recommended". Thus, instead of defining some sort of units, the NOST document RECOMMENDS (which all readers interpret as providing the license to ignore) the use of IAU units which include 17 versions of meters! More seriously, the NOST document does not provide any mechanism by which responsible FITS writers may document which units have been selected. (Note that the IAU Style Manual is a set of recommendations for publishing manuscripts on paper for humans to read. It is not a standards document, nor does it address issues of computer to computer communication.) The importance of the NOST documents, and the importance of providing a clear, unambiguous standard, must not be underestimated. Programmers are reading the NOST documents, correctly interpreting words like "recommend" to mean "do what you like", and writing tapes which are spreading throughout the community. Any errors or omissions in the NOST standard may quickly become "extensions" to the standard simply by occurring on a large number of tapes. For example, because of an error in an early but well circulated DRAFT of the NOST document, a major software package began to (and may still) write tapes with the value field of a floating keyword surrounded with quotation marks. AIPS users now expect AIPS FITS readers to interpret this aberration correctly for evermore. How do we improve the NOST standards? After some considerable reflection, I propose the following: (1) We create another keyword called CUNITn to describe the units along each axis. To assume that the CTYPE can imply the units uniquely, even if only a pure SI is used, is naive. If CUNITn is not given, then a pure SI unit consistent with CTYPEn is the default. (2) We change every place that units are mentioned in the NOST document to refer to a new section of the document which will define the limited set of units which will be allowed. NOST compliant data sets will use those units and only those units. (3) In the new units section in the NOST document, we define: (a) units (e.g., values for CRVALn, CDELTn, TSCALn, et al.) are always double-precision so questions of underflow and overflow do not arise. (b) units are pure SI without character or numeric prefixes (how can we require people to change from miles and mils to meters if we refuse to change cm to meters? (c) some extensions to the units are allowed, such as degrees, parsecs, atomic units, barnes and the like. I prefer not to include simple extensions like "Hz" since (Occam's razor) "/s" is so simple. (d) the grammar of the units string be defined in a manner like that espoused in the document by Ian George and Lorella Angellini entitled "Specification of Physical Units within OGIP FITS Files". (My copy was dated 21-May-93; is there a more recent one with fewer ambiguities?) I would recommend strongly that a more restricted grammar be defined to forbid the constructs those authors deprecate but allow and to forbid "un-normalized" constructs (e.g. "m**4 m**(-3)" for "m" or "barn Gpc" which is about 3 liters). (e) state that some conventions on CTYPEn may be made which limit the range of units still further (e.g., primary image spherical coordinates may be restricted to degrees if the CTYPEn is stated using the conventions of the Greisen/Calabretta proposal). One reason for this possible limitation is that we have to define a lot of math in such conventions and the constants in the formulae depend on the units. (f) that we explicitly allow even more extensions to the units for ASCII tables which are representations of what appears in journals, while binary tables and main headers are computer-to-computer communications. (g) that we limit the length of units strings to some reasonable number of characters (e.g. 40). Even with the simplifications to the full OGIP suggestions given above, I do not expect most computer systems to understand units all of the time. Nevertheless, to define a reasonable string to be echoed in all displays of units, would be a considerable service to FITS users. To allow almost any unit to be used, without documentation, is a potentially dangerous position for the NOST document to take. Conclusion and Proposed Plan of Action -------------------------------------- The NOST standard needs to be strengthened in how it deals with units, and needs to provide a clear framework within which applications developers may work. The proposal outlined above is an attempt to provide a reasonable, workable compromise between the desire to be able to use arbitrary units and the necessity of providing a workable, well-defined standard. How should we proceed forward? The following actions should be taken: (1) Barry Schlesinger should be designated to suggest a change, with all the necessary detail and exact language, in the way the NOST document deals with units. The NOST committee should then consider and modify the suggestions as needed, using input from other interested parties, in the manner already used to create the current document. (2) The official adoption of the NOST standards should be briefly delayed while these issues are resolved. If this is not done, it will create potential problems and delays in the eventual adoption of the NOST standards by the IAU. ----------------------------------------------------------------------- Eric W. Greisen, Scientist National Radio Astronomy Observatory 520 Edgemont Road, Charlottesville, VA 22903. egreisen at nrao.edu ----------------------------------------------------------------------- From fitsbits-request Thu Jul 21 16:29:11 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["410" "Thu" "21" "July" "1994" "16:29:26" "-0400" "Ian M. George" "GEORGE at lheavx.gsfc.nasa.gov" "<940721162926.20205380 at lheavx.gsfc.nasa.gov>" "10" "OGIP/93-001: \"Specification of Physical Units within OGIP FITS Files\"" "^From:" nil nil "7" "1994072120:29:26" "OGIP/93-001: \"Specification of Physical Units within OGIP FITS Files\"" (number " " mark " Ian M. George Jul 21 10/410 " thread-indent "\"OGIP/93-001: \"Specification of Physical Units within OGIP FITS Files\"\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA25685; Thu, 21 Jul 94 16:29:11 EDT Return-Path: Message-Id: <940721162926.20205380 at lheavx.gsfc.nasa.gov> From: "Ian M George, Code 668, NASA/GSFC, USA (Usque ad mortem bibendum)" Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Cc: GEORGE at lheavx.gsfc.nasa.gov Subject: OGIP/93-001: "Specification of Physical Units within OGIP FITS Files" Date: Thu, 21 Jul 1994 16:29:26 -0400 (EDT) The latest version of the above document is actually dated 1993 Aug 06 and is available via anon ftp from legacy.gsfc.nasa.gov as: fits_info/fits_formats/docs/general/ogip_93_001.tex (LaTeX source) ogip_93_001.ps (Postscript) [Unfortunately a number of people are having difficulties printing the flavour of PS we seem to be making ... sorry, we're investigating] Ian M George HEASARC From fitsbits-request Mon Jul 25 12:44:25 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1557" "Mon" "25" "July" "1994" "12:43:50" "-0400" "BARRY M. SCHLESINGER" "BSCHLESINGER at nssdca.gsfc.nasa.gov" "<940725124350.22800320 at NSSDCA.GSFC.NASA.GOV>" "32" "The Role of the NOST in FITS standardization" "^From:" nil nil "7" "1994072516:43:50" "The Role of the NOST in FITS standardization" (number " " mark " BARRY M. SCHLESIN Jul 25 32/1557 " thread-indent "\"The Role of the NOST in FITS standardization\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA06227; Mon, 25 Jul 94 12:44:25 EDT Return-Path: Message-Id: <940725124350.22800320 at NSSDCA.GSFC.NASA.GOV> From: "BARRY M. SCHLESINGER" Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at NRAO.EDU, fits at stsci.edu Subject: The Role of the NOST in FITS standardization Date: Mon, 25 Jul 1994 12:43:50 -0400 (EDT) Eric Greisen has proposed a number of changes to the treatment of units in the NOST Definition of FITS. In preparing the NOST standard, the objective has always been to codify the existing standard as presented in the FITS papers, eliminating contradictions and ambiguities. It has not been to modify the international standard. In this context, the issue of units has been brought before the new Technical Panel because of some comments from the community that some prescriptions for units in the original FITS papers had not been reflected in the NOST Definition of FITS, despite the extensive community review of the drafts of the NOST Definition of FITS. The Greisen proposals, however, go beyond the original FITS papers. They include introduction of a new keyword (CUNITn) and prescriptions for the grammar and length of the unit strings. Such proposals go beyond the original FITS papers; they are changes to FITS. The mission of the Technical Panel has been to {\em document} FITS as endorsed by the IAU. While these issues deserve careful discussion, the NOST is not the body to act on them. The appropriate forums are the IAU FITS Working Group and the regional FITS committees. Anything included in the NOST standard must have support either in the FITS papers or in the community understanding of them as reflected in practice. The role of the NOST in the FITS process is to codify the rules, not to make them. Donald Sawyer NOST Secretary Barry Schlesinger FITS Support Office Coordinator From fitsbits-request Thu Jul 28 17:15:47 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["593" "" "26" "July" "1994" "15:44:45" "GMT" "Guy Rixon" "gtr at mail.ast.cam.ac.uk" "<313b1d$p2c at lyra.csx.cam.ac.uk>" "13" "Interpretion of BLANK" "^From:" nil nil "7" "1994072615:44:45" "Interpretion of BLANK" (number " " mark " Guy Rixon Jul 26 13/593 " thread-indent "\"Interpretion of BLANK\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA06674; Thu, 28 Jul 94 17:15:47 EDT Return-Path: Message-Id: <313b1d$p2c at lyra.csx.cam.ac.uk> Organization: Royal Greenwich Observatory Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!howland.reston.ans.net!pipex!lyra.csx.cam.ac.uk!cast0.ast.cam.ac.uk!gtr From: gtr at mail.ast.cam.ac.uk (Guy Rixon) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Interpretion of BLANK Date: 26 Jul 1994 15:44:45 GMT I have found an ambiguity in the NOST FITS-standard concerning the use of the keyword BLANK. NOST 100-1.0 says: "BLANK Keyword [...] The value field shall contain an integer that specifies the representation of array values whose physical values are undefined." This is clear enough if BZERO = 0 and BSCALE = 1, but we have problems with the case BZERO = +32K. Our (that is the Isaac Newton Group's) FITS writer assumes that the test for dud pixels is made after correcting for BSCALE/BZERO; several FITS readers make the opposite assumption. Is an authoritative ruling possible here? From fitsbits-request Thu Jul 28 17:49:23 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["895" "Thu" "28" "July" "1994" "17:49:09" "-0400" "Arnold Rots" "arots at xebec.gsfc.nasa.gov" "<199407282149.RAA02052 at xebec.gsfc.nasa.gov>" "20" "Re: Interpretion of BLANK" "^From:" nil nil "7" "1994072821:49:09" "Interpretion of BLANK" (number " " mark " Arnold Rots Jul 28 20/895 " thread-indent "\"Re: Interpretion of BLANK\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA06944; Thu, 28 Jul 94 17:49:23 EDT Return-Path: Message-Id: <199407282149.RAA02052 at xebec.gsfc.nasa.gov> From: Arnold Rots Sender: fitsbits-request at fits.CV.NRAO.EDU To: gtr at mail.ast.cam.ac.uk Cc: fitsbits at NRAO.EDU Subject: Re: Interpretion of BLANK Date: Thu, 28 Jul 1994 17:49:09 -0400 I think it's fairly obvious that the standard intends BLANK to represent the value *before* application of BZERO or BSCALE: if either or both of them were non-integer, one could not satisfy the requirement that BLANK have an integer value. Remember, BZERO and BSCALE were originally introduced to facilitate the transport of floating point numbers through integers (back in the days when there was no IEEE float standard, but two's complement was well- defined). This also points at the danger of using BZERO (and BSCALE) in situations were one wants to convert an integer to an integer (e.g., unsigned to signed): there is no protection against overflows. I know, the target data type is not defined in FITS (i.e., you're on your own), but common practice is different: the IN group obviously implicitly assumes that the numbers will be cast to integers after conversion. - Arnold Rots From fitsbits-request Thu Jul 28 17:48:59 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1141" "" "26" "July" "1994" "16:12:32" "GMT" "Malcolm Currie" "mjc at rlsaxp.bnsc.rl.ac.uk" "<313clg$1qvk at unixfe.rl.ac.uk>" "31" "Re: IMPORTANT: FITS data from INT/JKT" "^From:" nil nil "7" "1994072616:12:32" "IMPORTANT: FITS data from INT/JKT" (number " " mark " Malcolm Currie Jul 26 31/1141 " thread-indent "\"Re: IMPORTANT: FITS data from INT/JKT\"\n") "<313ag8$p2c at lyra.csx.cam.ac.uk>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA06935; Thu, 28 Jul 94 17:48:59 EDT Return-Path: Message-Id: <313clg$1qvk at unixfe.rl.ac.uk> Organization: Rutherford Appleton Laboratory Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!howland.reston.ans.net!swrinde!pipex!lyra.csx.cam.ac.uk!warwick!ral!rlsaxp.bnsc.rl.ac.uk!mjc References: <3135fa$lu3 at lyra.csx.cam.ac.uk> <313ag8$p2c at lyra.csx.cam.ac.uk> Reply-To: cur at star.rl.ac.uk From: mjc at rlsaxp.bnsc.rl.ac.uk (Malcolm Currie) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: IMPORTANT: FITS data from INT/JKT Date: 26 Jul 1994 16:12:32 GMT In article <313ag8$p2c at lyra.csx.cam.ac.uk>, gtr at mail.ast.cam.ac.uk (Guy Rixon) writes: From: gtr at mail.ast.cam.ac.uk (Guy Rixon) Subject: Re: IMPORTANT: FITS data from INT/JKT Organization: Royal Greenwich Observatory In article <3135fa$lu3 at lyra.csx.cam.ac.uk>, nrt at mail.ast.cam.ac.uk (Nial Tanvir) writes: |> |> [There] is that a spurious [FITS descriptor] in the [files of the FITS tapes !> produced at the INT and JKT] which is telling |> the fits reader that all occurrances of 32768 are bad pixel |> values. > Not quite true. BLANK = 0 is saying that any pixel-values of zero are non-physical > (which is true for any CCD that's working correctly). The FITS reader is assuming > that the test against BLANK can be made before adjusting the pixels according to BZERO and > BSCALE. The FITS writer is making the opposite assumption. You took the words right out of my mouth. > The FITS standard (NOST 100-1.0) > doesn't make it clear which assumption is legitimate. The NOST FITS User's Guide does. "The value of BLANK is the one that appears in the actual data array, without scaling." Malcolm Currie Starlink Project From fitsbits-request Thu Jul 28 18:54:07 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1225" "" "26" "July" "1994" "17:30:19" "GMT" "Guy Rixon" "gtr at mail.ast.cam.ac.uk" "<313h7b$rkt at lyra.csx.cam.ac.uk>" "23" "Re: Interpretion of BLANK" "^From:" nil nil "7" "1994072617:30:19" "Interpretion of BLANK" (number " " mark " Guy Rixon Jul 26 23/1225 " thread-indent "\"Re: Interpretion of BLANK\"\n") "<313b1d$p2c at lyra.csx.cam.ac.uk>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA07258; Thu, 28 Jul 94 18:54:07 EDT Return-Path: Message-Id: <313h7b$rkt at lyra.csx.cam.ac.uk> Organization: Royal Greenwich Observatory Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!howland.reston.ans.net!pipex!lyra.csx.cam.ac.uk!cast0.ast.cam.ac.uk!gtr References: <313b1d$p2c at lyra.csx.cam.ac.uk> From: gtr at mail.ast.cam.ac.uk (Guy Rixon) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: Interpretion of BLANK Date: 26 Jul 1994 17:30:19 GMT In article <313b1d$p2c at lyra.csx.cam.ac.uk>, gtr at mail.ast.cam.ac.uk (Guy Rixon) writes: |> I have found an ambiguity in the NOST FITS-standard concerning the use of the keyword BLANK. |> |> NOST 100-1.0 says: |> |> "BLANK Keyword [...] The value field shall contain an integer that specifies the representation of array |> values whose physical values are undefined." |> |> This is clear enough if BZERO = 0 and BSCALE = 1, but we have problems with the case BZERO = +32K. |> Our (that is the Isaac Newton Group's) FITS writer assumes that the test for dud pixels is made after |> correcting for BSCALE/BZERO; several FITS readers make the opposite assumption. |> |> Is an authoritative ruling possible here? It has been pointed out, in another group, that that NOST 100-1.0 defines "array value" as the actual number written for a pixel/voxel in the FITS file, without any processing to get the physical value. The NOST "FITS user guide" (which I haven't seen an advert for; I'll check the FAQ for this group next time it comes round) makes it even clearer: "The value of BLANK is the one that appears in the actual data array, without scaling." Thanks to Martin Oldfield and Malcolm Currie for clearing this up. From fitsbits-request Thu Jul 28 21:26:21 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["923" "" "26" "July" "1994" "20:30:30" "GMT" "William Thompson" "thompson at orpheus.gsfc.nasa.gov" "<313rp6$qf3 at paperboy.gsfc.nasa.gov>" "22" "Re: Interpretion of BLANK" "^From:" nil nil "7" "1994072620:30:30" "Interpretion of BLANK" (number " " mark " William Thompson Jul 26 22/923 " thread-indent "\"Re: Interpretion of BLANK\"\n") "<313b1d$p2c at lyra.csx.cam.ac.uk>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA07370; Thu, 28 Jul 94 21:26:21 EDT Return-Path: Message-Id: <313rp6$qf3 at paperboy.gsfc.nasa.gov> Organization: NASA Goddard Space Flight Center -- InterNetNews site Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!news.umbc.edu!haven.umd.edu!ames!newsfeed.gsfc.nasa.gov!orpheus.gsfc.nasa.gov!thompson References: <313b1d$p2c at lyra.csx.cam.ac.uk> From: thompson at orpheus.gsfc.nasa.gov (William Thompson) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: Interpretion of BLANK Date: 26 Jul 1994 20:30:30 GMT gtr at mail.ast.cam.ac.uk (Guy Rixon) writes: >I have found an ambiguity in the NOST FITS-standard concerning the use of the keyword BLANK. >NOST 100-1.0 says: >"BLANK Keyword [...] The value field shall contain an integer that specifies the representation of array >values whose physical values are undefined." >This is clear enough if BZERO = 0 and BSCALE = 1, but we have problems with the case BZERO = +32K. >Our (that is the Isaac Newton Group's) FITS writer assumes that the test for dud pixels is made after >correcting for BSCALE/BZERO; several FITS readers make the opposite assumption. >Is an authoritative ruling possible here? I don't think it's ambiguous. The definition of BLANK says that it's an *integer*. In the most general case, once you have applied BZERO and BSCALE, then you have a floating point number. Hence, the test for dud pixels is made *before* applying BZERO/BSCALE. Bill Thompson