From heafits@legacy.gsfc.nasa.gov Thu Apr 1 11:10:27 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["2005" "Thu" "1" "April" "93" "11:09:50" "EST" "Bruce O'Neel" "oneel@arupa.gsfc.nasa.gov " nil "55" "Test and current heafits subscribers" "^From:" nil nil "4"]) Return-Path: Received: from arupa.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA11320; Thu, 1 Apr 93 11:10:21 EST Received: from (localhost.gsfc.nasa.gov) by arupa.gsfc.nasa.gov (4.1/1.35) id AA07749; Thu, 1 Apr 93 11:09:51 EST Message-Id: <9304011609.AA07704@arupa.gsfc.nasa.gov> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: oneel@arupa.gsfc.nasa.gov (Bruce O'Neel) Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: Test and current heafits subscribers Date: Thu, 1 Apr 93 11:09:50 EST Hi, This is a test to see how well the heafits mailing list works. The address to send mail to have it distributed to the whole list is: heafits@legacy.gsfc.nasa.gov The address to send additions/deletions/changes is listserv@legacy.gsfc.nasa.gov This is a listserv managed mailing list. This means that the processing of both requests and mailings is automatic. I'm still working with the listserv software so there may be some glitches for the next few days. Here is the current list of subscribers: ONEEL@LEGACY.GSFC.NASA.GOV Bruce Oneel pence@tetra.gsfc.nasa.gov Bill Pence zhang@qjet.bu.edu Yun Fei Zhang KUIN@nssdca.gsfc.nasa.gov N Paul Kuin dwells@fits.cv.nrao.edu Don Wells carolc@cea.berkeley.edu Carol Christian W3WHW@gibbs.gsfc.nasa.gov Wayne Warren ericco@cea.berkeley.edu Eric Olson leb@hypatia.gsfc.nasa.gov Lee Brotzman rnishim@c1.mtk.nao.ac.jp Shiro Nishimura bernlohr@eu6.mpi-hd.mpg.de Konrad Bernloehr forveill@gag.observ-gr.fr Thierry Forveille CGP@starlink.leicester.ac.uk Clive Page gibby@ssl.msfc.nasa.gov Gibby eric@cfa242.harvard.edu Eric Mandel levenson@unhsmm.unh.edu Ken Levenson MCGLYNN@grovx1.gsfc.nasa.gov Thomas McGlynn perry@lheavx.gsfc.nasa.gov Brendan Perry edgar@dxs.ssec.wisc.edu Richard J. Edgar ONEEL@ARUPA.GSFC.NASA.GOV Bruce Oneel Total number of subscribers: 21 -- "It's a nitwit idea. Nitwit ideas are for emergencies. The rest of the time you go by the Book, which is mostly a collection of nitwit ideas that worked." The Mote in God's Eye, Niven and Pournelle. From heafits@legacy.gsfc.nasa.gov Fri Apr 9 15:26:19 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["4226" "Fri" "9" "April" "93" "15:25:34" "-0400" "William Pence" "pence@tetra.gsfc.nasa.gov " nil "85" "heafits is open for business" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA23527; Fri, 9 Apr 93 15:26:16 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA06435; Fri, 9 Apr 93 15:25:34 -0400 Message-Id: <9304091925.AA23521@tetra.gsfc.nasa.gov> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: pence@tetra.gsfc.nasa.gov (William Pence) Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: heafits is open for business Date: Fri, 9 Apr 93 15:25:34 -0400 It appears that there are now sufficient members subscribed to the heafits mail exploder to make a meaningful discussion possible on issues related to FITS files in High Energy Astrophysics. This is an unmoderated group, so the floor is open to any and all... Heafits is an automated listserver which provides a number of useful services in addition to just sending messages out to other members of the list. To get information on these services, send a one line message to listserv@legacy.gsfc.nasa.gov (NOTE: don't send this to heafits!) where the body of the message contains the one word: help For example, there are commands that will return to you a list of all current subscribers, or commands to get a list (or a copy) of any previous messages that were posted to this listserver. Attached below is the current list of subscribers to heafits. Regards, Bill Pence -------------------------------------------------------------------------------- Here is the current list of subscribers: ONEEL@LEGACY.GSFC.NASA.GOV Bruce Oneel pence@tetra.gsfc.nasa.gov Bill Pence zhang@qjet.bu.edu Yun Fei Zhang KUIN@nssdca.gsfc.nasa.gov N Paul Kuin dwells@fits.cv.nrao.edu Don Wells carolc@cea.berkeley.edu Carol Christian W3WHW@gibbs.gsfc.nasa.gov Wayne Warren ericco@cea.berkeley.edu Eric Olson leb@hypatia.gsfc.nasa.gov Lee Brotzman rnishim@c1.mtk.nao.ac.jp Shiro Nishimura bernlohr@eu6.mpi-hd.mpg.de Konrad Bernloehr forveill@gag.observ-gr.fr Thierry Forveille CGP@starlink.leicester.ac.uk Clive Page gibby@ssl.msfc.nasa.gov Gibby eric@cfa242.harvard.edu Eric Mandel levenson@unhsmm.unh.edu Ken Levenson MCGLYNN@grovx1.gsfc.nasa.gov Thomas McGlynn perry@lheavx.gsfc.nasa.gov Brendan Perry edgar@dxs.ssec.wisc.edu Richard J. Edgar XREAG@ELDYN.GSFC.NASA.GOV Emily Greene JMORAN@CFA253.HARVARD.EDU John Moran RJB@KOALA.HARVARD.EDU Roger Brissenden GAETZ@CFA259.HARVARD.EDU Terry Gaetz FAP@CFA239.HARVARD.EDU Francis A. Primini NOLIVERSEN@NDADS.DNET.NASA.GOV Nancy Oliversen ONEEL@ARUPA.GSFC.NASA.GOV Bruce Oneel PLN@EGRET1.STANFORD.EDU Patrick Nolan RWHITE@DFTNIC.GSFC.NASA.GOV Richard A. White LANDSMAN@SORBET.DNET.NASA.GOV Landsman JHC@CFA207.HARVARD.EDU J. H. Chappell RUTLEDGE@SPACE.MIT.EDU Robert Rutledge FRH@CFA235.HARVARD.EDU Rick Harnden CGM0@LEHIGH.EDU George E. Mccluskey Jr. @DMSWWU1A.UNI-MUENSTER.DE:MICHAEL@AQUILA.UNI-MUENSTER.DE Michael Naumann SIMON@CFA255.HARVARD.EDU Rich Simon BSCHLESINGER@NSSDCA.GSFC.NASA.GOV Barry Schlesinger Nssdca.gsfc.nasa.gov HANISCH@STSCI.EDU Bob Hanisch GARCIA@CFA219.HARVARD.EDU Michael Garcia RPAYNE@SADIRA.GB.NRAO.EDU Bob Payne HSIEH@HUGH.HARVARD.EDU Paul Hsieh WHITE@HEAGIP.GSFC.NASA.GOV Nick AROTS@XEBEC.GSFC.NASA.GOV Arnold Rots CORCORAN@NDADSB.GSFC.NASA.GOV Mike Corcoran NELSON@AXL.STSCI.EDU Nelson Zarate KEMPER@NSSDCA.GSFC.NASA.GOV Kemper BARHAM@ALEXIS.LANL.GOV Barham W. Smith ESCHMERLING@NASAMAIL.NASA.GOV Erwin Schmerling LUCIO@POSEIDON.IFCTR.MI.CNR.IT Lucio Chiappetti JDEWEES@PRISM.NMT.EDU Jdeweesprism.nmt.edu NOUSEK@ASTRO.PSU.EDU John Nousek Total number of subscribers: 50 From heafits@legacy.gsfc.nasa.gov Wed Apr 14 20:21:07 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil t nil nil nil nil] ["1449" "Wed" "14" "April" "93" "20:20:18" "-0400" "Usque ad mortem bibendum" "GEORGE@heavax.gsfc.nasa.gov (Ian M George, Code 668, NASA/GSFC, USA )" "<930414201642.2280c4ac@heavax.gsfc.nasa.gov>" "34" "OGIP/93-001 - OGIP memo on the standard strings for physical units" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA03555; Wed, 14 Apr 93 20:21:05 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA04018; Wed, 14 Apr 93 20:20:18 -0400 Message-Id: <930414201642.2280c4ac@heavax.gsfc.nasa.gov> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: GEORGE@heavax.gsfc.nasa.gov (Ian M George, Code 668, NASA/GSFC, USA (Usque ad mortem bibendum)) Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: OGIP/93-001 - OGIP memo on the standard strings for physical units Date: Wed, 14 Apr 93 20:20:18 -0400 FITS gurus ... Lorella & I have put together a list of standard strings & rules we would like to see adopted OGIP-wide regarding the specification of physical units within FITS files. These have been written up as a [draft] OGIP memo (# OGIP/93-001), and copies circulated to: - a number of OGIP people in Bldg 2 directly - Don Jennings via GSFC internal mail (3 copies for CGRO SSC) - Barry Schlesinger via GSFC internal mail If you have not (or are not about to get) copies, but would like one, please see/e-mail me. .. During it's construction we were primarily thing about TUNITnnn keyword values, but there is no reason why the same strings can't be used elsewhere if necessary. We welcome all comments, suggestions & additions excluding those bemoaning the passing of imperial & c.g.s. units (after much debate, we decided that it was easiest [& most consistent] mainly to adopt the recommendations of the IAU). Ian M George HEASARC PS: Yes, thank you, we already know of most of the typos... NOTE ADDED IN PROOF ------------------- Since I photocopied & distributed the current draft, Bill Pence has made the very valid point that if we change the string for the (little used) unit of siemens from "S" to "siemens" (Table 1), then there is no case sensitivity amongst the currently listed strings. Since this has obvious s/w advantages, I fully support this, and the draft should be considered to have been ammended so. From heafits@legacy.gsfc.nasa.gov Wed Apr 14 21:11:16 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["888" "Wed" "14" "April" "93" "21:10:34" "-0400" "Don Wells" "dwells@fits.CV.NRAO.EDU " nil "17" "Re: OGIP/93-001 - OGIP memo on the standard strings for physical units" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA03621; Wed, 14 Apr 93 21:11:15 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA04353; Wed, 14 Apr 93 21:10:34 -0400 Message-Id: <9304150110.AA03618@fits.cv.nrao.edu> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: dwells@fits.CV.NRAO.EDU (Don Wells) Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: Re: OGIP/93-001 - OGIP memo on the standard strings for physical units Date: Wed, 14 Apr 93 21:10:34 -0400 Usque ad mortem bibendum writes: > ... have.. list.. of standard.. physical units within FITS files.. > [draft] OGIP memo (# OGIP/93-001).. after much debate, .. decided.. > easiest [& most consistent] .. adopt recommendations of the IAU.. I applaud this action, which is consistent with the FITS tradition which Eric Greisen and I started in 1979 when we specified use of SI units in FITS files. Please stand firm on this when the inevitable criticism comes. Your actions will set a good example for the rest of the FITS community and will, in the long run, pay handsome dividends for science. Donald C. Wells Associate Scientist dwells@nrao.edu National Radio Astronomy Observatory +1-804-296-0277 520 Edgemont Road Fax= +1-804-296-0278 Charlottesville, Virginia 22903-2475 USA 78:31.1W, 38:02.2N From heafits@legacy.gsfc.nasa.gov Thu Apr 15 10:07:06 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1415" "Thu" "15" "April" "93" "10:06:18" "-0400" "Lee E. Brotzman" "leb@hypatia.gsfc.nasa.gov " nil "28" "Re: OGIP/93-001 - OGIP memo on the standard strings for physical units" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA05637; Thu, 15 Apr 93 10:07:05 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA06566; Thu, 15 Apr 93 10:06:18 -0400 Message-Id: <9304151350.AA29582@Hypatia.gsfc.nasa.gov> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: leb@hypatia.gsfc.nasa.gov (Lee E. Brotzman) Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: Re: OGIP/93-001 - OGIP memo on the standard strings for physical units Date: Thu, 15 Apr 93 10:06:18 -0400 > ... Lorella & I have put together a list of standard strings & rules we > would like to see adopted OGIP-wide regarding the specification of physical > units within FITS files. These have been written up as a [draft] OGIP memo > (# OGIP/93-001), and copies circulated to: ... > If you have not (or are not about to get) copies, but would like one, > please see/e-mail me. There wasn't an e-mail address on the signature, so I'll ask publically. Please send me this draft. I have published 100 astronomical catalogs in FITS on CD-ROM and used a consistent set of units throughout, following the guidelines in the IAU Style guide. It would be interesting to see what you have come up with. I have also put in a proposal for ADP funds to extend the FITS formatting of ADC catalogs to an additional 150 or so catalogs and I fully intend to use the same unit and field name conventions I used on the CD-ROM. The first product (assuming of course that I get the funding) will be a data dictionary containing all of the common TTYPE and TUNIT keyword values I plan to use. That will be publically available over the Internet within the first month or two of starting work (say, late this summer). -- -- Lee E. Brotzman Internet: leb@hypatia.gsfc.nasa.gov -- Hughes STX DECNET: NDADSA::BROTZMAN -- NASA Goddard Space Flight Center BITNET: ZMLEB@GIBBS From heafits@legacy.gsfc.nasa.gov Thu Apr 15 11:05:30 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1799" "Thu" "15" "April" "93" "11:04:42" "-0400" "Lucio Chiappetti" "lucio@ifctr.mi.cnr.it" nil "37" "starting the discussion" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA05680; Thu, 15 Apr 93 11:05:29 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA08002; Thu, 15 Apr 93 11:04:42 -0400 Message-Id: <9304151504.AA08002@legacy.gsfc.nasa.gov> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: Lucio Chiappetti Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: starting the discussion Date: Thu, 15 Apr 93 11:04:42 -0400 Being one of those who solicited to set up this list (although I delayed subscribing because of some local rearrangements of our mail system ... but I am being told I haven't missed anything), I'd like to make a proposal to start with. As some of you perhaps remember, I raised originally a question over WGAS and FITSBITS about what is the "OGIP FITS" standard, and what is its scope, and how standard approach are best handled for different missions in different countries and agencies. Perhaps a few people at GSFC and elsewhere should post to HEAFITS : - some general purpose statement about OGIP FITS and alike - some mission specific description of plans about data formats and structures (I was sent privately by Mike Corcoran a description about ROSAT, which I feel could be an useful "template", and I encourage to be publicly posted). Starting from that everybody (myself included) could send in a similar description. This way we could state what are common areas worth a standard approach, and what are not. -------------------------------------------------------------------------- "The value of a dialogue depends mostly on the difference of concurrent opinions. If Babel tower were not there, we should have invented it." Karl R. Popper -------------------------------------------------------------------------- Lucio Chiappetti - IFCTR/CNR Milano | U N I C U I Q U E via Bassini 15 - I-20133 Milano - Italy | System Manager Internet: LUCIO@IFCTR.MI.CNR.IT | FORTUNAE SUAE Decnet: IFCTR::LUCIO (39610::LUCIO) | Bitnet: | ----------------------------------------------------------------------------- From heafits@legacy.gsfc.nasa.gov Thu Apr 15 11:52:38 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["5511" "Thu" "15" "April" "93" "11:51:55" "-0400" "Usque ad mortem bibendum" "GEORGE@heagip.gsfc.nasa.gov (Ian M George, Code 668, NASA/GSFC, USA )" nil "149" "Brief Summary of OGIP/93-001 (Physical units in FITS files)" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA05706; Thu, 15 Apr 93 11:52:36 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA08716; Thu, 15 Apr 93 11:51:55 -0400 Message-Id: <930415114900.220023c0@HEAGIP.GSFC.NASA.GOV> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: GEORGE@heagip.gsfc.nasa.gov (Ian M George, Code 668, NASA/GSFC, USA (Usque ad mortem bibendum)) Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: Brief Summary of OGIP/93-001 (Physical units in FITS files) Date: Thu, 15 Apr 93 11:51:55 -0400 Since Lorella & I inadvertantly opened a world-wide Pandora's box with our message to the heafits exploder yesterday, below we breifly list the proposed strings to specify physical units within FITS files. A couple of the (more constructive, physically plausible) suggestions recieved so far are tacked on the end. The memo itself is available via anonymous ftp on legacy.gsfc.nasa.gov (128.183.8.233) in the .caldb/docs/memos directory: ogip_93_001.tex is the LaTeX source ogip_93_001.ps is a PS version of the complete doc. (Note, I'm having some difficulty with PS "standards" (ha !) at present, thus unfortunately I can't guarentee it will print on all printers) Constructive comments are welcomed (I suggest to heafits)... .... Let the Games begin Ian M George HEASARC (george@lheavx.gsfc.nasa.gov) BRIEF OVERVIEW -------------- In the memo we give 3 tables of proposed strings: Table 1 for IAU-recommended units Table 2 for addition allowed units convenient for astronomy Table 3 for further allowed (but discouraged) convenient units It should be noted that wherever possible made the string the same as the corresponding IAU-recommended symbol. Notable exceptions are when the symbol is not part of the standard ASCII character set or when it would lead to potential upper/lower case confusion with another string. The final section of the memo then discusses proposed rules for the construction of compound units. TABLE 1 - Highly recommended --------------------------------------------------------------------------- Quantity & Proposed & Meaning & String --------------------------------------------------------------------------- (SI base & supplementary units) length & 'm' & metre mass & 'kg' & kilogram time & 's' & second of time (incl sidereal time) plane angle & 'rad' & radian solid angle & 'sr' & steradian temperature & 'K' & kelvin electric current & 'A' & ampere amount of substance & 'mol' & mole luminous intensity & 'cd' & candela (IAU-recognized derived units) frequency & 'Hz' & hertz energy & 'J' & joule power & 'W' & watt electric potential & 'V' & volts force & 'N' & newton pressure, stress & 'Pa' & pascal electric charge & 'C' & coulomb electric resistence & 'ohm' & ohm electric conductance & 'siemen' & siemens electric capacitance & 'F' & farad magnetic flux & 'Wb' & weber magnetic flux density & 'T' & tesla inductance & 'H' & henry luminous flux & 'lm' & lumen illuminance & 'lx' & lux --------------------------------------------------------------------------- TABLE 2 - Also Recommended --------------------------------------------------------------------------- Quantity & Proposed & Meaning & String --------------------------------------------------------------------------- plane angle & 'deg' & degree of arc energy & 'keV' & kiloelectron volt & 'erg' & erg length & 'angstrom' & angstrom events & 'count' & (detected) counts flux & 'mag' & (stellar) magnitude area & 'pixel' & (image/detector) pixel --------------------------------------------------------------------------- TABLE 3 Allowed, but discouraged --------------------------------------------------------------------------- Quantity & Proposed & Meaning & String --------------------------------------------------------------------------- time & 'us' & microsecond (note the "u") & 'ms' & millisecond & 'min' & minute (incl sidereal time) & 'h' & hour (incl sidereal time) & 'd' & day & 'yr' & year (Julian) plane angle & 'arcsec' & second of arc & 'arcmin' & minute of arc length & 'um' & micrometre (or micron - note the "u") & 'mm' & millimetre & 'cm' & centimetre & 'km' & kilometre & 'AU' & astronomical unit & 'lyr' & light year & 'pc' & parsec area & 'barn' & barn energy & 'eV' & electron volt & 'Mev' & megaelectron volts & 'GHz' & gigahertz mass & 'g' & gram events & 'photon' & (detected) photons temperature & 'C' & degrees celsius flux density & 'Jy' & jansky & 'mCrab' & millicrab (yuk, yuk, yuk) magnetic field & 'gauss' & gauss (note not using 'G') --------------------------------------------------------------------------- RULES FOR CONSTRUCTION OF COMPOUND UNITS ---------------------------------------- The rules for the construction of compound units are fairly straightforward, involving standard characters preceeding or following the string specifying the units. For a unit string $ustr$: o ' $ustr$' (with one or more preceeding spaces) denotes multiplied by $ustr$ o '/$ustr$' denotes divided by $ustr$ o '$ustr$**y denotes $ustr$ raised to the power y However, to minimize the risk of ambiguities, and to simplify the task of parsing the whole string giving the compound unit, the following rules are also required: - all the multiplicative units within the compound unit should be specified first - all units which are being divided by should have a preceeding '/' (slash) - raising units to a negative power should not be used, hence '/$ustr$**z should be used rather than '$ustr$**-z (for z>0) COMMENTS SO FAR --------------- 1) Bill Pence suggests: that multiplicative units should be separated by an * (asterix) eg m * s /Hz for metre seconds per hertz 2) Paul Barrett suggests: a) the promotion of eV to Table 2, b) the demotion of keV to Table 3, c) the inclusion of GeV in Table 3 From heafits@legacy.gsfc.nasa.gov Thu Apr 15 12:05:03 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["2277" "Thu" "15" "April" "93" "12:04:15" "-0400" "\"Clive Page, Leicester University, UK\"" "CGP@starlink.leicester.ac.uk" nil "42" "Physical units etc" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA05734; Thu, 15 Apr 93 12:05:02 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA09075; Thu, 15 Apr 93 12:04:15 -0400 Message-Id: <9304151604.AA09075@legacy.gsfc.nasa.gov> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: "Clive Page, Leicester University, UK" Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: Physical units etc Date: Thu, 15 Apr 93 12:04:15 -0400 I've now read the OGIP draft memo on strings for physical units written by Ian George and Lorella Angelini, and think their recommendations are quite sensible. I agree that it's desirable to avoid case-sensitivity when possible, but changing "S" to "siemen" to avoid confusion with "s" for "seconds" is perhaps going a bit far, given the infrequency with which units of electrical conductivity occur in FITS files. One think I particularly like is their recommendation that radians (string "rad") be preferred for angular coordinates (leaving degrees (string "deg") only as an alternative allowed form. I'd like to deprecate especially the use of hours-mins-secs and degree-arcmins-arcsecs for internal storage in FITS files. Although we are very grateful for the work on NSSDC/ADC in producing the CDs called "NASA Selected Astronomical Catalogs", it has to be said that the ease of use of these discs was greatly reduced by the fact that different tables have celestial coordinates given in such a wide variety of different ways. Thus some RAs are in hours and decimals, some in hours and minutes and decimals, some in hours minutes and seconds, and so on. In most cases the different parts of the same value are in different FITS columns. Although it is not to hard to work out how to read any single catalogue, it is extremely hard to devise software that can search through a set of such catalogs to list objects in a given part of the sky. It is of course true that astronomers commonly think (and even work) in sexagesimal units, and may prefer to have coordinates displayed on their computer terminals in this form. But it is easy enough to get the software to do the translation both ways. In my view we ought to construct FITS files using simple and relatively efficient internal forms and let the software do all conversions to/from external forms. In the same way that we use IEEE number formats for internal storage of floating-point values but convert them all to the familiar decimal number base on output, I feel that we ought to store angular coordinates internally in radians and only convert to sexagesimal format on output. Clive Page, Dept of Physics & Astronomy, University of Leicester, UK. DECNET: 19383::CGP Internet: cgp@star.le.ac.uk From heafits@legacy.gsfc.nasa.gov Thu Apr 15 12:59:50 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1792" "Thu" "15" "April" "93" "12:59:08" "-0400" "Lucio Chiappetti" "lucio@ifctr.mi.cnr.it" nil "36" "Re: Brief Summary of OGIP/93-001 (Physical units in FITS files)" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA05755; Thu, 15 Apr 93 12:59:49 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA09894; Thu, 15 Apr 93 12:59:08 -0400 Message-Id: <9304151659.AA09894@legacy.gsfc.nasa.gov> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: Lucio Chiappetti Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: Re: Brief Summary of OGIP/93-001 (Physical units in FITS files) Date: Thu, 15 Apr 93 12:59:08 -0400 Sorry Ian, your Postscript file looks ... unviewable. I did not even try to print it, I just looked at it with a previewer and gave me an offending command close to the beginning : %%BeginSetup BeginDviLaserDoc 300 300 RES %%EndSetup It looks like you call some external file or procedure which obviously defines some macros which are unknown to my previewer (DEC CDA Viewer). If you could perhaps make such macros available, perhaps anybody could then print the file. It looks quite strange to me anyhow (lot of binary stuff). I usually produce Postscript file from a Mac, in such case there is a HUGE printer-dependent prolog on top of the file, which makes the file very large, but as far as I know anybody could print it also on non-Apple printers. (Usually the problems I have encountered so far in exporting .ps files were due to the fact there were VMS machines at the other end. They often get confused by a file with very long records. Usually the trick is to ftp-binary the file onto a Vax : this forces records to 512 - which is manageable - while leaving the content intact and usable also by Unix machines. Of course this has nothing to do with the problem of YOUR file) ----------------------------------------------------------------------------- A member of G.ASS : Group for Astronomical Software Support ----------------------------------------------------------------------------- Lucio Chiappetti - IFCTR/CNR Milano | U N I C U I Q U E via Bassini 15 - I-20133 Milano - Italy | System Manager Internet: LUCIO@IFCTR.MI.CNR.IT | FORTUNAE SUAE Decnet: IFCTR::LUCIO (39610::LUCIO) | Bitnet: | ----------------------------------------------------------------------------- From heafits@legacy.gsfc.nasa.gov Thu Apr 15 13:23:49 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["6867" "Thu" "15" "April" "93" "13:23:01" "-0400" "CORCORAN@ndadsb.gsfc.nasa.gov" "CORCORAN@ndadsb.gsfc.nasa.gov" nil "161" "OGIP Rationalized Data Files" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA05771; Thu, 15 Apr 93 13:23:47 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA10675; Thu, 15 Apr 93 13:23:01 -0400 Message-Id: <930415132308.21206d47@ndadsb.gsfc.nasa.gov> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: CORCORAN@ndadsb.gsfc.nasa.gov Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: OGIP Rationalized Data Files Date: Thu, 15 Apr 93 13:23:01 -0400 Below is a copy of the e-mail I previously sent to Lucio about the data file structures that the OGIP has developed (with input from CFA) for archiving/distribution of data from high energy astrophysics missions. Mike Corcoran ----------------------Begin Include Message---------------------------- From: NDADS::CORCORAN 15-MAR-1993 10:52:21.73 To: SMTP%"lucio@ifctr.mi.cnr.it" CC: CORCORAN Subj: OGIP FITS conventions Hello Lucio, I received your e-mail concerning the OGIP FITS conventions and would like to give you some background concerning the OGIP file formats and respond to your questions. The OGIP has begun development of a "standard" set of file formats for distribution and archiving of data from high energy astrophysics missions. These formats are called "rationalized data file" (RDF) formats. The basic specifications for these files are 1) the files follow the standard FITS conventions for images and extensions (due to this requirement the rationalized data files are sometimes called "rationalized FITS files") 2) under the rationalized scheme, the data is grouped into 4 categories: a) BASIC data - this is the bare-bones science data, like good time intervals, photon event lists, and statuses. In other words, BASIC data is the information a user would need to perform scientific analysis. b) ANCILLARY data - data which is not expressly needed for standard science analysis but which is in general useful. This data includes engineering or housekeeping data, and spacecraft pointing (aspect) information. c) CALIBRATION data - data which should be available to the user to describe how the instrument is calibrated. This information includes detector response matrices, mirror effective areas, and point spread functions. d) DERIVED data - data which is derived from standard processing from the BASIC science information. This includes images, spectra, lightcurves, source lists, etc. 3) The OGIP has developed keywords and FITS table column names for use in the rationalized files. It is suggested that whenever possible projects should use defined keywords or table column names for description of similar quantities. However in the interest of accomodating a wide variety of missions these lists are expandable. 4) File formats should agree with previously defined OGIP formats and should be backward-compatible 5) Files should be self-documenting as much as practical. ROSAT is the first active mission to adopt the RDF format. Below I give you a rough outline of the ROSAT implementation of the RDF format: 1) for each observation, a BASIC file consisting of a FITS files with a NULL primary header and 4 binary table extensions giving a) good time intervals b) time intervals when the instrument was on, regardless of the quality flag settings c) a photon events list d) status histories 2) for each observation, an ANCILLARY file consisting of a FITS file with a NULL primary header and binary table extensions giving a) orbit information b) aspect information c) housekeeping information d) other instrument-specific information 3) for each observation, a CALIBRATION file consisting of a FITS file with a NULL primary header and binary table extensions giving a) effective areas b) response matrices c) instrument maps d) bad pixel locations e) etc. 4) for each observations, a set of DERIVED files consisting of FITS files with the following information a) FITS files with an energy selected image as the primary array and energy-selected source locations as binary table extensions b) FITS files with energy-selected backgrounds as the primary array c) A FITS file giving the total-band exposure map as the primary array d) A FITS file giving the source list and spectra as binary table extensions e) A FITS file giving a history of the data processing f) etc. The above is meant to give you some idea about how the data is structured in the RDF's for ROSAT. Individual considerations for other missions may require that the necessary information be divided up differently. For example, if a file grows too large it is ok to split the information up into individual files. One could envision that the BASIC file consisting of the status history and the event list may be too large, so it would be desireable to perhaps include the events list in a separate file. I'll send you along a list of keywords we've created for ROSAT, along with the binary table column names. If you want, I can send you header dumps for the files. Test versions of the ROSAT RDF files are available via anonymous FTP from legacy.gsfc.nasa.gov The test versions of the PSPC files are located in rosat/data/pspc/sample_files/ratfits while those for the HRI are located in rosat/data/hri/sample_files/ratfits Please beware that these files are developmental ONLY, and as such have a number of known (and unknown) BUGS, but will give you the general idea of how ROSAT is using the RDF format. In terms of your specific questions: > - Are these OGIP conventions a "private" convention of GSFC, or it is > intended to propose them as a draft standard in the traditional > FITS way ? These files follow the FITS standard so there are no plans to propose them as a draft standard in the traditional FITS way. > - Is the related documentation restricted in access or usage ? > And if not is it available electronically ? The documentation is not restricted, and I can send it to you if you so desire. I'm developing a ROSAT RDF spec which I can mail to you (at the moment it is not available electronically). I can e-mail you the basic specifications we're using to develop the files. Or you can ftp the files themselves from legacy. > - How stable are those conventions ? I mean, are they frozen ? Are > they liable to change duirng development dramatically ? etc. etc. For ROSAT the conventions are pretty stable, since we plan to start distributing data in this format by the end of April. The formats are meant to be flexible so as to be able to accomodate data from other missions, and some (hopefully minor) changes are foreseen. > - Are these conventions open for discussion in the context of future > missions in which European and Eastern countries may be involved > (e.g. SAX, Spektrum-RG, XMM etc.) ? Insofar as the impact on missions which have used older versions of the RDF formats is minimal, new changes can be added to the conventions so as to allow future missions to use the RDF. Like all FITS files, changes should be backward compatible. If you have any further questions please don't hesitate to contact me. Cheers, Mike Corcoran ROSAT Project Code 668 Goddard Space Flight Ctr Greenbelt, MD 20771 USA (301) 286-5576 From heafits@legacy.gsfc.nasa.gov Thu Apr 15 14:01:57 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["2745" "Thu" "15" "April" "93" "14:01:14" "-0400" "Jonathan McDowell" "jcm@urania.harvard.edu " nil "73" "" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA05837; Thu, 15 Apr 93 14:01:55 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA11484; Thu, 15 Apr 93 14:01:14 -0400 Message-Id: <9304151801.AA22894@urania.harvard.edu> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: jcm@urania.harvard.edu (Jonathan McDowell) Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: Date: Thu, 15 Apr 93 14:01:14 -0400 Some comments on Ian and Lorella's OGIP 93-001: 1. I would vote that Jy be promoted to table 2. It's as much a standard in astronomy as angstrom, nowadays. 2. I would prefer that SI prefixes in general should be permitted but discouraged at the level of table 3. (e.g. kpc, mJy). (I already have a subroutine that parses such prefixes, it's pretty trivial). 3. What about rescaling? Is 10**44 erg /s an acceptable unit? Otherwise you may have problems with overflowing real exponents. How about 10**(-26) W /m**2/Hz? 4. Negative powers in units. I actually feel that W m**(-2) Hz**(-1) is less ambiguous than W /m**2 /Hz which could be misinterpreted by someone really bloody minded to mean W/ (m**2/Hz) ie. W Hz /m**2 What about fractional powers, e.g sensitivities in W / Hz**0.5 or W / Hz**(1/2) (something I never really understand right anyway!) 5. I note that your argument against the use of mCrab applies precisely in the same way to mag (reading Vega for Crab) (although I agree they are both in the right tables). I also draw to your attention the standard unit of the diffuse background radiation, the S10(V) (surface brightness of a V=10 mag G2V star), which happily I have seen less of recently. 6. Pixel is not just a unit of area, it is a linear unit labelling each dimension of an n-dimensional array. I have also seen 'square pixels' as the unit of area. 7. How about 'Hubble length' as a unit? Indeed, Mpc is needed (I can imagine a header card as follows: H0 = 50 / Assumed Hubble Constant H0_UNIT= 'km /s /Mpc' / Units of Hubble Constant ). 8. Other units analogous to pixel are 'bin' (e.g. for PHA or Einstein IPC BAL) and 'step' (e.g. for instrument high voltage). Technically these are all 'dimensionless' in SI terms but in our context I think it's useful to have them, together with the existing 'photon' and 'count'. 9. Note the typo on Table 3 - should be MeV not Mev. also in section 2 - resistance is misspelt resistence. Finally I deprecate your omission of the standard FEF system of units (furlong-elephant-fortnight). Comments on other comments: I disagree with Clive's suggestion that we should mandate use of radians internally. Since as he points out the conversion is easy to do, we might as well leave it to the reading software. The key (and the power of FITS) is to make sure the header keywords allow the software to determine the meaning of the values. I would prefer to write a standard that allows as many different ways of doing things as possible but ensures that it's easy to figure out exactly what was done. - jonathan From heafits@legacy.gsfc.nasa.gov Fri Apr 16 08:26:06 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["730" "Fri" "16" "April" "93" "08:25:04" "-0400" "\"Don Jennings -- COSSC\"" "jennings@enemy.gsfc.nasa.gov" nil "13" "Another comment on George's and Lorella's FITS units draft" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA00172; Fri, 16 Apr 93 08:26:03 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA22266; Fri, 16 Apr 93 08:25:04 -0400 Message-Id: <9304161227.AA15607@enemy.gsfc.nasa.gov> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: "Don Jennings -- COSSC" Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: Another comment on George's and Lorella's FITS units draft Date: Fri, 16 Apr 93 08:25:04 -0400 My comment on the units draft is similar to that of Paul Barrett and concerns the relative importance of the eV, keV, MeV, GeV and even PeV units. While X-ray astronomers usually think in terms of keV, Gamma ray astronomers prefer MeV and GeV (and occasionally PeV) when describing the energy of photons. To be fair to astronomers of both wavelengths, I propose that we either do as Paul Barrett suggests and promote eV units to table 2 (recommended units), demote keV to table 3 (allowed but discouraged) and add GeV (PeV) to table 3, OR we promote all of these units to table 2. If the FITS readers can be made to understand the 'keV' string, then it takes little extra effort to make them understand 'MeV', 'GeV' and 'PeV'. From heafits@legacy.gsfc.nasa.gov Fri Apr 16 15:41:55 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1002" "Fri" "16" "April" "93" "15:41:09" "-0400" "\"Don Jennings -- COSSC\"" "jennings@enemy.gsfc.nasa.gov" nil "21" "Re: Another comment on George's and Lorella's FITS units draft " "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA01652; Fri, 16 Apr 93 15:41:52 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA29626; Fri, 16 Apr 93 15:41:09 -0400 Message-Id: <9304161943.AA17278@enemy.gsfc.nasa.gov> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: "Don Jennings -- COSSC" Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: Re: Another comment on George's and Lorella's FITS units draft Date: Fri, 16 Apr 93 15:41:09 -0400 >My comment on the units draft is similar to that of Paul Barrett and concerns >the relative importance of the eV, keV, MeV, GeV and even PeV units. > >While X-ray astronomers usually think in terms of keV, Gamma ray astronomers >prefer MeV and GeV (and occasionally PeV) when describing the energy of >photons. To be fair to astronomers of both wavelengths, I propose that we >either do as Paul Barrett suggests and promote eV units to table 2 (recommended >units), demote keV to table 3 (allowed but discouraged) and add GeV (PeV) to >table 3, OR we promote all of these units to table 2. > >If the FITS readers can be made to understand the 'keV' string, then it takes >little extra effort to make them understand 'MeV', 'GeV' and 'PeV'. This morning my tired brain forgot to include 'TeV' in the list of desired energy units. Also, how about adding nanometers (nm) to the list of allowed units in table 2. It is at least as useful as angstroms, if not more politically correct in the SI sense. From heafits@legacy.gsfc.nasa.gov Wed Apr 21 05:09:36 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["3338" "Wed" "21" "April" "93" "05:08:39" "-0400" "Usque ad mortem bibendum" "GEORGE@heavax.gsfc.nasa.gov (Ian M George, Code 668, NASA/GSFC, USA )" nil "66" "OGIP/93-001 - 3 Specific Questions" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA12438; Wed, 21 Apr 93 05:09:34 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA26121; Wed, 21 Apr 93 05:08:39 -0400 Message-Id: <930421050522.206002ae@heavax.gsfc.nasa.gov> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: GEORGE@heavax.gsfc.nasa.gov (Ian M George, Code 668, NASA/GSFC, USA (Usque ad mortem bibendum)) Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: OGIP/93-001 - 3 Specific Questions Date: Wed, 21 Apr 93 05:08:39 -0400 RE: OGIP/93-001 (Specification of Physical Units within FITS files) Thank you for the comments etc received so far. Most of these seem to be relatively straightforward suggestions for the promotion, demotion or addition of unit strings. Lorella & I will attempt to summarize and give our reactions (based either on some form of consistency or consensus arguments) in a week or so. However there are 3 (slightly harder) issues which have been raised (all by Jonathan McDowell), but received little debate. Thus I'd like to raise them again here since I'd be interested in your collective opinion. (Replies to the heafits@legacy.gsfc.nasa.gov exploder please) A) Jonathan correctly pointed out that pixels are used as units of both length and area. - Is everybody happy with such an ambiguity ? ==> Personally, I dont like it much, but have a feeling that it may be so engrained in FITS files & brains currently in circulation that it will be hard to break. - Is there any viable alternative ? ==> I believe the correct definition of a pixel is in terms of an area, thus dislike the scheme whereby a 'pixel' is redefined as a unit of length, and 'pixel**2' as a unit of area. Thus the only option is to come up with a name/unit for the side of a pixel (and please nobody suggest 'pixel**0.5' !). I think we have to live with the ambiguity.... B) Jonathan felt that units raised to negative powers could be better expressed using the format 'W m**(-2) Hz**(-1)' rather than 'W /m**2 /Hz' since any possible ambiguity is removed. - How do others feel ? ==> Personally I like this suggestion, especially since I consider it to remove the necessity of Bill Pence's earlier suggestion that all multiplicative units be separated by an asterix. - Are the brackets necessary ? ==> I find the string easier to read & parse if they are included, thus suggest yes, they be mandatory for all indices. - If accepted should it then follow that ALL unit strings must conform to this syntax, and thus that the '/' character is never used again ? ==> I suggest yes, lets simplify the parser and get rid of the '/' [==> Regarding the 2nd half of Jonathan's point 4, I assume that most people agree that in the case of units raised to fractional powers, then it preferable to express those fractions as decimals.] C) Finally, Jonathan also asked about the inclusion of numerical factors within the units string. Specifically he was concerned about preventing under/overflows - Are '10**(44) erg /s' and '10**(-26) W /m**2 /Hz' acceptable units ? ==> (apart from the use of the '/' -- see B) I'm undecided about this, but see that it could lead to additional interesting possibilities such as - the elimination of the need for SI prefixes eg '10**(6) eV', '10**(-10) m' - even implicit unit conversions eg '3600 s', '0.0254 m' ==> personally I *think* I like the idea of allowing factors of 10 to be specified in this way, but feel rather uneasy about any other numerical factors. In either case, however, I suggest that if allowed such factors were only allowed to be placed at the beginning of the string. - But do we really want to start such a trend ? Ian M George (george@lheavx.gsfc.nasa.gov) From heafits@legacy.gsfc.nasa.gov Wed Apr 21 06:32:56 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1724" "Wed" "21" "April" "93" "06:32:04" "-0400" "\"Clive Page, Leicester University, UK\"" "CGP@starlink.leicester.ac.uk" nil "35" "Units in FITS files" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA12445; Wed, 21 Apr 93 06:32:54 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA26484; Wed, 21 Apr 93 06:32:04 -0400 Message-Id: <9304211032.AA26484@legacy.gsfc.nasa.gov> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: "Clive Page, Leicester University, UK" Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: Units in FITS files Date: Wed, 21 Apr 93 06:32:04 -0400 My thoughts, for what they are worth, are as follows: (a) Pixels are units of both length and area: I don't think the ambiguity is a serious problem, but it's a pity we can't come up with a suitable name for the side of a pixel. After all the term "voxel" as the 3-d analogue of the pixel is now well established. (b) Units raised to negative powers: One problem with the use of forms such as "Hz**(-1)" is that the resulting strings can be rather long: 8 chars compared to 3 for the form "/Hz". An important use for units strings is to put them in table headings: when these are displayed on a screen, or printed out, you won't be able to get many columns across the screen if the units strings take up so much space. Note that "W m**(-2) Hz**(-1)" takes up nearly a fifth of my screen-width. Even worse, some bits of software may decide to truncate the strings at some arbitrary limit, say 16 chars. If the parentheses are removed, as in "Hz**-1" this reduces the length a bit, but on the whole I prefer the original form on the grounds of simplicity and brevity. (c) Numerical prefixes: I think the average user is going to be annoyed and will think the programmer somewhat stupid if the units string appeares as something as complicated as "10**(3) eV" when what is meant is just "keV", a term that all high-energy physicists are instantly familiar with. The former requires a few seconds though to be interpreted correctly, and is also harder for the machine to parse. I see no advantage at all in using it. Similarly I don't think it is a good idea to use powers of 10 in the units strings until you get beyond the range for which standard prefixes are defined (10**-18 to 10**+18 ??). Clive Page From heafits@legacy.gsfc.nasa.gov Wed Apr 21 08:48:20 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1691" "Wed" "21" "April" "93" "08:47:29" "-0400" "BARRY M. SCHLESINGER" "BSCHLESINGER@nssdca.gsfc.nasa.gov " nil "41" "RE: OGIP/93-001 - 3 Specific Questions" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA13344; Wed, 21 Apr 93 08:48:18 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA27257; Wed, 21 Apr 93 08:47:29 -0400 Message-Id: <930421084744.22407ea3@NSSDCA.GSFC.NASA.GOV> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: BSCHLESINGER@nssdca.gsfc.nasa.gov (BARRY M. SCHLESINGER) Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: RE: OGIP/93-001 - 3 Specific Questions Date: Wed, 21 Apr 93 08:47:29 -0400 The discussion on numerical factors within units, as summarized by Ian George: >C) Finally, Jonathan also asked about the inclusion of numerical factors > within the units string. Specifically he was concerned about preventing > under/overflows > - Are '10**(44) erg /s' and '10**(-26) W /m**2 /Hz' acceptable units ? > ==> (apart from the use of the '/' -- see B) I'm undecided about this, > but see that it could lead to additional interesting possibilities > such as > - the elimination of the need for SI prefixes > eg '10**(6) eV', '10**(-10) m' > - even implicit unit conversions > eg '3600 s', '0.0254 m' > ==> personally I *think* I like the idea of allowing factors of 10 > to be specified in this way, but feel rather uneasy about any other > numerical factors. In either case, however, I suggest that if > allowed such factors were only allowed to be placed at the > beginning of the string. > - But do we really want to start such a trend ? The current BINTABLE proposal provides for the TSCALn and TZEROn keywords, which can be used for scaling in the same way as the same keywords for the ASCII table extension and as BSCALE and BZERO for a data array. Thus if the values in column 6 were in units of 10^(-10)m, then one could use TSCAL6 = 1.E-10 TUNIT6 = 'm ' This way, only standard units are used as values for TUNITn, but the quantities in the table can still be of reasonable magnitude. Note that TUNITn (and BUNIT) give the units of the physical quantity derived by applying the scaling transformation to the value contained in the FITS file. Barry Schlesinger NSSDC/NOST FITS Support Office From heafits@legacy.gsfc.nasa.gov Wed Apr 21 13:42:51 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1776" "Wed" "21" "April" "93" "13:41:59" "-0400" "Jonathan McDowell" "jcm@urania.harvard.edu " nil "47" "" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA13420; Wed, 21 Apr 93 13:42:49 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA03798; Wed, 21 Apr 93 13:41:59 -0400 Message-Id: <9304211742.AA27900@urania.harvard.edu> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: jcm@urania.harvard.edu (Jonathan McDowell) Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: Date: Wed, 21 Apr 93 13:41:59 -0400 Response to Ian, Clive and Barry: 1) I think the correct 1-d analog of pixel is probably 'bin', but I agree in practice we have to accept 'pixel' and live with the ambiguity. 2) W m**(-2) I don't think we need to worry so much about simplifying the parser, so I would not suggest the parens should be required for simpler cases like cm**2. Clive's point is well taken about the conciseness of the / form. Again, I'm happy to code my parser to be able to read either form. I stand by my preference for the **(-n) form in complex cases. I think the argument about software dropping parts of the string is bogus; we should be able to use the full FITS character header card space; but the argument about the difficulty of displaying it has more force. For me, though, the deciding factor is that our string should clearly and unambiguously define the units used in the file. Displaying that string easily is less important since your software can parse and reformat it as it likes. 3) keV vs 10**(3) eV. I strongly agree with Clive and disagree with Ian on this point. It sort of revolves on whether we think it is mainly software or mainly humans who are going to read these strings. I think in practice they are often going to be passed directly to humans for simple applications, so let's use the SI prefixes we know and love. In particular if we mandate 10**(3) eV and forbid keV, no X-ray astronomer is actually going to obey our standard. By the way, the prefix range defined in SI is now 10**-24 to 10**+24 with the new y,z,Y,Z prefixes. 4) 10*44 erg/s I think Barry's point about the TSCALn keyword solves this issue elegantly, and hope all will agree that this removes the need for numerical prefixes in the unit strings themselves. - Jonathan From heafits@legacy.gsfc.nasa.gov Thu Apr 22 04:54:22 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1585" "Thu" "22" "April" "93" "04:53:30" "-0400" "\"Clive Page, Leicester University, UK\"" "CGP@starlink.leicester.ac.uk" nil "29" "More on units and scaling" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA14026; Thu, 22 Apr 93 04:54:19 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA27253; Thu, 22 Apr 93 04:53:30 -0400 Message-Id: <9304220853.AA27253@legacy.gsfc.nasa.gov> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: "Clive Page, Leicester University, UK" Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: More on units and scaling Date: Thu, 22 Apr 93 04:53:30 -0400 I agree with most of what Jonathan says, in particular I'd support the use of "bin" as the 1-d analogue of "pixel". (2) on the importance of conciseness: clearly "our" software (being by definition good software) won't ever do anything so silly as to truncate a units string, but we have to accept that an important reason for using FITS is to make files that all the other FITS readers out there can handle. Some of this may be nasty "not invented here" stuff, written by programmers who haven't been to the right schools. The shorter the units string the better the chances of it surviving being handled by such nasty foreign software. (4) On the use of TSCAL/TZERO. In theory this solves the problem, but if you work out what happens in practice, each FITS reader has to take the values in the file and apply the appropriate TSCAL and TZERO values to get an internal value. If this results in values outside the range 1e-38 to 1e38 (or thereabouts) it will overflow in single precision on most computers I know. On some systems (e.g. Vaxes) it will even overflow in D-floating. If such cases are to be treated sensibly, the FITS reader software has to be alive to such cases and convert to DOUBLE PRECISION. But this won't solve the problem in all cases, e.g. on a VAX using D-floating, or if the TSCAL value is as large as 1e300. These problems don't, of course, occur in the scaling factor is put in the units string. I can see that this argument goes against my preference for conciseness, but I think that pragmatism may be an even more important principle. Clive From heafits@legacy.gsfc.nasa.gov Thu Apr 22 09:52:06 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1197" "Thu" "22" "April" "93" "09:51:13" "-0400" "\"Jonathan McDowell\"" "jcm@urania.harvard.edu" nil "25" "Re: More on units and scaling " "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA14954; Thu, 22 Apr 93 09:52:05 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA01728; Thu, 22 Apr 93 09:51:13 -0400 Message-Id: <9304221351.AA05720@urania.harvard.edu> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: "Jonathan McDowell" Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: Re: More on units and scaling Date: Thu, 22 Apr 93 09:51:13 -0400 > (4) On the use of TSCAL/TZERO. > In theory this solves the problem, but if you work out what happens in > practice, each FITS reader has to take the values in the file and apply > the appropriate TSCAL and TZERO values to get an internal value. If this > results in values outside the range 1e-38 to 1e38 (or thereabouts) it > will overflow in single precision on most computers I know. On some Oooh. You're right, aren't you, Clive? I hadn't thought that one through. I withdraw my earlier recantation - the TSCAL problem does you no good whatsoever with the 1.E44 erg/s case unless you have a very clever reader. Now a way around this would be to mandate the TSCAL value to be considered part of the unit, and decree that our readers not convert the FITS file value by applying the TSCAL/TZERO value, but just leave it as it is. This goes against long FITS tradition though. Perhaps we need an extra set of TUSCLnnn values which are just like TSCAL except that they are considered to be part of the unit rather than part of the FITS internal rescaling. You can tell I quite like the idea of separating out the unit multiplier from the unit and want to save it if I can. - Jonathan From heafits@legacy.gsfc.nasa.gov Thu Apr 22 12:57:26 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1138" "Thu" "22" "April" "93" "12:56:32" "-0400" "William Pence" "pence@tetra.gsfc.nasa.gov " nil "25" "Re: Another comment on George's and Lorella's FITS units draft" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA15024; Thu, 22 Apr 93 12:57:24 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA05218; Thu, 22 Apr 93 12:56:32 -0400 Message-Id: <9304221657.AA06496@tetra.gsfc.nasa.gov> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: pence@tetra.gsfc.nasa.gov (William Pence) Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: Re: Another comment on George's and Lorella's FITS units draft Date: Thu, 22 Apr 93 12:56:32 -0400 Here's my input on some of the units issues: - I prefer 'bin' over 'pixel as a unit of length but would be willing to support both. - Negative powers: I think it is a small matter for parsers to be able to support the '/' character, so would prefer allowing which ever form is more concise or intelligible to the user. - Numerical factors as units: I can see a use for this in cases where the values could over or underflow a variable on typical machines. So I find '10**(44) erg /s' acceptable, but would not like to see this usage required to eliminate the use of SI prefixes. While it is a bit disagreeable, I think all the standared SI prefixes need to be supported since that is what people are used to seeing. As was previously mentioned the TSCAL and TZERO keywords may be useful in some cases, but this in itself does not solve the underflow problem. Finally, I don't seem much advantage to Jonathan's idea of having an additional TUSCLnnn keyword since FITS readers would then have to search for 2 keywords, not just one.. I'd rather see all the unit information contained in the single TUNIT keyword, -Bill Pence From heafits@legacy.gsfc.nasa.gov Thu Apr 22 13:49:58 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["433" "Thu" "22" "April" "93" "13:49:07" "-0400" "\"Clive Page, Leicester University, UK\"" "CGP@starlink.leicester.ac.uk" nil "9" "Re: More on units and scaling" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA15035; Thu, 22 Apr 93 13:49:57 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA07071; Thu, 22 Apr 93 13:49:07 -0400 Message-Id: <9304221749.AA07071@legacy.gsfc.nasa.gov> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: "Clive Page, Leicester University, UK" Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: Re: More on units and scaling Date: Thu, 22 Apr 93 13:49:07 -0400 By the way, Jonathan, in an earlier message you said that there were now standard prefixes from 1e-24 to 1e24. This is new to me, indeed I thought the standards committees stopped at around +-15 because they had run out of greek words for things that were extremely small or extremely large. What have they moved on to - Sanskit, Old Icelandic? Seriously though, it would be nice to see a complete list of them somewhere. Clive From heafits@legacy.gsfc.nasa.gov Thu Apr 22 15:33:33 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1841" "Thu" "22" "April" "93" "15:32:42" "-0400" "Jonathan McDowell" "jcm@urania.harvard.edu " nil "68" "" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA15073; Thu, 22 Apr 93 15:33:32 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA09301; Thu, 22 Apr 93 15:32:42 -0400 Message-Id: <9304221932.AA10635@urania.harvard.edu> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: jcm@urania.harvard.edu (Jonathan McDowell) Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: Date: Thu, 22 Apr 93 15:32:42 -0400 Clive (and everyone else) - I got this from John Roberts (roberts@cmr.ncsl.nist.gov): CIPM, 1990 [Comite International des Poids et Mesures] DRAFT RESOLUTION D TO THE 19th CGPM The 19th Conference Generale des Poids et Mesures, *decides* to add to the list of SI prefixes to be used for multiples and submultiples of units, adopted by the 11th Conference Generale des Poids et Mesures (CGPM), Resolution 12, paragraph 3, the 12th CGPM, Resolution 8, and the 15th CGPM, Resolution 10, the following prefixes: Multiplying factor Prefix Symbol 10^(21) zetta Z 10^(-21) zepto z 10^(24) yotta Y 10^(-24) yocto y [Footnote:] The names zepto and zetta are derived from septo suggesting the number seven (the seventh power of 10^3) and the letter "z" is substituted for the letter "s" to avoid the duplicate use of the letter "s" as a symbol. The names yocto and yotta are derived from octo, suggesting the number eight (the eighth power of 10^3); the letter "y" is added to avoid the use of the letter "o" as a symbol because it may be confused with the number zero. ---------- So this makes the full list: ! y -24 yocto ! z -21 zepto ! a -18 atto ! f -15 femto ! p -12 pico ! n -9 nano ! u -6 micro (Greek Mu) ! m -3 milli ! 0 ! k 3 kilo ! M 6 Mega ! G 9 Giga ! T 12 Tera ! P 15 Peta ! E 18 Eka ! Z 21 Zetta ! Y 24 Yotta ! c -2 centi ! d -1 deci ! D 1 Deka ! h 2 hecto - Jonathan From heafits@legacy.gsfc.nasa.gov Fri Apr 23 12:14:14 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["2176" "Fri" "23" "April" "93" "12:13:21" "-0400" "Lucio Chiappetti" "lucio@ifctr.mi.cnr.it" nil "43" "attempt to get Ian George's memo via ftp ..." "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA16475; Fri, 23 Apr 93 12:14:12 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA22004; Fri, 23 Apr 93 12:13:21 -0400 Message-Id: Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: Lucio Chiappetti Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: attempt to get Ian George's memo via ftp ... Date: Fri, 23 Apr 93 12:13:21 -0400 Hello, I attempted a few days ago to retrieve ogip_93_001.ps from legacy. Unfortunately the Postscript file makes reference to some macros produced by the DVILaser software which are not included in the file itself, so I could not print (or preview) it. Now I decided to retrieve the TeX version of the same file and process it myself. Unfortunately the ftp server on legacy is showing a very strange behaviour : it displays a light-year-long welcome message (some 100 pages ? surely longer than the scroll size of my DECterm .. 500 lines) interrupted by an ftp> prompt. Any command I type has the result to continue the typing of the welcome message. I insisted with a pwd command until the welcome message finished, but then it looks like the ls and get command do not work. Please let me know when it will be back in order. ----------------------------------------------------------------------------- A member of G.ASS : Group for Astronomical Software Support ----------------------------------------------------------------------------- Lucio Chiappetti - IFCTR/CNR Milano | U N I C U I Q U E via Bassini 15 - I-20133 Milano - Italy | System Manager Internet: LUCIO@IFCTR.MI.CNR.IT | FORTUNAE SUAE Decnet: IFCTR::LUCIO (39610::LUCIO) | Bitnet: | ----------------------------------------------------------------------------- Post Scriptum : Since one of our contractors told us they want to use DVILaser to produce PostScript documents to be distributed over the network, and since this is the first case I see of unprintable PostScript documents (I have sent and retrieved ps files of many different origins without problems), I would like to know whether there is an option to have the relevant macros included in the file or somehow distributed (or are they proprietary ?) Post Post Scriptum : I'd like to know the trick you use to have ftp print a welcome message (***but please**** DON'T make it THAT long !!!) From heafits@legacy.gsfc.nasa.gov Fri Apr 23 12:28:51 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1328" "Fri" "23" "April" "93" "12:27:59" "-0400" "Bruce O'Neel" "oneel@arupa.gsfc.nasa.gov " nil "37" "Re: attempt to get Ian George's memo via ftp ..." "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA16480; Fri, 23 Apr 93 12:28:48 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA22618; Fri, 23 Apr 93 12:27:59 -0400 Message-Id: <9304231628.AA05181@arupa.gsfc.nasa.gov> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: oneel@arupa.gsfc.nasa.gov (Bruce O'Neel) Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: Re: attempt to get Ian George's memo via ftp ... Date: Fri, 23 Apr 93 12:27:59 -0400 Hi, > > it displays a light-year-long welcome message (some 100 pages ? surely > longer than the scroll size of my DECterm .. 500 lines) interrupted by > an ftp> prompt. > Any command I type has the result to continue the typing of the welcome > message. I insisted with a pwd command until the welcome message finished, > but then it looks like the ls and get command do not work. Put a - character as the first character of the password you type in. This stops all the messages and makes our ftpd like a regular ftpd. > > Post Post Scriptum : > I'd like to know the trick you use to have ftp print a welcome message > (***but please**** DON'T make it THAT long !!!) This is a replacment ftpd program which runs on (at least) on suns, ultrix decstations, and Alpha OSF/1. It probably runs on others. It is produced by the people at wuarchve.wustl.edu. You can get it from ftp.uu.net:/networking/ftp/wu-ftpd-2.0.tar.Z. bruce p.s. I thought this might interest others so I am sending to the whole list.... -- ======================================= | Warning: File Corrupt! | | | | [OK] [CANCEL] | ======================================= From heafits@legacy.gsfc.nasa.gov Mon Apr 26 11:45:20 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["11191" "Mon" "26" "April" "93" "11:44:25" "-0400" "Lucio Chiappetti" "lucio@ifctr.mi.cnr.it" nil "223" "about OGIP/93-001" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA20670; Mon, 26 Apr 93 11:45:17 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA07689; Mon, 26 Apr 93 11:44:25 -0400 Message-Id: Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: Lucio Chiappetti Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: about OGIP/93-001 Date: Mon, 26 Apr 93 11:44:25 -0400 Finally I had time to read memo OGIP/93-001 and all related correspondence, and I'll try to summarize my comments below (in a few cases I will just repeat comments already raised by other people, in order to strengthen support to such opinions). ----------------------------------------------------------------------------- Lucio Chiappetti - IFCTR/CNR Milano | U N I C U I Q U E via Bassini 15 - I-20133 Milano - Italy | System Manager Internet: LUCIO@IFCTR.MI.CNR.IT | FORTUNAE SUAE Decnet: IFCTR::LUCIO (39610::LUCIO) | Bitnet: | ----------------------------------------------------------------------------- A) A STANDARD ON PHYSICAL UNITS - WHAT FOR ? First of all I'd like to have a clearer idea of what is the PURPOSE of such a standard. I can think myself of the following motivations : 0- to define the content of TUNITn (CUNIT, BUNIT) keywords (but this is not an exhaustive motivation, just an implementation) 1- these unit keywords provide (human-readable) DOCUMENTATION about the file content 2- these unit keywords are used to annotate plots, and to appear in printouts (still the emphasis is on human legibility, though software-aided) 3- for autoscaling by programs when reading the data (this implies some form of parsers in software) Personally I would put emphasis on motivations 1 and 2, and not be overly concerned about motivation 3, and pursue this systematically in the comments below. I would also say that I regard the question of units a "third order" issue (this does NOT mean a standard in this area is not important, it IS important for users to interpret the files, but is not essential in a way that not having it will hinder software development, or data usage by existing software). I would for instance regard as a "second order" issue the definition of STANDARD NAMES for PHYSICAL QUANTITIES (and I will cover this in a separate mail) and as a "first order" issue yhe definition of what are the basic file structures to be discussed within HEAFITS (and what are instead best left outside; this too I intend to cover in a separate mail) B) AESTHETICS, PROMOTIONS AND DEMOTIONS ETC. (comments to Tab I-II-III) B1) A minor comment to Tab. 1. Given for granted Clive Page's comment on the low (--> 0) frequency of electric conductance in astronomical data, and also given for granted that "S" may be confused with "s" for second in a case-insensitive environment I wonder why "siemens" are abbreviated "siemen". I do not see any reason to truncate the final "s" since there is other- wise no limitation to 6 character of the unit string. I do not believe poor Wilhelm and Werner Siemens deserve this truncation (although the SI decided that poor Alessandro Volta be subject to truncation in giving his name to the electric potential unit). Perhaps it would be useful to spell a clear rule like this : the unit string is the standard SI (or cgs or whatever) abbreviation (e.g. s, kg, Wg, cm, keV) in all cases where this is : a) in printable latin characters, b) not ambiguous considering case sensitivity ; in all other cases the long form of the unit name shall be used (so you correctly use ohm, siemenS, angstrom, deg) B2) As an application of the previous rule, I'd like to see the form "micro" instead of "u" used as the 10**-6 prefix. Therefore "micros" for microsecond and "microm" for micrometers etc. B3) Concerning table I, I regard almost all the second half of it (from Volts to Henry) of unfrequent use in astronomy (the same is true for Ampere and mole in the first part), anyhow it is fine by itself. I would promove to table II the following units : micros and ms (microsecond and millisecond) cm (centimetre, for NH in atoms/cm**2) pc, kpc, Mpc (parsec, kiloparsec, megaparsec) Jy, mJy, microJy (jansky, milli- and micro-jansky) photon (for dN/dE photons/cm**2/s/keV and alike) (perhaps also grams, eV and MeV) I would also appreciate a dimensionless "atom" to appear, still in order to express column densities like NH = 10**22 atom/cm**2). C) COMPOUND UNITS Since I am giving emphasis on use of strings for annotating plots and printouts, in order to keep the label short, I do not like parentheses around exponents. Actually I'd even accept to drop the ** power operator, and have cm2 instead of cm**2 etc., having forms like erg/cm2/s/Hz or erg cm-2 s-1 Hz-1. I believe they are fully acceptable for plots (even at publication quality) and work printouts. In all cases a smarter software (plotting, or word processing) must reformat them into superscripts, this is still possible if one assumes that a (eventually signed) number following a plain unit is to be intended as superscript. D) ANGULAR QUANTITIES Yes I agree to deprecate hh-mm-ss and similar sexadecimal notations for STORAGE (however they are valid for display). I'd also like a definition ("typedef") of special data types for "angles" (decimal degrees ?), "times" (seconds or fractions thereof) and "dates" (MJD) to be associated to a given column table or header keyword (particularly the latter however applies nto to FITS files but to some package's native internal format). For angles the question is then what to recommend (radians vs degree). I presume the discrimination should be the number of significant figures in the most restrictive double precision binary representation around. I give for granted that angles are represented as a REAL*8 value "in some units". REAL*8 are "about 16 decimal digits" on VAX D_FLOAT and IEEE, "about 15" on VAX G_FLOAT (it is disappearing on Alphas, isn't it) and I presume also on IBM VM (but I am not sure). As an arcsec is 2.777777E-4 deg, assuming an angle is limited to 0-360 deg, this makes nnn.fffxxx with xxx=278 at least necessary to keep the arcsec resolution with some precision, i.e. 9 decimal figures. This is more than usual single precision, hence double precision is needed for angles in degrees. This is even more true if one uses radians; an arcsec is 4.8E-6 rad. Now, what is the resolution we want to keep in general, a milliarcsec ? a microarcsec ? Depending on this, degrees might be preferred if we get close to the limit by about 2 decimal digits. E) UNIT PRESCALING Yes, pre-scaling like 10**44 erg/s is needed, and must not be interpreted by software (in order not to overflow on 1.0E-38 to 1.0E38 machines ...) i.e. no TSCAL and alike. I actually wonder whether even in IEEE floating point data special precautions are to be taken to avoid values outside this range, given that non-IEEE machines are often limited to the above range for single and ofter double precision ... again the most restrictive machine rules) In this case unfortunately my "labelling argument" fails (I can write 10-26 W/m2/Hz, but not 1044 erg/s ... is 1E44 any better than 10**44 ?) I regard prescaling is just a reminder for humans (so I would not be concerned too much about eV, MeV instead of keV and so on). In general I foresee some manual handling by user, based on the "dopcumentation" provided in the file,if the units AND prescaling used does no match that supported by my s/w( I would not overload my s/w with complex parsing routines trying to cope with many cases ... "build a system that even a fool can use, and only a fool will want to use it"). E.g. I have a set of programs which can handle photon spectra in photon/cm2/s/keV, wavelength spectra in in 1E-14 erg/cm2/s/angstrom, frequency spectra in mJy (1E-26 erg/cm2/s/Hz) and nu-Fnu spectra in erg/cm2/s. If I receive from somebody a spectrum in other units (say 1E-16 erg/cm2/s/keV) it is my business to scale it in advance. In principle I'd favour to limit prescaling to powers of ten, but I am concerned about powers of 2. Often spacecraft clocks are in steps of some 2**-n s, and binning of time profiles, accumulations of spectra, etc. are in some multiple of that. I personally believe that for plotting one best uses just (decimal fractions of) seconds, and for internal handling a different convention (storing the time resolution in some kewyord) shall be devised. F) PIXELS, BINS etc. I am happy to leave with the ambiguity of pixels as units of area and length. They are clear by context (if I say 70 photons/pixel I mean an unit of area; if I say "a box of 7*10 pixels" I mean an unit of length. "Bin" is a synonym for pixel when speaking of 1-d histograms (i.e. distribution y=f(x) with x in bin number; as z=f(x,y) with x,y pixel number. Concerning bin and pixels, I also would like to raise some additional issues. Sometimes one is happy to operate on bins or pixels as provided in the image (2-d x=f(x,y)) or histogram (1-d y=f(x)). This is e.g. the case of Burst Length or Rise Time spectra, in which one happily works in whatever ADC channels are provided and is not concerned with how many microsecond is Rise Time channel 18. In some other cases instead one may want to work in RA,Dec instead of x,y pixels (and this is covered by the well-known WCS stuff); or perhaps his image is a chi- square map with x,y logNH and alpha ... or one wants energy in keV or wavelength in angstrom instead of channel number. I am not sure about the WCS being able to accomodate the latter cases, probably yes. I need to read it again, specially for things whether the corner of a pixel (bin) is 0, 1, or 0.5 and alike. However there are intermediate cases of conversions, between "world" coordinates (RA,DEC or KeV) and "arbitrary" coordinates (image pixels or histogram bins), that is when some "standard detector coordinates" are used. Let us consider for example the case of a 1024*1024 Detector Pixels (DP), from whose datastream I accumulate a 256*256 image, zoom 2, covering the central 512*512 DPs. If I want to know some position-dependent correction factor, I need to use the position in DPs, not in "image pixels" (1-256). Likewise a detector may bin energy in 512 PHA channels, but the onboard or ground s/w may integrate it in 64-bin spectra, but to know channel boundaries or spectral response I'll need to refer to PHA channels. In the latter case it looks a quite widespread custom is to call "channels" the original detector PHA channels, and "bins" whatever multiple thereof (including 1* multiple). I've seen the proposal to introduce a similar jargon for images, calling "pixels" the "detector pixels" and "cells" whatever multiple thereof, but I believe this is NOT a widespread usage. I wonder whether such distinction shall be introduced at "unit string" level - or "third order" (differentiating "BIN" and "CHANNEL", "PIXEL" and "CELL" etc.) or instead at "quantity name" level (a "second order" question) putting in the column heading or otherwise appropriate kewyord a different name (for images it might be DETECTOR_X, DETECTOR_Y, or X_DET, Y_DET instead of ARBITRARY_X, ARBITRARY_Y or just X, Y, although I'm not happy with they way they sound, and even less I can find a satisfactory arrangement for PHA spectra). *** that's the end **** From heafits@legacy.gsfc.nasa.gov Mon Apr 26 14:42:36 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["2140" "Mon" "26" "April" "93" "14:40:56" "-0400" "\"Clive Page, Leicester University, UK\"" "CGP@starlink.leicester.ac.uk" nil "43" "RE: about OGIP/93-001" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA20788; Mon, 26 Apr 93 14:42:34 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA02566; Mon, 26 Apr 93 14:40:56 -0400 Message-Id: <9304261840.AA02566@legacy.gsfc.nasa.gov> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: "Clive Page, Leicester University, UK" Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: RE: about OGIP/93-001 Date: Mon, 26 Apr 93 14:40:56 -0400 Section (D) of the message from Lucio Chiapetti displays what I think is a fundamental misunderstanding of the way that floating-point arithmetic works. The precision you obtain is independent of the units you use (to within at most a factor of two), provided the numbers don't stray outside the permitted range. This is most unlikely to happen with angles whether you use radians, degrees, revolutions, or even microgrades. My preference is for radians simply because trig functions are normally defined to operate on them, and one can lose accuracy by repeatedly converting degrees to radians and vice-versa. My preference is to use internal storage to suit the machine, and convert to/from external forms (decimal degrees, hh:mm:ss, or whatever) to suit the user. The internal workings of all machines that I know about are more efficient if angles are stored internally in radians. As to the need for DOUBLE PRECISION: in my experience most astronomical positions are only precise to an arc second or so, so that REAL storage is usually sufficient. A note for Lucio: The REAL number format of IEEE arithmetic (and VAX arithmetic is very similar) uses 23 bits plus a sign bit to store the magnitude. But because the exponent is normalised, i.e. adjusted to make the mantissa as large as possible, its leading bit is always one and is therefore not stored (it is called the "hidden bit"). There are therefore effectively 24 bits, giving a resolution of 1 in 2**24, i.e. around 1 in 1.67e7. If our angles are normalised to a sensible range, the largest angle we need to handle is 360 degrees (or 2*pi radians) which has in REAL format a least-significant bit error of around 0.05 arcseconds. You can test this for yourself by storing 2*pi in a REAL and adding small numbers to it until it no longer equals two pi any more: I think that you will find that the smallest number you have to add is 2.4e-7, which converted from radians to arcseconds turns out to be 0.05. Althernatively if you store 360.0 in a REAL and do the same I think you will find that the least-bit resolution is very nearly the same. Regards Clive From heafits@legacy.gsfc.nasa.gov Tue Apr 27 08:43:36 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1061" "Tue" "27" "April" "93" "08:41:50" "-0400" "\"Wayne H. Warren Jr. 286-5419\"" "W3WHW@gibbs.gsfc.nasa.gov" nil "24" "Re: RE: about OGIP/93-001" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA22170; Tue, 27 Apr 93 08:43:35 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA03757; Tue, 27 Apr 93 08:41:50 -0400 Message-Id: <9304271241.AA03757@legacy.gsfc.nasa.gov> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: "Wayne H. Warren Jr. 286-5419" Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: Re: RE: about OGIP/93-001 Date: Tue, 27 Apr 93 08:41:50 -0400 > The precision you obtain is independent of the units you use (to > within at most a factor of two), provided the numbers don't stray > outside the permitted range. This is most unlikely to happen with > angles whether you use radians, degrees, revolutions, or even > microgrades. > I checked this out for units that I would have thought disproved it, but it came out the way Clive said. I converted 0.001 arcsecond (a typcial proper-motion value) to degrees, which comes out to 2.78E-07, still well within 32-bit single precision. > > As to the need for DOUBLE PRECISION: in my experience most astronomical > positions are only precise to an arc second or so, so that REAL storage > is usually sufficient. > I disagree with this, unless you're sure that you won't need positions to the accuracy currently available in the new astrometric catalogs. Single precision truncates to about 0.36 arcsec (depending on the RA). The new catalogs are better than this. I suggest storing *astrometric* equatorial positions in double precision (REAL*8) to be safe. From heafits@legacy.gsfc.nasa.gov Tue Apr 27 08:44:59 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["14940" "Tue" "27" "April" "93" "08:43:12" "-0400" "Lucio Chiappetti" "lucio@ifctr.mi.cnr.it" nil "308" "Re: OGIP FITS conventions (ROSAT et al.)" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA22175; Tue, 27 Apr 93 08:44:57 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA04000; Tue, 27 Apr 93 08:43:12 -0400 Message-Id: Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: Lucio Chiappetti Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: Re: OGIP FITS conventions (ROSAT et al.) Date: Tue, 27 Apr 93 08:43:12 -0400 As promised a while ago, I would address to HEAFITS a "first order" question about what, in my opinion, may and what may not be discussed over this exploder in the aim of converging about some multi-mission standard. For this I take as reference Mike Corcoran's mail, of which I will quote a few excerpts below, in order to make a common reference. On Thu, 20 Apr 1993 CORCORAN@ndadsb.gsfc.nasa.gov wrote: (forwarding to HEAFITS a message originally sent to me, of which I keep only hilights in the text below) > > .....respond to your questions. The OGIP has begun development of a > "standard" set of file formats for distribution and archiving of data > from high energy astrophysics missions. The fact OGIP is concerned on one hand ALSO with archiving of PAST missions, and on the other hand it is acting like a "filter" between a mission in which had no direct involvement at the level of defining the telemetry format or original data distribution format being this somehow ruled by other national agencies (e.g. ROSAT, ASTRO-D) puts it into a different framework than the one I'm operating in. I will make my points below based on the cases of SAX (a mission of the Italian Space Agency, with collaboration in the Netherlands and ESTEC), and XMM (a mission of the European Space Agency). In both cases the primary responsibility of data dissemination stays with the relevant agency (or its contractors). However there is a role of the scientists in giving as strict requirements as we can about how we want this to be done. > These formats are called > "rationalized data file" (RDF) formats. The basic specifications for > these files are > 1) the files follow the standard FITS conventions ..... *** see comment # 5 below ** > 2) under the rationalized scheme, the data is grouped into 4 categories: > a) BASIC data - this is the bare-bones science data, > like good time intervals, photon event lists, and statuses. I would call these "science telemetry data" *** see comment # 1 below ** > b) ANCILLARY data - ..... engineering or housekeeping > data, and spacecraft pointing (aspect) information. I would call these "housekeeping telemetry & auxiliary data" *** see comment # 2 below ** > c) CALIBRATION data -..... > response matrices, mirror effective areas, and point spread functions. for calibration data *** see comment # 3 below *** > d) DERIVED data ..... This includes images, > spectra, lightcurves, source lists, etc. I would call these "reduced data" or "data products" and put MOST EMPHASIS AT THIS LEVEL *** see comment # 4 below ** > 3) The OGIP has developed keywords and FITS table column names... *** see comment # 5 below ** --------------------------------------------------------------------------- In the following I'll make references to the case of SAX, which is in a stage of advanced definition, and XMM, for which I hope to be able to negotiate with ESA in a similar framework. # 1) SCIENCE "TELEMETRY" DATA Both SAX and XMM will use for their telemetry the ESA "Packet Telemetry" standard (actually this is a CCSDS standard, being the CCSDS a nice acronym which looks the same both in English and French - Consultative Committee for Space Data Systems or Comite' Consultatif pour Systemes de Donnees Spatiaux ... apologies about my bad French - that is some inter-agency-wide agreement). I do not know whether other space agencies are currently recommending/enforcing it for their missions. Being it a "packet" standard means in practice that each science instrument, operating in a variety of modes (i.e. direct modes producing photon lists, indirect spectral modes, indirect timing modes) produces one or more data streams with a defined layout. The standard itself is concerned about the way this user defined layout is encapsulated for transmission and not about forcing rules for this layout. I won't go here into any technical detail. Those of you being familiar with Exosat data may recognise that Exosat "floating telemetry" was a sort of rough, unstandardised precursor of the Packet Telemetry. Actually the Packet Telemetry applies both to science and HK telemetry. The basic facts here are : each instrument may have a large number of operating modes (i.e. different packet types), and not all of them are "direct" modes (i.e. there are spectral and timing modes). This is as far as I can say unlike missions having mainly only a direct (photon list) mode. for spacecraft and ground segment constraints one is often forced to optimize the telemetry bit rate using "unnatural" arrangements (i.e. having 10-bits for position, or 9 for energy, or packing events in some floating format with time every 6 photons only, etc.) These data cannot be reasonably given directly to observers, specially if one wants to give them some truly portable software. Furthermore one hand there are reasons to archive the telemetry as raw as possible (i.e. just sorted by packet type and observation BUT PRESERVING the packet arrangement : 1 packet = 1 record), on another hand whatever reformatting is done here must be simple, so that data can be reprocessed in the event of any mistake or update. [Finally since these things are to be done by a contractor, and contract requirement documents are unflexible, it is not possible to use standards not FULLY defined at the moment the requirements are written] The idea being pursued for SAX is to store packets of the same type in separate files for each observation (which is a sub-interval of a pointing interval where all instrument commandable parameters are kept constant), applying just a reformatting like "padding all quantities which are 1-8 bits to 8 bits, 9-16 bits to 16 bits, 17-32 bits to 32 bits" and "normalizing spacecraft time to a common resolution". [again those of you familiar with Exosat will recognise the "observation" and "observing period" distinction - I presume, given the similarity of operations, this might apply to XMM too] This sort of reformatted data, although not strictly telemetry, I'll term "science telemetry data". With it go some auxiliary files like "observation directories", "instrument configuration files" etc. which specify when and how data can be combined for analysis. Given the numerous mission specific issues at this level, I would personally NOT regard this kind of data as a subject of discussion (e.g. over HEAFITS) for standardization. (I will however incidentally mention here that we are thinking of having the "accumulation" programs, i.e. those which transform "science telemetry data" into "reduced data", driven by packet description kept in external files. We call such files "packetcap" for analogy with Unix termcap, printcap, graphcap etc. In such files one has for instance the NAMES of the fields present for each event. Being these names used later as column names in reduced data, THERE IS AN OBVIOUS NEED OF A STANDARD CONVENTION ON SUCH NAMES, and this CAN - MUST ? - BE MATTER FOR HEAFITS) # 2) HK TELEMETRY AND AUXILIARY DATA Payload and spacecraft housekeeping is handled exactly as science telemetry (at least whenever it is possible it be of any use for some observers - other s/c HK data are not even passed to the observer). In this case description of the content (so called Parameter Characteristics Files) are to be supplied by the Ground Segment contractor. Auxiliary data concern orbit, attitude, OBT to UTC conversion and alike, and are likewise supplied by the Ground Segment contractor, in ASCII form, which is sufficiently portable and self documented. Therefore these data too are NOT of concern to HEAFITS, in my opinion. # 3) CALIBRATION DATA I would consider here different kind of data : - the calibration observation (telemetry) data which are of no concern here (they are just like other observation data, only they are analysed by a particular PI or Agency team). Likewise are of no concern whatever intermediate files are produced during such analysis. But only the result of this analysis, i.e. ... - ...the calibration result files, i.e. files describing individual components (e.g. efficiency vs energy, optics area vs position, psf, gain, energy resolution, etc.). These are building blocks to be given to the observer. The general recommendation is that these data are not distributed along with the "telemetry" tapes, but made available for ftp or network copy (so that the observer can grab the latest update). I am unsure whether a standardization at this level is desirable or possible, but I will not a priori exclude related matters are discussed over HEAFITS - the ultimate files used for the analysis, like response matrices. These must be produced by the observer using supplied software, starting from the "building blocks". Such data in my view must be handled at all in the same way as "reduced data" (see below). # 4) REDUCED DATA Reduced data is what the observer produces using the accumulation programs (images, spectra, time profiles, photon lists AND response matrices). He can then manipulate them for background subtraction, dead time correction etc. and ultimately feed them in into one's favourite analysis package. One requirement at this stage is usage of a mission independent format (the only mission dependent stuff is storing reference information about instrument configuration somewhere in header keywords). The other basic requirement in my view is an economy principle, i.e. Ockham's Razor : FILE STRUCTURES NON SUNT MULTIPLICANDA PRAETER NECESSITATEM (are not to be multiplied beyond need). And also FILE LAYOUTS non sunt multiplicanda praeter necessitatem ! And hopefully they should be "natural looking". All this does not (yet necessarily) mean FITS (see comment #5 below), but means a preliminary agreement on WHAT ARE these basic file structures, and immediately after about their basic content. I'd like users of all various X-ray analysis packages in the world (from the general purpose ones like EXSAS, PROS or XANADU to any site or mission specific package) to comment on what is their feeling on current compatibility and requirements towards it (I would assume that one should not necessarily aim to "portability", i.e. use only one format 100% compatible with all packages, or even worse, all use a single package, but instead to "interoperability", i.e. import a file from one package, convert it hopefully with a SIMPLE and SINGLE operation to the other package format, use it in the second package, and so on) Concerning the list of basic data structures I presume (at least in X-ray astronomy ... I'm not sure how gamma-ray people feel about this) one can limit it to : - images - spectra - time profiles - photon lists - response matrices - plus perhaps minor ancillary files By images I mean not only the preponderant case of photon counts vs spatial x,y, but also any other z=f(x,y) case, like photon counts vs energy and rise time, or chi-square vs alpha and NH, or even a Fourier transform vs frequency and energy (the latter case are mainly for storage and display of results). By spectra I mean chiefly PHA spectra, but may also apply to any sort of 1-d histograms (Burst Length, Rise Time etc. ; but these can also be hanlded as 1-d images) By time profiles I mean any time profile, not just count rate vs time, but also any HK parameter vs time. Ancillary files are in my view things specific of some s/w package or program (like time windows, channel grouping masks and alike) and best stored as plain ASCII files, and anyhow outside the scope of the main discussion. Concerning the list of basic layouts, I would group the above in : - "images" (images, perhaps histograms, partly also response matrices) - "tables" (spectra, time profiles, photon lists) I would also as far as possible keep the above "physical" layout level hidden to the user, i.e. the user says "add spectrum A + spectrum B" and not "add column 1 of tables A and B, and quadratically add column 2 of tables A and B for the errors". I also note that while images are a simple univocally defined structure, tables allow a very wide range of choices, and it is quite easy to go astray from portability, interoperability or naturality. THIS IS WHERE CONVENTIONS MUST BE DISCUSSED BOTH ON THE LAYOUT, THE NAMES OF COLUMNS, THE KEYWORDS NEEDED FOR DESCRIPTION. Also in my quest for simplicity, I'd like as far as possible that each "logical" file (spectrum, image etc.) be a single disk file (or FITS extension if FITS is used either for storage or transport). I'd like to see the above things discussed, and of course I am ready to contribute to this discussion, but I believe this mail is getting quite long, and I'll better do that separately sometimes later. # 5) FITS OR NON-FITS I believe that discussion on the above topics can go on over HEAFITS (despite of the name of the exploder) EVEN if a particular mission decides to use a non-FITS format for its data. I'd personally favour (following a long-known trend) to store data in some native format, and use FITS for exchange only. However I have designed such format ("XAS" format) in such a way to be one-to-one mappable to FITS. The main differences are : data is kept in native binary representation, header keywords are kept in binary form, header keywords are located at the end of the file, in order to be able to extend the header freely e.g. to record in HISTORY any in-place mani- pulation to the file. Files are record-oriented with a "natural" record length dictated case-by-case (instead of the FITS 2880 ... however on a non-record oriented i.e. Unix, IEEE-compatible, e.g. Sun, machine, the data portion of a XAS file is IDENTICAL to the data part of a FITS HDU). Prototype programs exist to produce and manipulate XAS files, to convert them to FITS, and also to convert them between different operating systems without an intermediate FITS file Given this, the basic requirement from the beginning was to keep a one-to-one "mappability" to FITS. For this for instance I forced the LENGTH of a kewyord NAME to 8 characters, or the LENGTH of a CHARACTER keyword field to 68, etc. Therefore I believe there is PLENTY of conventions which can be discussed and agreed over HEAFITS, some of them being "first order" issues (like the identification of "basic" data structures mentioned above) and other being "second order" issues (like names of columns and keywords, as mentioned in Mike Corcoran's mail) or even lower order issues (one of these being the physical unit convention being recently discussed ... again the wording "lower order" has no contemptuous meaning ... actually the lowest the order, the easiest the agreement). From heafits@legacy.gsfc.nasa.gov Wed Apr 28 16:09:58 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["856" "Wed" "28" "April" "93" "16:08:04" "-0400" "William Pence" "pence@tetra.gsfc.nasa.gov " nil "17" "Re: OGIP FITS conventions (ROSAT et al.)" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA26441; Wed, 28 Apr 93 16:09:56 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA24655; Wed, 28 Apr 93 16:08:04 -0400 Message-Id: <9304282010.AA12345@tetra.gsfc.nasa.gov> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: pence@tetra.gsfc.nasa.gov (William Pence) Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: Re: OGIP FITS conventions (ROSAT et al.) Date: Wed, 28 Apr 93 16:08:04 -0400 I've been meaning to reply to some of the many issues raised in Lucio's last message but have been short of time. One of the problems is that there are so many good issues raised here, that it is hard to know where to start in replying to all of them. As a suggestion for future posts, it would make it much easier for me to comment on various issues if the postings to this group are kept relatively short and focused on a single issue. If too many different things are brought up at once, I get overloaded and may not be able to respond as well as I would like to all the individual issues. Also these issues will probably get a fuller discussion from a wider range of people if the issues are raised one by one over a reasonable period of time. I hope to reply to some of the specific issues raised by Lucio over the next few days... -Bill Pence From heafits@legacy.gsfc.nasa.gov Wed Apr 28 16:34:15 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1276" "Wed" "28" "April" "93" "16:32:28" "-0400" "William Pence" "pence@tetra.gsfc.nasa.gov " nil "26" "Re: RE: about OGIP/93-001" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA26447; Wed, 28 Apr 93 16:34:13 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA25455; Wed, 28 Apr 93 16:32:28 -0400 Message-Id: <9304282034.AA12375@tetra.gsfc.nasa.gov> Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: pence@tetra.gsfc.nasa.gov (William Pence) Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: Re: RE: about OGIP/93-001 Date: Wed, 28 Apr 93 16:32:28 -0400 Wayne Warren, Jr, in reply to Clive Page, wrote on 27 April: >> As to the need for DOUBLE PRECISION: in my experience most astronomical >> positions are only precise to an arc second or so, so that REAL storage >> is usually sufficient. >> >I disagree with this, unless you're sure that you won't need positions >to the accuracy currently available in the new astrometric catalogs. >Single precision truncates to about 0.36 arcsec (depending on the RA). >The new catalogs are better than this. I suggest storing *astrometric* >equatorial positions in double precision (REAL*8) to be safe. I don't see what the issue is here. The designer of a particular FITS file should be able to choose to use whatever precision is required when storing the RA and DEC values in the file. In some cases this will require using full double precision values, but in other cases single precision floating point, or even scaled integer*2 values (which can give a precision of about +/- 20 arcsec) may be more than adequate and may be usefully used to reduce the total size of the FITS file. FITS readers which process the RA and DEC values in some way should in general read the RA and DEC values internally into double precision variables, so as to not lose any precision. -Bill Pence From heafits@legacy.gsfc.nasa.gov Thu Apr 29 07:21:28 1993 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1039" "Thu" "29" "April" "93" "07:19:42" "-0400" "Lucio Chiappetti" "lucio@ifctr.mi.cnr.it" nil "22" "Re: OGIP FITS conventions (ROSAT et al.)" "^From:" nil nil "4"]) Return-Path: Received: from legacy.gsfc.nasa.gov by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA27837; Thu, 29 Apr 93 07:21:27 EDT Received: by legacy.gsfc.nasa.gov (5.57/Ultrix3.0-C) id AA01070; Thu, 29 Apr 93 07:19:42 -0400 Message-Id: Originator: heafits@legacy.gsfc.nasa.gov Errors-To: oneel@arupa.gsfc.nasa.gov Reply-To: heafits@legacy.gsfc.nasa.gov Version: 5.5 -- Copyright (c) 1991/92, Anastasios Kotsikonas From: Lucio Chiappetti Sender: heafits@legacy.gsfc.nasa.gov To: dwells@fits.CV.NRAO.EDU Subject: Re: OGIP FITS conventions (ROSAT et al.) Date: Thu, 29 Apr 93 07:19:42 -0400 On Wed, 28 Apr 1993, William Pence wrote: > I've been meaning to reply to some of the many issues raised in Lucio's last > message but have been short of time. One of the problems is that there > are so many good issues raised here, that it is hard to know where to > start in replying to all of them. As a suggestion for future posts, > it would make it much easier for me to comment on various issues if I do agree as general principle. I thought to condense most of the points in a single mail on one hand because maybe not all people would feel all points worth further discussion, so having the full list there may help the selection (this was just the start, if you want sort of an attempt to define a "charter" for HEAFITS). On the other hand I had also been quite busy in the previous days (and will be in the next ones, a common problem, isn't it ?) and I found easier to condense all in one. But for the future I strogly support Bill's recommendation. Lucio