From dwells@nrao.edu Mon Dec 9 13:19:29 1996 Path: solitaire.cv.nrao.edu!news.cv.nrao.edu!dwells From: dwells@nrao.edu (Don Wells) Newsgroups: sci.image.processing,sci.data.formats,comp.sys.sgi.graphics Subject: Re: FIT file format Date: 09 Dec 1996 18:16:32 GMT Organization: nrao Lines: 53 Message-ID: References: <32A5F2E6.721D@netime.com> NNTP-Posting-Host: fits.cv.nrao.edu In-reply-to: Christoph Schittko's message of Wed, 04 Dec 1996 22:53:42 +0100 Xref: solitaire.cv.nrao.edu sci.image.processing:24726 sci.data.formats:1740 comp.sys.sgi.graphics:17524 "CS" == Christoph Schittko writes: CS> I am trying to find some information on the FIT file format that is CS> supported by SGI's ImageVision Library. Does anyone know where to find CS> some information on the file structure and the different color models ? 'FIT' in the SGI library is probably FITS [Flexible Image Transport System], the standard data interchange and archival format of the worldwide astronomy community. See: The FITS newsgroup: sci.astro.fits FITS Basics and Information http://ssdoo.gsfc.nasa.gov/astro/fits/basics_info.html "Definition of FITS" [the FITS standard] Postscript, 80_pages, 430_KB ftp://nssdc.gsfc.nasa.gov/pub/fits/fits_standard.ps A User's Guide for the Flexible Image Transport System (FITS) Postscript, 100_pages, 574_KB ftp://nssdc.gsfc.nasa.gov/pub/fits/users_guide.ps FITS archive at NRAO http://fits.cv.nrao.edu/ FITS Support Office Home Page http://ssdoo.gsfc.nasa.gov/astro/fits/fits_home.html -=-=-=- FITS files are bytestreams which are regarded as being composed of logical records with length 2880_bytes; i.e. the byte count of a FITS file should be a multiple of 2880. The first 10 bytes of all FITS files are 'SIMPLE = ' (i.e., uppercase 7-bit ASCII codes for S, I, M, P, L, E, two ASCII blanks, ASCII equal and an ASCII blank; this is the "signature" of FITS). The first logical record of all FITS files is a header, which is regarded as 36 lines of 80 characters (36*80=2880), with the lines _not_ delimited by newline codes. Examination of a FITS file with GNU Emacs and a window width of about 80 characters is instructive. Simple FITS files (the most common case) transmit n-dimensional digital images with 8-, 16-, 32- or 64-bits per pixel. More complicated FITS files can transmit sets of tables and/or sets of images in addition to or instead of the primary image; the table and image "extensions" (in FITS-speak) have ASCII headers of their own in the same style as the primary header. There is no color model associated with FITS files. -- Donald C. Wells Associate Scientist dwells@nrao.edu http://fits.cv.nrao.edu/~dwells National Radio Astronomy Observatory +1-804-296-0277 520 Edgemont Road, Charlottesville, Virginia 22903-2475 USA From dwells@nrao.edu Mon Dec 9 17:32:54 1996 Path: solitaire.cv.nrao.edu!news.cv.nrao.edu!dwells From: dwells@nrao.edu (Don Wells) Newsgroups: sci.image.processing,sci.data.formats,comp.sys.sgi.graphics Subject: Re: FIT file format Date: 09 Dec 1996 18:16:32 GMT Organization: nrao Lines: 53 Message-ID: References: <32A5F2E6.721D@netime.com> NNTP-Posting-Host: fits.cv.nrao.edu In-reply-to: Christoph Schittko's message of Wed, 04 Dec 1996 22:53:42 +0100 Xref: solitaire.cv.nrao.edu sci.image.processing:24726 sci.data.formats:1740 comp.sys.sgi.graphics:17524 "CS" == Christoph Schittko writes: CS> I am trying to find some information on the FIT file format that is CS> supported by SGI's ImageVision Library. Does anyone know where to find CS> some information on the file structure and the different color models ? 'FIT' in the SGI library is probably FITS [Flexible Image Transport System], the standard data interchange and archival format of the worldwide astronomy community. See: The FITS newsgroup: sci.astro.fits FITS Basics and Information http://ssdoo.gsfc.nasa.gov/astro/fits/basics_info.html "Definition of FITS" [the FITS standard] Postscript, 80_pages, 430_KB ftp://nssdc.gsfc.nasa.gov/pub/fits/fits_standard.ps A User's Guide for the Flexible Image Transport System (FITS) Postscript, 100_pages, 574_KB ftp://nssdc.gsfc.nasa.gov/pub/fits/users_guide.ps FITS archive at NRAO http://fits.cv.nrao.edu/ FITS Support Office Home Page http://ssdoo.gsfc.nasa.gov/astro/fits/fits_home.html -=-=-=- FITS files are bytestreams which are regarded as being composed of logical records with length 2880_bytes; i.e. the byte count of a FITS file should be a multiple of 2880. The first 10 bytes of all FITS files are 'SIMPLE = ' (i.e., uppercase 7-bit ASCII codes for S, I, M, P, L, E, two ASCII blanks, ASCII equal and an ASCII blank; this is the "signature" of FITS). The first logical record of all FITS files is a header, which is regarded as 36 lines of 80 characters (36*80=2880), with the lines _not_ delimited by newline codes. Examination of a FITS file with GNU Emacs and a window width of about 80 characters is instructive. Simple FITS files (the most common case) transmit n-dimensional digital images with 8-, 16-, 32- or 64-bits per pixel. More complicated FITS files can transmit sets of tables and/or sets of images in addition to or instead of the primary image; the table and image "extensions" (in FITS-speak) have ASCII headers of their own in the same style as the primary header. There is no color model associated with FITS files. -- Donald C. Wells Associate Scientist dwells@nrao.edu http://fits.cv.nrao.edu/~dwells National Radio Astronomy Observatory +1-804-296-0277 520 Edgemont Road, Charlottesville, Virginia 22903-2475 USA From bljones@winter.ncsa.uiuc.edu Thu Dec 12 16:28:35 1996 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!aruba.odu.edu!news.larc.nasa.gov!night.primate.wisc.edu!news.he.net!news.dra.com!feed1.news.erols.com!howland.erols.net!vixen.cso.uiuc.edu!bljones From: bljones@winter.ncsa.uiuc.edu (Barbara Jones ) Newsgroups: sci.data.formats Subject: HDF Newsletter 23 Date: 10 Dec 1996 22:06:05 GMT Organization: University of Illinois at Urbana Lines: 213 Message-ID: <58kmsd$bmq@vixen.cso.uiuc.edu> NNTP-Posting-Host: winter.ncsa.uiuc.edu -------------------------------------------------------------------------- To subscribe/unsubscribe to the hdfnews mailing list, please send your request to ncsalist@ncsa.uiuc.edu with the appropriate command (e.g. subscribe hdfnews, unsubscribe hdfnews, help) in the *body* of the message. -------------------------------------------------------------------------- Newsletter #23 December 9, 1996 CONTENTS . HDF 4.1 Beta 1 Release A. New Features and Changes B. Platforms Tested C. Known Problems D. Important Fixes . The NCSA Collaborative HDF Viewer . How HDF mixes with Java . New Tools from Users HDF 4.1b1 Release ----------------- HDF 4.1 Beta 1 is now available on the NCSA anonymous ftp server (ftp.ncsa.uiuc.edu) in the directory HDF/HDF_4.1b1/. A. New Features and Changes: o Attributes are now supported in both the vdata and vgroup APIs. In the vdata API, attributes can be attached to either vdata fields or vdatas; in the vgroup API, attributes can be attached to vgroups. This new functionality can be used to attach attributes to vdatas and vgroups created by earlier versions of the HDF library. However, the old versions of the HDF library cannot read the new version vdatas and vgroups. A vdata/vgroup having attributes will become a new version vdata/vgroup. For more information, please refer to the file ../release_notes/vattr.txt, as well as the man pages for the new functions. o Data chunking is now supported in SD scientific data sets. When data chunking is used, an n-dimensional SDS is stored as a series of n-dimensional chunks, improving performance on certain types of partial read operations. New routines for creating and manipulating chunked SD scientific data sets have been provided, and two preexisting SD I/O routines, SDreaddata and SDwritedata, have also been modified to work with chunked SDSs. For more information, please refer to the file ../release_notes/sd_chunk_examples.txt, as well as the man page for sd_chunk. More information will be included in the HDF 4.1 documentation, which will be available with the release of HDF 4.1. o Due to certain limitations in the way compressed SDS datasets are stored, data which has been compressed is not completely writable in ways that uncompressed datasets are. The "rules" for writing to a compressed dataset are as follows: (1) Write an entire dataset that is to be compressed. i.e. build the dataset entirely in memory, then write it out with a single call. (2) Append to a compressed dataset. i.e. write to a compressed dataset that has already been written out by adding to the unlimited dimension for that dataset. (3) For users of HDF 4.1, write to any subset of a compressed dataset that is also chunked. Please refer to the ../release_notes/comp_SDS.txt file for more information. o A new file, ../release_notes/compile.txt, contains instructions on compiling applications on the supported platforms. If you encounter problems with it, please let us know at hdfhelp@ncsa.uiuc.edu. o SGI has changed some compiler default settings in IRIX 6.2. We decided to explicitly define the settings of various ABI related options. For the 64 bit OS ("uname -s" returns IRIX64), HDF uses "-64 -mips4" code. For the traditional 32 bit OS ("uname -s" returns IRIX), HDF uses "-32 -mips2". To use n32 mode on IRIX64, HDF uses "-n32 -mips3" code. Note that in the previous release (4.0r2), HDF used only "-n32". In IRIX 6.1 and before, "-n32" defaulted to "-mips4" code but in IRIX 6.2, it defaults to mips3 or mips4 code. We decided to explicitly set it to "-n32 -mips3". Therefore, applications linking with the HDF library must be compiled with the same explicit ABI options. B. Platforms Tested: HDF 4.1b1 has been tested on the following platforms: DEC Alpha/Digital Unix 3.2 DEC Alpha/OpenVMS AXP v6.2 DEC VAX OpenVMS v6.2 Free BSD 2.2 HP-UX 9.03 IRIX 5.3 IRIX 6.2_64 IRIX 6.2_n32 Linux ELF 1.2.13 (C only) Macintosh PowerPC (C only) (not ready yet) SP2 4.1 Solaris 2.5 SunOS 4.1.3 Windows NT/95 (C only) YMP 9.0.2asC C. Known Problems: o With the SD interface, you are unable to overwrite existing compressed data, that is not stored in "chunked" form. This is due to compression algorithms not being suitable for "local" modifications in a compressed datastream. For more information, please refer to the ../release_notes/comp_SDS.txt file. o With 4.0r1p1, you could type "hdp list -a " to get a list of the file attributes associated with a file. This does not currently work. o There are no plans to add the DF24writeref function to the DF24 interface. This function will be removed from the documentation. o When running "make test" on OpenVMS, Test 3 (float32) of the chunking tests fails, and has therefore been commented out. o When running the tests on Window NT/95, Test 2 (uint16) of the chunking tests fails, and will be commented out. o The ncgen test failed on VAX/OpenVMS. D. Important Fixes: o If you opened a file in Read Only mode with the SD interface (using SDstart), it would create the file if the file did not exist. This no longer occurs. o HDF 4.0r2 did not recognize JPEG images created by HDF 3.3r4. This has been fixed. See the ../release_notes/bug_fixed.txt file for more information on bugs fixed in this release. The NCSA Collaborative HDF Viewer --------------------------------- An alpha demonstration release of the NCSA Collaborative HDF Viewer is now available. The Collaborative HDF Viewer was created by adapting the stand-alone NCSA Java-based HDF Browser to use the NCSA Habanero Collaborative Framework. For information refer to: http://hdf.ncsa.uiuc.edu/hdf/java/chdv/index.html How HDF Mixes With Java ----------------------- In the last few months, the HDF group has been mulling over what to do with Java and HDF. Our investigation has led us to learn more than we really wanted to know about wrapping C libraries with Java and about Java's impoverished data types (compared to C). We still don't have a definite plan, but we do have some notes on what we've learned: http://hdf.ncsa.uiuc.edu/horizon/java-and-hdf.html Many of the comments are relevant to any Java application that wants to wrap existing C libraries, so they may be useful to others. We'd welcome feedback, expressions of interest, and pointers to similar work. New Tools From Users -------------------- Several users have let us know about new tools they have developed that use HDF. For more information on tools that users have developed, please refer to the "What Tools use HDF?" section off of our home page (http://hdf.ncsa.uiuc.edu/tools.html). If you have an application that you would like to add to the list, just let us know at hdfhelp@ncsa.uiuc.edu. HDFLook HDFLook is a friendly Motif HDF viewer, useful for quality control of Scientific Datasets. It allows easy access to physical values and ancillary data, and includes 2-D graphics (radial, histogram). It is currently available for Alpha VMS 3.2, HP-UX 10.1, IRIX 5.3, and AIX. (ftp://loaser.univ-lille1.fr/) WIM Windows Image Manager (WIM) is a general purpose image display and analysis tool for the Microsoft Windows graphical user interface, with special features for analyzing satellite images. (URL: http://spode.ucsd.edu/) HDF Browser Fortner Research has developed the HDF Browser, which is a free utility that lets you open an HDF file, examine its hierarchy and components, and then view and edit its pieces. It runs on Macintosh and Windows NT/95. (URL: http://www.fortner.com/docs/product_hdf_b.html) From warnock@clark.net Sat Dec 28 01:38:23 1996 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!news-server.ncren.net!concert!rutgers!iag.net!enews.sgi.com!news.sgi.com!mr.net!news.clark.net!usenet From: Archie Warnock Newsgroups: sci.data.formats Subject: Re: Why have a standard data format? Date: Thu, 26 Dec 1996 16:04:48 -0500 Organization: A/WWW Enterprises Lines: 44 Message-ID: <32C2E870.61BA85E6@clark.net> References: <59ub68$plb@noao.edu> NNTP-Posting-Host: awww.clark.net Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailer: Mozilla 3.0Gold (X11; I; Linux 2.0.20 i586) Rob Seaman wrote: > I think the Risks article points up two requirements for either a > portable data interchange format or an archival format. (One way to > look at a data archive is as one-way data interchange with the future.) Listen to the man - he speaks the truth :-) > The first requirement is that the data format must be explicitly > specified, not just defined via a software interface. Software > changes - data must be allowed to stay the same. Wide adoption of > a standard also benefits from separating those responsible for its > implementation(s) from whatever group is ultimately responsible for > its definition. This is an important point. Who out there seriously thought, 10 years ago, that a Fortran-based software API would be obsolete? And yet, we're not too many years from reaching a point where young programmers won't have a clue what to do with data if the only way to access it is via a set of Fortran calls. For a data format to be useful as an archival format, it's critical that there be a byte-level definition describing how the data is written. > Secondly, a portable standard requires portable documentation. > Certainly there isn't much logic in providing access to an alternative > format, but requiring access to tools that read the original format > in order to use the alternative. Not only must the documentation be portable, it must be both locatable and accessible. Even plain ASCII documents won't be of any use if you can't find them and get a copy. Those of us in astronomy (and other areas, I'm sure) are familiar with the dreaded "institutional reports" that contain important information but don't exist anywhere anymore. We cannot afford for this to happen to valuable scientific data. Gotta think long term... -- Archie -- Archie Warnock Internet: warnock@clark.net -- A/WWW Enterprises Phone/FAX: 301-854-2987 -- http://www.clark.net/pub/warnock/warnock.html -- As a matter of fact, I _do_ speak for my employer. From seaman@noao.edu Sat Dec 28 14:08:51 1996 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!portal.gmu.edu!newsfeed.internetmci.com!newshub.csu.net!news.Cerritos.edu!news.Arizona.EDU!CS.Arizona.EDU!noao!seaman From: seaman@noao.edu (Rob Seaman) Newsgroups: sci.data.formats Subject: Why have a standard data format? Date: 26 Dec 1996 17:04:08 GMT Organization: National Optical Astronomy Observatories, Tucson, AZ, USA Lines: 94 Message-ID: <59ub68$plb@noao.edu> NNTP-Posting-Host: bigx.tuc.noao.edu I posted the appended Risks Digest message in sci.astro.fits, and a colleague suggested posting a similar pointer in this newsgroup. The article can also be reached at the URL: http://catless.ncl.ac.uk/Risks/18.71.html#subj11 I think the Risks article points up two requirements for either a portable data interchange format or an archival format. (One way to look at a data archive is as one-way data interchange with the future.) The first requirement is that the data format must be explicitly specified, not just defined via a software interface. Software changes - data must be allowed to stay the same. Wide adoption of a standard also benefits from separating those responsible for its implementation(s) from whatever group is ultimately responsible for its definition. Secondly, a portable standard requires portable documentation. Certainly there isn't much logic in providing access to an alternative format, but requiring access to tools that read the original format in order to use the alternative. Rob Seaman -- RISKS-LIST: Risks-Forum Digest Monday 23 December 1996 Volume 18 : Issue 71 FORUM ON RISKS TO THE PUBLIC IN COMPUTERS AND RELATED SYSTEMS (comp.risks) ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator [...] Date: Sat, 21 Dec 1996 06:29:39 -0500 (EST) From: "Darrin B. Jewell" Subject: Microsoft documents and Rosetta stones (Giersig, RISKS-18.70) This past summer I was getting increasingly frustrated by users who were sending me e-mail with Microsoft Word ".DOC" files attached to them via MIME encodings. I do not own a copy of Microsoft Word and did not want to purchase one simply so that I could read documents people send to me. Instead, I wanted the specification for Microsoft Word file format so that I would have a Rosetta stone for the documents I wished to read. I contacted Microsoft's customer support and was informed of the following: 1. The specification for Microsoft Word file format is unpublished proprietary information. However, I could download a free reader for Microsoft Word files which would run under one of Microsoft's operating systems. Source code for the reader was not available. 2. I could ask the sender of the message to send me the attachment encoded in Rich Text Format which is the official export format for Microsoft Word documents. The specification for Rich Text Format was publically available from Microsoft. Since I did not own a computer running one of Microsoft's operating systems, I asked Microsoft for the specification for Rich Text Format files. As a computer programmer, this was also a more useful and interesting form of Rosetta stone than a precompiled translating program. I was then directed to the file GC0165.EXE on the Microsoft ftp site, which I was able to download and unzip. (Itself another decoding adventure.) Included was the file GC0165.DOC, a Microsoft Word format file containing the specification I desired. The included README.TXT file contained the following comment in plain ASCII: The GC0165.DOC file included in this compressed file is the Rich Text Format (RTF) Specification version 1.3. The document is in Word 6.0 for Windows format. If you have neither Word 6.0 for Windows nor Word 2.0 for Windows with the 6.0-to-2.0 converter, you will need to call Microsoft Product Support Services at (206) 462-9673 to obtain a hard copy of the document. At this point, I decided it was fastest to have my friend who owned Microsoft Word print out the RTF specification for me. Since this experience, I usually ask people who wish to send me Word documents to send them in RTF format. When I explain to people the RISKS involved in using documents without open standards, I get comments about being ridiculous and pedantic or perhaps a blink and a "So what?" Even though Microsoft Word supports an "official export format", it is clearly not obvious to everyone why it should be used. Darrin Reused without explicit authorization under blanket permission granted for all Risks-Forum Digest materials. The author(s), the RISKS moderator, and the ACM have no connection with this reuse. -- seaman@noao.edu, http://iraf.noao.edu/~seaman NOAO, 950 N Cherry Ave, Tucson AZ 85719, 520-318-8248 PGP: 98 8D 8B 49 74 9A 41 88 3A 43 87 54 51 BF 30 4B