From mskuhn@cip.informatik.uni-erlangen.de Mon Feb 5 10:11:08 1996 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!portal.gmu.edu!solaris.cc.vt.edu!news.mathworks.com!gatech!swrinde!howland.reston.ans.net!blackbush.xlink.net!news.uni-mainz.de!news.th-darmstadt.de!uni-erlangen.de!rznews.rrze.uni-erlangen.de!unrza3.dialin.rrze.uni-erlangen.de!usenet From: Markus Kuhn Newsgroups: sci.data.formats Subject: Re: time information in ISO 8601 format Date: 31 Jan 1996 14:19:01 GMT Organization: Markus Kuhn, 91080 Uttenreuth, Germany Lines: 418 Message-ID: <4entol$1bl@cortex.dialin.rrze.uni-erlangen.de> References: NNTP-Posting-Host: unrza3.dialin.rrze.uni-erlangen.de Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="-------------------------------334859247804279671789585569" X-Mailer: Mozilla 1.1N (X11; I; Linux 1.2.13 i486) X-URL: file:/home/mskuhn/proj/doc/iso-time.html This is a multi-part message in MIME format. ---------------------------------334859247804279671789585569 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii teuben@taurus.astro.umd.edu (Peter Teuben) writes: >Check > ftp.uni-erlangen.de:/pub/doc/ISO/ISO8601.ps.Z This address is now and there is also a web page available about ISO 8601 on in case this machine has already recovered from last weeks harddisk catastrophy when you read this. I'll append the Web page below. Markus -- Markus Kuhn, Computer Science student -- University of Erlangen, Internet Mail: - Germany WWW Home: ---------------------------------334859247804279671789585569 Content-Transfer-Encoding: 7bit Content-Type: text/html International Standard Date and Time Notation

A Summary of the International Standard Date and Time Notation

International Standard ISO 8601 specifies numeric representations of date and time. This standard notation helps to avoid confusion in international communication caused by the many different national notations and increases the portability of computer user interfaces. In addition, these formats have several important advantages for computer usage compared to other traditional date and time notations. The time notation described here is already the de-facto standard in almost all countries and the date notation is becoming increasingly popular.

Especially software engineers who design user interfaces, file formats, and communication protocols should be familiar with ISO 8601.

Contents: Date, Time of Day, Time Zone.

Date

The international standard date notation is

YYYY-MM-DD

where YYYY is the year in the usual Gregorian calendar, MM is the month of the year between 01 (January) and 12 (December), and DD is the day of the month between 01 and 31.

For example, the fourth day of February in the year 1995 is written in the standard notation as

1995-02-04

Other commonly used notations are e.g. 2/4/95, 4/2/95, 95/2/4, 4.2.1995, 04-FEB-1995, 4-February-1995, and many more. Especially the first two examples are dangerous, because as both are used quite often in the USA and in Great Britain and both can not be distinguished, it is unclear whether 2/4/95 means 1995-04-02 or 1995-02-04. The date notation 2/4/5 has at least six reasonable interpretations (assuming that only the twentieth and twenty-first century are reasonable candidates in our live time).

Advantages of the ISO 8601 standard date notation compared to other commonly used variants:

easily readable and writeable by software (no 'JAN', 'FEB', ... table necessary)
easily comparable and sortable with a trivial string comparison
language independent
can not be confused with other popular date notations
consistency with the common 24h time notation system, where the larger units (hours) are also written in front of the smaller ones (minutes and seconds)
strings containing a date followed by a time are also easily comparable and sortable (e.g. write "1995-02-04 22:45:00")
the notation is short and has constant length, which makes both keyboard data entry and table layout easier
identical to the Chinese date notation, so the largest cultural group (>25%) on this planet is already familiar with it :-)
date notations with the order "year, month, day" are in addition already widely used e.g. in Japan, Korea, Hungary, Sweden, Finland, Denmark, and a few other countries and people in the US are already used to at least the "month, day" order
a 4-digit year representation avoids overflow problems after 1999-12-31

As dates will look a little bit strange anyway starting with 2000-01-01 (e.g. like 1/1/0), it has been suggested that the year 2000 is an excellent opportunity to change to the standard date notation.

Apart from the recommended primary standard notation YYYY-MM-DD, ISO 8601 also specifies a number of alternative formats for use in applications with special requirements. All of these alternatives can easily and automatically be distinguished from each other:

The hyphens can be omitted if compactness of the representation is more important than human readability, for example as in

19950204

For situations where information about the century is really not required, a 2-digit year representation is available:

95-02-04 or 950204

If only the month or even only the year is of interest:

1995-02 or 1995

In commercial and industrial applications (delivery times, production plans, etc.), especially in Europe, it is often required to refer to a week of a year. Week 01 of a year is per definition the first week which has the Thursday in this year, which is equivalent to the week which contains the fourth day of January. In other words, the first week of a new year is the week which has the majority of its days in the new year. Week 01 might also contain days from the previous year and the week before week 01 of a year is the last week (52 or 53) of the previous year even if it contains days from the new year. A week starts with Monday (day 1) and ends with Sunday (day 7). For example, the first week of the year 1997 lasts from 1996-12-30 to 1997-01-05 and can be written in standard notation as

1997-W01 or 1997W01

The week notation can also be extended by a number indicating the day of the week. For example the day 1996-12-31 which is the Tuesday (day 2) of the first week of 1997 can also be written as

1997-W01-2 or 1997W012

for applications like industrial planning where many things like shift rotations are organized per week and knowing the week number and the day of the week is more handy than knowing the day of the month.

An abbreviated version of the year and week number like

95W05

is sometimes useful as a compact code printed on a product that indicates when it has been manufactured.

The ISO standard avoids explicitly stating the possible range of week numbers, but this can easily be deduced from the definition:

Theorem: Possible week numbers are in the range 01 to 53. A year always has a week 52.
Proof: Per definition, the first week of a year is W01 and consequently days before week W01 belong to the previous year and so there is no week with lower numbers. Considering the highest possible week number, the worst case is a leap year like 1976 that starts with a Thursday, because this keeps the highest possible number of days of W01 in the previous year, i.e. 3 days. In this case, the Sunday of W52 of the worst case year is day number 4+51*7=361 and 361-366=5 days of W53 belong still to this year which guarantees that in the worst case year day 4 (Thursday) of W53 is not yet in the next year, so a week number 53 is possible. For example, the 53 weeks of the worst case year 1976 started with 1975-12-29 = 1976-W01-1 and ended with 1977-01-02 = 1976-W53-7. On the other hand, considering the lowest number of the last week of a year, the worst case is a non-leap year like 1999 that starts with a Friday, which ensures that the first three days of the year belong to the last week of the previous year. In this case, the Sunday of week 52 would be day number 3+52*7=367, i.e. only the last 367-365=2 days of the W52 reach into the next year and consequently, even a worst case year like 1999 has a week W52 including the days 1999-12-27 to 2000-01-02. q.e.d.

[Unfortunately, the current version of the C programming language standard provides in the strftime() function no means to generate the ISO 8601 week notation. A required extension would be four new formatting codes: for the year of the week to which the specified day belongs (both 2-digit and 4-digit), for the number of the week between 01 and 53, and for the day of the week between 1 (Monday) and 7 (Sunday). Another trivial mistake in the description of strftime() in the C standard is that the range of seconds goes from 00 to 61, because at one time only one single leap second 60 can be inserted into UTC and consequently there will never be a leap second 61.]

Both day and year are useful units of structuring time, because the position of the sun on the sky, which influences our lives, is described by them. However the 12 months of a year are of some obscure mystic origin and have no real purpose today except that people are used to having them (they do not even describe the current position of the moon). In some applications, a date notation is prefered which uses only the year and the day of the year between 001 and 365 (366 in leap years). The standard notation for this variant representing the day 1995-02-04 (that is day 035 of the year 1995) is

1995-035 or 1995035

Leap years are years with an additional day YYYY-02-29, where the year number is a multiple of four with the following exception: If a year is a multiple of 100, then it is only a leap year if it is also a multiple of 400. For example, 1900 was not a leap year, but 2000 is one.

Time of Day

The international standard notation for the time of day is

hh:mm:ss

where hh is the number of complete hours that have passed since midnight (00-24), mm is the number of complete minutes that have passed since the start of the hour (00-59), and ss is the number of seconds since the start of the minute (00-60). The value 60 for ss appears only in case of an inserted leap second into an atomic time scale like UTC in order to keep it synchronized with a less constant astronomical time scale like UT1. The hour value 24 is only possible when the minute and second values are zero. An example time is

23:59:59

which represents the time one second before midnight.

As with the date notation, the separating colons can also be omitted as in

235959

and the precision can be reduced by omitting the seconds or both the seconds and minutes as in

23:59, 2359, or 23

It is also possible to add fractions of a second after a decimal dot, e.g. the time 5.8 ms before midnight can be written as

23:59:59.9942 or 235959.9942

As every day both starts and ends with midnight, the two notations 00:00 and 24:00 are available to distinguish the two midnights which can be associated with one date. This means that the following two strings refer to exactly the same point in time:

1995-02-04 24:00 = 1995-02-05 00:00

In case an unambiguous representation of time is required, 00:00 is usually the prefered notation for midnight, which is also what most digital clocks display.

A remark for readers from the US:

The 24h time notation specified here has already been the de-facto standard all over the world in written language for decades, except in some English speaking countries where still a notation with hours between 1 and 12 and additions like "a.m." and "p.m." is in wide use. The common 24h international standard notation starts to get widely used now even in England. Other languages than English don't even have abbreviations like "a.m." and "p.m." and the 12h notation is certainly hardly ever used on Continental Europe to write or display a time. The old English 12h notation has many disadvantages like

it is not clear, how 00:00, 12:00 and 24:00 are represented (even encyclopedias and style manuals contain contradicting descriptions and a common quick fix seems to be to avoid "12:00 a.m./p.m." altogether and write "noon", "midnight" or "12:01 a.m./p.m." instead)
it makes people often believe that the next day starts at the overflow from "12:59 a.m." to "1:00 a.m." (which is a common problem not only when people try to program the timer of VCRs shortly after midnight)
it is not easily comparable with a string compare operation
it is not immediately clear for the unaware, whether the time between "12:00 a.m./p.m." and "1:00 a.m./p.m." starts at 00:00 or at 12:00, i.e. the English 12h notation is more difficult to understand

The 12h time is simply a relic from the dark ages when Roman numerals were used, the number zero had not yet been invented and analog clocks where the only known form of displaying a time. Please avoid using it, especially in technical applications! Even in the US, the widely respected Chicago Manual of Style recommends now to use the international standard time notation in publications.

Time Zone

Without any further additions, a date and time as written above is assumed to be in some local time zone. In order to indicate that a time is measured in Universal Time (UTC), you can append a capital letter Z to a time as in

23:59:59Z or 2359Z

[The Z stands for the "zero meridian" which goes through Greenwich in London and is also commonly used in radio communication where it is pronounced "Zulu" (the word for Z in the NATO radio alphabet). Universal Time (sometimes also called "Zulu Time") was called Greenwich Mean Time (GMT) before 1972, however this term should no longer be used. Since the introduction of an international atomic time scale, almost all existing civil time zones are now related to UTC which is slightly different from the old and now unused GMT.]

The strings

+hh:mm, +hhmm, or +hh

can be added to the time to indicate that the used local time zone is hh hours and mm minutes ahead of UTC. For time zones west of the zero meridian, which are behind UTC, the notation

-hh:mm, -hhmm, or -hh

is used instead. For example, Middle European Time (MET) is +0100 and US/Canadian Eastern Standard Time (EST) is -0500. The following strings all indicate the same point of time:

12:00Z = 13:00+01:00 = 0700-0500

There exists no international standard which specifies abbreviations for civil time zones like MET, EST, etc. and sometimes the same abbreviation is even used for two very different time zones. In addition, politicians enjoy modifying the rules for civil time zones, especially for daylight saving times, every few years, so the only really reliable way of describing a local time zone is to specify numerically the difference of local time to UTC. Better use directly UTC as your only time zone where this is possible and then you have not to worry about time zones and daylight saving time changes at all.

This was a brief overview of the ISO 8601 standard that covers only the most useful notations and includes some additional related information. The full standard defines in addition a number of more exotic notations including some for periods of time. The ISO 8601 document is unfortunately not available online and interested people will have to order a paper copy from

International Organization for Standardization
Case postale 56
1, rue de Varembé
CH-1211 Genève 20
Switzerland

phone: +41 22 749 01 11
fax: +41 22 733 34 30
e-mail: sales@isocs.iso.ch

A more detailed online summary of ISO 8601 than this one is the text ISO 8601:1988 Date/Time Representations available >from ftp.informatik.uni-erlangen.de:pub/doc/ISO/ISO8601.ps.Z (PostScript, 16 kb, 5 pages) written by Gary Houston, which is now also available in HTML.

I wish to thank Edward M. Reingold for developing the fine GNU Emacs calendar functions, as well as Rich Wales, Mark Brader, and others in the comp.std.internat USENET discussion group for valuable comments about this document. Further comments and hyperlinks to this text are welcome.

Markus Kuhn <mskuhn@cip.informatik.uni-erlangen.de>, 1996-01-13 ---------------------------------334859247804279671789585569-- From Doug.Ilg@gsfc.nasa.gov Fri Feb 9 10:08:24 1996 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!maui.cc.odu.edu!news.larc.nasa.gov!ames!newsfeed.gsfc.nasa.gov!usenet From: Doug Ilg Newsgroups: sci.data.formats Subject: Re: HDF raster -> gif? Date: Thu, 08 Feb 1996 09:57:06 -0500 Organization: NASA Goddard Space Flight Center -- Greenbelt, Maryland USA Lines: 37 Message-ID: <311A0F42.41C6@gsfc.nasa.gov> References: <4fb5uf$t2k@news.cc.utah.edu> NNTP-Posting-Host: hops.stx.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailer: Mozilla 2.0b6a (X11; I; IRIX 5.3 IP22) Hi Mike, Mike Zulauf wrote: > I have some questions regarding raster images in HDF. Basically, > I need to translate them into gifs somehow. Does anyone know of a > utility (unix - Sun preferably) which could do this? You might try Reformat/XReformat from NCSA. I understand it's recently been updated, so I'm sure there's support for GIF (I think it was there before, anyway). You can get it from ftp://ftp.ncsa.uiuc.edu/ (same place you got HDF). > Can I use the following call to write a jpeg image to an HDF file? > > d8aimg(filename, image, width, height, compress) > > If I can, what do I use for 'compress', and how can I specify the > quality of the image? I would want to use a quality of 100. If this > isn't the proper way to do this, then what is? For your situation, you shouldn't need to, but to use JPEG in HDF, the most reasonable [and maybe the only] way is to call DFR8setcompress() before you make the call to DFR8addimage(). (Sorry, I don't remember all the FORTRAN names for these routines.) 'Setcompress will let you tune the JPEG algorithm in the normal way. You can find it on page 49 of the Feb 1994 edition of the HDF Version 3.3 Reference Manual. This page also gives the complete list of symbolic constants for use as the `compress' argument. Feel free to send email, if you have more questions. ==================================================================== Doug Ilg Hughes STX Corporation EOSDIS Standard Data Formats 7701 Greenbelt Road, Suite 400 Doug.Ilg@gsfc.nasa.gov Greenbelt, MD 20770 USA Voice: (301) 441-4089 FAX: (301) 441-1853 ================= I do not speak for HSTX or NASA ================== From gbryan@ncsa.uiuc.edu Mon Feb 12 12:00:53 1996 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!portal.gmu.edu!solaris.cc.vt.edu!newsfeed.internetmci.com!vixen.cso.uiuc.edu!usenet From: Greg Bryan Newsgroups: sci.data.formats Subject: Re: HDF raster -> gif? Date: 8 Feb 1996 19:08:38 GMT Organization: University of Illinois at Urbana Lines: 18 Message-ID: <4fdhnm$rsq@vixen.cso.uiuc.edu> References: <4fb5uf$t2k@news.cc.utah.edu> NNTP-Posting-Host: pangloss.ncsa.uiuc.edu Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailer: Mozilla 1.1N (X11; I; IRIX 5.2 IP22) X-URL: news:4fb5uf$t2k@news.cc.utah.edu > I have some questions regarding raster images in HDF. Basically, >I need to translate them into gifs somehow. Does anyone know of a >utility (unix - Sun preferably) which could do this? The San Diego Supercomputing Center's Image Tools should be able to do this directly. Get imconv from ftp://ftp.sdsc.edu/pub/sdsc/graphics/imtools (I know they have a sun version) and do something like: imconv image.hdf image.gif (A few caveats: I'm not sure if they support both 8 and 24 bit HDF rasters nor do I know what their current HDF version is, whether they support more than one image per file, etc. But it's worth a shot and imconv supports many different image types so its handy to have around.) Greg Bryan gbryan@ncsa.uiuc.edu From sxu@ncsa.uiuc.edu Tue Feb 13 13:53:27 1996 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!maui.cc.odu.edu!news.larc.nasa.gov!night.primate.wisc.edu!newsspool.doit.wisc.edu!uwm.edu!vixen.cso.uiuc.edu!sxu From: sxu@ncsa.uiuc.edu (Shiming Xu) Newsgroups: sci.data.formats Subject: HDF Newsletter #19 Date: 12 Feb 1996 20:29:46 GMT Organization: Nat'l Ctr for Supercomp App (NCSA) @ University of Illinois Lines: 140 Distribution: world Message-ID: <4fo7vq$imr@vixen.cso.uiuc.edu> NNTP-Posting-Host: void.ncsa.uiuc.edu Keywords: HDF4.0r1 release Originator: sxu@void HDF Newsletter 19 Feb. 9, 1996 Contents . HDF4.0r1 release . HDF4.0 documentation . Progress in HDF related projects . Platforms to be dropped . New HDFers HDF4.0r1 release ================ We are very glad to announce that HDF4.0r1 is now officially released. HDF4.0 represents a substantial revision, expansion and improvement of the HDF library over HDF3.3. Its new features include: . Interfaces that support multi-file/multi-object access for raster images and annotations, similar to the multifile SDS interface. . Support for n-bit integer data within the SDS interface. . Support for RLE, adaptive Huffman and Gzip deflation compression for SDSs. . Uses v5 of Independent JPEG Group (IJG) JPEG file access library . Support low level function for RLE, adaptive Huffman and Gzip deflation compression for any object. . Support for creation and search for external elements. . Support for external vdata elements. . Unified auto-configuration that eases the installation and test of HDF package on UNIX platforms. . A new version of dimension record decreasing the size of HDF files which contain large 1-dimensional SDS's. . A new HDF utility hdp -- an HDF file dumper . Full support for SGI Power Challenge running IRIX6.1 . Support for parallel I/O on the Thinking Machines'CM-5. . Modifications in the implementation of memory management that decrease the required memory size and execution time. . Integration of Pablo performance measurement software. . I/O performance improvements . Limited support for reading CDF files. Details are listed in the $(toplevel)/release_notes/ABOUT_4.0r1. HDF4.0r1 has been tested on: SunOS 4.1.4, Solaris 2.4, Solaris 2.5, IRIX5.3, IRIX6.1 (both 32-bit and 64-bit modes), HPUX 9.03, UNICOS 803, CM5, AIX v3.2, Digital Unix v3.2, OpenVMS AXP v6.2, Solarisx86, Linux(a.out) v1.2.4, and Free BSD 2.0. More details are included in the $(toplever)/INSTALL file. Many thanks to the HDF4.0 alpha and beta testers for their feedback, comments and suggestions. HDF4.0 documentation ==================== An HDF 4.0 User's Guide is available on the NCSA ftp server, ftp.ncsa.uiuc.edu, in subdirectory: /HDF/Documentation/HDF4.0/Users_Guide/. We are working on more documentation, including a new Getting started manual, a new HDF 4.0 Reference Manual, and a new HDF Specification and Developer's Guide. We will announce the availability of these documents as they are completed. Progress in HDF related projects ================================ BigHDF ------ In the last Newsletter we described the BigHDF project, in which we are building a prototype next-generation HDF. A proposed specification for the prototype is available on the web at http://hdf.ncsa.uiuc.edu/internal/NRA/BigHDFSpec.html There is also a reproduction of talk on BigHDF entitled "HDF: The Next Generation," given by Mike Folk and Quincey Koziol, at http://hdf.ncsa.uiuc.edu/nra/BigHDF/talk/coverpage.html We welcome your comments and ideas about BigHDF. HDF Support for Parallel Computing Environment ---------------------------------------------- With support from NASA's Advanced Information Systems Research project (AISR), we are designing new features to provide HDF library support for the parallel computing environment of Network of Workstations (HDF-NOW). Preliminary design has completed and we are in the stage of functional definitions. We plan to release a prototype version this year. Tentative platforms are the IBM SP2 and network of workstations like HP, SGI and Sun. More details are available at "http://hdf.ncsa.uiuc.edu/parallel_hdf.html". Platforms to be dropped ======================= Due to lack of resources and waning interest, we have decided to stop supporting HDF on the following platforms: DecStation/MIPSEL NeXT/Next-STEP Fujitsu VP/UXPM Intel i860 IRIX4.0 IRIX5.2 VAX/VMS If you need HDF support on these platforms, please contact us and we will try to help you port HDF 4.0 to them. New HDFers ========== Don Gibson, a University of Illinois M.B.A. student, has been working with the HDF group for several months. Don is helping to manage the concurrent HDF projects and assisting in the development of future projects. He is currently helping to write an HDF business plan. Barbara Jones recently joined us to help provide user-support to the HDF community. She comes to us with 16 years of programming and technical support experience. Barbara moved here from Providence, Rhode Island where she spent the last 6 years as a Technical Support Specialist at Cadre Tech. From ethan@ncar.ucar.edu Fri Feb 16 11:56:10 1996 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!portal.gmu.edu!solaris.cc.vt.edu!newsfeed.internetmci.com!vixen.cso.uiuc.edu!uchinews!ncar!usenet From: Ethan Alpert Newsgroups: sci.data.formats Subject: Re: HDF, netCDF Date: 14 Feb 1996 17:37:34 GMT Organization: very little Lines: 83 Message-ID: <4ft6kv$ovl@ncar.ucar.edu> References: <4floik$a74@controversy.admin.lsa.umich.edu> NNTP-Posting-Host: chiquita.scd.ucar.edu Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailer: Mozilla 1.1N (X11; I; IRIX 5.2 IP20) X-URL: news:EJH.96Feb14071712@larry.gsfc.nasa.gov ejh@larry.gsfc.nasa.gov (Edward Hartnett) wrote: >>>>>> "J" == Joel Parriott writes: > > J> I'm currently running Fortran simulations which produce multi-dimensional arrays [deleted: wants to move data from IBM RS6000 to Sun] > J> of HDF or netCDF. >I hate to be a data format heritic, but why can't you use binary? >Can't both machines write IEEE? I would have thought that this would >be the default output on both machines, but even if it is not, does >the IBM compiler not even offer such an option? Not all machines are IEEE binary. I believe the IBM machines DON'T use a hidden bit, but I might be wrong wrt AIX. This would make writing data on the IBM and moving it to the Sun difficult. Binary, regardless of the IEEE standard, is still very unportable. There are the DEC byte swapped machines and now you have 64 bit longs on some machines. CRAY's don't write IEEE by default, its a mess. > >ASCII obviously stinks. No way! I agree. > >(Again, playing the data format heritic) you could take this >oppurtunity to pack your data into 16 byte integers. It's a fairly >simple process. (I can send you a description of two possible methods >if you're interested). This would be nice and portable but think about the unpacking code given, the lack of standards, on the systems I mentioned before, would make this code highly machine dependent. GRIB packs data into integers, sort of, and my code to unpack the data has to be diferent on SUN's, CRAYS, and IBM's. So you don't gain anything here. Furthermore, the original poster's code is FORTRAN. I'd HATE to have to do all this bit-twidling in FORTRAN, yech! Anyways I'm really responding here to make a point that you are missing the point of "standard" data formats like HDF and netCDF. My point is the interface means everything and the format means nothing. For example, let's say you write your 16 bit integer packed format into a file. Two years later you find the binary file sitting in your directory. How do you know what the data is, what the units are, when you created it, what its dimensionality is and most importantly how do you know how its packed and where the code you wrote to unpack it is? "Standard" data formats alleviate these problems and provide you a way to store with your data information describing your data. Assign your data varibles and dimensions names, etc. But most importantly they give you a uniform interfacefor writing, reading and inquiring about data, that also is portable. These formats open the doors to develope usefull tools to browse and manipulate data. For example, here at NCAR I've worked on developing an interpreted language that is an example of what can be built using these kinds of standard interfaces.(http://ngwww.ucar.edu/ngdoc/ng4.0/ref/ncl/NclIntr.html). I'm currently adding the GRIB format to NCL. GRIB does not have an interface and it has been quite a chore to create one because of the forementioned binary difference between architectures. -ethan > > >-- >Edward Hartnett ejh@larry.gsfc.nasa.gov >* I don't speak for NASA or ARC, I just work for them! * > Goddard Distributed Active Archive Center (DAAC) >Check out cool Earth Science data at: http://daac.gsfc.nasa.gov/ > -- ------------------------------------------------------------------------------- Ethan Alpert ethan@ncar.ucar.edu ------------------------------------------------------------------------------- From hrodstein@aol.com Thu Feb 22 14:18:16 1996 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!maui.cc.odu.edu!news.larc.nasa.gov!lerc.nasa.gov!magnus.acs.ohio-state.edu!math.ohio-state.edu!howland.reston.ans.net!news-e1a.megaweb.com!newstf01.news.aol.com!newsbf02.news.aol.com!not-for-mail From: hrodstein@aol.com (HRodstein) Newsgroups: sci.data.formats Subject: Re: HDF, netCDF Date: 18 Feb 1996 23:48:40 -0500 Organization: America Online, Inc. (1-800-827-6364) Lines: 14 Sender: root@newsbf02.news.aol.com Message-ID: <4g8vf8$dr6@newsbf02.news.aol.com> References: Reply-To: hrodstein@aol.com (HRodstein) NNTP-Posting-Host: newsbf02.mail.aol.com > Are there any "functional" numerical packages (e.g. MatLab, > Mathematica, SAS, SYSTAT, or even Excel) that can directly import either > netCDF or HDF (and understand their headers) into their environment? Igor Pro 3.0 (which my company makes) can import HDF files. You can get information on it from www.wavemetrics.com. It is a Mac-only graphing, data anlaysis and automation program. Spyglass Transform from Fortner Research can also import HDF. I don't know about the other programs you mention. Howard Rodstein WaveMetrics From Doug.Ilg@gsfc.nasa.gov Thu Feb 22 14:18:47 1996 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!maui.cc.odu.edu!news.larc.nasa.gov!ames!newsfeed.gsfc.nasa.gov!usenet From: Doug Ilg Newsgroups: sci.data.formats Subject: Re: HDF -- r/w slabs Date: Tue, 20 Feb 1996 16:53:11 -0500 Organization: NASA Goddard Space Flight Center -- Greenbelt, Maryland USA Lines: 30 Message-ID: <312A42C7.41C6@gsfc.nasa.gov> References: <4ga9mr$erc@controversy.admin.lsa.umich.edu> NNTP-Posting-Host: hops.stx.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailer: Mozilla 2.0b6a (X11; I; IRIX 5.3 IP22) Joel Parriott wrote: > > Question1: sfwdata(sds_id, start, stride, edges, zrho)...do the > start, stride and edges vectors refer to SDS array or to my input data > array (zrho)? I would hope that they refer to the data array, and that > things are written to the SDS contiguously. Does the same thing apply for > reading back from an SDS to a large array? It's the other way around. HDF assumes that your buffer contains contiguous data and uses the start/stride/edge vectors to scatter data around your SDS. On reading, it's the same thing in reverse. That is, HDF will gather data from widely scattered places in your SDS and assemble them into a contiguous buffer, which you must provide. > Question2: Does a pointer to the SDS write/read location advance so that > I can make multiple writes to an SDS...just changing the data array from > say zrho-->zpressure....or will I overwrite? Perhaps I would need to add > another dimension to my SDS for each data array I want to grab values from... There is no read/write pointer to an SDS. Each operation performs direct access to the SDS based on the values supplied in start/stride/edge. If you want to march through the SDS, adding data as you go, just increment your start vector appropriately. ==================================================================== Doug Ilg Hughes STX Corporation EOSDIS Standard Data Formats 7701 Greenbelt Road, Suite 400 Doug.Ilg@gsfc.nasa.gov Greenbelt, MD 20770 USA Voice: (301) 441-4089 FAX: (301) 441-1853 ================= I do not speak for HSTX or NASA ================== From ROsborn@anl.gov Thu Feb 22 14:19:01 1996 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!news-server.ncren.net!concert!gatech!psuvax1!news.math.psu.edu!chi-news.cic.net!nntp.coast.net!news.kei.com!newsfeed.internetmci.com!vixen.cso.uiuc.edu!milo.mcs.anl.gov!msdmac4.msd.anl.gov!user From: ROsborn@anl.gov (Ray Osborn) Newsgroups: sci.data.formats Subject: Re: HDF, netCDF Date: Mon, 19 Feb 1996 11:21:41 -0600 Organization: Argonne National Laboratory Lines: 22 Message-ID: References: <4fucvh$mf0@controversy.admin.lsa.umich.edu> <3124CA20.28F1@anl.gov> NNTP-Posting-Host: msdmac4.msd.anl.gov X-Newsreader: Value-Added NewsWatcher 2.0b24.0+ In article , mehl@iastate.edu (Mark M Mehl) wrote: > >Are there any "functional" numerical packages (e.g. MatLab, >Mathematica, SAS, SYSTAT, or even Excel) that can directly import either >netCDF or HDF (and understand their headers) into their environment? > I'm away from my office at the moment so don't have access to a reliable list. However, I know that IDL and AVS have HDF and netCDF input procedures. In the case of IDL, they are in the process of being updated for HDF 4.0, though I don't know the current status. You could check the HDF and netCDF web page at and respectively, for other information. -- Ray Osborn E-mail: ROsborn@anl.gov Materials Science Division Tel: (708) 252-9011 Argonne National Laboratory Fax: (708) 252-7777 Argonne, IL 60439-4845 From sxu@ncsa.uiuc.edu Mon Feb 26 09:24:14 1996 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!news-server.ncren.net!concert!gatech!udel!news.mathworks.com!newsfeed.internetmci.com!vixen.cso.uiuc.edu!sxu From: sxu@ncsa.uiuc.edu (Shiming Xu) Newsgroups: sci.data.formats Subject: (Re-post) HDF Newsletter #19 Date: 23 Feb 1996 00:02:22 GMT Organization: Nat'l Ctr for Supercomp App (NCSA) @ University of Illinois Lines: 139 Distribution: world Message-ID: <4gj06e$enm@vixen.cso.uiuc.edu> NNTP-Posting-Host: void.ncsa.uiuc.edu Keywords: HDF4.0r1 Originator: sxu@void HDF Newsletter 19 Feb. 9, 1996 Contents . HDF4.0r1 release . HDF4.0 documentation . Progress in HDF related projects . Platforms to be dropped . New HDFers HDF4.0r1 release ================ We are very glad to announce that HDF4.0r1 is now officially released. HDF4.0 represents a substantial revision, expansion and improvement of the HDF library over HDF3.3. Its new features include: . Interfaces that support multi-file/multi-object access for raster images and annotations, similar to the multifile SDS interface. . Support for n-bit integer data within the SDS interface. . Support for RLE, adaptive Huffman and Gzip deflation compression for SDSs. . Uses v5 of Independent JPEG Group (IJG) JPEG file access library . Support low level function for RLE, adaptive Huffman and Gzip deflation compression for any object. . Support for creation and search for external elements. . Support for external vdata elements. . Unified auto-configuration that eases the installation and test of HDF package on UNIX platforms. . A new version of dimension record decreasing the size of HDF files which contain large 1-dimensional SDS's. . A new HDF utility hdp -- an HDF file dumper . Full support for SGI Power Challenge running IRIX6.1 . Support for parallel I/O on the Thinking Machines'CM-5. . Modifications in the implementation of memory management that decrease the required memory size and execution time. . Integration of Pablo performance measurement software. . I/O performance improvements . Limited support for reading CDF files. Details are listed in the $(toplevel)/release_notes/ABOUT_4.0r1. HDF4.0r1 has been tested on: SunOS 4.1.4, Solaris 2.4, Solaris 2.5, IRIX5.3, IRIX6.1 (both 32-bit and 64-bit modes), HPUX 9.03, UNICOS 803, CM5, AIX v3.2, Digital Unix v3.2, OpenVMS AXP v6.2, Solarisx86, Linux(a.out) v1.2.4, and Free BSD 2.0. More details are included in the $(toplevel)/INSTALL file. Many thanks to the HDF4.0 alpha and beta testers for their feedback, comments and suggestions. HDF4.0 documentation ==================== An HDF 4.0 User's Guide is available on the NCSA ftp server, ftp.ncsa.uiuc.edu, in subdirectory: /HDF/Documentation/HDF4.0/Users_Guide/. We are working on more documentation, including a new Getting started manual, a new HDF 4.0 Reference Manual, and a new HDF Specification and Developer's Guide. We will announce the availability of these documents as they are completed. Progress in HDF related projects ================================ BigHDF ------ In the last Newsletter we described the BigHDF project, in which we are building a prototype next-generation HDF. A proposed specification for the prototype is available on the web at http://hdf.ncsa.uiuc.edu/internal/NRA/BigHDFSpec.html There is also a reproduction of talk on BigHDF entitled "HDF: The Next Generation," given by Mike Folk and Quincey Koziol, at http://hdf.ncsa.uiuc.edu/nra/BigHDF/talk/coverpage.html We welcome your comments and ideas about BigHDF. HDF Support for Parallel Computing Environment ---------------------------------------------- With support from NASA's Advanced Information Systems Research project (AISR), we are designing new features to provide HDF library support for the parallel computing environment of Network of Workstations (HDF-NOW). Preliminary design has completed and we are in the stage of functional definitions. We plan to release a prototype version this year. Tentative platforms are the IBM SP2 and network of workstations like HP, SGI and Sun. More details are available at "http://hdf.ncsa.uiuc.edu/parallel_hdf.html". Platforms to be dropped ======================= Due to lack of resources and waning interest, we have decided to stop supporting HDF on the following platforms: DecStation/MIPSEL NeXT/Next-STEP Fujitsu VP/UXPM Intel i860 IRIX4.0 IRIX5.2 VAX/VMS If you need HDF support on these platforms, please contact us and we will try to help you port HDF 4.0 to them. New HDFers ========== Don Gibson, a University of Illinois M.B.A. student, has been working with the HDF group for several months. Don is helping to manage the concurrent HDF projects and assisting in the development of future projects. He is currently helping to write an HDF business plan. Barbara Jones recently joined us to help provide user-support to the HDF community. She comes to us with 16 years of programming and technical support experience. Barbara moved here from Providence, Rhode Island where she spent the last 6 years as a Technical Support Specialist at Cadre Tech.