Subject: Re: portable binary format? [CUJ reference needed]

From: [email-masked] ()
Date: 6 Feb 1995 17:28:09 GMT

In article <[email-masked]>, [email-masked] (Mumit Khan) writes:
|> I've been looking into writing a portable I/O interface (callable from
|> C/C++ and FORTRAN) that's based on IEEE floating pt format. XDR looks 
|> promising, but I don't know how readily I can get XDR routines for 
|> non-UNIX OS's (and something we can redistribute for free). There was a 
|> posting that mentioned some C User's Journal article dedicated to the 
|> topic ... anyone have a ref? Are there established pckgs that already 
|> solved this?

Yes, there are several established packages out there that have solved your
problem to various degrees, in different ways.  I'd suggest taking a look at
the FAQ for sci.data.formats:
 ftp://rtfm.mit.edu/pub/usenet/news.answers/sci-data-formats
I'd pay particular attention to CDF, netCDF, and HDF (my personal favorite).
They seem to cover the widest range of data on the widest range of platforms.

		-Doug Ilg

---

From: [email-masked] (Mumit Khan)
Date: 9 Feb 1995 16:35:18 GMT

I had posted the following query asking for info/suggestion on a portable
binary format that uses IEEE standard floating point format.

In article <[email-masked]>, I <[email-masked]> wrote:

    I've been looking into writing a portable I/O interface (callable from
    C/C++ and FORTRAN) that's based on IEEE floating pt format. XDR looks 
    promising, but I don't know how readily I can get XDR routines for 
    non-UNIX OS's (and something we can redistribute for free). There was a 
    posting that mentioned some C User's Journal article dedicated to the 
    topic ... anyone have a ref? Are there established pckgs that already 
    solved this?
    
    thanks
    mumit -- [email-masked]
    http://www.xraylith.wisc.edu/~khan/

All the responses I've received so far have pointed out the following
packages:
    NCSA HDF,
    UCAR netCDF, and 
    UCAR CDF

These packages have been designed specifically for this type of task.
Carlie Coats, Jr <[email-masked]> suggests writing a domain-specific
wrapper/interface on top of either UCAR netCDF or NCS HDF, which would 
be very much what my application needs.

I've built both HDF and netCDF since my posting and found them very
powerful and that prompts the next 2 questions:
    
    - What're the (dis)advantages of netCDF over HDF? Why would you pick
      one over the other?

    - What kind of runtime overhead can I expect to incur (over straight
      binary reads and writes using fread/fwrite, for example)?

I'd like to thank the following people for their help:
    
    Carlie Coats, Jr		<[email-masked]>
    Steve Emmerson		<[email-masked]>
    Doug Ilg			<[email-masked]>

---

From: [email-masked] (Albert Cheng)
Date: 9 Feb 1995 20:39:51 GMT

In article <[email-masked]>, [email-masked] (Mumit Khan) writes:
|> I've built both HDF and netCDF since my posting and found them very
|> powerful and that prompts the next 2 questions:
|>     
|>     - What're the (dis)advantages of netCDF over HDF? Why would you pick
|>       one over the other?

They each has their strength and drawbacks.  But HDF also supports a
netCDF interface--it can read and write netCDF files (but not to create
a brand new netCDF file.)  So, you have the best of both worlds.  :-)

|> 
|>     - What kind of runtime overhead can I expect to incur (over straight
|>       binary reads and writes using fread/fwrite, for example)?

We are doing a study on the overhead cost.  There *maybe* a report
released later but no promise since I am not the one doing it.

====
Albert Cheng
Research Programmer
HDF Team

---

From: [email-masked] (Edward Hartnett)
Date: 09 Feb 1995 20:40:32 GMT

>>>>> "M" == Mumit Khan <[email-masked]> writes:

    M> I've built both HDF and netCDF since my posting and found them very
    M> powerful and that prompts the next 2 questions:
    
    M>     - What're the (dis)advantages of netCDF over HDF? Why would you pick
    M>       one over the other?

HDF has been chosen by NASA's EOSDIS project, which is *BIG*
project. I expect that within the next few years it will increase in
popularity.

Since it's latest release HDF includes all the features of netCDF
anyway.

    M>     - What kind of runtime overhead can I expect to incur (over straight
    M>       binary reads and writes using fread/fwrite, for example)?

Pretty negligible. Biggest inventment is in programmer time. I've been
slamming my head against HDF for about a month now. Using any of these
formats is more complicated then working with binary data files.

--