Subject: Re: Standards for exchange of scientific data via WWW?

From: [email-masked] (Ilana Stern)
Date: 1 Apr 1994 00:36:27 GMT

In article <[email-masked]>, Jeff E Mandel <[email-masked]> writes:
 
> I have the following questions:
> 
> 1) Has anyone else discussed the possibility of distributing datasets via
> WWW?

There are two issues here:  the format of the dataset, and the method
of distribution.  We (NCAR) distribute some datasets by ftp, and we
have a quasi-WWW site (it's hypertext served by ftp, until we get a
real http server), so in a sense you could say we distribute datasets
via WWW.  But the data format is not MIMEish or anything -- it's mostly
just ascii (or occasionally packed binary) data.

See ftp://ncardata.ucar.edu/catalogs/.html/README.html for examples.

> 2) Are there any standards for dataset formats? I need something more
> efficient than tab-delimited text.

See the sci.data.formats FAQ (I've crossposted to that group) at
http://www.ncsa.uiuc.edu/SDG/Software/HDF/SciDataFormatsFAQ.html
I think HDF is understood by Mosaic.  Basically, packed binary data is
the way to go for most scientific datasets, because it's 1) efficient
and 2) not machine-dependent.

> 3) Is there a way to build a WAIS index of this, so that only the text is
> indexed, and not the data? Consider the file containing the 200 word 
> abstract of the paper "Creatine
> blood levels in 500 patients with chronic renal failure sampled at 10
> minute intervals for 3 years" followed by the gazillion values

Michael Schwartz at the University of Colorado has done some work
creating a Wais indexing scheme called Essence which does just this.  

-- 
/\     "This technique probably only makes the difference | dod#0009
  \_][     between serious injury and total disaster."    | [email-masked]
      \__<a href=ftp://ncardata.ucar.edu/catalogs/.html/me.html>Ilana Stern</a>