Subject: Re: Why have a standard data format?

From: Archie Warnock <[email-masked]>
Date: Thu, 26 Dec 1996 16:04:48 -0500

Rob Seaman wrote:
> I think the Risks article points up two requirements for either a
> portable data interchange format or an archival format.  (One way to
> look at a data archive is as one-way data interchange with the future.)

Listen to the man - he speaks the truth :-)

> The first requirement is that the data format must be explicitly
> specified, not just defined via a software interface.  Software
> changes - data must be allowed to stay the same.  Wide adoption of
> a standard also benefits from separating those responsible for its
> implementation(s) from whatever group is ultimately responsible for
> its definition.

This is an important point.  Who out there seriously thought, 10 years
ago, that a Fortran-based software API would be obsolete?  And yet,
we're not too many years from reaching a point where young programmers
won't have a clue what to do with data if the only way to access it is
via a set of Fortran calls.  For a data format to be useful as an
archival format, it's critical that there be a byte-level definition
describing how the data is written.

> Secondly, a portable standard requires portable documentation.
> Certainly there isn't much logic in providing access to an alternative
> format, but requiring access to tools that read the original format
> in order to use the alternative.

Not only must the documentation be portable, it must be both locatable
and accessible.  Even plain ASCII documents won't be of any use if you
can't find them and get a copy.  Those of us in astronomy (and other
areas, I'm sure) are familiar with the dreaded "institutional reports"
that contain important information but don't exist anywhere anymore.  We
cannot afford for this to happen to valuable scientific data.

Gotta think long term...

-- 

Archie

-- Archie Warnock                           Internet:  [email-masked]
-- A/WWW Enterprises                        Phone/FAX: 301-854-2987
--	     http://www.clark.net/pub/warnock/warnock.html
--	   As a matter of fact, I _do_ speak for my employer.

---

From: [email-masked] (Rob Seaman)
Date: 26 Dec 1996 17:04:08 GMT

I posted the appended Risks Digest message in sci.astro.fits, and
a colleague suggested posting a similar pointer in this newsgroup.
The article can also be reached at the URL:

    http://catless.ncl.ac.uk/Risks/18.71.html#subj11

Rob Seaman
--

RISKS-LIST: Risks-Forum Digest  Monday 23 December 1996  Volume 18 : Issue 71

   FORUM ON RISKS TO THE PUBLIC IN COMPUTERS AND RELATED SYSTEMS (comp.risks)
   ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator

[...]

Date: Sat, 21 Dec 1996 06:29:39 -0500 (EST)
From: "Darrin B. Jewell" <[email-masked]>
Subject: Microsoft documents and Rosetta stones (Giersig, RISKS-18.70)

This past summer I was getting increasingly frustrated by users who were
sending me e-mail with Microsoft Word ".DOC" files attached to them via MIME
encodings.  I do not own a copy of Microsoft Word and did not want to
purchase one simply so that I could read documents people send to me.
Instead, I wanted the specification for Microsoft Word file format so that I
would have a Rosetta stone for the documents I wished to read.

I contacted Microsoft's customer support and was informed of the following:
 
 1. The specification for Microsoft Word file format is unpublished
    proprietary information.  However, I could download a free reader
    for Microsoft Word files which would run under one of Microsoft's
    operating systems.  Source code for the reader was not available.

 2. I could ask the sender of the message to send me the attachment encoded
    in Rich Text Format which is the official export format for Microsoft
    Word documents.  The specification for Rich Text Format was publically
    available from Microsoft.

Since I did not own a computer running one of Microsoft's operating systems,
I asked Microsoft for the specification for Rich Text Format files.  As a
computer programmer, this was also a more useful and interesting form of
Rosetta stone than a precompiled translating program.

I was then directed to the file GC0165.EXE on the Microsoft ftp site, which
I was able to download and unzip.  (Itself another decoding adventure.)
Included was the file GC0165.DOC, a Microsoft Word format file containing
the specification I desired.  The included README.TXT file contained the
following comment in plain ASCII:

  The GC0165.DOC file included in this compressed file is the Rich Text
  Format (RTF) Specification version 1.3.  The document is in Word 6.0 for
  Windows format. If you have neither Word 6.0 for Windows nor Word 2.0 for
  Windows with the 6.0-to-2.0 converter, you will need to call Microsoft
  Product Support Services at (206) 462-9673 to obtain a hard copy of the
  document.

At this point, I decided it was fastest to have my friend who owned
Microsoft Word print out the RTF specification for me.

Since this experience, I usually ask people who wish to send me Word
documents to send them in RTF format.  When I explain to people the RISKS
involved in using documents without open standards, I get comments about
being ridiculous and pedantic or perhaps a blink and a "So what?"  Even
though Microsoft Word supports an "official export format", it is clearly
not obvious to everyone why it should be used.

Darrin

	Reused without explicit authorization under blanket 
	permission granted for all Risks-Forum Digest materials.  
	The author(s), the RISKS moderator, and the ACM have no 
	connection with this reuse.

-- 
[email-masked], http://iraf.noao.edu/~seaman
NOAO, 950 N Cherry Ave, Tucson AZ 85719, 520-318-8248
PGP: 98 8D 8B 49 74 9A 41 88  3A 43 87 54 51 BF 30 4B