Subject: Saving 100GB of data for 15 years cheaply

From: keith at msmri.med.ubc.ca
Date: 4 Aug 1995 15:33:05 GMT

Please post replies to sci.data.formats.

I need to save several hundred gigabytes of medical imaging data on a cheap 
reliable media.  We are currently saving the data to 10GB 8mm Exabyte tapes.
This is cheap but is only good for about 1 year.  Does anyone have any better
suggestions?

Thanks in advance.

Keith S Cover
Physics, UBC
Vancouver, Canada
keith at msmri.med.ubc.ca

---

From: cjones at ccserver.uoregon.edu (Christopher Michael Jones)
Date: 4 Aug 1995 19:23:05 GMT

keith at msmri.med.ubc.ca wrote:
: Please post replies to sci.data.formats.

: I need to save several hundred gigabytes of medical imaging data on a cheap 
: reliable media.  We are currently saving the data to 10GB 8mm Exabyte tapes.
: This is cheap but is only good for about 1 year.  Does anyone have any 
: better suggestions?

: Thanks in advance.

I'm not sure hom much getting a CD-ROM made would cost but that could be 
an option.  You could try distributing chunks of it over the internet for 
people to save for you :-).  Or you could wait about a while and put the 
data on 10 or 7 Gig DVD's.

---

From: tdp at ix.netcom.com (Tom Potter )
Date: 5 Aug 1995 14:33:34 GMT

In <mancini-0508951513050001 at usx121.maxlab.lu.se> mancini at maxlab.lu.se (Derrick C. Mancini) writes: 

>
>In article <3vts2p$4j0 at pith.uoregon.edu>, cjones at ccserver.uoregon.edu
>(Christopher Michael Jones) wrote:
>>an option. 
>CD-ROM blanks will be slow to write, but are cheap (about $20 each) and
>will hold about 700K.  They also have the advantage of being readable on
>a large number of machines, provided you put things into a readily accessible
>format.  You could also maintain an index CD.  Write times are slow compared
>to a hard drive, but should be better than the tape.  Backup should be good
>for 10 years, I think.  I would guess that price per gigabyte per year of
>storage would be competitive with tapes.  The CD-ROM writable drives can
>be had for as little as $2000.
>
>-- 
>Dr. Derrick C. Mancini
>MAXLab, Lund University         Advance Photon Source, ANL
>Box 118, Ole Romers Vag 1       9700 S. Cass Avenue - APS/431
>S-221 00, Lund, Sweden          Argonne, IL, USA  60439
>Tel: +46 46 222 3355            Tel: +1 708 252 0147
>FAX: +46 46 222 4710            FAX: +1 708 252 3222
>Email: mancini at maxlab.lu.se     Email: mancini at aps.anl.gov
>

CD's hold about 700 MEGABYTES of data, uncompressed.
They can hold at least two times this if the data is compressed ( Zipped ).

---

From: afl at cdc.noaa.gov (Andy Loughe)
Date: 4 Aug 1995 19:53:13 GMT

In article <3vtejh$mg at nntp.ucs.ubc.ca>, keith at msmri.med.ubc.ca writes:
|> Please post replies to sci.data.formats.
|> 
|> I need to save several hundred gigabytes of medical imaging data on a cheap 
|> reliable media.  We are currently saving the data to 10GB 8mm Exabyte tapes.
|> This is cheap but is only good for about 1 year.  Does anyone have any better
|> suggestions?
|> 
|> Thanks in advance.
|> 
|> Keith S Cover
|> Physics, UBC
|> Vancouver, Canada
|> keith at msmri.med.ubc.ca
|> 

What about a small optical drive with removable disks?

-- 

Andrew F. Loughe                        email: afl at cdc.noaa.gov
University of Colorado, CIRES           voice: (303) 492-0707
Campus Box 449                            fax: (303) 497-7013
Boulder, CO  80309-0449   USA

---

From: ralf at mail.ast.cam.ac.uk (Ralph Martin)
Date: 7 Aug 1995 11:24:58 GMT

The question of how to store digital information for long periods of
time is worth a hard look because there are already a number of
archives in astronomy which have had to ask the same question. Our archive
of telescope data was started in 1984 and originally used 1600 bpi nine track 
tape. The later computers had 6250 bpi nine track, then Exabyte and now DAT
tapes. By the beginning of last year we had some 7000 nine track tapes which
used about 60-70 square metres of floor space. More importantly we found
out that these kind of nine track tapes, which were mostly held in sealed 
canisters, were an immense fire risk. There are records of a test done
by the fire brigade at the Rutherford Laboratory. A few hundred of
these tapes were put into a former bomb store and a lighted rag thrown in.
A few minutes later, the fire was going so fiercely that it bent the
door lintel which was made of reinforced concrete. The brigade doused the blaze
with water but the fire restarted because the canisters provided their
own oxygen and the cooled outer shell protected the still hot inner core.
It took the fire crew serious amounts of foam to extinguish
the blaze. We thus had to choose what medium we were going to transfer
all this past data onto and it is here that the various options come into
play. How often do you want to reaccess the data? If it rare then
Exabyte will do the job but if you want to do it lots of times, the
Exabyte drive wear the tape. (This has something to do with the angle
that the tape makes with the reading heads). We asked around what was our
best option was and we decided to copy onto DAT tapes rather than CD-Rom
because a) DAT decks are cheap. b) DAT tapes are cheap c) It didn't
need any extra software effort to our derchiving software. I'm prepared
to believe that the choice could be made differently but the issue boils
down to what is going to be around for the next 10-15 years and I'd
say that DAT, Exabyte (maybe) and CD-Rom all have lifetimes of that
order. I now have two cabinets full of DATs and a spare room of 65 square
metres with students in it rather than tapes. The problem will probably
return in 10-15 years by which time I'll have the 'gone fishing' sign
over my door. The DAT's are being accessed regularly and so far we haven't
had any problems. Since we now dearchive the equivalent of nearly a full
telescope of data every year, it's important for it to be right. 
The DAT's are also considerably quicker than Exabytes when it comes
to reading FITS formatted tapes (which are the de facto standard in
astronomy) because Exabytes are hopeless at reading tape marks. Tape marks
occur after every FITS file on a tape....
The correspondent who wrote about optical storage is also correct
as the 10,000 Schmidt plates in our plate library and 150,000 glass plates
in our plate archive (some of which are well over a century old)
illustrate.
-- 
Ralph Martin                            Internet:      ralf at ast.cam.ac.uk
Royal Greenwich Observatory             Telephone:     UK+ 01223-374000
Cambridge CB3 0EZ                       ISDN:          UK+ 01223-328028
United Kingdom                          ISDN:          UK+ 01223-328087

---

From: rjw at crl.com (Ron Wickersham)
Date: 5 Aug 1995 16:30:09 -0700

Please elaborate on the 8mm Exabyte storage lifetime.  I thought that tape
would be good for the fifteen year requirement stated.  Do others report 
the Exabyte system only reliable for 1 year?

-ron wickersham
rjw at crl.com

keith at msmri.med.ubc.ca 
wrote: : Please post replies to sci.data.formats.

: I need to save several hundred gigabytes of medical imaging data on a cheap 
: reliable media.  We are currently saving the data to 10GB 8mm Exabyte tapes.
: This is cheap but is only good for about 1 year.  Does anyone have any better
: suggestions?

: Thanks in advance.

: Keith S Cover
: Physics, UBC
: Vancouver, Canada
: keith at msmri.med.ubc.ca

---

From: susanc at ssd3450.SanDiegoCA.ATTGIS.COM (susan cassidy)
Date: Mon, 7 Aug 1995 17:13:03 GMT

I would be reluctant to count on any form of magnetic tape lasting for 15
years.  When I visited the U.S. Geological Survey office where they 
process the data from space probes, they told us they had problems with
tapes distintegrating that had data that had not yet been processed.
Also, the reliability of tape is very dependent on the environmental
conditions.

One other factor is that anything like 8mm Exabyte tapes may not be
around (the drives/software to read them) 15 years from now.  They
did not exist 15 years ago.  I think the CD/ROM idea is probably
a better approach for now.

-- 
Susan Cassidy
email: susan.cassidy at sandiegoca.attgis.com

---

From: mleese at hudson.CS.unb.ca (Martin Leese - OMG)
Date: 8 Aug 1995 13:34:11 GMT

On 8 Aug 1995 01:32:22 -0400 Scott Fearing (sfearing at gwis2.circ.gwu.edu) wrote:
>> 	I should warn you of a potential danger of CD storage: CD 'rot'.
>> Apparently, the aluminum can oxidize under the plastic protective covering 
>> and data can be lost.  This is appearing on CDs as well as laserdiscs, 
>> but I assume it does not happen to the special gold plated discs that come 
>> out on the audiophile pressings.  There are discussions of this from time 
>> to time on rec.video.laserdisc.

Laserdiscs are made in two halves and then glued together.  There are
problems with the glue eating the aluminum layer - disc rot - but the 
glue was changed many years ago.  It takes years for the Laserdisc to 
rot, so we are currently reaping the "benefits" of a past mistake.

Audio CDs and CD-ROMs are not made in two halves so there is no glue.
There was a problem many years ago with acid from the cheap cardboard
sleeves of promo CDs attacking the aluminium. There was also a problem
many years ago with the ink used to print onto the CD.  Both of these 
problems are long gone.

Regards,
Martin
E-mail: mleese at atlantic.cs.unb.ca
WWW:    http://www.omg.unb.ca/~mleese/
________________________________________________________________________
 Want to know how Ambisonics can improve the sound of your LPs and CDs?
  See the Ambisonic Surround Sound FAQ.  Details on my WWW Home page.

---

From: bweiner at electron.rutgers.edu (Ben Weiner)
Date: 8 Aug 1995 10:56:29 -0400

>to time on rec.video.laserdisc.

Uh.  This sounds like an urban legend.  Or one of those audiophile stories.
If aluminum oxidizes, it forms, natch, aluminum oxide, which is clear.
In fact first-surface aluminized mirrors always form a thin layer of
clear aluminum oxide on the top, which is not bad, but good - aluminum
oxide is less reactive than raw aluminum, so it prevents tarnish.

This is not to in any way endorse claims that CDs are permanent.

But I believe that in 15 years your CDs will probably still be good.
Also, I believe that in 15 years you will be able to find a CD(ROM)
player.  I can barely read nine-track tapes now, and in a couple of
years I expect it would be a real pain in the ass finding a nine-track
drive.  Some of the tapes will be less than 10 years old then, yet
unreadable.  (Fortunately, there's nothing important on them.)  I
expect the situation to be better with Exabytes and even better with
DATs.  But still ...

-- 
History / has a stutter / it says / w-w-w-w-watch out!           - the mekons

---

From: grunes at news.nrl.navy.mil (Mitchell R Grunes)
Date: Tue, 8 Aug 1995 17:42:05 GMT

1.  Why not leave put the data on those 1 year lifetime (but high
capacity) tapes, but re-copy it every 5 or 6 months?  Since it would only 
take a few tapes, and you could copy one overnight, it really wouldn't 
take much of your time.  If you are paranoid, add an extra layer of 
error detection and/or correction coding, keep the last version's tape 
around in case an error happens, and I would think you could make it work.

This isn't based on extended experience--it just seems reasonable.

Re-copying needs to be done even with 9 tracks, though less often.  With
any media.  

2.  About your old 9 tracks.  How could they burn in sealed containers?
Surely there is a simple solution to this problem?  Inert gas?

3.  Since no one liked my idea of film, I've an even better idea.  
Print it all out in hex, on acid free paper.  When you need it, use a 
scanner.  Let's see: two characters/byte, about 100 characters/line 
and 60 lines/page...now that's a REAL fire hazard.  Compact, too.
                             (-:

4.  I've tried to think up a solution to a somewhat similar problem, and 
wasn't real happy with any of the solutions.  My personal solution mostly 
followed #1 above, because the tape drives were already around, whereas we 
would have to buy a CD recorder, and it would have taken ALOT of CDs--a lot 
of time and energy to deal with.

5.  For really long term storage, one would have to buy extra tape
drives, and computers to put them in, and a few decades of spare
parts.  I really wish libraries would keep a few old computers around,
of every conceivable common design, to make sure people could read
old media.  Would you believe that most scientific labs would now have
trouble reading 9 tracks?  It's not very sexy, but the need to read
old media (in spite of the errors) is going to be a very real problem.
What would you do if someone handed you an old TRS-80 audio tape with 
data on which the fate of the world (or at least your personal future) 
depended?
-------------------------------------------------------------------------
  Opinions are mine alone.
  I am not a "Real Programmer".  I only program in the Real World.
  Mitchell R Grunes (grunes at nrlvax.nrl.navy.mil)
  AlliedSignal Technical Services Corp.

---

From: sla at umbra.ucolick.org (Steve Allen)
Date: 8 Aug 1995 18:23:06 GMT

Note that I have restricted this to sci.astro.fits only, as it
is not relevant to any of the other original newsgroups.

In article <40778p$rd9 at noao.edu>, Rob Seaman  <seaman at noao.edu> wrote:
>Another way to address this problem is to write larger files onto the
>tapes - and thus fewer tape marks.  FITS image extensions (and other
>FITS extensions) allow a good way to do this by concatenating multiple
>images into a single large file.

But in the instance of non-IMAGE extensions how would it be possible
to determine the relationships between the various different kinds of
HDUs?  I suspect that this would be an excellent application for
the <A HREF="http://acadia.gsfc.nasa.gov/convert/group.html">
FITS Grouping Convention </A> which was proposed last year.

>any single image in the archive is accessible in under 10 minutes.  Not
>exactly what I'd call "online", but quite acceptable for our purposes.
>In addition, multiple retrievals are much more likely than single image
>retrievals due to the calibration data, multiple filters and exposures,
>and the initial seek time is shared between the entire list of images
>retrieved.

Again, the proposed FITS grouping extension seems to contain the necessary
structure to document the relationships between all such images stored
together.

I'm putting together specs for the Keck DEIMOS data storage, and as
a part of the data that comes out of that we think we will be generating
FITS tables that describe the slitlet configurations.  We're planning on
embedding the FITS grouping convention keywords (which are simple to
add and unobtrusive to a FITS reader that does not care about them).
--  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --
Steve Allen          UCO/Lick Observatory       Santa Cruz, CA 95064
sla at ucolick.org      Voice: +1 408 459 3046     FAX: +1 408 454 9863
Notice:  The ucolick.org domain has just changed from being lick.ucsc.edu

---

From: mancini at maxlab.lu.se (Derrick C. Mancini)
Date: Sat, 5 Aug 1995 14:13:05 GMT

---

From: grunes at news.nrl.navy.mil (Mitchell R Grunes)
Date: 9 Aug 1995 13:07:45 GMT

In article <409gdi$2qrq at yuma.ACNS.ColoState.EDU> dean at phobos.cira.colostate.edu writes:
>  CD Recorders would be my vote. I am currently using a Pinnicle Recorder
>to make 650Mb CD-ROMs...
>
>  The CD that I am making with the Pinnicle do have problems. Some CD-ROMs
> will not read them. It is not a problem for us, because all the CD we
> make work on our CD-ROMs on our Window NT system. However, there have been
> some people reporting that their DOS machines with a CD-ROM wil not see the
> disk in the drive.

Could other people give their experiences on this, with the Pinnicle
and with other drives?  Does anyone know whether it is an issue of:

(1) Format.  Windows NT file systems may be unreadable to DOS
    boxes.  In fact, Some new DOS file systems will be unreadable to
    old DOS's, and any compressed file system (e.g., Stacker) will be 
    unreadable if you aren't using the same compression system.

(2) Reliability.  I suppose it is possible that some/all CD writers
    (or some/all brands of blank disks) do a borderline job, producing 
    disks that are just barely readable by some drives, not readable by 
    others.  Kind of like writing 360 KB floppies on 1.2 MB drives.

If the latter is true, it would make a big difference to those
of us who might otherwise consider buying a CD-ROM writer.
-------------------------------------------------------------------------
  Opinions are mine alone.
  I am not a "Real Programmer".  I only program in the Real World.
  Mitchell R Grunes (grunes at nrlvax.nrl.navy.mil)
  AlliedSignal Technical Services Corp.

---

From: mcdonald at aries.scs.uiuc.edu (Doug McDonald)
Date: Wed, 9 Aug 1995 13:39:37 GMT

In article <mancini-0908951129210001 at usx121.maxlab.lu.se> mancini at maxlab.lu.se (Derrick C. Mancini) writes:

>>and data can be lost.

Actually, the data can be recovered from a CD anyway. The overcoat can be 
stripped and the metal completely removed with acid. Then recoat.
Or the uncoated disk can be played like a record, mechanically, with
a simple low-resolution AFM of the "beetle" type. In fact, uncoated CDs
are used as tests for these gizmos.

Doug McDonald

---

From: jack at itsnet.com (Jack Hudler)
Date: Mon, 07 Aug 1995 06:12:23 GMT

>They can hold at least two times this if the data is compressed ( Zipped ).

If you find the right software (Young Minds comes to mind :-) ), and
if your not worried about losses in the image, you could write
the data in raw CD data... in this case a CD can hold 1.2 to 1.6 gig.
The final amount would depend on the amount of correction information
you supply with the image. A CD has a huge overhead for the correction
logic.

---

From: EdStrick at io.com (Edwin Strickland)
Date: 7 Aug 1995 09:06:22 GMT

In article <3vtejh$mg at nntp.ucs.ubc.ca>,    keith at msmri.med.ubc.ca says...
>I need to save several hundred gigabytes of medical imaging data on a 
cheap >reliable media.  We are currently saving the data to 10GB 8mm 
Exabyte tapes >This is cheap but is only good for about 1 year.  Does 
anyone have any better suggestions?

CD-ROM recorders mail order for $1,599 US dollars, and I see prices 
of "less than" $8. for recordable CDs (when ordered in large quantity --
hundreds?).  Recordable CD's are supposed to be as stable as factor-
pressed ones, or so I think I've been told.  This is "decades" of shelf
life.  I doubt you can do better, especially since it is a medium that
will be supported for a couple decades, at least.  When they have high-
density recordable CD's (several gig, instead of 640 meg) switch to that.

---------------------------------------------------------------------
Edwin L. Strickland III                       Image Processor/Analyst
Austin, Tx.                          Planetary, Geologist (esp. Mars)
Ed Strick  at  IO .com                       SF, Hard Fantasy, Classical
|                                         Music, Wild-Cat Species Fan
|                                                                   |
IO.com is Illuminati Online, of Steve Jackson Games << Shameless Plug
---------------------------------------------------------------------

---

From: pausch at electra.saaf.se (Paul Schlyter)
Date: 6 Aug 1995 17:59:19 +0200

>This is cheap but is only good for about 1 year.  Does anyone have any better

Since the data consists of images, why not try photographic plates or film?
That medium is known to be long-lasting and reliable.

-- 
----------------------------------------------------------------
Paul Schlyter,  Swedish Amateur Astronomer's Society (SAAF)
Nybrogatan 75 A,  S-114 40 Stockholm,  SWEDEN
e-mail:  pausch at saaf.se       paul.schlyter at ausys.se

---

From: tferro at merlin.la.asu.edu (Anthony J. Ferro)
Date: 9 Aug 1995 15:56:48 GMT

Just to pitch in a couple of items to this discussion:

  CD-R (recordable CD-ROM) is different than normal, manufactured CDs.
  The process is more photographic (the laser changes the reflectivity
  of the media), and isn't magnetic (MO).  With both CD and CD-R, the
  lifetimes aren't well established.  I think the claims are that CD-R
  might be around the 15-year timescale.  A well made normal CD should
  be much longer, 50-100 years.

  The problems someone was having with Pinnacle CD-Rs is mostly to do with
  differences in how people interpret the ISO 9660 standard, and which level
  they support.  If you stick to the level 1 (the wonderful 8.3 file names,
  etc), I've been able read disks on almost any computer.  When you go above
  that, you can have problems.  Some of these problems can be solved
  using other software, besides that shipped by Pinnacle.

---------
Anthony J. Ferro				aferro at as.arizona.edu
Steward Observatory, NICMOS Project		Ph:  (520) 621-8683
University of Arizona				FAX: (520) 621-1891	
Tucson, AZ 85721
---------

---

From: swalton at galileo.csun.edu (Stephen Walton)
Date: 7 Aug 1995 10:47:53 -0700

In article <3vts2p$4j0 at pith.uoregon.edu>,
>: Please post replies to sci.data.formats.

OK.

>: This is cheap but is only good for about 1 year.

Wow, I had no idea Exabytes were good for such a short time.

>I'm not sure hom much getting a CD-ROM made would cost but that could be 

CD-ROM recorders are down to about $2000, and the recordable media are
about $7.50 per disk.  These are write-once, and (of course) have the
advantage that any $1000 'multimedia' PC can read them.  This didn't
become a good option until recently, when 'multisession' recorders
made it possible to record data in multiple pieces.  Before that, at
least as far as I understand, you had to have a complete image of all
660 MB on a hard disk and record the CD in one pass.  We just bought a
Yamaha 4X CD-ROM recorder with the 'EasyPro for Windows' software but
haven't tried it out yet.
-- 
Stephen Walton, California State University, Northridge
"Be careful what you wish for;  you might get it."	swalton at csun.edu

---

From: grunes at news.nrl.navy.mil (Mitchell R Grunes)
Date: Mon, 7 Aug 1995 23:31:55 GMT

>This is cheap but is only good for about 1 year.  Does anyone have any better

I guess saving imagery on film sounds ridiculous, doesn't it?  Surely
no one would do such a thing?

(-:

(Some films have fairly long lives, it's cheap, and it doesn't require 
specialized and soon-outdated hardware to read.  It's even obvious.)

---

From: Rob Seaman <seaman at noao.edu>
Date: 8 Aug 1995 08:29:13 GMT

In article <404t6a$fme at lyra.csx.cam.ac.uk> ralf at mail.ast.cam.ac.uk,
Ralph Martin writes:

> astronomy) because Exabytes are hopeless at reading tape marks. Tape
> marks occur after every FITS file on a tape....

The NOAO archive (http://iraf.noao.edu/~seaman/archive/archive.html)
multiplexes images from a half dozen or so of our telescopes on
Kitt Peak into FITS image extension files about 50 Mbytes in size.
These are taped onto duplicate exabyte 8505s (non-compressed) over
the network onto a central archive machine.  The software would just
as easily support DATs or other formats, but exabytes remain cheaper
per Gbyte.

With a capacity of 4.12 Gbytes (a very conservative choice amounting to
1.5M FITS records), this file size results in about 80 files per tape.
Combined with the much smaller tape marks on the 8505 drives compared
with the original 8200 drives, this small number of files means that
any single image in the archive is accessible in under 10 minutes.  Not
exactly what I'd call "online", but quite acceptable for our purposes.
In addition, multiple retrievals are much more likely than single image
retrievals due to the calibration data, multiple filters and exposures,
and the initial seek time is shared between the entire list of images
retrieved.

After a pair of tapes is full, active taping is shifted to a second
pair of drives and the tapes are FITS checksum verified.  The checksums
also provide a way to verify the readability of the tapes at a later
date (see ftp://iraf.noao.edu/misc/checksum/checksum.ps).  One copy
of each pair is shipped downtown, the other stays on the mountain
providing some disaster protection.

The tapes are stored in climate controlled computer labs in Tucson and
on Kitt Peak.  Note that climate control is important for blank media,
too.  Our tape handling procedures allow several days or weeks of
preconditioning in the mountain computer room before the blank tapes
are mounted in the drives - this allows the drives and media to
equilibrate thermally.

We also store the header catalog and the header/tape cross-index as
separate data products from the data tapes themselves.  This has obvious
advantages, but from the standpoint of simple survival of the data, the
biggest advantage is that the data can be easily recast onto new media,
perhaps reordering or regrouping the data into new tape or disk FITS
files, without actually having to read through and interpret the data.
This isn't a minor point with Tbyte size archives.  Recasting the data
will be required periodically no matter what the current medium - it's
only a question of how often.

Rob Seaman
seaman at noao.edu
National Optical Astronomy Observatories, Tucson

---

From: sfearing at gwis2.circ.gwu.edu (Scott Fearing)
Date: 8 Aug 1995 01:32:22 -0400

-- 
		Scott

Scott Fearing       <sfearing at gwis2.circ.gwu.edu>
National Symphony Orchestra and George Washington University:   Washington,DC

---

From: holden at oddjob.uchicago.edu (Bradford Holden)
Date: Tue, 8 Aug 1995 00:54:37 GMT

>no one would do such a thing?

Oddly enough large numbers of astronomers do this.
Film however, has some disadvantages.  It doesn't have a great dynamic range
(though fairly good for some of them) its not machine readable (unless you 
build a machine to do it), it has a nonlinear response, and its hard to make
good quality copies cheaply.  

-- 
Bradford "PROS is evil" Holden 

gradual student at the University of Chicago

---

From: mleese at hudson.CS.unb.ca (Martin Leese - OMG)
Date: 8 Aug 1995 20:23:05 GMT

>> the Exabyte system only reliable for 1 year?

I used to help run an archive of seismic data using 300 GByte WORM
jukeboxes.  The data arrived on Exabytes and our experience was that
if an Exabyte is less that 1 year old you should be surprised if you
cannot read it.  If an Exabyte is more than 1 year old you should not 
be surprised if you cannot read.

Having said this, most Exabytes will read for many years, just don't
rely on it.

Regards,
Martin

---

From: Paul Carpenter <paul at pcserv.demon.co.uk>
Date: Tue, 08 Aug 95 22:40:19 GMT

grunes at news.nrl.navy.mil "Mitchell R Grunes" writes:
-1.  Why not leave put the data on those 1 year lifetime (but high
-capacity) tapes, but re-copy it every 5 or 6 months?  Since it would only 
-take a few tapes, and you could copy one overnight, it really wouldn't 
-take much of your time.....

-This isn't based on extended experience--it just seems reasonable.

AFAIR major banks, used to do this practise with their archives, to avoid tape 
ageing and stretch.. This was with old 9 track tapes and the like.

How many people create archives/backups of important data without a true 
verification pass, let alone a compatability test. Too often I hear of backups 
being no use because:-

	1/	Data was not checked after being written

	2/	Drive being used for storage was out of calibration so backup
		media only worked on that drive.

	3/	No non-compressed etc. copy of software to read backup was kept 
		in an even safer place(s), that was verified to work!.

-Re-copying needs to be done even with 9 tracks, though less often.  With
-any media.  

-2.  About your old 9 tracks.  How could they burn in sealed containers?
-Surely there is a simple solution to this problem?  Inert gas?

Tried filling an individual tape canister with inert gas, and ensuring that 
oxygen was not released from the plastics used :-)

-3....

In short whatever the archive/backup method used ensure it works and test that
archive/backup can actually be used at some point in the future. No system is
infallible, assess the archive/backups means/media/procedures/software/host to
suit your needs now and in N years time.

Keeping a system with spares for large archives may seem silly but in fact is
very practical as long as people are around who still know how to fix them or
run them.

=-------------------------------------------------
Paul Carpenter		| paul at pcserv.demon.co.uk
PC Services		| tel (44) 1734-463634
PO Box 2030, Reading	| fax (44) 1734-463423
RG4 8YW, UK		| 
Consulting for DSP, Image processing, Data Acquisition, System tuning & more..

---

From: mleese at hudson.CS.unb.ca (Martin Leese - OMG)
Date: 8 Aug 1995 20:16:59 GMT

On Tue, 8 Aug 1995 17:42:05 GMT Mitchell R Grunes (grunes at news.nrl.navy.mil) wrote:
...
>> Surely there is a simple solution to this problem?  Inert gas?

Presumably the sealed container heated up, heating up the tape inside
until it ignited.  (Incidentially, this is how fire spreads on a ship,
even through airtight steel bulkheads.)  I cannot see how inert gas 
would help.  Once the fire was going, you would have sealed canisters 
popping open all over the place.

Regards,
Martin

---

From: adrianw at cassius (Adrian Whichello)
Date: 9 Aug 1995 02:12:51 GMT

>to time on rec.video.laserdisc.

yeah, but can't you paint all around the edge of the CD with a special green
felt pen...

Adrian.

---

From: dean at phobos.cira.colostate.edu
Date: 9 Aug 1995 05:17:38 GMT

CD Recorders would be my vote. I am currently using a Pinnicle Recorder
to make 650Mb CD-ROMs. It take about 30 min to make, but some time is
 involved with preparing the directory tree. We use 3M blank disk which are
 about $11. However, I hear TDK blank disk are better and they cost about
 $7.50 a piece.

  Kelly Dean
  CSU/CIRA

---

From: mancini at maxlab.lu.se (Derrick C. Mancini)
Date: Wed, 9 Aug 1995 09:29:21 GMT

In article <406st6$8le at gwis2.circ.gwu.edu>, sfearing at gwis2.circ.gwu.edu
>to time on rec.video.laserdisc.

I believe, Scott, that your comments may be only applicable to pressed CDs.
The writable CDs are a MO technology, I think.  I am not sure what they
use for reflective coat.   In properly manufactured CDs, the aluminum
should last for 15 years.  It is the presence of water and salts that
cause the damage.  I know I have never seen this effect on anything in my
CD collection, which started when CD's started, about 12 years (and a few
hundred CDs) ago.  By the same token, poor manufacture and improper storage
of tape is far more likely to fail than aluminum in CDs.  In either case,
you would be advised to store in a cool, dry place.  And make duplicates.

Another advantage of CDs is that they are a random access storage device,
which most tapes are not, and should have better ability to recover good
files if a particular sector or track were to go bad.  Many tape systems
will lose the whole tail of a tape from the point a defect is found.

---

From: pettsj at visigoth.demon.co.uk (James Petts)
Date: Wed, 09 Aug 1995 06:59:46 GMT

In article <grunes.302.807903725 at news.nrl.navy.mil>,

> and 60 lines/page...now that's a REAL fire hazard.  Compact, too.

I came into this discussion late (i.e. this is the first message on
the subject I've seen), but what was the main objection to film.

If it is the stability of the film base, that can almost certainly be
addressed by storing the film in vacuum containers. If it is image
permanence that can be addressed by using metallic replacement in the
developed film, substituting gold for silver.

Both techniques have been used at Kodak Limited in England: when Domesday
Book was taken apart and rebound in 1985-1986 it was photographed and the
film treated as mentioned above. Accelerated ageing test show that the
film is expected to retain its image in essentially he same form for at
least 100 years. I can speak with the person who did it, and get more
information if required.

-- 
=== James Petts ===
In the world of CD-ROM, a horse will always be an herbivorous
quadruped, and there'll be endless examples to prove just that.
But that world will never tell you how it feels to ride a horse.
Or fall off it.

---

From: ralf at mail.ast.cam.ac.uk (Ralph Martin)
Date: 10 Aug 1995 10:42:24 GMT

I checked back in my records for systems that store terabytes of data. 
(1 terabyte =1000 Gigabytes = 1,000,000 Megabytes).
Optical Memory News, February 1991, Published Monthly by
Rothchild Consultants, ran an article about a device marketed by ICI
Imagedata which wrote to optical tape. In it they described a device made by a
firm called CREO which had the following capabilities.
1 Terabyte capacity per 12" reel.
Cost ' a few dollars per gigabyte'
Transfer Rate  up to 3 Megs/sec sustained.
Media Life 30 years in office environment
SCSI-2 interface
Search speed 68 sec max for full reel.
The last contact I had with them was some years ago but the address
of the suppliers was

ICI Imagedata
PO BOX 6 Shire Park
Bessemer Road
Welwyn Garden City
Hertfordshire AL7 1HD
Telephone 01707 323400
Fax 01707 337896.

I have no connection either business or private with the afore mentioned
firm.

---

From: Rob Seaman <seaman at noao.edu>
Date: 10 Aug 1995 04:37:53 GMT

In article <408a2a$h50 at darkstar.UCSC.EDU> sla at umbra.ucolick.org,

> FITS Grouping Convention </A> which was proposed last year.

Well, this applies to IMAGE extensions as well as other extensions.
One difficulty with archiving raw data is that not all members of a
given group are defined - typically not even acquired yet - when an image
is archived.  Archiving the group interrelationships as a separate FITS
object at some later time is attractive - although this will complement,
not replace, a catalog database.

The grouping convention relies on URLs (or URNs) to refer to group
members that are external to the current file.  It isn't obvious (to
me, anyway) how best to construct a URL that references a tape sitting
on a shelf.  Examples, anybody?

Note also that since the physical media will be replaced periodically,
and since a new medium will likely have different constraints on size,
the arrangement of the data into files and HDUs in those files is likely
to change.  It may be tricky to define groupings whose URLs (or whatever)
will remain valid through a recasting of the data.  This would be
preferable
to discarding and then regenerating the grouping extensions, however.

Rob Seaman
seaman at noao.edu

---

From: hybl at umbc.edu (Dr. Albert Hybl)
Date: 10 Aug 1995 16:39:33 -0400

Recently, a TV station reported that Dr. Uzasny Blazen, CEO of
The Black Hole Storage Devices Inc., announced that they are about
to start shipping a new device that promises to provide a near
infinite storage capacity.
  A reporter and a technician have been dispatched with a mobile
satellite dish to obtain an interview with him.  Their report
had not arrived in time for the 6 pm news.  Hopefully, it will
be available for the 11 pm news.
  In related news, an airline pilot reported seeing a building
collapse into a huge subterranean chasm.  The cavern is located
exactly where Dr. Blazen's factory should be.  The last report
broadcast from the station's traffic helicopter estimated that
the crater is expanding at about five feet per hour.

:-)
Albert

---

From: Art Munson <artmuns at tape.com>
Date: 10 Aug 1995 13:53:08 GMT

If anyone would be interested in our latest prices on TDK CDR blanks, 
just let me know and I'll e-mail them to you. I'm also posting a recent
fax from TDK regarding longevity. 

Thanks

Art

Cassette House

http://www.tape.com/ch

dean at phobos.cira.colostate.edu wrote:
> We use 3M blank disk which are
> about $11. However, I hear TDK blank disk are better and they cost > about
> $7.50 a piece.
>

              T-TEC TDK-TECHNICAL EXCELLENCE CORNER

Topics:  Mix Magazine Article      

Compiled by Tom Izumi & Henry Dobashi

Number 4 - July, 1995

In MIX magazine (July 1995), an article purportedly comparing
cyanine to phthalocyanine-based CD-R by Steven St. Croix 
contained injurious comments about cyanine. Here is a summery
of our reactions.

What Mr. St. Croix says:  He speaks ill of many things about
cyanine-based CD-R, but most of his comments center on the liefspan
of cyanine-based CD-Rs.  He leads readers to believe that cyanine
CD-Rs are veritable time bombs, designed to self-destruct in as
little as five years, one year or even over the weekend.  The most
puzzling aspect of the article is that his conclusions are based on
secret data -- research that he refuses to make public.

TDK'S Reaction:  We are afraid that the readers may have serious
doubts about the longevity of our media.  It is our responsibility
to rebut.  You will see a two-page open letter in MIX, October
issue, showing our assertions as follows.

Discussions in our open letter:  We believe the writer may have
based many of his conclusions on data describing the performance of
raw cyanine dye which TDK does not use.  We will publish data that
demonstrates a big improvement in light resistivity, when metal-
stabilized cyanine dye is used.

We will share our aging test data, too.  The results demonstrate
our CD-R lasts at least 70 years when stored at 30 degrees C (86
degrees F).

We will also present our test results comparing the light fastness
of cyanine and phthalocyanine discs.  Even after 40 Mlux*h exposure
under Xenon test lamp, every parameter stays within spec for both
disc types.  This test indicates that both types of discs will last
10 years at least when left beside the windows with their dye side
up.

According to the test results mentioned above, phthalocyanine is
slightly more resistant to light than our metal-stabilized cyanine
formulation.  That's why phthalocyanine discs are specified at 100
years; ours at 70 years.

So why use cyanine?  The answer is compatibility with CD recorders
and long-term writability.  A graph will be published showing block
error rate (BLER) vs. write power, which compares our CD-R with a
typical phthalocyanine disc.  Our disc maintains its low BLER over
a broader range of laser power than phthalocyanine media.

This is significant because factory-fresh CD recorders may generate
higher or lower power output than the nominal manufacturer spec. 
Also, the laser ages over time and operation temperature may
fluctuate.  These factors may cause recording power to fluctuate.

Compatibility and long-term writability are the strongest and most
rational arguments for use of cyanine dye.  These arguments are
made very strongly in our open letter to MIX.

---

From: stgprao at sugarland.unocal.COM (Richard Ottolini)
Date: 14 Aug 1995 16:48:03 GMT

There was a nice review of the issue of digital archives in Scientific American
about a year ago.  Cost is not so much the issue as is recoverable formats
in a rapidly change technology.  If things aren't copied into the new formats
every few years, they could be lost.  Much like if a classical work wasn't
recopied by a medieval monk or Arab classicist, most books distintigrated
or were lost.  Much of early NASA planetary data is "lost" in the old formats.
And this doesn't even address the issue of physical media lifetimes.

---

From: atwoodj at ada.CS.ORST.EDU (John Atwood)
Date: 14 Aug 1995 18:35:08 GMT

I'm surprised no one has mentioned Optical Tape.  The August issue of
IEEE Spectrum shows it as a medium with good characteristics.  The 
article says Sony, Philips, Storage Technonlogy, E-Systems EMAS, and Kodak
are "actively investigating optical tape".  One graph shows the capacity of
optical tape to be, in 1993, 27GB, in 1995, 50GB, in 1999, 100GB.

John
-- 
_________________________________________________________________
Office phone: 503-737-5583 (Batcheller 349);home: 503-757-8772
Office mail:  303 Dearborn Hall, OSU, Corvallis, OR  97331
_________________________________________________________________

---

From: Guy Rixon <gtr at ast.cam.ac.uk>
Date: 18 Aug 1995 15:07:56 GMT

>the subject I've seen), but what was the main objection to film.
[Suggestions for safe storage of film deleted]

The point about film is that you can't do all the analysis of the images you
want to without re-digitizing the data and that's an inherently lossy process.
We're talking about science data here, of course.  If the pictures were held
for their aesthetic value, or if they were, say, medical images that could be
analyzed by looking at them on a light-box, then film would be ideal.