Subject: Re: Saving 100GB of data for 15 years cheaply

From: Clive Page <[email-masked]>
Date: 7 Aug 1995 10:26:05 +0100

The problem with saving data for ~15 years is not the reliability of the
medium [note for younger readers: "medium" is the almost obsolete
singular of "media"] but the pace of technological change.  Fifteen years
ago we had only just given up using paper tape and punched cards, and our
data were mostly stored on 9-track tapes at 800 bits/inch.  We can't read
them now, not because the tapes aren't legible, but because the decks
become obsolete several years ago.

My guess is that the only format that will last another 15 years is the
CD-ROM, mainly because it is based on a consumer item, and these formats
last longer than computer-only formats (c.f. the LP record).  Recordable
CDs are reliable and fairly cheap (around US$10/GB) and manufacturers
claim that if stored out of sunlight they are stable for at least 20
years.  The snag is the capacity, only 0.65 GB each, slowish speed, and
the fact that current CD recorders need a stead stream of data which is
hard to guarantee on multi-user systems.  But if longevity is the most
important criterion, then these are tolerable.  It's what we use for our
astronomical data.

-- 
-------------------------------------------------------------------------
Clive Page,                         Internet: [email-masked]
Dept of Physics & Astronomy,        
University of Leicester.

---

From: [email-masked] (Edward Hartnett)
Date: 07 Aug 1995 14:23:28 GMT

>>>>> "Clive" == Clive Page <[email-masked]> writes:

    Clive> The problem with saving data for ~15 years is not the reliability of the
    Clive> medium [note for younger readers: "medium" is the almost obsolete
    Clive> singular of "media"] but the pace of technological change.  Fifteen years
    Clive> ago we had only just given up using paper tape and punched cards, and our
    Clive> data were mostly stored on 9-track tapes at 800 bits/inch.  We can't read
    Clive> them now, not because the tapes aren't legible, but because the decks
    Clive> become obsolete several years ago.

    Clive> My guess is that the only format that will last another 15 years is the
    Clive> CD-ROM, mainly because it is based on a consumer item, and these formats
    Clive> last longer than computer-only formats (c.f. the LP record).  Recordable
    Clive> CDs are reliable and fairly cheap (around US$10/GB) and manufacturers
    Clive> claim that if stored out of sunlight they are stable for at least 20
    Clive> years.  The snag is the capacity, only 0.65 GB each, slowish speed, and
    Clive> the fact that current CD recorders need a stead stream of data which is
    Clive> hard to guarantee on multi-user systems.  But if longevity is the most
    Clive> important criterion, then these are tolerable.  It's what we use for our
    Clive> astronomical data.

I see your point, but must disagree.

Firstly, there are paper tape readers out there still. As recently as
1990 I saw one still in use at an old job. I saw punch card readers in
use at my University until about that time also.

Secondly, if you have a large archive of data and no hardware to read
it, you have no one to blame but yourself. When exabyte announces that
they are going out of business, buy a few of the tape drives and stash
them with the data.

Thirdly, one really doesn't need to worry about it that much. If
exabyte goes away it will only be because it is replaced by something
much better. There will be a long period where both technologies exist
side by side, and the worst case scenario is that you transfer all the
data to the new media.

Finally, while paper tape readers are not very common, neither was
much important data written to paper tape. With all the exabyte data
out there I would be completely amazed if exabyte tape drives
disappeared in 15 years. There are many orders of magnitude more data
on exabytes than there were on paper tapes. Even in it's heyday there
were never that many paper tape readers out there, because there
simply weren't that many computers around back then.

PS - I've had a lot of exabyte tapes go bad on my, BTW, so always
write important data to at least three tapes.

--

---

From: [email-masked] (David Clunie)
Date: 4 Aug 1995 21:55:43 GMT

In article [email-masked], [email-masked] () writes:

>Please post replies to sci.data.formats.

You might try posting this to alt.image.medical and comp.protocols.dicom
if you want medical imaging related responses.

>I need to save several hundred gigabytes of medical imaging data on a cheap 
>reliable media.  We are currently saving the data to 10GB 8mm Exabyte tapes.
>This is cheap but is only good for about 1 year.

Why is it only good for a year ? Do you read these tapes a lot ? I
understood that Exabytes lasted much longer than that. Are they less
reliable than DAT's ? Have you looked into DLT ? I gather it has a very
large storage capacity.

I presume that WORM's, MOD's and even CD-R are not cost effect for your
application ?

BTW. What format are you archiving the images in ?

---
David A. Clunie ([email-masked])
Soon leaving sunny Riyadh, Saudi Arabia.

at home in the desert:   http://www.rahul.net/dclunie/html/
alt.image.medical FAQ:    ftp://ftp.rahul.net/pub/dclunie/medical-image-faq/
                         http://www.rahul.net/dclunie/medical-image-faq/html/
dicom3tools:              ftp://ftp.rahul.net/pub/dclunie/dicom3tools.tar.gz

---

From: Clive Page <[email-masked]>
Date: 9 Aug 1995 13:56:20 +0100

In article <[email-masked]>,

>side by side, and the worst case scenario is that you transfer all the

The difficulty of transferring data shouldn't be underestimated.  We had
a large collection here of data on ~3000 9-track tapes. When we moved
>from VAXes to Unix systems and were about to lose our last 9-track deck
we wondered what to do about the data.  Tests showed they could be copied
to Exabyte or DAT at a rate of only 10 to 15 old tapes a day, so it would
take at least one person working for a whole year to transfer them all. We
couldn't afford to get someone to do this. Fortunately most of the data
tapes were just copies of those kept elsewhere by ESA, so we junked them,
and copied the few unique ones.  If this hadn't been so, we'd have had a
real problem.

We now have a large and growing collection of Exabytes.  I expect these
to become obsolete in a few years (it already looks as if DAT and DLT are
superior replacements because each was designed from the outset as a
digital medium).  I'm not at all confident that the transfer from Exabyte
to whatever replaces it will be any easier than the last one (Exabytes
are pretty slow devices).  But I'd bet that the CD-ROM outlasts the
Exabyte in regular use, even though its capacity is lower, and despite
the use of (analogue) 8mm tapes in camcorders.

>write important data to at least three tapes.

I'll agree with you there.
-- 
-------------------------------------------------------------------------
Clive Page,                         Internet: [email-masked]
Dept of Physics & Astronomy,        
University of Leicester.

---

From: [email-masked] (Markish boyperson)
Date: 8 Aug 95 21:13:14 EST

In article <[email-masked]>, [email-masked] writes:
> This is cheap but is only good for about 1 year.  Does anyone have any better
> suggestions?
> 
> Thanks in advance.
> 
> Keith S Cover
> Physics, UBC
> Vancouver, Canada
> [email-masked]
> 

	I'm not exactly sure of the tape costs, but try dlt (digital
linear tape).  Its a 1/2" format, and I think it lasts a lot longer
than exabyte.  It has 1.25 megs/sec uncompressed throughput, and
depending on the drive has 10, 15, or 20 gigs per tape, that's 
uncompressed, double that for compression, as well as the throuput.
The drives are the dlt 2000, the dlt 2000xt, and the dlt 4000.
I think the prices are $3000, $4000, and $5500.  I think the cost
of media for dlt 2000 is less than $40 per tape, somewhere around
the same cost for the dlt 2000xt, and double that for the dlt
4000.  The manufacturer is quantum, even though I could have sworn 
seeying them under dec's name.  Good luck, tell me what you find
out, we're going to be buying the dlt 2000xt for daily backup in
our digital editing suite of compressed video.
	Mark Turney	[email-masked]

---

From: [email-masked] (Walter Harms)
Date: Thu, 10 Aug 1995 10:35:13 GMT

[email-masked] writes:

>CD-ROM's are difficult to deal with because of there small size (700MB).  That means you
>you are dealling with ~130 CD's per 100GB with is not as nice as dealing with 
>10  10GB chunk's on 8mm tapes.  The access time of tape is adequate for our 
>needs.  The 10GB DTL tapes someone suggested sound interesting.  There is
>a writeup in the March 15, 95 Byte which I have not had a chance 
>to look at. 

	that are the std cd-rom for micro-computers, you may like to heare
	that there are drive capabel of terabytes. they are used in a local
	ship-yard for storring blue-prints of ships, maybe is that a 
	direction to look.

	walter-- 
####################################################################
 "Life starts at '030, fun starts at '040, impotence starts at '86"
              keyboard not connected -- press F1 to continue
 
<><><><><><>alles nur gestohlen, alles nur geraubt ..<><><><><><><><>