Subject: Draft Proposal re Fixed-block Sequential Media.

From: dwells at fits.cx.nrao.edu (Don Wells)
Date: Tue, 1 Oct 1991 14:40:40 GMT

This draft proposal has been placed in the anonymous-FTP server on
fits.cx.nrao.edu [192.33.115.8] in directory FITS/doc:
rw-r--r--  1 dwells   vlb          2183 Oct  1 09:51 blocking90.txt
This is a short file, and so I am appending it to this announcement.

FITS is a bitstream data format which is media-independent (contrary
to the impression given by the Wells-Greisen-Harten Basic FITS paper
of 1981). FITS *logical* records have length 2880 bytes. Blocking is
regarded as being a media-dependent matter, not a part of the FITS
standard per se. For 9-tk tapes the FITS community has agreed to use
blocking factors between 1 and 10 (28800 bytes/block) inclusive for
FITS files.  Other types of media need to be separately negotiated.
During the past few years it has become clear that 9tk tapes are being
superceded by several types of cartridge media. This text proposes
rules for several of these media.

This draft has been under consideration by the FITS committees for
about two years, and has not yet been endorsed by them. I encourage
everyone interested in FITS to consider the implications carefully.
For example, what would be the increase in efficiency if DATs and/or
Exabytes were allowed to have blocking factor greater than 10, and do
we want that efficiency enough to change our software which already
supports 28800-byte blocks? 

Donald C. Wells             Associate Scientist        dwells at nrao.edu
National Radio Astronomy Observatory                   +1-804-296-0277
Edgemont Road                                     Fax= +1-804-296-0278
Charlottesville, Virginia 22903-2475 USA            78:31.1W, 38:02.2N 

=-=-=-=-=-=-=-=-=-=-=-=-=-=-= cut here =-=-=-=-=-=-=-=-=-=-=-=-=-=-=

                            Draft Proposal
                                 for
               Blocking of Fixed-block Sequential Media.
                                 and
                           bitstream Devices.

                        P.Grosbol and D.Wells
                            1990-August-10

                             Introduction

A number of high density storage media (e.g. optical disks and helical
scan tapes) is of significant interest of exchange of large volumes of
astronomical data in FITS format. Many controllers and devices for these
media can only access data in blocks of fixed length, typically 2**n bytes.
Thus, a special blocking agreement for FITS files is required for this
type of media.

                 Draft Proposal for fixed-block media.

For fixed-block sequential media, FITS files consisting of an integer
number of 2880 byte logical records should be regarded as a bitstream
written out with the fixed blocking size of the media with the last
block being padded out with zeros to the length of the fixed blocks.
Reading an incomplete FITS logical record should be regarded as an
end-of-file. This proposal applies to optical disks (accessed as a
sequential set of records), QIC format 1/4inch cartridge tapes
and Local Area Networks.

The fixed-block blocking proposal conforms to the general rules
for blocking of FITS files (Grosbol et al. 1988, Astron. Astrophys.
Suppl. 73, p359) using a blocking factor of 2**n/2880 for a media
with a fixed block size of 2**n bytes.

                   Draft Proposal for bitstream devices.

For bitstream devices, FITS files should be written with a blocking
factor of one i.e. with fixed blocks of 2880 bytes corresponding to
the logical record size. This proposal applies to FITS files written
to logical file systems.

                 Draft Proposal for variable-block media.

For variable block length sequencial media, FITS files may be written
with an interger blocking factor between 1 and 10 including as specified
in the blocking agreement for 1/2 inch 9 track tapes. This proposal applies
to FITS files written to DDS/DAT 4mm cartridge tapes and
8mm cartridge tape (Exabyte).

---

From: sla at fast.ucsc.edu (Steve Allen)
Date: 1 Oct 91 22:07:37 GMT

>supports 28800-byte blocks?

Richard Stover at Lick observatory has investigated the issues of blocksize
on Exabyte tapes.  Richard is the author and maintainer of the Lick
data acquisition system.  His observations are included below.
--------------------
Years ago I looked into Exabyte efficiency with respect to blocking.
One can query the Exabyte for the number of 1024-byte blocks remaining
on the tape.  Therefore, one can write some number of n-byte variable
length blocks and then ask the Exabyte how many 1024-byte blocks are left.
>From this one can easily deduce the tape efficiency.  My results were:
	2048 or 4096 byte block are 100% efficient.
	2880 byte blocks are 93% efficient.
	28880 byte blocks are 96.7% efficient.
Given these small efficiency differences I have chosen to write 2880-byte
records, although the software was designed to handle a blocking factor
up to 10.  One reason to write 28800-byte records instead of 2880-byte
records might be that the number of system calls is reduced by a factor
of 10.  But machines are so fast these days that this didn't seems to
me like an overriding concern.

Richard Stover
--------------
_______________________________________________________________________________
Steve Allen          |                                |   sla at helios.ucsc.edu
UCO/Lick Observatory |     This space for rent.       | If the UC were opining,
Santa Cruz, CA 95064 |                                | it wouldn't tell me.