Subject: FITS Data Compression

From: "Dr. Immanuel Freedman" <immanuel at mother.com>
Date: 14 Nov 1995 22:07:13 GMT

Colleagues

Addressing the issues of FITS data compression:

1) I see data compression & data structures as within the accepted framework
   of approximation theory,

2) I prefer to think in terms of alternative representations - abstract data 
   types with efficiently-supported operations,

3) Separating the data into an approximated component + compressed residual
   preserves the original data exactly for future use (per Bill Press &
   Jim Tilton),

4) Several major NASA projects have already adopted compression methods:

   Cassini  - on-board data analysis & JPEG-ish image compression
   Galileo  - multiple dictionary textual substitution (non-image)
              Integer Cosine Transform approximation   (image)

   HST      - archival browse imaging via HCOMPRESS
   IHW      - archival Previous Pixel compression
   COBE     - packing, spherical harmonic analysis
            - Vector Quantization of star catalog (Pixel # for direction)
            - experimental packing, Modified Huffman Codes, Rice codes
   STS      - Block Adaptive Quantization.

   At least one project is already using k-d trees for range query & set
   operations within the astronomical data base and last year more than
   13 papers referred to some kind of compression (linear quadtree, k-d tree,
   pyramidal & wavelet transforms...),

5) Major standards (JPEG, MPEG, DICOM, NITFS, SDTS...) are NOT defined by 
   software.  An algorithmic definition of a minimal decoder is specified
   together with accuracy constraints.  Any implementation that is decodeable
   by the standard algorithm to the prescribe accuracy or higher is conformant.

Please feel free to email or post questions to this thread,

Dr. Immanuel Freedman
(916) 759-8673