From acheng@fuga.ncsa.uiuc.edu Sun Apr 13 16:51:26 1997 Path: solitaire.cv.nrao.edu!newsgate.duke.edu!news.mathworks.com!howland.erols.net!vixen.cso.uiuc.edu!fuga.ncsa.uiuc.edu!acheng From: acheng@fuga.ncsa.uiuc.edu (Albert Cheng) Newsgroups: sci.data.formats,comp.parallel.mpi Subject: HDFNOW new beta release Date: 10 Apr 1997 18:29:30 GMT Organization: University of Illinois at Urbana Lines: 76 Message-ID: <5ijbia$j6a@vixen.cso.uiuc.edu> NNTP-Posting-Host: fuga.ncsa.uiuc.edu Xref: solitaire.cv.nrao.edu sci.data.formats:1871 comp.parallel.mpi:2623 We would like to announce a new beta release for the HDFNOW library. The following is the release notes. Send comments/questions to hdfparallel@ncsa.uiuc.edu -- Albert Cheng HDF Group NCSA-UIUC ========================================================================= About HDFNOW 0.2 March 31, 1997 HDFNOW is a library that supports HDF for a Network Of Workstations. The initial design supports the distributed Scientific Datasets for HDF files. The Distributed SD (DSD) interface is designed for those who are familiar with the SD interface and experienced with message-passing programs. The DSD interface is a layer on top of the SD interface and using MPI for interprocess communications. Changes since HDFNOW Beta release --------------------------------- o Change the internal data structure so that the interface can run more efficiently: - Instead of having an universal message format, define message formats to customize different requests sent by processes so that the size of messages sent by processes are more concise. - Instead of storing fid/sdsid in the internal table, store the indices for different HDF files, SDS's and chunks to reduce the search time for the targeted files or SDS's. o Simplify the DSD interface by removing the argument mhp_rank from DSDnametoindex, DSDselect, DSDreaddata, DSDwritedata, DSDendaccess, and DSDend. Programming task is easier and more robust. Read the man pages for the new syntax of the above functions. Software Requirements: ---------------------- - HDF distribution release 4.0r2 - MPICH 1.0.13 distribution. Platforms/OS supported: ----------------------- - HP-UX A.09.03 - IRIX 5.3 - SunOS 4.1.3 Steps to install the DSD interface: ----------------------------------- - obtain the DSD interface from ftp://hdf.ncsa.uiuc.edu/pub/HDFNOW/dhdf0.2.tar - tar xvf dhdf.tar - to compile the interface, follow the instructions in src/README - to test the interface, follow the instructions in examples/test/README - to run examples, follow the instructions in examples/README From johan@technologist.com Wed Apr 16 11:18:53 1997 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!aruba.odu.edu!reznor.larc.nasa.gov!lll-winken.llnl.gov!uwm.edu!newsfeeds.sol.net!nntp.uio.no!solace!mn6.swip.net!mn5.swip.net!news From: johan@technologist.com (Johan Levin) Newsgroups: rec.games.programmer,alt.comp.compression,sci.data.formats,comp.graphics.algorithms Subject: Q: LZW-compression and GIF Date: 13 Apr 1997 13:26:08 GMT Organization: - Lines: 94 Message-ID: <5iqmtg$hmg@mn5.swip.net> NNTP-Posting-Host: dialup149-2-12.swipnet.se Mime-Version: 1.0 Content-Type: Text/Plain; charset=US-ASCII NNTP-Posting-User: s-49839 X-Newsreader: WinVN 0.99.6 Xref: solitaire.cv.nrao.edu rec.games.programmer:127243 alt.comp.compression:3647 sci.data.formats:1873 comp.graphics.algorithms:46737 I'm trying to make a gif-decoder and need some help about the LZW compression algorithm. This is the definition of LZW that I've read: /********************************************************************* 2.1 LZW Compression The LZW compressor reads 8-bit bytes from a data source and outputs N-bit codes each of which identifies a previously defined string. The value of N starts at 9. Thus, codes 0 through 255 ($ff) correspond with the standard character set, while codes 256 ($100) through 511 ($1ff) correspond to a byte-byte pair or a code-byte pair in the code table. After code 511 is output, 10 bit codes are used. This is repeated until the maximum number of bits per code is reached. LZW4P can use a maximum of 12, 13, or 14 bits per code. The recommended value is 14. The LZW compressor builds a code table as it compresses data. The code table consists of previously encountered strings. The basic LZW compression algorithm is as follows: STRING = get first input byte while there is more input data {BYTE = get next input byte if STRING+BYTE is in code table STRING=STRING+BYTE else {output code for STRING add STRING+BYTE to code table STRING = BYTE } } output the code for STRING 2.2 LZW Expansion The LZW expansion routine reads the N-bit codes previously created by the LZW compressor and reconstructs the code table (as previously constructed by the compressor) as it is outputing 8-bit bytes. A code corresponds to a single byte (the first 256 codes from $00 through $ff), or a byte-byte pair in the code table, or a code-byte pair in the code table. In the later case, the code part of the code-byte pair refers to another defined code pair in the table. As each code is read in, it is located in the code table and the corresponding 8-bit bytes are output. This means that codes must be defined before they are needed for expansion. Unlike older dictionary based compression schemes, the code dictionary produced by the compressor routine does not have to be provided to the expansion routine. The basic LZW de-compression algorithm is as follows: OLDCODE = input first code output OLDCODE while there is more input data {NEWCODE = get next input code STRING = translation of NEWCODE output STRING BYTE = 1st byte of STRING add OLDCODE+BYTE to the code table OLDCODE = NEWCODE } *********************************************************************/ According to these algoriths the stream: 0xff 0xff 0xff 0xff 0xff ... would generate these (9-bit) codes 0xff 0x100 0x101 ... When the codes are decoded, this happends: OLDCODE=0xff (input) output 0xff (oldcode) NEWCODE=0x100 (input) Then I'm suppose to translate the code 0x100, but there are no entries in the codetable. What am I doing wrong??? (I know that the first two codes are predefined in gif and that the initial codesize isn't always 9, that isn't the problem.) I'd appreciate any help, TIA. -- Johan Levin Sweden ------------------------------------------ homepage: http://home2.swipnet.se/~w-21572 mail : johan@technologist.com From odhv001@pf.nhpxynaq.np.am Thu Apr 17 13:42:11 1997 Path: solitaire.cv.nrao.edu!newsgate.duke.edu!news.mathworks.com!news-peer.sprintlink.net!news.sprintlink.net!sprint!howland.erols.net!news2.digex.net!uunet!in2.uu.net!202.37.60.10!comp.vuw.ac.nz!news.hn.netlink.co.nz!auckland.ac.nz!gate!gl13.cs.auckland.ac.nz!user From: odhv001@pf.nhpxynaq.np.am (Bevyn Douglas Quiding) Newsgroups: rec.games.programmer,alt.comp.compression,sci.data.formats,comp.graphics.algorithms Subject: Re: Q: LZW-compression and GIF Date: Mon, 14 Apr 1997 18:44:53 +1200 Organization: University of Auckland Lines: 51 Message-ID: References: <5iqmtg$hmg@mn5.swip.net> Reply-To: bqui001@cs.auckland.ac.nz NNTP-Posting-Host: gate.cs.auckland.ac.nz X-Original-Host: gl13.cs.auckland.ac.nz X-Authenticated: Sentinel-NNTP v0.9 on gate.cs.auckland.ac.nz X-Newsreader: Yet Another NewsWatcher 2.0b27.2 Xref: solitaire.cv.nrao.edu rec.games.programmer:127405 alt.comp.compression:3651 sci.data.formats:1874 comp.graphics.algorithms:46794 In article <5iqmtg$hmg@mn5.swip.net>, johan@technologist.com (Johan Levin) wrote: > According to these algoriths the stream: > > 0xff 0xff 0xff 0xff 0xff ... > > would generate these (9-bit) codes > > 0xff 0x100 0x101 ... > > When the codes are decoded, this happends: > > OLDCODE=0xff (input) > output 0xff (oldcode) > NEWCODE=0x100 (input) > Then I'm suppose to translate the code 0x100, but there > are no entries in the codetable. What am I doing wrong??? > if you look at the problem you aren't doing anything wrong it's just a offshoot with the algo that you have to be aware of. when the decoder recieves a code which is equal to the next free slot in it's dictionary (as in this case) it isn't necessarily invalid. but could equal the last code recieved + the first character of the new code. the first character of the new code = the first character of the old code. (if the old code dosen't exist then it is an invalid code ie when the table has just been cleared ) so in this case the decoder should do something like this get 0xff output 0xff last_first_char = 0xff old_code = Oxff get 0x100 add_to_dictionary (old_code + last_fist_char) output 0xff 0xff last_first_char = 0xff old_code = Oxff 0xff get 0x101 add_to_dictionary (old_code + last_fist_char) output 0xff 0xff 0xff last_first_char = 0xff old_code = Oxff 0xff 0xff bevyn. -- email address has been rot 13'd. please no junk mail (own business blah blah yakkety shmack) From timon@clark.net Fri Apr 18 08:26:32 1997 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!uunet!in1.uu.net!206.229.87.25!news-peer.sprintlink.net!news.sprintlink.net!sprint!cpk-news-hub1.bbnplanet.com!news.bbnplanet.com!newsfeed.internetmci.com!news.wco.com!not-for-mail From: Timon Marmex Trzepacz Newsgroups: rec.games.programmer,alt.comp.compression,sci.data.formats,comp.graphics.algorithms Subject: Re: Q: LZW-compression and GIF Date: Wed, 16 Apr 1997 02:51:23 -0700 Organization: SoftEgg Entertainment Lines: 29 Message-ID: <3354A11B.66E1@clark.net> References: <5iqmtg$hmg@mn5.swip.net> Reply-To: *NOSPAM*timon@clark.net NNTP-Posting-Host: 206.155.34.173 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailer: Mozilla 3.0Gold (Win95; I) To: Johan Levin Xref: solitaire.cv.nrao.edu rec.games.programmer:127678 alt.comp.compression:3660 sci.data.formats:1877 comp.graphics.algorithms:46878 Johan Levin wrote: > According to these algoriths the stream: > > 0xff 0xff 0xff 0xff 0xff ... > > would generate these (9-bit) codes > > 0xff 0x100 0x101 ... > > When the codes are decoded, this happends: > > OLDCODE=0xff (input) > output 0xff (oldcode) > NEWCODE=0x100 (input) > Then I'm suppose to translate the code 0x100, but there > are no entries in the codetable. What am I doing wrong??? When you get an entry beyond the last entry in the current code table, that is your signal to add a new entry there with whatever the last actual character (not code) was and the last code. In this case, the last actual character and the last code are the same character. Then follow your algorithm normally. It's a wierd special case, I know, but it works ... -- Tim Trzepacz --