From flehmann@orion.oac.uci.edu Sat Jul 1 22:36:49 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!howland.reston.ans.net!usc!news.service.uci.edu!orion.oac.uci.edu!flehmann From: flehmann@orion.oac.uci.edu (Fritz Lehmann) Newsgroups: sci.data.formats,alt.snail-mail,comp.std.internat,comp.databases.theory,bit.listserv.edi-l Subject: Re: Data format integration, was addresses Date: 1 Jul 1995 19:24:47 GMT Organization: GRANDAI Software, Irvine Lines: 91 Message-ID: <3t47dv$9s1@news.service.uci.edu> References: <3slf2h$46@news.service.uci.edu> <804189712snz@intersec.demon.co.uk> <1995Jun30.220520.13611@rossinc.com> <804601391snz@intersec.demon.co.uk> NNTP-Posting-Host: orion.oac.uci.edu Xref: solitaire.cv.nrao.edu sci.data.formats:1022 alt.snail-mail:2322 comp.std.internat:3393 comp.databases.theory:4586 I've changed the name of this thread since the issue is universal for all data types and not just a matter of names and addresses. Bernard Peek wrote in comp.databases.theory: ---begin quote--- For the name of individuals I hold: Title Firstname SecondName(s) Prefix Lastname Succession There's other information I could, even should, hold. I don't send personalised letters so although I have a field for titles like "Mr.", "Ms." etc, I have never had to print it. I should also hold data on academic achievements and honorifics. I don't have a field for "Duke of xxxxxx" which I should technically include after the name for some people. ---end quote--- The key thing for data like this that will have long-term, varied uses is that this is but one of many legitimate ways of representing the same (semantic) information. Other format designers will do it differently. Even if the syntax is identical, the names and meanings of fields will not coincide -- this forever prevents _automated_ integration of databases, unless the semantics of the fields is represented directly in the computer. I've never seen "Succession" in a name field! I assume it means (about) what, in my address system, is (with crosslinks in brackets): GENERATION-DISTINGUISHER/NUMERATOR (Jr., Sr., III, XI, the Younger) [FDEF PLACEHOLDER] [EDEF A word or number, occurring immediately after a personal name, which indicates which person, among persons of the same name within a family, is intended, based on the relative age or temporal order of birth of the persons to be distinguished from one another.|FL] [USPS.Pub28.92 III C{Suffix Title - Maturity}|FL] [X12 1104:10{Generation}|FL] [USMARC-B.94 100/400/600/700/800:$b{Numeration}|FL] [BrackettB.94 Person Name Extension|FL] [WordMenu.92 see: DoFaKiKi312{generation}|FL] [Laffal.73 SUB YNG, LEAD PAST, NUMR|FL] [Roget.4 124.15;1{junior, Jr.}, 127.5;1{senior,Sr.,elder,older}|FL] [WordMenu.92 DoFaKiFa313{elder,junior,seniors}, DoFaPaNa321{junior,master}|FL] Notice all the different names for this: Generation-distinguisher, Maturity, Generation, Numeration, Person-Name-Extension, and now Succession. These are taken from a US Postal standard, an EDI (Electronic Data Interchange) standard for commercial transactions, the main library standard for data on publications, a well-known book on data definition, two thesauri and one "conceptual dictionary", plus Peek's version. There are two distinct issues: 1. What fields to use in a particular application, and what field names to use. This will vary, and properly so. 2. What are all the possible semantic building-blocks which are likely to be used in an address? When this is known, and all the categories are well-defined, then it is possible to _annotate_ any particular implementation with the semantic classes. So an application can note whether "Mr." is included in "name" (some do, some don't include it), whether the "von" is in a name, whether an Amerindian name like Lone Horse is to be one surname or two names, etc. The point here is not to dictate what to do in an application, but rather to establish the set -- the union -- of all the semantic classes that any application is likely to use. Then the syntax specification can specify what classes get mashed together in a field like "Address2". Then, if two applications' formats are properly annotated (their syntax is specified and the semantics are specified using the "universal" set of semantic classes), automatic translation from one to the other is possible, regardless of the field (and code) names used or the syntax differences. Exactly the same problem (and potential solution) exists in the area of EDI (Electronic Data Interchange) and Electronic Commerce. For EDI there is a current effort called the BSR (Basic Semantic Repository) which is taking the first steps toward the solution in ISO 11179. In EDI the "source" application happens to be in a different organization >from the "target" application, but the principle is the same. What the BSR is doing, and what certain groups like CYC, Pangloss, CCAT and others are doing within Artificial Intelligence, is building an "ontology" of commercially relevant concepts in the real world. That's what is needed for Semantic Data Integration. This is why I have recommended a very detailed breakdown of data like names, addresses, dates, accounts, etc. -- not to suggest that anyone will actually want to use all of the detailed categories as fields in a particular application. Yours truly, Fritz Lehmann From dilg@newsroom.hitc.com Wed Jul 5 22:55:19 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!swrinde!elroy.jpl.nasa.gov!ames!newsfeed.gsfc.nasa.gov!newsroom.hitc.com!dilg From: dilg@newsroom.hitc.com (Doug Ilg) Newsgroups: sci.data.formats Subject: Re: Using Tags to write Annotations in HDF with FORTRAN Date: 5 Jul 1995 17:30:05 GMT Organization: NASA Goddard Space Flight Center -- Greenbelt, Maryland USA Lines: 73 Distribution: world Message-ID: <3tei6t$q9v@post.gsfc.nasa.gov> References: <3tcj1d$pl7@dingo.cc.uq.oz.au> NNTP-Posting-Host: newsroom.gsfc.nasa.gov Neil, I've got a couple tips on annotations for you. I think your problem stems from some confusion over the purpose of the annotation functions. Here's a quick explanation: There are three major types of annotations provided by the HDF annotation interface (DFAN). o File annotations, attached to whole files o Object annotations, attached to individual data objects (tag/ref pairs) o Tag annotations, attached to a specific type of data object (rarely used) Under each of these, there are two subtypes. o Labels, used for naming things o Descriptions, used for extended textual descriptions of things In addition, each of the other interfaces can define any descriptive auxilliary data objects it needs. The SDS interface contains many of these interface-specific features, like labels, units, and formats for each dimension of a data array, as well as for the data array, itself. The routines you've shown below are of the object annotation type, which is definitely not appropriate for the file description, but *may* be okay for the SDS label you want to store. In article <3tcj1d$pl7@dingo.cc.uq.oz.au>, n.white@ctpm.uq.edu.au (Neil White) writes: |> |> |> I'm trying to write out a HDF consisting of 22 SDS where each is |> real rdata(2796,52,4) |> character fdesc*350, flabel*25 |> I have no real problem generating the HDF and adding the file |> desciption using |> |> iret=dapdesc(hdfnme,101,2,fdesc,350) Actually, this isn't the proper way to add a file description. The above call would produce an annotation *to* a file description, not an annotation that *is* a file description. The routine you should use is DAAFDS(), but you must first open the file with Hopen(). You'll find documentation for the routine on page 19 of the "HDF Reference Manual" for Version 3.3. |> But putting a label is doesn't work |> |> After writing the SDS I'ved used |> lastref=dslref() |> to get the ref of the last SDS |> |> iret=dsplab(hdfnme,704,lastref,flabel,25) |> |> doesn't create the tag refeference (ie it doesn't show when using the |> utility HDFLS |> |> Is this the right tag to be using inconjunction with dsplab ???? The only explanation I can think of for this behavior is the fact that you've chosen to annotate the tag 704, which is DFTAG_SDL (scientific data labels). The problem with that tag is that (I think) you're not guaranteed to have a unique DFTAG_SDL for each SDS. You'd be much better off annotating the DFTAG_SD (scientific data), which would be 702. (I'm assuming that you intend to put a label on an SDS, rather than the whole file, in which case you'd want to use DAAFID.) Now that I think about it, what you really want to do is probably write out the actual DFTAG_SDL (704) with the DSSDAST (set data strings) and DSSDIST (set dimension strings). Those functions appear on pages 81 and 84, respectively, in the Reference Manual. |> All help greatfully appreciated Finally, I'd like to encourage you to use the new SD interface to write your files (assuming you don't already). The SD interface has the capability (borrowed from netCDF) to store global and local attributes for SDSs. -Doug Ilg EOSDIS Project Hughes STX From ritter@earthlink.net Fri Jul 14 01:14:48 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!caen!spool.mu.edu!howland.reston.ans.net!usc!elroy.jpl.nasa.gov!mtritter.jpl.nasa.gov!user From: ritter@earthlink.net (Niles D. Ritter) Newsgroups: sci.data.formats,sci.image.processing,comp.std.misc Subject: Georeferenced TIFF Release 0.1 Date: Tue, 11 Jul 1995 17:37:45 -0700 Organization: Sometimes a Good Idea Lines: 54 Message-ID: Reply-To: ndr@tazboy.jpl.nasa.gov NNTP-Posting-Host: 128.149.24.55 Keywords: tiff,standard,georeference,geotiff,gis Xref: solitaire.cv.nrao.edu sci.data.formats:1045 sci.image.processing:15482 comp.std.misc:1045 This is an announcement of the Beta release of a new set of TIFF tag extensions for georeferencing raster data within the TIFF 6.0 interchange format. The extension is called GeoTIFF and is the result of a collaboration between over 100 remote sensing, surveying and cartographic raster data providers and users, including SPOT Image, ARC/INFO, ERDAS, MapInfo, ER Mapper, PCI, JPL Cartographic group, and the USGS. The extension uses tags registered to Aldus/Adobe, and is fully compliant with the TIFF 6.0 standard, using no private directories, binary structures or proprietary implementations. The GeoTIFF format is not intended to be a competitor with a full featured metadata standard such as as the Canadian SAIF, the USGS SDTS standard, or the FGDC metadata standard. It is merely an effort to encapsulate into the popular TIFF standard data elements for encoding the datums, ellipsoids, projections, tiepoints, etc, so that the raster data can be precisely tied down to the surface of the earth. The USGS is planning on releasing prototype versions of their 7.5 minute quadrangles on CD-ROM in their "DRG" (Digital Raster Graphic) format, which in turn will be based on the GeoTIFF spec. Other providers such as SPOT Image will also be releasing products in this format, and many of the companies listed above have committed to producing an import/export capability in support of GeoTIFF. GeoTIFF is currently at Revision 0.1 (Beta), and following several months of comments and critiques will be promoted to Revision 1.0 (Final), with further revisions and extensions on a yearly basis, if necessary. For more information, a WWW page in now enabled, and is located at: http://www-mipl.jpl.nasa.gov/~ndr/cartlab/geotiff/geotiff.html Which has pointers to hypertext versions of the specification. The spec and related source code is also available via FTP to ftp://mtritter.jpl.nasa.gov/pub/geotiff/ or its mirror site at: ftp://ftpmcmc.cr.usgs.gov/release/geotiff/ A mailing list has been created for discussing the development of this standard; to subscribe send email to geotiff-request@tazboy.jpl.nasa.gov with subject: none and the body of the message reading: subscribe geotiff your-name-here. The address of the mailing list itself is geotiff@tazboy.jpl.nasa.gov. --Niles Ritter (admin, geotiff mailing-list). From mskuhn@cip.informatik.uni-erlangen.de Tue Jul 18 00:35:59 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!news-server.ncren.net!taco.cc.ncsu.edu!lll-winken.llnl.gov!sol.ctr.columbia.edu!howland.reston.ans.net!xlink.net!nntp.gmd.de!nntp.darmstadt.gmd.de!news.th-darmstadt.de!fauern!rrze.uni-erlangen.de!cip.informatik.uni-erlangen.de!mskuhn From: mskuhn@cip.informatik.uni-erlangen.de (Markus Kuhn) Newsgroups: comp.std.misc,comp.protocols.time.ntp,comp.infosystems.www.authoring.misc,sci.data.formats Subject: International Standard Date Notation Date: 16 Jul 1995 10:09:12 GMT Organization: Student Pool, CSD., University of Erlangen Lines: 53 Distribution: inet Message-ID: <3uaog8$l3u@cd4680fs.rrze.uni-erlangen.de> Reply-To: mskuhn@cip.informatik.uni-erlangen.de NNTP-Posting-Host: faui04g.informatik.uni-erlangen.de Xref: solitaire.cv.nrao.edu comp.std.misc:1048 comp.protocols.time.ntp:762 comp.infosystems.www.authoring.misc:1602 sci.data.formats:1049 International Standard Date Notation ------------------------------------ For those of you wondering, what the professional and correct way of writing a calendar date in network publications is: The international standard date notation is: YYYY-MM-DD For example, February 4, 1995 is written as 1995-02-04. This notation is standardized in International Standard ISO 8601. For a detailed description of this standard, please download from ftp.uni-erlangen.de the file pub/doc/ISO/ISO8601.ps.Z. Other commonly used notations are e.g. 2/4/95, 4/2/95, 4.2.1995, 04-FEB-1995, 4-February-1995, and many more. Especially the first two examples are dangerous, because as both are used quite often and can not be distinguished, it is unclear whether 2/4/95 means 1995-04-02 or 1995-02-04. Advantages of the ISO standard date notation are: - easily parsable by software (no 'JAN', 'FEB', ... table necessary) - easily sortable with a trivial string compare - language independent - can not be confused with other popular date notations - consistent with 24-h time notation hh:mm:ss which comes also with the most significant component first and is consequently also easily sortable (e.g. write 1999-12-31 23:59:59). - short and has constant length (makes keyboard data entry easier) - identical to the Chinese date notation, so the largest cultural group (>25%) on this planet is already familiar with it. - 4-digit year representation avoids overflow problems after 1999-12-31. In shell scripts, use date "+%Y-%m-%d %H:%M:%S" in order to print the date and time in ISO format. In C, use the string "%Y-%m-%d %H:%M:%S" as the format specifier for strftime(). The beginning of the next century might be a good opportunity to change to a common worldwide date notation. Feel free to redistribute this text. Markus --- Markus Kuhn, Computer Science student -- University of Erlangen, Internet Mail: - Germany WWW Home: From dundas@salt.jpl.nasa.gov Tue Jul 18 00:36:05 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!maui.cc.odu.edu!kuroshio.ccpo.odu.edu!xanth.cs.odu.edu!lll-winken.llnl.gov!uwm.edu!vixen.cso.uiuc.edu!howland.reston.ans.net!usc!elroy.jpl.nasa.gov!netline-fddi.jpl.nasa.gov!salt.jpl.nasa.gov!dundas From: dundas@salt.jpl.nasa.gov (John A. Dundas III) Newsgroups: comp.std.misc,comp.protocols.time.ntp,comp.infosystems.www.authoring.misc,sci.data.formats Subject: Re: International Standard Date Notation Date: 17 Jul 1995 14:44:38 GMT Organization: Jet Propulsion Laboratory Lines: 42 Distribution: inet Message-ID: <3udt0m$s42@netline-fddi.jpl.nasa.gov> References: <3uaog8$l3u@cd4680fs.rrze.uni-erlangen.de> <1995Jul17.131006.27286@rsg1.er.usgs.gov> NNTP-Posting-Host: salt.jpl.nasa.gov Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Xref: solitaire.cv.nrao.edu comp.std.misc:1047 comp.protocols.time.ntp:761 comp.infosystems.www.authoring.misc:1596 sci.data.formats:1048 In article <1995Jul17.131006.27286@rsg1.er.usgs.gov>, peter@geochange.er.usgs.gov (Peter N. Schweitzer) writes: |> In article <3uaog8$l3u@cd4680fs.rrze.uni-erlangen.de>, |> mskuhn@cip.informatik.uni-erlangen.de (Markus Kuhn) writes: |> > |> > International Standard Date Notation |> > ------------------------------------ |> > |> > For those of you wondering, what the professional and correct way of |> > writing a calendar date in network publications is: |> > |> > The international standard date notation is: YYYY-MM-DD |> > |> > For example, February 4, 1995 is written as 1995-02-04. This notation |> > is standardized in International Standard ISO 8601. For a detailed |> > description of this standard, please download from ftp.uni-erlangen.de |> > the file pub/doc/ISO/ISO8601.ps.Z. |> |> (very informative post truncated here) |> |> > Markus Kuhn, Computer Science student -- University of Erlangen, |> > Internet Mail: - Germany |> > WWW Home: |> |> Unfortunately, the US Federal Information Processing Standards |> include a date format that is significantly different, i.e. |> YYYYMMDD (no hyphens). Does anyone know whether US is planning |> to revise the FIPS to match the ISO standard? (Peter's signature deleted...) Actually, I just happen to have a copy of the full ISO 8601 standard in front of me and there are two different complete representations for calendar date. Section 5.2.1.1 describes the "basic" format as CCYYMMDD, which corresponds to Peter's example, and the "extended" format as CCYY-MM-DD, which is what Markus noted. Note that the standard says that BOTH are acceptable. I say, both are easily machine readable and distinguishable. For time jockeys, this standard is an important one and should be studied carefully. I highly recommend obtaining a (legitimate) copy and reading it often. John Dundas From erik@naggum.no Tue Jul 18 00:36:18 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!portal.gmu.edu!europa.chnt.gtegsc.com!howland.reston.ans.net!Germany.EU.net!EU.net!dkuug!Norway.EU.net!nac.no!ifi.uio.no!nntp!enag From: erik@naggum.no (Erik Naggum) Newsgroups: comp.std.misc,comp.protocols.time.ntp,comp.infosystems.www.authoring.misc,sci.data.formats Subject: Re: International Standard Date Notation Date: 17 Jul 1995 14:53:40 GMT Organization: Naggum Software; +47 2295 0313 Lines: 17 Distribution: inet Message-ID: <19950717T145340Z@naggum.no> References: <3uaog8$l3u@cd4680fs.rrze.uni-erlangen.de> <1995Jul17.131006.27286@rsg1.er.usgs.gov> NNTP-Posting-Host: gyda.ifi.uio.no Xref: solitaire.cv.nrao.edu comp.std.misc:1049 comp.protocols.time.ntp:763 comp.infosystems.www.authoring.misc:1604 sci.data.formats:1050 [Peter N. Schweitzer] | Unfortunately, the US Federal Information Processing Standards | include a date format that is significantly different, i.e. | YYYYMMDD (no hyphens). Does anyone know whether US is planning | to revise the FIPS to match the ISO standard? the hyphens are optional in ISO 8601. the FIPS should be updated to include hyphens, as well. (actually, ISO 8601 is way too permissive.) my message-id has the compact form of ISO 8601, as in 19950717T144705Z, # -- NETSCAPISM /net-'sca-,pi-z*m/ n (1995): habitual diversion of the mind to purely imaginative activity or entertainment as an escape from the realization that the Internet was built by and for someone else. From jrs@dclf.npl.co.uk Fri Jul 21 00:21:55 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!portal.gmu.edu!europa.chnt.gtegsc.com!news.mathworks.com!newshost.marcam.com!usc!math.ohio-state.edu!howland.reston.ans.net!tank.news.pipex.net!pipex!uknet!nplpsg!merlin.dclf.npl.co.uk!jrs From: jrs@dclf.npl.co.uk (Dr John Stockton, NPL UK) Newsgroups: sci.data.formats Subject: Re: International Standard Date Notation Date: Tue, 18 Jul 1995 10:23:28 Organization: National Physical Laboratory, UK Lines: 15 Message-ID: References: <3uaog8$l3u@cd4680fs.rrze.uni-erlangen.de> <1995Jul17.131006.27286@rsg1.er.usgs.gov> <3udt0m$s42@netline-fddi.jpl.nasa.gov> NNTP-Posting-Host: merlin.dclf.npl.co.uk X-Newsreader: Trumpet for Windows [Version 1.0 Rev A] In article <3udt0m$s42@netline-fddi.jpl.nasa.gov> dundas@salt.jpl.nasa.gov (John A. Dundas III) writes: >Actually, I just happen to have a copy of the full ISO 8601 standard in front of>me and there are two different complete representations for calendar date. >Section 5.2.1.1 describes the "basic" format as CCYYMMDD, which corresponds >to Peter's example, and the "extended" format as CCYY-MM-DD... How should one do the date&time? I have used 1995/07/18-10:22:49, but from the above I suspect that may not be the preferred punctuation. -- John Stockton : JRS@dclf.npl.co.uk from off-site. MIME. WP. National Physical Laboratory, Teddington, Middlesex, TW11 0LW, UK Direct Phone +44 181-943 6087, Nearby Fax +44 181-943 7138 ** Postings out, Email in/out are fast; News in is often slow. ** Offshore news takes 0..10+ days to arrive; please mail copies of replies! Regret system puts unzoned (UK civil) time on messages. From Curtis Emerson Fri Jul 21 00:22:34 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!maui.cc.odu.edu!kuroshio.ccpo.odu.edu!xanth.cs.odu.edu!lll-winken.llnl.gov!ames!newsfeed.gsfc.nasa.gov!usenet From: Curtis Emerson Newsgroups: sci.data.formats Subject: Re: International Standard Date Notation Date: 19 Jul 1995 19:47:17 GMT Organization: NASA Goddard Space Flight Center -- Greenbelt, Maryland USA Lines: 25 Message-ID: <3ujng5$pp2@post.gsfc.nasa.gov> References: <3uaog8$l3u@cd4680fs.rrze.uni-erlangen.de> Reply-To: Curtis.Emerson@gsfc.nasa.gov NNTP-Posting-Host: next532-9.gsfc.nasa.gov Markus Kuhn writes > > > International Standard Date Notation > ------------------------------------ > > For those of you wondering, what the professional and correct way of > writing a calendar date in network publications is: > > The international standard date notation is: YYYY-MM-DD > > For example, February 4, 1995 is written as 1995-02-04. This notation > is standardized in International Standard ISO 8601. For a detailed Other older standards include ANSI X3.30,43,51 CCSDS (Consultative Committee for Space Data Systems) http://ddwilson.gsfc.nasa.gov/CCSDS-A.html Note that there is no RDBMS SQL standard for date/time format. Would be nice if everyone adopted ISO format. Curtis From mskuhn@cip.informatik.uni-erlangen.de Sat Jul 22 18:41:39 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!maui.cc.odu.edu!kuroshio.ccpo.odu.edu!xanth.cs.odu.edu!lll-winken.llnl.gov!simtel!fu-berlin.de!cs.tu-berlin.de!fauern!rrze.uni-erlangen.de!cip.informatik.uni-erlangen.de!mskuhn From: mskuhn@cip.informatik.uni-erlangen.de (Markus Kuhn) Newsgroups: sci.data.formats Subject: Re: International Standard Date Notation Date: 21 Jul 1995 14:48:25 GMT Organization: Student Pool, CSD., University of Erlangen Lines: 29 Message-ID: <3uoenp$e98@cd4680fs.rrze.uni-erlangen.de> References: <3uaog8$l3u@cd4680fs.rrze.uni-erlangen.de> <1995Jul17.131006.27286@rsg1.er.usgs.gov> <3udt0m$s42@netline-fddi.jpl.nasa.gov> Reply-To: mskuhn@cip.informatik.uni-erlangen.de NNTP-Posting-Host: faui04g.informatik.uni-erlangen.de jrs@dclf.npl.co.uk (Dr John Stockton, NPL UK) writes: >How should one do the date&time? I have used 1995/07/18-10:22:49, but from >the above I suspect that may not be the preferred punctuation. Replace the '/' by '-'. Usually, I separate the date and time fields by a space, in case you need a non-space joiner for the two fields, ISO 8601 suggests a capital T: 1995-07-18T10:22:49 I prefer 1995-07-18 10:22:49 ISO 8601 defines a whole compatible family of date and time notations for various requirements. E.g. you can also write 19950718T102249 which is sometimes used in compact file formats. They are all automatically distinguishable. For details, please download from ftp.uni-erlangen.de the file pub/doc/ISO/ISO8601.ps.Z or get a copy of ISO 8601. Markus --- Markus Kuhn, Computer Science student -- University of Erlangen, Internet Mail: - Germany WWW Home: From mskuhn@cip.informatik.uni-erlangen.de Mon Jul 24 10:54:11 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!caen!hookup!news.mathworks.com!fu-berlin.de!cs.tu-berlin.de!fauern!rrze.uni-erlangen.de!cip.informatik.uni-erlangen.de!mskuhn From: mskuhn@cip.informatik.uni-erlangen.de (Markus Kuhn) Newsgroups: sci.data.formats Subject: Re: International Standard Date Notation Date: 23 Jul 1995 10:53:35 GMT Organization: Student Pool, CSD., University of Erlangen Lines: 61 Message-ID: <3ut9nf$4ih@cd4680fs.rrze.uni-erlangen.de> References: <3uaog8$l3u@cd4680fs.rrze.uni-erlangen.de> <3ujng5$pp2@post.gsfc.nasa.gov> Reply-To: mskuhn@cip.informatik.uni-erlangen.de NNTP-Posting-Host: faui00b.informatik.uni-erlangen.de Curtis Emerson writes: >Markus Kuhn writes >> >> The international standard date notation is: YYYY-MM-DD >> >> For example, February 4, 1995 is written as 1995-02-04. This notation >> is standardized in International Standard ISO 8601. For a detailed >Other older standards include >ANSI X3.30,43,51 >CCSDS (Consultative Committee for Space Data Systems) > http://ddwilson.gsfc.nasa.gov/CCSDS-A.html >Would be nice if everyone adopted ISO format. I have just checked the CCSDS publications. In the file there are both binary and ASCII date/time formats specified and the ASCII versions are perfectly conforming to ISO 8601 (this standard is even explicitely referenced). The two ASCII variants from ISO 8601 used in this standard are: --------------------------------------------------------------------------- 2.5.1.1 ASCII TIME CODE A, Month/Day of Month Calendar Variation: The format for ASCII Time Code A is as follows: YYYY-MM-DDThh:mm:ss.d->dZ [...] 2.5.1.2 ASCII TIME CODE B, Year/Day of Year Calendar Variation: The format for ASCII Time Code B is as follows: YYYY-DDDThh:mm:ss.d->dZ --------------------------------------------------------------------------- E.g. 1988-01-18T17:20:43.123456Z (the Z after the time indicates according to ISO 8601 that the time is in UTC (Z for ZERO meridian, in international radio communication pronounced as "zulu" or "zulu time"). The second date format (YYYY-DDD) consists of the 4-digit year and the 3-digit day number in the year (values 001 - 365 or 366 in leap years) which is preferred over year-month-day in some organizations. E.g. 1988-01-18 = 1988-018 Markus --- Markus Kuhn, Computer Science student -- University of Erlangen, Internet Mail: - Germany WWW Home: From tg3@u.washington.edu Wed Aug 2 10:20:15 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!caen!hookup!news.mathworks.com!news.bluesky.net!news.sprintlink.net!howland.reston.ans.net!vixen.cso.uiuc.edu!news.uoregon.edu!news.u.washington.edu!gillespy.rad.washington.edu!user From: tg3@u.washington.edu (Thurman Gillespy III) Newsgroups: sci.data.formats Subject: Re: Medical Image Formats? Date: Tue, 01 Aug 1995 11:02:00 -0800 Organization: Dept of Radiology, Univ of Washington Lines: 19 Message-ID: References: <3vlk10$ldd@kira.cc.uakron.edu> NNTP-Posting-Host: gillespy.rad.washington.edu In article <3vlk10$ldd@kira.cc.uakron.edu>, r2dnf@dax.cc.uakron.edu (Deepa N Fernandes-Prabhu ) wrote: >Hi, > >Can anybody let me know what the image formats widely used in the >Medical Industry are and where I can get information/specifications for >them?. Any help is appreciated. > >My e-mail address: r2dnf@guts.biomed.uakron.edu Start here: -- Thurman Gillespy III | tg3@u.washington.edu Department of Radiology, SB-05 | voice (206)543-3320 University of Washington | fax (206)543-6317 Seattle, WA 98195 | From meta@harlequin.co.uk Sun Aug 6 14:37:35 1995 Newsgroups: comp.std.misc,comp.protocols.time.ntp,comp.infosystems.www.authoring.misc,sci.data.formats Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!portal.gmu.edu!europa.chnt.gtegsc.com!cantaloupe.srv.cs.cmu.edu!bb3.andrew.cmu.edu!nntp.sei.cmu.edu!news.psc.edu!hudson.lm.com!hookup!gatech!swrinde!howland.reston.ans.net!plug.news.pipex.net!pipex!tank.news.pipex.net!pipex!dish.news.pipex.net!pipex!harlqn.co.uk!harlequin.co.uk!mcgyorgy.cam.harlequin.co.uk!user From: meta@harlequin.co.uk (the ideal copy) Subject: Re: International Standard Date Notation Message-ID: Sender: usenet@harlequin.co.uk (Usenet Maintainer) Organization: Harlequin Information Systems X-Newsreader: Value-Added NewsWatcher 2.0b24.0+ References: <3uaog8$l3u@cd4680fs.rrze.uni-erlangen.de> <1995Jul17.131006.27286@rsg1.er.usgs.gov> <3udt0m$s42@netline-fddi.jpl.nasa.gov> <19950717T163721Z@naggum.no> Distribution: inet Date: Fri, 28 Jul 1995 13:58:05 GMT Lines: 29 Xref: solitaire.cv.nrao.edu comp.std.misc:1073 comp.protocols.time.ntp:839 comp.infosystems.www.authoring.misc:2018 sci.data.formats:1089 In article <19950717T163721Z@naggum.no>, erik@naggum.no (Erik Naggum) wrote: > [John A. Dundas, III] > | calendar date. Section 5.2.1.1 describes the "basic" format as > | CCYYMMDD, which corresponds to Peter's example, and the "extended" > | format as CCYY-MM-DD, which is what Markus noted. Note that the > | standard says that BOTH are acceptable. I say, both are easily machine > | readable and distinguishable. > > sorry to say so, John, but you point out what the real problem is. a > century (CC) is not a distinct part of a date specification, it's an > integral part of the year. likewise, we don't say CCDYMMdd, D for Decadem > we say YYYYMMDD. What's more, today is 1995-07-28, but it's in the 20th century. But 2000-07-28 is in the 20th century as well, as the 21st century doesn't start until January 1st 2001. So calling the first two digits of the year the "century" is really very misleading. Every time I see someone do it, I think "Ah, there's a person who doesn't really know what's going on." mathew -- the ideal copy is the same, the ideal copy has your name when you can't it makes you can, when you aren't it makes you am http://www.mantis.co.uk/~mathew/ From dundas@salt.jpl.nasa.gov Mon Aug 7 12:22:16 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!portal.gmu.edu!europa.chnt.gtegsc.com!news.mathworks.com!usenet.eel.ufl.edu!netline-fddi.jpl.nasa.gov!salt.jpl.nasa.gov!dundas From: dundas@salt.jpl.nasa.gov (John A. Dundas III) Newsgroups: comp.std.misc,comp.protocols.time.ntp,comp.infosystems.www.authoring.misc,sci.data.formats Subject: Re: International Standard Date Notation Date: 31 Jul 1995 19:58:00 GMT Organization: Jet Propulsion Laboratory Lines: 50 Distribution: inet Message-ID: <3vjck8$stb@netline-fddi.jpl.nasa.gov> References: <3uaog8$l3u@cd4680fs.rrze.uni-erlangen.de> <1995Jul17.131006.27286@rsg1.er.usgs.gov> <3udt0m$s42@netline-fddi.jpl.nasa.gov> <19950717T163721Z@naggum.no> NNTP-Posting-Host: salt.jpl.nasa.gov Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Xref: solitaire.cv.nrao.edu comp.std.misc:1074 comp.protocols.time.ntp:849 comp.infosystems.www.authoring.misc:2070 sci.data.formats:1090 In article meta@harlequin.co.uk (the ideal copy) writes: |> In article <19950717T163721Z@naggum.no>, erik@naggum.no (Erik Naggum) wrote: |> > [John A. Dundas, III] |> > | calendar date. Section 5.2.1.1 describes the "basic" format as |> > | CCYYMMDD, which corresponds to Peter's example, and the "extended" |> > | format as CCYY-MM-DD, which is what Markus noted. Note that the |> > | standard says that BOTH are acceptable. I say, both are easily machine |> > | readable and distinguishable. |> > |> > sorry to say so, John, but you point out what the real problem is. a |> > century (CC) is not a distinct part of a date specification, it's an |> > integral part of the year. likewise, we don't say CCDYMMdd, D for Decadem |> > we say YYYYMMDD. |> |> What's more, today is 1995-07-28, but it's in the 20th century. |> |> But 2000-07-28 is in the 20th century as well, as the 21st century doesn't |> start until January 1st 2001. |> |> So calling the first two digits of the year the "century" is really very |> misleading. Every time I see someone do it, I think "Ah, there's a person |> who doesn't really know what's going on." Gee, and here I thought I was contributing a useful point to an interesting discussion (rather than making a personal assault on another individual's intelligence). At the risk of violating some international copyright treaty, here is the direct quote from 5.2.1.1 (page 4, for those playing along at home): "5.2.1.1 Complete representation When the application clearly identifies the need for an expres- sion only of a calendar date, then the complete representation shall be a single numeric data element comprising eight digits, where [CCYY] represents a calendar year, [MM] the ordinal number of a calendar month within the calendar year, and [DD] the ordinal number of a day within the calendar month. Basic format: CCYYMMDD Example: 19850412 Extended format: CCYY-MM-DD Example: 1985-04-12" Before making any more comments regarding my ability to distinguish what is and is not "going on" READ THE DOCUMENT. It's not my opinion; it's an ISO standard that I had nothing to do with. From cgp@le.ac.uk Mon Aug 7 12:24:52 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!caen!spool.mu.edu!agate!sunsite.doc.ic.ac.uk!warwick!leicester!leicester!not-for-mail From: Clive Page Newsgroups: sci.data.formats Subject: Re: Saving 100GB of data for 15 years cheaply Date: 7 Aug 1995 10:26:05 +0100 Organization: University of Leicester, UK Lines: 24 Message-ID: <404m7d$357@hawk.le.ac.uk> References: <3vtejh$mg@nntp.ucs.ubc.ca> <3vttr9$8r2@lace.colorado.edu> NNTP-Posting-Host: irix.le.ac.uk The problem with saving data for ~15 years is not the reliability of the medium [note for younger readers: "medium" is the almost obsolete singular of "media"] but the pace of technological change. Fifteen years ago we had only just given up using paper tape and punched cards, and our data were mostly stored on 9-track tapes at 800 bits/inch. We can't read them now, not because the tapes aren't legible, but because the decks become obsolete several years ago. My guess is that the only format that will last another 15 years is the CD-ROM, mainly because it is based on a consumer item, and these formats last longer than computer-only formats (c.f. the LP record). Recordable CDs are reliable and fairly cheap (around US$10/GB) and manufacturers claim that if stored out of sunlight they are stable for at least 20 years. The snag is the capacity, only 0.65 GB each, slowish speed, and the fact that current CD recorders need a stead stream of data which is hard to guarantee on multi-user systems. But if longevity is the most important criterion, then these are tolerable. It's what we use for our astronomical data. -- ------------------------------------------------------------------------- Clive Page, Internet: cgp@star.le.ac.uk Dept of Physics & Astronomy, University of Leicester. From ejh@larry.gsfc.nasa.gov Tue Aug 8 14:57:43 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!caen!hookup!ames!newsfeed.gsfc.nasa.gov!gsfc.nasa.gov!ejh From: ejh@larry.gsfc.nasa.gov (Edward Hartnett) Newsgroups: sci.data.formats Subject: Re: Saving 100GB of data for 15 years cheaply Date: 07 Aug 1995 14:23:28 GMT Organization: NASA Goddard Space Flight Center -- Greenbelt, Maryland USA Lines: 52 Message-ID: References: <3vtejh$mg@nntp.ucs.ubc.ca> <3vttr9$8r2@lace.colorado.edu> <404m7d$357@hawk.le.ac.uk> NNTP-Posting-Host: larry.gsfc.nasa.gov In-reply-to: Clive Page's message of 7 Aug 1995 10:26:05 +0100 >>>>> "Clive" == Clive Page writes: Clive> The problem with saving data for ~15 years is not the reliability of the Clive> medium [note for younger readers: "medium" is the almost obsolete Clive> singular of "media"] but the pace of technological change. Fifteen years Clive> ago we had only just given up using paper tape and punched cards, and our Clive> data were mostly stored on 9-track tapes at 800 bits/inch. We can't read Clive> them now, not because the tapes aren't legible, but because the decks Clive> become obsolete several years ago. Clive> My guess is that the only format that will last another 15 years is the Clive> CD-ROM, mainly because it is based on a consumer item, and these formats Clive> last longer than computer-only formats (c.f. the LP record). Recordable Clive> CDs are reliable and fairly cheap (around US$10/GB) and manufacturers Clive> claim that if stored out of sunlight they are stable for at least 20 Clive> years. The snag is the capacity, only 0.65 GB each, slowish speed, and Clive> the fact that current CD recorders need a stead stream of data which is Clive> hard to guarantee on multi-user systems. But if longevity is the most Clive> important criterion, then these are tolerable. It's what we use for our Clive> astronomical data. I see your point, but must disagree. Firstly, there are paper tape readers out there still. As recently as 1990 I saw one still in use at an old job. I saw punch card readers in use at my University until about that time also. Secondly, if you have a large archive of data and no hardware to read it, you have no one to blame but yourself. When exabyte announces that they are going out of business, buy a few of the tape drives and stash them with the data. Thirdly, one really doesn't need to worry about it that much. If exabyte goes away it will only be because it is replaced by something much better. There will be a long period where both technologies exist side by side, and the worst case scenario is that you transfer all the data to the new media. Finally, while paper tape readers are not very common, neither was much important data written to paper tape. With all the exabyte data out there I would be completely amazed if exabyte tape drives disappeared in 15 years. There are many orders of magnitude more data on exabytes than there were on paper tapes. Even in it's heyday there were never that many paper tape readers out there, because there simply weren't that many computers around back then. PS - I've had a lot of exabyte tapes go bad on my, BTW, so always write important data to at least three tapes. -- From dclunie@flash.us.com Wed Aug 9 07:20:16 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!news.mathworks.com!news.ultranet.com!news.sprintlink.net!howland.reston.ans.net!swrinde!sgigate.sgi.com!genmagic!bug.rahul.net!a2i!rahul.net!a2i!flash.us.com!britt!dclunie From: dclunie@flash.us.com (David Clunie) Newsgroups: sci.data.formats Subject: Re: Saving 100GB of data for 15 years cheaply Date: 4 Aug 1995 21:55:43 GMT Organization: Her Master's Voice Lines: 30 Distribution: world Message-ID: <3vu50v$emq@flash.ksapax> References: <3vtejh$mg@nntp.ucs.ubc.ca> Reply-To: dclunie@flash.us.com NNTP-Posting-Host: britt.ksapax In article mg@nntp.ucs.ubc.ca, keith@msmri.med.ubc.ca () writes: >Please post replies to sci.data.formats. You might try posting this to alt.image.medical and comp.protocols.dicom if you want medical imaging related responses. >I need to save several hundred gigabytes of medical imaging data on a cheap >reliable media. We are currently saving the data to 10GB 8mm Exabyte tapes. >This is cheap but is only good for about 1 year. Why is it only good for a year ? Do you read these tapes a lot ? I understood that Exabytes lasted much longer than that. Are they less reliable than DAT's ? Have you looked into DLT ? I gather it has a very large storage capacity. I presume that WORM's, MOD's and even CD-R are not cost effect for your application ? BTW. What format are you archiving the images in ? --- David A. Clunie (dclunie@flash.us.com) Soon leaving sunny Riyadh, Saudi Arabia. at home in the desert: http://www.rahul.net/dclunie/html/ alt.image.medical FAQ: ftp://ftp.rahul.net/pub/dclunie/medical-image-faq/ http://www.rahul.net/dclunie/medical-image-faq/html/ dicom3tools: ftp://ftp.rahul.net/pub/dclunie/dicom3tools.tar.gz From cgp@le.ac.uk Wed Aug 9 13:17:12 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!caen!zip.eecs.umich.edu!newshost.marcam.com!news.mathworks.com!tank.news.pipex.net!pipex!warwick!leicester!leicester!not-for-mail From: Clive Page Newsgroups: sci.data.formats Subject: Re: Saving 100GB of data for 15 years cheaply Date: 9 Aug 1995 13:56:20 +0100 Organization: University of Leicester, UK Lines: 38 Message-ID: <40ab9k$7ac@hawk.le.ac.uk> References: <3vtejh$mg@nntp.ucs.ubc.ca> <3vttr9$8r2@lace.colorado.edu> <404m7d$357@hawk.le.ac.uk> NNTP-Posting-Host: irix.le.ac.uk In article , Edward Hartnett wrote: >Thirdly, one really doesn't need to worry about it that much. If >exabyte goes away it will only be because it is replaced by something >much better. There will be a long period where both technologies exist >side by side, and the worst case scenario is that you transfer all the >data to the new media. The difficulty of transferring data shouldn't be underestimated. We had a large collection here of data on ~3000 9-track tapes. When we moved >from VAXes to Unix systems and were about to lose our last 9-track deck we wondered what to do about the data. Tests showed they could be copied to Exabyte or DAT at a rate of only 10 to 15 old tapes a day, so it would take at least one person working for a whole year to transfer them all. We couldn't afford to get someone to do this. Fortunately most of the data tapes were just copies of those kept elsewhere by ESA, so we junked them, and copied the few unique ones. If this hadn't been so, we'd have had a real problem. We now have a large and growing collection of Exabytes. I expect these to become obsolete in a few years (it already looks as if DAT and DLT are superior replacements because each was designed from the outset as a digital medium). I'm not at all confident that the transfer from Exabyte to whatever replaces it will be any easier than the last one (Exabytes are pretty slow devices). But I'd bet that the CD-ROM outlasts the Exabyte in regular use, even though its capacity is lower, and despite the use of (analogue) 8mm tapes in camcorders. >PS - I've had a lot of exabyte tapes go bad on my, BTW, so always >write important data to at least three tapes. I'll agree with you there. -- ------------------------------------------------------------------------- Clive Page, Internet: cgp@star.le.ac.uk Dept of Physics & Astronomy, University of Leicester. From maturney@acad.ursinus.edu Fri Aug 11 15:15:10 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!portal.gmu.edu!europa.chnt.gtegsc.com!news.mathworks.com!newsfeed.internetmci.com!news.sprintlink.net!howland.reston.ans.net!newsserver.jvnc.net!newsserver2.jvnc.net!netnews.upenn.edu!netnews.ursinus.edu!acad.ursinus.edu!maturney From: maturney@acad.ursinus.edu (Markish boyperson) Newsgroups: sci.data.formats,sci.astro,sci.image.processing,sci.physics.accelerators Subject: Re: Saving 100GB of data for 15 years cheaply Message-ID: <1995Aug8.211314.1436@acad.ursinus.edu> Date: 8 Aug 95 21:13:14 EST References: <3vtejh$mg@nntp.ucs.ubc.ca> Organization: Ursinus College Lines: 30 Xref: solitaire.cv.nrao.edu sci.data.formats:1119 sci.astro:82680 sci.image.processing:16003 sci.physics.accelerators:1567 In article <3vtejh$mg@nntp.ucs.ubc.ca>, keith@msmri.med.ubc.ca writes: > Please post replies to sci.data.formats. > > I need to save several hundred gigabytes of medical imaging data on a cheap > reliable media. We are currently saving the data to 10GB 8mm Exabyte tapes. > This is cheap but is only good for about 1 year. Does anyone have any better > suggestions? > > Thanks in advance. > > Keith S Cover > Physics, UBC > Vancouver, Canada > keith@msmri.med.ubc.ca > I'm not exactly sure of the tape costs, but try dlt (digital linear tape). Its a 1/2" format, and I think it lasts a lot longer than exabyte. It has 1.25 megs/sec uncompressed throughput, and depending on the drive has 10, 15, or 20 gigs per tape, that's uncompressed, double that for compression, as well as the throuput. The drives are the dlt 2000, the dlt 2000xt, and the dlt 4000. I think the prices are $3000, $4000, and $5500. I think the cost of media for dlt 2000 is less than $40 per tape, somewhere around the same cost for the dlt 2000xt, and double that for the dlt 4000. The manufacturer is quantum, even though I could have sworn seeying them under dec's name. Good luck, tell me what you find out, we're going to be buying the dlt 2000xt for daily backup in our digital editing suite of compressed video. Mark Turney maturney@acad.ursinus.edu From gwood@bmr.gov.au Fri Aug 11 21:02:08 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!news.mathworks.com!zombie.ncsc.mil!simtel!harbinger.cc.monash.edu.au!newshost.anu.edu.au!garnet.bmr.gov.au!garnet!gwood From: gwood@bmr.gov.au (Geoff Wood) Newsgroups: sci.data.formats Subject: Re: Saving 100GB of data for 15 years cheapl Date: 8 Aug 1995 23:11:08 GMT Organization: Bureau of Mineral Resources Lines: 2 Distribution: world Message-ID: <408qucINNpgk@garnet.bmr.gov.au> References: Reply-To: gwood@bmr.gov.au NNTP-Posting-Host: garnet.bmr.gov.au But will CD-ROm be around in 15 years time either ?? From Terry_norton@HP-Loveland-om10.om.hp.com Sun Aug 13 07:49:49 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!portal.gmu.edu!europa.chnt.gtegsc.com!news.mathworks.com!usenet.eel.ufl.edu!col.hp.com!fc.hp.com!news.lvld.hp.com!news From: Terry Norton Newsgroups: comp.arch.storage,sci.data.formats Subject: Re: QIC format specs, where ? Date: 9 Aug 1995 23:02:58 GMT Organization: Hewlett-Packard Co., Loveland, CO Lines: 32 Message-ID: <40ber2$8av@hplvec.lvld.hp.com> References: <3vbf5n$pln@surt.ifi.uio.no> NNTP-Posting-Host: hpcm6b1e.lvld.hp.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailer: Mozilla 1.1N (Windows; I; 16bit) Xref: solitaire.cv.nrao.edu comp.arch.storage:6972 sci.data.formats:1129 michaels@ifi.uio.no (Michael Shuldman) wrote: >Where can I find the specifications for the QIC format ? > > >-- > _ // > \X/ -- Michael Shuldman > You need to know the QIC specification number. There are different specifications covering many areas of QIC type technology. Standard Formats you may be interested in are: QIC-80-MC Standard QIC-80 Tape Format w/Travan Support QIC-3020-MC New Floppy interface format tape QIC-117 Floppy Tape Interface Spec and commands I don't think any of the QIC documents are available online (legally). Anyway, the documents are available through: Quarter-Inch Cartridge Drive Standards, Inc. 311 East Carrillo Street Santa Barbara, California 93101 USA PH#(805)963-3853 FAX(805)962-1541 I don't know if the specifications are readily available to non-QIC members. Terry Norton HP/Colorado Memory Systems From Walter.Harms@arbi.informatik.uni-oldenburg.de Sun Aug 13 22:16:06 1995 Newsgroups: sci.data.formats Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!news.mathworks.com!fu-berlin.de!zrz.TU-Berlin.DE!zib-berlin.de!uniol!Walter.Harms From: Walter.Harms@arbi.informatik.uni-oldenburg.de (Walter Harms) Subject: Re: Saving 100GB of data for 15 years cheaply Organization: University of Oldenburg, Germany Date: Thu, 10 Aug 1995 10:35:13 GMT Message-ID: <1995Aug10.104119.5777@arbi.Informatik.Uni-Oldenburg.DE> References: <3vtejh$mg@nntp.ucs.ubc.ca> <3vts2p$4j0@pith.uoregon.edu> <400hdq$8se@nntp.ucs.ubc.ca> Sender: news@arbi.Informatik.Uni-Oldenburg.DE Lines: 24 keith@msmri.med.ubc.ca writes: >CD-ROM's are difficult to deal with because of there small size (700MB). That means you >you are dealling with ~130 CD's per 100GB with is not as nice as dealing with >10 10GB chunk's on 8mm tapes. The access time of tape is adequate for our >needs. The 10GB DTL tapes someone suggested sound interesting. There is >a writeup in the March 15, 95 Byte which I have not had a chance >to look at. that are the std cd-rom for micro-computers, you may like to heare that there are drive capabel of terabytes. they are used in a local ship-yard for storring blue-prints of ships, maybe is that a direction to look. walter-- #################################################################### "Life starts at '030, fun starts at '040, impotence starts at '86" keyboard not connected -- press F1 to continue <><><><><><>alles nur gestohlen, alles nur geraubt ..<><><><><><><><> From Curtis Emerson Wed Aug 23 15:42:36 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!caen!spool.mu.edu!agate!ames!newsfeed.gsfc.nasa.gov!usenet From: Curtis Emerson Newsgroups: sci.data.formats Subject: Re: Saving 100GB of data for 15 years cheaply Date: 22 Aug 1995 20:56:59 GMT Organization: NASA Goddard Space Flight Center -- Greenbelt, Maryland USA Lines: 7 Message-ID: <41dgar$1sv@post.gsfc.nasa.gov> References: <40o50s$mej@engr.orst.edu> Reply-To: Curtis.Emerson@gsfc.nasa.gov NNTP-Posting-Host: next532-9.gsfc.nasa.gov Another source for information is Defense Electronics magazine (ISSN 0278-3479). Articles and advertisements have appeared in Oct, Nov, Dec 1993 editions, and July 1994. Probably more focused on avionic and terabyte requirements. Curtis From jeff@gg.caltech.edu Thu Aug 24 17:08:09 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!news-server.ncren.net!news.duke.edu!news.mathworks.com!newsfeed.internetmci.com!news.msfc.nasa.gov!pendragon.jsc.nasa.gov!ames!lll-winken.llnl.gov!fnnews.fnal.gov!nntp-server.caltech.edu!jeff From: jeff@gg.caltech.edu (Jeff Goldsmith) Newsgroups: sci.data.formats Subject: Visualization Freeware Release Date: 23 Aug 1995 22:55:03 GMT Organization: California Institute of Technology, Pasadena Lines: 44 Message-ID: <41gbk7$i7a@gap.cco.caltech.edu> NNTP-Posting-Host: muggy.gg.caltech.edu X-Newsreader: NN version 6.5.0 (NOV) LinkWinds 2.1 Release The new LinkWinds, version 2.1, is ready for release. For those unfamiliar with LinkWinds, a brief description is below. The software executes only on SGI platforms under IRIX 4 or IRIX 5. An alpha-test build for the Sun workstation running under Solaris 2.4 is also available. It contains the 2- dimensional subset of LinkWinds applications only. The Linked Windows Interactive Data System (LinkWinds) is a visual data exploration system which has served as a testbed and prototyping environment for a NASA/JPL program of research into graphical methods for rapidly and interactively accessing, displaying and analyzing large multivariate multidisciplinary datasets. It provides a variety of functions and services, including interactive 2- dimensional and 3-dimensional graphical displays of data, hard copy of graphical displays and text, interactive color manipulation, animation creation and display, data subsetting either at the input or output, a journal and macro capability, context-sensitive help, and network support for data search and retrieval as well as collaborative data analysis. It is an integrated multi- application execution environment, running under UNIX, with a full graphical user interface (GUI). The system has several facilities for database access and provides for the ingestion of very large data files in a variety of database formats. Data formats currently accepted are: 1) Raw binary integer (1 - 4 byte) and floating point (4, 8 byte) data. 2) The Hierarchical Data Format (HDF). 3) The Common Data Format (CDF). 4) NetCDF. 5) The Silicon Graphics, Inc. native RGB image format. 6) Data with Planetary Data System (PDS) headers. 7) The astrophysics Flexible Image Transport System (FITS). Various LinkWinds packages are available via anonymous ftp from: twinky.jpl.nasa.gov in the /pub/LinkWinds directory. YOU MUST MAKE THE TRANSFER IN BINARY MODE. From dilg@newsroom.hitc.com Mon Aug 28 10:25:24 1995 Path: solitaire.cv.nrao.edu!hearst.acc.Virginia.EDU!caen!spool.mu.edu!howland.reston.ans.net!swrinde!elroy.jpl.nasa.gov!lll-winken.llnl.gov!ames!newsfeed.gsfc.nasa.gov!newsroom.hitc.com!dilg From: dilg@newsroom.hitc.com (Doug Ilg) Newsgroups: sci.data.formats Subject: Re: HDF data structures support questions Date: 25 Aug 1995 14:33:08 GMT Organization: NASA Goddard Space Flight Center -- Greenbelt, Maryland USA Lines: 77 Distribution: world Message-ID: <41kmv4$q55@post.gsfc.nasa.gov> References: <41i537$cmf@fnnews.fnal.gov> NNTP-Posting-Host: newsroom.gsfc.nasa.gov In article <41i537$cmf@fnnews.fnal.gov>, kennedy@b0ru01.fnal.gov (Robert Daniel Kennedy) writes: |> Hello, |> |> I am currently evaluating HDF for use in the storage of event data |> organized by a rich set of data structures, for experiments in the year 1999. |> Data in an event comes from a large number of detectors, each sub-divided by elements |> and readout devices. Efficiency of storage and the comprehensibility of the data |> structure are both issues in the evaluation, as well as the speed of reading/writing |> data of course. I have gone through the documentation, FAQ, and so on... and have a few |> questions that remain unanswered. Perhaps someone would be good enough to shed some light |> on these, or perhaps provide some off-the-cuff work-around. No work-arounds necessary, Rob. HDF does exactly what you want, already. |> (1) Is there support for a variable-length array? We often summarize a detector with a |> just a few elements "hit" by a variable-length array such as |> {Nhits, hit_location_1, hit_location2, ... hit_location_n} |> where Nhits << number_of_elements in the detector. Both SDSs and Vdatas are variable length structures. SDSs can be extended along the first dimension (using C dimension order) by specifying SD_UNLIMITED or NC_UNLIMITED as the size of that dimension. Vdatas can be extended in the record/element "dimension." There's no need to keep around info like Nhits, though. HDF takes care of that. |> (2) Is there a way of "grouping" type information in the Vdata/Vgroup model |> to reduce the type information overhead? Many blocks in our data are the same |> set of types repeated N times for N detector elements. Our current data storage |> format has a repeat count to reduce the size of the type specification tremendously... |> N(type1, type2) means N consecutive groups of type1 followed by type2. Okay, so maybe this is a work-around. I didn't understand your question on first reading. Anyway, we're doing similar things in the EOS project at NASA. There are several ways that come to mind to handle this. 1) Re-order your data so that you have multiple instances of type1 and type2 sitting next to each other. Then you can use the `order` argument when you define your vdata fields to get N(type1), N(type2). Maybe not the best, but it would work. 2) Define each as an individual field and get type1(1), type2(1), type1(2), type2(2),... type1(N), type2(N). This way seems quite tedious and really ignores the point of grouping. 3) Define one field of a type whose size (times its order) is equal to sizeof(type1) + sizeof(type2), then do your own monkeying around with picking apart the values after you read them. This is not a good solution at all, but it *is* an option. 4) Define a vdata with one field of type1 and one of type2. Put all data records for detector 1 in the vdata, followed by all records for detector 2, etc. Then create an index vdata with (at least) a "begin" field and an "extent" field. The index has one record for each detector which tells where in the other vdata to look to find the data for each detector. For instance, if detector 1 has 6 records and detector 2 has 9, the index would look like this: begin extent 0 6 6 9 You can add any extra info in the index or the data vdata, wherever it makes sense. This is the scheme we're using in EOS to handle what we call "Point Data." It works well if your data is not growing dynamically. It's a real pain to insert a new record for detector 1 after you have begun to write out records from detector 2, though. If you need to do that, you could split the data records into several vdatas, one per detector, and each one could grow independently. Of course, then you need to worry about a bunch of horribly fragmented vdatas, if you're not careful about how they're written out. A good buffering scheme goes a long way in this situation. If you want to know more about our "Point Data Model," let me know. I can probably arrange to get you some documentation. Hope this helps. -Doug Ilg PS. Check out the HDF Web server at "http://hdf.ncsa.uiuc.edu:8001/" It's got lots of good info, including a page on HDF's limitations and copies of all the docs. Make sure you read the new "Users Guide."