Subject: Metadata Workshop Meeting Meetings for 17-18 Febuary 1994

From: [email-masked] (Dale Lancaster)
Date: Mon, 28 Feb 94 16:28:54 -0600

Greetings to the reflector folks.  Below are the minutes from our last
meeting.  My PC could not recover a portion of the file I had stored
the minutes in.  But I did take some written notes that I think may
have filled in the gaps.  Please review and let me know what I missed
or should be corrected.  After a few days of collecting changes, I
will repost to the reflector and to other internet places that make
sense.

regards,

dale

			  Metadata Workshop
			   Meeting Minutes
			 17-18 February 1994

This workshop was a second in a series focused on understanding and
defining the "metadata" or data access problem.  The goal is to create
awareness of the problem and/or standards for how to deal with it.

The workshop was held at the Center for High Performance Computing
(CHPC) in Austin, Texas on 17 and 18 February 1994.  Jim Almond
(CHPC), Dale Lancaster (Convex) and Otis Graf (IBM FSC) co-lead the
workshop.  The workshop was sponsored by the IEEE Mass Storage Systems
and Technology Committee (MSS&TC) as part of the Data Management
Technology sub-committee.

The minutes include the following categories:

A) Actions Items from the meeting
B) Attendance List
C) IEEE Structure
D) Metadata White Paper
E) Metadata Types and taxonomy
F) Metadata Reference Model
G) Next Workshop in May on the Reference Model
H) Other related metadata projects/items
I) Miscellaneous Notes
J) Who are the attendees and their reasons for attending

A) Actions Items from the meeting

* Kick-off an email discussion on the [email-masked] reflector
to:

	- Suggest agenda items/structure for the upcoming Metadata
	Reference Model workshop in May.

	- Define the scope of a Metadata Reference Model
	- Define the functional requirements of a Metadata Reference Model

	The intent is to have various members of the metadata reflector
	propose various options and definitions for the above and have
	them debated electronically.  There will be a need at some point
	to "moderate" and/or conclude the discussions before the workshop
	on the Metadata Reference Model to be held in May.

* Update the Metadata whitepaper to include the various thoughts and
conclusions from this workshops.  Robyn Sumpter will do this.

* Setup a Xmosaic master home page for Metadata.  This home page
can/will point to other sources of information relating to the
Metadata effort.  The intent is to create a clearing house for
tracking all the various technologies and efforts related to data
access and the metadata problem.  Robyn Sumpter and Ron Pfaff will
work on this.  Also it was suggested to send a general message to the
internet to invite other organizations to feed their information/work
on metadata into this system.

* Several people wanted information on the DMIG (Data Management
Interface Group) effort which is defining a standard system call
interface for handling low level file migration requirements.  Dale
Lancaster will post this information to the reflector.

* Create a "real" effort to attempt to solve the metadata access
problem in a practical way.  Several people were interested in getting
together to do this.  This would not be part of the IEEE, but would be
consistent with the metadata effort under it.

B) Attendance List

Name		  Organization	Email				Phone
===============================================================================

Jim Almond	  CHPC		[email-masked]	512-471-2442
Dale Lancaster	  Convex	[email-masked]			214-497-4581
Otis Graf	  IBM FSC	[email-masked]	713-282-8216
Roy Skruggs	  Triada	[email-masked]	404-951-5493
Mike Daily	  Mobil R&D	[email-masked]		214-851-8836
Robyn Sumpter	  LLNL		[email-masked]		510-423-5054
Carol Hunter 	  LLNL		[email-masked]		510-422-1657
Becky Springmeyer LLNL		[email-masked]		510-423-0794
Ann Wheeler	  Britton Lee	[email-masked]			408-370-1400
Lynn Wheeler	  Britton Lee	[email-masked]			408-370-1400
Tony Baraghimian  Hughes	[email-masked]		703-759-1392
Parris Caulk	  Loral 	[email-masked]		301-925-0610
Charles Dollar	  National
		  Archives	[email-masked]	301-713-7076
William Farrell	  SAIC		[email-masked]		619-458-2645
Gregory Jirak	  Xidak		[email-masked]			415-855-9271
Maria Zemankova	  MITRE		[email-masked]		703-883-6217
Michael Josephs	  MITRE		[email-masked]		703-883-6567
Ron Pfaff	  LANL		[email-masked]			505-667-8182

C) IEEE Structure

Bob Coyne of IBM Federal Systems, representing the IEEE Mass Storage
Systems and Technology Committee (MSS&TC) reviewed how this workshop
and its participants fit into the IEEE MSS&TC structure.

This workshop is known as the "Metadata Project" which is coordinated
by Jim, Dale and Otis.  This project comes under the "Data Management
Project Committee made up of Bob Coyne (Chair), Otis Graf (Co-chair)
and Dale Lancaster.  This committee is responsible for coordinating
and sponsoring workshops and projects under the IEEE MSS&TC that deal
with data access and management with particular interest from a
storage systems perspective.  This committee had, at one time, created
another workshop effort called "Data Management Workshop" that was
originally to deal with large scientific databases and how to handle
them in regards to a storage system.  

The "Data Management Workshop" coordination committee met and decided
that their interests where identical to that of this metadata workshop
series.  As a result they have decided to hold a workshop in May to
deal with a "Metadata Reference Model".  It is not clear exactly how
these two efforts should merge.  According to discussion at this
workshop, the participants felt that they should play a major role in
deciding the agenda and structure of such a workshop.

Bob noted that the IEEE MSS&TC wants a common set of terms to talk
about metadata and to provide an intellectural framework to discuss
such solutions. Such models take on four forms: whitepaper, guide,
recommended practice or standard).  The MSS&TC wants to provide terms
and framework and then recommend where standards work should be done.
(The MSS&TC cannot actually work on, or sponsor standards).  Standards
may then go over to ANSI and/or and/or the Storage Systems Standards
Working Group (P1244 effort) or another place in IEEE. Bob stated that
it would be desirable to avoid the problem of creating an implied
standard such as was done when the Mass Storage Reference Model was
created.  He also suggested that we not concentrate on metadata just
in terms of a filesystem, but that other storage methods and structures
could exist that will need a metadata access solution.

Washington D.C. Workshop - Francis Bretherton will chair this
workshop.  He has a reference model for metadata and wants to review
and change it.  Maybe have alternatives.  Who should be invited to
this?  Will cost $300 to get in, 2-3 days, meals, small groups,
assignments.  Let Otis or Dale know if interested.

D) Metadata White Paper.

Robyn Sumpter outlined her paper to the group.  Major items:

* At least two types of metadata: application and filesystem.  It was
noted that other major types of metadata existed.  It was suggested
that another way to split it would be "system" and "non-system
(application, user, etc)" metadata.

* Robyn showed a block diagram of interfaces to a storage system
It was recommended that "filesystem" and "data management" logical
blocks shown be interlaced rather than layered on top of each other.

* Need to define the type and structure of metadata for applications
     - promote portability
     (some discussion on abstractions)
     (more discussion on metadata is data as well and how to
        handle this)
     (be concerned about relationships between data)

* SNMP model for metadata - 
    - may need meta agent (things that describe the agent)
    - how to handle relationships between metadata
    - applications give meaning to data (can this be determined
        from a bottom up approach?).

Robyn will update this whitepaper based on input received from 
the workshop.

E) Metadata types and Taxonomy

Jim Almond reviewed his whitepaper on Collection of Metadata Types and
Taxonomy.

* Two types of metadata
	- metadata describing the informational entity itself
            - abstract data type (What is this?, book,image,matrix)
            - informational essence (What basic information 
                 does it give me?)
            - relationships between the entity and other entities

        - metadata pertaining to the storage and use of the 
            informational entity
            - pertaining to storage and replication of entities
                on physical 2media (computer and non-computer)
            - metadata attributes recording or controlling the use 
                of entities.
* Went through an example of the above using the definition
        of how and what a matrix of numbers is represented.

F) Metadata Reference Model

The group discussed the merits and possible structure of a Metadata
Reference Model.  

There was agreement that there is a need for some type of Metadata
Reference Model.  As a minimum it would facilitate the need to develop
a agreed upon terminology to describe the problem of metadata and
intelligent access to data.  It would also establish an intellectural
framework within which new ideas could be developed. 

Several graphical diagrams were proposed during the meeting about how
this model could appear and/or what its boundries are in a computer
system.  However there was some agreement that a common set of APIs
that define the interaction of applications and system software with a
metadata system is needed.  One of the major concerns is how to handle
"multiple views" of data.  That is, the ability for a metadata and
storage system to easily provide for different users to define and
access the same data from different perspectives and requirements.

There was no clear agreement about the scope or definition of the
model.  It was agreed to debate this electronically via the metadata
reflector ([email-masked]) and to feed the results into the May
workshop.

G) Next Workshop in May on the Reference Model

Bob Coyne announced this workshop will take place in May (see
announcement later on in this section).  He has suggested that a
primary theme is to review a version of a Metadata Model from Francis
Bretherton (who will be Chairing this meeting).  Everyone agreed that
this workshop and its participants should be heavily involved and
provide as much input as possible.  The group agreed to debate/gen up
a proposed agenda for the workshop and submit that to Francis.  We agreed
that it is desirable not to make one person's view of the model the
theme or focus of the workshop.  It is hoped that debate of various
models can be done electronically on the reflector and the results
of that be discussed at the May workshop.  The announcement for this
workshop is:

From: [email-masked]
To: [email-masked]
Subject: IEEE Metadata Workshop, May 16-18, Wash DC
Date: Mon, 21 Feb 94 15:43:30 CST

IEEE MSS&TC plans to hold a Metadata workshop in the Washington area May
16-18, 1994. Francis Bretherton is the program chair.

We are soliciting input for the workshop and inviting folks to volunteer
to support the workshop.

Attendance may be limited to 50 persons (combination of invitation and
open registration for IEEE members). Registration fee target is $300.00

One topic of the workshop will be to discuss and critique Francis' Strawman
Reference Document for Metadata. Another topic of interest is agreement
on a suite of standards and public protocols require to ensure
stewardship of data over a long period (decades). Other topics are
solicited. We will probably have 4-5 separate working groups of 8-10 folks.

We are looking for recommendation for the invitation list.

It is not clear what role the folks attending the UT Austin Workshop
and using this reflector desire through the IEEE MSS&TC Data Management
committee efforts. Hopefully that will sort it self out over the next couple
of workshops. Participants of the UT Austin Workshop and this reflector
are welcome to participate in the May 16th workshop.

Send comments and questions to me or Otis Graf.

Regards, Bob
Chair, IEEE MSS&TC Data Management Committee

H) Other activities:

- Massive Digital Data Workshop held in Reston, VA this past January.

Dale Lancaster gave out copies of the notes from subgroup at this
workshop that dealt with Metadata.  (These notes were also sent
to the reflector).  The priorization of the metadata problems
identified by the MDDS group include:

	1. Presentation of metadata.  It was emphasized that it be based on
        how users naturally would access the data based on the
        application they are using.

	1. Defining a structure/method/standard for how metadata is created
        and defined (I like the MIB approach suggested by Robyn in
        the whitepaper).

	2. Automation of the derivation of relationships in data and metadata

	2. Defining an API to applications and storage (and maybe other
        major functional areas).

	3. Automatic generation of metadata

	4. How to evolve a metadata system (especially from legacy systems
        to state-of-the-art systems)

	5. Tools for metadata creation and management

	6. Optimization (some wanted to say parallelization, but the real issue
        is how to make it fast and efficient)

- POSC - PetroTechnical Open Software Corp. Mike Daily gave an overview
of this effort in the petroleum industry. Notes from his presentation:

AAPG - American Association of Petroleum Geologists.  They establish
recommended practices for petroleum geologists.  One of those is a
standard data format called RP-66, a self-describing data format.

SEG - Society of Exploration Geophysicists.  SEG has established two
types of tape formats:

	SEG-Y - current tape format which is header+data
	SEG-DEF - RP-66 flavored format
	RODE - Record Oriented Data Encapsulation

POSC - PetroTechnical Open Software Corporation

* 4 technical tracks:
	- base standards (C, POSIX, MOTIF, etc )
	- GUI Style Guide (superset of Motif)
	- Data Access - is hybrid object-relational
		- pre-standard SQL3 (UniSQL, HP OpenODB)
		- PEF POSC Exchange Format extended RP-66
	- Data Model - is object oriented
		- maintained in Express language
		- 100's of entities with multiple inheritance
			and complex dataq types (vector, array, etc)
		- relational projection > 900 tables
		- lots of reference entities - 'legal values'

* 2 implementation tracks
	- Migration: tools, methods, policies, procedures for
	data load/validation, application (re)development, etc
	- Application Views: process-specific application "packages".

A graphical view of the above was presented.  Roughly drawn:

		User	DBA	Programmer
Migration	A p p l i c a t i o n  V i e w s
Migration	U s e r  I n t e r f a c e
Migration	D a t a   A c c e s s
Migration	API	  and	Exchange Format
Migration	Logical Model  and Base Standards

- NEONS (Naval Environ. Operation Nowcasting system)
- Sequoia - William Farrell gave an overview of the effort.
    - goal is to develop information management and
    presentation applications to handle global change research
    issues.
    - Mainly sponsored by DEC and several researchers.
    - systems approach to a real world problem.
        - storage systems
        - filesystems
        - database systems
        - WAN protocols
        - visualization of data
    - goal to have 10 TB of accessible data by Jun 1994; not
            going to meet this goal
    - continuing series of technical reports available from Berkeley
- Visualization '93 subgroup on metadata
- MetaStore - CHPC effort reviewed by Jim Almond.

Access to unix files based on descriptive metadata.  Files in a HSM.
User interface is GUI.  Gain experience with Informational Entities.

Currently used only as an experiment.  No real data used in this
system.  Uses Abstract Data Types (ADTs) to classify different types
of data used.

- DMIG - Data Management Interface Group - Dale Lancaster gave a brief
intro and stated he would post detailed information to the reflector.
This group is defining a common operating system interface for system
applications that implement file migration.

I) Miscellaneous Notes

* There was the usual discussion about "data is metadata is data is
metadata".  I think we all agreed that someone's metadata is someone 
else's metadata and vice versa.  [So maybe we should just say that
all data is metadata?? :-].

* There was a small effort to just pay a person/grad student to just
go off and create a "standard" model and we all hash it out afterwards.
This has some merit, but was agreed it wasn't really worthwhile.

* Several people expressed an interest in getting together to "just do
it".  That is, to go head and create some kind of practical system
from various parts to flesh out the problems and ideas of a metadata
based data access and storage system.  Several people had interest in
this.  

* Another issued discussed was how to get metadata forms filled-in
properly by users.  User generally don't like to fill-in anything.
Solutions ranged from automatically generated metadata to "if no
metadata, then the data is deleted/not stored".

* There still seemed to be a need to define broad classes of metadata.
This will have to be discussed/addressed in the metadata reference
model.

* There was a discussion about have classes instead of objects and the
question of "Why not objects to define data types?".  And the
associated question of where to store "algorithms and methods"
associated with the data.  This was not really resolved.

J) Who are the attendees and their reasons for attending

Bob Coyne - UBM Federal Systems - Representing IEEE MSS&TC and how
	it sees this workshop.
Otis Graf - IBM Federal Systems - Representing IEEE MSS&TC Data 
	Management Project Committee.  Also interested in how to link
	metadata rdbms to a mass storage system and begin to classify 
	information entities and research on tools for users and applications.
Dale Lancaster - Interested in gathering requirements for metadata
	based products.  Also interested in helping move things along
	in this area so that this products can be built with standard 
	interfaces.
Jim Almond - CHPC - Information Density and Abstract Data Types.
Ron Pfaff - LLNL -  Tying different user groups together for virtual teaming.
Greg Jirak - Aurora Data management system - structure of scientific data.
        Interested in self-defining/extensible data structures.
Tony Baraghimian - Hughes - interested in machine learning.  
	Accessing petabytes of data
Carol Hunter - LLNL - Intelligent Archive - how to search and browse
    large amounts of data.
Becky Spinmeyer - LLNL - system level metadata, application
    metadata and a "middle level".
Robyn Sumpter - LLNL - Interested in metadata from a storage perspective.
Anne Wheeler - BLI - new DBMS to help deal with metadata access.  Using a
    top down approach to handling the metadata problem.
Roy Skruggs - Triada - N-Gram transform for data and intelligent pattern
	matching.
Parris Caulk - Loral - NASA EOS - 3 TB data/day - have lots of data
    with different formats - "metadata" is a bad term - scientist
    don't want to be restricted in their search and metadata seems
    to imply that.  Interested in content based searching.
William Farrell - SAIC - Global Change Data Sets - Sequoia 2000 project
Mike Daily - Mobil - Petabyte of data - involved in POSC - similar
    to OSF.  POSC trying to set standard for data models and
    access.
Maria Zemankova - MITRE - Database people need to work hand in hand
    with scientific people to solve this problem.  (Our customer
    is the end user, not the standard itself).  Don't want to
    be worse off than we are now because of standards.
Charles Dollar - National Archives - Guidance to federal agencies
    to ensure information is stored, used, processed in such a way
    that it will be accessible for long term. 

Respectfully Submitted,

Dale Lancaster