Subject: Alan Emtage on Bunyip

From: [email-masked] (Hans-Martin Adorf)
Date: Thu, 24 Mar 1994 12:09:57 +0100

Hi All,

> I'm working on trying to provide search capabilities for email discussion
> group listings - initially I'm working on the List of Lists.  I've just
> added WAIS capabilities to our gopher, and one of the perennial user
> questions concerns searching for email lists on a given topic.
>
> First, I'm wondering if anyone else has tackled this.  My guess is that
> someone probably has, but I didn't find any leads in the normal places.

Just FYI, this comes under the scope of two interrelated projects. The
first is the IAFA templates which (he says hopefully) should be sent up
to FYI-class RFC status in the near future. Without going into it in
great detail here the IAFA group of the IETF is chartered (along with
other things) to create "cataloging" templates so that meta-data such of
the kind you are talking about would have a standardized format on the
network. The other project is the Bunyip Internet Directory Project
(BIDP) which should be up in the next little while. I have included the
annnouncement about this below. We are in the process of writing
"quick-and-dirty" documents so that people are not required to read
through the standards documents themselves, but for the moment they are
all we have.

Within the scope of the BIDP is the hope that we would be able to collect
and maintain something akin to the List of Lists. Under our strucuture
the list administrators themselves would maintain their own information
about the mailing list and retrieval of this information would be
automated. I don't believe that manual maintenance of this kind of data
is feasible any longer.... the Internet is now too big for this kind of
thing. Humans do however have a place in doing value added work as
classifying and editing the raw information.

-Alan

-----------------------------------------------------------------------------
Alan Emtage,                    "The Left in Canada is more gauche
Bunyip Information Systems       than sinister"
Montreal Canada                  -The Economist
[email-masked]
-----------------------------------------------------------------------------

                Bunyip Internet Directory Project

Over the next several months we at Bunyip Information Systems (creators
of the archie directory system), will be running a pilot project to
assess the viability of collecting additional data cataloging information
directly from the Internet. Ultimately, this information will be made
available to the Internet community via the distribution mechanisms now
in place for the familiar archie anonymous FTP database.

One of the biggest problems currently facing the use of the wealth of
information now available on the Internet is locating the files,
documents or services that you are looking for. While archie goes some
way to addressing this problem, "cataloging" information such as contact
information, abstracts, descriptions, keywords and access policies for
the files and services are simply not available through simple filename
searches. The problem of trying to keep this cataloging information up to
date manually in such a dynamic environment is well illustrated by the
current archie Software Description Database (or "whatis" database as it
is more commonly known).

We are asking you, the anonymous FTP archive site administrators and
operators of Internet services to cooperate with us by preparing your
information for automated gathering and serving. In doing so, we believe
you will be supporting a data distribution model that is well suited to
the Internet.  Control over your content will remain with you, and the
automatic preparation of additional information services will be made
possible. The data served to the community will be updated when you
change your own files.

You need not tell us that you have these files: if you have asked to be
part of the archie service in the past, this information will be
automatically retrieved from your archive.

Our project will use an extended version of the archie system to collect
data "templates" on anonymous FTP archives. These templates will be
automatically retrieved, collated and indexed into a form easily
searchable by users on the network. Over the period of the pilot a number
of search and retrieval techniques will be assessed to determine their
usability.

The hope is that the general user will be able to query this database and
obtain descriptive information about freely available or Public Domain
documents, images, sounds and services on the network...worldwide. In
addition to the descriptions, the templates point the user to the source
of the original data or service which may then be accessed.

By filling out and making these templates available for retrieval, you
can advertise the files and services available from your department or
institution. The templates can be used to describe

      Services

        The archive can offer an overall description of each the various
        Internet services offered by your organization's systems, along
        with corresponding contact information.

        This description would then indicate whether the the parent
        organization offers such services as:

        o on-line library catalogues

        o Interactive online information services such as WAIS, gopher,
          Prospero, World Wide Web or archie

        o specialized information servers such as those providing
          weather, geographic information, newswire feeds etc.

        o Other information services

      People

        By putting non-sensitive personal contact information into the
        templates we hope to provide the start of the first truly global
        White Pages service: the ability to locate people anywhere on the
        network.

      Documents, Datasets, Mailing List Archives, USETNET Archives,
      Software Packages, Images and other Objects

        You might wish to make available a brief description of available
        software, documents, images, sounds, video, datasets, USENET
        archives and mailing list information through the anonymous FTP
        archive.

A typical "services" template might look like this (no offense to the
Census Bureau intended :-)

Template-Type:          SERVICES
Name:                   Census Bureau information server
Host-Name:              census.ispy.gov
Host-Port:              1234
Protocol:               telnet
Admin-Name:             Jay Bond
Admin-Postal:           PO Box. 42, A Street Washington DC, USA 20001
Admin-Work-Phone:       +1-202-222-3333
Admin-Work-Fax:         +1 202 444 5555
Admin-Email:            [email-masked]
Description:            This server provides information from the
                        latest USA Census Bureau statistics (1990)
                        Type "help" for more information.
Authentication:         Once connected type your email address at
                        the "login:" prompt. No password is
                        required.
Registration:           No formal registration is required
Charging-Policy:        There is no charge for the use of this service
Access-Times:           9:00 EST / 17:00 EST
Access-Policy:          This service may not be used by sites in
                        the Republic of the VTTS
Keywords:               census, population, 1990, statistics
Last-Modified-Name:     Miss Moneypenny
Last-Modified-Email:    [email-masked]
Last-Modified-Date:     Wed, 1 Jan 1970 12:00:00 GMT

Note that the machine on which the actual service or documents reside
need not be the same as that of the anonymous FTP archive: you may place
these templates on any anonymous FTP archive. The template contains the
date on where to locate the information itself.

The templates to be used will be those constructed by the Internet
Anonymous FTP Archives working group of the Internet Engineering Task
Force. These documents describing the templates are available as Internet
Drafts from the standard network repositories. They are also available
from archive.cc.mcgill.ca in the pub/Network/iafa directory. The two
files are

draft-ietf-iafa-publish-00.txt

and

draft-ietf-iafa-templates-00.txt

the templates themselves are generally extensible and you may add fields
if you do not think that the ones provided suit your particular needs.

Please do not be put off by the size of the documents: the first contains
several examples explaining and illustrating how they are filled out; the
second provides empty templates for you to use. Also, do not feel that
you are obliged to complete one of every template or even every field in
each template. We ask only that you document current practice at your
site as completely as possible.

If you have any questions about this project or suggestions about the
templates, we would be happy to answer them. Please send mail to

        [email-masked]

Please feel free to redistribute this message to groups you think
appropriate.