Subject: draft of AstroWeb article for *ESO Messenger*

From: [email-masked] (Hans-Martin Adorf)
Date: Wed, 26 Oct 1994 16:30:10 +0100

% Converted from RTF format using rtf2LaTeX
% Comments and bugs to Erwin Wechtl
%                      Woertg. 2/18
%                      A-2500 Baden
%
\documentstyle{article}
\pagestyle{myheadings}
\newlength{\defaultparindent}
\setlength{\defaultparindent}{\parindent}

% definition of reference%
\newenvironment{reference}{%
\normalsize %
\setlength{\parindent}{-27pt}%
\begin{list}{ }{%
\setlength{\leftmargin}{28pt}\setlength{\rightmargin}{0pt}%
\setlength{\topsep}{0pt}\setlength{\partopsep}{0pt}}
\item }{%
\end{list}}
% definition of indented Paragraph%
\newenvironment{indented Paragraph}{%
\setlength{\parindent}{28pt}}{}
% definition of comment%
\newenvironment{comment}{}{}
% definition of affiliation%
\newenvironment{affiliation}{%
\begin{center}}{%
\end{center}}
% definition of acknowledgement%
\newenvironment{acknowledgement}{%
\normalsize }{}
% Normal will be ignored
% heading 1 will be translated as section
% heading 2 will be translated as subsection
%*****************************************************************%
\begin{document}
\title{The AstroWeb database of Internet resources}
\author{}
\maketitle

\begin{center}
{\normalsize Hans-Martin Adorf}%
{\normalsize $^{1}$}%
{\normalsize , Daniel Egret}%
{\normalsize $^{2}$}%
{\normalsize , Andr\'{e} Heck}%
{\normalsize $^{3}$}%
{\normalsize , Robert Jackson}%
{\normalsize $^{4}$}%
{\normalsize , Anton Koekemoer}%
{\normalsize $^{5}$}%
{\normalsize , Fionn Murtagh}%
{\normalsize $^{1}$}%
{\normalsize , Don Wells}%
{\normalsize $^{6}$}%
\\
{\normalsize $^{1}$}ST-ECF, %
{\normalsize $^{2}$}CDS, %
{\normalsize $^{3}$}Strasbourg Astronomical Observatory, %
{\normalsize $^{4}$}CSC/STScI, %
{\normalsize $^{5}$}MSSSO, %
{\normalsize $^{6}$}NRAO
\end{center}

\section{Introduction}
The Internet has succeeded --- and with it the World Wide Web.
This mould of hypertext information pages with embedded pictures
and hyperlinks to other local or remote information pages ---
as well as to other Internet services --- has greatly facilitated
public participation of the Shoemaker-Levy-9 comet crash (West
1994, Murtagh \& Fendt 1994). The World Wide Web, or WWW for
short, is also increasingly being used for scientific communication
and information dissemination in all natural sciences, and astronomy
is one of the most active communities in this respect.

With the success of the Internet, a mechanism better than individual
``hotlists'' for organising useful collections of astronomical
Internet resources is clearly needed. This is where the ``AstroWeb''
database (Adorf %
{\it et al. }1994, Jackson %
{\it et al.} 1994, 1995a, b) enters. It aims at being the most
complete and useful on-line collection of Uniform Resource Locators
(URLs) for astronomical Internet resources, pointing to entities
as diverse as societies, observatories, databases, preprint servers,
telescopes, telescope schedules, weather information or individual
astronomers. As such the AstroWeb database complements the recent
annotated compilation of astronomical Internet resources (Andernach,
Hanisch \& Murtagh 1994).

\section{Organisation}

\setlength{\parskip}{0pt}The AstroWeb database, which is being
maintained by the ``AstroWeb Consortium'', currently comprises
more than 1100 records of FTP, Gopher, Telnet, News, WAIS and
WWW resources. Each record minimally consists of a headline and
an associated URL. Many records are augmented by a paragraph
of descriptive text. If known, the e-mail address of the resource's
maintainer is stored along with the record. As added value, the
AstroWeb consortium has classified each record according into
one or more of about 25 categories, such as observatory, radio,
optical, infrared, telescope, database or software.

The AstroWeb database is actually a distributed system of databases
maintained at five different sites (CDS, MSSSO, NRAO, ST-ECF,
and STScI ) on three continents. The master copy is currently
kept at the Space Telescope Science Institute, Baltimore. There
the aliveness of all URLs (except of Telnet records) is verified
three times a day, and ``unreliable'' or ``dead'' URLs are flagged.
Once a day each site automatically fetches a copy from the master,
and computes a local rendition of the AstroWeb database.

Each site offers the AstroWeb database in a different style of
presentation. The ST-ECF rendition, for instance, is terse and
suitable for quick reference, whereas the STScI or NRAO renditions
are more verbose and particularly suitable for browsing. Some
sites even offer more than one rendition for the convenience
of the users.

\section{Accessing the AstroWeb database}

\setlength{\parskip}{0pt}The natural place for offering the
AstroWeb database is the WWW. The ST-ECF rendition can be accessed
either through hyperlinks originating from the ESO web
(http://www.hq.eso.org/eso-homepage.html),
or from the ST-ECF web (http://ecf.hq.eso.org/), or directly
via the ``yellow pages'' URL
http://ecf.hq.eso.org/astroweb/yp\_astro\_resources.html
(Fig. 1), from where links to the other AstroWeb renderings are
provided. The AstroWeb database is an excellent starting point
for browsing the astronomical Internet branch (Figs. 2 and 3),
as well as for finding other resources of potential interest
to astronomers.

At a size of well over 1000 records, it is mandatory that the
database is also %
{\it searchable}. To this end one may use the simple text matching
facility built into most, if not all, WWW clients. A more powerful
search mechanism, however, is offered by the AstroWeb WAIS index,
which can be queried using natural (English) language from all
AstroWeb sites. Since complete records are indexed, queries may
include categories and URLs, which are usually hidden in HTML-comments.
The results are returned as WWW-pages with ready-to-use hyperlinks
to interesting resources found.

\section{Submission of new records}

\setlength{\parskip}{0pt}The AstroWeb Consortium, consisting
of the authors above, %
{\it welcomes} submissions of new records from the astronomical
community; WWW forms have been put in place to facilitate this
process. Submissions are reviewed by Consortium members and entered
manually to ensure a minimum quality of the database. The list
of recent submissions, not yet entered into the main database,
may be viewed.

The Consortium also encourages corrections to existing records
(e.g. via e-mail to [email-masked]). AstroWeb Consortium members
can globally edit the database, in order to better respond to
change requests.

\section{Possible future steps}

\setlength{\parskip}{0pt}After a rapid growth, the AstroWeb
is currently being consolidated. It is hoped that, with the help
of the astronomical community, the degree of completeness and
quality of the AstroWeb database can be further improved. Certainly,
database internal cross-referencing within the descriptive paragraphs
is far from complete. Here the situation can presumably be remedied
with an appropriate software tool.

In the future the AstroWeb database might be used as a starting
base for indexing all astronomical WWW pages out on the Internet.
This would involve a ``robot'' repeatedly fetching all relevant
HTML pages, similarly to how Archie indexes all files in all
registered  anonymous FTP servers. Such a project is by no means
out of question. In fact, general indices spanning the whole
WWW already exist, but none thus far is specific to astronomy.

\section{References}

\setlength{\parskip}{0pt}%
\begin{reference}
Adorf, H.-M., Egret, D., Heck, A., Jackson, R., Koekemoer, A.,
Murtagh, F., Wells, D.: 1994, ``AstroWeb --- Internet resources
for astronomers'', %
{\it ST-ECF Newsl. }%
{\bf 22}, (in press).

Andernach, H., Hanisch, R.J., Murtagh, F.D.: 1994, ``Network
resources for astronomers'', %
{\it Publ. Astron. Soc. Pacific, }(in press).

Jackson, R., Wells, D., Adorf, H.-M., Egret, D., Heck, A., Koekemoer,
A., Murtagh, F.: 1994, ``The AstroWeb Database'', %
{\it Bull. CDS }%
{\bf 45}, 21---25.

Jackson, R., Wells, D., Adorf, H.-M., Egret, D., Heck, A., Koekemoer,
A., Murtagh, F.: 1995a, ``AstroWeb --- A database of links to
astronomy resources --- Announcement of a database'', %
{\it Astron. Astrophys. Suppl., }(in press).

Jackson, R.E., Adorf, H.-M., Egret, D., Heck, A., Koekemoer,
A., Murtagh, F., Wells, D.: 1995b, ``AstroWeb --- Internet resources
for astronomers'', in: Proc. %
{\it Astronomical Data Analysis Software and Systems (ADASS)
'94,} Baltimore, MD, D. Shaw, H. Payne, and J. Hayes (eds.),
Astronomical Society of the Pacific, (in press).

Murtagh, F., Fendt, M.: 1994, ``The SL-9/ESO Web Encounter'',
ESO Messenger %
{\bf 77}, 47.

West, R.: 1994, ``Comet Shoemaker-Levy 9 Collides with Jupiter'',
ESO Messenger %
{\bf 77}, 28---31.%
\end{reference}

\setlength{\parindent}{\defaultparindent}%
\begin{reference}
\end{reference}

\section{Figures}

\setlength{\parskip}{0pt}Fig. 1: View of the top-level page
of the AstroWeb database at the ST-ECF. Hyperlinks to forms for
entering new records, and for error corrections are provided.

Fig. 2: The list of ``telescope schedules'' accessible from the
AstroWeb information pages.

Fig. 3: Portion of the ``abstracts'' resource listing in the
AstroWeb database.

\end{document}