Subject: Notes on the current status of AstroWeb

From: [email-masked] (anton koekemoer)
Date: Fri, 10 Jun 94 22:58:07 EST

Dear friends,

Here are some of my thoughts regarding Daniel/Andre's comments. Apologies for
not replying to each point in turn; some of the paragraphs below cover a number
of points simultaneously (while I'll leave other points for later discussion).
However the structure of Daniel's email is useful; it may be a good idea to
keep adding to that file, letting it serve as a repository for the ideas
resulting from our discussions.

Firstly, I agree that AstroWeb should have a clearly stated, well-defined
purpose, preferably one that will be able to take into account the constant
appearance of new types of services on the web. One way of addressing this
may be to talk about the question "why are we doing this?"

Personally, I spend as much time using AstroWeb as I do maintaining it; I tend
to use it to look for anything that can help with my research. One use involves
looking for information about "departments" (ie any place where researchers are
based (semi-)full-time and whose work is published in the major astronomical
journals). What I'm interested in are annual reports, research interests
(general and individual), colloquium schedules, job opportunities. Many
departments don't have much information available yet but some do. I'm also
interested in any telescopes which I could apply for time on, particularly
their instrumentation, manuals and observing schedules. In addition I use the
databases (observational and literature) and regularly check the lists of
meetings and jobs. I would not expect this to be representative of a general
usage pattern for AstroWeb, it's just an example to get the discussion going.

So as a starting point for discussion, how about the following aims:
  1) Be as comprehensive and up-to-date as possible.
      - provide information of relevance and use to anyone doing research in
        astrophysics, or involved with supporting it
      - provide a forum where departments/individuals can publicise their
        activities in the knowledge that it will be seen as widely as possible.
  2) Present the information in a number of different ways. It seems to me that
     there are 3 general presentation strategies, all of which are useful and
     should be kept:
      - search mechanisms (Bob, Daniel/Andre)
      - classify by access protocol (Bob) - possibly sub-classify by service
      - classify by service (my list, Don's and Hans-Martin)

In order to achieve (1) we need to actively encourage the community to
contribute to AstroWeb - since advertising their activities is ultimately of
mutual benefit to everyone (otherwise what is the point of the internet).
A practical way of doing this, closely related to raising the profile of
AstroWeb, may be for most sites to have some consistent way of referring to it.
We should probably provide this; something along the lines of a short paragraph
(2 - 3 sentences) explaining what AstroWeb is all about and encouraging anyone
involved in astrophysics research to contribute to it (and actually include,
for example, pointers to Don's scripts). Perhaps we should start contacting
the maintainers of home-pages, firstly those who have copied astroweb databases
and secondly those who still refer to only one of the versions, and offer them
such a "standard" paragraph. As well as perhaps pointing out the benefits of
what AstroWeb is on about. It is worth noting here that a number of sites have
done their best to give AstroWeb a high profile - eg ATNF, WebStars. But the
ability for users to contribute should be emphasised much more - (along the
lines of "AstroWeb: *the* place to be seen!" :) ). I also agree that a logo
competition would be a great way to raise its profile even further.

I like the current mechanisms for contributing to AstroWeb, whereby people
use the forms to input URLs into a specific file and these are then added to
the database at our discretion (possibly modifying them slightly for the sake
of consistency). But this will only work well (ie- be popular with the
community in general) if the changes are implemented relatively soon, within
24 hours or so of them being added. If this happens then contributions from
others may be much more forthcoming. I suggest below that any new URLs found
by us should also go into the same file.

As for keeping AstroWeb up-to-date with a minimum of effort, I'd agree that the
first step should definitely be for all sites to be brought to the same level
of completeness. This is probably best achieved by using Don's awk scripts to
create each astroweb presentation from the same master database file (currently
yp_topzone.html, although a more fundamental set of input files is inp_*.html).
As far as I know this is already the case for all sites except CDS; however it
should be quite easy to adapt the scripts so that the two CDS databases can
be created from yp_topzone.html. This would of course involve first adding to
the database all the CDS URLs which are not currently there.

Don's current system firstly uses inp_*.html to create yp_topzone.html, and
then uses yp_topzone.html to create whichever AstroWeb presentation is being
asked for in the 'make' command. My implementation of this uses the following
input files:
     23278 Jun  1 20:19 inp_cds.html
       605 Jun 10 20:32 inp_mssso.html
    221350 Jun  6 22:37 inp_nrao.html
         0 May  7 21:46 inp_stecf.html
         0 May  7 21:46 inp_stsci.html
If we all have an identical version of AstroWeb, then these files could each
become the general input file for each particular site. For example, in order
to keep my version up-to-date, I would (1) add new URLs to inp_mssso.html as
I discover them (and check/fix those that others have added to this file via
the input form), and (2) use an automatic script which regularly fetches
inp_cds.html from CDS, inp_nrao.html from NRAO, etc. and then re-makes the
database after it has fetched all the input files (re-making all the
presentations, not just the 'mssso' one). All the above is currently in place,
except for the fact that all the above input files need to be brought into
existence at the other sites. (I also can't get 'aref1' to work in order to
allow my inp_mssso.html to be changed but I probably just need some advice
here..). [The idea of each of us having an input file is simply to distribute
the load evenly; if there is only "master" input file then it does place some
extra responsibility on the person at that site, and seems to introduce some
reluctance on the part of the other members to make changes to it.]
We could also use the astroweb exploder to announce particularly interesting
new sites; however I would view this as an optional extra, the primary input
mechanism being inp_*.html.
What do you all feel about this way of keeping things up-to-date?

If we all have the same collection of AstroWeb presentations, this would
also start to reduce the importance of their physical location. One option
then is to have the same AstroWeb "home page" at each of our sites, and in that
home page refer to 2 things:
  1) links pointing to the local copies of each of the different presentations,
     but listing them by their classification details rather than their
     geographical origin.
  2) pointers to the other AstroWeb home pages, listing them by continent (with
     the institution in brackets) and pointing out that this is purely for the
     sake of network speed and convenience.
I'll make up such a draft homepage for the purposes of illustration/discussion,
although if anyone else would like to do so then please feel free.

Another question, relating perhaps to our name 'consortium': what are the
implications for involvement on the part of our institutions? In other words,
what happens when we move to other institutions? This may in fact be happening
for me in the near future, as I'll probably be leaving MSSSO to take up a
postdoc at the Institut d'Astrophysique de Paris; but it should form an
important part of our policy in general. I would expect to continue remaining
just as involved with AstroWeb, since I view it as an on-going collaboration
between ourselves, and through my discussions at MSSSO it is clear that they
don't feel any formal obligation towards either AstroWeb itself or any of the
other institutions. But the prominence of this question will be reduced
somewhat if we all have identical copies of the lists and emphasise their
different styles of classification. The "sites" will then primarily serve as
local mirrors, which could be of increasing importance in the near future if
network traffic continues to increase. I'd still believe in giving credit where
its due - but perhaps by using a separate page listing our names and current
affiliations, in a similar fashion to the title/abstract page of a paper.

Looking forward to continued discussion...

						cheers,
						 - anton.