Subject: Mirror copy at Villafranca: Status.

From: Jose Daniel Ponz VILSPA/VCS <[email-masked]>
Date: Mon, 19 Dec 1994 18:13:02 +0200 (CET)

Dear AstroWeb colleages,

   Thanks again for your help to set up the mirror copy at Villafranca. I 
have received all the scripts from Daniel Egret (Thanks!) and, today,
we have implemented a simple procedure (Phase 1 below), mainly based on the 
information at NRAO and ST/ScI. Please, visit us and send me your comments. 
The AstroWeb information is accessible from our Welcome page 
     http://www.vilspa.esa.es/
or can be directly addressed as 
     http://www.vilspa.esa.es/astroweb/astronomy.html

I have structured the page 'astronomy.html' in a slightly different form,
and added the HTML validation service for testing new pages. If you agree,
you could add the URL 
  http://www.vilspa.esa.es/div/help/validation-form.html 
into the AstroWeb main pages at each center, this is a service that we can 
offer to the Consortium.

In order to continue with the collaboration, I will explain our 
implementation plan, please, give me your comments.

Phase 1
-------
- Simple copy of fits.cv.nrao.edu/www/yp_*.html and 
                 lor.stsci.edu/astroweb/newmaster.html
  using lynx in a procedure defined in the cron table.
- The main page (astronomy.html) is not changed.
- URLs are periodically checked using MOMspider.
 
Question to Don and Bob: Which is the best time for data transfer?

Status: already implemented.

Phase 2
-------
- Implement local WAIS index for AstroWeb. This capability is already 
  available for our complete information system.
- Implement dead and unreliable URL lists (as seen from Villafranca).
  Comment: It could be interesting to implement this service at each center.

Status: starting soon.

Phase 3
-------
- Generate yp_*.html files from the Master Database directly, using
  a generic Perl procedure with a configuration file defining the
  categories. The file astronomy.html will also be generated.
- Merge of AstroWeb and Virtual library into a single AstroWeb Virtual 
  Library Database (?). I have contacted Sergio Paoli on this subject and 
  he is not against the idea. I will circulate a proposal in a few days.

Status: Thinking on it.

This is all for the moment.

Best regards

Daniel

---

From: [email-masked] (Don Wells)
Date: Mon, 19 Dec 94 13:38:01 EST

Jose Daniel Ponz VILSPA/VCS writes:
 > The AstroWeb information is accessible from our Welcome page ..

GOOD! 
I have revised http://fits.cv.nrao.edu/www/astroweb.html appropriately.

 > Question to Don and Bob: Which is the best time for data transfer?

My daily daemon script for fits.cv.nrao.edu runs at 05:01 Eastern time
every day. In the wintertime, Eastern time differs from UT by
5h. Previously, the AstroWeb update was the last item in the script
for the daemon; my AstroWeb page said that it is updated at about
10:45 UT. In fact, yesterday it updated at 11:11 UT. I have just
changed my daily daemon to make the AstroWeb update be the first job
for the daemon.  I have edited my AstroWeb page; now it will say
"These NRAO AstroWeb pages are updated at about 10:05 UT each day, if
the master database has changed." Note that last phrase -- for
example, my pages did *not* change today!

My AstroWeb daemon is in file://fits.cv.nrao.edu/www/astroweb/every_day.sh.
The MSSSO and ST-ECF installations are using a variation on my code.
The code fetches the master database from STScI using Lynx.
It decides whether the database has changed. You may find this logic useful.
If there has been a change, the daemon recomputes all of the pages.
The computation is invoked by the 'make install' command.
The Makefile and the AWK programs which it calls are in the same directory.

Regarding HTML validation, the new Arena client tool from CERN is
quite useful: it checks the syntax against the official SGML DTD and
reports errors in its View-Source rendering. I now have three clients
(Mosaic, Netscape and Arena) and find that testing pages with all
three is a good exercise to assure portability. 

 >   he is not against the idea. I will circulate a proposal in a few days.

GOOD! Thank you taking this action. Sergio should be invited to join
this Consortium. He should compute a version of the AstroWeb database
every day to be the Virtual Library page for Astronomy/Astrophysics.

-Don