Subject: recent modifications of the AstroWeb database

From: [email-masked] (Hans-Martin Adorf)
Date: Fri, 21 Oct 1994 10:34:26 +0100

Dear friends,

for the first time I have modified the AstroWeb database. I entered my
backlog of 40 or so records, mainly WAIS sources.

Here are my experiences:

In general, I found the AstroWeb update mechanism to be ok, but I see room
for improvements.

I chose to edit the whole database which, however, comes not in the agreed
HTML format, but in abbreviated Tcl. My 40 or so new records which I had
entered, following Don Wells' suggestion, using his entry form, however
were in the agreed-upon HTML format.

How to merge these two files? I decided not to cut and paste (again), in
fear of introducing more errors.

I conversed with Bob Jackson, who finally converted for me the new HTML
records into the abbreviated Tcl format. I am sure he used a Tcl tool, and
it would be nice to have it generally available.

I merged the two files in Emacs - and had to submit it about a dozen times,
since Bob's checker (good as it is) reports only one error at a time and
then quits. I suggest that the checker goes as far as possible and produces
an error report with multiple errors, if that is not too much effort for
Bob.

The database increased in size by 15 % (from 308 kByte by 48 kByte to 354
kByte).

I see two important improvements to be made to the database:

1. check that the descriptive text of a resource always mentions the
institute it comes from; we may ask the resource maintainers to do the
necessary modifications.

2. fully crossreference the AstroWeb database (i.e. automatically insert
anchors into the descriptions); I presume that this is not too difficult,
given that Bob has all the basic Tcl parsing tools at hand. I offer to
write the specifications and to cooperate, if that is an agreed-upon action
item within the consortium

Cheers
Hans-Martin

---

From: [email-masked] (anton koekemoer)
Date: Sun, 23 Oct 94 17:10:27 EST

Dear Friends,

Firstly, it's great to see Hans-Martin's ST-ECF version also now up-to-date;
also, thanks very much Hans-Martin for sharing all the new resources you found
(and thanks to those who helped him out). That was a truly monumental effort;
the total number of entries increased by about 70 in the space of one day!

I would suggest that the "ADD New Resource" option in Bob's astroweb Entry form
is probably the preferred way for any of us to directly enter any new resources
we find (while Don's entry form should be regarded as a buffer for additions by
the general community; it is up to us to transfer those to the database by
hand, also making modifications to them if necessary). However, while Bob's
form works perfectly well for single entries, I would tend to agree with
Hans-Martin that if one wants to add multiple entries there is probably
a little room for improvement. Certainly my current preferred mode of operating
is to accumulate new URLs and corrections over the space of a week or two,
and when I have time I use "Edit Entire Database" and add them all at once.
But this does involve hacking the {}'s, &'s etc, which is really only
of therapeutic value.

After having allowed my mind to digest this matter over the last few months
(ie every time I add new resources) here are some suggestions supplementary to
those of Hans-Martin's, if they do not involve too much effort for Bob to
implement:

1. It would be useful to be able to enter an indefinite number of newly found
resources using the "ADD" form, creating a single intermediate file containing
all the new resources, and having the option of only updating the database
itself when all entries have been added. The format of this intermediate file
should probably be either the tcl format we use when editing the entire
database, or else the original html format (or even allow this to be
specified?- I suspect that we may have different preferences anyway. If in html
than it could be internally translated to tcl when the database is updated).
It would also be good to be able to edit this file at any stage (even if all
the new resources have not yet been added) in order to fix any mistakes. Also,
when this intermediate file is finally added to the database, I'd support the
suggestion of a "log-file" of errors, and the ability to subsequently re-edit
the file and re-submit it after fixing the errors.

2. This would also solve the problem of translating entries from Don's
inp_incoming.html file into the tcl database format. One could use Bob's "ADD"
form to edit the empty intermediate file immediately, specifying html format,
and just paste the entries from inp_incoming.html into this file. Then update
the database when all entries have been pasted in (and optionally fixed).

3. Allow one to specify the comment describing the changes made, in the same
way as is currently done for "Edit Entire Database".

Also, a suggestion for the "MODIFY Existing Resource" option - have it work
much the same as "DELETE Existing Resource", except that it says "Modify>" next
to each URL instead of "Delete>". Clicking on "Modify" then brings up a little
editing window which lets you modify that particular resource. One way of
updating the database might then be to first create a new database with that
entry removed, and then use a function similar to the "Add New Resource" option
to add the modified version (of course all this should be invisible to whoever
has just clicked the "Update" button). And again, it may be useful to allow a
number of modifications to be made, storing the deleted/modified resources in
temporary buffer files, before doing the database update.

I suspect that if the above options are implemented then I'd rarely need to use
the "Edit Entire Database" option again.

						cheers,
						 - anton.