Subject: and Daniel inquired...

From: Bob Jackson <[email-masked]>
Date: Tue, 22 Aug 95 10:02:31 EDT

Hi Bob,
    I have been testing the database update functions in tastroweb-cvs.html
It seems to work well for the few cases I tried. 

As remarks:
1) It will be interesting to include in tastroweb-cvs.html a short 
description of the new update procedure, indicating when and how are the 
updates applied to the master database.

>>> Currently I have a CRON script which would update the master
>>> database every hour from the files created or modified by
>>> this tool
>>> Is once per hour sufficient?

2) It is not clear to me how are you planning to handle the list of 
pending URLs with this new procedure... this could be also described on the 
page.

>>> 'Somebody' gets to enter them.
>>> I think Don has some ideas?

3) The log of Database changes for a session could be improved...    

>>> There is no 'session' log.
>>> I an just using the naive 'cvs log <filename>' output.
>>> It would be possible to parse the output and generate
>>> the DIFF's for each change
>>> How useful would this be?

4) What kind of quality control tests do you apply to the modified/added 
records ?  (duplicated entries, availability of the URL, ...)

>>> Only the uniqueness of the URL is checked.
>>> The availability is checked by other tools.
>>> This checking can be slow and has a non-zero
>>> false negative rate.

---

From: [email-masked] (Don Wells)
Date: Tue, 22 Aug 1995 10:56:32 -0400

"BJ" == Bob Jackson <[email-masked]> writes:
BJ> 2) It is not clear to me how are you planning to handle the list of 
BJ> pending URLs with this new procedure... 
>>>> I think Don has some ideas?

Yes, I do. 

I have long wanted to modify my CGI program which processes the record
entry forms so that it would compute separate forms which would update
the master database. I still intend to do this. In fact, I worked on
the problem in March, and built some experimental prototype code
(files aref95*.* and inp_incoming95.* and the inp_incoming95/
subdirectory in my astroweb directory).  Bob's new procedures make my
ideas even more attractive, and I now intend to work on the code again
and try to make a production version.

The general idea is that our user community would fill out a form as
they do now, but the information would be re-formatted into another
HTML form ready to execute an ADD operation on the master
database. This new form would have user/password fields which one of
us would fill in in order to authorize the transaction. One little
detail is how to authorize deleting the form file on my machine when
the operation is finished; for this purpose a "verify password"
command in Bob's procedure would be useful to me (of course, I could
get the effect of it by trying to read the logfile using the
password).  Probably I would compute an incoming HTML page (and
possibly an index page) similar to the present one, but I would do it
by concatenating information from the pending input forms. The
prototype of this feature is:
http://fits.cv.nrao.edu/www/astroweb/inp_incoming95/index.html

It is my intention to transform the existing input records to the new
format to test this new technology, and to facilitate processing those
records. We have 58 kilobytes in inp_incoming.html currently...

This whole concept leads to the additional idea of automating
corrections to the database.  Currently the users send us change
requests via Email using our form. It would be better if they edited
their own records themselves, directly. I have in mind providing a
scheme for them to fetch their record and edit it in a form
themselves, and then submit the MODIFY form to us for
authorization. This would mean that any of us could authorize the
change, and would be able to edit the record before making the
transaction, but the users would do all the heavy lifting and would be
able to mostly control their own records. If this mechanism can be
made to work, it would then encourage me to construct a tool to send
Email messages to OWNERs suggesting that they review their records and
submit modified versions.

-Don