The XTE Data Finder:
How to Get your Own Virtual Copy of a Large Database

A.H.Rots
USRA-NASA/GSFC

K.C.Hilldrup
HSTX-NASA/GSFC

Session ID: T7.05   Type: oral

Abstract:

The mission database for the Rossi X-ray Timing Explorer (RXTE or XTE) is implemented as a hierarchical collection of FITS files. All data files can be accessed through a two-level system of index tables. The Master Index contains one record for each observation with pointers to subsystem index tables. Each observation's subsystem index tables contain pointers to the actual data tables which hold all information extracted from the telemetry in FITS format. With the exception of ASCII files containing the SHA (Secure Hash Algorithm) message digests of the data files, all database contents is stored in FITS binary tables and all files are checksummed using the proposed FITS checksum convention. All data files in the mission archive are gzipped. The XTE Data Finder (XDF) is a graphical tool that allows the user to navigate the database and select data files of interest; it is interfaced for the XTE-GOF's data processing and analysis tools. It is built using Tcl/Tk, object-oriented extensions, and some C code. It is given the path to a Master Index file and from there can find its way through the system of observations and subsystem index tables. In order to pinpoint the desired data file(s), it allows the user to select by source name, time range, observation, spacecraft subsystem, data source, and data mode. It works equally well on the entire mission database, or databases that just contain a single proposal's or even a single observation's data. The crucial issue is that XDF only needs the index files in order to find the appropriate data files, not the data files themselves. We have added a user-selectable option to the final selection step that will check for the existence of the selected data files in the user's local file system. If this check is negative, XDF will transfer the missing files from the XTE public archive through FTP. Thus, the user only needs to retrieve the two levels of index files and can analyze any data from the RXTE public archive as if (s)he had the whole mission database available on a local file system - except, of course, for the inevitable data transfer times. In the future, we intend to extend this transparent direct access to a ``virtual'' local mission database to include the index tables as well; XDF will then take care of the retrieval of the index files, too.





Patrick P. Murphy
Tue Sep 10 22:23:59 EDT 1996