Subject: Re: Unix based FITS table browsers?

From: dwells at azalea.cx.nrao.edu (Don Wells)
Date: Sat, 14 Mar 1992 17:56:28 GMT

In article <1992Mar13.211558.5133 at rice.edu> dirk at spacsun.rice.edu
(Dirk Valk) writes:

 DV> "...   Does anyone know where I can lay hands on a UNIX based
 DV> program that will let me manipulate, search and extract subsets
 DV> of FITS tables? ..."

Dirk wants the program to be able to read FITS files directly from a
CD-ROM, without copying to disk. I am unaware of any Unix-based code
that will do this.

If someone out there wants to work on the problem, I suggest that the
best approach would be to code the application in Lisp under GNU
Emacs. The resulting application would be able to display/manipulate
FITS tables on local disk as well as CD-ROM, and would be portable to
environments ranging from DOS to VMS VAXen to Unicos Crays as well as
essentially *all* Unix boxes, because GNU Emacs implementations exist
in all of these environments, and Lisp code is portable to all of
them. The application would operate under X-windows in workstation
environments. On workstations Emacs windows can utilize a million
pixels or more, permitting examination of many columns and many rows.

The programming and window management environment of GNU Emacs is
rich, much like an operating system. There are existing applications
coded in Lisp which would be a rich source of ideas for coding the
application. 

Any takers?

--

Donald C. Wells             Associate Scientist        dwells at nrao.edu
National Radio Astronomy Observatory                   +1-804-296-0277
520 Edgemont Road                                 Fax= +1-804-296-0278
Charlottesville, Virginia 22903-2475 USA            78:31.1W, 38:02.2N

---

From: cflatter at nrao.edu (Chris Flatters)
Date: Sun, 15 Mar 1992 22:22:35 GMT

In article 92Mar14125628 at azalea.cx.nrao.edu, dwells at azalea.cx.nrao.edu (Don Wells) writes:
>Emacs. 

If the tables are ASCII tables then you should be able to knock something
together fairly easily with PERL.  If you have binary tables then both
PERL and emacs LISP are going to have problems.

A FITS table browser is an interesting idea.  I'm going to think about
that for a bit (not to be taken as a guarantee that I'll actually do anything
it).

==============================================================================
Chris Flatters				|  cflatter at nrao.edu
National Radio Astronomy Observatory	|
==============================================================================
Desperation is the raw material of drastic change --- W. S. Burroughs
==============================================================================

---

From: dirk at spacsun.rice.edu (Dirk Valk)
Date: Mon, 16 Mar 1992 16:31:31 GMT

In article <DWELLS.92Mar14125628 at azalea.cx.nrao.edu>, dwells at azalea.cx.nrao.edu (Don Wells) writes:
|> In article <1992Mar13.211558.5133 at rice.edu> dirk at spacsun.rice.edu
|> (Dirk Valk) writes:
|> 
|>  DV> "...   Does anyone know where I can lay hands on a UNIX based
|>  DV> program that will let me manipulate, search and extract subsets
|>  DV> of FITS tables? ..."
|> 
|> Dirk wants the program to be able to read FITS files directly from a
|> CD-ROM, without copying to disk. I am unaware of any Unix-based code
|> that will do this.
|> 

Is this any different than simply browsing a FITS file that isn't on CD-ROM?
When you mount the CD it's directories can be traversed like a normal file 
structure.  I was under the impression that the mounted CD 'looked' just 
like files on the main hard drive to the Sun OS.  As such, you wouldn't 
need a FITS browser that was specifically tailored for reading the CD-ROM,
right?  

Please correct me if I'm wrong -- we're a little out of my normal expertise.
-- 
: Dirk Valk             |"And if it doesn't work out,                      :
: Rice Univeristy       | There'll never be any doubt,                     :
: dirk at spacsun.rice.edu | That the pleasure was worth all the pain."       :
: Space Physics & Astro.|                         - Jimmy Buffett          :

---

From: dwells at azalea.cx.nrao.edu (Don Wells)
Date: Mon, 16 Mar 1992 18:51:40 GMT

In article <1992Mar16.163131.12036 at rice.edu> dirk at spacsun.rice.edu
	(Dirk Valk) writes:
   In article <DWELLS.92Mar14125628 at azalea.cx.nrao.edu>,
	dwells at azalea.cx.nrao.edu (Don Wells) writes:
   |> Dirk wants the program to be able to read FITS files directly from a
   |> CD-ROM, without copying to disk...
 Is this any different than simply browsing a FITS file that isn't
	on CD-ROM?

No. My wording was misleading; for "copying to disk" read "converting
to internal format on disk". As you pointed out in your original
posting, IRAF/ST-SDAS, MIDAS, AIPS, etc., have table manipulation
software that works on their respective internal table formats, and
the FITS file must be converted first. Lee Brotzmann's browser for DOS
works on FITS files directly, either on CD-ROM or disk.

--

---

From: leb at gsfc.nasa.gov (Lee E. Brotzman)
Date: Mon, 16 Mar 1992 05:12:47 GMT

I'm the author of the MS-DOS FITS Table Browser (FTB) program distributed with
the ADC CD-ROM.  I am now working on a Unix version of FTB "by popular demand".
However the work is proceeding just like the MS-DOS version -- slowly in my own
time on the PC in my basement, with occasional uploads and compiles on
NASA/GSFC Unix boxes.

FTB's source code is pretty non-platform specific.  The interface is
command-based and any commands that FTB doesn't understand internally are
simply thrown out to the OS with a system() call.  Screen manipulation was
originally done by encapsulating Borland Turbo C cursor-control calls in curses
look-alike functions, since I always planned on porting the code completely to
curses when I found a decent implementation for the PC.  Now that I have found
a good PC curses, I have replaced all of the Turbo C-specific code.

But, I have found a few little quirks with curses that have driven me to revamp
the user interface somewhat.  For instance, scrolling in curses really sucks,
especially over a modem, which is how I test the program.  I'm working now to
eliminate scrolling entirely, which isn't too hard.  Once that's out of the
way, I should be able to release a beta version 2.0. (And after I clean up the
source code so that I wouldn't be ashamed to let other people read it :-)

Unfortunately, the Unix port has taken me away from some other serious things 
on my FTB to-do list, like fixing a stupid bug in the way I skip over 
Generalized Extensions other than 'TABLE', and another more insidious bug 
caused by the Fortran origins of the TABLE field format definition.

Maybe someone out there can help me with that last one.  You see, TFORMn is
defined as a FORTRAN 77 format specification, which is fine until you try to
read floating point numbers in C.  You can read a floating point number with an
implied decimal point in Fortran with something like: 

	READ(UNIT=STRING,FMT="F9.5") FPVAL

but there is no equivalent capability in C (you can use "%9.5f" in sprintf() to
print a float, but you can't use it in sscanf() to read it, only the field
width -- the "9" in this case -- is allowed in the format specifier).  I can
try to do some pre-processing of the input numeric field so that a decimal
point exists in the proper place, but I'd really appreciate it if someone had a
generalized Fortran floating point read function in C that can handle implied
decimal points cleanly.

When I asked about this in comp.lang.c all I got was bitching about how I
shouldn't be using implied decimal points in my data, like I can control that
in a general FITS table reader.  Sheesh!

--
-- Lee E. Brotzman                    Internet:  leb at hypatia.gsfc.nasa.gov
-- Hughes STX                         DECNET:    NDADSA::BROTZMAN
-- National Space Science Data Center BITNET:    ZMLEB at SCFMVS
-- NASA Goddard Space Flight Center

---

From: pence at heawk1.gsfc.nasa.gov ( William Pence)
Date: Mon, 16 Mar 1992 17:23:19 GMT

dirk at spacsun.rice.edu (Dirk Valk) on Fri, 13 Mar 1992 21:15:58 GMT wrote:

>Does anyone know where I can lay hands on a UNIX based program that will let
>me manipulate, search and extract subsets of FITS tables?  
>
>You can sort of do it by using the IRAF package STSDAS.FITSIO.STRFITS to turn 
>the FITS file into a GEIS file, and then manipulate it with the STSDAS.TTOOLS
>package.  But this means that you end up copying the entire table onto disk 
>storage -- that's like 56Mb for some of the tables -- which negates a lot of 
>the utility of the cdrom.

The HEASARC (High Energy Astrophysics Science Archive and Research
Center) at NASA/GSFC is currently developing a set of software
utilities for manipulating FITS files (especially ASCII and Binary
tables).   I was planning to wait until this project was a little
further advanced before publicizing it, but since there seems to be
interest in this sort software it may be better to describe it sooner
to possibly avoid duplication of effort by other groups.

We are developing a suite of programs to manipulate the contents of FITS
tables.  For example, some of the more general tasks include:

      fsort    - sort the rows in a FITS table based on the contents of one
                 or more columns
      fproject - create a new FITS table from selected columns from an
                 existing FITS table
      fselect  - select rows from a table based on some logical selection
                 criteria
      fdump    - dump the contents of a FITS table to an ASCII file
      fcalc    - perform arithmetic on  table columns

The functionality of these FITS tasks is being closely modelled on the
tasks in the STSDAS TTOOLS package.  All the tasks operate directly on
FITS format files and create FITS files as output (and in a few cases,
the tasks also input or output simple ASCII files).

These utilities are being developed as part of the general HEASARC
philosophy of directly using the  FITS format as the primary data
format for archiving,  reducing and analysing data sets from past and
present high-energy astronomy missions.  These tools are being
specifically developed to process the X-Ray data that will be received
>from the Japanese/US Astro-D satellite to be launched next February,
but the tools are intended to be general enough to be useful for
manipulating any FITS format file.

Our software design philosophy is to write the main body of the code in
strict ANSI-standard Fortran-77 which is then portable to any machine
with a F77 compiler.  The FITSIO subroutine interface is used to read
and write the FITS files, which is also machine independent.  The
interface to the user (or command language) to get the input parameters
needed to run the task (e.g., the name of the input FITS table) has
also been isolated through a standard subroutine interface to make the
code more portable.  Currently we can link these tasks to 2 different
parameter interface libraries:

      1.  by linking to the F77 Interface Library for IRAF developed
          at the STScI, we produce an IRAF task.

      2.  by linking to the 'Host Interface' library developed at SAO/CfA
          we generate a standalone executable task which has the look
          and feel of an IRAF task, but in fact does not have any
          IRAF dependencies and can be envoked directly from the
          host machine operating system prompt.  (we are currently
          using a prototype version of this interface)

In both cases an IRAF-format parameter file is used as the basic mechanism
for passing parameters between the user and the tasks.

Currently we have 2+ programmers working full time on this project, and
we expect to have a significant amount of it finished later this year.
I will be giving a status report on this project at the WGAS session at
the June AAS meeting, for anyone interested in hearing more details, or
they can contact me directly.

William Pence          pence at tetra.gsfc.nasa.gov  or  HEASRC::PENCE
(301)286-4599

---

From: dwells at azalea.cx.nrao.edu (Don Wells)
Date: Tue, 17 Mar 1992 03:37:37 GMT

In article <1992Mar16.184014.21997 at draco.macsch.com>
bds at woim.id.macsch.com () writes:

"...   Enclosed is a   utility that I wrote for    my   brother.  This
program.. is an executable and runs on his sun4 and my sparc2..."

I have  placed this executable in directory FITS/OSsupport/SunOS/SPARC
on fits.cx.nrao.edu [192.33.115.8]:

  -rwxr-xr-x  1 dwells     172032 Mar 16 16:59 fits-bds
  -rw-r--r--  1 dwells      89711 Mar 16 16:59 fits-bds.Z
  -rw-r--r--  1 dwells       1336 Mar 16 16:58 fits-bds.txt

I confirm that SPARC-binary "fits-bds"  will execute  on a SPARC under
SunOS 4.1.1 and  OpenWindows-2, and that  it will  display FITS-format
images. It creates two small windows, one for a command menu (labelled
"FitsImaging v2.1") and the other a color map. It appears that it will
only  accept  files with names of the   form *.fits.  When   a file is
selected a   third window is   created for the image display.    It is
somewhat slow, even on our Sparcstation-2, but it does work. 
--