Subject: Re: ISO Data Management, Graphing, and Visualization Tools

From: Andy Glew <[email-masked]>
Date: Tue, 24 Feb 1998 17:00:27 -0600

A little while back I posted in search of Data Management, Graphing, and Data Visualization tools.

Off and on in the past week I have been evaluating such tools, reading the glossies,
running the demos, waiting for sales and technical support to tell me that their package
does not do what I hoped it would do...

I thought that I would share the results of this evaluation of existing software packages
with y'all.

$Header: /u/g/l/glew/work/data-exploration-tool-search/RCS/evaluation-notes,v 1.4 1998/02/24 22:54:10 glew Exp $

Evaluation of Data Management, Graphing, and Exploration Tools
==============================================================

BOTTOM LINE: there is no single answer, unfortunately.

SPSS seems to be the nicest Windows enabled data management, stats,
and graphing tool. However, it has distinct limitations when it comes
to data exploration; specifically, it can't do "mouse based zoom", or
any of the other nice things that make data exploration quicker.

DataDesk is the nicest interactive data exploration tool, has mouse
based zoom and a whole slew of other interactive features, but it has
only an extremely limited repertoire of graphs, plots, and charts.

JMP does more graphs than DataDesk, and is reasonably interactive, has
mouse based zoom, but not much more - but it cannot do the most common
graph types in my field (e.g. clustered bar graphs, let alone
clustered stacked bar graphs).

None of these tools seem to do the "advanced" graphs, such as
clustered stacked bar charts.

In an ideal world I'd buy SPSS and Datadesk, and also a good "batch
mode" package such as MatLAB to draw the really sophisticated stuff.

Requirements
============

    As I go through this process, I am learning more about what
    I want in a tool.

    Chart Interaction

 mouse based zoom
     draw a box around some data points on the screen,
     and have axes adjusted so that box fills full screen.
  Many (most) tools do not have a mouse based zoom,
     but instead require axis limits to be typed
     into a dialog box, or worse.
  Occasionally hides as a magnifying glasstool,
     or as lasso or point selection.

 select/query
     select points, and be told

    Chart Types
 MUST
     axis types
  {linear,log} x, {linear,log} y

 MUST

     {scatter, lines {w/wo symbols}} multiple datasets
  editable colors, symbols, linestyles, etc., assumed

     bar {vertical, horizontal}

  clustered bar
  stacked bar
  stacked clustered bar

 WANT
     ability to add comments (arrows, text, etc.) to charts
     to explain features.

 NICE
     overlay arbitrary graphs on top of each other

     3D graphs - 2D arry of bars, etc.

     thumbnails

     small multiples
  (typically of scatterplots)

 Survey: I scanned quickly over the last volume of ISCA and MICRO
     conference journals to see what sorts of graphs are in use.
     Briefly:   Micro97     ISCA97
  XY Scatter  7     6
  XY line graphs  30     103
  Simple Bar Graphs 7     2
  Clustered Bar graphs 73     38
  Cluster Cluster Bar 8     0
  Stacked Bar graphs 22     13
  Clustered Stacked Bars 20     8
  3D surface plots 2     0

    Data Management:
 MUST
     Import arbitrary textfiles in some reasonable
     format (CSV,TSV,WSV, etc.)

 WANT
     Import from a pipe, or through a program filter
     - so that can write a program to extract and reformat
     data from the database of a different program
     (and thus avoid the problems of proliferating data formats,
     temporary files, etc.)
 NICE
     Import Excel.
     OLE automation on import.

 WANT
     Ability to embed comments in a file - so that
     can track data provenance, etc.

 WANT
     No limit on points in dataset - e.g. I have prepared
     graphs with 200,000 points in the past.
     However, it is often hard to verify, from demoware,
     what the capabilities are.

    Output:
 MUST:
     EPS (Emeddable Post Script)
 OR MUST:
     Windows objects that can be embedded in Word
     (OLE automation).

 MUST:
     Finally produce Postscript viewable by Ghostview
     (for acceptance by conferences).

Products
========

Evaluated
---------

DataDesk
    Good magazine reviews

    Evaluation: crippleware demo downloaded from web

    Graph Interaction: pretty good

 MUST(OK): graphical zoom
     Seems to be the most interactive package that I have tried so far.

 Overall, has the nicest forms of interaction with the graph I have seen.
 Not only does it have mouse based zoom (box based),
 but it also has the ability to drag the graph around, changing
 i.e. changing xmin,xmax,ymin,ymax without changing the scale,
 and other interactive features.

    Graph Types: poor

 MUST(FAILS): LIMITED GRAPH TYPES
     Unfortunately, while it seems to be very interactive,
     it does not seem to have a good selection of graph/plot/chart types.
     For example, its concept of lineplots is to plot the
     variable as Y against the case number as X. I.e. it does not
     have XY lineplots.

     Similarly, Datadesk's barcharts are restricted in a similar manner,
     and understand nothing of clustering, let alone stacking, etc.

    While I might purchase DataDesk just for data exploration of scatterplots,
    I definitely cannot consider it as the answer to my graphing needs.
    Particularly not at its cost (600$ or so).

    Klugey User Interface: originally MAC based,
 not at all natural feeling in the Windows environment.

SPSS

    Cost: 175$, as a "student gradpack".

    Comments:

 SPSS 7 and 8 look reasonably good on Windows
 - much better than the old interface, which carried
 over from mainframes.

 However, still have access to the old command language
 - can run from command line, or within a procedure automation tool -
 which is good.

 All of the SPSS products (SPSS, Sigmaplot, etc.)
 seem to have good tutorials and help.

    Database
 still SPSS sequential files
 ???? unclear if can do a JOIN

    Pivoting table editor - VERY GOOD!!!

    Graphs/Charts - a reasonable variety of graphs.
 Q: can it do clustered bars, stacked bars, clustered stacked bars?

 BAD: doesn't seem to be able to do mouse based zooms.

SPSS/Deltagraph
    Cost: 295$
    Recently acquired company.

    http://www.spss.com/software/DeltaGraph/

    Comment:
 Seems to be the "PowerPoint" of PC graphing packages
 - produces sexy looking graphs, fancy backgrounds, etc.
 Seems to be recommended by many social science departments
 (psychology, demographics).

    Chart Types

                     2-D Charts Area, Area %, Bar, Bar-Stacked,
                     Bar-Floating,* Bar-Segmentation,* Bar-Stacked
                     Segmentation,* Bubble, Build-Up,*
                     Build-Up-Stacked,* Column, Column-Stacked,
                     Column-Segmentation,* Column-Stacked
                     Segmentation,* Column-Floating,* Column-XY,*
                     Combination, Contour Fill, Contour Line, Double X
                     axis, Double XY axes, Double Y axis, High-Low,
                     High Low Open Close (Candlestick and Whisker),
                     Line, Line-Filled, Line-XY, Line-Paired XY,
                     Pictograph, Pie, Pie-Donut, Pie-Multiple, Pie-Stacked,
                     Polar, Quality Control X bar r, Quality Control X bar
                     s, Quality Control p, Quality Control n, Quality
                     Control u, Quality Control c, Radar, Range, Scatter,
                     Scatter-Paired XY, Scatter (with optional droplines),
                     Spider, Step, Table, Ternary, Ternary %, Time Line,
                     Vector-Gridded, Vector-Radius/Angle, Vector-XY,
                     XYZ Contour Fill, and XYZ Contour Line.

                     Text Charts Bullet, Organization, Table.

                     3-D Charts Area, Column, Ribbon, Scatter,
                     Scatterline, Surface Fill, Surface Line, Wireframe,
                     True 3-D XYZ Surface Fill, True 3-D XYZ Surface
                     Line.

                     Statistical Charts Histogram, Ogive, Pareto, Box
                     Plot, Survival.*

     Doesn't seem to have clustered stacked bars (as are a fad
     in computer architecture papers recently).

     UNKNOWN: mouse based zoom?

    Data management

 Limit: 32,000 points per data sert

 MUST(OK): text files.
     Unclear if interaction with Excell, etc., is by
     producing files, or by OLE automation.

SAS + SAS/GRAPH
    Cost:
 no student price
 150$ departmental
 >1000$ industrial
 very complicated and particular licencing
     - even though Intel has a licence,
     I can't install it on my office machine
     because I can't access the licence server

    Comments:
 Very clunky mainframe like interface.
 Lots of typing in old command language.
 Poor tutorial and help.

    Charts:
 MUST (FAILS):
     Limited types, it seems.
     They even use line printer examples!!!!!

 MUST (FAILS) (Supposed to be OK, not verified):
     Mouse Based Zoom:
  I tried to find it, could,
  although later SAS/GRAP technical support
  said it is in
  "Edit/Graphics/Magnify Tool".

  => lousy help and tutorials if I couldn't find this!!!

SAS/JMP
    Recently acquired company
    => SAS will probably absorb its function into main SAS.

    Evaluation
 Demo: crippleware

 Poor help, no tutorial to speak of.

    Chart Interaction:
 MUST (OK): has mouse based zoom (magnify toll)
     (BUG: selecting magnify tool from right click menu is broken;
     only works when selected from toolbar)
 Nice GUI for spinning 3D

    Chart:
 MUST (FAILS):
     Limited graph types.
     MISSING clustered bars, stacked bars, etc.
     Has scattergraphs - are in help file
  - but I could not find how to make them.

XMGR
    Cost: freeware
    Chart Interaction:
 MUST (OK): mouse based zooming
    Chart:
 MUST (FAILS):
     Limited graph types
     MISSING clustered bars, etc.
  (so write them yourself...)

DEVise
    University of Wisconsin project
    Cost: freeware?

    Written in TCL/TK, hence supposedly extensible,

    Limited functionality SQL database underneath it.
    (will be extended to Oracle?)

    Chart Interaction:
 MUST (FAILS): no mouse based zoom.

    Chart:
 MUST (FAILS):
     Limited graph types.
     Very poor quality graphs.

Sigmaplot
    From SPSS
    Demo online
    NICE!!!!: Tufte macro package

    Evaluation:
 demo online

 very badly behaved demo: requires admin to install, etc.

 good help and tutorial (true for all SPSS products)

    Data Management
 NICE (OK): OLE automation
 MUST (OK): EPS output (GIFF, etc.)
 Limits: 16K columns by 64K rows

    Chart Interaction
 NICE (FAILS): doesn't seem to have "click on a point
     to query" - i.e. doesn't highlight that
     point in dataset (at least not trivially)
 MUST (FAILS): doesn't have "mose based zoom".
     Actually, has "mouse based magnify",
     but it just magnifies, doesn't redraw axes.

    Chart types:
     lots of types, including (more on web page):
     note that it has grouped bar and stacked bar
  Q: does it have grouped stacked bar?

                    2D
                        Scatter - 14 types
                        Line - 4 types
                        Scatter and Line - 10 types
                        Step - 8 types
                        Vertical Bar - 2 types
                        Horizontal Bar - 2 types
                        Vertical, Grouped Bar - 2
                        types
                        Horizontal, Grouped Bar -
                        2 types
                        Vertical, Stacked Bar
                        Horizontal, Stacked Bar
                        Box - 2 types
                        Polar - 3 types
                        Histograms - 6 types
                        Ternary - 3 types
                        Time-Series
                        Bubble
                        Pie
                        Control Charts
                        Needle
                        Quadrant
                        Population
                    3D
                        Multiple, intersecting plots
                        with hidden line removal,
                        smooth or discrete shading,
                        transparent or opaque fills,
                        and light source shading
                        3D rotation
                        Perspective preview
                        Scatter
                        Bar
                        3D line - trajectory and
                        waterfall
                        Mesh
                        Contour

S-Plus
    From Mathsoft
    Cost: 500$

    Checked out web page. Extensible embedded language.
    Website with lots of S-plus libraries.

    MUST(Fails?): mouse based zoom
 As is usual from the silly demos online, cannot tell if
 it has mouse based zoom.

    MUST(OK): clustered bars, scatter

    Mathsoft also sells:
 Axum - 200$ - graphing
 Mathcad - 130$ - misc stuff

Axum
    From Mathsoft
    Cost: 200$

    MUST(Fails?) Mouse based zoom
 As is usual from the silly demos online, cannot tell if
 it has mouse based zoom. My guess is not.

    MUST(OK): clustered bars

    Basically cannot evaluate it from its demos or literature

Statistica
    Evaluated: Statistica 5.1 demo
    MUST(FAILS?): no mouse based zoom
 Actually has a mouse based magnify,
 but it doesn't rescale or reaxis - instead
 it is just a visual magnify
    Overall feels very cluttered - not a good UI.
    Competent techsupport, though

Systat
    Cost: 185$ gradpak.
    MUST(OK-50%as mouse based zoom
 Specifically, can lasso or otherwise select points,
 and replot so that only those points are visible.

 Unfortunately, the selection tools are available only for scatterplots.

 Wasn't able to find out how to unselect data without
 reloading the whole bloody data set.
 (The demo says "Be careful to unselect, or else you
 will be restricted to the selected points.)

    Looks possible, but looks like it will have some kluges
    that make it awkward to use.

Statview
    From SPSS
    Demo in the mail

    MUST(Fails): does not have mouse based zoom
 (reported by SPSS sales)

Excel
    Microsoft

    Although everybody tells me "you can do that in Excel",
    I am reasonably certain that you cannot.

    MUST(FAILS): no mouse based zoom

    Reasonable sekection of graph types.

    Can only handle 32K data points.

dataplot
    http://www.itl.nist.gov/div898/software/dataplot
    cost: freeware?
    Recommended by net.folk

    command driven.
 somewhat powerful language
 doesn't seem to have mouse interface
    F77

    lots of nice graphs
 SEMATECH standard

Scilab
    INRIA/France
    Cost: freeware.
    Web page:  http://www-rocq.inria.fr/scilab/
    Seems to be command driven (based on reading the web page).
    Seems to be comparable to MATlab in abilities.

To be evaluated
---------------

Minitab
    Chart Interaction
 MUST (FAILS?): no mouse based zoom.

    Charts:
 MUST (OK): lots of graph types,
     language easily allows customizing,
     lots of examples in tutorial

MATLAB
    Data Management:
 MUST (FAILS?):
     Only textual import seems to be M-files,
     which are trivially related to ascii text files,
     but still not the same.
    See: Octave

Octave:
    MATLAB free clone?

IPL
    Cost: freeware?
    Charts: MUST (OK) lots of graph types
    Chart interaction: MUST (FAILS): batch mode only

IDL
    A Fortran like language advertised in IEEE magazines
    as producing lots of neat graphs.

xplot
    obsolete???

Mathematica
    Barely possible.
    Net.folk say difficult to explore data with.
    Good internal language - anything possible.
    Slow for large datasets?

Statsoft

HiQ
    Demo in the mail

S
    old stat package (from AT&T, if I remember correctly)
    haven't seen anyone using it.
    see: R

R
    S free clone

Mineset
    vendor: SGI
    Highly recommended by net.folk "could do complicated data
 exploration in only five clicks"
    cost: 20,000$
    host: SGI

JGraph
    unclear what this is.
    net.searching yields several shareware Java Graph tool references.

Other
-----

MySQL
    http://www.tcx.se
    cost: freeware?
    database

MiniSQL
    cost: freeware?
    database

---

From: [email-masked] (Michael Lyle)
Date: Wed, 25 Feb 1998 08:35:13 -0800

> I thought that I would share the results of this evaluation of existing
software packages

If you decide to roll your own, the X/Motif based XRT graph widget (and all
his cousins in the PDS suite) is a great starting point.  It meets most of
your criteria for the graphing part, and is not performance and size
limited.  You still have to do the statistics, but you could probably
interface your own stuff to one of the standard stat packages.  Using
X-Windows is no easy task, but you get exactly what you want, and in an
environment that is reliable and portable.

www.klg.com

I have no affilitation with klg, just a satisfied customer.  However, the
one or two times I had to use their tech support, they were very
unresponsive.  YMMV.

-- 
Michael
Michael Lyle ([email-masked])

---

From: Andy Glew <[email-masked]>
Date: Wed, 25 Feb 1998 12:15:48 -0600

Life is like this....

I think that I have finished my (vain) search for data management, graphing,
and exploration tools, when a new one comes to my attention, first by somebody
responding to my "final" post, and then in the form of an ad in Scientific Computing
and Automation magazine.

I just downloaded the demo for Origin 5.0 from www.microcal.com.
It passes all of the tests that I can perform on the demo with flying colours.

My "bottom line" section of my notes now reads:

Wed Feb 25 1998: NEW BOTTOM LINE: Origin 5.0 seesm to be the winner!
It has the chart interaction features I want, e.g. mouse based zoom.
It has most of the chart types I want built in.
Its extension language seems to be able to build the chart
    types that are missing - e.g. clustered bar
    - at least, such chart types appear in their gallery of examples.
It integrates well with Excel and Windows NT.
Origin 5.0 Professional appears to allow extensions,
    such as allowing to read new input file formats.
    Not clear, however, whether it is flexible enough to
    allow filters to be specified.

More evaluation seems to need to be done by usage. So I'm
going to buy it... (damn, it's expensive!).