Steve Scott's email of Thu, 18 Sep 2003 10:08:54 -0700

A few comments on data rate issues.

History
The SSR proposal to increase the datarates that was formalized in May
2002 was initiated by the feasibility study for an enhanced correlator
published in ALMA Memo #350. Some excellent science cases were also
distributed at the same time that emphasized the utility of a large
number of channels. The ASAC examined the requirements for an enhanced
correlator in their report in October 2001. These requirements were
welcomed to the extent that they could also apply to the baseline
correlator, so an SSR sub-committee re-evaluated the datarates. The
proposal (attached) was forwarded to the Computing IPT in May 2002. In
January of this year the enhancement to the baseline correlator was
proposed. The SSR requested input from the Science IPT on datarates and
a report (also attached) was sent to them as background information.


My 2 cents worth

I completely agree with the opinions that the data rates should be
science driven. That is not to say that the discussions have not been
tempered with a view to feasibility. The limiting factor is probably not
the actual archiving of the data, but the downstream image processing.
While the idea of using the DSRP is sound in theory, we do not have the
resources to utilize it in a timely way to affect the implementation
that is now underway.

Keeping in mind the ASAC's recommendations along with the capabilities
of the enhanced correlator, it seems reasonable to assume an average
number of channels of at least 8K (not including the atmospheric
pathlength corrected data). This also recognizes the opinion of the
Science IPT that the tendency will be for the astronomer to stick with
the high resolution spectra and only degrade the resolution when
required. The estimated average integration time of 10 seconds is
probably sound. The 8K channel target leads to a factor of four increase
over the existing average rate. Considerations of a factor of five or
even eight would not be unreasonable.

The recommendation for the peak data rate has been tempered by technical
considerations, which we will temporarily ignore, and examine three test
cases that ALMA should be able to handle. The first is  all 32K channels
at the "average" 10 second integration rate, recording both the
corrected and uncorrected data. The second is the average number of
channels (8K) recorded at the timescale of the atmospheric fluctuations
(1 second) for self-calibration. No pathlength correction is required
for this data. The third case is the average number of channels (8K)
recorded fast enough to avoid fringe smearing at the longest baseline.
Scaling from Barry's experience at the VLA, we get an integration time
of about 2.5 seconds for a 14 km baseline. For the peak rate we deal
with the total number of channels recorded, not necessarily the number
of channels in a single sideband (for double sideband receivers). These
three cases give the following peak rate for visibility data only:

32K/10s:   12.9 MVPS     51 MBPS
 8K/1.0s:  16.2 MVPS     65 MBPS
 8K/2.5s:  12.9 MVPS     51 MBPS

In summary, I think that we should:
1) Size the average visibility rate to handle 8K channels at 10 second
integration time.
2) Size the peak visibility rate to handle 8K channels at 1 second (the
worst case above).
3) Base the average image pixel rates on 8K channel data.

Using the equations and assumptions in the attachment and a peak 16K
channel/10 minute interval for image production gives:

              Vis                 Pix              Total
Ave     4.5MVPS/18.0MBPS    1.8MPPS/ 7.2MBPS     25.2MBPS
Peak   16.2MVPS/64.8MBPS    7.2MPPS/28.8MBPS     93.6MBPS


Cheers,
Steve