Imaging Requirements for the ALMA M. Holdaway 22 Feb 2002 DRAFT 13.1 Observing Modes Imaging requirements for ALMA is a complicated topic which is partially covered in the Calibration, Computing, and Configuration chapters. We will try to minimize the overlap with these other areas, but will have to repeat some information for clarity. ALMA's imaging is complicated by the diversity of observing modes which ALMA is designed to perform, the physical limitations of the antennas as we push to extremely high frequencies, and the extraodinarilly high sensitivity of the array. A low sensitivity array may suffer from the physical limitations of the antennas, but a high sensitivity array such as ALMA will have the SNR to actually fix some of these problems. Also, ALMA's imaging must be understood in unprecedented detail because of the requirements of dynamic scheduling and pipeline imaging. In the past, these demands have been glossed over, and we must look at the imaging requirements with a new seriousness. Consider the subsystems of conceptual bodies of Observing Modes, Proposal Submission Materials (or tools), Data Simulator, Current Environmental Data, Dynamic Scheduler, On-Line Computing, and the Imaging Pipeline. We are mainly concerned with certain interactions between these bodies, and observing modes will be fundamental in framing these interactions, as we need to be able to reduce data from all supported observing modes. ALMA will have a number of observing modes: interferometric single field interferometric multi-field interferometric fast switching pointing measurement total power single pointing frequency switching position switching subreflector nutation on-the-fly (OTF) ... and combinations of the above modes pointing measurement interferometric plus total power (mosaicing) interferometric plus dedicated total power data homogeneous array mosaicing (total power data taken with the full interferometric array) OTF mosaicing interferometric plus ACA plus dedicated total power data holography interferometric single dish (interferometry with reference reciever) VLBI Some of these observing modes require nothing of the imaging algorithms. For example, data taken in pointing measurement mode will be treated through calibration, and will usually not require any special attention from the imaging algorithms. Additionally, there are a number of data collection modes which might distinguish several observations with the same observing mode: continuum, spectral line, polarization, pulsar gating, phased array, etc. The data collection modes correspond broadly with correlator modes, as the correlator is the primary backend device for filtering and manipulating which data are to be collected. Finally, there are a number of reduction methods. There should be one or more reduction methods for each reasonable (ie, desirable) combination of data collection mode and observing mode. There will be a number of levels of support for reduction methods: unsupported (ie, the algorithm has not been implemented in ALMA software), beta support (ie, the algorithm works, but may need scientific supervision to proceed), and pipeline support (the algorithm is understood sufficiently well to be automated). Let us say the combination of observing mode, data collection mode, and reduction method corresponds to an "observing intent". Two different observations may have identical observing modes and data collection modes, but if one observation is a long, thermal noise-limited, detection observation while the other is a dynamic range-limited observation of a bright source, they present very different requirements to the ALMA scheduling system and reduction system, and may have different reduction methods associated with them. Two observations with different "observing intent" might still be imaged with the same reduction method. Conversely, two observations with the same "observing intent" might be imaged with different algorithms. 13.2 High Level Statement of ALMA Imaging Requirements A top-level statement of the imaging requirements for ALMA are: - to borrow, optimize, or generate, imaging algorithms for each reasonable "observing intent", and to elevate the robustness and understanding of each such algorithm to the level of pipeline support in a manner which reflects both the demand for that algorithm and the difficulty of the implementation of that algorithm. It is understood that the set of "reasonable observing intents" may evolve with time. - to identify interactions between the imaging algorithms' performance and other aspects of the ALMA system, including the proposal process, the simulator, the scheduler, the site environment, and the imaging pipeline. - to quantify inherent limitations on the various imaging algorithms, and the relationships between the site's environmental data and the imaging algorithm's success. For example, there will be a correlation between wind and antenna pointing, and also correlations between antenna pointing and image quality for different observing modes. The scheduler will need to have a clear statement of the relationship between environmental data and potential image quality to make good choices with sheduling. 13.3 Relationships Between Observing Modes (Observing Intents) and Other Subsystems The subsystems of Proposal Submission, Simulator, Environmental Data, Scheduler, On-Line System, and Pipeline rely upon the observing modes for specific content and instruction, and their purpose is to fulfill the goals of the observing intent. Most of the interactions of these subsystems is outside the scope of imaging, but the interactions of these subsystems with the observing modes is central to imaging. The current capabilities of the ALMA will be clearly identified and advertised in the Proposal Submission subsystem. We need to state which observing modes and intents are supported by the simulator, the on-line system, the automatic scheduling system, the off-line reduction software, and the automated pipeline imaging. Furthermore, the Proposal Submission system must know the various parameters which are required for each observing mode or intent. Many parameters will be common to nearly all observing modes, but some will be more specific. For example, pointing errors are usually of little concern for single pointing obervations, but may become an important limitation for multi-field observations, and a pointing error limit parameter would need to be included for mosaicing modes. As alluded to above, the simulator should understand the relevant details of each supported observing mode or intent. As the simulator matures, it will also be able to simulate the most likely limiting errors for a given observing mode or intent. One of the most important areas of interactions between the observing intents and the rest of the ALMA computing system is the estimation of quality of an observation which is about to be performed, given the current environmental site data. This is the calculation which the dynamic scheduler must perform constantly to determine the optimal use of the array. This area of interactions can be summarized by a list of quasi-analytical, numerical, or purely empiracle relationships between envirmonmental conditions and the final image quality. For example, a very simple relationship might be high SNR mosaic observations above a certain frequency are never performed during daytime hours (ie, when solar radiation and strong winds will cause adverse surface, voltage pattern, and pointing effects, thereby decreasing the quality of the mosaiced images). However, such a simple rule (no daytime mosaics at high frequency) could be fine tuned a great deal. For example, a detection experiment may not require very high image quality, and may tolerate some day time conditions. Or more quantitatively, we might find that when the solar flux is less than some amount and there is no solar shadowing of antennas by other antennas, we can make moderate quality images up to a certain observing frequency. Finally, we may have some sort of correction algorithm which improves the image quality further, or pushes the maximum recommended observing frequency higher for a given set of environmental conditions (see Image-Plane Effects below). So, one of our tasks is to compile the set of all such relevant rules which relate the environmental conditions and the observing mode to an estimated image quality. These rules must be compiled for all observing intents to be considered by the dynamic scheduler. Initially, many of these rules will be rough, ad hoc rules, based on intuitive understanding of the behavior of the array and the imaging algorithms. However, experience on the array, especially experience gleaned from analysis of test data which has been correlated with environmental conditions, and extensive simulations of the various observing modes and either environmental conditions or antenna errors derived from the environmental conditions, will be very useful in formulating improved rules. Obviously, this is an area of exploration which must begin now, but will be ongoing through the life of ALMA. Of course, there is an obvious relationship between the observing intents and the off-line reduction system or the Pipeline system. Each mode and intent needs to be supported with off-line reduction software, and will eventually be supported in an automated manner through the imaging pipeline. The observing intent needs to be present, either explicitly by some category name, or implicitly embedded in information about the required image quality, the source SNR, and the correlator setup, at the time of observation so the imaging pipeline can sensibly image the data. Likewise, each observing mode needs to be archivable. In addition to the usually archived information, we will also need to archive any information available on the intent of the observation. Data from the archive will at times be reconstituted and passed through the imaging pipeline, so there needs to be some sort of persistence of observing intent, yet also some flexibility so that the pipeline recognizes a superior algorithm might be used which was not available at the time of the archived observations. 13.4 Observing Intents The following attempts to be a complete list of observing intents for ALMA. Each broad observing category is followed by a number of lines, each with an exclusive switch such as [ continuum | spectral line ]. A large number of possible observing intents follows from the outer product of all the exclusive switches. Many of the observing intents under a given observing category might be imaged with the same algroithms. Some intents will require specialized algorithms either for higher efficiency (when low SNR justifies an approximation) or for high imaging accuracy (when high SNR justifies a more stringent approach and various errors must be handled explicitly). At this time, no effort has been made to prioritize the support for the various intents. Intents required for ALMA commissioning would be top priority, followed by common observing modes, intents which can be split off and imaged more efficiently, and finally the most demanding intents which require new imaging algorithms. - total power point detection: [ Beam Switched | Frequency Switched | Position Switched ] [ continuum | spectral line ] [ total intensity | polarization ] - total power imaging: [ OTF | point & shoot ] [ Beam Switched | Frequency Switched | No Switching ] [ continuum | spectral line ] [ total intensity | polarization ] [ high SNR | low SNR] For high SNR total power imaging, effects such as pointing errors and beam irregularities might be accounted for. - compact single field For compact single field imaging, a single region in the primary beam (presumably the center) is being sampled. Hence, image-plane effects such as pointing errors and beam errors become negligible or manifest themselves as visibility-based errors which can be corrected by self-calibration. [ continuum | spectral line ] [ total intensity | polarization ] [ high SNR | moderate SNR | low SNR] Low SNR observations require Fourier inversion, but no deconvolution. Moderate SNR observations require deconvolution. High SNR observations require iterative rounds of deconvolution and self-calibration. - non-compact single field While a single field is sufficient to image the target source, the object is spread over the primary beam. High SNR observations may require attention to image-plane effects. Polarization observations may require attention to the polarization beam, another image-plane effect. [ continuum | spectral line ] [ total intensity | polarization ] [ high SNR | moderate SNR | low SNR] Low SNR observations require Fourier inversion, but no deconvolution. Moderate SNR observations require deconvolution. High SNR observations require iterative rounds of deconvolution and treatment of image-plane effects and/or self-calibration. - multi-field observations of many compact sources Interferometric observations alone can accurately represent the source structure, or the extended source structure is being ignored, so total power measurements are not required. High SNR observations may require attention to image-plane effects. Separately deconvolved images of each field, combined into a single mosaic image after deconvolution, will reduce the magnitude of many image-plane effects. Holography observations are a special case here (a multi-field observation of a single compact source). Since there is only one source, the image-plane effects reduce to visibility-based effects and can be used to solve for the image-plane errors. [ point & shoot | OTF ] [ continuum | spectral line ] [ total intensity | polarization ] [ high SNR | moderate SNR | low SNR] Low SNR observations require Fourier inversion, but no deconvolution. Moderate SNR observations require deconvolution (presumably prior to combination into a single mosaiced image). High SNR observations require iterative rounds of deconvolution and treatment of image-plane effects and/or self-calibration. - multi-field observations of extended sources Interferometer and total power data must be combined to produce a single image. In some cases, the total power may be collected with the all dishes and at the same time as the interferometric data (homogeneous array). As nutators will be present on only four antennas, continuum observations probably cannot be performed with the homogeneous array concept. The four antennas with nutators will form a dedicated total power subarray, which will likely operate independently of the rest of the array, observing the target source at a different time than the main array. The ACA, another means of measuring short spacings, is not currently budgeted, but is listed for completeness. [ dedicated total power | homogeneous array | ACA & dedicated total power] [ point & shoot | OTF ] [ continuum | spectral line ] [ total intensity | polarization ] [ high SNR | moderate SNR | low SNR] Low SNR observations can use linear mosaicing with a single approximate deconvolution. Moderate SNR observations require full deconvolution. High SNR observations require treatment of image-plane effects, dealing with both the interferometric and total power data. 13.5 What Imaging Algorithms Do We Need? COMPLETE THIS Total Power Imaging Single Field Imaging -reduction of single field interferometer data. Many different algorithms exist to accomplish this, and it is pretty well understood. There will be more work in this area when ALMA comes on line, but we can't realy predict what this work will be. Much of the single field interferometer data will result in thermal noise-limited images. These images will have signal to noise ratios (SNR) ranging from tens to one to hundreds to one. If this SNR is less than the ratio of point spread function (PSF) peak to maximum sidelobe ratio, the image may not require deconvolution. One issue to be aware of in undeconvolved images which are limited by thermal noise is that the potential SNR will likely be much greater on shorter baselines or at low resolution, but that the PSF sidelobe level may not be similarly improved at that resolution. In this case, the undeconvolved images would be limited on the large scales by failure to deconvolve. This is an important reason for multi-scale PSF optimization with respect to both weighting and array design. -weighting (also applies to multi-field): The Fourier plane coverage for the various configurations of ALMA is exceptionally good, resulting in a nicely shaped PSF with sidelobes of only a few percent. Furthermore, a simple reweighting of the data which gives an optimally Gaussian PSF and sidelobes of a few tenths of a percent will only cost on the order of 10\% of the array's sensitivity (Boone, 2002). This indicates that many observations will not require any deconvolution at all. We should embark upon a study of such reweighted but undeconvolved images to determine the nature of their errors and to know under what conditions deconvolution will improve the images (for example, if the source size is equal to or greater than the recirpocal of the shortest measured spacing, deconvolution could greatly increase the recovered flux, generally imroving the image quality). -fast switching (also applies to multi-field); while the imaging system is not primarily concerned with fast switching (it is a calibration technique), the fast switching can result in some baseline-dependent decorrelation leading to imaging sources which appear to be a bit more resolved than they actually are. A simple correction algorithm is suggested by the fast switching method. The phase detection carried out on the calibrator will provide us with the statistics of the residual phase errors on each baseline. From this, we can estimate the decorrelation each baseline experiences on the target source and divide the visibility amplitude by the decorrelation factor. If this is done, the noise will increase on the longest baselines, so we should also explicitly downweight the visibilities by a commensurate amount, leaving the net visibility amplitude times the weight unchanged. However, the beam's weights are also changed, which will match the beam to the manner in which point sources are incorrectly resolved by the baseline-dependent decorrelation. Hence, baseline-dependent decorrelation can be handled by simply adjusting the weights for the calculation of the PSF, and not adjusting the weights or visibilities that go into the dirty image calculation at all. Upon deconvolution with the reweighted PSF, point sources will be recovered where true point sources are located. This is a topic for more work. Multi-Field Imaging and Other Wide Field Issues -linear mosaicing: Just as moderate dynamic range single-field observations can sometimes forego deconvolution, multi-field observations can also make some shortcuts. The design of the most compact ALMA configuration (ie, the ``mosaicing configuration'') is driven mainly by the need for high surface brightness sensitivity, and the antennas are so close together that there is not a great deal of room for sidelobe minimization and beam shaping, but the compact configuration's PSF will still be pretty good. A first order mosaic image would be the linear combination of the dirty images. This option, and it's implications for total power, need further exploration; the homogeneous array's dirty images suffer from the zero-spacing problem (ie, negative bowels masking extended structure), and require deconvolution to reconstruct the large scale emission. As mosaicing will be one of the most CPU-intensive imaging pipeline activities, a great easing of the Pipeline's computational requirements would result if a significant fraction of observations could utilize linear mosaicing without deconvolution. Another option for a quick mosaic, which takes more CPU time, but which has been demonstrated to work for homogeneous array data, is to deconvolve the entire linear dirty mosaic image for the effects of a single mean PSF (Cornwell, Holdaway, and Uson, 1994; AIPS++ User's Manual, 2002). In reality, each pointing will have a slightly different Fourier plane coverage and a different PSF. However, if the mosaic observation was made wisely, each pointing will have a very similar PSF, and the main effects of the PSF on the image can be removed by deconvolving the mean or effective PSF from the linear mosaic of the dirty images. This deconvolution strategy can be applied to objects with potential SNR of several hundred to one, and it too represents a significant savings over a full non-linear mosaic. Several different deconvolution methods could be used at this point, including MEM, clean, multi-scale clean, and, if complete effective Fourier plane coverage is achieved (as it always in in the ALMA compact configuration), Wiener filtering or linear deconvolution. -non-linear mosaicing: Moderate and high SNR mosaics will require a more exact treatment of the differences in (u,v) coverage among each pointing. Effective non-linear mosaicing algorithms have been implemented in SDE, Miriad, Gildas, and AIP++. Deconvolution methods in non-linear mosaicing algorithms include MEM, maximum emptiness, clean, and multi-scale clean. The algorithmic limitations of non-linear mosaicing are generally beyond the SNR expected of most ALMA observations. However, even with a bright source and low thermal noise in the ALMA system (ie, a high potential SNR), there are a host of image plane effects which could otherwise limit the image quality of the mosaic images. The following sections detail some of these effects and propose a possible solution to move beyond these limitations. -wide field (image plane) errors that ALMA must be concerned with COMPLETE THIS * pointing COMPLETE THIS * surface errors and other miscelaneous voltage pattern errors * illumination allignment; The tapered illumination of the primary dish by the feed will not be perfectly centered on the primary. The shift in the aperture illumination corresponds to a phase gradient in the sky voltage pattern. If not corrected, this voltage pattern error could severely limit mosaics at frequencies up to several hundred GHz. However, the shift in aperture illumination is almost totally equivalent to a shift in the (u,v) coordinates which rotates as the parallactic angle. An excellent fix to this problem has been demonstrated (Holdaway, 2001), and it is handled in the calibration chapter. *polarization: On-axis polarization issues are treated in the calibration chapter. Wide-field polarization imaging (ie, non-compact single field or multi-field observations of extended sources) will be affected by the impurity of the voltage patterns. The off axis feeds of two supposedly orthogonal result in non-orthogonal voltage patterns, which is equivalent to an instrumental polarization which changes with position in the beam. Like the instrumental polarization, this polarization beam rotates with parallactic angle. If the voltage patterns are known, and we can generate a good model of the total intensity image, we can calculate the spurious polarization signal for each measured visibility and subtract it from the data visibilities. A simple image-plane version of this problem has been implemented for snapshot observation on the VLA (Cotton, 1995). More on this sort of correction algorithm is found in the section on image-plane errors below. -an approach to image-plane errors in AIPS++ At high frequencies, ALMA mosaic images will be limited by pointing errors and surface errors. We expect that there will be many cases where the source will be bright enough and ALMA will be sensitive enough to solve for these errors and push beyond this dynamic range limit into a new regime of high SNR mosaicing. However, it is unclear how much success we might expect from this direction of inquiry. Image plane errors, such as pointing errors, destroy the nice conceptual break between visibility data and images. A pointing error is a calibration parameter, yet a correction for a pointing error cannot be applied to the visibilities for a non-compact source. However, given a model brightness distribution and the form of the image-plane error, it is straightforward to compute the visibility. Pointing errors and beam errors are essentially direction dependent gains. A general solution to the problem of direction-dependent gains is quite demanding. The design of the AIPS++ system has the concept of such direction-dependent gains built in. Furthermore, the possibility exists of fixing the image-plane errors for just the brightest sources, which, if uncorrected, would scatter the most flux, thereby limiting the dynamic range and obscuring fainter emission. Hence, the proposed strategy for image-plane problems is: 1) identify the bright problem sources which would limit the imaging, 2) perform a cycle of imaging, collecting "flux components" down to the level where we are beginning to become corrputed by scattered spurious emission, 3) perform a sort of direction-dependent self-cal in these directions, comparing the data visibilities to the model visibilities to solve for the parameters image plane errors, 4) calculate the model visibilities implied by the source distribution revealed so far, augmented by the solved image-plane error parameters, 5) subtract the model visibilities from the data visibilities 6) continue with step 2), performing the next cycle of imaging of weaker emission. This proposed algorithm is new and uncertain territory, and should be prototyped as soon as possible. -wide field errors that we probably don't have to worry about: * focus errors; Focus errors will introduce a concentric phase error pattern in the illumination voltage, which results in phase errors in the sky voltage pattern. If two antennas have the same focus error, this effect cancels as that baseline's primary beam is formed by multiplying one sky voltage pattern by the complex conjugate of the other's voltage pattern. Focus errors may help confuse other sorts of errors, such as pointing errors, but numerical simulations indicate focus errors will not be a problem for mosaicing with ALMA. * total intensity beam asymmetries from offset feeds; as the feeds get further and further off axis, the beams are expected to become deformed and asymmetric. Deformed or asymmetric beams will rotate on the sky with parallactic angle and can be handled in the mosaicing process with a small increase in computation, as long as they remain constant with time. Calculations based on James Lamb's aperture illumination voltage calculations indicate that the 30~GHz beam will be symmetric to a very high degree, less than a fraction of a percent. * non-coplanar baselines will not be a problem. Low frequency observations of wide fields at high resolution results in the situation where the synthesized beam varies over the primary beam. This becomes an issue when the quantity \lambda B_{max} / D^2 becomes larger than 1. For 14~km baselines, 1~cm wavelength, and 12~m antennas, this diagnostic is 1.0, indicating that for the highest resolution, lowest frequency ALMA observations, the non-coplanar array will be a minor problem for complicated fields, and it will not be any sort of problem on shorter baselines or at higher frequencies. 13.5 The Simulator COMPLETE THIS! 13.6 What Image Quality-Environmental Condition Relationships Do We Need? As stated previously, this will be a growing web of logic which will start out very sketchy and will be filled in through simulations and experience on the ALMA array. These are the relationships which will drive the dynamic scheduler, and the array's efficiency will be closely tied to the utility and accuracy of these relations. However, we seek to document a starting point with a series of guesses. These relations should be redocumented here as they are adopted in the dynamic scheduling process. The first two relations do not really belong here, as they are not directly related to imaging, but more to calibration. However, in the interest of colocating all scientific equations which indicate some aspect of observational success in relation to the environmental data, we include them here as well. COMPLETE THIS noise -- opacity -- elevation, trade against (u,v) coverage phase stability pointing -- wind pointing -- anomalous refraction pointing -- thermal pointing -- errors in pointing determination frozen in until next calibration cycle beam shape -- wind beam shape -- thermal what else? These image quality - environmental data relationships are a key part of the dynamic scheduler, but we still need an algorithm for using them to choose the optimal observing project with time. Other non-environmental factors, such as calibration overhead for a project switch and the need to complete projects which have been partially observed, will also enter in to the project selection process. References AIPS++ User's Manual, 2002. Boone, 2002. Cornwell, Holdaway, and Uson, 1994. Cotton, 1995. Holdaway, 2001, Illumination Offset Memo Lamb, James, 2001, private communication (James provided an aperture voltage pattern for the 30 GHz system, and I calculated the sky voltage patterns and made inferences).