Report of the ALMA Scientific Advisory Committee: March 2002 Meeting

April 8, 2002

ALMA Scientific Advisory Committee

R. Bachiller (Spain), A. Benz (Switzerland)^{∗}, G. Blake (USA, Chair), R. Booth (Sweden), L. Bronfman

(Chile), P. Cox (France, Vice Chair), R. Crutcher (USA)^{∗}, N. Evans (USA), Y. Fukui (Japan, Lead Observer), M. Gurwell (USA)^{∗}, T. Hasegawa (Japan), D. Mardones (Chile), H. Matsuo (Japan), N. Nakai (Japan), J. Richer (UK), S. Sakamoto (Japan), P. Schilke (Germany), L. Mundy (USA)^{∗}, K. Tatematsu (Japan), M. Tsuboi (Japan), E. van Dishoeck (Netherlands), M. Walmsley (Italy), D. Wilner^{∗} (USA), C.

Wilson (Canada)^{∗}, S. Yamamoto (Japan), M. Yun (USA)

Ex-officio members

S. Guilloteau (ESO), R. Kawabe (Japan), J. Mangum (NRAO), P. Shaver (ESO), A. Wootten (NRAO)

Other participants

R. Brown (NRAO), K. Chiba (NAOJ), Y. Chikada (NAOJ), T. Cornwell (NRAO), C. Cunningham (NRC), P. Gray (NRAO), K. Kohno (NAOJ), M. Ishiguro (NAOJ), N. Kaifu (NAOJ), R. Kurz (ESO), S. Miyama

(NAOJ), K. Morita (NAOJ), T. Noguchi (NAOJ), M. Rafal (NRAO), Y. Sekimoto (NAOJ), W. Wild (NOVA/SRON)

∗ Not present at Mitaka meeting, but provided input to ASAC report.

1. Science Operations: Proposal Review and Time Accounting

Following the E-ACC and E-AEC meetings in June 2001, the ASAC was asked to consider ALMA operations from an astronomer’s point of view. As a result, an ASAC Operations Study Group was formed chaired by N. Evans, C. Wilson and Y. Fukui. The Study Group did not address technical operations issues, such as the siting of the OSF, the work schedules, etc. Instead it focused on the operational issues that might affect the scientific productivity and vitality of ALMA, looking at the questions from the point of view of the future ALMA observer.

In the initial report from the Study Group, the major recommendations on the ALMA operations centered on the role of the Regional Support Centers (RSCs) in assisting observers from the preparation of proposals through the analysis of ALMA data. During our Santiago face-to-face meeting, it became clear that certain aspects of ALMA made it rather different than other observatories, and that further deliberations by the study group and discussions by the ASAC were warranted. Here we present a summary of our March 2002 discussions on possible modes of proposal review for ALMA. In § 2, we report on our further thoughts on the roles of the RSCs. For background information, please consult the September 2001 ASAC report.

The ASAC Operations Study Group made a study of how proposal review and time allocation is done at other multi-partner observatories. They further considered the unique characteristics of ALMA, in which all observing is scheduled by a dynamic scheduler. Time cannot be “allocated” in the usual sense because the dynamic scheduler actually decides what project will be observed at a given time, based on the current weather conditions and array status. The outcome of the proposal review process is then a scientific ranking and a maximum time. In this situation, the only possible place to account for and adjust time going to different partners is in the dynamic scheduler. These considerations were agreed to by the entire ASAC, leading to our first recommendation to the ACC:

1. The ASAC recommends that time accounting and establishment of parity among the partners be separated from the proposal review process.

The distribution of requested observations over the range of weather conditions and array status conditions (encapsulated into the concept of stringency in our last report) may be very uneven for a given partner. In addition, the fraction of time during a given semester that a certain stringency will be achieved is unpredictable because of long-term weather variations. Imagine a partner that produced almost all projects with high stringency (e.g., requiring the very lowest water vapor) in a semester with bad weather (e.g., an El Niño event). A rigid requirement that the observing time for a partner adds up to a fixed fraction would lead to observations being done under inappropriate conditions, resulting in bad data and possible embarrassment for the observatory. This problem leads to our second and third recommendations:

2. The ASAC recommends that the ALMA Agreement adopt some flexibility in the precision within which and the timescale over which the time accounted to any partner must balance.

3. Alternatively, we recommend that the time accounting be weighted by the stringency so that observations under rarely-achieved conditions count more.

Between these two alternatives, the ASAC was more comfortable with the first, but if the flexibility implied by the first suggestion is not feasible, the second alternative is almost certainly needed. It would require further study to define the optimum weighting factors, which would depend on the statistics of stringency at the site. Such a study needs to be done, leading to another recommendation to the ALMA Project:

4. The ASAC recommends that the Project use existing site data and projections of performance under various conditions to assess the statistics of stringency on a range of timescales.

Stringency is defined in our previous report as t_{a}/t_{p}, where t_{a} is the total available observing time and t_{p} is the observing time during which the particular observation can be done. However, one should also think of stringency as a function that depends on opacity, seeing, pointing, etc. The value of this function, including all the dependencies, would yield the first definition.

Once we realize that time accounting is logically separate from the proposal review process, we can consider the methods for proposal review free of the need to balance time shares. That function is done by the dynamic scheduler. Thus we use the term Proposal Review Committee (PRC) rather than a Time Allocation Committee (TAC). The Study Group developed a set of principles, assumptions, and hypothetical projects and two alternative models for Proposal Review Committees, using features of existing committees, adapted to the ALMA situation. These are subject-based PRCs (with multi-partner panels), similar to the HST and ESO systems, and partner-based PRCs (with an overall International PRC), similar to the JCMT and CFHT systems. We explicitly attempted to mitigate the negative features of each of these, leading to models that would satisfy the largest possible group. Many members of the ASAC started out with strong preferences for one or the other but realized that either model could be made to work.

These models are presented in Appendices § A and § B in some detail only to reveal the issues that they raise. They are strictly “straw-person” models for the sake of discussion. The main issues that our considerations raised for the ACC to consider are these:

5. The ASAC recommends that some time (6–10%) be set aside for Director’s discretionary time, international (non-partner) proposals, and special programs such as Key or Legacy programs.

6. We recommend that policies be adopted that encourage collaborative proposals with members from different partners.

A number of the hypothetical projects involved time-critical or target-of-opportunity observations that could be handled easily only with discretionary time. In addition, we hope that some project with an impact like that of the Hubble Deep Field will be done with ALMA. The role of Director’s discretionary time in obtaining such paradigm-shifting results was recognized. We also believe that worthy projects from outside the ALMA partners should have a chance to be executed. Finally, we believe that Key or Legacy (no proprietary time) programs are important to demonstrate the power of ALMA to the larger community. We expect that such programs will combine the expertise in the different partner communities, helping to encourage collaborative proposals. Taking the time for such proposals “off the top” makes these a joint effort by the partners. However, we also seek to encourage smaller, more typical projects with members from different partners. This can be difficult in the partner-based PRC model in particular, where there may be a bias against them. While opinion was divided, we considered the idea that the time accounting method be adjusted to lower the barriers to such “mixed proposals.” If the time accounting goes partially to the partner of the co-Is, rather than all being assigned to the partner of the PI, mixed proposals can compete more fairly in a partner-based PRC. However, this means that one partner could favor a proposal that led to time being counted for another partner. In any case, the method of time accounting should be considered by the ACC; if possible, it would be good to leave some flexibility in the agreement on this point.

2. Science Operations: Regional Support Centers

In the September 2001 ASAC report, the roles of the ‘Science Operations Center’ (SOC) and the Regional Support Centers (RSCs) were discussed. It was recommended that there should be “a single SOC, operated by the ALMA observatory, where the pipeline produces and stores the official archive” and that the RSCs “should be responsible for the support of the observer, from proposal preparation through data reduction and analysis.” Further study and discussion were recommended on the core functionality of the RSCs and on the number of RSCs that are needed. The ASAC has considered the issues of the RSCs further and we report here on the conclusions based on the various discussions which were held in the partner communities.

The need for a RSC has been recognized as being a critical aspect in the success of ALMA as a means to support the the larger astronomical community and so enhance the scientific return of the project. The main problem is that interferometry represents a ‘cultural change’ for most astronomers, so that assistance is essential for any new user. Astronomers therefore need support in their own time zone, and new users need physical access to a Center. There should thus be one ALMA supported RSC for each partner community. Even experienced observers need to top-up their knowledge and talk to a team of people who are expert users of the instrument. The interaction of ‘ALMA-qualified’ astronomers with the ALMA users will also be essential to develop tools and strategies for the best use of ALMA. Therefore it is important that such a Center has close interactions with the instrument and the Operations Support Facility in Chile. The RSC should also provide easy access to the Data Archive.

These are the core functions that each RSC should have and which are under the control of the ALMA project, to be distinguished from additional functions that different partners may add. It is the functions themselves, rather than the particular means to realize them, that we discuss here; the means may differ among the partners. The core functions that we have identified, in order of importance, are:

1. Providing user support for observing proposals, data reduction beyond the pipeline products, and archival research;

2. Providing feedback from the user to the project on performance;

3. Ensuring rapid access to the ALMA Archive, either via a local copy or a fast link.

We note that the role in supporting archival research overlaps to some degree with the role of Virtual Observatories and the interaction between the RSCs and these VOs needs to be defined. Whether each RSC has a physical copy of the archive or merely provides a link is a matter of local implementation. Various communities have considered other functions to be important (e.g., financial support for the US community, software development for the Canadian community), but the core functions listed above are agreed to be the common denominator.

Other aspects, which could be supported by the communities’ own resources outside the ALMA project, (and which may even be distributed among the individual institutes) include, e.g.:

• Software development & maintenance beyond the nominal operations

• Interferometric data handling & new techniques

• Support for special projects such as public surveys with limited time-priority, legacy programs or projects which cannot be handled with the pipeline • Organization of post-docs (ALMA Fellowship), training program & interferometry schools

• Financial support to observers to help with student support, data analysis, and publication of results

These non-core functions or development activities will depend on the community. Finally, the ASAC reiterates that it is crucial that the RSCs are operated with an international and collaborative spirit leading to close interactions between the RSCs for the benefit of the astronomical community at large.

Recommendation: The core functions identified above for the RSCs should be adopted by the Project.

3. Summary

The major ASAC recommendations are summarized below. These are in the order discussed in the text and not in any priority order. More detailed recommendations can be found in the section referenced by the major recommendations.

1. The ASAC continues to consider Science Operations in detail, particularly the likely interfaces between the observing community and the observatory, and has the following major recommendations for the submission and review of proposals (see §1):

– The ASAC recommends that time accounting and establishment of parity among the partners be separated from the proposal review process, and that

– The ALMA agreement adopt some flexibility in the precision within which and the timescale over which the allocation to any partner must balance. Alternatively, time accounting could be weighted by the stringency of the proposed observations.

– Accordingly, the ASAC recommends that the Project work (with the ASAC) to use existing site data along with projections of array performance under various conditions to assess the statistics of stringency. The latter analysis should consider a wide range of timescales.

– The ASAC recommends that some time (6–10%) be set aside for Director’s discretionary time, international (non-partner) proposals, and special programs such as Key or legacy programs. We also recommend that policies be adopted that encourage collaborative proposals with members from different partners.

2. The ASAC reiterates our earlier recommendation that the Regional Support Centers (RSC) should be responsible for support of the observer, from proposal preparation through data reduction and analysis. They may also provide data portal and software development. They should be operated with an international and collaborative spirit. The ASAC recommends that the following be the core functions that are provided by each Regional Support Center (RSC):

1. Providing user support for observing proposals, data reduction beyond the pipeline products, and archival research;

2. Providing feedback from the user to the project on performance;

3. Ensuring rapid access to the ALMA Archive, either via a local copy or a fast link.

APPENDIX

A. Model for Subject-Based PRCs

General Structure: This is a model for a Subject-Based or unified ALMA Proposal Review Committee (PRC). It is based on the systems used at HST and ESO and adapted to the circumstances of ALMA. It is assumed below that Japan is an equal partner in ALMA, but the adaptation to a bilateral ALMA is straightforward. We also assume that the host nation, Chile, chooses to participate in this PRC.

We assume that there is an overall committee, the APC or ALMA Program Committee, and Subject-Based Panels. The Panels would provide a ranking of proposals within their subject area. The APC would merge these into a single set of rankings and make recommendations regarding balance between general proposals and special programs (Key or Legacy proposals, etc.). We also assume that the ALMA Director has some discretionary time. We propose that the panels and the APC meet in a location that rotates among the partners.

A.1. Composition of the Panels and the APC

We assume 4 panels, divided into subject areas (for an example, see below). Each panel would have 7 members. Four panel members should be chosen by the partners and host nation as representatives of their regions (Europe, Japan, N. America, Chile). Three members (one from each of the partners) would be added by the ALMA Director. The reason for partner representatives is to reassure the regional communities that they are fairly treated. The three additional members would ensure that all specialities are covered in a given panel. The Panel chairpersons would be members of the PRC and would be distributed among the partners and host country evenly (1 Europe, 1 North America, 1 Japan, 1 Chile). We propose 3 year terms for each panelist to balance experience with the need for new ideas and to avoid overly stressing individuals. The Panel chairpersons would rotate every year.

The APC would consist of the chairpersons of each Panel and additional members chosen by the partners (one for each partner or host nation) and additional members chosen by the ALMA Director. These extra members should have an equal say to the Panel Chairs. The members of the APC would serve 3 year terms. The final composition of the APC would then be 3 Europeans, 3 North Americans, 3 Japanese, and 2 Chileans (11 in all).

A.2. Proposed Panel Structure

We suggest for ALMA 4 panels. Based on the assumed proposal pressure (up to 500 proposals per semester seem plausible based on ALMA’s throughput and the experience with HST), each panel would review 125 proposals, close to the pain barrier for proposal review. At any one time, there would be 28 people (8 Europe, 8 N.America, 8 Japan, 4 Chile) on the ALMA panels.

The Panels could be distributed as follows:

1. Stellar Astronomy including the Sun, circumstellar envelopes, planetary nebulae.

2. Interstellar Medium, Star and Planet Formation, Solar System including planets and comets.

3. Nearby Galaxies including the Magellanic clouds and AGNs with z < 0.1.

4. Galaxies and Cosmology including high-redshift objects, the S-Z effect.

The idea behind this breakdown is to have roughly equal numbers of proposals coming to the different Panels. The subject areas should be redefined periodically based on proposal pressure.

A.3. Charge to the Panels

The Panels are charged with producing a rank-ordered list of all proposals assigned to them along with any adjustments deemed necessary to the maximum time for a project. International (no proposer from a partner) and ALMA staff proposals are assigned to the Panels in the same way as any other proposal. The Panels should not explicitly consider partner balance, but they may consider partner priorities for style of science (e.g., continuum versus line, survey versus case study, ...) as expressed by the partner representative. They may also consider balance between sub-areas within their large subject area. While they do not assign time, they may consider time requirements in their ranking and decrease the maximum time from that requested. They also consider stringency, as determined by the simulator in the proposal process.

A.4. Charge to the APC

The APC is charged with review of special programs and merging the panel rankings into a final rankordered list. Special programs include proposals for large blocks of time with no proprietary period (Legacy proposals), Key projects, and anything else that requires an overall view. They may ask the subject-based panels for scientific evaluation of such proposals, but responsibility for their placement in the overall ranking rests with the APC. They should also consider whether international proposals have been fairly dealt with on the panels. The APC is also responsible for deciding the fraction of time going to the different subject areas using the proposal pressure as a guide. They may recommend to the ALMA Director a redefinition of the areas in order to equalize proposal pressure. Finally, they are responsible for advice to the ALMA Director regarding the proposal evaluation process.

A.5. Charge to the Director

The APC reports its rankings to the ALMA Director. The Director exercises discretionary time by inserting programs into the rank-ordered list. The Director must also monitor partner parity and, if necessary, adjust the parameters of the Dynamic Scheduler in such a manner as to redress any imbalance outside the allowed range on the appropriate time scale.

A.6. Pros and Cons of a Subject-Based PRC

• Pros

1. More expertise on subject-based panels 2. Can favor multi-partner collaborations including large programs or surveys.

3. Competition leads to better proposals.

• Cons

1. Issues of partner parity and style of science can arise. The above model tries to deal with this problem partly by having a fair representation of the partners on the panels and APC and partly by empowering ALMA management to take partner parity into account in the Dynamic Scheduler.

2. The APC has to consider balance among subjects when producing its rankings and this will always be somewhat arbitrary. In the above model, the Panel Chairpersons have the responsibility of defending their area. Over and above this, we recommend periodic redefinition of areas in order to obtain roughly equal proposal pressure in all panels.

B. Model for Partner-Based PRCs

General Structure A partner-based time-allocation system is successfully used at several multinational observatories (the JCMT and CFHT being examples). We assume that North America, Europe, Japan, and Chile have separate PRCs, which would report to an International PRC (IPRC). The composition of the IPRC is not specified in this model, but it could be similar to that of the APC in the subject-based model.

In this system, the partners can choose the style of science they prefer. For example, one partner may have invested heavily in another telescope (e.g., a deep submm space survey telescope), and its community may wish to capitalize on this investment by spending a large fraction of their ALMA time following up the sources they have detected. Or they may choose to give special priority to graduate student projects, survey work, or public outreach work. These issues of scientific style have proved important in previous multi-partner observatories.

We assume the following for Partner PRCs:

1. Each partner has its own Proposal Review Committee (PRC) that weighs the scientific merits of the proposals with PI’s from that partner.

2. ALMA will utilize dynamic scheduling that uses a weighted combination of science ranking, stringency, execution status, and aggregate partner share to ascertain which project in the dynamic pool will be observed under current conditions. Usually, but not always, this will be at the project with the highest scientific ranking that has a stringency range consistent with the current stringency. The science ranking is the only input from the PRC to the dynamic scheduler.

3. The International Proposal Review Committee is needed to provide a single point of contact between the time allocation process and the ALMA Director, and to resolve conflicts between identical or similar proposals from different partners. In addition, it could handle proposals from applicants without partner affiliation, and any calls from the Director for Legacy/Key projects.

B.1. Suggested Organization

1. Well before each semester, the Director makes an estimate of expected integration time available in different stringency ranges. Director’s time, engineering time and an estimate of International time are top-sliced before partner and host country allocations are made. The partners are then given an indicative figure of how much time they are likely to receive in the coming semester, in the given stringency ranges.

2. Each partner solicits proposals from their science community, at roughly common deadlines.

3. The Partner PRCs rank scientifically all the proposals from their community. Each Partner can choose how it wishes to handle such things as Long Term proposals or student theses.

4. Ranked proposals are then forwarded to the IPRC. The IPRC attempts to identify and resolve conflicts between partners (see later), and allocates international time and Legacy time if appropriate. The IPRC then sends on recommendations to the Director, who deals with Director’s discretionary time.

5. On a slower cycle (perhaps 2-3 years) the Director will solicit Legacy/Key Project proposals from the entire community. These proposals will be ranked scientifically by the IPRC, and a certain fraction will be accepted into the project queue.

B.2. Issues still to be resolved

1. There are various natural ways of handling large Key or Legacy proposals in this scheme. Each partner could individually allocate its own large programmes depending on its own priorities. Alternatively, an agreed amount of time could be top-sliced from the ALMA time, and a special call for proposals issued by the Director periodically, with assessment done by the IPRC.

2. The mechanism by which a proposal is matched to the correct PRC is usually the PI’s affiliation. If the PI is not affiliated to a partner, any partner co-Is’ affiliations are used. Alternatively, the PI and co-I’s could be given different weights, and the PRC chosen according to this weighted share.

3. At some point, it must be decided whether mixed projects count as time from a single partner (the PI) or if time is counted against all the co-I’s partners in some way. The latter method could have the result that the European PRC could allocate some “North American” time. On the other hand, it would remove barriers to inter-partner collaborations in partner-based PRCs. In addition, it must be decided if multi-partner proposals can apply simultaneously to several PRCs, or must go to only one PRC.

4. There are several possible ways that the Director could act on the recommendations from the IPRC. For example,

(a) Model A: A sequential list of the Partners is created, with the number of entries for each Partner approximating the fraction of time each Partner is expected to receive. Each entry corresponds to a set amount of time. The top proposal from Partner A is given a rank of 1, and the expected amount of time to complete the project is used to determine how many entries for Partner A in the sequential list are “skipped” until another proposal from Partner A is ranked. Priority is then allocated sequentially by running through this list for all partners.

(b) Model B: No merging of ranked lists is done. When a observation block needs queuing up, the scheduler looks at the four lists and chooses an appropriate block, using the usual criteria of ranking, completion, stringency and also partner share to date.

5. The task of resolving overlapping observing proposals from different partners has been left to the IPRC. Where proposals are identical (i.e., observe the same galaxy with the same sensitivity and resolution), the proposal with the higher scientific ranking could simply be chosen. However, note that even if the two proposals are exactly the same, the winning proposal would be the one ranked higher by the PRC that reviewed it. This will be dependent upon the number of good proposals each PRC reviews as well as the style of science priorities that each Partner may have. This will work, but could seem quite arbitrary to the losing proposers. Where proposals are only similar (i.e., deep continuum imaging of different blank fields) the case becomes trickier. This type of overlap may also not be allowed for very well matched proposals, but could shade to allowable if, say, one group had mid-IR and VLA data to match while another had optical and X-ray data (although experts might well say one group had a better data set for comparison than the other). This type of overlap will require some human intervention to identify. Perhaps once they are identified they could again be dealt with on the basis of scientific ranking from the PRCs. The more labor intensive but perhaps more satisfying alternative is to use the IPRC to review and rank such conflicting proposals.

B.3. Pros and Cons of Partner-Based PRCs

• Pros

1. Different partners can choose different scientific priorities.

• Cons

1. Multi-partner collaborations require special handling. One suggestion to mitigate this is to count some of the time to the partner of the co-Is.

2. In some implementations of the system, duplicate observations and science may be allowed to happen. One suggestion to mitigate this problem is to give the IPRC considerable authority to decide which proposals have too much overlap. Both these mitigations tend to decrease the main advantage of this system, the freedom of partners to choose their own priorities.