pipeline.h.tasks.restoredata package

Submodules

pipeline.h.tasks.restoredata.restoredata module

The restore data module provides a class for reimporting, reflagging, and recalibrating a subset of the ASDMs belonging to a member OUS, using pipeline flagging and calibration data products.

The basic restore data module assumes that the ASDMs, flagging, and calibration data products are on disk in the rawdata directory in the format produced by the ExportData class.

This class assumes that the required data products have been

o downloaded from the archive along with the ASDMs (not yet possible) o are sitting on disk in a form which is compatible with what is

produced by ExportData

To test these classes, register some data with the pipeline using ImportData, then execute:

import pipeline vis = [ ‘<ASDM name>’ ]

# Create a pipeline context and register some data context = pipeline.Pipeline().context inputs = pipeline.tasks.RestoreData.Inputs(context, vis=vis) task = pipeline.tasks.RestoreData(inputs) results = task.execute(dry_run=False) results.accept(context)

class pipeline.h.tasks.restoredata.restoredata.RestoreData(inputs)

Bases: pipeline.infrastructure.basetask.StandardTaskTemplate

RestoreData is the base class for restoring flagged and calibrated data produced during a previous pipeline run and archived on disk.

• Imports the selected ASDMs from rawdata

• Imports the flagversions for the selected ASDMs from ../rawdata

• Imports the calibration data for the selected ASDMs from ../rawdata

• Restores the final set of pipeline flags

• Restores the final calibration state

• Applies the calibrations

Inputs

alias of RestoreDataInputs

analyse(results)

Determine the best parameters by analysing the given jobs before returning any final jobs to execute.

Parameters

jobs (a list ofJobRequest) – the job requests generated by prepare()

Return type

Result

is_multi_vis_task = True

prepare()

Prepare job requests for execution.

Parameters

parameters – the parameters to pass through to the subclass. Refer to the implementing subclass for specific information on what these parameters are.

Return type

a class implementing Result

class pipeline.h.tasks.restoredata.restoredata.RestoreDataInputs(context, copytoraw=None, products_dir=None, rawdata_dir=None, output_dir=None, session=None, vis=None, bdfflags=None, lazy=None, asis=None, ocorr_mode=None)

Bases: pipeline.infrastructure.vdp.StandardInputs

RestoreDataInputs manages the inputs for the RestoreData task.

context

the (Context) holding all pipeline state

products_dir

the directory containing the archived pipeline flagging and calibration data products. Data products will be unpacked from this directory into rawdata_dir. Support for this parameter is not yet implemented.

rawdata_dir

the directory containing the raw data ASDM(s) and the pipeline flagging and calibration data products.

output_dir

the working directory where the restored data will be written

session

a string or list of strings containing the sessions(s) one for each vis.

vis

a string or list of strings containing the ASDM(s) to be restored.

asis

VisDependentProperty is a Python data descriptor that standardises the behaviour of pipeline Inputs properties and lets them create default values more easily.

On reading a VisDependentProperty (ie. using the dot prefix: inputs.solint), one of two things happens:

1. If a NullMarker is found - signifying that no user input has been provided - and a ‘getter’ function has been defined, the getter function will be called to provide a default value for that measurement set.

2. If a user has overridden the value (eg. inputs.solint = 123), that value will be retrieved.

3. The value, either the default from step 1 or user-provided from step 2, is run through the optional postprocess function, which gives a final opportunity to change the value depending on the state/value of other properties.

A VisDependentProperty can be made read-only by specifying ‘readonly=True’ when creating the instance.

A VisDependentProperty can be hidden from the containing Inputs string representation by setting ‘hidden=True’ when creating the instance. This will hide the property from the web log and CLI getInputs calls.

Each VisDependentProperty has a set of values it considers equivalent to null. When the user sets the VDP value to one of these null values, the VDP machinery converts this to a private NullObject marker that signifies the property is now unset, resulting in the default value being returned next time the property is read. Developers can specify which values should be converted to NullObject by specifying null_input at creation time, e.g.,

solint = @VisDependentProperty(default=5, null_input=[None, ‘’, ‘RESET’, -1])

bdfflags

VisDependentProperty is a Python data descriptor that standardises the behaviour of pipeline Inputs properties and lets them create default values more easily.

On reading a VisDependentProperty (ie. using the dot prefix: inputs.solint), one of two things happens:

1. If a NullMarker is found - signifying that no user input has been provided - and a ‘getter’ function has been defined, the getter function will be called to provide a default value for that measurement set.

2. If a user has overridden the value (eg. inputs.solint = 123), that value will be retrieved.

3. The value, either the default from step 1 or user-provided from step 2, is run through the optional postprocess function, which gives a final opportunity to change the value depending on the state/value of other properties.

A VisDependentProperty can be made read-only by specifying ‘readonly=True’ when creating the instance.

A VisDependentProperty can be hidden from the containing Inputs string representation by setting ‘hidden=True’ when creating the instance. This will hide the property from the web log and CLI getInputs calls.

Each VisDependentProperty has a set of values it considers equivalent to null. When the user sets the VDP value to one of these null values, the VDP machinery converts this to a private NullObject marker that signifies the property is now unset, resulting in the default value being returned next time the property is read. Developers can specify which values should be converted to NullObject by specifying null_input at creation time, e.g.,

solint = @VisDependentProperty(default=5, null_input=[None, ‘’, ‘RESET’, -1])

copytoraw

VisDependentProperty is a Python data descriptor that standardises the behaviour of pipeline Inputs properties and lets them create default values more easily.

On reading a VisDependentProperty (ie. using the dot prefix: inputs.solint), one of two things happens:

1. If a NullMarker is found - signifying that no user input has been provided - and a ‘getter’ function has been defined, the getter function will be called to provide a default value for that measurement set.

2. If a user has overridden the value (eg. inputs.solint = 123), that value will be retrieved.

3. The value, either the default from step 1 or user-provided from step 2, is run through the optional postprocess function, which gives a final opportunity to change the value depending on the state/value of other properties.

A VisDependentProperty can be made read-only by specifying ‘readonly=True’ when creating the instance.

A VisDependentProperty can be hidden from the containing Inputs string representation by setting ‘hidden=True’ when creating the instance. This will hide the property from the web log and CLI getInputs calls.

Each VisDependentProperty has a set of values it considers equivalent to null. When the user sets the VDP value to one of these null values, the VDP machinery converts this to a private NullObject marker that signifies the property is now unset, resulting in the default value being returned next time the property is read. Developers can specify which values should be converted to NullObject by specifying null_input at creation time, e.g.,

solint = @VisDependentProperty(default=5, null_input=[None, ‘’, ‘RESET’, -1])

lazy

VisDependentProperty is a Python data descriptor that standardises the behaviour of pipeline Inputs properties and lets them create default values more easily.

On reading a VisDependentProperty (ie. using the dot prefix: inputs.solint), one of two things happens:

1. If a NullMarker is found - signifying that no user input has been provided - and a ‘getter’ function has been defined, the getter function will be called to provide a default value for that measurement set.

2. If a user has overridden the value (eg. inputs.solint = 123), that value will be retrieved.

3. The value, either the default from step 1 or user-provided from step 2, is run through the optional postprocess function, which gives a final opportunity to change the value depending on the state/value of other properties.

A VisDependentProperty can be made read-only by specifying ‘readonly=True’ when creating the instance.

A VisDependentProperty can be hidden from the containing Inputs string representation by setting ‘hidden=True’ when creating the instance. This will hide the property from the web log and CLI getInputs calls.

Each VisDependentProperty has a set of values it considers equivalent to null. When the user sets the VDP value to one of these null values, the VDP machinery converts this to a private NullObject marker that signifies the property is now unset, resulting in the default value being returned next time the property is read. Developers can specify which values should be converted to NullObject by specifying null_input at creation time, e.g.,

solint = @VisDependentProperty(default=5, null_input=[None, ‘’, ‘RESET’, -1])

ocorr_mode

VisDependentProperty is a Python data descriptor that standardises the behaviour of pipeline Inputs properties and lets them create default values more easily.

On reading a VisDependentProperty (ie. using the dot prefix: inputs.solint), one of two things happens:

1. If a NullMarker is found - signifying that no user input has been provided - and a ‘getter’ function has been defined, the getter function will be called to provide a default value for that measurement set.

2. If a user has overridden the value (eg. inputs.solint = 123), that value will be retrieved.

3. The value, either the default from step 1 or user-provided from step 2, is run through the optional postprocess function, which gives a final opportunity to change the value depending on the state/value of other properties.

A VisDependentProperty can be made read-only by specifying ‘readonly=True’ when creating the instance.

A VisDependentProperty can be hidden from the containing Inputs string representation by setting ‘hidden=True’ when creating the instance. This will hide the property from the web log and CLI getInputs calls.

Each VisDependentProperty has a set of values it considers equivalent to null. When the user sets the VDP value to one of these null values, the VDP machinery converts this to a private NullObject marker that signifies the property is now unset, resulting in the default value being returned next time the property is read. Developers can specify which values should be converted to NullObject by specifying null_input at creation time, e.g.,

solint = @VisDependentProperty(default=5, null_input=[None, ‘’, ‘RESET’, -1])

products_dir

VisDependentProperty is a Python data descriptor that standardises the behaviour of pipeline Inputs properties and lets them create default values more easily.

On reading a VisDependentProperty (ie. using the dot prefix: inputs.solint), one of two things happens:

1. If a NullMarker is found - signifying that no user input has been provided - and a ‘getter’ function has been defined, the getter function will be called to provide a default value for that measurement set.

2. If a user has overridden the value (eg. inputs.solint = 123), that value will be retrieved.

3. The value, either the default from step 1 or user-provided from step 2, is run through the optional postprocess function, which gives a final opportunity to change the value depending on the state/value of other properties.

A VisDependentProperty can be made read-only by specifying ‘readonly=True’ when creating the instance.

A VisDependentProperty can be hidden from the containing Inputs string representation by setting ‘hidden=True’ when creating the instance. This will hide the property from the web log and CLI getInputs calls.

Each VisDependentProperty has a set of values it considers equivalent to null. When the user sets the VDP value to one of these null values, the VDP machinery converts this to a private NullObject marker that signifies the property is now unset, resulting in the default value being returned next time the property is read. Developers can specify which values should be converted to NullObject by specifying null_input at creation time, e.g.,

solint = @VisDependentProperty(default=5, null_input=[None, ‘’, ‘RESET’, -1])

rawdata_dir

VisDependentProperty is a Python data descriptor that standardises the behaviour of pipeline Inputs properties and lets them create default values more easily.

On reading a VisDependentProperty (ie. using the dot prefix: inputs.solint), one of two things happens:

1. If a NullMarker is found - signifying that no user input has been provided - and a ‘getter’ function has been defined, the getter function will be called to provide a default value for that measurement set.

2. If a user has overridden the value (eg. inputs.solint = 123), that value will be retrieved.

3. The value, either the default from step 1 or user-provided from step 2, is run through the optional postprocess function, which gives a final opportunity to change the value depending on the state/value of other properties.

A VisDependentProperty can be made read-only by specifying ‘readonly=True’ when creating the instance.

A VisDependentProperty can be hidden from the containing Inputs string representation by setting ‘hidden=True’ when creating the instance. This will hide the property from the web log and CLI getInputs calls.

Each VisDependentProperty has a set of values it considers equivalent to null. When the user sets the VDP value to one of these null values, the VDP machinery converts this to a private NullObject marker that signifies the property is now unset, resulting in the default value being returned next time the property is read. Developers can specify which values should be converted to NullObject by specifying null_input at creation time, e.g.,

solint = @VisDependentProperty(default=5, null_input=[None, ‘’, ‘RESET’, -1])

session

VisDependentProperty is a Python data descriptor that standardises the behaviour of pipeline Inputs properties and lets them create default values more easily.

On reading a VisDependentProperty (ie. using the dot prefix: inputs.solint), one of two things happens:

1. If a NullMarker is found - signifying that no user input has been provided - and a ‘getter’ function has been defined, the getter function will be called to provide a default value for that measurement set.

2. If a user has overridden the value (eg. inputs.solint = 123), that value will be retrieved.

3. The value, either the default from step 1 or user-provided from step 2, is run through the optional postprocess function, which gives a final opportunity to change the value depending on the state/value of other properties.

A VisDependentProperty can be made read-only by specifying ‘readonly=True’ when creating the instance.

A VisDependentProperty can be hidden from the containing Inputs string representation by setting ‘hidden=True’ when creating the instance. This will hide the property from the web log and CLI getInputs calls.

Each VisDependentProperty has a set of values it considers equivalent to null. When the user sets the VDP value to one of these null values, the VDP machinery converts this to a private NullObject marker that signifies the property is now unset, resulting in the default value being returned next time the property is read. Developers can specify which values should be converted to NullObject by specifying null_input at creation time, e.g.,

solint = @VisDependentProperty(default=5, null_input=[None, ‘’, ‘RESET’, -1])

class pipeline.h.tasks.restoredata.restoredata.RestoreDataResults(importdata_results=None, applycal_results=None, flagging_summaries=None, casa_version_orig=None, pipeline_version_orig=None, orig_mpi_servers=0)

Bases: pipeline.infrastructure.basetask.Results

merge_with_context(context)

Merge these results with the given context.

This method will be called during the execution of accept(). For calibration tasks, a typical implementation will register caltables with the pipeline callibrary.

At this point the result is deemed safe to merge, so no further checks on the context need be performed.

Parameters

context (Context) – the target Context

Module contents