pipeline.hsd.tasks.k2jycal package

Submodules

pipeline.hsd.tasks.k2jycal.display module

class pipeline.hsd.tasks.k2jycal.display.K2JyHistDisplay(stage, spw, valid_factors, bandname='')[source]

Bases: object

A display class to generate histogram of Jy/K factors

plot()[source]
pipeline.hsd.tasks.k2jycal.display.collect_dict_values(in_value)[source]

Returns a list of values in in_value. When in_value = dict(a=1, b=dict(c=2, d=4)) the method collects all values in tips of branches and returns, [1, 2, 4] When in_value is a simple number or an array, it returns a list of the number or the array

in_value: a dictionary, number or array to collect values and construct a list

pipeline.hsd.tasks.k2jycal.jyperkdbaccess module

class pipeline.hsd.tasks.k2jycal.jyperkdbaccess.ALMAJyPerKDatabaseAccessBase(context=None)[source]

Bases: object

BASE_URL = 'https://asa.alma.cl/science/jy-kelvins'
ENDPOINT_TYPE = None
access(queries)[source]
filter_jyperk(vis, factors)[source]
format_jyperk(vis, jyperk)[source]

Format given dictionary to the formatted list as below.

[[‘MS_name’, ‘antenna_name’, ‘spwid’, ‘pol string’, ‘factor’],

[‘MS_name’, ‘antenna_name’, ‘spwid’, ‘pol string’, ‘factor’], … [‘MS_name’, ‘antenna_name’, ‘spwid’, ‘pol string’, ‘factor’]]

Parameters

  • vis {str} – Name of MS

  • jyperk {dict} – Dictionary containing Jy/K factors with meta data

Returns

[list] – Formatted list of Jy/K factors

get(vis)[source]

Access Jy/K DB and return its response.

Parameters

vis {str} – Name of MS

Raises

  • urllib2.HTTPError –

  • urllib2.URLError –

Returns

[dict] – Response from the DB as a dictionary. It should contain
the following keys:

’query’ – query data ‘total’ – number of data ‘data’ – data

getJyPerK(vis)[source]

getJyPerK returns list of Jy/K conversion factors with their meta data (MS name, antenna name, spwid, and pol string).

Parameters

vis {str} – Name of MS

Returns

[list] – List of Jy/K conversion factors with meta data

get_params(vis)[source]
property url
validate(vis)[source]
class pipeline.hsd.tasks.k2jycal.jyperkdbaccess.JyPerKAbstractEndPoint(context=None)[source]

Bases: pipeline.hsd.tasks.k2jycal.jyperkdbaccess.ALMAJyPerKDatabaseAccessBase

access(queries)[source]
get_params(vis)[source]
class pipeline.hsd.tasks.k2jycal.jyperkdbaccess.JyPerKAsdmEndPoint(context=None)[source]

Bases: pipeline.hsd.tasks.k2jycal.jyperkdbaccess.ALMAJyPerKDatabaseAccessBase

ENDPOINT_TYPE = 'asdm'
access(queries)[source]
get_params(vis)[source]
class pipeline.hsd.tasks.k2jycal.jyperkdbaccess.JyPerKInterpolationEndPoint(context=None)[source]

Bases: pipeline.hsd.tasks.k2jycal.jyperkdbaccess.JyPerKAbstractEndPoint

ENDPOINT_TYPE = 'interpolation'
class pipeline.hsd.tasks.k2jycal.jyperkdbaccess.JyPerKModelFitEndPoint(context=None)[source]

Bases: pipeline.hsd.tasks.k2jycal.jyperkdbaccess.JyPerKAbstractEndPoint

ENDPOINT_TYPE = 'model-fit'
class pipeline.hsd.tasks.k2jycal.jyperkdbaccess.QueryStruct(param, subparam)

Bases: tuple

property param

Alias for field number 0

property subparam

Alias for field number 1

class pipeline.hsd.tasks.k2jycal.jyperkdbaccess.ResponseStruct(response, subparam)

Bases: tuple

property response

Alias for field number 0

property subparam

Alias for field number 1

pipeline.hsd.tasks.k2jycal.jyperkdbaccess.get_mean_elevation(context, vis, antenna_id)[source]
pipeline.hsd.tasks.k2jycal.jyperkdbaccess.get_mean_frequency(spw)[source]
pipeline.hsd.tasks.k2jycal.jyperkdbaccess.get_mean_temperature(vis)[source]
pipeline.hsd.tasks.k2jycal.jyperkdbaccess.mjd_to_datestring(epoch)[source]
pipeline.hsd.tasks.k2jycal.jyperkdbaccess.translate_spw(data, ms)[source]
pipeline.hsd.tasks.k2jycal.jyperkdbaccess.vis_to_uid(vis)[source]

Convert MS name like uid___A002_Xabcd_X012 into uid://A002/Xabcd/X012

Parameters

vis {str} – Name of MS

Raises

RuntimeError –

Returns

str – Corresponding ASDM uid

pipeline.hsd.tasks.k2jycal.jyperkreader module

class pipeline.hsd.tasks.k2jycal.jyperkreader.JyPerK[source]

Bases: object

Parse session based jyperk csv and store. * meta stores meta data information from the lines in the form, ‘#name=value’,

as a dictionary, meta[name]=value.

  • header stores column label from the line in the form ‘#header0, header1, …’

    as a list, header = [‘header0’, ‘header1’, …]

  • data stores values in csv file as a dictionary,

    data[‘header0’] = [data00, data01, …]

register_data(content)[source]
register_meta(content)[source]
class pipeline.hsd.tasks.k2jycal.jyperkreader.JyPerKDataParser[source]

Bases: object

classmethod get_content(line)[source]
classmethod parse_data(line)[source]
classmethod parse_header(line)[source]
pipeline.hsd.tasks.k2jycal.jyperkreader.associate(context, factors)[source]

Convert data collected from session based jyperk csv as JyPerK object to MS-beased csv, i.e., a string list of [‘MS,ant,spwid,polid,factor’, …]

pipeline.hsd.tasks.k2jycal.jyperkreader.inspect_coverage(minval, maxval, minref, maxref)[source]
pipeline.hsd.tasks.k2jycal.jyperkreader.inspect_type(filename)[source]
pipeline.hsd.tasks.k2jycal.jyperkreader.read(context, filename)[source]

Reads jyperk factors from a file and returns a string list of [[‘MS’,’ant’,’spwid’,’polid’,’factor’], …]

pipeline.hsd.tasks.k2jycal.jyperkreader.read_ms_based(reffile)[source]
pipeline.hsd.tasks.k2jycal.jyperkreader.read_session_based(context, reffile)[source]

pipeline.hsd.tasks.k2jycal.k2jycal module

class pipeline.hsd.tasks.k2jycal.k2jycal.SDK2JyCal(inputs)[source]

Bases: pipeline.infrastructure.basetask.StandardTaskTemplate

Inputs

alias of SDK2JyCalInputs

analyse(result)[source]

Determine the best parameters by analysing the given jobs before returning any final jobs to execute.

Parameters

jobs (a list ofJobRequest) – the job requests generated by prepare()

Return type

Result

prepare()[source]

Prepare job requests for execution.

Parameters

parameters – the parameters to pass through to the subclass. Refer to the implementing subclass for specific information on what these parameters are.

Return type

a class implementing Result

class pipeline.hsd.tasks.k2jycal.k2jycal.SDK2JyCalInputs(context, output_dir=None, infiles=None, caltable=None, reffile=None, dbservice=None, endpoint=None)[source]

Bases: pipeline.infrastructure.vdp.StandardInputs

caltable

VisDependentProperty is a Python data descriptor that standardises the behaviour of pipeline Inputs properties and lets them create default values more easily.

On reading a VisDependentProperty (ie. using the dot prefix: inputs.solint), one of two things happens:

  1. If a NullMarker is found - signifying that no user input has been provided - and a ‘getter’ function has been defined, the getter function will be called to provide a default value for that measurement set.

  2. If a user has overridden the value (eg. inputs.solint = 123), that value will be retrieved.

  3. The value, either the default from step 1 or user-provided from step 2, is run through the optional postprocess function, which gives a final opportunity to change the value depending on the state/value of other properties.

A VisDependentProperty can be made read-only by specifying ‘readonly=True’ when creating the instance.

A VisDependentProperty can be hidden from the containing Inputs string representation by setting ‘hidden=True’ when creating the instance. This will hide the property from the web log and CLI getInputs calls.

Each VisDependentProperty has a set of values it considers equivalent to null. When the user sets the VDP value to one of these null values, the VDP machinery converts this to a private NullObject marker that signifies the property is now unset, resulting in the default value being returned next time the property is read. Developers can specify which values should be converted to NullObject by specifying null_input at creation time, e.g.,

solint = @VisDependentProperty(default=5, null_input=[None, ‘’, ‘RESET’, -1])

infiles

VisDependentProperty is a Python data descriptor that standardises the behaviour of pipeline Inputs properties and lets them create default values more easily.

On reading a VisDependentProperty (ie. using the dot prefix: inputs.solint), one of two things happens:

  1. If a NullMarker is found - signifying that no user input has been provided - and a ‘getter’ function has been defined, the getter function will be called to provide a default value for that measurement set.

  2. If a user has overridden the value (eg. inputs.solint = 123), that value will be retrieved.

  3. The value, either the default from step 1 or user-provided from step 2, is run through the optional postprocess function, which gives a final opportunity to change the value depending on the state/value of other properties.

A VisDependentProperty can be made read-only by specifying ‘readonly=True’ when creating the instance.

A VisDependentProperty can be hidden from the containing Inputs string representation by setting ‘hidden=True’ when creating the instance. This will hide the property from the web log and CLI getInputs calls.

Each VisDependentProperty has a set of values it considers equivalent to null. When the user sets the VDP value to one of these null values, the VDP machinery converts this to a private NullObject marker that signifies the property is now unset, resulting in the default value being returned next time the property is read. Developers can specify which values should be converted to NullObject by specifying null_input at creation time, e.g.,

solint = @VisDependentProperty(default=5, null_input=[None, ‘’, ‘RESET’, -1])

reffile

VisDependentProperty is a Python data descriptor that standardises the behaviour of pipeline Inputs properties and lets them create default values more easily.

On reading a VisDependentProperty (ie. using the dot prefix: inputs.solint), one of two things happens:

  1. If a NullMarker is found - signifying that no user input has been provided - and a ‘getter’ function has been defined, the getter function will be called to provide a default value for that measurement set.

  2. If a user has overridden the value (eg. inputs.solint = 123), that value will be retrieved.

  3. The value, either the default from step 1 or user-provided from step 2, is run through the optional postprocess function, which gives a final opportunity to change the value depending on the state/value of other properties.

A VisDependentProperty can be made read-only by specifying ‘readonly=True’ when creating the instance.

A VisDependentProperty can be hidden from the containing Inputs string representation by setting ‘hidden=True’ when creating the instance. This will hide the property from the web log and CLI getInputs calls.

Each VisDependentProperty has a set of values it considers equivalent to null. When the user sets the VDP value to one of these null values, the VDP machinery converts this to a private NullObject marker that signifies the property is now unset, resulting in the default value being returned next time the property is read. Developers can specify which values should be converted to NullObject by specifying null_input at creation time, e.g.,

solint = @VisDependentProperty(default=5, null_input=[None, ‘’, ‘RESET’, -1])

class pipeline.hsd.tasks.k2jycal.k2jycal.SDK2JyCalResults(vis=None, final=[], pool=[], reffile=None, factors={}, all_ok=False, dbstatus=None)[source]

Bases: pipeline.infrastructure.basetask.Results

merge_with_context(context)[source]

Merge these results with the given context.

This method will be called during the execution of accept(). For calibration tasks, a typical implementation will register caltables with the pipeline callibrary.

At this point the result is deemed safe to merge, so no further checks on the context need be performed.

Parameters

context (Context) – the target Context

pipeline.hsd.tasks.k2jycal.k2jycal.export_jyperk(outfile, factors)[source]
pipeline.hsd.tasks.k2jycal.k2jycal.rearrange_factors_list(factors_list)[source]

Rearrange scaling factor list to dictionary which looks like {‘MS’: {‘spw’: {‘Ant’: {‘pol’: factor}}}}

pipeline.hsd.tasks.k2jycal.qa module

class pipeline.hsd.tasks.k2jycal.qa.SDK2JyCalListQAHandler[source]

Bases: pipeline.infrastructure.pipelineqa.QAPlugin

child_cls

alias of pipeline.hsd.tasks.k2jycal.k2jycal.SDK2JyCalResults

handle(context, result)[source]
result_cls

alias of collections.abc.Iterable

class pipeline.hsd.tasks.k2jycal.qa.SDK2JyCalQAHandler[source]

Bases: pipeline.infrastructure.pipelineqa.QAPlugin

child_cls = None
handle(context, result)[source]
result_cls

alias of pipeline.hsd.tasks.k2jycal.k2jycal.SDK2JyCalResults

pipeline.hsd.tasks.k2jycal.renderer module

class pipeline.hsd.tasks.k2jycal.renderer.JyperKTR(spw, msname, antenna, pol, factor)

Bases: tuple

property antenna

Alias for field number 2

property factor

Alias for field number 4

property msname

Alias for field number 1

property pol

Alias for field number 3

property spw

Alias for field number 0

class pipeline.hsd.tasks.k2jycal.renderer.JyperKTRV(virtualspw, msname, realspw, antenna, pol, factor)

Bases: tuple

property antenna

Alias for field number 3

property factor

Alias for field number 5

property msname

Alias for field number 1

property pol

Alias for field number 4

property realspw

Alias for field number 2

property virtualspw

Alias for field number 0

class pipeline.hsd.tasks.k2jycal.renderer.T2_4MDetailsSingleDishK2JyCalRenderer(uri='hsd_k2jycal.mako', description='Generate Kelvin to Jy calibration table.', always_rerender=False)[source]

Bases: pipeline.infrastructure.renderer.basetemplates.T2_4MDetailsDefaultRenderer

update_mako_context(ctx, context, results)[source]

pipeline.hsd.tasks.k2jycal.worker module

class pipeline.hsd.tasks.k2jycal.worker.SDK2JyCalWorker(inputs)[source]

Bases: pipeline.infrastructure.basetask.StandardTaskTemplate

Per MS caltable creation

Inputs

alias of SDK2JyCalWorkerInputs

analyse(result)[source]

Define factors actually used and analyze if the factors are provided to all relevant data in MS.

prepare()[source]

Prepare job requests for execution.

Parameters

parameters – the parameters to pass through to the subclass. Refer to the implementing subclass for specific information on what these parameters are.

Return type

a class implementing Result

class pipeline.hsd.tasks.k2jycal.worker.SDK2JyCalWorkerInputs(context, output_dir, vis, caltable, factors)[source]

Bases: pipeline.infrastructure.vdp.StandardInputs

caltype

VisDependentProperty is a Python data descriptor that standardises the behaviour of pipeline Inputs properties and lets them create default values more easily.

On reading a VisDependentProperty (ie. using the dot prefix: inputs.solint), one of two things happens:

  1. If a NullMarker is found - signifying that no user input has been provided - and a ‘getter’ function has been defined, the getter function will be called to provide a default value for that measurement set.

  2. If a user has overridden the value (eg. inputs.solint = 123), that value will be retrieved.

  3. The value, either the default from step 1 or user-provided from step 2, is run through the optional postprocess function, which gives a final opportunity to change the value depending on the state/value of other properties.

A VisDependentProperty can be made read-only by specifying ‘readonly=True’ when creating the instance.

A VisDependentProperty can be hidden from the containing Inputs string representation by setting ‘hidden=True’ when creating the instance. This will hide the property from the web log and CLI getInputs calls.

Each VisDependentProperty has a set of values it considers equivalent to null. When the user sets the VDP value to one of these null values, the VDP machinery converts this to a private NullObject marker that signifies the property is now unset, resulting in the default value being returned next time the property is read. Developers can specify which values should be converted to NullObject by specifying null_input at creation time, e.g.,

solint = @VisDependentProperty(default=5, null_input=[None, ‘’, ‘RESET’, -1])

to_casa_args()[source]

Express these inputs as a dictionary of CASA arguments. The values in the dictionary are in a format suitable for CASA and can be directly passed to the CASA task.

Return type

a dictionary of string/??? kw/val pairs

class pipeline.hsd.tasks.k2jycal.worker.SDK2JyCalWorkerResults(vis, calapp=None, factors={})[source]

Bases: pipeline.infrastructure.basetask.Results

merge_with_context(context)[source]

Merge these results with the given context.

This method will be called during the execution of accept(). For calibration tasks, a typical implementation will register caltables with the pipeline callibrary.

At this point the result is deemed safe to merge, so no further checks on the context need be performed.

Parameters

context (Context) – the target Context

Module contents