pipeline.infrastructure.renderer package

Subpackages

• pipeline.infrastructure.renderer.templates package
  • Subpackages
    • pipeline.infrastructure.renderer.templates.resources package
      • Module contents
  • Module contents

Submodules

pipeline.infrastructure.renderer.basetemplates module

Created on 8 Sep 2014

@author: sjw

class pipeline.infrastructure.renderer.basetemplates.CommonRenderer(uri, context, result)

Bases: object

get_file()

render()

update_mako_context(mako_context)

class pipeline.infrastructure.renderer.basetemplates.JsonPlotRenderer(uri, context, result, plots, title, outfile)

Bases: pipeline.infrastructure.renderer.basetemplates.CommonRenderer

update_json_dict(d, plot)

Hook function that can be used by extending classes to extract extra parameters from the plot object and insert them into the JSON dictionary for that plot.

update_mako_context(mako_context)

class pipeline.infrastructure.renderer.basetemplates.T2_4MDetailsDefaultRenderer(uri='t2-4m_details-generic.mako', description='No description set for this task', always_rerender=False)

Bases: object

get_display_context(context, result)

render(context, result)

update_mako_context(mako_context, pipeline_context, result)

class pipeline.infrastructure.renderer.basetemplates.T2_4MDetailsFailedTaskRenderer(uri='t2-4m_details-failed.mako', description='Failed task', always_rerender=False)

Bases: pipeline.infrastructure.renderer.basetemplates.T2_4MDetailsDefaultRenderer

Renders detailed HTML output for a failed task.

update_mako_context(mako_context, pipeline_context, results)

pipeline.infrastructure.renderer.basetemplates.reorder_dicts(order_mapping, d)

pipeline.infrastructure.renderer.basetemplates.time_order_dicts(mako_context, pipeline_context)

Time-sort any MS-related entries in the Mako context dictionary.

pipeline.infrastructure.renderer.htmlrenderer module

class pipeline.infrastructure.renderer.htmlrenderer.LogCopier

Bases: object

LogCopier copies and handles the CASA logs so that they may be referenced by the pipeline web logs.

Capturing the CASA log gives us a few problems: The previous log is renamed upon starting a new session. To be reliably referenced from the web log, we must give it an immutable name and copy it to a safe location within the web log directory.

The user may want to view the web log at any time during a pipeline session. To avoid broken links to the CASA log, the log should be copied across to the web log location at the end of each task.

Pipeline sessions may be interrupted and restored, resulting in multiple CASA logs for such sessions. These logs must be consolidated into one file alongside any previous log information.

Adding HTML tags such as ‘<pre>’ and HTML anchors causes the CASA log reader to render such entries as empty entries at the bottom of the log. The result is that you must scroll up to find the last log entry. To prevent this, we need to output anchors as CASA log comments, possibly timestamps, and then use javascript to navigate to the log location.

static copy(context)

class pipeline.infrastructure.renderer.htmlrenderer.MetadataRendererBase

Bases: pipeline.infrastructure.renderer.htmlrenderer.RendererBase

classmethod rerender(context)

class pipeline.infrastructure.renderer.htmlrenderer.MsgTableRow(stage, task, type, message, target)

Bases: tuple

property message

Alias for field number 3

property stage

Alias for field number 0

property target

Alias for field number 4

property task

Alias for field number 1

property type

Alias for field number 2

class pipeline.infrastructure.renderer.htmlrenderer.RendererBase

Bases: object

Base renderer class.

classmethod get_file(context)

classmethod get_path(context)

classmethod render(context)

classmethod rerender(context)

class pipeline.infrastructure.renderer.htmlrenderer.Session(mses=None, name='Unnamed Session')

Bases: object

static get_sessions(context)

class pipeline.infrastructure.renderer.htmlrenderer.T1_1Renderer

Bases: pipeline.infrastructure.renderer.htmlrenderer.RendererBase

T1-1 OUS Splash Page renderer

class EnvironmentTableRow(hostname, num_mpi_servers, num_cores, cpu, ram, os, ulimit)

Bases: tuple

property cpu

Alias for field number 3

property hostname

Alias for field number 0

property num_cores

Alias for field number 2

property num_mpi_servers

Alias for field number 1

property os

Alias for field number 5

property ram

Alias for field number 4

property ulimit

Alias for field number 6

TableRow

alias of Tablerow

TableRowNRO

alias of TablerowNRO

static get_cluster_tablerows()

static get_display_context(context)

output_file = 't1-1.html'

template = 't1-1.mako'

class pipeline.infrastructure.renderer.htmlrenderer.T1_2Renderer

Bases: pipeline.infrastructure.renderer.htmlrenderer.RendererBase

T1-2 Observation Summary renderer

TableRow

alias of Tablerow

static get_display_context(context)

output_file = 't1-2.html'

template = 't1-2.mako'

class pipeline.infrastructure.renderer.htmlrenderer.T1_3MRenderer

Bases: pipeline.infrastructure.renderer.htmlrenderer.RendererBase

T1-3M renderer

class MsgTableRow(stage, task, type, message, target)

Bases: tuple

property message

Alias for field number 3

property stage

Alias for field number 0

property target

Alias for field number 4

property task

Alias for field number 1

property type

Alias for field number 2

classmethod get_display_context(context)

output_file = 't1-3.html'

template = 't1-3m.mako'

class pipeline.infrastructure.renderer.htmlrenderer.T1_4MRenderer

Bases: pipeline.infrastructure.renderer.htmlrenderer.RendererBase

T1-4M renderer

static get_display_context(context)

output_file = 't1-4.html'

template = 't1-4m.mako'

class pipeline.infrastructure.renderer.htmlrenderer.T2_1DetailsRenderer

Bases: object

static get_display_context(context, ms)

classmethod get_file(filename)

classmethod get_filename(context, session, ms)

output_file = 't2-1_details.html'

classmethod render(context)

template = 't2-1_details.mako'

static write_listobs(context, ms)

class pipeline.infrastructure.renderer.htmlrenderer.T2_1Renderer

Bases: pipeline.infrastructure.renderer.htmlrenderer.RendererBase

T2-4M renderer

static get_display_context(context)

output_file = 't2-1.html'

template = 't2-1.mako'

class pipeline.infrastructure.renderer.htmlrenderer.T2_2_1Renderer

Bases: pipeline.infrastructure.renderer.htmlrenderer.T2_2_XRendererBase

T2-2-1 renderer - spatial setup

static get_display_context(context, ms)

output_file = 't2-2-1.html'

template = 't2-2-1.mako'

class pipeline.infrastructure.renderer.htmlrenderer.T2_2_2Renderer

Bases: pipeline.infrastructure.renderer.htmlrenderer.T2_2_XRendererBase

T2-2-2 renderer

static get_display_context(context, ms)

output_file = 't2-2-2.html'

template = 't2-2-2.mako'

class pipeline.infrastructure.renderer.htmlrenderer.T2_2_3Renderer

Bases: pipeline.infrastructure.renderer.htmlrenderer.T2_2_XRendererBase

T2-2-3 renderer

static get_display_context(context, ms)

output_file = 't2-2-3.html'

template = 't2-2-3.mako'

class pipeline.infrastructure.renderer.htmlrenderer.T2_2_4Renderer

Bases: pipeline.infrastructure.renderer.htmlrenderer.T2_2_XRendererBase

T2-2-4 renderer

static get_display_context(context, ms)

output_file = 't2-2-4.html'

template = 't2-2-4.mako'

class pipeline.infrastructure.renderer.htmlrenderer.T2_2_5Renderer

Bases: pipeline.infrastructure.renderer.htmlrenderer.T2_2_XRendererBase

T2-2-5 renderer - weather page

static get_display_context(context, ms)

output_file = 't2-2-5.html'

template = 't2-2-5.mako'

class pipeline.infrastructure.renderer.htmlrenderer.T2_2_6Renderer

Bases: pipeline.infrastructure.renderer.htmlrenderer.T2_2_XRendererBase

T2-2-6 renderer - scans page

class TableRow(id, time_start, time_end, duration, intents, fields, spws)

Bases: tuple

property duration

Alias for field number 3

property fields

Alias for field number 5

property id

Alias for field number 0

property intents

Alias for field number 4

property spws

Alias for field number 6

property time_end

Alias for field number 2

property time_start

Alias for field number 1

static get_display_context(context, ms)

output_file = 't2-2-6.html'

template = 't2-2-6.mako'

class pipeline.infrastructure.renderer.htmlrenderer.T2_2_7Renderer

Bases: pipeline.infrastructure.renderer.htmlrenderer.T2_2_XRendererBase

T2-2-7 renderer (single dish specific)

static get_display_context(context, ms)

output_file = 't2-2-7.html'

classmethod render(context)

template = 't2-2-7.mako'

class pipeline.infrastructure.renderer.htmlrenderer.T2_2_XRendererBase

Bases: object

Base renderer for T2-2-X series of pages.

classmethod get_file(filename)

classmethod get_filename(context, ms)

classmethod render(context)

class pipeline.infrastructure.renderer.htmlrenderer.T2_3_1MRenderer

Bases: pipeline.infrastructure.renderer.htmlrenderer.T2_3_XMBaseRenderer

Renderer for T2-3-1M, the data set topic.

classmethod get_topic()

output_file = 't2-3-1m.html'

template = 't2-3-1m.mako'

class pipeline.infrastructure.renderer.htmlrenderer.T2_3_2MRenderer

Bases: pipeline.infrastructure.renderer.htmlrenderer.T2_3_XMBaseRenderer

Renderer for T2-3-2M: the QA calibration section.

classmethod get_topic()

output_file = 't2-3-2m.html'

template = 't2-3-2m.mako'

class pipeline.infrastructure.renderer.htmlrenderer.T2_3_3MRenderer

Bases: pipeline.infrastructure.renderer.htmlrenderer.T2_3_XMBaseRenderer

Renderer for T2-3-3M: the QA flagging section.

classmethod get_topic()

output_file = 't2-3-3m.html'

template = 't2-3-3m.mako'

class pipeline.infrastructure.renderer.htmlrenderer.T2_3_4MRenderer

Bases: pipeline.infrastructure.renderer.htmlrenderer.T2_3_XMBaseRenderer

Renderer for T2-3-4M: the QA line finding section.

classmethod get_topic()

output_file = 't2-3-4m.html'

template = 't2-3-4m.mako'

class pipeline.infrastructure.renderer.htmlrenderer.T2_3_5MRenderer

Bases: pipeline.infrastructure.renderer.htmlrenderer.T2_3_XMBaseRenderer

Renderer for T2-3-5M: the imaging topic

classmethod get_topic()

output_file = 't2-3-5m.html'

template = 't2-3-5m.mako'

class pipeline.infrastructure.renderer.htmlrenderer.T2_3_6MRenderer

Bases: pipeline.infrastructure.renderer.htmlrenderer.T2_3_XMBaseRenderer

Renderer for T2-3-6M: the miscellaneous topic

classmethod get_topic()

output_file = 't2-3-6m.html'

template = 't2-3-6m.mako'

class pipeline.infrastructure.renderer.htmlrenderer.T2_3_XMBaseRenderer

Bases: pipeline.infrastructure.renderer.htmlrenderer.RendererBase

classmethod get_display_context(context)

output_file = 'overrideme'

template = 'overrideme'

class pipeline.infrastructure.renderer.htmlrenderer.T2_4MDetailsContainerRenderer

Bases: pipeline.infrastructure.renderer.htmlrenderer.RendererBase

classmethod get_file(context, result)

classmethod get_path(context, result)

output_file = 't2-4m_details-container.html'

classmethod render(context, result, urls)

template = 't2-4m_details-container.mako'

class pipeline.infrastructure.renderer.htmlrenderer.T2_4MDetailsDefaultRenderer(template='t2-4m_details-generic.mako', always_rerender=False)

Bases: object

get_display_context(context, result)

render(context, result)

update_mako_context(mako_context, pipeline_context, result)

class pipeline.infrastructure.renderer.htmlrenderer.T2_4MDetailsRenderer

Bases: object

classmethod get_file(context, result, root)

classmethod get_path(context, result, root='')

output_file = 't2-4m_details.html'

classmethod render(context)

classmethod render_result(renderer, context, result, root='')

class pipeline.infrastructure.renderer.htmlrenderer.T2_4MRenderer

Bases: pipeline.infrastructure.renderer.htmlrenderer.RendererBase

T2-4M renderer

static get_display_context(context)

output_file = 't2-4m.html'

template = 't2-4m.mako'

class pipeline.infrastructure.renderer.htmlrenderer.WebLogGenerator

Bases: object

static copy_resources(context)

static render(context)

renderers = [<class 'pipeline.infrastructure.renderer.htmlrenderer.T1_1Renderer'>, <class 'pipeline.infrastructure.renderer.htmlrenderer.T1_2Renderer'>, <class 'pipeline.infrastructure.renderer.htmlrenderer.T1_3MRenderer'>, <class 'pipeline.infrastructure.renderer.htmlrenderer.T2_1Renderer'>, <class 'pipeline.infrastructure.renderer.htmlrenderer.T2_1DetailsRenderer'>, <class 'pipeline.infrastructure.renderer.htmlrenderer.T2_2_1Renderer'>, <class 'pipeline.infrastructure.renderer.htmlrenderer.T2_2_2Renderer'>, <class 'pipeline.infrastructure.renderer.htmlrenderer.T2_2_3Renderer'>, <class 'pipeline.infrastructure.renderer.htmlrenderer.T2_2_4Renderer'>, <class 'pipeline.infrastructure.renderer.htmlrenderer.T2_2_6Renderer'>, <class 'pipeline.infrastructure.renderer.htmlrenderer.T2_2_7Renderer'>, <class 'pipeline.infrastructure.renderer.htmlrenderer.T2_3_1MRenderer'>, <class 'pipeline.infrastructure.renderer.htmlrenderer.T2_3_2MRenderer'>, <class 'pipeline.infrastructure.renderer.htmlrenderer.T2_3_3MRenderer'>, <class 'pipeline.infrastructure.renderer.htmlrenderer.T2_3_5MRenderer'>, <class 'pipeline.infrastructure.renderer.htmlrenderer.T2_3_6MRenderer'>, <class 'pipeline.infrastructure.renderer.htmlrenderer.T2_4MRenderer'>, <class 'pipeline.infrastructure.renderer.htmlrenderer.T2_4MDetailsRenderer'>, <class 'pipeline.infrastructure.renderer.htmlrenderer.T1_4MRenderer'>]

pipeline.infrastructure.renderer.htmlrenderer.cmp(a, b)

pipeline.infrastructure.renderer.htmlrenderer.compute_az_el_for_ms(ms, observatory, func)

pipeline.infrastructure.renderer.htmlrenderer.compute_az_el_to_field(field, epoch, observatory)

pipeline.infrastructure.renderer.htmlrenderer.create_tablerow(results, message: str, msgtype: str, target='') → pipeline.infrastructure.renderer.htmlrenderer.MsgTableRow

Create a table entry struct from a web log message.

pipeline.infrastructure.renderer.htmlrenderer.filter_qascores(results_list, lo: float, hi: float) → List[pipeline.infrastructure.pipelineqa.QAScore]

pipeline.infrastructure.renderer.htmlrenderer.get_ms_attr_for_result(context, vis, accessor)

pipeline.infrastructure.renderer.htmlrenderer.get_ms_start_time_for_result(context, result)

pipeline.infrastructure.renderer.htmlrenderer.get_mses_by_time(context)

pipeline.infrastructure.renderer.htmlrenderer.get_plot_dir(context, stage_number)

pipeline.infrastructure.renderer.htmlrenderer.get_results_by_time(context, resultslist)

pipeline.infrastructure.renderer.htmlrenderer.get_rootdir(r)

pipeline.infrastructure.renderer.htmlrenderer.get_stage_number(result_obj)

pipeline.infrastructure.renderer.htmlrenderer.get_task_description(result_obj, context, include_stage=True)

pipeline.infrastructure.renderer.htmlrenderer.get_task_name(result_obj, include_stage=True)

pipeline.infrastructure.renderer.htmlrenderer.group_by_root(context, task_results)

pipeline.infrastructure.renderer.htmlrenderer.group_into_measurement_sets(context, task_results)

pipeline.infrastructure.renderer.htmlrenderer.group_into_sessions(context, task_results)

Return results grouped into lists by session.

pipeline.infrastructure.renderer.htmlrenderer.is_singledish_ms(context)

pipeline.infrastructure.renderer.htmlrenderer.logrecords_to_tablerows(records, results, msgtype='ERROR') → List[pipeline.infrastructure.renderer.htmlrenderer.MsgTableRow]

Convert a list of LogRecords to a list of table entries, ready for insertion into a Mako template.

pipeline.infrastructure.renderer.htmlrenderer.qascores_to_tablerows(qascores: List[pipeline.infrastructure.pipelineqa.QAScore], results, msgtype: str = 'ERROR') → List[pipeline.infrastructure.renderer.htmlrenderer.MsgTableRow]

Convert a list of QAScores to a list of table entries, ready for insertion into a Mako template.

pipeline.infrastructure.renderer.htmlrenderer.scan_has_intent(scans, intent)

Returns True if the list of scans includes a specified intent

pipeline.infrastructure.renderer.htmlrenderer.sort_by_time(mses)

Return measurement sets sorted by time order.

pipeline.infrastructure.renderer.htmlrenderer.wrap_in_resultslist(task_result)

pipeline.infrastructure.renderer.logger module

class pipeline.infrastructure.renderer.logger.Parameters

Bases: object

Provides a set of utility functions that describe how the plot parameters given as keys of the optional parameters dictionary given to Plot() should be interpreted.

css_ids = {'ant': 'ant', 'chnl': 'chnl', 'field': 'field', 'file': 'file', 'intent': 'intent', 'map': 'map', 'pol': 'pol', 'solint': 'solint', 'spw': 'spw', 'type': 'type', 'vis': 'vis'}

descriptions = {'ant': 'Antenna', 'chnl': 'Channels', 'field': 'Field', 'file': 'File', 'intent': 'Intent', 'map': 'Map', 'pol': 'Polarisation', 'solint': 'Solution Interval', 'spw': 'Spectral Window', 'type': 'Type', 'vis': 'Measurement Set'}

static getCssId(s) → string

Get the CSS class associated with this parameter in HTML output.

static getDescription(parameter)

getCssId(s) -> string

Get the plain English description of this parameter.

class pipeline.infrastructure.renderer.logger.Plot(filename, x_axis='Unknown', y_axis='Unknown', field=None, parameters=None, qa_score=None, command=None)

Bases: object

property css_class

The CSS class to be used for this plot.

property thumbnail

property title

title -> string

Construct and return a meaningful plot title using the internal state (axis names, parameter names etc.) of this plot.

class pipeline.infrastructure.renderer.logger.PlotGroup(plots=[])

Bases: object

property buttons

static create_plot_groups(plots=[])

Returns a list of PlotGroups, each containing a series of plots with the same axes.

full_template = <string.Template object>

static numericalSort(selector)

Sorts strings numerically, eg. [b13, a2, a10] -> [a2, a10, b13]

Based on Recipe 5.5 from Python Cookbook 2nd Edition.

selector_template = <string.Template object>

property selectors

property thumbnails

property title

property x_axis

property y_axis

class pipeline.infrastructure.renderer.logger.Selector(parameter, value)

Bases: object

pipeline.infrastructure.renderer.logger.THUMBNAIL_CMD(full, thumb)

pipeline.infrastructure.renderer.logger.getPath(filename)

pipeline.infrastructure.renderer.qaadapter module

The qaadapter module holds classes that regroup the results list, ordered by stage number, into a structure ordered by task type. This regrouping is used by the QA sections of the weblog.

class pipeline.infrastructure.renderer.qaadapter.CalibrationTopic

Bases: pipeline.infrastructure.renderer.qaadapter.Topic

CalibrationTopic collects together those results generated by calibration tasks.

class pipeline.infrastructure.renderer.qaadapter.DataSetTopic

Bases: pipeline.infrastructure.renderer.qaadapter.Topic

DataSetTopic collects together those results generated by data set handling tasks.

class pipeline.infrastructure.renderer.qaadapter.FlaggingTopic

Bases: pipeline.infrastructure.renderer.qaadapter.Topic

FlaggingTopic collects together those results generated by flagging tasks.

class pipeline.infrastructure.renderer.qaadapter.ImagingTopic

Bases: pipeline.infrastructure.renderer.qaadapter.Topic

ImagingTopic collects together those results generated by imaging tasks.

class pipeline.infrastructure.renderer.qaadapter.LineFindingTopic

Bases: pipeline.infrastructure.renderer.qaadapter.Topic

LineFindingTopic collects together those results generated by line-finding tasks.

class pipeline.infrastructure.renderer.qaadapter.MiscellaneousTopic

Bases: pipeline.infrastructure.renderer.qaadapter.Topic

MiscellaneousTopic collects together those results generated by miscellaneous tasks that don’t belong in any other category.

class pipeline.infrastructure.renderer.qaadapter.QABaseAdapter(context, result)

Bases: object

class pipeline.infrastructure.renderer.qaadapter.Topic(url, description)

Bases: object

Topic is the base class for a Topic web log section. It should not be instantiated directly.

assign_to_topics(results)

Allocating Results to the appropriate task type based on the Results classes registered to this Topic.

Parameters

results (a list of Result) – the results

handles_result(result)

Return True if the given Result is part of this topic.

Parameters

result (Result) – the task result

register_result_class(result_class)

class pipeline.infrastructure.renderer.qaadapter.TopicRegistry

Bases: object

assign_to_topics(results)

get_calibration_topic()

get_dataset_topic()

get_flagging_topic()

get_imaging_topic()

get_linefinding_topic()

get_miscellaneous_topic()

get_topics()

get_url(result)

Get the URL of the QA section appropriate to the given result.

Parameters

result (Result) – the task result

Return type

the filename of the appropriate QA section

register_to_calibration_topic(result_class)

register_to_dataset_topic(result_class)

register_to_flagging_topic(result_class)

register_to_imaging_topic(result_class)

register_to_linefinding_topic(result_class)

register_to_miscellaneous_topic(result_class)

topic_classes = [<class 'pipeline.infrastructure.renderer.qaadapter.DataSetTopic'>, <class 'pipeline.infrastructure.renderer.qaadapter.CalibrationTopic'>, <class 'pipeline.infrastructure.renderer.qaadapter.FlaggingTopic'>, <class 'pipeline.infrastructure.renderer.qaadapter.LineFindingTopic'>, <class 'pipeline.infrastructure.renderer.qaadapter.ImagingTopic'>, <class 'pipeline.infrastructure.renderer.qaadapter.MiscellaneousTopic'>]

pipeline.infrastructure.renderer.registry module

Created on 2 Nov 2018

@author: sjw

class pipeline.infrastructure.renderer.registry.RendererRegistry

Bases: object

add_renderer(task_cls, renderer, key_fn=None, key=None)

Register a renderer to be used to generate HTML for results from a given task.

There are two modes of registration:

1. registration of a context-specific renderer

2. registration of a universal renderer, which wil be used if no context-specific renderer is found

Context-specific renderers are registered by supplying key and key_fn arguments. key_fn should be a function that accepts a context and returns a key from it. This key is used to look up the renderer. Specifying a key value of ‘X’ says ‘this renderer should be used for this task if this key_fn returns ‘X’ for this context’.

Parameters

• task_cls – the target pipeline Task class

• renderer – the renderer to use for the task

• key – optional key to retrieve this renderer by

• key_fn – optional function that accepts a pipeline context and returns the renderer key

Returns

get_renderer(cls, context, result)

Get the registered renderer for a class.

The pipeline context argument may be passed to a registered function that returns the key for the given context.

Parameters

• cls – the class to look up

• context – pipeline context

• result – pipeline task result

Returns

registered renderer class, or KeyError if no renderer was registered

pipeline.infrastructure.renderer.regression module

The regression module contains base classes and plugin registry for the pipeline’s regression test value extractor framework.

This module contains two classes:

• RegressionExtractor: the base class for extractor plug-ins

• RegressionExtractorRegistry: the registry and manager for plug-ins

Tasks provide and register their own extractors that each extends RegressionExtractor. These extractor plug-ins analyse the results of a task, extracting pertinent values and writing them to a dict. The keys of the output dict identify the value; the values of the dict are the extracted values themselves.

The pipeline QA framework is activated whenever a Results instance is accepted into the pipeline context. The pipeline QA framework operates by calling is_handler_for(result) on each registered QAPlugin, passing it the the accepted Results instance for inspection. QAPlugins that claim to handle the Result are given the Result for processing. In this step, the QA framework calls QAPlugin.handle(context, result), the method overridden by the task-specific QAPlugin.

class pipeline.infrastructure.renderer.regression.ApplycalRegressionExtractor

Bases: pipeline.infrastructure.renderer.regression.RegressionExtractor

Regression test result extractor for applycal tasks.

child_cls = None

generating_task

alias of pipeline.hif.tasks.applycal.ifapplycal.IFApplycal

handle(result)

This method should return a dict of

{‘applycal.new_flags.science’: 0.34,

‘applycal.new_flags.bandpass’: 0.53}

Parameters

result –

Returns

result_cls

alias of pipeline.h.tasks.applycal.applycal.ApplycalResults

class pipeline.infrastructure.renderer.regression.FluxcalflagRegressionExtractor

Bases: pipeline.infrastructure.renderer.regression.RegressionExtractor

Regression test result extractor for hifa_gfluxscaleflag.

The extracted values are:

• the number of flagged rows before this task

• the number of flagged rows after this task

child_cls = None

handle(result)

This method should return a dict of

{‘applycal.new_flags.science’: 0.34,

‘applycal.new_flags.bandpass’: 0.53}

Parameters

result –

Returns

result_cls

alias of pipeline.hifa.tasks.gfluxscaleflag.resultobjects.GfluxscaleflagResults

class pipeline.infrastructure.renderer.regression.GcorFluxscaleRegressionExtractor

Bases: pipeline.infrastructure.renderer.regression.RegressionExtractor

Regression test result extractor for hifa_gcorfluxscale

The extracted values are:

• Stokes I for each field and spw, in Jy

child_cls = None

generating_task

alias of pipeline.hifa.tasks.fluxscale.gcorfluxscale.GcorFluxscale

handle(result)

This method should return a dict of

{‘applycal.new_flags.science’: 0.34,

‘applycal.new_flags.bandpass’: 0.53}

Parameters

result –

Returns

result_cls

alias of pipeline.h.tasks.common.commonfluxresults.FluxCalibrationResults

class pipeline.infrastructure.renderer.regression.RegressionExtractor

Bases: object

RegressionExtractor is the mandatory base class for all regression test result extractors.

child_cls = None

generating_task = None

abstract handle(result) → Dict[str, float]

This method should return a dict of

{‘applycal.new_flags.science’: 0.34,

‘applycal.new_flags.bandpass’: 0.53}

Parameters

result –

Returns

is_handler_for(result)

Return True if this RegressionExtractor can process the Result.

Parameters

result – the task Result to inspect

Returns

True if the Result can be processed

result_cls = None

class pipeline.infrastructure.renderer.regression.RegressionExtractorRegistry

Bases: object

The registry and manager of the regression result extractor framework.

The responsibility of the RegressionResultRegistry is to pass Results to RegressionExtractors that can handle them.

add_handler(handler)

handle(result)

pipeline.infrastructure.renderer.regression.extract_qa_score_regression(prefix, result)

pipeline.infrastructure.renderer.regression.extract_regression_results(context)

pipeline.infrastructure.renderer.regression.get_prefix(result)

pipeline.infrastructure.renderer.regression.key_intersection(d1: dict, d2: dict)

Compare keys of two dicts, returning duplicate keys.

pipeline.infrastructure.renderer.regression.union(d1: dict, d2: dict)

Return the union of two dicts, raising an exception if duplicate keys are detected in the input dicts.

pipeline.infrastructure.renderer.rendererutils module

pipeline.infrastructure.renderer.rendererutils.format_notification(tr_class, alert, msg, icon_class=None)

pipeline.infrastructure.renderer.rendererutils.format_score(pqascore)

pipeline.infrastructure.renderer.rendererutils.format_shortmsg(pqascore)

pipeline.infrastructure.renderer.rendererutils.get_badge_class(pqascore)

pipeline.infrastructure.renderer.rendererutils.get_bar_class(pqascore)

pipeline.infrastructure.renderer.rendererutils.get_bar_width(pqascore)

pipeline.infrastructure.renderer.rendererutils.get_command_markup(ctx, command)

pipeline.infrastructure.renderer.rendererutils.get_errors_badge(result)

pipeline.infrastructure.renderer.rendererutils.get_failures_badge(result)

pipeline.infrastructure.renderer.rendererutils.get_notification_trs(result, alerts_info, alerts_success)

pipeline.infrastructure.renderer.rendererutils.get_sidebar_style_for_task(result)

pipeline.infrastructure.renderer.rendererutils.get_suboptimal_badge(result)

pipeline.infrastructure.renderer.rendererutils.get_symbol_badge(result)

pipeline.infrastructure.renderer.rendererutils.get_warnings_badge(result)

pipeline.infrastructure.renderer.rendererutils.group_plots(data, axes)

pipeline.infrastructure.renderer.rendererutils.num_lines(abspath)

Report number of non-empty non-comment lines in a file specified by abspath. If the file does not exist, report N/A.

pipeline.infrastructure.renderer.rendererutils.printTsysFlags(tsystable, htmlreport)

Method that implements a version of printTsysFlags by Todd Hunter.

pipeline.infrastructure.renderer.rendererutils.renderflagcmds(flagcmds, htmlflagcmds)

Method to render a list of flagcmds into html format.

pipeline.infrastructure.renderer.rendererutils.sanitize_data_selection_string(text)

pipeline.infrastructure.renderer.rendererutils.scores_in_range(pool: List[pipeline.infrastructure.pipelineqa.QAScore], lo: float, hi: float) → List[pipeline.infrastructure.pipelineqa.QAScore]

Filter QA scores by range.

pipeline.infrastructure.renderer.rendererutils.scores_with_location(pool: List[pipeline.infrastructure.pipelineqa.QAScore], locations: Optional[List[pipeline.infrastructure.pipelineqa.WebLogLocation]] = None) → List[pipeline.infrastructure.pipelineqa.QAScore]

Filter QA scores by web log location.

pipeline.infrastructure.renderer.rendererutils.sort_row_by(row, axes)

pipeline.infrastructure.renderer.weblog module

Created on 8 Sep 2014

@author: sjw

class pipeline.infrastructure.renderer.weblog.WebLogRendererRegistry

Bases: pipeline.infrastructure.renderer.registry.RendererRegistry

add_renderer(task_cls, renderer, group_by=None, key_fn=None, key=None)

Register a renderer to be used to generate HTML for results from a given task.

There are two modes of registration:

1. registration of a context-specific renderer

2. registration of a universal renderer, which wil be used if no context-specific renderer is found

Context-specific renderers are registered by supplying key and key_fn arguments. key_fn should be a function that accepts a context and returns a key from it. This key is used to look up the renderer. Specifying a key value of ‘X’ says ‘this renderer should be used for this task if this key_fn returns ‘X’ for this context’.

Parameters

• task_cls – the target pipeline Task class

• renderer – the renderer to use for the task

• group_by – grouping directive - either “session” or “ungrouped”

• key – optional key to retrieve this renderer by

• key_fn – optional function that accepts a pipeline context and returns the renderer key

Returns

render_by_session(task_name)

render_ungrouped(task_name)

pipeline.infrastructure.renderer.weblog.add_renderer(*args, **kwargs)

pipeline.infrastructure.renderer.weblog.register_mako_templates(directory, prefix='')

Find Mako templates in the given directory, registering them to the module template lookup. Templates will be registered to a URI composed of the URI prefix argument (optional) plus the template filename, minus filename extension.

For example, a call with prefix=’hif’ finding a file called ‘importdata.mako’ would register the template to the Mako URI ‘hif/importdata’.

Module contents