1.42. doHebCorrection

Full Name: herschel.hifi.pipeline.generic.DoHebCorrectionTask
Alias: doHebCorrection
Type: Java Task - Java Task
Import: from herschel.hifi.pipeline.generic import DoHebCorrectionTask
Category

HIFI/Pipeline/Level 1 Pipeline

Description

Task to correct (remove) Electric Standing Waves in Bands 6 & 7.

The correction is performed on Level 1 spectra (type 'science': on-target spectra by default, and also off-target spectra if input parameter offsource=True).

The user will then have to (re)process the observation from Level 1 onward to create a final product.

The HRS bands are corrected using the fit parameters found for the WBS.

Regions with lines are automatically excluded from the fit, or can be specified using the "excludeIF/excludeSky" parameters. Automatic disregard of lines is done by (iterative) robust fitting. All channels with a flag not equal to 0 are disregarded in the fitting.

Expert options:

By default the model set for fitting is taken from the Calibration tree. The model set can also be supplied on the command line with the "catalog" parameter. See the HIFI Data Reduction Guide section on Electrical Standing Wave removal for example usage and how to obtain the model file.

Experts may also set the number of knots in the background spline, the order of that spline, the length of the boxcar filter, the Tukey scale and/or the order of the multiplicative polynomial. See JavaDoc for info.

Example

Example 1: DoHebCorrectionTask
# The task is run on individual HTP.
# It always needs the spline model catalog input from the calibration tree
# If options 'reuse' or 'redo' are used, it also needs a prepared solution
# also from the calibration tree
# WARNING: The solution tables used by the tasks doHebCorrection and hebCorrection
#          are NOT interchangeable, due to table formating differences.
#
# We recommend using the hebCorrection task to correct observations by fitting ESW. It works on all backends
# in one go. Task doHebCorrection is used by the pipeline to apply pre-computed solutions from the calTree.
#
# If the HEB Correction has been applied to a dataset the metadatum 'hebCorrection' will be present in the dataset.
# This can be checked in the following, way on the command line of HIPE, for example,
#
print obs.refs["level1"].product.refs["WBS-V"].product.refs["box_001"].product["0005"].meta['hebCorrection']
# ## {description="Heb Standing Wave Correction has been done", boolean=true}
#
# Undo the HEB correction. It assumes that the necessary tables are present in the caltree.
# The spline model catalog should be in the caltree at
# obs.refs["calibration"].product.refs["Downlink"].product.refs["Generic"].product.refs["hebCorrection"]
     .product.refs["WBS-Splines"].product
# The precomputed solutions should be in the caltree at
# obs.refs["calibration"].product.refs["Downlink"].product.refs["Generic"].product.refs["hebCorrection"]
     .product.refs["fitParameters"].product
#
htpWBSVCorr = obs.getLevel1().getProduct('WBS-V')
htpWBSVUnCorr, tds = doHebCorrection( htp=htpWBSVCorr, catalog=obs.calibration, undo=obs.calibration)
# hebCorrection metadatum now changed to False
htpHRSVCorr = obs.getLevel1().getProduct('HRS-V')
htpHRSVUnCorr, tds = doHebCorrection( htp=htpHRSVCorr, catalog=obs.calibration, undo=obs.calibration)
obs.level["level1"].setProduct("WBS-V",htpWBSVUnCorr)
obs.level["level1"].setProduct("HRS-V",htpHRSVUnCorr)
obs = hifiPipeline( obs=obs, fromLevel=1.0 )
#
# Apply the correction to V again
for backend in ['WBS-V', 'HRS-V']:
   htpUnCorr = obs.getLevel1().getProduct(backend)
   htpCorr, tds = doHebCorrection( htp=htpUnCorr, catalog=obs.calibration, redo=obs.calibration, redoRequired=1)
   obs.level["level1"].setProduct(backend,htpCorr)
#
obs = hifiPipeline( obs=obs, fromLevel=1.0 )

API details

Properties

HifiTimelineProduct htp [INOUT, OPTIONAL, default=null]

A HifiTimelineProduct from the level 1 of an observation context

Product fitpars [OUTPUT, OPTIONAL, default=(none)]

The product containing all fit parameters.

Double1d excludeIF [INPUT, OPTIONAL, default=(none)]

One or more pairs of [low,high] IF frequencies which should be exclude from fitting. E.g. because a line is present in that region.

When excludeIF is set, robust fitting is turned off.

Double1d excludeSky [INPUT, OPTIONAL, default=(none)]

One or more pairs of [low,high] sky frequencies which should be exclude from fitting.

Note: Robust fitting is not turned off. Use parameter robust when needed.

Boolean robust [INPUT, OPTIONAL, default=(true)]

Robust fitting is turned on/off. It overrides consequences of the exclude keywords

Product catalog [INPUT, OPTIONAL, default=no default value]

This parameter can take either take a user supplied catalog (Product) of spline-forms that suffice to model all existing ESW shape or a reference to the calibration tree (MapContext) from which the Spline models are stored.

Object redo [INPUT, OPTIONAL, default=null]

Use fit parameters derived in a previous run to reprocess the same observation again. The parameters are either taken from a TableDataset (if redo is a TDS) or from the caltree.

Only one of the parameters [redo, reuse, undo] can be given.

Boolean redoRequired [INPUT, OPTIONAL, default=false]

A flag that indicates to the task that the redo TableDataset or calibration data must be available. This is required by the SPG pipeline to ensure that the task is only executed when existing solutions are available. Requiring this reduces the time it takes to execute this task.

Object reuse [INPUT, OPTIONAL, default=null]

Use info (exclude, robust, offSource) from a previous run to process the data. The info is either taken from a TableDataset (if reuse is a TDS) or from the caltree.

The can be replaced by parameter settings.

Only one of the parameters [redo, reuse, undo] can be given.

Object undo [INPUT, OPTIONAL, default=null]

Use fit parameters in the TableDatset to undo the corrections. The parameters are either taken from a TableDataset (if undo is a TDS) or from the caltree.

Undo leaves a mark in MetaData: HebCorrection is set to false.

Only one of the parameters [redo, reuse, undo] can be given.

Boolean offSource [INPUT, OPTIONAL, default=true (on-source & off-source is done)]

When set to false, only the on-source science spectra are corrected.

Boolean ignore [INPUT, OPTIONAL, default=no default value]

If set to True no fit correction is performed.

PipelineConfiguration params [INPUT, OPTIONAL, default=no default value]

Pipeline configuration parameters that can be passed to the task.

See also

History

  • 2012-03-10 - DK: Initial version
  • 2014-17-07 - KE: Modified task for use in level 1 pipeline
  • 2014-06-08 - KE: Added to level 1 pipeline