1.38. doFluxHotCold

Full Name: herschel.hifi.pipeline.generic.DoFluxHotColdTask
Alias: doFluxHotCold
Type: Java Task - Java Task
Import: from herschel.hifi.pipeline.generic import DoFluxHotColdTask

HIFI/Pipeline/Level 1 Pipeline


Applies the bandpass spectra (computed by MkFluxHotCold) to the science spectra.

The bandpass spectra are passed to the module by using the keyword 'cal' and should be of type {@link CalFluxHotCold}. This product contains both, bandpass spectra and receiver temperature - for this module, only the bandpass is needed.

Typically, several bandpass spectra are included in the calibration product - each associated with different observation times and/or LO frequencies. For each science spectrum to be calibrated a suitable bandpass spectrum is constructed by first picking the set of bandpass spectra with consistent LO frequency and then applying a suitable interpolation scheme so that a bandpass with consistent observation time is obtained.

A validation mechanism checks the spectra to be subtracted from each other and, if needed, sets in the result data a row flag. The default mechanism inspects the mixer currents (columns 'MJC_hor' or 'MJC_Ver', respectively) and tests their relative deviation (of the science data and the bandpass) to be less than a given tolerance. Once the tolerance is exceeded, for the result spectrum a row flag (bit 16) is set. The tolerance can be set by specifying the 'validatorTolerance'-parameter. By default it is set to 0.025. This default checking mechanism can be overruled by passing as 'validator'-parameter an instance of the {@link DataValidator} interface. When setting this parameter to None, no checking is done.

In addition to a possible row flag associated with the mixer current deviation checks a flag is set in case no suitable bandpass has been found. In this situation, bit 17 is set in the row flag.


Example 1: In HIPE:
from herschel.hifi.pipeline.product import CalFluxHotCold
# htp is a HifiTimelineProduct with spectra to be calibrated.
# calHotCold is a CalFluxHotCold
res = doFluxHotCold(htp=htp, cal=calHotCold)
res = doFluxHotCold(htp=htp, cal=calHotCold, interpolator="LINEAR")
Example 2: In HIPE:
from herschel.hifi.pipeline.product import CalFluxHotCold
from herschel.hifi.cal import  CalCoupCoeff
htp is a HifiTimelineProduct with  spectra to be calibrated.
coupl are coupling coefficients (CalCoupCoeff)
cal=mkFluxHotCold(htp=htp, coeff=coupl)
res = doFluxHotCold(htp=htp, cal=cal)
Example 3: In HIPE:
# the validator tolerance is re-set.
htp = doFluxHotCold(htp=htp, cal=cal, validatorTolerance=0.1)
Example 4: In HIPE:
# validation is switched off.
htp = doFluxHotCold(htp=htp, cal=cal, validator=None)
Example 5: In HIPE:
# an alternative validation mechanism is set (here based, as dummy, on the LoFrequency)
from herschel.hifi.pipeline.generic.dorefsubtract import SubtractionDataValidatorImpl
validator = SubtractionDataValidatorImpl()
validator.configure('LoFrequency', 0.001, True)
htp = doFluxHotCold(htp=htp, cal=cal, validator=validator)

API details


HifiTimelineProduct htp [INOUT, MANDATORY, default=No default value]

The timeline product (observation) to be processed.

CalFluxHotCold cal [INPUT, MANDATORY, default=no default value]

Provides the product with the bandpass and receiver temperature spectra.

PipelineConfiguration params [INPUT, OPTIONAL, default=no default value]

Pipeline configuration parameters that can be passed to the task.

Boolean ignore [INPUT, OPTIONAL, default=no default value]

Flag to indicate whether the execution of the module should be ignored.

Object validatorTolerance [INPUT, OPTIONAL, default=0.025]

The tolerance to be used for the default validator that checks the difference in the mixer currents and sets a row flag once the mixer currents found in the science data and the bandpass deviate by more than the tolerance from each other. A relative distance measure is used to quantify the deviation (2abs(x1-x2)/(abs(x1)+abs(x2))). Alternatively, a calibration object of type GenericPipelineCalProductcan be passed from which the band-specific tolerances will be extracted. Yet another possibility is to pass the whole calibration context - the task retrieves the appropriate product with the tolerances. Furthermore, you can use the parameter 'calibration' for that purpose. Once a product of type GenericPipelineCalProduct is obtained, the appropriate tolerance is looked up from a table (associated with the given date) from a column named 'doFluxHotCold'.

DataValidator validator [INPUT, OPTIONAL, default=no default value]

Custom validator that can be passed to the task to check source and ref that is subtracted from each other and flag data once the tolerance exceeds a given tolerance. Note that the validatorTolerance cannot be used to specify this threshold. Furthermore, the default validator is overwritten once a validator or None is passed here.

MapContext calibration [INPUT, OPTIONAL, default=no default value]

Parameter to pass a calibration context from which all the calibration input, here just mixer current tolerances, can be retrieved.

Boolean useCalTree [INPUT, OPTIONAL, default=no default value]

Parameter to specify whether the calibration tree should be used as primary source of information. It plays a role for task parameters that can explicitly be set as task parameter value (here, for the validatorTolerance-parameter) or, alternatively, retrieved from the calibration tree (see parameter calibration). If useCalTree=True but no reference to the calibration tree is provided the validatorTolerance parameter is used.

String interpolator [INPUT, OPTIONAL, default=no default value]

Interpolation scheme to be used. Currently it accepts the following options:

  • "PREVIOUS" the previous hot-cold measurement in time is used.

  • "NEXT" the next hot-cold measurement in time is used.

  • "LINEAR": a linear interpolation is used.

  • "NEAREST": a nearest neighbor interpolation is used.

  • "CUBIC_SPLINE" : a polynomial cubic Spline fit is done.

  • "TRUNC_LINEAR" : linear for bracketed scans, nearest neighbour scans with only a single neighbouring scan.

Integer rowFlagMask [INPUT, OPTIONAL, default=RowMask.IGNORE_DATA.getValue() | RowMask.BAD_DATA.getValue()]

Mask to identify the bandpass spectra that should be discarded when interpolating bandpasses. Flagged rows that are not masked by this integer will be used in the interpolation and propagated as informative flags. For example, if you want to ignore rows that have the flag 1,4 or 16 set you specify a rowFlagMask=21. By default, the rowFlagMask is set to discard rows with the flags RowMask.IGNORE_DATA or RowMask.BAD_DATA set. If in a given bandpass group all spectra are flagged (i.e. the configured bits set) no bandpass division is carried through for those spectra. Accordingly, a quality flag is raised (intensityCalibrationFlag). If some of the bandpass spectra are discarded a badHcDataDiscarded-flag is raised. Associated entries are added to the meta data of the htp. If you want to disregard rows with any flag set specify a rowFlagMask = 0. If you want to include all rows irrespective of the row flag set specify a negative rowFlagMask.

See also


  • 2011-07-17 - Melchior: : History added.
  • 2011-08-14 - Melchior: : Renamed to DoFluxHotColdTask.
  • 2012-01-16 - Melchior: : Mixer current deviation row flag bit set to RowMask.MCD_HOT.getIndex().
  • 2012-01-16 - Melchoir: : No bandpass row flag bit set to RowMask.NO_HOT_COLD.getIndex()
  • 2012-04-09 - Melchior: : HIFI-4641
  • 2012-04-13 - Melchior: : HIFI-4769