1.35. doCleanUp

Full Name: herschel.hifi.pipeline.generic.DoCleanUpTask
Alias: doCleanUp
Type: Java Task - Java Task
Import: from herschel.hifi.pipeline.generic import DoCleanUpTask

HIFI/Pipeline/Level 2 Pipeline


Cleans up the timeline product by removing data no longer needed after running the level1 pipeline.

Furthermore, it packs all the data belonging to the same frequency group to on single dataset. Here, the temporal order is conserved.


Example 1: In HIPE:
cleanUp(htp=htp) # apply just the defaults
params = herschel.hifi.pipeline.generic.PipelineConfiguration.getConfig(htp)
cleanUp(htp=htp, params=params) # apply the defaults set for the pipeline
cleanUp(htp=htp, mergeDatasets=False) # do not merge the datasets belonging to the same group
cleanUp(htp=htp, retain='science') # retain all the science data in the timeline product, ON and OFF's.

API details


HifiTimelineProduct htp [INOUT, MANDATORY, default=no default value]

The timeline product (observation) to be passed to the module.

PipelineConfiguration params [INPUT, OPTIONAL, default=no default value]

Pipeline configuration parameters that can be passed to the task.

Boolean ignore [INPUT, OPTIONAL, default=no default value]

Flag to indicate whether the execution of the module should be ignored.

String retain [INPUT, OPTIONAL, default=scienceOn]

String parameter specifying which data to retain while cleaning up. Options: 'scienceOn' (default), 'science'.

Boolean mergeDatasets [INPUT, OPTIONAL, default=True]

If set to true datasets that belong to the same "group" are merged. The data typically is structured into groups where the groups are given an observing mode specific meaning:

  • For spectral scans, all the scans with the same LO tuning form a group.

  • For raster maps, all the scans that refer to the same raster point ("rasterColumnNum", "rasterLineNum") form a group.

  • For otf maps, all the scans in the same line ("scanLineNum") form a group.

  • For cross maps, all the scans associated with the same point within the cross ("customMapPointNum") form a group.

In addition, scans that belong to the same group but are not subsequent in observation time will not be merged.

Integer datasetSize [INPUT, OPTIONAL, default=100]

Maximum size of the datasets (number of rows) included in the result timeline product. Note that datasets are not teared in the sense that datasets are not merged if the resulting size is not within the specified size. Furthermore, if a given input dataset is already above the limit this will be moved to the output without change.

See also


  • 2011-07-17 - Melchior: : History added
  • 2011-08-14 - Melchior: : Renamed to DoCleanUpTask