1.105. PipelineTask

Full Name: herschel.hifi.pipeline.PipelineTask
Alias: PipelineTask
Type: Jython Task -
Import: from herschel.hifi.pipeline import PipelineTask
Category:

HIFI/Pipeline

Description

Task to wrap pipeline algorithms

Example

Example 1: PipelineTask
 
<pre>
from herschel.hifi.pipeline.PipelineTask import *
pipelineTask=PipelineTask()
#
# Example 1, sgenerate an observation context import scratch
# the default db = retrieved import the property: var.database.devel, check with propgen.
#
obs = pipelineTask(obsid=268435583, db="ds1@iccdb.sron.rug.nl 0 READ")
#
# Note: you can use the dataset inspector to inspect the outcoming observation context
#
#
# Example 2, like ex. 1, now use of obsMode parameter,
# for ILT data you have to provide the obsMode name because the data itself does not have it!#
#
obs = pipelineTask(obsid=268516902, db="ilt_fm_5_prop@iccdb1.sron.rug.nl 0 READ", obsMode="HifiPointModeLoadChop",gui=1)
#
#
# Example 3, like ex. 1,  now use of apids parameter,
# pass an apid for level 1 processing (level 0 always process all data / 4 apids, is not yet optional)
#
obs = pipelineTask(obsid=268516902, apids=["1030"], obsMode="HifiPointModeLoadChop")
#
#
# Example 4, like ex. 1 now using your own algorithm for wbs /hrs or generic:
# one can look / edit / use the algo"s by loading the following scripts into HIPE:
# The WBS pipeline algo is found in the {build_root}/lib/herschel/hifi/pipeline/wbs/WbsPipelineAlgo.py
# The HRS pipeline algo is found in the {build_root}/lib/herschel/hifi/pipeline/hrs/HrsPipelineAlgo.py
# The Generic pipeline algois found in the {build_root}/lib/herschel/hifi/pipeline/generic/GenericPipelineAlgo.py
#
obs = pipelineTask(obsid=268435583, db="ds1@iccdb.sron.rug.nl 0 READ", wbsAlgo=myWbsAlgo, hrsAlgo=myHrsAlgo,level1Algo=mylevel1Algo)
#
#
#
# Example 6, reprocess an existing observation context
# only reprocess hrs/wbs/generic pipelines:
pipelineTask(obs=obs)
#
#
# Example 7: like ex. 1-6, now only generate level0
# Note that for level o processing all available apids are processed, i.e. it
# does not react on the apids parameter!
#
obs = pipelineTask(obsid=268516902,  db="ilt_fm_5_prop@iccdb1.sron.rug.nl 0 READ",obsMode="HifiPointModeLoadChop", upToLevel=0)
#
#similar:
#
obs = pipelineTask(obsid=268516902, db="ilt_fm_5_prop@iccdb1.sron.rug.nl 0 READ", obsMode="HifiPointModeLoadChop", upToLevel=1)
#
#
# Example 8: like ex. 1-6, now reprocess all levels incl. level 0
# Note that all calibration and other products (auxilary etc) are not replaced.
#
pipelineTask(obs=obs, fromLevel=-1)
#
#
#
# Example 9: like 1 now providing a tm_version_map
# in case the selecated database requires a different mission phase than defined in your
# binstruct property: hcss.binstruct.mib.pal.tm_version_map (hifi default = "ilt-fm")
#
obs = pipelineTask(obsid=268435583, db="ilt_par_7_prop@iccdb.sron.rug.nl 0 READ", tmVersion="ilt-par")
#
#
# Example 10:
# clear CachedStoreHandler to avoid a block by this Handler due to none closed stores
# note: closes ALL stores availbale in this cache and thus might affect other applications
# running in this session.
#
obs = pipelineTask(obsid=268435583, db="ds1@iccdb.sron.rug.nl 0 READ", upToLevel=0.5, clearCachedStoreHandler=1)
#
#
# Example 11: like 1 now providing your own palStore
# Note that the pipeline tasks needs the hifi-cal@iccdb pool (or simple or lstore copy) to generate the level0 product
# so one should always include it:
#
myStore = herschel.ia.pal.ProductStorage()
myStore.register(SimplePool.getInstance("my-dedicated-pool-to-be-applied")
#
# the next are registered to retrieve your calibration data:
# local:
myStore.register(SimplePool.getInstance("hifi-cal")) # or hifi-cal_dev for ex.
# or, server pool:
import herschel.share.util.Configuration
Configuration.setProperty("hifical", self.cal)
myStore.register(DbPool.getInstance("hifical"))
#
obs = pipelineTask(obsid=268435583, db="ds1@iccdb.sron.rug.nl 0 READ", palStore = myStore)
#
#
# to browse the products in the pool:
def startProductBrowser(storage):
    from herschel.ia.pal.browser import ProductBrowser
    result = ProductBrowser.browseProduct(storage)
    return result
#
res = startProductBrowser(myStore)
</pre>

API Summary

Properties
array(String) apids [IN, OPTIONAL, default=default=all apids]
ObservationContext obsOut [OUT, OPTIONAL, default=No default value]
ObservationContext obs [IO, OPTIONAL, default=(alternative to obsid)]
Long obsid [IN, OPTIONAL, default=No default value]
String db [IN, OPTIONAL, default=default= Configuration.getProperty("var.database.devel")]
String cal [IN, OPTIONAL, default=default= Configuration.getProperty("hcss.binstruct.mib.pal.database")]
String obsMode [IN, OPTIONAL, default=default=self determined]
ProductStorage palStore [IN, OPTIONAL, default=default= ProductStorage:"hifi-pipeline"]
PyFunction wbsAlgo [INPUT, OPTIONAL, default=default=]
PyFunction hrsAlgo [INPUT, OPTIONAL, default=default=]
PyFunction genericAlgo [INPUT, OPTIONAL, default=default=]
PyFunction level1Algo [INPUT, OPTIONAL, default=default=]
PyFunction level2Algo [INPUT, OPTIONAL, default=default=]
Boolean reprocessAllLevels [IN, OPTIONAL, default=default=False.]
Boolean gui [IN, OPTIONAL, default=default=False.]

API details

Properties

array(String) apids [IN, OPTIONAL, default=default=all apids]
 

the apid that have to be processed

ObservationContext obsOut [OUT, OPTIONAL, default=No default value]
 

The Observation Context processed until level 2 when the input is just: (obsid).

ObservationContext obs [IO, OPTIONAL, default=(alternative to obsid)]
 

No default value, null allowed. The Observation Context processed , in is none, level0 or level0_5, out is level1 and level2

Long obsid [IN, OPTIONAL, default=No default value]
 

Alternative input for the observation to be processed. ObservationContext created on the fly

String db [IN, OPTIONAL, default=default= Configuration.getProperty("var.database.devel")]
 

the database containing the specified obsid.

String cal [IN, OPTIONAL, default=default= Configuration.getProperty("hcss.binstruct.mib.pal.database")]
 

the remote pal database containing MIB and the calibration products.

String obsMode [IN, OPTIONAL, default=default=self determined]
 

the observation mode to be used in the pipline. Overwrite the obsMode present in the ObservationContext

ProductStorage palStore [IN, OPTIONAL, default=default= ProductStorage:"hifi-pipeline"]
 

The store where the result will be stored and where the calibration product can be retrieved.

PyFunction wbsAlgo [INPUT, OPTIONAL, default=default=]
 

The algorithm for the wbs pipeline.

PyFunction hrsAlgo [INPUT, OPTIONAL, default=default=]
 

The algorithm for the hrs pipeline.

PyFunction genericAlgo [INPUT, OPTIONAL, default=default=]
 

The algorithm for the generic pipeline.

PyFunction level1Algo [INPUT, OPTIONAL, default=default=]
 

The algorithm for the level1 pipeline.

PyFunction level2Algo [INPUT, OPTIONAL, default=default=]
 

The algorithm for the level2 pipeline.

Boolean reprocessAllLevels [IN, OPTIONAL, default=default=False.]
 

In case you pass an (adopted) observation context and you want to reprocess level 0 and following switch this one to TRUE

Boolean gui [IN, OPTIONAL, default=default=False.]
 

If the progress bar is displayed or not.

History

  • 12-Feb-08 PZ Add JTAGs for this task
  • 07-Mar-08 PZ Add update example for this task
  • 28-March-2008 PZ make pipelineTask with API for CS