Unhide doPipelineConfiguration by clicking on the arrow, all of the steps in the levels 1 and 2 pipelines are displayed.

If an ObservationContext is loaded into the hifiPipeline GUI then doPipelineConfiguration will read the observing mode from the metadata and the steps that are not applicable for that observing mode will be greyed out.

By unchecking the tick box by each pipeline step name you can omit that step when you run the pipeline.

By unhiding the panels in the GUI you can see the parameters used in each step with the default settings for that observing mode, which you can modify as you prefer. Tooltips found by hovering over parameter names give indications of the options available to you, while more information is available in Chapter 4 and the pipeline steps are described in detail in the Pipeline Specification Document>.

If the observing mode is a mapping mode then you can also modify the doGridding panel beneath. The doGridding task is used to grid data into cubes, see Chapter 12 for usage, and the doGridding panel is identical to the GUI of the doGridding task with the exception that anything related to spectrometer subbands is omitted; that is, the choice of subbands and the number of channels to be included per subband. This is because doGridding reads this information from the HTP but here an ObservationContext is passed. This panel gives you the option to specify how maps are created for all backends in the observation, which is more efficient than creating cubes individually from HTP

The command to run the pipeline as you have configured it is echoed to the console, allowing you to transfer your customised pipeline command to a script to run on other observations of the same observing mode: the configuration is dependant on the observing mode and a different configuration must be created for a different mode.

You can also run doPipelineConfiguration from the command line as:

config = doPipelineConfiguration (observingMode="HifiPointModeFastDBS") # It is mandatory to specify the observing mode
obs_1 = hifiPipeline(obs=obs, params=config)

It is recommended to use the pipeline GUI to configure the pipeline because the syntax to modify pipeline steps is rather unwieldy, see for example the configuration required to omit the doAverage step for a DBS Fast Chop observation:

config = doPipelineConfiguration(observingMode='HifiPointModeFastDBS', params={'doCleanUp': {'mergeDatasets': 'True', \
'ignore': 'False', 'datasetSize': 100}, 'mkFreqGrid': {'ignore': 'False'}, 'doAvg': {'preserveGroups': 'True', \
'return_single_ds': 'False', 'ignore': 'True'}, 'doAntennaTemp': {'ignore': 'False', 'useCalTree': 'False'}, \
'doVelocityCorrection': {'ignore': 'False'}, 'doOffSubtract': {'mode': u'addOff', 'ignore': 'False', 'useCalTree': \
'False', 'ignoreFlagged': 'False', 'interpolator': u'trunc_linear', 'validatorTolerance': 0.025}, 'doChannelWeights': \
{'ignore': 'False', 'extrapolate': 'False', 'width': 20, 'definition': u'radiometric', 'interpolator': u'TruncuatedLinear', \
'smoothing': u'Gaussian'}, 'doFluxHotCold': {'ignore': 'False', 'useCalTree': 'False', 'interpolator': u'trunc_linear', \
'validatorTolerance': 0.025}, 'doFreqGrid': {'ignore': 'False', 'scheme': u'euler'}, 'mkFluxHotCold': {'ignore': 'False', \
'useCalTree': 'False', 'indicator': u'buffer', 'isFSwitch': 'False', 'isABBA': 'True', 'validatorTolerance': 0.025}, \
'doFilterLoads': {'ignore': 'True', 'filterMethod': u'True', 'scheme': u'scheme1'}, 'checkFreqGrid': {'ignore': 'False', \
'tolerance': 5.0, 'isFSwitch': 'False'}, 'doRefSubtract': {'ignore': 'False', 'useCalTree': 'False', 'indicator': u'pattern', \
'isABBA': 'False', 'offStartsWithOpposite': 'True', 'startWithRef': 'False', 'validatorTolerance': 0.025}, 'mkOffSmooth': \
{'mode': u'filter', 'ignore': 'True', 'widthUnit': u'MHz', 'useCalTree': 'False', 'width': 10.0, 'filter': u'Gaussian'}, \
'doSpectrumStatistics': {'ignore': 'False', 'stitchedSegments': 'False'}, 'doSidebandGain': {'ignore': 'False'}, \
'mkSidebandGain': {'ignore': 'False'}})

Edit the pipeline script and save it. You can pass the algorithm to the pipeline by specifying the location of the script in the appropriate "algo" field of the GUI, you can browse for your saved script by clicking on folder icon.

You can also pass the path to the saved algorithm script in the command line, e.g.,

obs = hifiPipeline(obs=obs, level2Algo="/home/me/MyScripts/MyLevel2PipelineAlgo.py")

Alternatively, you can make use of the fact that the pipeline algorithm scripts define a function ( def TaskName (Parameters):) and pass the function name to the pipeline. To do this you must first compile your new pipeline algorithm by running the script with the double arrows (>>) in the HIPE toolbar. In the example below the function has been called MyLevel1Algo:

# Include your own algorithm for the Level 1 pipeline, for all spectrometers, from Level 0 to 1:
obs = hifiPipeline(obs=obs, fromLevel=0, upToLevel=1, level1Algo=MyLevel1Algo)
#

Running the pipeline, or one part of the pipeline, step by step allows you to inspect the results of each step and change the default parameters of the pipeline. If you wish to create your own algorithm, which must be written in jython, for a part of the pipeline, then this will likely be your first step.

It is not expected that there will be much need to customise the spectrometer pipelines (up to Level 0.5) and indeed there are only a few steps of the spectrometer pipelines that have some options. It is more likely that you may wish to play with how off and reference spectra are subtracted in the Level 1 pipeline, although it is expected that the default settings should work well.

To step through the pipeline you must work directly on the appropriate level HifiTimeLine (HTP - the dataset containing all the spectra, including calibration spectra, made during an observation for a given spectrometer). So the first thing you must do is extract the HTP you want to work on from your ObservationContext:

When you select an HTP in the Variables view in HIPE you will notice that many tasks with names like DoWbsDark, mkFreqGrid. These are the names of all of the steps in the HIFI pipeline; mk... signifies a step where a calibration product is being made, Do... is a step where a calibration is applied. You can step through the pipeline using these tasks or (more efficiently) use and modify the scripts that are supplied with the software from the Pipeline menu in HIPE or in the scripts/hifi/Pipeline directory in the installation directory of HIPE. To learn about moving about and running scripts, see the HIPE Owners Guide

For information on the steps of each level of the pipeline (their names, the order to run them in, and what options you can change) see the Pipeline Specification Document>.