HIFI Pipeline Specification

Document: ICC2008-154

Custodian: Sylvie F Beaulieu

co-Editor: Carolyn McCoey

HIFI ICC

Revision History
Revision 0.1 1 February 2008
Outline on Dokuwikki
Revision 0.6 30 July 2008
Minor Updates
Revision 0.7 13 Nov 2009
Updates to all sections
Revision 0.8 24 Nov 2009
Updates to HRS pipeline flow diagram, clarification of MkWbsZero, change to manual properties for new documentation framework
Revision 0.9 28 July 2010
Updates to Level-1 and -2 pipeline flow diagram, split "Generic Pipeline" chapter into Level-1 and Level-2 Pipeline chapters, rearrangement and additions of new sections to reflect status of pipeline.
Revision 1.0 20 Sept 2011
Updates to code examples.
Revision 1.1 March-May 2013
Added missing tasks, and removed tasks no longer applied in current pipeline. Added section Level 2.5, with subsections 'doDeconvolution' and 'doGridding".
Revision 1.2 June 2013
Continuing adding missing information for some tasks. Document ungergoing major review. Stay tune...
Revision 1.3 July 2013
Continuing adding missing information for some tasks. Correcting typos and editorial style formating.
Revision 1.4 27 September 2013
Added doFold task in Level 2 Pipeline section.
Revision 1.5 9 October 2013
Switched from olink to ulink.
Revision 1.6 12 November 2013
Added tasks to Level 2.5: DoMainBeamTemp (moved from Level 2), PolarPair, DoFold (moved from Level 2), MkRms.
Revision 1.7 8 January 2014
Corrected bad links and validation errors.
Revision 1.8 4 March 2014
Removal of doRadialVelocity (Level 1 task), superseded by doVelocityCorrection.
Revision 1.9 15 August 2014
Documentation of doBadLo step in Level 0 pipeline.
Revision 1.10 27 March 2015
Restructuring of document to make separate chapters for Levels 1, 2 and 2.5 pipelines. Also fixing broken links, removal of unnecessary sections and numerous small corrections
Revision 1.11 June-July 2015
Updates to several tasks and refreshed figures.
Revision 1.12 October 2015
Updated text in some tasks and updated pipeline flow diagrams figures from Level 0 to 2.0.
Revision 1.13 20 November 2015
Updated text in some tasks, specifically the flags information.

Table of Contents

1. Introduction
2. Level 0 Pipeline
2.1. Introduction to the Level 0 Pipeline
2.2. doBadLo
2.3. doHkCheck
2.4. doPointing
2.5. doTimeCorr
2.6. doUplink
2.7. cleanDF
3. HRS Pipeline
3.1. Introduction to HRS pipeline
3.2. doHrsSubbands
3.3. doHrsOffsetPow
3.4. doHrsNorm
3.5. doHrsQDCFast
3.6. doHrsQDCFull
3.7. doHrsPowCorr
3.8. doHrsWindow
3.9. doHrsSymm
3.10. doHrsFFT
3.11. doHrsSmooth
3.12. doHrsFreq
3.13. doHrsCorrSP
3.14. doHrsCutBandEdges
4. WBS Pipeline
4.1. Introduction to the WBS pipeline
4.2. doWbsScanCount
4.3. mkWbsBadPixels
4.4. doWbsCrFlag
4.5. doWbsBadPixels
4.6. doWbsDark
4.7. doWbsNonLin
4.8. mkWbsZero
4.9. doWbsZero
4.10. mkWbsFreq
4.11. mkWbsFreqFromHrs
4.12. doWbsFreq
4.13. mkWbsFluxAtten
4.14. doWbsSubbands
5. Level 1 Pipeline
5.1. checkDataStructure
5.2. checkFreqGrid
5.3. checkPhases
5.4. doFilterLoads
5.5. mkFluxHotCold
5.6. doChannelWeights
5.7. doRefSubtract
5.8. mkOffSmooth
5.9. doOffSubtract
5.10. doFluxHotCold
5.11. doVelocityCorrection
5.12. doHebCorrection
5.13. mkFlagSummary
6. Level 2 Pipeline
6.1. mkRef
6.2. doCleanUp
6.3. doAntennaTemp
6.3.1. doMainBeamTemp
6.4. mkSidebandGain
6.5. doSidebandGain
6.6. undoSidebandGain
6.7. convertFrequency
6.8. mkFreqGrid
6.9. doFreqGrid
6.10. doHpbw
6.11. doAvg
6.12. checkPlatforming
6.13. mkRms
6.14. doChannelFlags
6.15. mkUncertaintyTable
7. Level 2.5 Pipeline
7.1. doStitch
7.2. mkRms
7.3. doFold
7.4. doGridding
7.5. doDeconvolution
8. Appendix:
8.1. Introduction to the Generic Pipeline
8.2. Configuration of the Generic Pipeline
8.3. Standard Observing Modes
8.4. Observing Modes Groups
8.5. Some Details on Spectrum Data
8.6. Initialization of Chopper Positions
8.7. Pipeline Modules

List of Figures

5.1. Summary Table before Applying the Module
5.2. Summary Table after Applying the Module
5.3. A dataset of the FreqRanges-product corresponding to the single group.
5.4. Summary Table after Applying the Module
6.1. Sample table with forward efficiencies
6.2. Sample table specifying the LO dependent gain levels.
6.3. Sample table specifying the IF dependent gain shape.
6.4. Calibration product containing the frequency grids created by the MkFreqGrid task.
7.1. An LSB emission line plotted on the LSB frequency scale.
7.2. An LSB emission line plotted on the USB frequency scale.