HIFI data reduction tools (expected updates for HIPE 6):

• Pipeline will no longer attempt to access HSA when processing using calibration included in ObservationContext As a consequence, the pipeline must be configured with ConfigureHifiPipeline to use a different location (e.g., HSA) for the calibration/auxilliary and to save each level immediately after processing
• Bug fixed: in HIPE 5.1 if aux=True is specified the uplink product is not updated and pipeline fails
• New algorithm for load chop and frequency switch data to avoid ‘CCD scratches’ and improve baselines
• Possible to create a cal pool from HSA with HifiCalTask - efficient way to reprocess observations with updated calibration
• It will be possible to configure the pipeline to omit steps or use non-default task parameters in Level 1 and 2 pipelines. Proto-type already in HIPE 6 developer builds but has issues…
• Bug fix: Updated calibration in HSA appears as proprietary to new non super-users. To be fixed by HSA in 6.0 or 7.0
• New trend product: rms. This is calculated for the full bandwidth with no line masking
• Appropriate smoothing of OFF position in Load Chop with OFF spectra. Expected to remove many of the ripple issues seen using this mode with reprocessing.
• Removal of 'scratches' in calibrated data in FSW and LChop observations (notably WBS-V) by applying the intensity calibration before the reference position is smoothed and subtracted removes the ‘scratches’. This also improves baseline noise level.

-- AnthonyMarston - 31 Jan 2011