Chapter 3. HIFI Products Description

Table of Contents

3.1. HIFI observational products
3.1.1. HIFI Observation Context metadata
3.1.2. HIFI Level 0, Level 0.5, Level 1 and level 2 contexts
3.1.3. HIFI Level 2 context
3.1.4. HIFI Level 2.5 context
3.2. HIFI calibration products
3.2.1. HIFI calibration products 1: Predefined calibration products
3.2.2. HIFI calibration products 2: Calibration products derived from the HifiTimelineProduct
3.3. HIFI Quality products
3.4. HIFI Trend Analysis products

3.1. HIFI observational products

3.1.1. HIFI Observation Context metadata

The HIFI Observation Context Product contains a list of metadata keywords which give the majority of the parameters that identify the observation. These are examples for the three HIFI modes:

3.1.1.1. ObservationContext for HIFI Single Point

Table 3.1. Example of the metadata keywords contained in a HIFI Single Point observation.

StringParameter  type (description="Product Type Identification")
StringParameter  creator (description="Generator of this product")
DateParameter  creationDate (description="Creation date of this product")
StringParameter  description (description="Name of this product")
StringParameter  instrument (description="Instrument attached to this product")
StringParameter  modelName (description="Model name attached to this product")
DateParameter  startDate (description="Start date of the observation")
DateParameter  endDate (description="End date of the observation")
StringParameter  formatVersion (description="Version of product format")
StringParameter  obsState (description="One of CREATED, LEVEL0_PROCESSED, LEVEL0_5_PROCESSED, LEVEL1_PROCESSED, LEVEL2_PROCESSED, LEVEL2_5_PROCESSED, LEVEL3_PROCESSED")
LongParameter  obsid (description="Observation identifier")
LongParameter  odNumber (description="Operational day number")
StringParameter  cusMode (description="CUS observation mode")
StringParameter  instMode (description="Instrument Mode")
StringParameter  obsMode (description="Observation mode name")
StringParameter  processingMode (description="Processing mode selected to execute the pipeline")
StringParameter  observer (description="Observer name")
StringParameter  proposal (description="Proposal name")
StringParameter  pointingMode (description="Pointing mode")
DateParameter  slewTime (description="Predicted start time for slew before the observation")
StringParameter  origin (description="Site that created the product")
StringParameter  aorLabel (description="AOR Label as entered in HSpot")
StringParameter  aot (description="AOT Identifier")
DoubleParameter  equinox (description="Equinox of celestial coordinate system")
StringParameter  missionConfig (description="Mission configuration")
StringParameter  object (description="Target name")
StringParameter  raDeSys (description="Coordinate reference frame for the RA and DEC")
DoubleParameter  pmRA (description="Target's proper motion RA (arcsec/yr) as given by the observer", quantity="arcsec a-1")
DoubleParameter  pmDEC (description="Target's proper motion Dec (arcsec/yr) as given by the observer", quantity="arcsec a-1")
DoubleParameter  raNominal (description="Requested Right Ascension of pointing", quantity="deg")
DoubleParameter  decNominal (description="Requested Declination of pointing", quantity="deg")
DoubleParameter  ra (description="Average of ra in level 2 datasets", quantity="deg")
DoubleParameter  dec (description="Average of dec in level 2 datasets", quantity="deg")
DoubleParameter  posAngle (description="Spacecraft pointing Position angle", quantity="deg")
StringParameter  telescope (description="Name of telescope")
StringParameter  velocityDefinition (description="The velocity definition and frame")
DoubleParameter  radialVelocity (description="Spacecraft velocity along the l-of-s of the telescope wrt the LSR", quantity="km s-1")
StringParameter  calVersion (description="HIFI calibration version")
StringParameter  Band (description="Active band")
StringParameter  attitudeQuaternion (description="Pointing product quaternion applied")
DoubleParameter  tmbReference (description="Temperature (main beam) at noise reference frequency", quantity="K")
DoubleParameter  noiseRefFrequency (description="Noise reference frequency", quantity="GHz")
DoubleParameter  totNoiseEfficiency (description="Total noise efficiency", quantity="%")
DoubleParameter  driftNoiseContrib (description="Drift noise contribution", quantity="%")
StringParameter  orbitEphemerisSourceFile (description="Name of the file from where data was extracted")
DoubleParameter  solarAspectAngleMean (description="Mean Solar Aspect Angle", quantity="degrees")
DoubleParameter  solarAspectAngleRms (description="RMS Solar Aspect Angle", quantity="degrees")
BooleanParameter  hrsHscience (description="Science data are obtained with HRS-H")
BooleanParameter  hrsVscience (description="Science data are obtained with HRS-V")
BooleanParameter  wbsHscience (description="Science data are obtained with WBS-H")
BooleanParameter  wbsVscience (description="Science data are obtained with WBS-V")
DoubleParameter  loFrequencyRequest (description="User requested local oscillator frequency", quantity="GHz")
BooleanParameter  gyroAttSuspicious (description="Suspicious quality of the attitude reconstruction.")
DoubleParameter  loFreqAvg (description="Average LO frequency Doppler-corrected to freqFrame (SPECSYS)", quantity="GHz")
DoubleParameter  obsFreqUsbMin (description="Observed min frequency for USB in freqFrame (SPECSYS)", quantity="GHz")
DoubleParameter  obsFreqUsbMax (description="Observed max frequency for USB in freqFrame (SPECSYS)", quantity="GHz")
DoubleParameter  obsFreqLsbMin (description="Observed min frequency for LSB in freqFrame (SPECSYS)", quantity="GHz")
DoubleParameter  obsFreqLsbMax (description="Observed max frequency for LSB in freqFrame (SPECSYS)", quantity="GHz")
DoubleParameter  loFrequency (description="Actual local oscillator frequency", quantity="GHz")
DoubleParameter  rmsMinUsb (description="Rms Noise USB at minimum bandwidth H and V polarizations averaged", quantity="K")
DoubleParameter  rmsNativeUsb (description="Rms Noise USB rescaled to native WBS resolution H and V polarizations averaged", quantity="K")
DoubleParameter  rmsMaxUsb (description="Rms Noise USB at maximum bandwidth H and V polarizations averaged", quantity="K")
DoubleParameter  rmsMinLsb (description="Rms Noise LSB at minimum bandwidth H and V polarizations averaged", quantity="K")
DoubleParameter  rmsNativeLsb (description="Rms Noise LSB rescaled to native WBS resolution H and V polarizations averaged", quantity="K")
DoubleParameter  rmsMaxLsb (description="Rms Noise LSB at maximum bandwidth H and V polarizations averaged", quantity="K")
                             


3.1.1.2. ObservationContext for HIFI Mapping

Table 3.2. Example of the metadata keywords contained in a HIFI Mapping observation.

StringParameter  type (description="Product Type Identification")
StringParameter  creator (description="Generator of this product")
DateParameter  creationDate (description="Creation date of this product")
StringParameter  description (description="Name of this product")
StringParameter  instrument (description="Instrument attached to this product")
StringParameter  modelName (description="Model name attached to this product")
DateParameter  startDate (description="Start date of the observation")
DateParameter  endDate (description="End date of the observation")
StringParameter  formatVersion (description="Version of product format")
StringParameter  obsState (description="One of CREATED, LEVEL0_PROCESSED, LEVEL0_5_PROCESSED, LEVEL1_PROCESSED, LEVEL2_PROCESSED, LEVEL2_5_PROCESSED, LEVEL3_PROCESSED")
LongParameter  obsid (description="Observation identifier")
LongParameter  odNumber (description="Operational day number")
StringParameter  cusMode (description="CUS observation mode")
StringParameter  instMode (description="Instrument Mode")
StringParameter  obsMode (description="Observation mode name")
StringParameter  processingMode (description="Processing mode selected to execute the pipeline")
StringParameter  observer (description="Observer name")
StringParameter  proposal (description="Proposal name")
StringParameter  pointingMode (description="Pointing mode")
DateParameter  slewTime (description="Predicted start time for slew before the observation")
StringParameter  origin (description="Site that created the product")
StringParameter  aorLabel (description="AOR Label as entered in HSpot")
StringParameter  aot (description="AOT Identifier")
DoubleParameter  equinox (description="Equinox of celestial coordinate system")
StringParameter  missionConfig (description="Mission configuration")
StringParameter  object (description="Target name")
StringParameter  raDeSys (description="Coordinate reference frame for the RA and DEC")
DoubleParameter  pmRA (description="Target's proper motion RA (arcsec/yr) as given by the observer", quantity="arcsec a-1")
DoubleParameter  pmDEC (description="Target's proper motion Dec (arcsec/yr) as given by the observer", quantity="arcsec a-1")
DoubleParameter  raNominal (description="Requested Right Ascension of pointing", quantity="deg")
DoubleParameter  decNominal (description="Requested Declination of pointing", quantity="deg")
DoubleParameter  ra (description="Average of ra in level 2 datasets", quantity="deg")
DoubleParameter  dec (description="Average of dec in level 2 datasets", quantity="deg")
DoubleParameter  posAngle (description="Spacecraft pointing Position angle", quantity="deg")
StringParameter  telescope (description="Name of telescope")
StringParameter  velocityDefinition (description="The velocity definition and frame")
DoubleParameter  radialVelocity (description="Spacecraft velocity along the l-of-s of the telescope wrt the LSR", quantity="km s-1")
StringParameter  calVersion (description="HIFI calibration version")
StringParameter  Band (description="Active band")
StringParameter  attitudeQuaternion (description="Pointing product quaternion applied")
DoubleParameter  pattAngle (description="Spectral map rotation angle", quantity="degrees")
DoubleParameter  tmbReference (description="Temperature (main beam) at noise reference frequency", quantity="K")
DoubleParameter  noiseRefFrequency (description="Noise reference frequency", quantity="GHz")
DoubleParameter  totNoiseEfficiency (description="Total noise efficiency", quantity="%")
DoubleParameter  driftNoiseContrib (description="Drift noise contribution", quantity="%")
StringParameter  orbitEphemerisSourceFile (description="Name of the file from where data was extracted")
DoubleParameter  solarAspectAngleMean (description="Mean Solar Aspect Angle", quantity="degrees")
DoubleParameter  solarAspectAngleRms (description="RMS Solar Aspect Angle", quantity="degrees")
BooleanParameter  hrsHscience (description="Science data are obtained with HRS-H")
BooleanParameter  hrsVscience (description="Science data are obtained with HRS-V")
BooleanParameter  wbsHscience (description="Science data are obtained with WBS-H")
BooleanParameter  wbsVscience (description="Science data are obtained with WBS-V")
DoubleParameter  loFrequencyRequest (description="User requested local oscillator frequency", quantity="GHz")
BooleanParameter  gyroAttSuspicious (description="Suspicious quality of the attitude reconstruction.")
DoubleParameter  loFreqAvg (description="Average LO frequency Doppler-corrected to freqFrame (SPECSYS)", quantity="GHz")
DoubleParameter  obsFreqUsbMin (description="Observed min frequency for USB in freqFrame (SPECSYS)", quantity="GHz")
DoubleParameter  obsFreqUsbMax (description="Observed max frequency for USB in freqFrame (SPECSYS)", quantity="GHz")
DoubleParameter  obsFreqLsbMin (description="Observed min frequency for LSB in freqFrame (SPECSYS)", quantity="GHz")
DoubleParameter  obsFreqLsbMax (description="Observed max frequency for LSB in freqFrame (SPECSYS)", quantity="GHz")
DoubleParameter  loFrequency (description="Actual local oscillator frequency", quantity="GHz")
DoubleParameter  rmsMinUsb (description="Rms Noise USB at minimum bandwidth H and V polarizations averaged", quantity="K")
DoubleParameter  rmsNativeUsb (description="Rms Noise USB rescaled to native WBS resolution H and V polarizations averaged", quantity="K")
DoubleParameter  rmsMaxUsb (description="Rms Noise USB at maximum bandwidth H and V polarizations averaged", quantity="K")
DoubleParameter  rmsMinLsb (description="Rms Noise LSB at minimum bandwidth H and V polarizations averaged", quantity="K")
DoubleParameter  rmsNativeLsb (description="Rms Noise LSB rescaled to native WBS resolution H and V polarizations averaged", quantity="K")
DoubleParameter  rmsMaxLsb (description="Rms Noise LSB at maximum bandwidth H and V polarizations averaged", quantity="K")
                             


3.1.1.3. ObservationContext for HIFI Spectral Scan

Table 3.3. Example of the metadata keywords contained in a HIFI Spectral Scan.

StringParameter  type (description="Product Type Identification")
StringParameter  creator (description="Generator of this product")
DateParameter  creationDate (description="Creation date of this product")
StringParameter  description (description="Name of this product")
StringParameter  instrument (description="Instrument attached to this product")
StringParameter  modelName (description="Model name attached to this product")
DateParameter  startDate (description="Start date of the observation")
DateParameter  endDate (description="End date of the observation")
StringParameter  formatVersion (description="Version of product format")
StringParameter  obsState (description="One of CREATED, LEVEL0_PROCESSED, LEVEL0_5_PROCESSED, LEVEL1_PROCESSED, LEVEL2_PROCESSED, LEVEL2_5_PROCESSED, LEVEL3_PROCESSED")
LongParameter  obsid (description="Observation identifier")
LongParameter  odNumber (description="Operational day number")
StringParameter  cusMode (description="CUS observation mode")
StringParameter  instMode (description="Instrument Mode")
StringParameter  obsMode (description="Observation mode name")
StringParameter  processingMode (description="Processing mode selected to execute the pipeline")
StringParameter  observer (description="Observer name")
StringParameter  proposal (description="Proposal name")
StringParameter  pointingMode (description="Pointing mode")
DateParameter  slewTime (description="Predicted start time for slew before the observation")
StringParameter  origin (description="Site that created the product")
StringParameter  aorLabel (description="AOR Label as entered in HSpot")
StringParameter  aot (description="AOT Identifier")
DoubleParameter  equinox (description="Equinox of celestial coordinate system")
StringParameter  missionConfig (description="Mission configuration")
StringParameter  object (description="Target name")
StringParameter  raDeSys (description="Coordinate reference frame for the RA and DEC")
DoubleParameter  pmRA (description="Target's proper motion RA (arcsec/yr) as given by the observer", quantity="arcsec a-1")
DoubleParameter  pmDEC (description="Target's proper motion Dec (arcsec/yr) as given by the observer", quantity="arcsec a-1")
DoubleParameter  raNominal (description="Requested Right Ascension of pointing", quantity="deg")
DoubleParameter  decNominal (description="Requested Declination of pointing", quantity="deg")
DoubleParameter  ra (description="Average of ra in level 2 datasets", quantity="deg")
DoubleParameter  dec (description="Average of dec in level 2 datasets", quantity="deg")
DoubleParameter  posAngle (description="Spacecraft pointing Position angle", quantity="deg")
StringParameter  telescope (description="Name of telescope")
StringParameter  velocityDefinition (description="The velocity definition and frame")
DoubleParameter  radialVelocity (description="Spacecraft velocity along the l-of-s of the telescope wrt the LSR", quantity="km s-1")
StringParameter  calVersion (description="HIFI calibration version")
StringParameter  Band (description="Active band")
StringParameter  attitudeQuaternion (description="Pointing product quaternion applied")
LongParameter  redundancy (description="Spectral Scan redundancy requested")
DoubleParameter  tmbReference (description="Temperature (main beam) at noise reference frequency", quantity="K")
DoubleParameter  noiseRefFrequency (description="Noise reference frequency", quantity="GHz")
DoubleParameter  totNoiseEfficiency (description="Total noise efficiency", quantity="%")
DoubleParameter  driftNoiseContrib (description="Drift noise contribution", quantity="%")
StringParameter  orbitEphemerisSourceFile (description="Name of the file from where data was extracted")
DoubleParameter  solarAspectAngleMean (description="Mean Solar Aspect Angle", quantity="degrees")
DoubleParameter  solarAspectAngleRms (description="RMS Solar Aspect Angle", quantity="degrees")
BooleanParameter  hrsHscience (description="Science data are obtained with HRS-H")
BooleanParameter  hrsVscience (description="Science data are obtained with HRS-V")
BooleanParameter  wbsHscience (description="Science data are obtained with WBS-H")
BooleanParameter  wbsVscience (description="Science data are obtained with WBS-V")
DoubleParameter  loFrequencyStartRequest (description="User requested beginning frequency of the spectral scan.", quantity="GHz")
DoubleParameter  loFrequencyEndRequest (description="User requested final frequency of the spectral scan.", quantity="GHz")
BooleanParameter  gyroAttSuspicious (description="Suspicious quality of the attitude reconstruction.")
DoubleParameter  loFreqMin (description="Min LO frequency of the spectral scan Doppler-corrected to freqFrame (SPECSYS)", quantity="GHz")
DoubleParameter  loFreqMax (description="Max LO frequency of the spectral scan Doppler-corrected to freqFrame (SPECSYS)", quantity="GHz")
DoubleParameter  obsFreqMin (description="Observed min frequency in freqFrame (SPECSYS)", quantity="GHz")
DoubleParameter  obsFreqMax (description="Observed max frequency in freqFrame (SPECSYS)", quantity="GHz")
DoubleParameter  loFrequencyStart (description="Actual start local oscillator frequency", quantity="GHz")
DoubleParameter  loFrequencyEnd (description="Actual end local oscillator frequency", quantity="GHz")
DoubleParameter  rmsNoiseHvVsTsys (description="The noise ratio of the two polarisation exceeds that expected from the measured Tsys ratio by more than 10%. Their ratio is given in the value column.")
DoubleParameter  rmsNoiseHV (description="One of the two polarisations is noisier than the other by more than SQRT(2). Their noise ratio is given in the value column")
DoubleParameter  rmsDSBMin (description="Rms DSB Noise at minimum bandwidth H and V polarizations averaged", quantity="K")
DoubleParameter  rmsDSBNative (description="Rms DSB Noise rescaled to native WBS resolution H and V polarizations averaged", quantity="K")
DoubleParameter  rmsDSBMax (description="Rms DSB Noise at maximum bandwidth H and V polarizations averaged", quantity="K")
DoubleParameter  rmsSSBMin (description="Rms Deconvolved SSB Noise at minimum bandwidth H and V polarizations averaged", quantity="K")
DoubleParameter  rmsSSBNative (description="Rms Deconvolved SSB Noise rescaled to native WBS resolution H and V polarizations averaged", quantity="K")
DoubleParameter  rmsSSBMax (description="Rms Deconvolved SSB Noise at maximum bandwidth H and V polarizations averaged", quantity="K")
                             


3.1.2. HIFI Level 0, Level 0.5, Level 1 and level 2 contexts

The HIFI Level 0, Level 0.5, Level 1 and Level 2 contexts contain one type of context. Its global structure does not change when the data is processed between Level 0 and Level 2. The HIFI Levels 0, 0.5, 1 and 2 contain the HIFI Timeline product, which is a map context that groups a number of HIFI Spectrum Datasets, wrapped in products, normally one per building block.

Product description Product type
HIFI level 0 context HifiRawData
  HIFI WBS-H context HifiTimelineProduct
    WBS-H product per building block HifiSpectrumDataset
  HIFI WBS-V context HifiTimelineProduct
    WBS-V products per building block HifiSpectrumDataset
  HIFI HRS-H context HifiTimelineProduct
    HRS-H products per building block HifiSpectrumDataset
  HIFI HRS-V context HifiTimelineProduct
    HRS-V products per building block HifiSpectrumDataset

  • HIFI Level 0.5 context: Globally the Level 0.5 context has the same structure as the Level 0 product. Backend (HRS or WBS) specific effects are removed as much as possible. A frequency scale has been added.

  • HIFI level 1 context: As mentioned above, the level 1 context still has the same global structure but now the flux columns are calibrated. Each row in the HIFI Spectrum Dataset corresponds to a calibrated single on-board integration. The integrations are not added in the standard processing.

  • HIFI Level 2 context: The Level 2 context also has the same global structure but now the fluxes are on the Ta* scale and the data at the same LO frequency and spatial position are averaged together.

Level 0.5 is removed from the HIFI observational products, upon successful generation of a Level 1 context. The reason for this is to save disk space. Level 0.5 can always be reprocessed from level 0 data.

3.1.3. HIFI Level 2 context

A high level description of the HIFI level-2 products is given here. Depending on the AOT different products will be made. Broadly there are 3 kinds of HIFI AOTs: Single point observations, spectral scan observations and mapping observations. The first two of these will produce one or more single spectra while the last will produce a spectral cube.

HIFI Single Point Observation. By processing the data in a HifiTimelineProduct by e.g. removing the reference, averaging the on-source spectra and concatenating subbands smaller SpectrumDatasets are obtained. All of these products are full HifiSpectrumDatasets and, as such, fundamentally identical to the spectrum data sets generated in level-1 processing (except for the number of spectra). In general only one spectrum will be contained in the HifiSpectrumDataset. The meta information associated with these data sets is more extended than what is available for the level-1 spectrum data set. The overall container of these reduced HifiSpectrumDatasets is still a HifiTimelineProduct.

HIFI Spectral Scan Observation. A spectral scan observation contains several point observations at different LO settings such that they might form a single continuous spectrum, when deconvolved. As deconvolution is still beyond level-2 spectral scan observations are similar to the single point observations of the previous paragraph.

HIFI Mapping Observations. HIFI on the fly mapping and raster data are three dimensional in nature; the data contain spectra corresponding to a (fairly) regular position grid on the sky. Normally OTF cq. raster data is processed from a HifiSpectrumDataset into a regularly gridded data cube with equally spaced pixels in position and frequency coordinates. Such data cubes are stored as a HifiSpectralCube product which will be directly derived from the generic SpectralSimpleCube data product.

3.1.4. HIFI Level 2.5 context

A high-level description of the HIFI Level 2.5 products is given here. Level 2.5 products are those that will be provided as stand-alone browse products in the HSA. Depending on the AOT different products will be made. Broadly there are three kinds of HIFI AOTs: single point observations, spectral scan observations and spectral mapping observations. The first two of these will produce one or more single spectra while the last will produce a spectral cube.

Single point observations.  Level 2.5 products correspond to stitched (i.e. only one concatenated spectrometer sub-band) level 2 spectra for each of the polarisations and backends applicable to the obsid. In the case of the HRS, individual level 2 sub-bands may not necessarily overlap in frequency and the corresponding gaps will be filled with NaNs. Finally, Frequency Switching observations will be folded.

Spectral scan observations. Level 2.5 products correspond to the outcome of the doDeconvolution task run on both polarisation of the WBS. No deconvolved products are provided for the HRS data, even if they have been taken serendipitously during the observation. No baseline correction is done prior to deconvolution and it is expected that in many cases you will need to go back to the Level 2 HTP and clean the data prior to re-running doDeconvolution. This can be done using the interactive Level 2.5 pipeline.

Mapping observations.  Level 2.5 products are cubes created from level 2 products regridded onto a regular spatial grid for each of the polarisations and backend applicable to the obsid. The cube dimension are derived from the geometry of the executed map. For moving targets, the maps are in the comoving frame. Unlike the point mode observations, only WBS spectra are systematically stitched. HRS spectra are stitched only in case the corresponding level 2 sub band overlap in frequency. Finally, maps combined with Frequency Switching observations will be folded.