Difference: HifiCalibrationWeb (1 vs. 109)

Revision 1092017-04-17 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page


Changed:
<
<
NEW NEW NEW Latest updates -- 10 January 2017 NEW NEW NEW
  • HIPE 15.0 is the latest user release
>
>
NEW NEW NEW Latest updates -- 20 April 2017 NEW NEW NEW
  • HIPE 15.0.1 is the latest user release, fixing an issue discovered with the use of interactively-assigned line masks in FitBaseline
 
  • A new HIFI calibration tree has been released, with very minor updates
  • The first version of the HIFI handbook is now available
  • Check the important calibration updates and associated improvements expected in the HIPE 14 (and final) reprocessed data - this includes in particular updated sideband gain ratio in bands 1 to 4
Line: 32 to 32
 

Reducing HIFI data

Recommended User release

Changed:
<
<
>
>
  HIPE and Java 8 : please consult the HIPE Known Issues page regarding important information on issues with Java for different versions of HIPE.

Should I reprocess my date with HIPE 15.0 ?

Revision 1082017-01-10 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page

Line: 50 to 50
  If your data have been processed with HIPE 13.0, HIPE 12.0 or HIPE 11.1

  • Intensity calibration uncertainty
Changed:
<
<
    • HIPE 14 introduces a new product providing an estimated breakdown of the intensity calibration uncertainty. The uncertainties are given separately for each component, as well as in a quadratic sum fashion for all statistically independent elements. This sum applies to the calibration of data in the Ta* scale. Further systematic and random error apply for data converted into different intensity scales (e.g. Tmb or Jy). See further details in section 9 of the DRG.
>
>
    • HIPE 14 introduces a new product providing an estimated breakdown of the intensity calibration uncertainty. The uncertainties are given separately for each component, as well as in a quadratic sum fashion for all statistically independent elements. This sum applies to the calibration of data in the Ta* scale. Further systematic and random errors apply for data converted into different intensity scales (e.g. Tmb or Jy). See further details in section 9 of the DRG.
 
  • Pointing reconstruction
    • In HIPE 14, those observations where the interlacing mode was activated (i.e. using more than just 9 guide stars in the star tracker) will benefit from an more accurate reconstructed astrometry and so positions may change in those cases. On top of that, the quality figure associated to the new gyro-propagated method introduced back in HIPE 13 (see also the following Pointing Information page) is now computed in a slightly different fashion. Since the application or not of the gyro-propagated pointing is, for HIFI, conditional upon a certain threshold on this quality (avoiding to apply the new pointing reconstruction to under-performing case), the number of case making us of one or the other approach will differ in HIPE 14 - we recall that in case the gyro-propagated method is discarded, the pointing is the same as that used back in HIPE 12.For the HIFI data such a correction is applied in a conditional fashion depending on a quality figure computed for each individual observation. The new pointing reconstruction will not apply to under-performing cases, and those latter will still use the pointing files used back in HIPE 12. Details about the new attitude reconstruction, and the way it is approached and may impact the HIFI data, can be found in this memo.
Line: 65 to 65
 
    • In HIPE 14, a new channel flag has been introduced ("warning") that will be assigned to the data based on a knowledge base of spurious features built from the spectral scan flagging (see bullet above). These flags will be applied to all point and mapping observations. They should be taken as indicative as they not necessarily accurately match spurious features in the data where they got applied. For this reason, this new flag is not honoured by any of the standard interactive post-processing tasks.

  • Flags in OFF positions.
Changed:
<
<
    • In HIPE 13 the data used in the OFF positions were already processed up to an equivalent Level 2 calibration (both in intensity and frequency) in order to be directly comparable to the ON-target data. In HIPE 14, those OFF spectra will also hold spur and warning flags that will be propagated from the ON-targert ones. This feature is for example very interesting in order to perform a deconvolution of the OFF spectras in a spectral scan and be able to compare the OFF spectra to the level 2.5 deconvolved products of those observations.
>
>
    • In HIPE 13 the data used in the OFF positions were already processed up to an equivalent Level 2 calibration (both in intensity and frequency) in order to be directly comparable to the ON-target data. In HIPE 14, those OFF spectra will also hold spur and warning flags that will be propagated from the ON-targert ones. This feature is for example very interesting in order to perform a deconvolution of the OFF spectra in a spectral scan and be able to compare the OFF spectra to the level 2.5 deconvolved products of those observations.
  If your data have been processed with a version earlier than 11.1

  • Solar System Object ephemerides
Changed:
<
<
    • A bug (see here) was fixed in the calculation of SSO ephemeris positions (ra_centre/dec_centre) that led to offsets of up to ~10". This bug affected users that use cubes in a co-moving frame (including the standard Level 2.5 cubes), use the doOffset task, or make explicit use of ra_centre / dec_centre in their HIPE scripts. The offset is practically constant over a map and could, e.g., make the emission of a comet appear to be off-center when it really isn't. The fixed bug is in the Level 0 pipeline, which users cannot run easily. The issue was fixed in HIPE 11.1.
>
>
    • A bug (see here) was fixed in the calculation of SSO ephemeris positions (ra_centre/dec_centre) that led to offsets of up to ~10". This bug affected users that use cubes in a co-moving frame (including the standard Level 2.5 cubes), use the doOffset task, or make explicit use of ra_centre / dec_centre in their HIPE scripts. The offset is practically constant over a map and could, e.g., make the emission of a comet appear to be off-center when it really isn't. The fixed bug is in the Level 0 pipeline, which users cannot run easily. The issue was fixed in HIPE 11.1.
  If your data have been processed with a version earlier than 10.3

  • DBS Observations If you have data processed with < HIPE 10
Line: 217 to 217
 
Changed:
<
<
  • This repository of system temperature plots across the IF for each HIFI subband (updated 24 Jan 2011) inform about the variation in system temperature (sensitivity) across the HIFI intermediate frequency band at various frequencies.
>
>
  • This repository of system temperature plots across the IF for each HIFI subband (updated 24 Jan 2011) informs about the variation in system temperature (sensitivity) across the HIFI intermediate frequency band at various frequencies.
 

Calibration error budget

Revision 1072017-01-09 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page


Changed:
<
<
NEW NEW NEW Latest updates -- 26 September 2016 NEW NEW NEW
  • HIPE 14.2 is the latest user release
>
>
NEW NEW NEW Latest updates -- 10 January 2017 NEW NEW NEW
  • HIPE 15.0 is the latest user release
 
  • A new HIFI calibration tree has been released, with very minor updates
  • The first version of the HIFI handbook is now available
Changed:
<
<
  • Check the important calibration updates and associated improvements expected in the HIPE 14.0.1 and upcoming 14.1 reprocessed data - this includes in particular updated sideband gain ratio in bands 1 to 4
>
>
  • Check the important calibration updates and associated improvements expected in the HIPE 14 (and final) reprocessed data - this includes in particular updated sideband gain ratio in bands 1 to 4
 
  • The HIFI products now come with a frequency-dependent intensity calibration uncertainty budget
  • Channel flags for spurs are now populated in all HIFI observations (this was limited to spectral scan up to HIPE 13)
  • Re-organisation of the HIFI quality flags for a better assessement of issues being relevant or not to the end products
Line: 32 to 32
 

Reducing HIFI data

Recommended User release

Changed:
<
<		 HIPE 14 and Java 8 : please consult the HIPE Known Issues page regarding important information on issues with Java for different versions of HIPE.

Which data would most benefit from being reprocessed in HIPE 14.1/14.2 ?

>
>		 HIPE and Java 8 : please consult the HIPE Known Issues page regarding important information on issues with Java for different versions of HIPE.

Should I reprocess my date with HIPE 15.0 ?

 
Changed:
<
<		 If your data have been processed with HIPE 14.1
>
>		HIPE 15.0 is the final user release of HIPE. This version provides an optimisation of various interactive analysis tools offered within HIPE, however, it does not provide any change in the instrument pipeline algorithm. As such products generated with HIPE 15.0 will be identical (within negligible differences for very particular cases - see the DP known issues page) to those generated during the last bulk reprocessing, which corresponds essentially to HIPE 14.1 for HIFI products. The main new or upgraded functionalities offered by HIPE 15 are summarised in the What's New page.

Which data did most benefit from being reprocessed in HIPE 14.1/14.2 ?

 
Changed:
<
<
  • Isolated problematic processing with 14.1
    • A dozen of observations was not properly processed in 14.1 and 14.0.1. The list of those obsids is: 1342197147, 1342210032, 1342216806, 1342219255, 1342219454, 134230216, 1342245299, 1342246507, 1342245997, 1342250406, 1342250721, 1342265972, 1342263227, 1342192673, 1342218631. These observations have been reprocessed correctly with 14.2.
>
>		If your data have been processed with HIPE 14.1
 
Added:
>
>
  • Isolated problematic processing with 14.1
    • A dozen of observations was not properly processed in 14.1 and 14.0.1. The list of those obsids is: 1342197147, 1342210032, 1342216806, 1342219255, 1342219454, 134230216, 1342245299, 1342246507, 1342245997, 1342250406, 1342250721, 1342265972, 1342263227, 1342192673, 1342218631. These observations have been reprocessed correctly with 14.2
  If your data have been processed with HIPE 14.0.1

  • Spectra mean and median statistical numbers
Line: 48 to 50
  If your data have been processed with HIPE 13.0, HIPE 12.0 or HIPE 11.1

  • Intensity calibration uncertainty
Changed:
<
<
    • HIPE 14 introduces a new product providing an estimated breakdown of the intensity calibration uncertainty. The uncertainties are given separately for each component, as well as in a quadratic sum fashion for all statistically independent elements. This sum applies to the calibration of data in the Ta* scale. Further systematic and random error apply for data converted into different intensity scales (e.g. Tmb or Jy). See further details in section 9 of the DRG.
>
>
    • HIPE 14 introduces a new product providing an estimated breakdown of the intensity calibration uncertainty. The uncertainties are given separately for each component, as well as in a quadratic sum fashion for all statistically independent elements. This sum applies to the calibration of data in the Ta* scale. Further systematic and random error apply for data converted into different intensity scales (e.g. Tmb or Jy). See further details in section 9 of the DRG.
 
  • Pointing reconstruction
    • In HIPE 14, those observations where the interlacing mode was activated (i.e. using more than just 9 guide stars in the star tracker) will benefit from an more accurate reconstructed astrometry and so positions may change in those cases. On top of that, the quality figure associated to the new gyro-propagated method introduced back in HIPE 13 (see also the following Pointing Information page) is now computed in a slightly different fashion. Since the application or not of the gyro-propagated pointing is, for HIFI, conditional upon a certain threshold on this quality (avoiding to apply the new pointing reconstruction to under-performing case), the number of case making us of one or the other approach will differ in HIPE 14 - we recall that in case the gyro-propagated method is discarded, the pointing is the same as that used back in HIPE 12.For the HIFI data such a correction is applied in a conditional fashion depending on a quality figure computed for each individual observation. The new pointing reconstruction will not apply to under-performing cases, and those latter will still use the pointing files used back in HIPE 12. Details about the new attitude reconstruction, and the way it is approached and may impact the HIFI data, can be found in this memo.

  • Band 6 and 7 observations
Changed:
<
<
    • From HIPE 13 onward observations in bands 6 and 7 have been automatically corrected from the Electrical Standing Wave affecting those data. The correction is based on an optimised fit to the baseline artefact stored in the HIFI calibration files and applied by the pipeline. In HIPE 14, some of those stored solutions have been refined and so the resulting products will be improved. For instructions on how to benefit from this reprocessing and see the typical improvement expected in the data please refer to this section from the HIFI Data Reduction Guide (DRG).
>
>
    • From HIPE 13 onward observations in bands 6 and 7 have been automatically corrected from the Electrical Standing Wave affecting those data. The correction is based on an optimised fit to the baseline artefact stored in the HIFI calibration files and applied by the pipeline. In HIPE 14, some of those stored solutions have been refined and so the resulting products will be improved. For instructions on how to benefit from this reprocessing and see the typical improvement expected in the data please refer to this section from the HIFI Data Reduction Guide (DRG).
 
  • Spectral Scan observations
Changed:
<
<
    • The reprocessing of spectral scans in HIPE 13 made use of optimised mask tables for spurs and unruly baseline ranges, resulting in improved deconvolved solutions at the Level 2.5 (an example of such improvement between HIPE 12 (left - red) and HIPE 13 (right - black) is shown in this plot - see also section 5.4 of the DRG). In HIPE 14, a dozen of additional spectral scans (mostly from the calibration programme) have been added to this list.In order to benefit from this update you should reprocess your data from Level 0 with the new calibration tree.
>
>
    • The reprocessing of spectral scans in HIPE 13 made use of optimised mask tables for spurs and unruly baseline ranges, resulting in improved deconvolved solutions at the Level 2.5 (an example of such improvement between HIPE 12 (left - red) and HIPE 13 (right - black) is shown in this plot - see also section 5.4 of the DRG). In HIPE 14, a dozen of additional spectral scans (mostly from the calibration programme) have been added to this list.In order to benefit from this update you should reprocess your data from Level 0 with the new calibration tree.
 
  • Spur warning flags
    • In HIPE 14, a new channel flag has been introduced ("warning") that will be assigned to the data based on a knowledge base of spurious features built from the spectral scan flagging (see bullet above). These flags will be applied to all point and mapping observations. They should be taken as indicative as they not necessarily accurately match spurious features in the data where they got applied. For this reason, this new flag is not honoured by any of the standard interactive post-processing tasks.
Line: 70 to 72
 
    • A bug (see here) was fixed in the calculation of SSO ephemeris positions (ra_centre/dec_centre) that led to offsets of up to ~10". This bug affected users that use cubes in a co-moving frame (including the standard Level 2.5 cubes), use the doOffset task, or make explicit use of ra_centre / dec_centre in their HIPE scripts. The offset is practically constant over a map and could, e.g., make the emission of a comet appear to be off-center when it really isn't. The fixed bug is in the Level 0 pipeline, which users cannot run easily. The issue was fixed in HIPE 11.1.
If your data have been processed with a version earlier than 10.3
Changed:
<
<
  • DBS Observations If you have data processed with < HIPE 10
>
>
  • DBS Observations If you have data processed with < HIPE 10
 
    • A new pipeline step, mkDbsReference, calculates the differences in the chop positions in all DBS observations, and after applying the band-pass correction, stores them in a product in calibration->pipeline-out called ReferenceSpectra. This allows you to check for contamination in chop positions for all types of DBS observations.
Changed:
<
<
  • Improved pointing reconstruction for observations taken between OD 320 and 761 If you have data processed with < HIPE 9
    • The pointing information attached to the data for observations taken between OD 320 and OD761 did not use the most accurate representation of the star tracker focal length. This was done for the bulk reprocessing with HIPE 9. As a consequence, some observations will experience a shift in astrometry, that can be as high as 8 arcsec. Details about the consequences for a particular obsid, and recipes to reconstruct the improved pointing yourself can be found at http://herschel.esac.esa.int/twiki/bin/view/Public/HowToUseImprovedPointingProducts as well as in the Level 0 section of the HIFI Data Reduction Guide.
>
>
  • Improved pointing reconstruction for observations taken between OD 320 and 761 If you have data processed with < HIPE 9
    • The pointing information attached to the data for observations taken between OD 320 and OD761 did not use the most accurate representation of the star tracker focal length. This was done for the bulk reprocessing with HIPE 9. As a consequence, some observations will experience a shift in astrometry, that can be as high as 8 arcsec. Details about the consequences for a particular obsid, and recipes to reconstruct the improved pointing yourself can be found at http://herschel.esac.esa.int/twiki/bin/view/Public/HowToUseImprovedPointingProducts as well as in the Level 0 section of the HIFI Data Reduction Guide.
 
Changed:
<
<
  • Backfilling of observational parameters If you have data processed with < HIPE 9
>
>
  • Backfilling of observational parameters If you have data processed with < HIPE 9
 
    • From 9.1 onwards, most of the observational parameters that got optimised in HSpot will be propagated into the observation context, under a new product called HifiUplinkProduct (in the auxiliary product branch). Some of these parameters will be used in the pipeline to e.g. estimate the dimension of the cubes in the mapping and have them more representative of how the map was really obtained.
Changed:
<
<
  • Solar System Object maps If you have data processed with < HIPE 9
    • Cubes for moving targets are now created in the comoving frames in HIPE 9. In order to benefit from that, you should re-pipeline from Level 0 up to Level 2.5. Note that the hifiPipeline task should be called with the option "Aux=True". Please check the Pipeline chapter of the HIFI Data Reduction Guide for more details about how to do that.
>
>
  • Solar System Object maps If you have data processed with < HIPE 9
    • Cubes for moving targets are now created in the comoving frames in HIPE 9. In order to benefit from that, you should re-pipeline from Level 0 up to Level 2.5. Note that the hifiPipeline task should be called with the option "Aux=True". Please check the Pipeline chapter of the HIFI Data Reduction Guide for more details about how to do that.
 
Changed:
<
<
>
>
 

Documentation and Cookbooks

  • The HIFI Launch Pad is intended to help you quickly off the ground with HIFI data reduction
Changed:
<
<
>
>
 
  • Cookbooks now exist for each HIFI observing mode
Changed:
<
<
>
>
 
  • The HIFI Data Reduction Guide and the HIFI AOT Observing Mode Release and Performance Notes contain information you need to know about HIFI data, regardless of the software you use for data reduction
Changed:
<
<
>
>
 

  • A series of useful scripts can be found following the HIPE menu Scripts -> HIFI Useful Scripts

  • If you want to know more about the details of the HIFI pipeline, you should look at the HIFI Pipeline Specification Document
Changed:
<
<
>
>
 
Changed:
<
<
  • If you are working in HIPE, the HIFI Data Reduction Guide will be your prime resource for all things HIFI but you should also look at the Herschel Data Analysis Guide for information about general tools, such as those for viewing and manipulating spectra and spectral cubes, and for obtaining data from the archive.
>
>
  • If you are working in HIPE, the HIFI Data Reduction Guide will be your prime resource for all things HIFI but you should also look at the Herschel Data Analysis Guide for information about general tools, such as those for viewing and manipulating spectra and spectral cubes, and for obtaining data from the archive.
 
  • The full set of online documentation for the current user release also contains detailed information about the HIFI pipeline, information about scripting in HIPE, and reference manuals for command listings. New users of HIPE are recommended to read through the Quick Start Guide and the HIPE Owner's Guide
    • You can create pdfs of each document by clicking on the pdf symbol at the top of the main page.
Changed:
<
<
>
>
 
  • The full set of Herschel documentation for the developer track is also available. This documentation contains revisions and updates to the documentation associated with the current User Release but you should also be aware that it may describe functionality not available in the current User Release
Line: 118 to 120
 

Typical Data Reduction Workflow

  • Download data from the HSA. If you are downloading several observations it is better to use the tar ball.
Changed:
<
<
>
>
 
  • Inspect data quality. Both point spectra and spectral cubes can be viewed in the SpectrumExplorer. In addition to inspecting the quality of your Level 2 data, you should always check the Level 1 data for any bad scans. If data at Level 2 appears strange then looking at Level 1 data may provide insight to the problem, see the Data Primer chapter of the HIFI Data Reduction Guide for more information about levels of data
Changed:
<
<
>
>
 
  • Assess whether data should be re-pipelined.
    • Look at the section above and also at the What's New in the latest User Release to decide if you should re-pipeline.
    • Note that the ICC recommends that data processing should not be allowed to fall more than one version behind the HIPE version being used.
    • Everything you need to re-pipeline data is available in the ObservationContext, and the instructions for using the pipeline, including tips on re-pipelining using new calibration data and customising the pipeline, can be found in the Pipeline chapter of the HIFI Data Reduction Guide
Changed:
<
<
>
>
 
  • Flag data. A user-friendly task has been developed in order to easily flag your data: flagTool. It can be used both as a GUI (similar concept as for the fitBaseline task) or in command line
Changed:
<
<
>
>
 
  • Remove standing waves. Standing waves are a common problem in HIFI data, particularly in bands 3, 4, 6, and 7. The HIFI Data Reduction Guide provides some information about typical standing waves in HIFI data and describes how to remove them in HIPE using fitHifiFringe and the modified band pass technique
Changed:
<
<
>
>
 
  • Correct baseline drift. Baselines can be flattened by subtracted, or division in the case of real continuum. This can be done in HIPE with the fitBaseline task
Changed:
<
<
>
>
 
  • Fold frequency switch data. Frequency switch data is not folded in the pipeline and can be done with the doFold task.
Changed:
<
<
>
>
 
  • Average together H and V polarisation.
    • The noise estimates given by HSpot assume that the H and V polarisations are averaged together. This can be done in HIPE with the PolarPair task, which resamples the spectra to the same frequency scale (they are frequency calibrated with different comb measurements) and then averages them. Alternatively, you can use the Accumulate task, which allows you to specify the resampling width if you desire.
Changed:
<
<
>
>
 
    • Note that differences may be seen in H and V profiles, see the note on H and V positions. If you are particularly interested in the spatial structure of your source you may prefer not to average the H and V polarisations together.
    • You can also average maps taken with the H and V polarisations:
Changed:
<
<
>
>
 
  • Spectral manipulation. The Spectral Toolbox in HIPE allows you to perform typical spectral arithmetic tasks on spectra and spectral cubes
Changed:
<
<
>
>
 
  • Fit lines. The SpectrumFitterGUI, which is also a part of the Spectral Toolbox, can be used to fit line profiles.
Changed:
<
<
>
>
 
  • Save modified data You can save data to a pool, or store your session in order to continue working in HIPE.
Changed:
<
<
>
>
 
Changed:
<
<
  • You may wish to export data as FITS, ASCII or use the hiClass task to export data as CLASS readable FITs files.
    • Saving as FITS
    • Saving as ASCII
    • Products readable in GILDAS/CLASS: for products processed with HIPE 12 or later, you can directly read the HCSS-generated FITS files into CLASS (more details about the FITS reader can be found in this report). For products processed with an earlier version, you should use the HiClass task to export data as CLASS readable FITS files. Notes: (1) If you import products processed in HIPE 12 or 13 some warnings will be raised about missing meta-data, but the import will proceed. (2) The FITS file from Level 2 and 2.5 observations downloaded from the HSA are gzipped files. It is recommended to test that the FITS are properly un-zipped before reading to CLASS (for example using the utility ‘fv’ (FITS Viewer) among others). (3) HSC cannot guarantee that future CLASS version will be able read the HCSS-generated FITS files.
>
>
  • You may wish to export data as FITS, ASCII or read the data in GILDAS/CLASS.
    • Saving as FITS
    • Saving as ASCII
    • Products readable in GILDAS/CLASS: for products processed with HIPE 12 or later, you can directly read the HCSS-generated FITS files into CLASS (more details about the FITS reader can be found in this report). For products processed with an earlier version, you should use the HiClass task to export data as CLASS readable FITS files. Notes: (1) If you import products processed in HIPE 12 or 13 some warnings will be raised about missing meta-data, but the import will proceed. (2) The FITS file from Level 2 and 2.5 observations downloaded from the HSA are gzipped files. It is recommended to test that the FITS are properly un-zipped before reading to CLASS (for example using the utility ‘fv’ (FITS Viewer) among others). (3) HSC cannot guarantee that future CLASS version will be able read the HCSS-generated FITS files.
 
Line: 175 to 177
 

Dedicated data reduction tips for Spectral Scan data

  • The data needs to be deconvolved to a single sideband solution. This can be done with the doDeconvolution task, which works with an ObservationContext.
Changed:
<
<
>
>
 
  • It is important to clean the data before deconvolving it: this means removing standing waves, correcting baselines, and flagging out any spurs or bad data that the pipeline missed. The calibration tree for HIPE 14 provides optimised mask tables for each spectral scan obsids so it is recommended that your make use of those, typically be reprocessing your data with HIPE 14 and this calibration file (see above).
Line: 191 to 193
 
  • Regridding in HIPE > 9 will automatically use the comoving frame for SSOs. Considering the section above, mapping observations of SSOs will benefit from reprocessing in HIPE > 11.1.

  • You can then grid your cleaned data into a spectral cube using the doGridding task, you can also use this task to customise the cube creation to best suit your science goals.
Changed:
<
<
>
>
 

Dedicated data reduction tips for Strong continuum observations

  • Strong continuum sources
    • An optional step in the pipeline can be used to remove standing waves arising in the loads, this is particularly effective for strong continuum sources. A report describing the technique can be found here. Instructions for using this modified passband technique are available in the Standing Wave Removal chapter of the HIFI Data Reduction Guide:
Changed:
<
<
>
>
 

HIFI performance and calibration

Line: 204 to 206
 
Added:
>
>
    • Kester et al. 2016: This paper describes the derivation of the final sideband gain ratio in use in the HIFI product calibration
 
    • Sideband ratio technical note: This note describes the general calibration problem associated with the sideband gain imbalance existing in double-sideband heterodyne systems such as HIFI, and provides some recipes and calibration tables applicable to the particular case of the HIFI mixers.
    • http://arxiv.org/abs/1404.2806: "The effect of sideband ratio on line intensity for Herschel/HIFI", Experimental Astronomy.
Changed:
<
<
    • After an extensive investigation, the HIFI ICC has introduced updated sideband gain ratio in the calibration tree. A final documentation is still in preparation but the overview of the sideband gain ratio frequency dependence in each band is illustrated in this plot.
>
>
    • After an extensive investigation, the HIFI ICC has introduced updated sideband gain ratio in the calibration tree. A final documentation is still in preparation but the overview of the sideband gain ratio frequency dependence in each band is illustrated in this plot.
 
Line: 214 to 217
 
Changed:
<
<
  • This FTP repository of system temperature plots across the IF for each HIFI subband (updated 24 Jan 2011) inform about the variation in system temperature (sensitivity) across the HIFI intermediate frequency band at various frequencies.
>
>
  • This repository of system temperature plots across the IF for each HIFI subband (updated 24 Jan 2011) inform about the variation in system temperature (sensitivity) across the HIFI intermediate frequency band at various frequencies.
 

Calibration error budget

Line: 233 to 236
 
Planetary model error 5 5 5 5
Beam/aperture efficiency 1.1-1.4 1.2-1.3 1-1.1 1.2-1.3
Flux loss due to pointing** 2-4 3-7 3-4 8-18
Changed:
<
<		*Note that the "sideband gain ratio" convention used in the HIFI pipeline does not correspond to the ratio between the respective sideband gains (equal to 1 for a balanced system), but to the gain in one sideband normalised by the combined gain of the two sidebands. Consequently, perfectly-balanced sidebands will feature a sideband ratio of 0.5 (see also section 5.3.1 of the Observers' Manual, and the Intensity Calibration Framework document).
>
>		*Note that the "sideband gain ratio" convention used in the HIFI pipeline does not correspond to the ratio between the respective sideband gains (equal to 1 for a balanced system), but to the gain in one sideband normalised by the combined gain of the two sidebands. Consequently, perfectly-balanced sidebands will feature a sideband ratio of 0.5 (see also Kester et al. 2016, and the Intensity Calibration Framework document).
 
Changed:
<
<		**For a point source (worst case scenario) - this flux loss estimates takes into account an APE of 1.2", a residual SIAM bias of order 1.5" and the fact that the H/V respective apertures are slightly offset from the middle synthetic aperture used to point the telescope (offset ranging between 3" in band 1 and 0.5" in band 7)
>
>		**For a point source (worst case scenario) - this flux loss estimates takes into account an APE of 1.2", a residual SIAM bias of order 1.5" and the fact that the H/V respective apertures are slightly offset from the middle synthetic aperture used to point the telescope (offset ranging between 3" in band 1 and 0.5" in band 7). See also section 5.6 of the HIFI handbook.
  Frequency Calibration Accuracy
Line: 252 to 255
  HIFI beam coupling efficiencies
Changed:
<
<		A revision of the HIFI beam analysis has been performed, based on a multi-epoch set of measurements on Mars. As a result, a more accurate description of the HIFI beams has been obtained, leading to some change in the beam coupling efficiencies, typically a drop. The details of the analysis and the revised coupling efficiencies are given in this release note. For the record, the first release of the HIFI beam efficiencies was described in this document (17 Nov 2010). Please note that these updates will only be featured in the HIFI calibration tree from HIPE 13 onwards.
>
>		A revision of the HIFI beam analysis has been performed, based on a multi-epoch set of measurements on Mars. As a result, a more accurate description of the HIFI beams has been obtained, leading to some change in the beam coupling efficiencies, typically a drop. The details of the analysis and the revised coupling efficiencies are given in this release note. For the record, the first release of the HIFI beam efficiencies was described in this document (17 Nov 2010). Please note that these updates will only be featured in the HIFI calibration tree from HIPE 13 onwards.
  HIFI beam maps and Encircled Energy Fraction
Line: 261 to 264
 The beam material is concatenated as a single FITS file per mixer, frequency, and polarisation. Each file has three extensions tables: one for the 2-D beam model maps (2705x2705 pixels), one for the EEF, and one for the 1-D azimuthal beam averages. Note that 2-D beam model of smaller size (201x201 pixels) will also be made available in the calibration tree (not before HIPE 13), and used in the generation of Level 2.5 cubes by the pipeline. The following page provides links to each individual file (27 Mb each). Plots of the 1-D azimuthal beam averages, and EEF for each band can be found in the Appendix B of the release note.

Access to the calibration tree:

Changed:
<
<
  • Information about what calibration is found in the HIFI ObservationContext is found in the Data Primer of the HIFI Data Reduction Guide.
>
>
  • Information about what calibration is found in the HIFI ObservationContext is found in the Data Primer of the HIFI Data Reduction Guide.
 
Changed:
<
<
  • Information about how to find what calibration version was used on your data, how to get the latest calibration, and reprocess observations with it is found in the Pipeline chapter of the HIFI Data reduction Guide.
>
>
  • Information about how to find what calibration version was used on your data, how to get the latest calibration, and reprocess observations with it is found in the Pipeline chapter of the HIFI Data reduction Guide.
 
  • The latest calibration tree pool can also be retrieved directly here (26 September 2016).
Line: 297 to 300
 
  • Standing waves:
    • there are several standing waves that can affect the HIFI data at various processing levels. Their nature and impact on the HIFI calibration are described in the Standing Wave technical note and, more briefly, in the Standing Wave removal chapter of the HIFI Data Reduction Guide
Changed:
<
<
>
>
 
    • There are various techniques currently offered to clean these baseline distortions:
Changed:
<
<
      • Remove sine waves, or combinations of sine waves. Check the bullet about usage of the fitHifiFringe task in the data reduction section above.
>
>
      • Remove sine waves, or combinations of sine waves. Check the bullet about usage of the fitHifiFringe task in the data reduction section above.
 
      • Electrical Standing Wave (aka Matching technique): to correct for the IF standing wave present in bands 6 and 7
        • A new task hebCorrection has been implemented in HIPE 12.0 and is being used by default by the pipeline in HIPE 13 (see also the Reducing HIFI data section above). We warn that the task does not guarantee an accurate correction in all circumstances .
Changed:
<
<
        • The task is described in the following document. The mathematical background can be found in this proceeding. Unlike the script made available in previous version of HIPE, this task is applicable to any HIFI observing mode.
>
>
        • The task is described in the following document. The mathematical background can be found in this proceeding. Unlike the script made available in previous version of HIPE, this task is applicable to any HIFI observing mode.
 
Changed:
<
<
>
>
 
Changed:
<
<
  • Baseline removal: imperfect ON-OFF calibration scheme can result in residual baseline distortion (not necessarily standing waves). Check the bullet about usage of the fitBaseline task in the data reduction section above.
>
>
  • Baseline removal: imperfect ON-OFF calibration scheme can result in residual baseline distortion (not necessarily standing waves). Check the bullet about usage of the fitBaseline task in the data reduction section above.
 
Changed:
<
<
  • Spectral purity: there are places over the HIFI frequency range where the Local Oscillator does not offer a single frequency tone, so that spurious spectral line may end up in the data, and the calibration of the targeted line gets affected. A dedicated cookbook on the matter is in preparation. In the meantime, we refer to section 5.4.6 of the Observer's Manual, and to the release notes provided above.
>
>
  • Spectral purity: in a small number of cases, there are places over the HIFI frequency range where the Local Oscillator does not offer a single frequency tone, so that spurious spectral line may end up in the data, and the calibration of the targeted line gets affected. Details are given in Section 5.3.2 of the HIFI handbook.
 
Changed:
<
<
  • Spurs: Spectral Spurs are narrow and broad spectral features due to instabilities in the Local Oscillator. Up to HIPE 12, spurs have been assigned by the pipeline in an automatic fashion. In HIPE 13, this is no longer the case and the masks are assigned based on a priori tables provided by the ICC in the calibration tree. As of now, all spectral scans benefit from such tables, but only a handful of single point and spectral mapping observations. This limitation has little impact on the top level data though since no pipeline module makes use of these masks in those modes. A list of the spurs registered up to HIPE 12 can be found in section 5.4.6 of the Observer's Manual. This table will be updated soon based on the new statistics derived from the HIPE 13 mask tables.
>
>
  • Spurs: Spectral Spurs are narrow and broad spectral features due to instabilities in the Local Oscillator. Up to HIPE 12, spurs have been assigned by the pipeline in an automatic fashion. In HIPE 13, this is no longer the case and the masks are assigned based on a priori tables provided by the ICC in the calibration tree. As of now, all spectral scans benefit from such tables, but only a handful of single point and spectral mapping observations. This limitation has little impact on the top level data though since no pipeline module makes use of these masks in those modes. A list of the spurs registered in all spectral scans has been incorporated into the HIFI calibration tree, and is used to automatically flag data accordingly in the pipeline.
 

Interest Groups and Scripts

Deleted:
<
<
  • HIPI: a library of HIFI Plug-In's has been prepared by the NHSC - check the HIPI website for more information. From HIPE 13 onwards, most of those utilities have been ported to Useful scripts (Menu Scripts > HIFI Useful Scripts)
 
  • The following interest groups relate to processing of observations taken with HIFI. The links provided allow subscription to these interest groups.
    • subscribe to PACS, SPIRE and HIFI spectral maps interest group
    • subscribe to HIFI point source and spectral scan interest group
Line: 329 to 331
 

Further Information

Deleted:
<
<
 
Changed:
<
<		-- DavidTeyssier , CarolynMcCoey, SylvieFBeaulieu and RengelMiriam - 11 April 2016 -
>
>		-- DavidTeyssier , CarolynMcCoey, SylvieFBeaulieu and RengelMiriam - 09 January 2017 -
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521151" name="freq_vel_3.PDF" path="freq_vel_3.PDF" size="312508" user="Main.DavidTeyssier" version="1"

Revision 1062016-11-10 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page

Line: 170 to 170
 
Changed:
<
<
>
>
 

Dedicated data reduction tips for Spectral Scan data

Line: 369 to 369
 
META FILEATTACHMENT attachment="HIFI_Uncertainty_HIPE14.0.png" attr="" comment="" date="1449696012" name="HIFI_Uncertainty_HIPE14.0.png" path="HIFI_Uncertainty_HIPE14.0.png" size="129135" user="DavidTeyssier" version="1"
META FILEATTACHMENT attachment="HIFI_SBR_summary_HIPE14.0.png" attr="" comment="" date="1449762023" name="HIFI_SBR_summary_HIPE14.0.png" path="HIFI_SBR_summary_HIPE14.0.png" size="110415" user="DavidTeyssier" version="2"
META FILEATTACHMENT attachment="class-herschel-fits.pdf" attr="" comment="" date="1449699928" name="class-herschel-fits.pdf" path="class-herschel-fits.pdf" size="429415" user="DavidTeyssier" version="1"
Added:
>
>
META FILEATTACHMENT attachment="linelist.txt" attr="" comment="" date="1478790230" name="linelist.txt" path="linelist.txt" size="817939" user="DavidTeyssier" version="1"
META FILEATTACHMENT attachment="smallerLinelist.txt" attr="" comment="" date="1478790231" name="smallerLinelist.txt" path="smallerLinelist.txt" size="79154" user="DavidTeyssier" version="1"

Revision 1052016-09-26 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page


Changed:
<
<
NEW NEW NEW Latest updates -- 22 July 2016 NEW NEW NEW
>
>
NEW NEW NEW Latest updates -- 26 September 2016 NEW NEW NEW
 
  • HIPE 14.2 is the latest user release
Added:
>
>
  • A new HIFI calibration tree has been released, with very minor updates
 
  • The first version of the HIFI handbook is now available
  • Check the important calibration updates and associated improvements expected in the HIPE 14.0.1 and upcoming 14.1 reprocessed data - this includes in particular updated sideband gain ratio in bands 1 to 4
  • The HIFI products now come with a frequency-dependent intensity calibration uncertainty budget
Line: 31 to 32
 

Reducing HIFI data

Recommended User release

Changed:
<
<
>
>
  HIPE 14 and Java 8 : please consult the HIPE Known Issues page regarding important information on issues with Java for different versions of HIPE.

Which data would most benefit from being reprocessed in HIPE 14.1/14.2 ?

If your data have been processed with HIPE 14.1

  • Isolated problematic processing with 14.1
Changed:
<
<
    • A dozen of observations was not properly processed in 14.1 and 14.0.1. The list of those obsids is: 1342197147, 1342210032, 1342216806, 1342219255, 1342219454, 134230216, 1342245299, 1342246507, 1342245997, 1342250406, 1342250721, 1342265972, 1342263227. These observations will be reprocessed correctly with 14.2 - meanwhile, it is possible to use the HIPE 13 products, bearing in mind that all other improvements introduced in the 14.x products will not yet be featured in those products
>
>
    • A dozen of observations was not properly processed in 14.1 and 14.0.1. The list of those obsids is: 1342197147, 1342210032, 1342216806, 1342219255, 1342219454, 134230216, 1342245299, 1342246507, 1342245997, 1342250406, 1342250721, 1342265972, 1342263227, 1342192673, 1342218631. These observations have been reprocessed correctly with 14.2.
  If your data have been processed with HIPE 14.0.1
Line: 165 to 166
 
Changed:
<
<
    • HiClass task - Note, however, that GILDAS/Class can now directly read the HCSS-generated FITS files (as of release October 2015) and so it is no longer required to convert the HCSS FITS files into a dedicated format. More details about this new FITS reader can be found in this report.
>
>
    • Products readable in GILDAS/CLASS: for products processed with HIPE 12 or later, you can directly read the HCSS-generated FITS files into CLASS (more details about the FITS reader can be found in this report). For products processed with an earlier version, you should use the HiClass task to export data as CLASS readable FITS files. Notes: (1) If you import products processed in HIPE 12 or 13 some warnings will be raised about missing meta-data, but the import will proceed. (2) The FITS file from Level 2 and 2.5 observations downloaded from the HSA are gzipped files. It is recommended to test that the FITS are properly un-zipped before reading to CLASS (for example using the utility ‘fv’ (FITS Viewer) among others). (3) HSC cannot guarantee that future CLASS version will be able read the HCSS-generated FITS files.
 
Line: 264 to 265
 
  • Information about how to find what calibration version was used on your data, how to get the latest calibration, and reprocess observations with it is found in the Pipeline chapter of the HIFI Data reduction Guide.
Changed:
<
<
  • The latest calibration tree pool can also be retrieved directly here (1 April 2016).
>
>
  • The latest calibration tree pool can also be retrieved directly here (26 September 2016).
 
  • Updates to the HIFI calibration data are generally concurrent with the release of each major version of the HCSS-HIFI software. However, it is possible to have updates to the calibration data in between major releases of the software as the software and data are independent of each other, in the table below the HIPE or the OD from which the calibration updates apply are given. The calibration versions available since HIPE 5 are listed in the table below:
Line: 288 to 289
 
22_0 09-04-15   13.0 Update of the beam parameters (coupling efficiencies, HPBW, 2-D beam maps - see also the beam section above), addition of a new table of expected spectral spurs to be masked by the pipeline in Spectral Scans, addition of a database of Electrical Standing Wave correction solution for observations in bands 6 and 7, addition of a new Uncertainty table that will be propagated into an error budget in HIPE 14 (currently only placeholder), introduction of IF-dependent sideband gain ratio (currently only true for the lower end of band 2a), update of the list of corrupted data-frames for flagging at Level 0
24_0 10-12-15   14.0 Major update of all sideband ratio tables. Introduction of uncertainty budget tables. Introduction of spur warning tables for all point and mapping observations. Update of some Electrical Standing Wave correction solutions. Added some spur flag tables for a handful of spectral scans, as well as some remaining corrupted data-frame entries for flagging at level 0. Updated smoothing width for load chop processing in band 6a, allowing to mitigate the affect of strong artefacts present at tunings between 1457 and 1459 GHz.
25_0 01-04-15   14.1 Minor update to table of BBIDs in order to allow processing of a handful of calibration observations.
Added:
>
>
26_0 26-09-16   14.2 Minor update to table of spur masks for obsids 1342192673 and 1342218631
 

Outstanding calibration issues

Revision 1042016-07-22 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page


Changed:
<
<
NEW NEW NEW Latest updates -- 11 July 2016 NEW NEW NEW
  • HIPE 14.2 will be imminently released - in the meantime 14.1 is the latest user release
>
>
NEW NEW NEW Latest updates -- 22 July 2016 NEW NEW NEW
  • HIPE 14.2 is the latest user release
 
  • The first version of the HIFI handbook is now available
  • Check the important calibration updates and associated improvements expected in the HIPE 14.0.1 and upcoming 14.1 reprocessed data - this includes in particular updated sideband gain ratio in bands 1 to 4
  • The HIFI products now come with a frequency-dependent intensity calibration uncertainty budget

Revision 1032016-07-12 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page



NEW NEW NEW Latest updates -- 11 July 2016 NEW NEW NEW

Changed:
<
<
  • HIPE 14.2 has been released !
>
>
  • HIPE 14.2 will be imminently released - in the meantime 14.1 is the latest user release
 
  • The first version of the HIFI handbook is now available
  • Check the important calibration updates and associated improvements expected in the HIPE 14.0.1 and upcoming 14.1 reprocessed data - this includes in particular updated sideband gain ratio in bands 1 to 4
  • The HIFI products now come with a frequency-dependent intensity calibration uncertainty budget
Line: 31 to 31
 

Reducing HIFI data

Recommended User release

Changed:
<
<
>
>
  HIPE 14 and Java 8 : please consult the HIPE Known Issues page regarding important information on issues with Java for different versions of HIPE.

Which data would most benefit from being reprocessed in HIPE 14.1/14.2 ?

Revision 1022016-07-06 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page


Changed:
<
<
NEW NEW NEW Latest updates -- 12 April 2016 NEW NEW NEW
  • HIPE 14.1 has been released !
>
>
NEW NEW NEW Latest updates -- 11 July 2016 NEW NEW NEW
  • HIPE 14.2 has been released !
  • The first version of the HIFI handbook is now available
 
  • Check the important calibration updates and associated improvements expected in the HIPE 14.0.1 and upcoming 14.1 reprocessed data - this includes in particular updated sideband gain ratio in bands 1 to 4
  • The HIFI products now come with a frequency-dependent intensity calibration uncertainty budget
  • Channel flags for spurs are now populated in all HIFI observations (this was limited to spectral scan up to HIPE 13)
  • Re-organisation of the HIFI quality flags for a better assessement of issues being relevant or not to the end products
Changed:
<
<
  • Users wishing to import their HIFI data into Gildas/CLASS can now do it directly with the Gildas oct15 release without the need of any previous conversion
>
>
  • Users wishing to import their HIFI data into Gildas/CLASS can now do so directly with the Gildas oct15 release without the need of any previous conversion
 
Changed:
<
<

Observing with HIFI

>
>

Overview of HIFI performances and data

 
  • Useful Documentation
Changed:
<
<
>
>
 
Line: 27 to 28
 
    • HIFI fact sheet A 2-page guide to the HIFI instrument
    • A&A paper describing the method used to measure stability performance with HIFI (added 4 Oct 2011)
    • A&A paper about the optimisation of mapping modes for heterodyne instruments (added 18 March 2014)
Changed:
<
<
>
>
 

Reducing HIFI data

Deleted:
<
<
 

Recommended User release

Changed:
<
<
>
>
  HIPE 14 and Java 8 : please consult the HIPE Known Issues page regarding important information on issues with Java for different versions of HIPE.
Added:
>
>

Which data would most benefit from being reprocessed in HIPE 14.1/14.2 ?

If your data have been processed with HIPE 14.1

 
Changed:
<
<

Which data would most benefit from being reprocessed in HIPE 14.1 ?

>
>
  • Isolated problematic processing with 14.1
    • A dozen of observations was not properly processed in 14.1 and 14.0.1. The list of those obsids is: 1342197147, 1342210032, 1342216806, 1342219255, 1342219454, 134230216, 1342245299, 1342246507, 1342245997, 1342250406, 1342250721, 1342265972, 1342263227. These observations will be reprocessed correctly with 14.2 - meanwhile, it is possible to use the HIPE 13 products, bearing in mind that all other improvements introduced in the 14.x products will not yet be featured in those products
 
Changed:
<
<		 If your data have been processed with the latest bulk reprocessing (13.0), HIPE 12.0 or HIPE 11.1
>
>		 If your data have been processed with HIPE 14.0.1
 
  • Spectra mean and median statistical numbers
    • The HIFI 14.0.1 products suffered from a minor bug leading to a wrong computation of the mean and median spectra values (i.e. a measure of the DSB continuum to first order). This has been repaied in 14.1
Added:
>
>		 If your data have been processed with HIPE 13.0, HIPE 12.0 or HIPE 11.1
 
  • Intensity calibration uncertainty
    • HIPE 14 introduces a new product providing an estimated breakdown of the intensity calibration uncertainty. The uncertainties are given separately for each component, as well as in a quadratic sum fashion for all statistically independent elements. This sum applies to the calibration of data in the Ta* scale. Further systematic and random error apply for data converted into different intensity scales (e.g. Tmb or Jy). See further details in section 9 of the DRG.

Revision 1012016-04-11 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page


Changed:
<
<
NEW NEW NEW Latest updates -- 4 April 2016 NEW NEW NEW
  • HIPE 14.0 has been released
  • Check the important calibration updates and associated improvements expected in the HIPE 14 reprocessed data- this includes in particular updated sideband gain ratio in bands 1 to 4
>
>
NEW NEW NEW Latest updates -- 12 April 2016 NEW NEW NEW
  • HIPE 14.1 has been released !
  • Check the important calibration updates and associated improvements expected in the HIPE 14.0.1 and upcoming 14.1 reprocessed data - this includes in particular updated sideband gain ratio in bands 1 to 4
 
  • The HIFI products now come with a frequency-dependent intensity calibration uncertainty budget
  • Channel flags for spurs are now populated in all HIFI observations (this was limited to spectral scan up to HIPE 13)
  • Re-organisation of the HIFI quality flags for a better assessement of issues being relevant or not to the end products
Line: 31 to 31
 

Reducing HIFI data

Recommended User release

Changed:
<
<
>
>
  HIPE 14 and Java 8 : please consult the HIPE Known Issues page regarding important information on issues with Java for different versions of HIPE.
Changed:
<
<

Which data would most benefit from being reprocessed in HIPE 14.0?

>
>

Which data would most benefit from being reprocessed in HIPE 14.1 ?

  If your data have been processed with the latest bulk reprocessing (13.0), HIPE 12.0 or HIPE 11.1
Added:
>
>
  • Spectra mean and median statistical numbers
    • The HIFI 14.0.1 products suffered from a minor bug leading to a wrong computation of the mean and median spectra values (i.e. a measure of the DSB continuum to first order). This has been repaied in 14.1
 
  • Intensity calibration uncertainty
    • HIPE 14 introduces a new product providing an estimated breakdown of the intensity calibration uncertainty. The uncertainties are given separately for each component, as well as in a quadratic sum fashion for all statistically independent elements. This sum applies to the calibration of data in the Ta* scale. Further systematic and random error apply for data converted into different intensity scales (e.g. Tmb or Jy). See further details in section 9 of the DRG.
Line: 323 to 326
 
Changed:
<
<		-- DavidTeyssier , CarolynMcCoey, SylvieFBeaulieu and RengelMiriam - 4 April 2016 -
>
>		-- DavidTeyssier , CarolynMcCoey, SylvieFBeaulieu and RengelMiriam - 11 April 2016 -
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521151" name="freq_vel_3.PDF" path="freq_vel_3.PDF" size="312508" user="Main.DavidTeyssier" version="1"

Revision 1002016-04-10 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page

Line: 279 to 279
 
18_0 02-07-14   12.1 Update of the list of corrupted data-frames for flagging at Level 0, Absolute Pointing Error (APE) added to allow a quality flag to be raised in future releases in case of large pointing offsets, updates to allow IF-dependent corrections to sideband ratio, update to spur tables to store entry of spurs found by the ICC or a Key Programme, addition of an instrument uncertainty table (currently only placeholder)
22_0 09-04-15   13.0 Update of the beam parameters (coupling efficiencies, HPBW, 2-D beam maps - see also the beam section above), addition of a new table of expected spectral spurs to be masked by the pipeline in Spectral Scans, addition of a database of Electrical Standing Wave correction solution for observations in bands 6 and 7, addition of a new Uncertainty table that will be propagated into an error budget in HIPE 14 (currently only placeholder), introduction of IF-dependent sideband gain ratio (currently only true for the lower end of band 2a), update of the list of corrupted data-frames for flagging at Level 0
24_0 10-12-15   14.0 Major update of all sideband ratio tables. Introduction of uncertainty budget tables. Introduction of spur warning tables for all point and mapping observations. Update of some Electrical Standing Wave correction solutions. Added some spur flag tables for a handful of spectral scans, as well as some remaining corrupted data-frame entries for flagging at level 0. Updated smoothing width for load chop processing in band 6a, allowing to mitigate the affect of strong artefacts present at tunings between 1457 and 1459 GHz.
Changed:
<
<
25_0 To be released soon   14.1 Minor update to table of BBIDs in order to allow processing of a handful of calibration observations.
>
>
25_0 01-04-15   14.1 Minor update to table of BBIDs in order to allow processing of a handful of calibration observations.
 

Outstanding calibration issues

Revision 992016-04-04 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page


Changed:
<
<
NEW NEW NEW Latest updates -- 14 December 2015 NEW NEW NEW
>
>
NEW NEW NEW Latest updates -- 4 April 2016 NEW NEW NEW
 
  • HIPE 14.0 has been released
  • Check the important calibration updates and associated improvements expected in the HIPE 14 reprocessed data- this includes in particular updated sideband gain ratio in bands 1 to 4
  • The HIFI products now come with a frequency-dependent intensity calibration uncertainty budget
  • Channel flags for spurs are now populated in all HIFI observations (this was limited to spectral scan up to HIPE 13)
  • Re-organisation of the HIFI quality flags for a better assessement of issues being relevant or not to the end products
Added:
>
>
  • Users wishing to import their HIFI data into Gildas/CLASS can now do it directly with the Gildas oct15 release without the need of any previous conversion
 

Observing with HIFI

  • Useful Documentation
Line: 35 to 36
 

Which data would most benefit from being reprocessed in HIPE 14.0?

Changed:
<
<		If your data have been processed with the latest bulk reprocessing (13.0), HIPE 12.0 or HIPE 11.1
>
>		 If your data have been processed with the latest bulk reprocessing (13.0), HIPE 12.0 or HIPE 11.1
 
  • Intensity calibration uncertainty
    • HIPE 14 introduces a new product providing an estimated breakdown of the intensity calibration uncertainty. The uncertainties are given separately for each component, as well as in a quadratic sum fashion for all statistically independent elements. This sum applies to the calibration of data in the Ta* scale. Further systematic and random error apply for data converted into different intensity scales (e.g. Tmb or Jy). See further details in section 9 of the DRG.
Line: 54 to 55
 
  • Flags in OFF positions.
    • In HIPE 13 the data used in the OFF positions were already processed up to an equivalent Level 2 calibration (both in intensity and frequency) in order to be directly comparable to the ON-target data. In HIPE 14, those OFF spectra will also hold spur and warning flags that will be propagated from the ON-targert ones. This feature is for example very interesting in order to perform a deconvolution of the OFF spectras in a spectral scan and be able to compare the OFF spectra to the level 2.5 deconvolved products of those observations.
Changed:
<
<		If your data have been processed with a version earlier than 11.1
>
>		 If your data have been processed with a version earlier than 11.1
 
  • Solar System Object ephemerides
    • A bug (see here) was fixed in the calculation of SSO ephemeris positions (ra_centre/dec_centre) that led to offsets of up to ~10". This bug affected users that use cubes in a co-moving frame (including the standard Level 2.5 cubes), use the doOffset task, or make explicit use of ra_centre / dec_centre in their HIPE scripts. The offset is practically constant over a map and could, e.g., make the emission of a comet appear to be off-center when it really isn't. The fixed bug is in the Level 0 pipeline, which users cannot run easily. The issue was fixed in HIPE 11.1.
Changed:
<
<		If your data have been processed with a version earlier than 10.3
>
>		 If your data have been processed with a version earlier than 10.3
 
Changed:
<
<
  • DBS Observations If you have data processed with < HIPE 10
>
>
  • DBS Observations If you have data processed with < HIPE 10
 
    • A new pipeline step, mkDbsReference, calculates the differences in the chop positions in all DBS observations, and after applying the band-pass correction, stores them in a product in calibration->pipeline-out called ReferenceSpectra. This allows you to check for contamination in chop positions for all types of DBS observations.
Changed:
<
<
  • Improved pointing reconstruction for observations taken between OD 320 and 761 If you have data processed with < HIPE 9
>
>
  • Improved pointing reconstruction for observations taken between OD 320 and 761 If you have data processed with < HIPE 9
 
    • The pointing information attached to the data for observations taken between OD 320 and OD761 did not use the most accurate representation of the star tracker focal length. This was done for the bulk reprocessing with HIPE 9. As a consequence, some observations will experience a shift in astrometry, that can be as high as 8 arcsec. Details about the consequences for a particular obsid, and recipes to reconstruct the improved pointing yourself can be found at http://herschel.esac.esa.int/twiki/bin/view/Public/HowToUseImprovedPointingProducts as well as in the Level 0 section of the HIFI Data Reduction Guide.
Changed:
<
<
  • Backfilling of observational parameters If you have data processed with < HIPE 9
>
>
  • Backfilling of observational parameters If you have data processed with < HIPE 9
 
    • From 9.1 onwards, most of the observational parameters that got optimised in HSpot will be propagated into the observation context, under a new product called HifiUplinkProduct (in the auxiliary product branch). Some of these parameters will be used in the pipeline to e.g. estimate the dimension of the cubes in the mapping and have them more representative of how the map was really obtained.
Changed:
<
<
  • Solar System Object maps If you have data processed with < HIPE 9
>
>
  • Solar System Object maps If you have data processed with < HIPE 9
 
    • Cubes for moving targets are now created in the comoving frames in HIPE 9. In order to benefit from that, you should re-pipeline from Level 0 up to Level 2.5. Note that the hifiPipeline task should be called with the option "Aux=True". Please check the Pipeline chapter of the HIFI Data Reduction Guide for more details about how to do that.
Deleted:
<
<		 <!--* Calibration of data in bands 2a, 5a and 5b If you have data processed with < HIPE 8
* Un-balanced sideband gains will apply for some ranges of these bands (see also the calibration section). Reprocessing data from band 2a is recommended if your data are still processed with HIPE 6. Reprocessing data from band 5a and 5b is recommended if your data are processed with HIPE 8.

* Spectral Scan Data in bands 6 and 7 If you have data processed with < HIPE 8
* It is recommended to reprocess spectral scan data taken in band 6 and 7 if they are still processed with HIPE <8. This is not needed in data processed with HIPE 8 onward.
* Reprocessing with HIPE 9 onward will create a new Level 2.5 product containing a deconvolved single-sideband spectrum for each applicable spectrometer

* Maps If you have data processed with < HIPE 8
* Spectral cubes from mapping observations after OD 835 gridded using actual scan line and readout used in observation, earlier maps use a better approximation than in HIPE 7. These improvements are a consequence of changes in the Level 0 pipeline to read in information about how the observation was performed and require the pipeline to be re-run after running the doUplink step:

<verbatim>
from herschel.hifi.pipeline.level0 import DoUplink
doUplink=DoUplink()
backend = "WBS-V"
htp=obs.refs["level1"].product.refs[backend].product
htp1=doUplink(htp=htp.copy(), auxiliary=obs.getAuxiliary())
l1=obs.refs["level1"].product
l1.setProduct(backend,htp1)
obs.setProduct("level1",l1)
obs = hifiPipeline( obs=obs, fromLevel=1, upToLevel=2.5,save=0,apids=[backend])
#
# Now repeat replacing backend with "WBS-H", "HRS-H", "HRS-V"
</verbatim> -->
 
Line: 323 to 323
 
Changed:
<
<		-- DavidTeyssier , CarolynMcCoey, SylvieFBeaulieu and RengelMiriam - 14 December 2015 -
>
>		-- DavidTeyssier , CarolynMcCoey, SylvieFBeaulieu and RengelMiriam - 4 April 2016 -
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521151" name="freq_vel_3.PDF" path="freq_vel_3.PDF" size="312508" user="Main.DavidTeyssier" version="1"

Revision 982016-04-01 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page

Line: 256 to 256
 
  • Information about how to find what calibration version was used on your data, how to get the latest calibration, and reprocess observations with it is found in the Pipeline chapter of the HIFI Data reduction Guide.
Changed:
<
<
  • The latest calibration tree pool can also be retrieved directly here (10 December 2015).
>
>
  • The latest calibration tree pool can also be retrieved directly here (1 April 2016).
 
  • Updates to the HIFI calibration data are generally concurrent with the release of each major version of the HCSS-HIFI software. However, it is possible to have updates to the calibration data in between major releases of the software as the software and data are independent of each other, in the table below the HIPE or the OD from which the calibration updates apply are given. The calibration versions available since HIPE 5 are listed in the table below:
Line: 279 to 279
 
18_0 02-07-14   12.1 Update of the list of corrupted data-frames for flagging at Level 0, Absolute Pointing Error (APE) added to allow a quality flag to be raised in future releases in case of large pointing offsets, updates to allow IF-dependent corrections to sideband ratio, update to spur tables to store entry of spurs found by the ICC or a Key Programme, addition of an instrument uncertainty table (currently only placeholder)
22_0 09-04-15   13.0 Update of the beam parameters (coupling efficiencies, HPBW, 2-D beam maps - see also the beam section above), addition of a new table of expected spectral spurs to be masked by the pipeline in Spectral Scans, addition of a database of Electrical Standing Wave correction solution for observations in bands 6 and 7, addition of a new Uncertainty table that will be propagated into an error budget in HIPE 14 (currently only placeholder), introduction of IF-dependent sideband gain ratio (currently only true for the lower end of band 2a), update of the list of corrupted data-frames for flagging at Level 0
24_0 10-12-15   14.0 Major update of all sideband ratio tables. Introduction of uncertainty budget tables. Introduction of spur warning tables for all point and mapping observations. Update of some Electrical Standing Wave correction solutions. Added some spur flag tables for a handful of spectral scans, as well as some remaining corrupted data-frame entries for flagging at level 0. Updated smoothing width for load chop processing in band 6a, allowing to mitigate the affect of strong artefacts present at tunings between 1457 and 1459 GHz.
Added:
>
>
25_0 To be released soon   14.1 Minor update to table of BBIDs in order to allow processing of a handful of calibration observations.
 

Outstanding calibration issues

Revision 972015-12-16 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page

Line: 217 to 217
 
Sideband ratio* 1.5-4 1.3-3 3 5-6
Hot load coupling 0.1-0.8 0.2-0.4 1-1.7 2-2.3
Cold load coupling 0.1-0.2 0.1-0.3 1.7 1.5
Changed:
<
<
Hot load temperature 0.5 0.5 0.5 0.5
Cold load temperature 1 1 1 1
>
>
Hot load temperature 0.6 0.6-0.7 0.7 0.7-0.8
Cold load temperature 0.1 0.1 <0.1 <0.1
 
Opt. standing waves: loads 1.1-1.7 0.7-1.2 0.7-1.1 0.7-0.8
Changed:
<
<
Opt. standing waves: diplexer N/A 0.4-0.9 N/A 0.7-2.2
>
>
Opt. standing waves: diplexer N/A 0.4-0.9 N/A 4.2-4.9
 
Planetary model error 5 5 5 5
Beam/aperture efficiency 1.1-1.4 1.2-1.3 1-1.1 1.2-1.3
Flux loss due to pointing** 2-4 3-7 3-4 8-18

Revision 962015-12-16 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page

Line: 223 to 223
 
Opt. standing waves: diplexer N/A 0.4-0.9 N/A 0.7-2.2
Planetary model error 5 5 5 5
Beam/aperture efficiency 1.1-1.4 1.2-1.3 1-1.1 1.2-1.3
Changed:
<
<
Flux loss due to pointing** 4-7 6-11 5-7 10-20
>
>
Flux loss due to pointing** 2-4 3-7 3-4 8-18
 *Note that the "sideband gain ratio" convention used in the HIFI pipeline does not correspond to the ratio between the respective sideband gains (equal to 1 for a balanced system), but to the gain in one sideband normalised by the combined gain of the two sidebands. Consequently, perfectly-balanced sidebands will feature a sideband ratio of 0.5 (see also section 5.3.1 of the Observers' Manual, and the Intensity Calibration Framework document).

**For a point source (worst case scenario) - this flux loss estimates takes into account an APE of 1.2", a residual SIAM bias of order 1.5" and the fact that the H/V respective apertures are slightly offset from the middle synthetic aperture used to point the telescope (offset ranging between 3" in band 1 and 0.5" in band 7)

Revision 952015-12-14 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page

Line: 320 to 320
 
Changed:
<
<
>
>
 -- DavidTeyssier , CarolynMcCoey, SylvieFBeaulieu and RengelMiriam - 14 December 2015 -

META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"

Revision 942015-12-14 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page

Changed:
<
<
>
>
 
Changed:
<
<		NEW NEW NEW Latest updates -- 14 April 2015 NEW NEW NEW
  • HIPE 13.0 has been released
  • Check below the important calibration updates and associated improvements expected in the HIPE 13 reprocessed data
  • HIFI now feature a series of useful scripts
>
>
NEW NEW NEW Latest updates -- 14 December 2015 NEW NEW NEW
  • HIPE 14.0 has been released
  • Check the important calibration updates and associated improvements expected in the HIPE 14 reprocessed data- this includes in particular updated sideband gain ratio in bands 1 to 4
  • The HIFI products now come with a frequency-dependent intensity calibration uncertainty budget
  • Channel flags for spurs are now populated in all HIFI observations (this was limited to spectral scan up to HIPE 13)
  • Re-organisation of the HIFI quality flags for a better assessement of issues being relevant or not to the end products
 

Observing with HIFI

  • Useful Documentation
Changed:
<
<
>
>
 
Changed:
<
<
>
>
 

  • HIFI calibration data
    • A significant number of the data taken by HIFI in the framework of the Performance Validation (PV) and routine phase are now publicly available. There are particular caveats that apply to those data. They are described in this disclaimer note.

  • See also
Changed:
<
<
    • HIFI fact sheet A 2-page guide to the HIFI instrument
    • A&A paper describing the method used to measure stability performance with HIFI (added 4 Oct 2011)
    • A&A paper about the optimisation of mapping modes for heterodyne instruments (added 18 March 2014)]
>
>
    • HIFI fact sheet A 2-page guide to the HIFI instrument
    • A&A paper describing the method used to measure stability performance with HIFI (added 4 Oct 2011)
    • A&A paper about the optimisation of mapping modes for heterodyne instruments (added 18 March 2014)
 

Reducing HIFI data

Recommended User release

Changed:
<
<
>
>		 HIPE 14 and Java 8 : please consult the HIPE Known Issues page regarding important information on issues with Java for different versions of HIPE.
 
Changed:
<
<		HIPE 13 and Java 8 : please consult the HIPE Known Issues page regarding important information on issues with Java for different versions of HIPE.
>
>

Which data would most benefit from being reprocessed in HIPE 14.0?

 
Changed:
<
<

Which data would most benefit from being reprocessed in HIPE 13.0?

>
>		If your data have been processed with the latest bulk reprocessing (13.0), HIPE 12.0 or HIPE 11.1
 
Changed:
<
<		 If your data have been processed with the latest bulk reprocessing (12.1):
>
>
  • Intensity calibration uncertainty
    • HIPE 14 introduces a new product providing an estimated breakdown of the intensity calibration uncertainty. The uncertainties are given separately for each component, as well as in a quadratic sum fashion for all statistically independent elements. This sum applies to the calibration of data in the Ta* scale. Further systematic and random error apply for data converted into different intensity scales (e.g. Tmb or Jy). See further details in section 9 of the DRG.
 
Changed:
<
<
  • Pointing in all observations
    • A new approach to pointing reconstruction has been implemented in HIPE 13 by introducing additional correction based on the gyroscope information (see the following Pointing Information page). For the HIFI data such a correction will be applied in a conditional fashion depending on a quality figure computed for each individual observation. The new pointing reconstruction will not apply to under-performing cases, and those latter will still use the pointing files used back in HIPE 12. Details about the new attitude reconstruction, and the way it is approached and may impact the HIFI data, can be found in this memo.
>
>
  • Pointing reconstruction
    • In HIPE 14, those observations where the interlacing mode was activated (i.e. using more than just 9 guide stars in the star tracker) will benefit from an more accurate reconstructed astrometry and so positions may change in those cases. On top of that, the quality figure associated to the new gyro-propagated method introduced back in HIPE 13 (see also the following Pointing Information page) is now computed in a slightly different fashion. Since the application or not of the gyro-propagated pointing is, for HIFI, conditional upon a certain threshold on this quality (avoiding to apply the new pointing reconstruction to under-performing case), the number of case making us of one or the other approach will differ in HIPE 14 - we recall that in case the gyro-propagated method is discarded, the pointing is the same as that used back in HIPE 12.For the HIFI data such a correction is applied in a conditional fashion depending on a quality figure computed for each individual observation. The new pointing reconstruction will not apply to under-performing cases, and those latter will still use the pointing files used back in HIPE 12. Details about the new attitude reconstruction, and the way it is approached and may impact the HIFI data, can be found in this memo.
 
  • Band 6 and 7 observations
Changed:
<
<
    • From HIPE 13 onward observations in bands 6 and 7 will automatically be corrected from the Electrical Standing Wave affecting those data. The correction is based on an optimised fit to the baseline artefact stored in the HIFI calibration files and applied by the pipeline. This also means that the optimum solution may not necessarily completely correct the standing wave. Note that the continuum present in band 6 and 7 data can only be fully trusted if the Electrical Standing Wave is corrected (see also the Outstanding Calibration Issues section below). Refined solutions will be provided on a case by case basis in HIPE 14. For instructions on how to benefit from this reprocessing and see the typical improvement expected in the data please refer to this section from the HIFI Data Reduction Guide (DRG).
>
>
    • From HIPE 13 onward observations in bands 6 and 7 have been automatically corrected from the Electrical Standing Wave affecting those data. The correction is based on an optimised fit to the baseline artefact stored in the HIFI calibration files and applied by the pipeline. In HIPE 14, some of those stored solutions have been refined and so the resulting products will be improved. For instructions on how to benefit from this reprocessing and see the typical improvement expected in the data please refer to this section from the HIFI Data Reduction Guide (DRG).
 
  • Spectral Scan observations
Changed:
<
<
    • The reprocessing of spectral scans in HIPE 13 will make use of optimised mask tables for spurs and unruly baseline ranges. This will result in improved deconvolved solutions at the Level 2.5. An example of such improvement between HIPE 12 (left - red) and HIPE 13 (right - black) is shown in this plot. In order to benefit from this update you should reprocess your data from Level 0 with the new calibration tree (see section 5.4 of the DRG).
>
>
    • The reprocessing of spectral scans in HIPE 13 made use of optimised mask tables for spurs and unruly baseline ranges, resulting in improved deconvolved solutions at the Level 2.5 (an example of such improvement between HIPE 12 (left - red) and HIPE 13 (right - black) is shown in this plot - see also section 5.4 of the DRG). In HIPE 14, a dozen of additional spectral scans (mostly from the calibration programme) have been added to this list.In order to benefit from this update you should reprocess your data from Level 0 with the new calibration tree.
 
Changed:
<
<
  • Calibrated OFF position spectra (any mode)
    • In HIPE 13 the data used in the OFF positions will be processed up to an equivalent Level 2 calibration (both in intensity and frequency) and can therefore be directly compared to the ON-target data. Details about the peculiarity of each OFF spectra can be found in the cookbooks (see next section). In order to generate those spectra you need to reprocess the data from the Level 1, and can work from the calibration tree present in the data processed with HIPE 12.1 - see the pipeline section of the DRG for more details.
>
>
  • Spur warning flags
    • In HIPE 14, a new channel flag has been introduced ("warning") that will be assigned to the data based on a knowledge base of spurious features built from the spectral scan flagging (see bullet above). These flags will be applied to all point and mapping observations. They should be taken as indicative as they not necessarily accurately match spurious features in the data where they got applied. For this reason, this new flag is not honoured by any of the standard interactive post-processing tasks.
 
Changed:
<
<
  • Single Point and Spectral Mapping modes
    • The Level 2.5 products in these modes have been optimised by stitching the respective spectrometer subbands. In maps, for example, this will result in a smaller number of cubes. More details can be found in section 4.6 of the DRG.

If your data have been processed with a version earlier than 11.1:

>
>
  • Flags in OFF positions.
    • In HIPE 13 the data used in the OFF positions were already processed up to an equivalent Level 2 calibration (both in intensity and frequency) in order to be directly comparable to the ON-target data. In HIPE 14, those OFF spectra will also hold spur and warning flags that will be propagated from the ON-targert ones. This feature is for example very interesting in order to perform a deconvolution of the OFF spectras in a spectral scan and be able to compare the OFF spectra to the level 2.5 deconvolved products of those observations.
If your data have been processed with a version earlier than 11.1
 
  • Solar System Object ephemerides
Changed:
<
<
    • A bug (see here) was fixed in the calculation of SSO ephemeris positions (ra_centre/dec_centre) that led to offsets of up to ~10". This bug affected users that use cubes in a co-moving frame (including the standard Level 2.5 cubes), use the doOffset task, or make explicit use of ra_centre / dec_centre in their HIPE scripts. The offset is practically constant over a map and could, e.g., make the emission of a comet appear to be off-center when it really isn't. The fixed bug is in the Level 0 pipeline, which users cannot run easily. The issue was fixed in HIPE 11.1.

If your data have been processed with a version earlier than 10.3:

>
>
    • A bug (see here) was fixed in the calculation of SSO ephemeris positions (ra_centre/dec_centre) that led to offsets of up to ~10". This bug affected users that use cubes in a co-moving frame (including the standard Level 2.5 cubes), use the doOffset task, or make explicit use of ra_centre / dec_centre in their HIPE scripts. The offset is practically constant over a map and could, e.g., make the emission of a comet appear to be off-center when it really isn't. The fixed bug is in the Level 0 pipeline, which users cannot run easily. The issue was fixed in HIPE 11.1.
If your data have been processed with a version earlier than 10.3
 
Changed:
<
<
  • DBS Observations If you have data processed with < HIPE 10
    • A new pipeline step, mkDbsReference, calculates the differences in the chop positions in all DBS observations, and after applying the band-pass correction, stores them in a product in calibration->pipeline-out called ReferenceSpectra. This allows you to check for contamination in chop positions for all types of DBS observations.
>
>
  • DBS Observations If you have data processed with < HIPE 10
    • A new pipeline step, mkDbsReference, calculates the differences in the chop positions in all DBS observations, and after applying the band-pass correction, stores them in a product in calibration->pipeline-out called ReferenceSpectra. This allows you to check for contamination in chop positions for all types of DBS observations.
 
Changed:
<
<
  • Improved pointing reconstruction for observations taken between OD 320 and 761 If you have data processed with < HIPE 9
    • The pointing information attached to the data for observations taken between OD 320 and OD761 did not use the most accurate representation of the star tracker focal length. This was done for the bulk reprocessing with HIPE 9. As a consequence, some observations will experience a shift in astrometry, that can be as high as 8 arcsec. Details about the consequences for a particular obsid, and recipes to reconstruct the improved pointing yourself can be found at http://herschel.esac.esa.int/twiki/bin/view/Public/HowToUseImprovedPointingProducts as well as in the Level 0 section of the HIFI Data Reduction Guide.
>
>
  • Improved pointing reconstruction for observations taken between OD 320 and 761 If you have data processed with < HIPE 9
    • The pointing information attached to the data for observations taken between OD 320 and OD761 did not use the most accurate representation of the star tracker focal length. This was done for the bulk reprocessing with HIPE 9. As a consequence, some observations will experience a shift in astrometry, that can be as high as 8 arcsec. Details about the consequences for a particular obsid, and recipes to reconstruct the improved pointing yourself can be found at http://herschel.esac.esa.int/twiki/bin/view/Public/HowToUseImprovedPointingProducts as well as in the Level 0 section of the HIFI Data Reduction Guide.
 
Changed:
<
<
  • Backfilling of observational parameters If you have data processed with < HIPE 9
>
>
  • Backfilling of observational parameters If you have data processed with < HIPE 9
 
    • From 9.1 onwards, most of the observational parameters that got optimised in HSpot will be propagated into the observation context, under a new product called HifiUplinkProduct (in the auxiliary product branch). Some of these parameters will be used in the pipeline to e.g. estimate the dimension of the cubes in the mapping and have them more representative of how the map was really obtained.
Changed:
<
<
  • Solar System Object maps If you have data processed with < HIPE 9
    • Cubes for moving targets are now created in the comoving frames in HIPE 9. In order to benefit from that, you should re-pipeline from Level 0 up to Level 2.5. Note that the hifiPipeline task should be called with the option "Aux=True". Please check the Pipeline chapter of the HIFI Data Reduction Guide for more details about how to do that. 

<--* Calibration of data in bands 2a, 5a and 5b   If you have data processed with < HIPE 8  
    
 
  • 
 
      

    •  Un-balanced sideband gains will apply for some ranges of these bands (see also the calibration section). Reprocessing data from band 2a is recommended if your data are still processed with HIPE 6. Reprocessing data from band 5a and 5b is recommended if your data are processed with HIPE 8.

     

 

 
    

  •  Spectral Scan Data in bands 6 and 7  If you have data processed with < HIPE 8  
      

    •  It is recommended to reprocess spectral scan data taken in band 6 and 7 if they are still processed with HIPE <8. This is not needed in data processed with HIPE 8 onward.

      •  
    •  Reprocessing with HIPE 9 onward will create a new Level 2.5 product containing a deconvolved single-sideband spectrum for each applicable spectrometer

     

 

 
    

  •  Maps  If you have data processed with < HIPE 8  
      

    •   Spectral cubes from mapping observations after OD 835 gridded using actual scan line and readout used in observation, earlier maps use a better approximation than in HIPE 7.  These improvements are a consequence of changes in the Level 0 pipeline to read in information about how the observation was performed and require the pipeline to be re-run after running the doUplink step:

     

 



from herschel.hifi.pipeline.level0 import DoUplink
doUplink=DoUplink()
backend = "WBS-V"
htp=obs.refs["level1"].product.refs[backend].product
htp1=doUplink(htp=htp.copy(), auxiliary=obs.getAuxiliary())
l1=obs.refs["level1"].product
l1.setProduct(backend,htp1)
obs.setProduct("level1",l1)
obs = hifiPipeline( obs=obs, fromLevel=1, upToLevel=2.5,save=0,apids=[backend])
#
# Now repeat replacing backend with "WBS-H", "HRS-H", "HRS-V"

 -->
>
>
  • Solar System Object maps If you have data processed with < HIPE 9
    • Cubes for moving targets are now created in the comoving frames in HIPE 9. In order to benefit from that, you should re-pipeline from Level 0 up to Level 2.5. Note that the hifiPipeline task should be called with the option "Aux=True". Please check the Pipeline chapter of the HIFI Data Reduction Guide for more details about how to do that. <!--* Calibration of data in bands 2a, 5a and 5b If you have data processed with < HIPE 8
      * Un-balanced sideband gains will apply for some ranges of these bands (see also the calibration section). Reprocessing data from band 2a is recommended if your data are still processed with HIPE 6. Reprocessing data from band 5a and 5b is recommended if your data are processed with HIPE 8.

      * Spectral Scan Data in bands 6 and 7 If you have data processed with < HIPE 8
      * It is recommended to reprocess spectral scan data taken in band 6 and 7 if they are still processed with HIPE <8. This is not needed in data processed with HIPE 8 onward.
      * Reprocessing with HIPE 9 onward will create a new Level 2.5 product containing a deconvolved single-sideband spectrum for each applicable spectrometer

      * Maps If you have data processed with < HIPE 8
      * Spectral cubes from mapping observations after OD 835 gridded using actual scan line and readout used in observation, earlier maps use a better approximation than in HIPE 7. These improvements are a consequence of changes in the Level 0 pipeline to read in information about how the observation was performed and require the pipeline to be re-run after running the doUplink step:

      <verbatim>
      from herschel.hifi.pipeline.level0 import DoUplink
      doUplink=DoUplink()
      backend = "WBS-V"
      htp=obs.refs["level1"].product.refs[backend].product
      htp1=doUplink(htp=htp.copy(), auxiliary=obs.getAuxiliary())
      l1=obs.refs["level1"].product
      l1.setProduct(backend,htp1)
      obs.setProduct("level1",l1)
      obs = hifiPipeline( obs=obs, fromLevel=1, upToLevel=2.5,save=0,apids=[backend])
      #
      # Now repeat replacing backend with "WBS-H", "HRS-H", "HRS-V"
      </verbatim> -->
 
Added:
>
>
 
Changed:
<
<

>
>
 

Documentation and Cookbooks

  • The HIFI Launch Pad is intended to help you quickly off the ground with HIFI data reduction
Changed:
<
<
>
>
 
  • Cookbooks now exist for each HIFI observing mode
Changed:
<
<
>
>
 
  • The HIFI Data Reduction Guide and the HIFI AOT Observing Mode Release and Performance Notes contain information you need to know about HIFI data, regardless of the software you use for data reduction
Changed:
<
<
>
>
 
Changed:
<
<
  • A series of useful scripts can be found following the HIPE menu Scripts -> HIFI Useful Scripts
>
>
  • A series of useful scripts can be found following the HIPE menu Scripts -> HIFI Useful Scripts
 
  • If you want to know more about the details of the HIFI pipeline, you should look at the HIFI Pipeline Specification Document
Changed:
<
<
>
>
 
Changed:
<
<
  • If you are working in HIPE, the HIFI Data Reduction Guide will be your prime resource for all things HIFI but you should also look at the Herschel Data Analysis Guide for information about general tools, such as those for viewing and manipulating spectra and spectral cubes, and for obtaining data from the archive.
>
>
  • If you are working in HIPE, the HIFI Data Reduction Guide will be your prime resource for all things HIFI but you should also look at the Herschel Data Analysis Guide for information about general tools, such as those for viewing and manipulating spectra and spectral cubes, and for obtaining data from the archive.
 
  • The full set of online documentation for the current user release also contains detailed information about the HIFI pipeline, information about scripting in HIPE, and reference manuals for command listings. New users of HIPE are recommended to read through the Quick Start Guide and the HIPE Owner's Guide
    • You can create pdfs of each document by clicking on the pdf symbol at the top of the main page.
Changed:
<
<
>
>
 
  • The full set of Herschel documentation for the developer track is also available. This documentation contains revisions and updates to the documentation associated with the current User Release but you should also be aware that it may describe functionality not available in the current User Release
Changed:
<
<
>
>
 
Line: 129 to 107
 

Deleted:
<
<
 

Typical Data Reduction Workflow

  • Download data from the HSA. If you are downloading several observations it is better to use the tar ball.
Changed:
<
<
>
>
 
  • Inspect data quality. Both point spectra and spectral cubes can be viewed in the SpectrumExplorer. In addition to inspecting the quality of your Level 2 data, you should always check the Level 1 data for any bad scans. If data at Level 2 appears strange then looking at Level 1 data may provide insight to the problem, see the Data Primer chapter of the HIFI Data Reduction Guide for more information about levels of data
Changed:
<
<
>
>
 
  • Assess whether data should be re-pipelined.
Changed:
<
<
    • Look at the section above and also at the What's New in the latest User Release to decide if you should re-pipeline.
>
>
    • Look at the section above and also at the What's New in the latest User Release to decide if you should re-pipeline.
 
    • Note that the ICC recommends that data processing should not be allowed to fall more than one version behind the HIPE version being used.
    • Everything you need to re-pipeline data is available in the ObservationContext, and the instructions for using the pipeline, including tips on re-pipelining using new calibration data and customising the pipeline, can be found in the Pipeline chapter of the HIFI Data Reduction Guide
Changed:
<
<
>
>
 
Changed:
<
<
  • Flag data. A new user-friendly task has been developed in order to easily flag your data: FlagTool. It can be used both as a GUI (similar concept as for the FitBaseline task) or in command line
>
>
  • Flag data. A user-friendly task has been developed in order to easily flag your data: flagTool. It can be used both as a GUI (similar concept as for the fitBaseline task) or in command line
 
  • Remove standing waves. Standing waves are a common problem in HIFI data, particularly in bands 3, 4, 6, and 7. The HIFI Data Reduction Guide provides some information about typical standing waves in HIFI data and describes how to remove them in HIPE using fitHifiFringe and the modified band pass technique
Changed:
<
<
>
>
 
  • Correct baseline drift. Baselines can be flattened by subtracted, or division in the case of real continuum. This can be done in HIPE with the fitBaseline task
Changed:
<
<
>
>
 
  • Fold frequency switch data. Frequency switch data is not folded in the pipeline and can be done with the doFold task.
Changed:
<
<
>
>
 
  • Average together H and V polarisation.
    • The noise estimates given by HSpot assume that the H and V polarisations are averaged together. This can be done in HIPE with the PolarPair task, which resamples the spectra to the same frequency scale (they are frequency calibrated with different comb measurements) and then averages them. Alternatively, you can use the Accumulate task, which allows you to specify the resampling width if you desire.
Changed:
<
<
>
>
 
    • Note that differences may be seen in H and V profiles, see the note on H and V positions. If you are particularly interested in the spatial structure of your source you may prefer not to average the H and V polarisations together.
Changed:
<
<
>
>
 
    • You can also average maps taken with the H and V polarisations:
Changed:
<
<
>
>
 
  • Spectral manipulation. The Spectral Toolbox in HIPE allows you to perform typical spectral arithmetic tasks on spectra and spectral cubes
Changed:
<
<
>
>
 
  • Fit lines. The SpectrumFitterGUI, which is also a part of the Spectral Toolbox, can be used to fit line profiles.
Changed:
<
<
>
>
 
  • Save modified data You can save data to a pool, or store your session in order to continue working in HIPE.
Changed:
<
<
>
>
 
Changed:
<
<
  • You may wish to export data as FITS, ASCII or use the HiClass task to export data as CLASS readable FITs files.
    • Saving as FITS
    • Saving as ASCII
    • HiClass task.
      • Warning, important WARNING! When converting to velocity, CLASS corrects the reference frequency (the frequency at the reference channel) for the Doppler shift, but not the channel width so there is a very small mismatch between the channel width before and after Doppler correction. Lines are shifted from the correct velocity as a consequence, with an increasing shift away from the reference channel. The ICC recommends that line frequencies are checked in HIPE before exporting to CLASS.
>
>
  • You may wish to export data as FITS, ASCII or use the hiClass task to export data as CLASS readable FITs files.
    • Saving as FITS
    • Saving as ASCII
    • HiClass task - Note, however, that GILDAS/Class can now directly read the HCSS-generated FITS files (as of release October 2015) and so it is no longer required to convert the HCSS FITS files into a dedicated format. More details about this new FITS reader can be found in this report.
 
Changed:
<
<
  • Line identification. IdentifyLines is a new task that allows you to identify lines in your spectrum and then run a comparison of known lines with a Linelist. To learn how to use the task identifyLines you will need the following three files:
>
>
  • Line identification. IdentifyLines and exportLines are two new tasks that allow you to identify and export lines in your spectrum, and then run a comparison of known lines with a Linelist. To learn how to use the tasks identifyLines and exportLines, you will need the following three files, and we direct you to HIFI Data Reduction Guide for further details on how to run the tasks:
 
Changed:
<
<
>
>
 

Dedicated data reduction tips for Spectral Scan data

  • The data needs to be deconvolved to a single sideband solution. This can be done with the doDeconvolution task, which works with an ObservationContext.
Changed:
<
<
>
>
 
Changed:
<
<
  • It is important to clean the data before deconvolving it: this means removing standing waves, correcting baselines, and flagging out any spurs or bad data that the pipeline missed. The calibration tree for HIPE 13 provides optimised mask tables for each spectral scan obsids so it is recommended that your make use of those, typically be reprocessing your data with HIPE 13 and this calibration file (see above).
>
>
  • It is important to clean the data before deconvolving it: this means removing standing waves, correcting baselines, and flagging out any spurs or bad data that the pipeline missed. The calibration tree for HIPE 14 provides optimised mask tables for each spectral scan obsids so it is recommended that your make use of those, typically be reprocessing your data with HIPE 14 and this calibration file (see above).
 
  • If you are planning to deconvolve your data with CLASS, please contact the Helpdesk, and cc to Claudia Comito (email:ccomito@ph1.uni-koeln.de) if you need further details.
Changed:
<
<
>
>
 

Dedicated data reduction tips for Mapping observations

  • Spectral cubes from all HIFI mapping observations are produced as part of the SPG pipeline (that is, the pipeline run for the HSA) and are now found in the Level 2.5 product. However, baseline and standing wave corrections are not done automatically in the pipeline because of the risk of harming the scientific content of the data. This can now be customised in the Level 2.5 step and be done prior to the cube creation.

  • It is strongly recommended that you inspect your Level 2 HTPs for baseline drift and residual standing waves, and decide if some, or all, datasets need to be cleaned up before re-running the gridding task.
Changed:
<
<
  • Regridding in HIPE > 9 will automatically use the comoving frame for SSOs. Considering the section above, mapping observations of SSOs will benefit from reprocessing in HIPE > 11.1.
>
>
  • Regridding in HIPE > 9 will automatically use the comoving frame for SSOs. Considering the section above, mapping observations of SSOs will benefit from reprocessing in HIPE > 11.1.
 
  • You can then grid your cleaned data into a spectral cube using the doGridding task, you can also use this task to customise the cube creation to best suit your science goals.
Changed:
<
<
>
>
 

Dedicated data reduction tips for Strong continuum observations

  • Strong continuum sources
    • An optional step in the pipeline can be used to remove standing waves arising in the loads, this is particularly effective for strong continuum sources. A report describing the technique can be found here. Instructions for using this modified passband technique are available in the Standing Wave Removal chapter of the HIFI Data Reduction Guide:
Changed:
<
<
>
>
 

HIFI performance and calibration

Line: 222 to 196
 
  • Intensity calibration: Intensity Calibration Framework document
  • Sideband ratio
    • Sideband ratio technical note: This note describes the general calibration problem associated with the sideband gain imbalance existing in double-sideband heterodyne systems such as HIFI, and provides some recipes and calibration tables applicable to the particular case of the HIFI mixers.
Changed:
<
<
>
>
    • http://arxiv.org/abs/1404.2806: "The effect of sideband ratio on line intensity for Herschel/HIFI", Experimental Astronomy.
    • After an extensive investigation, the HIFI ICC has introduced updated sideband gain ratio in the calibration tree. A final documentation is still in preparation but the overview of the sideband gain ratio frequency dependence in each band is illustrated in this plot.
 
Changed:
<
<
>
>
 
Changed:
<
<
  • This FTP repository of system temperature plots across the IF for each HIFI subband (updated 24 Jan 2011) inform about the variation in system temperature (sensitivity) across the HIFI intermediate frequency band at various frequencies.
>
>
  • This FTP repository of system temperature plots across the IF for each HIFI subband (updated 24 Jan 2011) inform about the variation in system temperature (sensitivity) across the HIFI intermediate frequency band at various frequencies.
 

Calibration error budget

Line Intensity Calibration Accuracy

Changed:
<
<		The following table provides the percentage flux error associated with each component of the error budget (from Roelfsema et al. 2012).
>
>		The following table provides the percentage flux error associated with each component of the error budget. These numbers are updates from the Roelfsema et al. 2012 paper and reflect what was implemented in the calibration files of HIPE 14. The detailed frequency dependence of this uncertainty budget is shown in this plot.
 
Error source Bands 1/2 Bands 3/4 Band 5 Bands 6/7
Changed:
<
<
Sideband ratio* 3-4 4-6 4-6 5-8
Hot load coupling < 1 < 1 < 1 < 1
Cold load coupling < 1 < 1 < 1 < 1
Hot load temperature < 1 < 1 < 1 < 1
Cold load temperature < 1 < 1 < 1 < 1
Planetary model error < 3 < 3 < 3 < 3
Beam efficiency < 5 < 5 < 10 < 5
Pointing < 1 < 2 < 2 < 4
Opt. standing waves 4 4 3 3
>
>
Sideband ratio* 1.5-4 1.3-3 3 5-6
Hot load coupling 0.1-0.8 0.2-0.4 1-1.7 2-2.3
Cold load coupling 0.1-0.2 0.1-0.3 1.7 1.5
Hot load temperature 0.5 0.5 0.5 0.5
Cold load temperature 1 1 1 1
Opt. standing waves: loads 1.1-1.7 0.7-1.2 0.7-1.1 0.7-0.8
Opt. standing waves: diplexer N/A 0.4-0.9 N/A 0.7-2.2
Planetary model error 5 5 5 5
Beam/aperture efficiency 1.1-1.4 1.2-1.3 1-1.1 1.2-1.3
Flux loss due to pointing** 4-7 6-11 5-7 10-20
*Note that the "sideband gain ratio" convention used in the HIFI pipeline does not correspond to the ratio between the respective sideband gains (equal to 1 for a balanced system), but to the gain in one sideband normalised by the combined gain of the two sidebands. Consequently, perfectly-balanced sidebands will feature a sideband ratio of 0.5 (see also section 5.3.1 of the Observers' Manual, and the Intensity Calibration Framework document).
 
Changed:
<
<		*Note that the "sideband gain ratio" convention used in the HIFI pipeline does not correspond to the ratio between the respective sideband gains (equal to 1 for a balanced system), but to the gain in one sideband normalised by the combined gain of the two sidebands. Consequently, perfectly-balanced sidebands will feature a sideband ratio of 0.5 (see also section 5.3.1 of the Observers' Manual, and the Intensity Calibration Framework document).
>
>		**For a point source (worst case scenario) - this flux loss estimates takes into account an APE of 1.2", a residual SIAM bias of order 1.5" and the fact that the H/V respective apertures are slightly offset from the middle synthetic aperture used to point the telescope (offset ranging between 3" in band 1 and 0.5" in band 7)
 
Changed:
<
<		Frequency Calibration Accuracy
>
>		Frequency Calibration Accuracy
 
Spectrometer Resolution Bandwidth (kHz) Accuracy (kHz)
HRS (high-res. mode, Hann appodisation) 207 - 364 20
Line: 267 to 243
  HIFI beam coupling efficiencies
Changed:
<
<		A revision of the HIFI beam analysis has been performed, based on a multi-epoch set of measurements on Mars. As a result, a more accurate description of the HIFI beams has been obtained, leading to some change in the beam coupling efficiencies, typically a drop. The details of the analysis and the revised coupling efficiencies are given in this release note. For the record, the first release of the HIFI beam efficiencies was described in this document (17 Nov 2010). Please note that these updates will only be featured in the HIFI calibration tree in HIPE 13.
>
>		A revision of the HIFI beam analysis has been performed, based on a multi-epoch set of measurements on Mars. As a result, a more accurate description of the HIFI beams has been obtained, leading to some change in the beam coupling efficiencies, typically a drop. The details of the analysis and the revised coupling efficiencies are given in this release note. For the record, the first release of the HIFI beam efficiencies was described in this document (17 Nov 2010). Please note that these updates will only be featured in the HIFI calibration tree from HIPE 13 onwards.
 
Changed:
<
<		HIFI beam maps and Encircled Energy Fraction
>
>		HIFI beam maps and Encircled Energy Fraction
  In addition to this, a detailed description of the HIFI beam models is provided as 2-D beam maps, 1-D azimuthal beam averages, and Encircled Energy Fractions (EEF). Each are provided at two frequencies per mixer band and polarisation, and can be extrapolated (within the LO frequencies of a given band) as a function of wavelength. The release notes provides further details about the delivered files and how to use them. In particular, some useful functions are provided in this Jython script.

The beam material is concatenated as a single FITS file per mixer, frequency, and polarisation. Each file has three extensions tables: one for the 2-D beam model maps (2705x2705 pixels), one for the EEF, and one for the 1-D azimuthal beam averages. Note that 2-D beam model of smaller size (201x201 pixels) will also be made available in the calibration tree (not before HIPE 13), and used in the generation of Level 2.5 cubes by the pipeline. The following page provides links to each individual file (27 Mb each). Plots of the 1-D azimuthal beam averages, and EEF for each band can be found in the Appendix B of the release note.

Access to the calibration tree:

Changed:
<
<
  • Information about what calibration is found in the HIFI ObservationContext is found in the Data Primer of the HIFI Data Reduction Guide.
>
>
  • Information about what calibration is found in the HIFI ObservationContext is found in the Data Primer of the HIFI Data Reduction Guide.
 
Changed:
<
<
  • Information about how to find what calibration version was used on your data, how to get the latest calibration, and reprocess observations with it is found in the Pipeline chapter of the HIFI Data reduction Guide.
>
>
  • Information about how to find what calibration version was used on your data, how to get the latest calibration, and reprocess observations with it is found in the Pipeline chapter of the HIFI Data reduction Guide.
 
Changed:
<
<
  • The latest calibration tree pool can also be retrieved directly here (11 April 2015).
>
>
  • The latest calibration tree pool can also be retrieved directly here (10 December 2015).
 
  • Updates to the HIFI calibration data are generally concurrent with the release of each major version of the HCSS-HIFI software. However, it is possible to have updates to the calibration data in between major releases of the software as the software and data are independent of each other, in the table below the HIPE or the OD from which the calibration updates apply are given. The calibration versions available since HIPE 5 are listed in the table below:
Line: 302 to 277
 
14_0 04-07-13   11.0 Update of the list of corrupted data-frames for flagging at Level 0
15_0 18-03-14   12.0 Update of the list of corrupted data-frames for flagging at Level 0, ingestion of the Electrical Standing Wave correction reference table
18_0 02-07-14   12.1 Update of the list of corrupted data-frames for flagging at Level 0, Absolute Pointing Error (APE) added to allow a quality flag to be raised in future releases in case of large pointing offsets, updates to allow IF-dependent corrections to sideband ratio, update to spur tables to store entry of spurs found by the ICC or a Key Programme, addition of an instrument uncertainty table (currently only placeholder)
Changed:
<
<
22_0 11-04-15   13.0 Update of the beam parameters (coupling efficiencies, HPBW, 2-D beam maps - see also the beam section above), addition of a new table of expected spectral spurs to be masked by the pipeline in Spectral Scans, addition of a database of Electrical Standing Wave correction solution for observations in bands 6 and 7, addition of a new Error table that will be propagated into an error budget in HIPE 14 (currently only placeholder), introduction of IF-dependent sideband gain ratio (currently only true for the lower end of band 2a), update of the list of corrupted data-frames for flagging at Level 0

  • The sideband ratio assumed in the pipeline as of the latest calibration tree pool are illustrated in this plot.

  • The HIFI beam release #1 (see section above) including updated aperture efficiencies, main-beam efficiencies, and HPBW is not yet reflected in the HIFI caltree.
>
>
22_0 09-04-15   13.0 Update of the beam parameters (coupling efficiencies, HPBW, 2-D beam maps - see also the beam section above), addition of a new table of expected spectral spurs to be masked by the pipeline in Spectral Scans, addition of a database of Electrical Standing Wave correction solution for observations in bands 6 and 7, addition of a new Uncertainty table that will be propagated into an error budget in HIPE 14 (currently only placeholder), introduction of IF-dependent sideband gain ratio (currently only true for the lower end of band 2a), update of the list of corrupted data-frames for flagging at Level 0
24_0 10-12-15   14.0 Major update of all sideband ratio tables. Introduction of uncertainty budget tables. Introduction of spur warning tables for all point and mapping observations. Update of some Electrical Standing Wave correction solutions. Added some spur flag tables for a handful of spectral scans, as well as some remaining corrupted data-frame entries for flagging at level 0. Updated smoothing width for load chop processing in band 6a, allowing to mitigate the affect of strong artefacts present at tunings between 1457 and 1459 GHz.
 

Outstanding calibration issues

Changed:
<
<
>
>
 
  • Standing waves:
    • there are several standing waves that can affect the HIFI data at various processing levels. Their nature and impact on the HIFI calibration are described in the Standing Wave technical note and, more briefly, in the Standing Wave removal chapter of the HIFI Data Reduction Guide
Changed:
<
<
>
>
 
    • There are various techniques currently offered to clean these baseline distortions:
Changed:
<
<
      • Remove sine waves, or combinations of sine waves. Check the bullet about usage of the fitHifiFringe task in the data reduction section above.
>
>
      • Remove sine waves, or combinations of sine waves. Check the bullet about usage of the fitHifiFringe task in the data reduction section above.
 
      • Electrical Standing Wave (aka Matching technique): to correct for the IF standing wave present in bands 6 and 7
Changed:
<
<
        • A new task hebCorrection has been implemented in HIPE 12.0 and is being used by default by the pipeline in HIPE 13 (see also the Reducing HIFI data section above). We warn that the task does not guarantee an accurate correction in all circumstances.
        • The task is described in the following document. The mathematical background can be found in this proceeding. Unlike the script made available in previous version of HIPE, this task is applicable to any HIFI observing mode.
>
>
        • A new task hebCorrection has been implemented in HIPE 12.0 and is being used by default by the pipeline in HIPE 13 (see also the Reducing HIFI data section above). We warn that the task does not guarantee an accurate correction in all circumstances .
        • The task is described in the following document. The mathematical background can be found in this proceeding. Unlike the script made available in previous version of HIPE, this task is applicable to any HIFI observing mode.
 
Changed:
<
<
      • in case of strong source continuum some standing waves can be enhanced, as described in the Alternative Calibration Scheme report. Such standing waves can be significantly reduced in amplitude using an alternative pipeline algorithm known as the Modified Passband Technique , which is described in the Standing Wave Removal chapter of the HIFI Data Reduction Guide. Alternatively, you can run the Level1PipelineAlgo_hc_filtered_v1.py script. 
        <--Note that in 9.0, a bug still affects the doFilterLoad task in such a way that it will ignore the user-input parameters, resulting in the task always applying the "cubic_spline" method with default parameters - i.e. the "fft" method will not be applied despite being set in the task inputs. This will be fixed in 9.1.-->
>
>
      • in case of strong source continuum some standing waves can be enhanced, as described in the Alternative Calibration Scheme report. Such standing waves can be significantly reduced in amplitude using an alternative pipeline algorithm known as the Modified Passband Technique , which is described in the Standing Wave Removal chapter of the HIFI Data Reduction Guide. Alternatively, you can run the Level1PipelineAlgo_hc_filtered_v1.py script.
 
Changed:
<
<
>
>
 
Changed:
<
<
  • Baseline removal: imperfect ON-OFF calibration scheme can result in residual baseline distortion (not necessarily standing waves). Check the bullet about usage of the fitBaseline task in the data reduction section above.
>
>
  • Baseline removal: imperfect ON-OFF calibration scheme can result in residual baseline distortion (not necessarily standing waves). Check the bullet about usage of the fitBaseline task in the data reduction section above.
 
Changed:
<
<
  • Spectral purity: there are places over the HIFI frequency range where the Local Oscillator does not offer a single frequency tone, so that spurious spectral line may end up in the data, and the calibration of the targeted line gets affected. A dedicated cookbook on the matter is in preparation. In the meantime, we refer to section 5.4.6 of the Observer's Manual, and to the release notes provided above.
>
>
  • Spectral purity: there are places over the HIFI frequency range where the Local Oscillator does not offer a single frequency tone, so that spurious spectral line may end up in the data, and the calibration of the targeted line gets affected. A dedicated cookbook on the matter is in preparation. In the meantime, we refer to section 5.4.6 of the Observer's Manual, and to the release notes provided above.
 
Changed:
<
<
  • Spurs: Spectral Spurs are narrow and broad spectral features due to instabilities in the Local Oscillator. Up to HIPE 12, spurs have been assigned by the pipeline in an automatic fashion. In HIPE 13, this is no longer the case and the masks are assigned based on a priori tables provided by the ICC in the calibration tree. As of now, all spectral scans benefit from such tables, but only a handful of single point and spectral mapping observations. This limitation has little impact on the top level data though since no pipeline module makes use of these masks in those modes. A list of the spurs registered up to HIPE 12 can be found in section 5.4.6 of the Observer's Manual. This table will be updated soon based on the new statistics derived from the HIPE 13 mask tables.
>
>
  • Spurs: Spectral Spurs are narrow and broad spectral features due to instabilities in the Local Oscillator. Up to HIPE 12, spurs have been assigned by the pipeline in an automatic fashion. In HIPE 13, this is no longer the case and the masks are assigned based on a priori tables provided by the ICC in the calibration tree. As of now, all spectral scans benefit from such tables, but only a handful of single point and spectral mapping observations. This limitation has little impact on the top level data though since no pipeline module makes use of these masks in those modes. A list of the spurs registered up to HIPE 12 can be found in section 5.4.6 of the Observer's Manual. This table will be updated soon based on the new statistics derived from the HIPE 13 mask tables.
 

Interest Groups and Scripts

Changed:
<
<
  • HIPI: a library of HIFI Plug-In's has been prepared by the NHSC - check the HIPI website for more information. In HIPE 13, most of those utilities have been ported to Useful scripts (Menu Scripts > HIFI Useful Scripts)
>
>
  • HIPI: a library of HIFI Plug-In's has been prepared by the NHSC - check the HIPI website for more information. From HIPE 13 onwards, most of those utilities have been ported to Useful scripts (Menu Scripts > HIFI Useful Scripts)
 
  • The following interest groups relate to processing of observations taken with HIFI. The links provided allow subscription to these interest groups.
Changed:
<
<
    • subscribe to PACS, SPIRE and HIFI spectral maps interest group
    • subscribe to HIFI point source and spectral scan interest group
  • User scripts: Users are welcome to submit scripts they believe could be of general interest to the community to the Herschel Helpdesk.
>
>
    • subscribe to PACS, SPIRE and HIFI spectral maps interest group
    • subscribe to HIFI point source and spectral scan interest group
  • User scripts: Users are welcome to submit scripts they believe could be of general interest to the community to the Herschel Helpdesk.
 

Further Information

Changed:
<
<
>
>
 
Changed:
<
<		 -- DavidTeyssier , CarolynMcCoey, SylvieFBeaulieu and RengelMiriam - 14 Apr 2015-

>
>		-- DavidTeyssier , CarolynMcCoey, SylvieFBeaulieu and RengelMiriam - 14 December 2015 -
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521151" name="freq_vel_3.PDF" path="freq_vel_3.PDF" size="312508" user="Main.DavidTeyssier" version="1"

Revision 932015-12-10 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page

Line: 388 to 388
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1428653680" name="HIFI_PointingAccuracyNote.pdf" path="HIFI_PointingAccuracyNote.pdf" size="382180" user="Main.RengelMiriam" version="2"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1323853023" name="Alternative_Calibration_Schemes_v1.0.pdf" path="Alternative_Calibration_Schemes_v1.0.pdf" size="2883364" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attachment="HIFI_Uncertainty_HIPE14.0.png" attr="" comment="" date="1449696012" name="HIFI_Uncertainty_HIPE14.0.png" path="HIFI_Uncertainty_HIPE14.0.png" size="129135" user="DavidTeyssier" version="1"
Changed:
<
<
META FILEATTACHMENT attachment="HIFI_SBR_summary_HIPE14.0.png" attr="" comment="" date="1449697572" name="HIFI_SBR_summary_HIPE14.0.png" path="HIFI_SBR_summary_HIPE14.0.png" size="85817" user="DavidTeyssier" version="1"
>
>
META FILEATTACHMENT attachment="HIFI_SBR_summary_HIPE14.0.png" attr="" comment="" date="1449762023" name="HIFI_SBR_summary_HIPE14.0.png" path="HIFI_SBR_summary_HIPE14.0.png" size="110415" user="DavidTeyssier" version="2"
 
META FILEATTACHMENT attachment="class-herschel-fits.pdf" attr="" comment="" date="1449699928" name="class-herschel-fits.pdf" path="class-herschel-fits.pdf" size="429415" user="DavidTeyssier" version="1"

Revision 922015-12-09 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page

Line: 387 to 387
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309110936" name="freq_vel_1.1.pdf" path="freq_vel_1.1.pdf" size="261519" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1428653680" name="HIFI_PointingAccuracyNote.pdf" path="HIFI_PointingAccuracyNote.pdf" size="382180" user="Main.RengelMiriam" version="2"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1323853023" name="Alternative_Calibration_Schemes_v1.0.pdf" path="Alternative_Calibration_Schemes_v1.0.pdf" size="2883364" user="Main.DavidTeyssier" version="1"
Added:
>
>
META FILEATTACHMENT attachment="HIFI_Uncertainty_HIPE14.0.png" attr="" comment="" date="1449696012" name="HIFI_Uncertainty_HIPE14.0.png" path="HIFI_Uncertainty_HIPE14.0.png" size="129135" user="DavidTeyssier" version="1"
META FILEATTACHMENT attachment="HIFI_SBR_summary_HIPE14.0.png" attr="" comment="" date="1449697572" name="HIFI_SBR_summary_HIPE14.0.png" path="HIFI_SBR_summary_HIPE14.0.png" size="85817" user="DavidTeyssier" version="1"
META FILEATTACHMENT attachment="class-herschel-fits.pdf" attr="" comment="" date="1449699928" name="class-herschel-fits.pdf" path="class-herschel-fits.pdf" size="429415" user="DavidTeyssier" version="1"

Revision 912015-05-07 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page

Line: 280 to 280
 
  • Information about how to find what calibration version was used on your data, how to get the latest calibration, and reprocess observations with it is found in the Pipeline chapter of the HIFI Data reduction Guide.
Changed:
<
<
  • The latest calibration tree pool can be also retrieved directly here (11 April 2015).
>
>
  • The latest calibration tree pool can also be retrieved directly here (11 April 2015).
 
  • Updates to the HIFI calibration data are generally concurrent with the release of each major version of the HCSS-HIFI software. However, it is possible to have updates to the calibration data in between major releases of the software as the software and data are independent of each other, in the table below the HIPE or the OD from which the calibration updates apply are given. The calibration versions available since HIPE 5 are listed in the table below:

Revision 902015-04-14 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page

Changed:
<
<		NEW NEW NEW Latest updates -- 11 April 2015 NEW NEW NEW
  • HIPE 13.0 will be released any time soon
>
>		NEW NEW NEW Latest updates -- 14 April 2015 NEW NEW NEW
  • HIPE 13.0 has been released
 
  • Check below the important calibration updates and associated improvements expected in the HIPE 13 reprocessed data
Changed:
<
<
  • HIFI will feature a series of useful scripts
>
>
  • HIFI now feature a series of useful scripts
 

Observing with HIFI

  • Useful Documentation
Line: 352 to 352
 
Changed:
<
<		-- DavidTeyssier , CarolynMcCoey, SylvieFBeaulieu and RengelMiriam - 09 Apr 2015-
>
>		-- DavidTeyssier , CarolynMcCoey, SylvieFBeaulieu and RengelMiriam - 14 Apr 2015-
 

Revision 892015-04-11 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page

Changed:
<
<		NEW NEW NEW Latest updates -- 9 April 2015 NEW NEW NEW
  • HIPE 13.0 has been released
  • Check the important calibration updates and associated improvements expected in the HIPE 13 reprocessed data
  • HIFI now features a series of useful scripts
>
>		NEW NEW NEW Latest updates -- 11 April 2015 NEW NEW NEW
  • HIPE 13.0 will be released any time soon
  • Check below the important calibration updates and associated improvements expected in the HIPE 13 reprocessed data
  • HIFI will feature a series of useful scripts
 

Observing with HIFI

  • Useful Documentation
Line: 280 to 280
 
  • Information about how to find what calibration version was used on your data, how to get the latest calibration, and reprocess observations with it is found in the Pipeline chapter of the HIFI Data reduction Guide.
Changed:
<
<
  • The latest calibration tree pool can be also retrieved directly here (9 April 2015).
>
>
  • The latest calibration tree pool can be also retrieved directly here (11 April 2015).
 
  • Updates to the HIFI calibration data are generally concurrent with the release of each major version of the HCSS-HIFI software. However, it is possible to have updates to the calibration data in between major releases of the software as the software and data are independent of each other, in the table below the HIPE or the OD from which the calibration updates apply are given. The calibration versions available since HIPE 5 are listed in the table below:
Line: 302 to 302
 
14_0 04-07-13   11.0 Update of the list of corrupted data-frames for flagging at Level 0
15_0 18-03-14   12.0 Update of the list of corrupted data-frames for flagging at Level 0, ingestion of the Electrical Standing Wave correction reference table
18_0 02-07-14   12.1 Update of the list of corrupted data-frames for flagging at Level 0, Absolute Pointing Error (APE) added to allow a quality flag to be raised in future releases in case of large pointing offsets, updates to allow IF-dependent corrections to sideband ratio, update to spur tables to store entry of spurs found by the ICC or a Key Programme, addition of an instrument uncertainty table (currently only placeholder)
Changed:
<
<
21_0 09-04-15   13.0 Update of the beam parameters (coupling efficiencies, HPBW, 2-D beam maps - see also the beam section above), addition of a new table of expected spectral spurs to be masked by the pipeline in Spectral Scans, addition of a database of Electrical Standing Wave correction solution for observations in bands 6 and 7, addition of a new Error table that will be propagated into an error budget in HIPE 14 (currently only placeholder), introduction of IF-dependent sideband gain ratio (currently only true for the lower end of band 2a), update of the list of corrupted data-frames for flagging at Level 0
>
>
22_0 11-04-15   13.0 Update of the beam parameters (coupling efficiencies, HPBW, 2-D beam maps - see also the beam section above), addition of a new table of expected spectral spurs to be masked by the pipeline in Spectral Scans, addition of a database of Electrical Standing Wave correction solution for observations in bands 6 and 7, addition of a new Error table that will be propagated into an error budget in HIPE 14 (currently only placeholder), introduction of IF-dependent sideband gain ratio (currently only true for the lower end of band 2a), update of the list of corrupted data-frames for flagging at Level 0
 

  • The sideband ratio assumed in the pipeline as of the latest calibration tree pool are illustrated in this plot.
Line: 385 to 385
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1345720705" name="HIFI_SBR_Summary_V9.0.png" path="HIFI_SBR_Summary_V9.0.png" size="88066" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="HIFI_SBR_Cookbook_V1.0" date="1372338964" name="HIFI_SBR_Cookbook_V1.0.pdf" path="HIFI_SBR_Cookbook_V1.0.pdf" size="1474786" user="Main.SylvieFBeaulieu" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309110936" name="freq_vel_1.1.pdf" path="freq_vel_1.1.pdf" size="261519" user="Main.DavidTeyssier" version="1"
Changed:
<
<
META FILEATTACHMENT attachment="HIFI_PointingAccuracyNote.pdf" attr="" comment="" date="1428653680" name="HIFI_PointingAccuracyNote.pdf" path="HIFI_PointingAccuracyNote.pdf" size="382180" stream="HIFI_PointingAccuracyNote.pdf" user="Main.RengelMiriam" version="2"
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1428653680" name="HIFI_PointingAccuracyNote.pdf" path="HIFI_PointingAccuracyNote.pdf" size="382180" user="Main.RengelMiriam" version="2"
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1323853023" name="Alternative_Calibration_Schemes_v1.0.pdf" path="Alternative_Calibration_Schemes_v1.0.pdf" size="2883364" user="Main.DavidTeyssier" version="1"

Revision 882015-04-10 - RengelMiriam

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page

Line: 385 to 385
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1345720705" name="HIFI_SBR_Summary_V9.0.png" path="HIFI_SBR_Summary_V9.0.png" size="88066" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="HIFI_SBR_Cookbook_V1.0" date="1372338964" name="HIFI_SBR_Cookbook_V1.0.pdf" path="HIFI_SBR_Cookbook_V1.0.pdf" size="1474786" user="Main.SylvieFBeaulieu" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309110936" name="freq_vel_1.1.pdf" path="freq_vel_1.1.pdf" size="261519" user="Main.DavidTeyssier" version="1"
Changed:
<
<
META FILEATTACHMENT attachment="HIFI_PointingAccuracyNote.pdf" attr="" comment="" date="1428593439" name="HIFI_PointingAccuracyNote.pdf" path="HIFI_PointingAccuracyNote.pdf" size="382180" stream="HIFI_PointingAccuracyNote.pdf" user="Main.DavidTeyssier" version="2"
>
>
META FILEATTACHMENT attachment="HIFI_PointingAccuracyNote.pdf" attr="" comment="" date="1428653680" name="HIFI_PointingAccuracyNote.pdf" path="HIFI_PointingAccuracyNote.pdf" size="382180" stream="HIFI_PointingAccuracyNote.pdf" user="Main.RengelMiriam" version="2"
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1323853023" name="Alternative_Calibration_Schemes_v1.0.pdf" path="Alternative_Calibration_Schemes_v1.0.pdf" size="2883364" user="Main.DavidTeyssier" version="1"

Revision 872015-04-09 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web page

Line: 385 to 385
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1345720705" name="HIFI_SBR_Summary_V9.0.png" path="HIFI_SBR_Summary_V9.0.png" size="88066" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="HIFI_SBR_Cookbook_V1.0" date="1372338964" name="HIFI_SBR_Cookbook_V1.0.pdf" path="HIFI_SBR_Cookbook_V1.0.pdf" size="1474786" user="Main.SylvieFBeaulieu" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309110936" name="freq_vel_1.1.pdf" path="freq_vel_1.1.pdf" size="261519" user="Main.DavidTeyssier" version="1"
Changed:
<
<
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1428483971" name="HIFI_PointingAccuracyNote.pdf" path="HIFI_PointingAccuracyNote.pdf" size="381099" user="Main.DavidTeyssier" version="1"
>
>
META FILEATTACHMENT attachment="HIFI_PointingAccuracyNote.pdf" attr="" comment="" date="1428593439" name="HIFI_PointingAccuracyNote.pdf" path="HIFI_PointingAccuracyNote.pdf" size="382180" stream="HIFI_PointingAccuracyNote.pdf" user="Main.DavidTeyssier" version="2"
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1323853023" name="Alternative_Calibration_Schemes_v1.0.pdf" path="Alternative_Calibration_Schemes_v1.0.pdf" size="2883364" user="Main.DavidTeyssier" version="1"

Revision 862015-04-09 - RengelMiriam

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Changed:
<
<

HIFI instrument and calibration web pages

>
>

HIFI instrument and calibration web page

 
Changed:
<
<		NEW NEW NEW Latest updates -- 19 September 2014 NEW NEW NEW
  • New beam and coupling efficiency release - see the "HIFI performance and calibration" section
>
>		NEW NEW NEW Latest updates -- 9 April 2015 NEW NEW NEW
  • HIPE 13.0 has been released
  • Check the important calibration updates and associated improvements expected in the HIPE 13 reprocessed data
  • HIFI now features a series of useful scripts
 

Observing with HIFI

  • Useful Documentation
Line: 16 to 18
 

  • HIFI calibration data
Changed:
<
<
    • A significant number of the data taken by HIFI in the framework of the Performance Validation (PV) and routine phase are now publicly available. There are particular caveats that apply to those data. They are described in this disclaimer note
>
>
    • A significant number of the data taken by HIFI in the framework of the Performance Validation (PV) and routine phase are now publicly available. There are particular caveats that apply to those data. They are described in this disclaimer note.
 
  • See also
    • HIFI fact sheet A 2-page guide to the HIFI instrument
    • A&A paper describing the method used to measure stability performance with HIFI (added 4 Oct 2011)
Changed:
<
<
    • A&A paper about the optimization of mapping modes for heterodyne instruments (added 18 March 2014)]
>
>
    • A&A paper about the optimisation of mapping modes for heterodyne instruments (added 18 March 2014)]
 

Reducing HIFI data

Recommended User release

Changed:
<
<
>
>
 
Changed:
<
<		Mac OS X and Java 7 Issues : please consult the HIPE Known Issues page regarding important information on issues with Java for different versions of HIPE.
>
>		HIPE 13 and Java 8 : please consult the HIPE Known Issues page regarding important information on issues with Java for different versions of HIPE.
 
Changed:
<
<

Which data would most benefit from being reprocessed in HIPE 12.1?

>
>

Which data would most benefit from being reprocessed in HIPE 13.0?

 
Changed:
<
<		 If your data have been processed with the latest bulk reprocessing (11.1):
>
>		 If your data have been processed with the latest bulk reprocessing (12.1):
 
Changed:
<
<
  • Rotated maps
    • Rotated cubes are now produced in the Level 2.5 product by default for maps carried out with a non-zero position angle, a non-rotated cube is also produced.
>
>
  • Pointing in all observations
    • A new approach to pointing reconstruction has been implemented in HIPE 13 by introducing additional correction based on the gyroscope information (see the following Pointing Information page). For the HIFI data such a correction will be applied in a conditional fashion depending on a quality figure computed for each individual observation. The new pointing reconstruction will not apply to under-performing cases, and those latter will still use the pointing files used back in HIPE 12. Details about the new attitude reconstruction, and the way it is approached and may impact the HIFI data, can be found in this memo.
 
Changed:
<
<
  • Solar System Object ephemerides
    • A bug (see here) was fixed in the calculation of SSO ephemeris positions (ra_centre/dec_centre) that led to offsets of up to ~10". This bug affected users that use cubes in a co-moving frame (including the standard level 2.5 cubes), use the doOffset task, or make explicit use of ra_centre / dec_centre in their HIPE scripts. The offset is practically constant over a map and could, e.g., make the emission of a comet appear to be off-center when it really isn't. The fixed bug is in the level 0 pipeline, which users cannot run easily. The issue is fixed by the HIPE 11.1 bulk reprocessing.
>
>
  • Band 6 and 7 observations
    • From HIPE 13 onward observations in bands 6 and 7 will automatically be corrected from the Electrical Standing Wave affecting those data. The correction is based on an optimised fit to the baseline artefact stored in the HIFI calibration files and applied by the pipeline. This also means that the optimum solution may not necessarily completely correct the standing wave. Note that the continuum present in band 6 and 7 data can only be fully trusted if the Electrical Standing Wave is corrected (see also the Outstanding Calibration Issues section below). Refined solutions will be provided on a case by case basis in HIPE 14. For instructions on how to benefit from this reprocessing and see the typical improvement expected in the data please refer to this section from the HIFI Data Reduction Guide (DRG).

  • Spectral Scan observations
    • The reprocessing of spectral scans in HIPE 13 will make use of optimised mask tables for spurs and unruly baseline ranges. This will result in improved deconvolved solutions at the Level 2.5. An example of such improvement between HIPE 12 (left - red) and HIPE 13 (right - black) is shown in this plot. In order to benefit from this update you should reprocess your data from Level 0 with the new calibration tree (see section 5.4 of the DRG).

  • Calibrated OFF position spectra (any mode)
    • In HIPE 13 the data used in the OFF positions will be processed up to an equivalent Level 2 calibration (both in intensity and frequency) and can therefore be directly compared to the ON-target data. Details about the peculiarity of each OFF spectra can be found in the cookbooks (see next section). In order to generate those spectra you need to reprocess the data from the Level 1, and can work from the calibration tree present in the data processed with HIPE 12.1 - see the pipeline section of the DRG for more details.
 
Changed:
<
<
  • Improved pointing reconstruction for observations taken before OD 320
>
>
  • Single Point and Spectral Mapping modes
    • The Level 2.5 products in these modes have been optimised by stitching the respective spectrometer subbands. In maps, for example, this will result in a smaller number of cubes. More details can be found in section 4.6 of the DRG.

If your data have been processed with a version earlier than 11.1:

  • Solar System Object ephemerides
    • A bug (see here) was fixed in the calculation of SSO ephemeris positions (ra_centre/dec_centre) that led to offsets of up to ~10". This bug affected users that use cubes in a co-moving frame (including the standard Level 2.5 cubes), use the doOffset task, or make explicit use of ra_centre / dec_centre in their HIPE scripts. The offset is practically constant over a map and could, e.g., make the emission of a comet appear to be off-center when it really isn't. The fixed bug is in the Level 0 pipeline, which users cannot run easily. The issue was fixed in HIPE 11.1.
  If your data have been processed with a version earlier than 10.3:

  • DBS Observations If you have data processed with < HIPE 10
Changed:
<
<
    • A new pipeline step, mkDbsReference, calculates the differences the chop positions in all DBS observations and after applying the band-pass correction stores them in a product in calibration->pipeline-out called ReferenceSpectra. This allows you to check for contamination in chop positions for all types of DBS observations.
>
>
    • A new pipeline step, mkDbsReference, calculates the differences in the chop positions in all DBS observations, and after applying the band-pass correction, stores them in a product in calibration->pipeline-out called ReferenceSpectra. This allows you to check for contamination in chop positions for all types of DBS observations.
 
  • Improved pointing reconstruction for observations taken between OD 320 and 761 If you have data processed with < HIPE 9
Changed:
<
<
    • The pointing information attached to the data for observations taken between OD 320 and OD761 did not use the most accurate representation of the star tracker focal length. This will be done for the bulk reprocessing with HIPE 9. As a consequence, some observations will experience a shift in astrometry, that can be as high as 8 arcsec. All the details about the consequences for a particular obsid, and recipes to reconstruct yourself the improved pointing can be found at http://herschel.esac.esa.int/twiki/bin/view/Public/HowToUseImprovedPointingProducts as well as in the level0 section of the Hifi Data Reduction Guide
>
>
    • The pointing information attached to the data for observations taken between OD 320 and OD761 did not use the most accurate representation of the star tracker focal length. This was done for the bulk reprocessing with HIPE 9. As a consequence, some observations will experience a shift in astrometry, that can be as high as 8 arcsec. Details about the consequences for a particular obsid, and recipes to reconstruct the improved pointing yourself can be found at http://herschel.esac.esa.int/twiki/bin/view/Public/HowToUseImprovedPointingProducts as well as in the Level 0 section of the HIFI Data Reduction Guide.
 
  • Backfilling of observational parameters If you have data processed with < HIPE 9
    • From 9.1 onwards, most of the observational parameters that got optimised in HSpot will be propagated into the observation context, under a new product called HifiUplinkProduct (in the auxiliary product branch). Some of these parameters will be used in the pipeline to e.g. estimate the dimension of the cubes in the mapping and have them more representative of how the map was really obtained.

  • Solar System Object maps If you have data processed with < HIPE 9
Changed:
<
<
    • Cubes for moving targets are now created in the comoving frames in HIPE 9. In order to benefit from that, you should re-pipeline from level 0 up to level 2.5. Note the hifiPipeline task should be called with the option "Aux=True". Please check the Pipeline chapter of the HIFI Data Reduction Guide for more details about how to do that.
>
>
    • Cubes for moving targets are now created in the comoving frames in HIPE 9. In order to benefit from that, you should re-pipeline from Level 0 up to Level 2.5. Note that the hifiPipeline task should be called with the option "Aux=True". Please check the Pipeline chapter of the HIFI Data Reduction Guide for more details about how to do that.
 
Changed:
<
<
>
>
 
Changed:
<
<
>
>
 

Documentation and Cookbooks

  • The HIFI Launch Pad is intended to help you quickly off the ground with HIFI data reduction
Changed:
<
<

>
>
 
Added:
>
>
 
  • The HIFI Data Reduction Guide and the HIFI AOT Observing Mode Release and Performance Notes contain information you need to know about HIFI data, regardless of the software you use for data reduction
Changed:
<
<
>
>
 
Added:
>
>
  • A series of useful scripts can be found following the HIPE menu Scripts -> HIFI Useful Scripts
 
  • If you want to know more about the details of the HIFI pipeline, you should look at the HIFI Pipeline Specification Document
Changed:
<
<
>
>
 
Changed:
<
<
  • If you are working in HIPE, the HIFI Data Reduction Guide will be your prime resource for all things HIFI but you should also look at the Herschel Data Analysis Guide for information about general tools, such as those for viewing and manipulating spectra and spectral cubes, and for obtaining data from the archive.
    • [http://herschel.esac.esa.int/hcss-doc-12.0/load/dag/html/dag.html Herschel Data Analysis Guide]]
>
>
  • If you are working in HIPE, the HIFI Data Reduction Guide will be your prime resource for all things HIFI but you should also look at the Herschel Data Analysis Guide for information about general tools, such as those for viewing and manipulating spectra and spectral cubes, and for obtaining data from the archive.
 
  • The full set of online documentation for the current user release also contains detailed information about the HIFI pipeline, information about scripting in HIPE, and reference manuals for command listings. New users of HIPE are recommended to read through the Quick Start Guide and the HIPE Owner's Guide
    • You can create pdfs of each document by clicking on the pdf symbol at the top of the main page.
Changed:
<
<
>
>
 
  • The full set of Herschel documentation for the developer track is also available. This documentation contains revisions and updates to the documentation associated with the current User Release but you should also be aware that it may describe functionality not available in the current User Release
Changed:
<
<
>
>
 
Line: 118 to 133
 

Typical Data Reduction Workflow

  • Download data from the HSA. If you are downloading several observations it is better to use the tar ball.
Changed:
<
<
>
>
 
Changed:
<
<
  • Inspect data quality. Both point spectra and spectral cubes can be viewed in the SpectrumExplorer. In addition to inspecting the quality of your level 2 data, you should always check the level 1 data for any bad scans. If data at level 2 appears strange then looking at level 1 data may provide insight to the problem, see the Data Primer chapter of the HIFI Data Reduction Guide for more information about levels of data
>
>
  • Inspect data quality. Both point spectra and spectral cubes can be viewed in the SpectrumExplorer. In addition to inspecting the quality of your Level 2 data, you should always check the Level 1 data for any bad scans. If data at Level 2 appears strange then looking at Level 1 data may provide insight to the problem, see the Data Primer chapter of the HIFI Data Reduction Guide for more information about levels of data
 
  • Assess whether data should be re-pipelined.
    • Look at the section above and also at the What's New in the latest User Release to decide if you should re-pipeline.
    • Note that the ICC recommends that data processing should not be allowed to fall more than one version behind the HIPE version being used.
Changed:
<
<
    • Everything you need to re-pipeline data is available in the ObservationContext, and the instructions for using the pipeline, including tips on re-pipelining using new calibration data and customising the pipeline, can be found in the Pipeline chapter of the Hifi Data Reduction Guide
>
>
    • Everything you need to re-pipeline data is available in the ObservationContext, and the instructions for using the pipeline, including tips on re-pipelining using new calibration data and customising the pipeline, can be found in the Pipeline chapter of the HIFI Data Reduction Guide
 
Changed:
<
<
  • Flag data. A new user-friendly task has been developed in order to easily flag your data: FlagTool. It can be used both as a GUI (similar concept as for the FitBaseline task) or in command line
>
>
  • Flag data. A new user-friendly task has been developed in order to easily flag your data: FlagTool. It can be used both as a GUI (similar concept as for the FitBaseline task) or in command line
 
  • Remove standing waves. Standing waves are a common problem in HIFI data, particularly in bands 3, 4, 6, and 7. The HIFI Data Reduction Guide provides some information about typical standing waves in HIFI data and describes how to remove them in HIPE using fitHifiFringe and the modified band pass technique
Changed:
<
<
>
>
 
  • Correct baseline drift. Baselines can be flattened by subtracted, or division in the case of real continuum. This can be done in HIPE with the fitBaseline task
Changed:
<
<
>
>
 
  • Fold frequency switch data. Frequency switch data is not folded in the pipeline and can be done with the doFold task.
Changed:
<
<
>
>
 
  • Average together H and V polarisation.
    • The noise estimates given by HSpot assume that the H and V polarisations are averaged together. This can be done in HIPE with the PolarPair task, which resamples the spectra to the same frequency scale (they are frequency calibrated with different comb measurements) and then averages them. Alternatively, you can use the Accumulate task, which allows you to specify the resampling width if you desire.
Changed:
<
<
>
>
 
    • Note that differences may be seen in H and V profiles, see the note on H and V positions. If you are particularly interested in the spatial structure of your source you may prefer not to average the H and V polarisations together.
    • You can also average maps taken with the H and V polarisations:
Changed:
<
<
>
>
 
  • Spectral manipulation. The Spectral Toolbox in HIPE allows you to perform typical spectral arithmetic tasks on spectra and spectral cubes
Changed:
<
<
>
>
 
  • Fit lines. The SpectrumFitterGUI, which is also a part of the Spectral Toolbox, can be used to fit line profiles.
Changed:
<
<
>
>
 
  • Save modified data You can save data to a pool, or store your session in order to continue working in HIPE.
Changed:
<
<
>
>
 
  • You may wish to export data as FITS, ASCII or use the HiClass task to export data as CLASS readable FITs files.
Changed:
<
<
    • Saving as FITS
    • Saving as ASCII
    • HiClass task.
      • Warning, important WARNING! When converting to velocity, CLASS corrects the reference frequency (the frequency at the reference channel) for the Doppler shift, but not the channel width so there is a very small mismatch between the channel width before and after Doppler correction. Lines are shifted from the correct velocity as a consequence, with an increasing shift away from the reference channel.. The ICC recommends that line frequencies are checked in HIPE before exporting to CLASS.
>
>
    • Saving as FITS
    • Saving as ASCII
    • HiClass task.
      • Warning, important WARNING! When converting to velocity, CLASS corrects the reference frequency (the frequency at the reference channel) for the Doppler shift, but not the channel width so there is a very small mismatch between the channel width before and after Doppler correction. Lines are shifted from the correct velocity as a consequence, with an increasing shift away from the reference channel. The ICC recommends that line frequencies are checked in HIPE before exporting to CLASS.

 

Dedicated data reduction tips for Spectral Scan data

  • The data needs to be deconvolved to a single sideband solution. This can be done with the doDeconvolution task, which works with an ObservationContext.
Changed:
<
<
>
>
 
Changed:
<
<
  • It is important to clean the data before deconvolving it, this means removing standing waves, correcting baselines and flagging out any spurs or bad data that the pipeline missed.
>
>
  • It is important to clean the data before deconvolving it: this means removing standing waves, correcting baselines, and flagging out any spurs or bad data that the pipeline missed. The calibration tree for HIPE 13 provides optimised mask tables for each spectral scan obsids so it is recommended that your make use of those, typically be reprocessing your data with HIPE 13 and this calibration file (see above).
 
Changed:
<
<
  • If you are planning to deconvolve your data with CLASS, please contact the Helpdesk, or Claudia Comito (email:ccomito@ph1.uni-koeln.de) if you need further details.
>
>
  • If you are planning to deconvolve your data with CLASS, please contact the Helpdesk, and cc to Claudia Comito (email:ccomito@ph1.uni-koeln.de) if you need further details.
 
Changed:
<
<
>
>
 

Dedicated data reduction tips for Mapping observations

Changed:
<
<
  • Spectral cubes from all HIFI mapping observations are produced as part of the SPG pipeline (that is, the pipeline run for the HSA) and are now found in the level 2.5 product. However, baseline and standing wave corrections are not done automatically in the pipeline because of the risk of harming the scientific content of the data. This can now be customised in the level 2.5 step and be done prior to the cube creation.
>
>
  • Spectral cubes from all HIFI mapping observations are produced as part of the SPG pipeline (that is, the pipeline run for the HSA) and are now found in the Level 2.5 product. However, baseline and standing wave corrections are not done automatically in the pipeline because of the risk of harming the scientific content of the data. This can now be customised in the Level 2.5 step and be done prior to the cube creation.
 
Changed:
<
<
  • It is strongly recommended that you inspect your level 2 HTPs for baseline drift and residual standing waves and decide if some, or all, datasets need to be cleaned up before re-running the gridding task.
>
>
  • It is strongly recommended that you inspect your Level 2 HTPs for baseline drift and residual standing waves, and decide if some, or all, datasets need to be cleaned up before re-running the gridding task.
 
Changed:
<
<
  • Regridding in HIPE > 9 will automatically use the comoving frame for SSOs. See the section above about the benefits from reprocessing with HIPE 11 for more details on how to do that
>
>
  • Regridding in HIPE > 9 will automatically use the comoving frame for SSOs. Considering the section above, mapping observations of SSOs will benefit from reprocessing in HIPE > 11.1.
 
  • You can then grid your cleaned data into a spectral cube using the doGridding task, you can also use this task to customise the cube creation to best suit your science goals.
Changed:
<
<
>
>
 

Dedicated data reduction tips for Strong continuum observations

  • Strong continuum sources
Changed:
<
<
    • An optional step in the pipeline can be used to remove standing waves arising in the loads, this is particularly effective for strong continuum sources. A report describing the technique can be found here. Instructions for using this modified passband technique are available in the Standing Wave Removal chapter of the HIFI Data Reduction Guide:
>
>
    • An optional step in the pipeline can be used to remove standing waves arising in the loads, this is particularly effective for strong continuum sources. A report describing the technique can be found here. Instructions for using this modified passband technique are available in the Standing Wave Removal chapter of the HIFI Data Reduction Guide:
 

HIFI performance and calibration

Line: 202 to 221
 
Changed:
<
<
    • Side band ratio technical note: This note describes the general calibration problem associated with the side-band gain imbalance existing in double-side-band heterodyne systems such as HIFI, and provides some recipes and calibration tables applicable to the particular case of the HIFI mixers.
>
>
    • Sideband ratio technical note: This note describes the general calibration problem associated with the sideband gain imbalance existing in double-sideband heterodyne systems such as HIFI, and provides some recipes and calibration tables applicable to the particular case of the HIFI mixers.
 
Changed:
<
<
  • Beam calibration
>
>
  • Beam calibration ( see also HIFI beam sub-section thereafter )
 
Changed:
<
<
>
>
 
Changed:
<
<
  • This FTP repository of system temperature plots across the IF for each HIFI subband (updated 24 Jan 2011) inform about the variation in system temperature (sensitivity) across the HIFI intermediate frequency band at various frequencies
>
>
  • This FTP repository of system temperature plots across the IF for each HIFI subband (updated 24 Jan 2011) inform about the variation in system temperature (sensitivity) across the HIFI intermediate frequency band at various frequencies.
 

Calibration error budget

Line: 220 to 239
 The following table provides the percentage flux error associated with each component of the error budget (from Roelfsema et al. 2012).

Error source Bands 1/2 Bands 3/4 Band 5 Bands 6/7
Changed:
<
<
Side-band ratio* 3-4 4-6 4-6 5-8
>
>
Sideband ratio* 3-4 4-6 4-6 5-8
 
Hot load coupling < 1 < 1 < 1 < 1
Cold load coupling < 1 < 1 < 1 < 1
Hot load temperature < 1 < 1 < 1 < 1
Line: 230 to 249
 
Pointing < 1 < 2 < 2 < 4
Opt. standing waves 4 4 3 3
Changed:
<
<		*Note that the "side-band gain ratio" convention used in the HIFI pipeline does not correspond to the ratio between the respective side-band gains (equal to 1 for a balanced system), but to the gain in one side-band normalised by the combined gain of the two side-band. Consequently, perfectly-balanced side-bands will feature a side-band ratio of 0.5 (see also section 5.3.1 of the Observers' Manual, and the Intensity Calibration Framework document).
>
>		*Note that the "sideband gain ratio" convention used in the HIFI pipeline does not correspond to the ratio between the respective sideband gains (equal to 1 for a balanced system), but to the gain in one sideband normalised by the combined gain of the two sidebands. Consequently, perfectly-balanced sidebands will feature a sideband ratio of 0.5 (see also section 5.3.1 of the Observers' Manual, and the Intensity Calibration Framework document).
  Frequency Calibration Accuracy

Spectrometer Resolution Bandwidth (kHz) Accuracy (kHz)
Changed:
<
<
HRS (high-res. mode, Hann appodization) 207 - 364 20
>
>
HRS (high-res. mode, Hann appodisation) 207 - 364 20
 
WBS 1100 100

Dominant Contributions:

Line: 252 to 271
  HIFI beam maps and Encircled Energy Fraction
Changed:
<
<		In addition to this, a detailed description of the HIFI beam models is provided as 2-D beam maps, 1-D azimuthal beam averages, and Encircled Energy Fractions (EEF). Each are provided at two frequencies per mixer band and polarisation, and can be extrapolated (within the LO frequencies of a given band) as function of wavelength. The release note provides further details about the delivered files and how to use them. In particular, some useful functions are provided in this Jython script.
>
>		In addition to this, a detailed description of the HIFI beam models is provided as 2-D beam maps, 1-D azimuthal beam averages, and Encircled Energy Fractions (EEF). Each are provided at two frequencies per mixer band and polarisation, and can be extrapolated (within the LO frequencies of a given band) as a function of wavelength. The release notes provides further details about the delivered files and how to use them. In particular, some useful functions are provided in this Jython script.
 
Changed:
<
<		The beam material is concatenated as single FITS file per mixer, frequency and polarisation. Each file has three extensions table: one for the 2-D beam model maps (2705x2705 pixels), one for the EEF, and one for the 1-D azimuthal beam averages. Note that 2-D beam model of smaller size (201x201 pixels) will also be made available in the calibration tree (not before HIFI 13), and used in the generation of level 2.5 cubes by the pipeline. The following page provides links to each individual file (27 Mb each). Plots of the 1-D azimuthal beam averages and EEF for each band can be found in the Appendix B of the release note.
>
>		The beam material is concatenated as a single FITS file per mixer, frequency, and polarisation. Each file has three extensions tables: one for the 2-D beam model maps (2705x2705 pixels), one for the EEF, and one for the 1-D azimuthal beam averages. Note that 2-D beam model of smaller size (201x201 pixels) will also be made available in the calibration tree (not before HIPE 13), and used in the generation of Level 2.5 cubes by the pipeline. The following page provides links to each individual file (27 Mb each). Plots of the 1-D azimuthal beam averages, and EEF for each band can be found in the Appendix B of the release note.
 

Access to the calibration tree:

Changed:
<
<
  • Information about what calibration is found in the HIFI ObservationContext is found in the Data Primer of the HIFI Data Reduction Guide
>
>
  • Information about what calibration is found in the HIFI ObservationContext is found in the Data Primer of the HIFI Data Reduction Guide.
 
Changed:
<
<
  • Information about how to find what calibration version was used on your data, how to get the latest calibration and reprocess observations with it is found in the Pipeline chapter of the HIFI Data reduction Guide
>
>
  • Information about how to find what calibration version was used on your data, how to get the latest calibration, and reprocess observations with it is found in the Pipeline chapter of the HIFI Data reduction Guide.
 
Changed:
<
<
  • the latest calibration tree pool can be also retrieved directly here (2 July 2014)
>
>
  • The latest calibration tree pool can be also retrieved directly here (9 April 2015).
 
Changed:
<
<
  • Updates to the HIFI calibration data are generally concurrent with the release of each major version of the HCSS-HIFI software. However, it is possible to have updates to the calibration data in between major releases of the software as the software and data are independent of each other, in the table below the HIPE or the OD from which the calibration updates apply are given. The calibration versions available since HIPE 5 are listed in the table below
>
>
  • Updates to the HIFI calibration data are generally concurrent with the release of each major version of the HCSS-HIFI software. However, it is possible to have updates to the calibration data in between major releases of the software as the software and data are independent of each other, in the table below the HIPE or the OD from which the calibration updates apply are given. The calibration versions available since HIPE 5 are listed in the table below:
 

Calibration version number Release date OD HIPE version Changes
Line: 275 to 294
 
6_0 21-06-11 779   Sideband ratios in band 2a, addition of strong spur at 1108 GHz in band 5a to spur table
7_0 12-12-11   8.0 Prevention of unnecessary Quality Flags by correction and addition of units in calibration products and correction of thresholds, addition of Quality Flags when LO multiplier currents are out of limits, improved saturated pixel flagging, addition to spur table, uplink product
8_0 03-02-12 995 8.1 Update of a priori table of IF saturations, used in order to flag bad LO data to be discarded in the deconvolution
Changed:
<
<
9_0 24-07-12   9.0 Sideband ratios in bands 5a and 5b, update of a number of engineering threshold for more accurate quality flagging related to hardware housekeeping, introduction of a list of a priori known corrupted data-frames for dedicated flagging in level0 data
>
>
9_0 24-07-12   9.0 Sideband ratios in bands 5a and 5b, update of a number of engineering threshold for more accurate quality flagging related to hardware housekeeping, introduction of a list of a priori known corrupted data-frames for dedicated flagging in Level 0 data
 
10_0 08-11-12   9.1 Introduction of a new HifiUplink product to back-fill most the observational parameters as of HSpot optimisation back into the Uplink product, update of the list of corrupted data-frames for flagging at Level 0, update of a number of engineering threshold for more accurate quality flagging related to hardware housekeeping
11_0 22-01-13   10.0 Update of the list of corrupted data-frames for flagging at Level 0, Quality flags created for SEUs (Singe Event Upsets leading to on-board software corruption)
12_0 10-05-13   10.1 Update of the list of corrupted data-frames for flagging at Level 0, updates to quality flag meta-data and uplink information for mapping modes
Line: 283 to 302
 
14_0 04-07-13   11.0 Update of the list of corrupted data-frames for flagging at Level 0
15_0 18-03-14   12.0 Update of the list of corrupted data-frames for flagging at Level 0, ingestion of the Electrical Standing Wave correction reference table
18_0 02-07-14   12.1 Update of the list of corrupted data-frames for flagging at Level 0, Absolute Pointing Error (APE) added to allow a quality flag to be raised in future releases in case of large pointing offsets, updates to allow IF-dependent corrections to side band ratio, update to spur tables to store entry of spurs found by the ICC or a Key Programme, addition of an instrument uncertainty table (currently only placeholder)
Added:
>
>
21_0 09-04-15   13.0 Update of the beam parameters (coupling efficiencies, HPBW, 2-D beam maps - see also the beam section above), addition of a new table of expected spectral spurs to be masked by the pipeline in Spectral Scans, addition of a database of Electrical Standing Wave correction solution for observations in bands 6 and 7, addition of a new Error table that will be propagated into an error budget in HIPE 14 (currently only placeholder), introduction of IF-dependent sideband gain ratio (currently only true for the lower end of band 2a), update of the list of corrupted data-frames for flagging at Level 0
 
Changed:
<
<
  • The side-band ratio assumed in the pipeline as of the latest calibration tree pool are illustrated in this plot.
>
>
  • The sideband ratio assumed in the pipeline as of the latest calibration tree pool are illustrated in this plot.
 
  • The HIFI beam release #1 (see section above) including updated aperture efficiencies, main-beam efficiencies, and HPBW is not yet reflected in the HIFI caltree.
Line: 294 to 315
 
  • Standing waves:
    • there are several standing waves that can affect the HIFI data at various processing levels. Their nature and impact on the HIFI calibration are described in the Standing Wave technical note and, more briefly, in the Standing Wave removal chapter of the HIFI Data Reduction Guide
Changed:
<
<
>
>
 
    • There are various techniques currently offered to clean these baseline distortions:
Changed:
<
<
      • Remove sine waves, or combinations of sine waves. Check the bullet about usage of the fitHifiFringe task in the data reduction section above.
>
>
      • Remove sine waves, or combinations of sine waves. Check the bullet about usage of the fitHifiFringe task in the data reduction section above.
 
      • Electrical Standing Wave (aka Matching technique): to correct for the IF standing wave present in bands 6 and 7
Changed:
<
<
        • A new task hebCorrection has been implemented in HIPE 12.0. We warn that the task is still under testing and does not guarantee an accurate correction in all circumstances.
        • The task is described in the following document. The mathematical background can be found in this proceeding. Unlike the script made available in previous version of HIPE, this task is applicable to any HIFI observing mode.
        • The task needs to work from level 1 data, then reprocess the corrected data up to level 2.5. Two example codes are provided here:
>
>
        • A new task hebCorrection has been implemented in HIPE 12.0 and is being used by default by the pipeline in HIPE 13 (see also the Reducing HIFI data section above). We warn that the task does not guarantee an accurate correction in all circumstances.
        • The task is described in the following document. The mathematical background can be found in this proceeding. Unlike the script made available in previous version of HIPE, this task is applicable to any HIFI observing mode.
        • The task needs to work from Level 1 data, then reprocess the corrected data up to Level 2.5. Two example codes are provided here:
 
      • in case of strong source continuum some standing waves can be enhanced, as described in the Alternative Calibration Scheme report. Such standing waves can be significantly reduced in amplitude using an alternative pipeline algorithm known as the Modified Passband Technique , which is described in the Standing Wave Removal chapter of the HIFI Data Reduction Guide. Alternatively, you can run the Level1PipelineAlgo_hc_filtered_v1.py script. 
        <--Note that in 9.0, a bug still affects the doFilterLoad task in such a way that it will ignore the user-input parameters, resulting in the task always applying the "cubic_spline" method with default parameters - i.e. the "fft" method will not be applied despite being set in the task inputs. This will be fixed in 9.1.-->
Changed:
<
<
>
>
 
Changed:
<
<
  • Baseline removal: imperfect ON-OFF calibration scheme can result in residual baseline distortion (not necessarily standing waves). Check the bullet about usage of the fitBaseline task in the data reduction section above.
>
>
  • Baseline removal: imperfect ON-OFF calibration scheme can result in residual baseline distortion (not necessarily standing waves). Check the bullet about usage of the fitBaseline task in the data reduction section above.
 
  • Spectral purity: there are places over the HIFI frequency range where the Local Oscillator does not offer a single frequency tone, so that spurious spectral line may end up in the data, and the calibration of the targeted line gets affected. A dedicated cookbook on the matter is in preparation. In the meantime, we refer to section 5.4.6 of the Observer's Manual, and to the release notes provided above.
Changed:
<
<
  • Spurs: Spurs are reported in HSpot when using the frequency editor and general information is also provided in the HIFI Observer's Manual (see above link). Spurs are also being checked in the pipeline and any detected feature is reported in the TrendAnalysis product of an observation context. The list of currently known spurs can be found in section 5.4.6 of the Observer's Manual. However not all possible spurs can be predicted or caught by the data processing, so that Users are invited to inform the Helpdesk about any features they found in their data and may not have been properly flagged by the system.
>
>
  • Spurs: Spectral Spurs are narrow and broad spectral features due to instabilities in the Local Oscillator. Up to HIPE 12, spurs have been assigned by the pipeline in an automatic fashion. In HIPE 13, this is no longer the case and the masks are assigned based on a priori tables provided by the ICC in the calibration tree. As of now, all spectral scans benefit from such tables, but only a handful of single point and spectral mapping observations. This limitation has little impact on the top level data though since no pipeline module makes use of these masks in those modes. A list of the spurs registered up to HIPE 12 can be found in section 5.4.6 of the Observer's Manual. This table will be updated soon based on the new statistics derived from the HIPE 13 mask tables.
 

Interest Groups and Scripts

Changed:
<
<
  • HIPI: a library of HIFI Plug-In's has been prepared by the NHSC, and is regularly upgraded with new features - check the HIPI website for more information
>
>
  • HIPI: a library of HIFI Plug-In's has been prepared by the NHSC - check the HIPI website for more information. In HIPE 13, most of those utilities have been ported to Useful scripts (Menu Scripts > HIFI Useful Scripts)
 
  • The following interest groups relate to processing of observations taken with HIFI. The links provided allow subscription to these interest groups.
    • subscribe to PACS, SPIRE and HIFI spectral maps interest group
    • subscribe to HIFI point source and spectral scan interest group
Line: 331 to 352
 
Changed:
<
<		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 03 March 2015
>
>		-- DavidTeyssier , CarolynMcCoey, SylvieFBeaulieu and RengelMiriam - 09 Apr 2015-

 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521151" name="freq_vel_3.PDF" path="freq_vel_3.PDF" size="312508" user="Main.DavidTeyssier" version="1"
Line: 362 to 385
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1345720705" name="HIFI_SBR_Summary_V9.0.png" path="HIFI_SBR_Summary_V9.0.png" size="88066" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="HIFI_SBR_Cookbook_V1.0" date="1372338964" name="HIFI_SBR_Cookbook_V1.0.pdf" path="HIFI_SBR_Cookbook_V1.0.pdf" size="1474786" user="Main.SylvieFBeaulieu" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309110936" name="freq_vel_1.1.pdf" path="freq_vel_1.1.pdf" size="261519" user="Main.DavidTeyssier" version="1"
Added:
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1428483971" name="HIFI_PointingAccuracyNote.pdf" path="HIFI_PointingAccuracyNote.pdf" size="381099" user="Main.DavidTeyssier" version="1"
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1323853023" name="Alternative_Calibration_Schemes_v1.0.pdf" path="Alternative_Calibration_Schemes_v1.0.pdf" size="2883364" user="Main.DavidTeyssier" version="1"
Deleted:
<
<
META FILEATTACHMENT attachment="HIFI_PointingAccuracyNote.pdf" attr="" comment="" date="1428483971" name="HIFI_PointingAccuracyNote.pdf" path="HIFI_PointingAccuracyNote.pdf" size="381099" stream="HIFI_PointingAccuracyNote.pdf" user="Main.DavidTeyssier" version="1"

Revision 852015-04-08 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 346 to 346
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1395147438" name="hebCorrection_externalModels_example.py.txt" path="hebCorrection_externalModels_example.py.txt" size="2508" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309104004" name="Sherry_et_al_AAS217_poster.pdf" path="Sherry_et_al_AAS217_poster.pdf" size="104086" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521138" name="freq_framework.pdf" path="freq_framework.pdf" size="185626" user="Main.DavidTeyssier" version="1"
Added:
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1428441791" name="1342180554_band4b.png" path="1342180554_band4b.png" size="56249" user="Main.DavidTeyssier" version="1"
 
META FILEATTACHMENT attr="" autoattached="1" comment="CASSIS lineList for the idenfifyLines task (HDRG, chapter 'The HIFI line identification tool')" date="1412866259" name="16293_SIS.txt" path="16293_SIS.txt" size="175836" user="Main.SylvieFBeaulieu" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1392023495" name="spatial_response_framework_v1.9.pdf" path="spatial_response_framework_v1.9.pdf" size="603181" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309104638" name="thesis_ronan_higgins_nuim.pdf" path="thesis_ronan_higgins_nuim.pdf" size="19376465" user="Main.DavidTeyssier" version="1"
Line: 362 to 363
 
META FILEATTACHMENT attr="" autoattached="1" comment="HIFI_SBR_Cookbook_V1.0" date="1372338964" name="HIFI_SBR_Cookbook_V1.0.pdf" path="HIFI_SBR_Cookbook_V1.0.pdf" size="1474786" user="Main.SylvieFBeaulieu" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309110936" name="freq_vel_1.1.pdf" path="freq_vel_1.1.pdf" size="261519" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1323853023" name="Alternative_Calibration_Schemes_v1.0.pdf" path="Alternative_Calibration_Schemes_v1.0.pdf" size="2883364" user="Main.DavidTeyssier" version="1"
Changed:
<
<
META FILEATTACHMENT attachment="1342180554_band4b.png" attr="" comment="" date="1428441791" name="1342180554_band4b.png" path="1342180554_band4b.png" size="56249" stream="1342180554_band4b.png" user="Main.DavidTeyssier" version="1"
>
>
META FILEATTACHMENT attachment="HIFI_PointingAccuracyNote.pdf" attr="" comment="" date="1428483971" name="HIFI_PointingAccuracyNote.pdf" path="HIFI_PointingAccuracyNote.pdf" size="381099" stream="HIFI_PointingAccuracyNote.pdf" user="Main.DavidTeyssier" version="1"

Revision 842015-04-07 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 362 to 362
 
META FILEATTACHMENT attr="" autoattached="1" comment="HIFI_SBR_Cookbook_V1.0" date="1372338964" name="HIFI_SBR_Cookbook_V1.0.pdf" path="HIFI_SBR_Cookbook_V1.0.pdf" size="1474786" user="Main.SylvieFBeaulieu" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309110936" name="freq_vel_1.1.pdf" path="freq_vel_1.1.pdf" size="261519" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1323853023" name="Alternative_Calibration_Schemes_v1.0.pdf" path="Alternative_Calibration_Schemes_v1.0.pdf" size="2883364" user="Main.DavidTeyssier" version="1"
Added:
>
>
META FILEATTACHMENT attachment="1342180554_band4b.png" attr="" comment="" date="1428441791" name="1342180554_band4b.png" path="1342180554_band4b.png" size="56249" stream="1342180554_band4b.png" user="Main.DavidTeyssier" version="1"

Revision 832015-03-03 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 203 to 203
 
  • Intensity calibration: Intensity Calibration Framework document
  • Side band ratio
    • Side band ratio technical note: This note describes the general calibration problem associated with the side-band gain imbalance existing in double-side-band heterodyne systems such as HIFI, and provides some recipes and calibration tables applicable to the particular case of the HIFI mixers.
Added:
>
>
 
Line: 330 to 331
 
Changed:
<
<		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 21 Jan 2015
>
>		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 03 March 2015
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521151" name="freq_vel_3.PDF" path="freq_vel_3.PDF" size="312508" user="Main.DavidTeyssier" version="1"

Revision 822015-01-21 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 298 to 298
 
      • Remove sine waves, or combinations of sine waves. Check the bullet about usage of the fitHifiFringe task in the data reduction section above.
      • Electrical Standing Wave (aka Matching technique): to correct for the IF standing wave present in bands 6 and 7
        • A new task hebCorrection has been implemented in HIPE 12.0. We warn that the task is still under testing and does not guarantee an accurate correction in all circumstances.
Changed:
<
<
        • The task is described in the following document. Unlike the script made available in previous version of HIPE, this task is applicable to any HIFI observing mode.
>
>
        • The task is described in the following document. The mathematical background can be found in this proceeding. Unlike the script made available in previous version of HIPE, this task is applicable to any HIFI observing mode.
 
Line: 330 to 330
 
Changed:
<
<		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 1 Dec 2014
>
>		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 21 Jan 2015
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521151" name="freq_vel_3.PDF" path="freq_vel_3.PDF" size="312508" user="Main.DavidTeyssier" version="1"
Line: 351 to 351
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1417446382" name="getHifiBeam.py.txt" path="getHifiBeam.py.txt" size="5224" user="Main.DavidTeyssier" version="2"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1359637018" name="HIFI_Calibration_ReleaseNotes.pdf" path="HIFI_Calibration_ReleaseNotes.pdf" size="178716" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1406141554" name="HIFISubbands.pdf" path="HIFISubbands.pdf" size="8715" user="Main.CarolynMcCoey" version="1"
Added:
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1421844556" name="ESWCorrection.pdf" path="ESWCorrection.pdf" size="655707" user="Main.DavidTeyssier" version="1"
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1313009238" name="calibration_hifi_stdw_2011_v0_1.pdf" path="calibration_hifi_stdw_2011_v0_1.pdf" size="4118561" user="Main.CarolynMcCoey" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1322760092" name="aa15120-10.pdf" path="aa15120-10.pdf" size="386925" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521098" name="calibframework.pdf" path="calibframework.pdf" size="310766" user="Main.DavidTeyssier" version="1"

Revision 812014-12-01 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 330 to 330
 
Changed:
<
<		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 12 Nov 2014

>
>		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 1 Dec 2014
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521151" name="freq_vel_3.PDF" path="freq_vel_3.PDF" size="312508" user="Main.DavidTeyssier" version="1"
Line: 351 to 348
 
META FILEATTACHMENT attr="" autoattached="1" comment="CASSIS lineList for the idenfifyLines task (HDRG, chapter 'The HIFI line identification tool')" date="1412866259" name="16293_SIS.txt" path="16293_SIS.txt" size="175836" user="Main.SylvieFBeaulieu" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1392023495" name="spatial_response_framework_v1.9.pdf" path="spatial_response_framework_v1.9.pdf" size="603181" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309104638" name="thesis_ronan_higgins_nuim.pdf" path="thesis_ronan_higgins_nuim.pdf" size="19376465" user="Main.DavidTeyssier" version="1"
Changed:
<
<
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1415785392" name="getHifiBeam.py.txt" path="getHifiBeam.py.txt" size="4740" user="Main.DavidTeyssier" version="1"
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1417446382" name="getHifiBeam.py.txt" path="getHifiBeam.py.txt" size="5224" user="Main.DavidTeyssier" version="2"
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1359637018" name="HIFI_Calibration_ReleaseNotes.pdf" path="HIFI_Calibration_ReleaseNotes.pdf" size="178716" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1406141554" name="HIFISubbands.pdf" path="HIFISubbands.pdf" size="8715" user="Main.CarolynMcCoey" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1313009238" name="calibration_hifi_stdw_2011_v0_1.pdf" path="calibration_hifi_stdw_2011_v0_1.pdf" size="4118561" user="Main.CarolynMcCoey" version="1"

Revision 802014-11-12 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 330 to 330
 
Changed:
<
<		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 7 July 2014
>
>		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 12 Nov 2014

 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521151" name="freq_vel_3.PDF" path="freq_vel_3.PDF" size="312508" user="Main.DavidTeyssier" version="1"
Line: 338 to 341
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521125" name="spatial_response_framework.pdf" path="spatial_response_framework.pdf" size="595755" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1357815487" name="calibration_hifi_stdw_2011_v0_1.PDF" path="calibration_hifi_stdw_2011_v0_1.PDF" size="2966633" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1317720447" name="Ossenkopf_2008_AllanVariance.pdf" path="Ossenkopf_2008_AllanVariance.pdf" size="1536309" user="Main.DavidTeyssier" version="1"
Added:
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="Script for the idenfifyLines task (HDRG, chapter 'The HIFI line identification tool')" date="1412866352" name="line_identification_guide.py.txt" path="line_identification_guide.py.txt" size="12299" user="Main.SylvieFBeaulieu" version="1"
 
META FILEATTACHMENT attr="" autoattached="1" comment="de Graauw et al 2010 -- HIFI instrument paper" date="1295602694" name="aa14698-10.pdf" path="aa14698-10.pdf" size="785388" user="Main.AnthonyMarston" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1395147451" name="hebCorrectionModels_HICAL15.fits" path="hebCorrectionModels_HICAL15.fits" size="247680" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1317389173" name="HifiObservingModesPerformance_110926a.pdf" path="HifiObservingModesPerformance_110926a.pdf" size="4428174" user="Main.JeanMatagne" version="1"
Line: 347 to 351
 
META FILEATTACHMENT attr="" autoattached="1" comment="CASSIS lineList for the idenfifyLines task (HDRG, chapter 'The HIFI line identification tool')" date="1412866259" name="16293_SIS.txt" path="16293_SIS.txt" size="175836" user="Main.SylvieFBeaulieu" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1392023495" name="spatial_response_framework_v1.9.pdf" path="spatial_response_framework_v1.9.pdf" size="603181" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309104638" name="thesis_ronan_higgins_nuim.pdf" path="thesis_ronan_higgins_nuim.pdf" size="19376465" user="Main.DavidTeyssier" version="1"
Changed:
<
<
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1411119005" name="getHifiBeam.py.txt" path="getHifiBeam.py.txt" size="4697" user="Main.DavidTeyssier" version="1"
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1415785392" name="getHifiBeam.py.txt" path="getHifiBeam.py.txt" size="4740" user="Main.DavidTeyssier" version="1"
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1359637018" name="HIFI_Calibration_ReleaseNotes.pdf" path="HIFI_Calibration_ReleaseNotes.pdf" size="178716" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1406141554" name="HIFISubbands.pdf" path="HIFISubbands.pdf" size="8715" user="Main.CarolynMcCoey" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1313009238" name="calibration_hifi_stdw_2011_v0_1.pdf" path="calibration_hifi_stdw_2011_v0_1.pdf" size="4118561" user="Main.CarolynMcCoey" version="1"
Line: 359 to 363
 
META FILEATTACHMENT attr="" autoattached="1" comment="HIFI_SBR_Cookbook_V1.0" date="1372338964" name="HIFI_SBR_Cookbook_V1.0.pdf" path="HIFI_SBR_Cookbook_V1.0.pdf" size="1474786" user="Main.SylvieFBeaulieu" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309110936" name="freq_vel_1.1.pdf" path="freq_vel_1.1.pdf" size="261519" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1323853023" name="Alternative_Calibration_Schemes_v1.0.pdf" path="Alternative_Calibration_Schemes_v1.0.pdf" size="2883364" user="Main.DavidTeyssier" version="1"
Deleted:
<
<
META FILEATTACHMENT attachment="line_identification_guide.py.txt" attr="" comment="Script for the idenfifyLines task (HDRG, chapter 'The HIFI line identification tool')" date="1412866352" name="line_identification_guide.py.txt" path="line_identification_guide.py" size="12299" stream="line_identification_guide.py" user="Main.SylvieFBeaulieu" version="1"

Revision 792014-10-09 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 334 to 334
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521151" name="freq_vel_3.PDF" path="freq_vel_3.PDF" size="312508" user="Main.DavidTeyssier" version="1"
Added:
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="Spectrum1d (fits) for the idenfifyLines task (HDRG, chapter 'The HIFI line identification tool')" date="1412866311" name="Orion_S_baselined_485_490.fits" path="Orion_S_baselined_485_490.fits" size="172800" user="Main.SylvieFBeaulieu" version="1"
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521125" name="spatial_response_framework.pdf" path="spatial_response_framework.pdf" size="595755" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1357815487" name="calibration_hifi_stdw_2011_v0_1.PDF" path="calibration_hifi_stdw_2011_v0_1.PDF" size="2966633" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1317720447" name="Ossenkopf_2008_AllanVariance.pdf" path="Ossenkopf_2008_AllanVariance.pdf" size="1536309" user="Main.DavidTeyssier" version="1"
Line: 343 to 344
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1395147438" name="hebCorrection_externalModels_example.py.txt" path="hebCorrection_externalModels_example.py.txt" size="2508" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309104004" name="Sherry_et_al_AAS217_poster.pdf" path="Sherry_et_al_AAS217_poster.pdf" size="104086" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521138" name="freq_framework.pdf" path="freq_framework.pdf" size="185626" user="Main.DavidTeyssier" version="1"
Added:
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="CASSIS lineList for the idenfifyLines task (HDRG, chapter 'The HIFI line identification tool')" date="1412866259" name="16293_SIS.txt" path="16293_SIS.txt" size="175836" user="Main.SylvieFBeaulieu" version="1"
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1392023495" name="spatial_response_framework_v1.9.pdf" path="spatial_response_framework_v1.9.pdf" size="603181" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309104638" name="thesis_ronan_higgins_nuim.pdf" path="thesis_ronan_higgins_nuim.pdf" size="19376465" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1411119005" name="getHifiBeam.py.txt" path="getHifiBeam.py.txt" size="4697" user="Main.DavidTeyssier" version="1"
Line: 357 to 359
 
META FILEATTACHMENT attr="" autoattached="1" comment="HIFI_SBR_Cookbook_V1.0" date="1372338964" name="HIFI_SBR_Cookbook_V1.0.pdf" path="HIFI_SBR_Cookbook_V1.0.pdf" size="1474786" user="Main.SylvieFBeaulieu" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309110936" name="freq_vel_1.1.pdf" path="freq_vel_1.1.pdf" size="261519" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1323853023" name="Alternative_Calibration_Schemes_v1.0.pdf" path="Alternative_Calibration_Schemes_v1.0.pdf" size="2883364" user="Main.DavidTeyssier" version="1"
Added:
>
>
META FILEATTACHMENT attachment="line_identification_guide.py.txt" attr="" comment="Script for the idenfifyLines task (HDRG, chapter 'The HIFI line identification tool')" date="1412866352" name="line_identification_guide.py.txt" path="line_identification_guide.py" size="12299" stream="line_identification_guide.py" user="Main.SylvieFBeaulieu" version="1"

Revision 782014-10-01 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 205 to 205
 
    • Side band ratio technical note: This note describes the general calibration problem associated with the side-band gain imbalance existing in double-side-band heterodyne systems such as HIFI, and provides some recipes and calibration tables applicable to the particular case of the HIFI mixers.
  • Beam calibration
Changed:
<
<
>
>
 
Line: 352 to 352
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1322760092" name="aa15120-10.pdf" path="aa15120-10.pdf" size="386925" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521098" name="calibframework.pdf" path="calibframework.pdf" size="310766" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1395147422" name="hebCorrection_calTree_example.py.txt" path="hebCorrection_calTree_example.py.txt" size="2449" user="Main.DavidTeyssier" version="1"
Changed:
<
<
META FILEATTACHMENT attachment="HifiBeamReleaseNote_Sep2014.pdf" attr="" comment="" date="1412165437" name="HifiBeamReleaseNote_Sep2014.pdf" path="HifiBeamReleaseNote_Sep2014.pdf" size="1549504" stream="HifiBeamReleaseNote_Sep2014.pdf" user="Main.DavidTeyssier" version="2"
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1412165437" name="HifiBeamReleaseNote_Sep2014.pdf" path="HifiBeamReleaseNote_Sep2014.pdf" size="1549504" user="Main.DavidTeyssier" version="2"
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1345720705" name="HIFI_SBR_Summary_V9.0.png" path="HIFI_SBR_Summary_V9.0.png" size="88066" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="HIFI_SBR_Cookbook_V1.0" date="1372338964" name="HIFI_SBR_Cookbook_V1.0.pdf" path="HIFI_SBR_Cookbook_V1.0.pdf" size="1474786" user="Main.SylvieFBeaulieu" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309110936" name="freq_vel_1.1.pdf" path="freq_vel_1.1.pdf" size="261519" user="Main.DavidTeyssier" version="1"

Revision 772014-10-01 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 352 to 352
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1322760092" name="aa15120-10.pdf" path="aa15120-10.pdf" size="386925" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521098" name="calibframework.pdf" path="calibframework.pdf" size="310766" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1395147422" name="hebCorrection_calTree_example.py.txt" path="hebCorrection_calTree_example.py.txt" size="2449" user="Main.DavidTeyssier" version="1"
Changed:
<
<
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1411119017" name="HifiBeamReleaseNote_Sep2014.pdf" path="HifiBeamReleaseNote_Sep2014.pdf" size="1541231" user="Main.DavidTeyssier" version="1"
>
>
META FILEATTACHMENT attachment="HifiBeamReleaseNote_Sep2014.pdf" attr="" comment="" date="1412165437" name="HifiBeamReleaseNote_Sep2014.pdf" path="HifiBeamReleaseNote_Sep2014.pdf" size="1549504" stream="HifiBeamReleaseNote_Sep2014.pdf" user="Main.DavidTeyssier" version="2"
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1345720705" name="HIFI_SBR_Summary_V9.0.png" path="HIFI_SBR_Summary_V9.0.png" size="88066" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="HIFI_SBR_Cookbook_V1.0" date="1372338964" name="HIFI_SBR_Cookbook_V1.0.pdf" path="HIFI_SBR_Cookbook_V1.0.pdf" size="1474786" user="Main.SylvieFBeaulieu" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309110936" name="freq_vel_1.1.pdf" path="freq_vel_1.1.pdf" size="261519" user="Main.DavidTeyssier" version="1"

Revision 762014-09-25 - PedroGarciaLario

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 251 to 251
  HIFI beam maps and Encircled Energy Fraction
Changed:
<
<		In addition to this, a detailed description of the HIFI beam models is provided as 2-D beam maps, 1-D azimuthal beam averages, and Encircled Energy Fractions (EEF). Each are provided at two frequencies per mixer band and polarisation, and can be extrapolated (within the LO frequencies of a given band) as function of wavelength. The release notes provides further details about the delivered files and how to use them. In particular, some useful functions are provided in this Jython script.
>
>		In addition to this, a detailed description of the HIFI beam models is provided as 2-D beam maps, 1-D azimuthal beam averages, and Encircled Energy Fractions (EEF). Each are provided at two frequencies per mixer band and polarisation, and can be extrapolated (within the LO frequencies of a given band) as function of wavelength. The release note provides further details about the delivered files and how to use them. In particular, some useful functions are provided in this Jython script.
  The beam material is concatenated as single FITS file per mixer, frequency and polarisation. Each file has three extensions table: one for the 2-D beam model maps (2705x2705 pixels), one for the EEF, and one for the 1-D azimuthal beam averages. Note that 2-D beam model of smaller size (201x201 pixels) will also be made available in the calibration tree (not before HIFI 13), and used in the generation of level 2.5 cubes by the pipeline. The following page provides links to each individual file (27 Mb each). Plots of the 1-D azimuthal beam averages and EEF for each band can be found in the Appendix B of the release note.

Revision 752014-09-19 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 205 to 205
 
    • Side band ratio technical note: This note describes the general calibration problem associated with the side-band gain imbalance existing in double-side-band heterodyne systems such as HIFI, and provides some recipes and calibration tables applicable to the particular case of the HIFI mixers.
  • Beam calibration
Changed:
<
<
>
>
 
Line: 285 to 285
 
  • The side-band ratio assumed in the pipeline as of the latest calibration tree pool are illustrated in this plot.
Changed:
<
<
  • Beam efficiencies are derived from Mars maps. Deeper beam maps are currently being taken and analysed, and will result in a possible update of the coupling efficiencies. Those will be posted together with 2D beam map for each of the HIFI bands, which could be of particular interest for the interpretation of extended emission
>
>
  • The HIFI beam release #1 (see section above) including updated aperture efficiencies, main-beam efficiencies, and HPBW is not yet reflected in the HIFI caltree.
 

Outstanding calibration issues

Revision 742014-09-19 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Changed:
<
<		NEW NEW NEW Latest updates -- 2 July 2014 NEW NEW NEW
  • HIPE 12.1 has been released !
  • A new task for Electrical Standing Wave correction is available from HIPE 12.0 - see the "Outstanding calibration issues" section
>
>		NEW NEW NEW Latest updates -- 19 September 2014 NEW NEW NEW
  • New beam and coupling efficiency release - see the "HIFI performance and calibration" section
 

Observing with HIFI

  • Useful Documentation
Line: 244 to 243
 
  • WBS accuracy: COMB algorithm and interpolation interval
  • WBS resolution: optical alignment & thermal expansion effects within spectrometer
Added:
>
>

HIFI beams

 
Added:
>
>		HIFI beam coupling efficiencies

A revision of the HIFI beam analysis has been performed, based on a multi-epoch set of measurements on Mars. As a result, a more accurate description of the HIFI beams has been obtained, leading to some change in the beam coupling efficiencies, typically a drop. The details of the analysis and the revised coupling efficiencies are given in this release note. For the record, the first release of the HIFI beam efficiencies was described in this document (17 Nov 2010). Please note that these updates will only be featured in the HIFI calibration tree in HIPE 13.

HIFI beam maps and Encircled Energy Fraction

In addition to this, a detailed description of the HIFI beam models is provided as 2-D beam maps, 1-D azimuthal beam averages, and Encircled Energy Fractions (EEF). Each are provided at two frequencies per mixer band and polarisation, and can be extrapolated (within the LO frequencies of a given band) as function of wavelength. The release notes provides further details about the delivered files and how to use them. In particular, some useful functions are provided in this Jython script.

The beam material is concatenated as single FITS file per mixer, frequency and polarisation. Each file has three extensions table: one for the 2-D beam model maps (2705x2705 pixels), one for the EEF, and one for the 1-D azimuthal beam averages. Note that 2-D beam model of smaller size (201x201 pixels) will also be made available in the calibration tree (not before HIFI 13), and used in the generation of level 2.5 cubes by the pipeline. The following page provides links to each individual file (27 Mb each). Plots of the 1-D azimuthal beam averages and EEF for each band can be found in the Appendix B of the release note.

 

Access to the calibration tree:

  • Information about what calibration is found in the HIFI ObservationContext is found in the Data Primer of the HIFI Data Reduction Guide
Line: 336 to 345
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521138" name="freq_framework.pdf" path="freq_framework.pdf" size="185626" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1392023495" name="spatial_response_framework_v1.9.pdf" path="spatial_response_framework_v1.9.pdf" size="603181" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309104638" name="thesis_ronan_higgins_nuim.pdf" path="thesis_ronan_higgins_nuim.pdf" size="19376465" user="Main.DavidTeyssier" version="1"
Added:
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1411119005" name="getHifiBeam.py.txt" path="getHifiBeam.py.txt" size="4697" user="Main.DavidTeyssier" version="1"
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1359637018" name="HIFI_Calibration_ReleaseNotes.pdf" path="HIFI_Calibration_ReleaseNotes.pdf" size="178716" user="Main.DavidTeyssier" version="1"
Added:
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1406141554" name="HIFISubbands.pdf" path="HIFISubbands.pdf" size="8715" user="Main.CarolynMcCoey" version="1"
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1313009238" name="calibration_hifi_stdw_2011_v0_1.pdf" path="calibration_hifi_stdw_2011_v0_1.pdf" size="4118561" user="Main.CarolynMcCoey" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1322760092" name="aa15120-10.pdf" path="aa15120-10.pdf" size="386925" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521098" name="calibframework.pdf" path="calibframework.pdf" size="310766" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1395147422" name="hebCorrection_calTree_example.py.txt" path="hebCorrection_calTree_example.py.txt" size="2449" user="Main.DavidTeyssier" version="1"
Added:
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1411119017" name="HifiBeamReleaseNote_Sep2014.pdf" path="HifiBeamReleaseNote_Sep2014.pdf" size="1541231" user="Main.DavidTeyssier" version="1"
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1345720705" name="HIFI_SBR_Summary_V9.0.png" path="HIFI_SBR_Summary_V9.0.png" size="88066" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="HIFI_SBR_Cookbook_V1.0" date="1372338964" name="HIFI_SBR_Cookbook_V1.0.pdf" path="HIFI_SBR_Cookbook_V1.0.pdf" size="1474786" user="Main.SylvieFBeaulieu" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309110936" name="freq_vel_1.1.pdf" path="freq_vel_1.1.pdf" size="261519" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1323853023" name="Alternative_Calibration_Schemes_v1.0.pdf" path="Alternative_Calibration_Schemes_v1.0.pdf" size="2883364" user="Main.DavidTeyssier" version="1"
Deleted:
<
<
META FILEATTACHMENT attachment="HIFISubbands.pdf" attr="" comment="" date="1406141554" name="HIFISubbands.pdf" path="HIFISubbands.pdf" size="8715" stream="HIFISubbands.pdf" user="Main.CarolynMcCoey" version="1"

Revision 732014-07-23 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 345 to 345
 
META FILEATTACHMENT attr="" autoattached="1" comment="HIFI_SBR_Cookbook_V1.0" date="1372338964" name="HIFI_SBR_Cookbook_V1.0.pdf" path="HIFI_SBR_Cookbook_V1.0.pdf" size="1474786" user="Main.SylvieFBeaulieu" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309110936" name="freq_vel_1.1.pdf" path="freq_vel_1.1.pdf" size="261519" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1323853023" name="Alternative_Calibration_Schemes_v1.0.pdf" path="Alternative_Calibration_Schemes_v1.0.pdf" size="2883364" user="Main.DavidTeyssier" version="1"
Added:
>
>
META FILEATTACHMENT attachment="HIFISubbands.pdf" attr="" comment="" date="1406141554" name="HIFISubbands.pdf" path="HIFISubbands.pdf" size="8715" stream="HIFISubbands.pdf" user="Main.CarolynMcCoey" version="1"

Revision 722014-07-07 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 272 to 272
 
13_0 30-05-13   11.0 Update of the list of corrupted data-frames for flagging at Level 0, updates to spur table to account for band 3B purification
14_0 04-07-13   11.0 Update of the list of corrupted data-frames for flagging at Level 0
15_0 18-03-14   12.0 Update of the list of corrupted data-frames for flagging at Level 0, ingestion of the Electrical Standing Wave correction reference table
Changed:
<
<
18_0 02-07-14   12.1 Update of the list of corrupted data-frames for flagging at Level 0, Absolute Pointing Error (APE) added to allow a quality flag to be raised if the pointing outside of expected range during ODs 320-761, updates to allow IF-dependent corrections to side band ratio, update to spur tables allow entry of spurs found by the ICC or a Key Programme, addition of instrument uncertainty
>
>
18_0 02-07-14   12.1 Update of the list of corrupted data-frames for flagging at Level 0, Absolute Pointing Error (APE) added to allow a quality flag to be raised in future releases in case of large pointing offsets, updates to allow IF-dependent corrections to side band ratio, update to spur tables to store entry of spurs found by the ICC or a Key Programme, addition of an instrument uncertainty table (currently only placeholder)
 
  • The side-band ratio assumed in the pipeline as of the latest calibration tree pool are illustrated in this plot.
Line: 321 to 321
 
Changed:
<
<		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 2 July 2014
>
>		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 7 July 2014
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521151" name="freq_vel_3.PDF" path="freq_vel_3.PDF" size="312508" user="Main.DavidTeyssier" version="1"

Revision 712014-07-03 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 207 to 207
 
Deleted:
<
<
 

Revision 702014-07-03 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 6 to 6
  NEW NEW NEW Latest updates -- 2 July 2014 NEW NEW NEW
  • HIPE 12.1 has been released !
Deleted:
<
<
  • A revision of the HIFI beam analysis and efficiencies is available - see the "HIFI performance and calibration" section
 
  • A new task for Electrical Standing Wave correction is available from HIPE 12.0 - see the "Outstanding calibration issues" section

Observing with HIFI

Line: 205 to 204
 
  • Intensity calibration: Intensity Calibration Framework document
  • Side band ratio
    • Side band ratio technical note: This note describes the general calibration problem associated with the side-band gain imbalance existing in double-side-band heterodyne systems such as HIFI, and provides some recipes and calibration tables applicable to the particular case of the HIFI mixers.
Changed:
<
<
  • Beam calibration ( see also HIFI beam sub-section thereafter )
>
>
  • Beam calibration
 
Added:
>
>
 
Line: 245 to 245
 
  • WBS accuracy: COMB algorithm and interpolation interval
  • WBS resolution: optical alignment & thermal expansion effects within spectrometer
Deleted:
<
<

HIFI beams

 
Deleted:
<
<		HIFI beam coupling efficiencies

A revision of the HIFI beam analysis has been performed, based on a multi-epoch set of measurements on Mars. As a result, a more accurate description of the HIFI beams has been obtained, leading to some change in the beam coupling efficiencies. The details of the analysis and the revised coupling efficiencies are given in this note. For the record, the first release of the HIFI beam efficiencies was described in this document (17 Nov 2010). Please note that these updates will only be featured in the HIFI calibration tree in HIPE 13.0.

HIFI beam maps and Encircled Energy Function

In addition to this, a detailed description of the HIFI beam models is provided as 2-D beam maps, 1-D azimuthal beam averages, and 1-D azimuthally averaged Encircled Energy Functions (EEF). Each are provided at one frequency per mixer band and can be extrapolated (within the LO frequencies of a given band) as function of wavelength.

  • EEF plots as ascii and PNG files (one per mixer band and polarization - 14 in total).
  • 1-D beam model profiles as FITS files (one per mixer band and polarization - 14 in total).
  • 2-D beam model maps as FITS files (200x200 pixels, one per mixer band and polarization - 14 in total).
 

Access to the calibration tree:

  • Information about what calibration is found in the HIFI ObservationContext is found in the Data Primer of the HIFI Data Reduction Guide

Revision 692014-07-02 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Changed:
<
<		NEW NEW NEW Latest updates -- 18 March 2014 NEW NEW NEW
  • HIPE 12.0 has been released !
  • A new task for Electrical Standing Wave correction is available in HIPE 12.0 - see the "Outstanding calibration issues" section
  • Cookbooks are now available for each HIFI observing mode - see the "Documentation and Cookbooks" section
>
>		NEW NEW NEW Latest updates -- 2 July 2014 NEW NEW NEW
  • HIPE 12.1 has been released !
  • A revision of the HIFI beam analysis and efficiencies is available - see the "HIFI performance and calibration" section
  • A new task for Electrical Standing Wave correction is available from HIPE 12.0 - see the "Outstanding calibration issues" section
 

Observing with HIFI

  • Useful Documentation
Line: 15 to 15
 
Added:
>
>
 
  • HIFI calibration data
    • A significant number of the data taken by HIFI in the framework of the Performance Validation (PV) and routine phase are now publicly available. There are particular caveats that apply to those data. They are described in this disclaimer note
Line: 27 to 28
 

Reducing HIFI data

Recommended User release

Changed:
<
<
>
>
  Mac OS X and Java 7 Issues : please consult the HIPE Known Issues page regarding important information on issues with Java for different versions of HIPE.
Changed:
<
<

Which data would most benefit from being reprocessed in HIPE 12.0?

>
>

Which data would most benefit from being reprocessed in HIPE 12.1?

  If your data have been processed with the latest bulk reprocessing (11.1):

  • Rotated maps
Changed:
<
<
    • Rotated cubes are now produced in the Level 2.5 product by default for maps carried out with a non-zero position angle, a non-rotated cube is also produced. BEWARE however that a bug was introduced in HIPE 12.0 whereby non-rotated cubes are not properly created if option flyAngle=0 is selected. You can temporarily work this around by specifying e.g. flyAngle=1e-5 (i.e. a very small, non-zero, rotation). This will be fixed in HIPE 12.1.
>
>
    • Rotated cubes are now produced in the Level 2.5 product by default for maps carried out with a non-zero position angle, a non-rotated cube is also produced.
 
  • Solar System Object ephemerides
    • A bug (see here) was fixed in the calculation of SSO ephemeris positions (ra_centre/dec_centre) that led to offsets of up to ~10". This bug affected users that use cubes in a co-moving frame (including the standard level 2.5 cubes), use the doOffset task, or make explicit use of ra_centre / dec_centre in their HIPE scripts. The offset is practically constant over a map and could, e.g., make the emission of a comet appear to be off-center when it really isn't. The fixed bug is in the level 0 pipeline, which users cannot run easily. The issue is fixed by the HIPE 11.1 bulk reprocessing.
Line: 82 to 83
  -->
Changed:
<
<
  • See the What's New page for more details of improvements in HIPE 12.0
>
>
  • See the What's New page for more details of improvements in HIPE 12.1
 

Changed:
<
<
>
>
 

Documentation and Cookbooks

  • The HIFI Launch Pad is intended to help you quickly off the ground with HIFI data reduction
Line: 204 to 205
 
  • Intensity calibration: Intensity Calibration Framework document
  • Side band ratio
    • Side band ratio technical note: This note describes the general calibration problem associated with the side-band gain imbalance existing in double-side-band heterodyne systems such as HIFI, and provides some recipes and calibration tables applicable to the particular case of the HIFI mixers.
Changed:
<
<
  • Beam calibration
>
>
  • Beam calibration ( see also HIFI beam sub-section thereafter )
 
Changed:
<
<
>
>
 
Deleted:
<
<
 
Line: 245 to 245
 
  • WBS accuracy: COMB algorithm and interpolation interval
  • WBS resolution: optical alignment & thermal expansion effects within spectrometer
Added:
>
>

HIFI beams

HIFI beam coupling efficiencies

A revision of the HIFI beam analysis has been performed, based on a multi-epoch set of measurements on Mars. As a result, a more accurate description of the HIFI beams has been obtained, leading to some change in the beam coupling efficiencies. The details of the analysis and the revised coupling efficiencies are given in this note. For the record, the first release of the HIFI beam efficiencies was described in this document (17 Nov 2010). Please note that these updates will only be featured in the HIFI calibration tree in HIPE 13.0.

HIFI beam maps and Encircled Energy Function

In addition to this, a detailed description of the HIFI beam models is provided as 2-D beam maps, 1-D azimuthal beam averages, and 1-D azimuthally averaged Encircled Energy Functions (EEF). Each are provided at one frequency per mixer band and can be extrapolated (within the LO frequencies of a given band) as function of wavelength.

  • EEF plots as ascii and PNG files (one per mixer band and polarization - 14 in total).
  • 1-D beam model profiles as FITS files (one per mixer band and polarization - 14 in total).
  • 2-D beam model maps as FITS files (200x200 pixels, one per mixer band and polarization - 14 in total).
 

Access to the calibration tree:

  • Information about what calibration is found in the HIFI ObservationContext is found in the Data Primer of the HIFI Data Reduction Guide

  • Information about how to find what calibration version was used on your data, how to get the latest calibration and reprocess observations with it is found in the Pipeline chapter of the HIFI Data reduction Guide
Changed:
<
<
  • the latest calibration tree pool can be also retrieved directly here (18 March 2014)
>
>
  • the latest calibration tree pool can be also retrieved directly here (2 July 2014)
 
  • Updates to the HIFI calibration data are generally concurrent with the release of each major version of the HCSS-HIFI software. However, it is possible to have updates to the calibration data in between major releases of the software as the software and data are independent of each other, in the table below the HIPE or the OD from which the calibration updates apply are given. The calibration versions available since HIPE 5 are listed in the table below
Line: 272 to 285
 
13_0 30-05-13   11.0 Update of the list of corrupted data-frames for flagging at Level 0, updates to spur table to account for band 3B purification
14_0 04-07-13   11.0 Update of the list of corrupted data-frames for flagging at Level 0
15_0 18-03-14   12.0 Update of the list of corrupted data-frames for flagging at Level 0, ingestion of the Electrical Standing Wave correction reference table
Added:
>
>
18_0 02-07-14   12.1 Update of the list of corrupted data-frames for flagging at Level 0, Absolute Pointing Error (APE) added to allow a quality flag to be raised if the pointing outside of expected range during ODs 320-761, updates to allow IF-dependent corrections to side band ratio, update to spur tables allow entry of spurs found by the ICC or a Key Programme, addition of instrument uncertainty
 
  • The side-band ratio assumed in the pipeline as of the latest calibration tree pool are illustrated in this plot.
Line: 320 to 334
 
Changed:
<
<		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 26 March 2014
>
>		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 2 July 2014
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521151" name="freq_vel_3.PDF" path="freq_vel_3.PDF" size="312508" user="Main.DavidTeyssier" version="1"

Revision 682014-06-16 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 110 to 110
 

  • The full set of Herschel documentation for the developer track is also available. This documentation contains revisions and updates to the documentation associated with the current User Release but you should also be aware that it may describe functionality not available in the current User Release
Changed:
<
<
>
>
 

Revision 672014-03-26 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 36 to 36
  If your data have been processed with the latest bulk reprocessing (11.1):

  • Rotated maps
Changed:
<
<
    • Rotated cubes are now produced in the Level 2.5 product by default for maps carried out with a non-zero position angle, a non-rotated cube is also produced.
>
>
    • Rotated cubes are now produced in the Level 2.5 product by default for maps carried out with a non-zero position angle, a non-rotated cube is also produced. BEWARE however that a bug was introduced in HIPE 12.0 whereby non-rotated cubes are not properly created if option flyAngle=0 is selected. You can temporarily work this around by specifying e.g. flyAngle=1e-5 (i.e. a very small, non-zero, rotation). This will be fixed in HIPE 12.1.
 
  • Solar System Object ephemerides
    • A bug (see here) was fixed in the calculation of SSO ephemeris positions (ra_centre/dec_centre) that led to offsets of up to ~10". This bug affected users that use cubes in a co-moving frame (including the standard level 2.5 cubes), use the doOffset task, or make explicit use of ra_centre / dec_centre in their HIPE scripts. The offset is practically constant over a map and could, e.g., make the emission of a comet appear to be off-center when it really isn't. The fixed bug is in the level 0 pipeline, which users cannot run easily. The issue is fixed by the HIPE 11.1 bulk reprocessing.
Line: 320 to 320
 
Changed:
<
<		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 18 March 2014
>
>		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 26 March 2014
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521151" name="freq_vel_3.PDF" path="freq_vel_3.PDF" size="312508" user="Main.DavidTeyssier" version="1"

Revision 662014-03-20 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 33 to 33
 

Which data would most benefit from being reprocessed in HIPE 12.0?

Changed:
<
<		 If your data have been processed with the latest bulk reprocessing (10.3):
>
>		 If your data have been processed with the latest bulk reprocessing (11.1):
 
  • Rotated maps
    • Rotated cubes are now produced in the Level 2.5 product by default for maps carried out with a non-zero position angle, a non-rotated cube is also produced.

Revision 652014-03-18 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 12 to 12
 

Observing with HIFI

  • Useful Documentation
Changed:
<
<
>
>
 
Line: 39 to 39
 
    • Rotated cubes are now produced in the Level 2.5 product by default for maps carried out with a non-zero position angle, a non-rotated cube is also produced.

  • Solar System Object ephemerides
Changed:
<
<
    • A bug (see here) was fixed in the calculation of SSO ephemeris positions (ra_centre/dec_centre) that led to offsets of up to ~10". This bug affected users that use cubes in a co-moving frame (including the standard level 2.5 cubes), use the doOffset task, or make explicit use of ra_centre / dec_centre in their HIPE scripts. The offset is practically constant over a map and could, e.g., make the emission of a comet appear to be off-center when it really isn't. The fixed bug is in the level 0 pipeline, which users cannot run easily. The issue will be fixed by the HIPE 11 bulk reprocessing. In the meantime, you can use a HIPE snippet such as the one attached here.
>
>
    • A bug (see here) was fixed in the calculation of SSO ephemeris positions (ra_centre/dec_centre) that led to offsets of up to ~10". This bug affected users that use cubes in a co-moving frame (including the standard level 2.5 cubes), use the doOffset task, or make explicit use of ra_centre / dec_centre in their HIPE scripts. The offset is practically constant over a map and could, e.g., make the emission of a comet appear to be off-center when it really isn't. The fixed bug is in the level 0 pipeline, which users cannot run easily. The issue is fixed by the HIPE 11.1 bulk reprocessing.
 
  • Improved pointing reconstruction for observations taken before OD 320
Line: 49 to 49
 
    • A new pipeline step, mkDbsReference, calculates the differences the chop positions in all DBS observations and after applying the band-pass correction stores them in a product in calibration->pipeline-out called ReferenceSpectra. This allows you to check for contamination in chop positions for all types of DBS observations.

  • Improved pointing reconstruction for observations taken between OD 320 and 761 If you have data processed with < HIPE 9
Changed:
<
<
    • The pointing information attached to the data for observations taken between OD 320 and OD761 did not use the most accurate representation of the star tracker focal length. This will be done for the bulk reprocessing with HIPE 9. As a consequence, some observations will experience a shift in astrometry, that can be as high as 8 arcsec. All the details about the consequences for a particular obsid, and recipes to reconstruct yourself the improved pointing can be found at http://herschel.esac.esa.int/twiki/bin/view/Public/HowToUseImprovedPointingProducts as well as in the level0 section of the Hifi Data Reduction Guide
>
>
    • The pointing information attached to the data for observations taken between OD 320 and OD761 did not use the most accurate representation of the star tracker focal length. This will be done for the bulk reprocessing with HIPE 9. As a consequence, some observations will experience a shift in astrometry, that can be as high as 8 arcsec. All the details about the consequences for a particular obsid, and recipes to reconstruct yourself the improved pointing can be found at http://herschel.esac.esa.int/twiki/bin/view/Public/HowToUseImprovedPointingProducts as well as in the level0 section of the Hifi Data Reduction Guide
 
  • Backfilling of observational parameters If you have data processed with < HIPE 9
    • From 9.1 onwards, most of the observational parameters that got optimised in HSpot will be propagated into the observation context, under a new product called HifiUplinkProduct (in the auxiliary product branch). Some of these parameters will be used in the pipeline to e.g. estimate the dimension of the cubes in the mapping and have them more representative of how the map was really obtained.

  • Solar System Object maps If you have data processed with < HIPE 9
Changed:
<
<
    • Cubes for moving targets are now created in the comoving frames in HIPE 9. In order to benefit from that, you should re-pipeline from level 0 up to level 2.5. Note the hifiPipeline task should be called with the option "Aux=True". Please check the Pipeline chapter of the HIFI Data Reduction Guide for more details about how to do that.
>
>
    • Cubes for moving targets are now created in the comoving frames in HIPE 9. In order to benefit from that, you should re-pipeline from level 0 up to level 2.5. Note the hifiPipeline task should be called with the option "Aux=True". Please check the Pipeline chapter of the HIFI Data Reduction Guide for more details about how to do that.
 

Revision 642014-03-18 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Changed:
<
<		NEW NEW NEW Latest updates -- 19 November 2013 NEW NEW NEW
  • HIPE 11.1 has been released !
  • HIPI 2.1 (a library of Hifi Plug-In's) has been released - see Interest Groups and Scripts section
  • New matching technique script is available in the Expert Version of HIPE - see the "Outstanding calibration issues" section
  • A large amount of calibration data has been released ! - see the "Observing with HIFI" section

>
>		NEW NEW NEW Latest updates -- 18 March 2014 NEW NEW NEW
  • HIPE 12.0 has been released !
  • A new task for Electrical Standing Wave correction is available in HIPE 12.0 - see the "Outstanding calibration issues" section
  • Cookbooks are now available for each HIFI observing mode - see the "Documentation and Cookbooks" section
 

Observing with HIFI

  • Useful Documentation
Changed:
<
<
>
>
 
Line: 23 to 21
 
Changed:
<
<
>
>
    • A&A paper describing the method used to measure stability performance with HIFI (added 4 Oct 2011)
    • A&A paper about the optimization of mapping modes for heterodyne instruments (added 18 March 2014)]
 

Reducing HIFI data

Recommended User release

Changed:
<
<
>
>

Mac OS X and Java 7 Issues : please consult the HIPE Known Issues page regarding important information on issues with Java for different versions of HIPE.

 
Changed:
<
<

Which data would most benefit from being reprocessed in HIPE 11.1?

>
>

Which data would most benefit from being reprocessed in HIPE 12.0?

  If your data have been processed with the latest bulk reprocessing (10.3):
Line: 81 to 82
  -->
Changed:
<
<
>
>
 
Changed:
<
<
>
>
 

Documentation and Cookbooks

  • The HIFI Launch Pad is intended to help you quickly off the ground with HIFI data reduction
Changed:
<
<
>
>

 
Deleted:
<
<
 
  • The HIFI Data Reduction Guide and the HIFI AOT Observing Mode Release and Performance Notes contain information you need to know about HIFI data, regardless of the software you use for data reduction
Changed:
<
<
>
>
 

  • If you want to know more about the details of the HIFI pipeline, you should look at the HIFI Pipeline Specification Document
Changed:
<
<
>
>
 
Changed:
<
<
  • If you are working in HIPE, the HIFI Data Reduction Guide will be your prime resource for all things HIFI but you should also look at the Herschel Data Analysis Guide for information about general tools, such as those for viewing and manipulating spectra and spectral cubes, and for obtaining data from the archive.
>
>
  • If you are working in HIPE, the HIFI Data Reduction Guide will be your prime resource for all things HIFI but you should also look at the Herschel Data Analysis Guide for information about general tools, such as those for viewing and manipulating spectra and spectral cubes, and for obtaining data from the archive.
    • [http://herschel.esac.esa.int/hcss-doc-12.0/load/dag/html/dag.html Herschel Data Analysis Guide]]
 
  • The full set of online documentation for the current user release also contains detailed information about the HIFI pipeline, information about scripting in HIPE, and reference manuals for command listings. New users of HIPE are recommended to read through the Quick Start Guide and the HIPE Owner's Guide
    • You can create pdfs of each document by clicking on the pdf symbol at the top of the main page.
Changed:
<
<
>
>
 
  • The full set of Herschel documentation for the developer track is also available. This documentation contains revisions and updates to the documentation associated with the current User Release but you should also be aware that it may describe functionality not available in the current User Release
Line: 117 to 119
 

Typical Data Reduction Workflow

  • Download data from the HSA. If you are downloading several observations it is better to use the tar ball.
Changed:
<
<
>
>
 
  • Inspect data quality. Both point spectra and spectral cubes can be viewed in the SpectrumExplorer. In addition to inspecting the quality of your level 2 data, you should always check the level 1 data for any bad scans. If data at level 2 appears strange then looking at level 1 data may provide insight to the problem, see the Data Primer chapter of the HIFI Data Reduction Guide for more information about levels of data
Changed:
<
<
>
>
 
  • Assess whether data should be re-pipelined.
    • Look at the section above and also at the What's New in the latest User Release to decide if you should re-pipeline.
    • Note that the ICC recommends that data processing should not be allowed to fall more than one version behind the HIPE version being used.
    • Everything you need to re-pipeline data is available in the ObservationContext, and the instructions for using the pipeline, including tips on re-pipelining using new calibration data and customising the pipeline, can be found in the Pipeline chapter of the Hifi Data Reduction Guide
Changed:
<
<
>
>
 
  • Flag data. A new user-friendly task has been developed in order to easily flag your data: FlagTool. It can be used both as a GUI (similar concept as for the FitBaseline task) or in command line
Changed:
<
<
>
>
 
  • Remove standing waves. Standing waves are a common problem in HIFI data, particularly in bands 3, 4, 6, and 7. The HIFI Data Reduction Guide provides some information about typical standing waves in HIFI data and describes how to remove them in HIPE using fitHifiFringe and the modified band pass technique
Changed:
<
<
>
>
 
  • Correct baseline drift. Baselines can be flattened by subtracted, or division in the case of real continuum. This can be done in HIPE with the fitBaseline task
Changed:
<
<
>
>
 
  • Fold frequency switch data. Frequency switch data is not folded in the pipeline and can be done with the doFold task.
Changed:
<
<
>
>
 
  • Average together H and V polarisation.
    • The noise estimates given by HSpot assume that the H and V polarisations are averaged together. This can be done in HIPE with the PolarPair task, which resamples the spectra to the same frequency scale (they are frequency calibrated with different comb measurements) and then averages them. Alternatively, you can use the Accumulate task, which allows you to specify the resampling width if you desire.
Changed:
<
<
>
>
 
    • Note that differences may be seen in H and V profiles, see the note on H and V positions. If you are particularly interested in the spatial structure of your source you may prefer not to average the H and V polarisations together.
    • You can also average maps taken with the H and V polarisations:
Changed:
<
<
>
>
 
  • Spectral manipulation. The Spectral Toolbox in HIPE allows you to perform typical spectral arithmetic tasks on spectra and spectral cubes
Changed:
<
<
>
>
 
  • Fit lines. The SpectrumFitterGUI, which is also a part of the Spectral Toolbox, can be used to fit line profiles.
Changed:
<
<
>
>
 
  • Save modified data You can save data to a pool, or store your session in order to continue working in HIPE.
Changed:
<
<
>
>
 
  • You may wish to export data as FITS, ASCII or use the HiClass task to export data as CLASS readable FITs files.
Changed:
<
<
>
>
 
      • Warning, important WARNING! When converting to velocity, CLASS corrects the reference frequency (the frequency at the reference channel) for the Doppler shift, but not the channel width so there is a very small mismatch between the channel width before and after Doppler correction. Lines are shifted from the correct velocity as a consequence, with an increasing shift away from the reference channel.. The ICC recommends that line frequencies are checked in HIPE before exporting to CLASS.

Dedicated data reduction tips for Spectral Scan data

  • The data needs to be deconvolved to a single sideband solution. This can be done with the doDeconvolution task, which works with an ObservationContext.
Changed:
<
<
>
>
 
  • It is important to clean the data before deconvolving it, this means removing standing waves, correcting baselines and flagging out any spurs or bad data that the pipeline missed.
Changed:
<
<
  • If you are planning to deconvolve your data with CLASS, please contact the Helpdesk, or Claudia Comito (email:ccomito@mpifr-bonn.mpg.de) if you need further details.
>
>
  • If you are planning to deconvolve your data with CLASS, please contact the Helpdesk, or Claudia Comito (email:ccomito@ph1.uni-koeln.de) if you need further details.
 
Changed:
<
<
  • Note that deconvolution does not work properly on data take in the Frequency Switching mode (see also the Data Known Issues Page)
>
>
 

Dedicated data reduction tips for Mapping observations

  • Spectral cubes from all HIFI mapping observations are produced as part of the SPG pipeline (that is, the pipeline run for the HSA) and are now found in the level 2.5 product. However, baseline and standing wave corrections are not done automatically in the pipeline because of the risk of harming the scientific content of the data. This can now be customised in the level 2.5 step and be done prior to the cube creation.
Line: 187 to 189
 
  • Regridding in HIPE > 9 will automatically use the comoving frame for SSOs. See the section above about the benefits from reprocessing with HIPE 11 for more details on how to do that

  • You can then grid your cleaned data into a spectral cube using the doGridding task, you can also use this task to customise the cube creation to best suit your science goals.
Changed:
<
<
>
>
 

Dedicated data reduction tips for Strong continuum observations

  • Strong continuum sources
    • An optional step in the pipeline can be used to remove standing waves arising in the loads, this is particularly effective for strong continuum sources. A report describing the technique can be found here. Instructions for using this modified passband technique are available in the Standing Wave Removal chapter of the HIFI Data Reduction Guide:
Changed:
<
<
>
>
 

HIFI performance and calibration

Line: 245 to 247
 

Access to the calibration tree:

Changed:
<
<
  • Information about what calibration is found in the HIFI ObservationContext is found in the Data Primer of the HIFI Data Reduction Guide
>
>
  • Information about what calibration is found in the HIFI ObservationContext is found in the Data Primer of the HIFI Data Reduction Guide
 
Changed:
<
<
  • Information about how to find what calibration version was used on your data, how to get the latest calibration and reprocess observations with it is found in the Pipeline chapter of the HIFI Data reduction Guide
>
>
  • Information about how to find what calibration version was used on your data, how to get the latest calibration and reprocess observations with it is found in the Pipeline chapter of the HIFI Data reduction Guide
 
Changed:
<
<
  • the latest calibration tree pool can be also retrieved directly here (4 Jul 2013)
>
>
  • the latest calibration tree pool can be also retrieved directly here (18 March 2014)
 
  • Updates to the HIFI calibration data are generally concurrent with the release of each major version of the HCSS-HIFI software. However, it is possible to have updates to the calibration data in between major releases of the software as the software and data are independent of each other, in the table below the HIPE or the OD from which the calibration updates apply are given. The calibration versions available since HIPE 5 are listed in the table below
Line: 269 to 271
 
12_0 10-05-13   10.1 Update of the list of corrupted data-frames for flagging at Level 0, updates to quality flag meta-data and uplink information for mapping modes
13_0 30-05-13   11.0 Update of the list of corrupted data-frames for flagging at Level 0, updates to spur table to account for band 3B purification
14_0 04-07-13   11.0 Update of the list of corrupted data-frames for flagging at Level 0
Added:
>
>
15_0 18-03-14   12.0 Update of the list of corrupted data-frames for flagging at Level 0, ingestion of the Electrical Standing Wave correction reference table
 
  • The side-band ratio assumed in the pipeline as of the latest calibration tree pool are illustrated in this plot.
Line: 280 to 283
 
  • Standing waves:
    • there are several standing waves that can affect the HIFI data at various processing levels. Their nature and impact on the HIFI calibration are described in the Standing Wave technical note and, more briefly, in the Standing Wave removal chapter of the HIFI Data Reduction Guide
Changed:
<
<
>
>
 
    • There are various techniques currently offered to clean these baseline distortions:
Changed:
<
<
      • Remove sine waves, or combinations of sine waves. Check the bullet about usage of the fitHifiFringe task in the data reduction section above.
      • Matching technique: for the strong IF standing waves seen in bands 6 and 7.
        • A script is available in the source directory of the 'expert' install of HIPE: scripts/hifi/scripts/users/engineering/HEBStWvCatalogCorrection.py It is extensively commented. Alternatively, the latest version can be found here
        • At the HIPE Forum 2012, a demo of a pre-release version of the script provided in HIPE 11.1 was given. The presentation, that version of the script, a prepared input catalog for use by the script, and some result plots can be found on the twiki page:
>
>
      • Remove sine waves, or combinations of sine waves. Check the bullet about usage of the fitHifiFringe task in the data reduction section above.
      • Electrical Standing Wave (aka Matching technique): to correct for the IF standing wave present in bands 6 and 7
        • A new task hebCorrection has been implemented in HIPE 12.0. We warn that the task is still under testing and does not guarantee an accurate correction in all circumstances.
        • The task is described in the following document. Unlike the script made available in previous version of HIPE, this task is applicable to any HIFI observing mode.
        • The task needs to work from level 1 data, then reprocess the corrected data up to level 2.5. Two example codes are provided here:
 
Changed:
<
<
        • Published astronomical results using the technique:
          • attached poster (Sherry, Tchernyshyov and Martin, AAS Meeting #217, #255.07; Bulletin of the American Astronomical Society, Vol. 43, 2011) poster
        • for more information please contact Ian Avruch (I.Avruch@sron.nl) at the HIFI ICC. You can also check the progress on this effort in the following internal ICC pages.
      • in case of strong source continuum some standing waves can be enhanced, as described in the Alternative Calibration Scheme report. Such standing waves can be significantly reduced in amplitude using an alternative pipeline algorithm known as the Modified Passband Technique , which is described in the Standing Wave Removal chapter of the HIFI Data Reduction Guide. Alternatively, you can run the Level1PipelineAlgo_hc_filtered_v1.py script. 
        <--Note that in 9.0, a bug still affects the doFilterLoad task in such a way that it will ignore the user-input parameters, resulting in the task always applying the "cubic_spline" method with default parameters - i.e. the "fft" method will not be applied despite being set in the task inputs. This will be fixed in 9.1.-->
>
>
      • in case of strong source continuum some standing waves can be enhanced, as described in the Alternative Calibration Scheme report. Such standing waves can be significantly reduced in amplitude using an alternative pipeline algorithm known as the Modified Passband Technique , which is described in the Standing Wave Removal chapter of the HIFI Data Reduction Guide. Alternatively, you can run the Level1PipelineAlgo_hc_filtered_v1.py script. 
        <--Note that in 9.0, a bug still affects the doFilterLoad task in such a way that it will ignore the user-input parameters, resulting in the task always applying the "cubic_spline" method with default parameters - i.e. the "fft" method will not be applied despite being set in the task inputs. This will be fixed in 9.1.-->
 
Changed:
<
<
>
>
 
Changed:
<
<
  • Baseline removal: imperfect ON-OFF calibration scheme can result in residual baseline distortion (not necessarily standing waves). Check the bullet about usage of the fitBaseline task in the data reduction section above.
>
>
  • Baseline removal: imperfect ON-OFF calibration scheme can result in residual baseline distortion (not necessarily standing waves). Check the bullet about usage of the fitBaseline task in the data reduction section above.
 
  • Spectral purity: there are places over the HIFI frequency range where the Local Oscillator does not offer a single frequency tone, so that spurious spectral line may end up in the data, and the calibration of the targeted line gets affected. A dedicated cookbook on the matter is in preparation. In the meantime, we refer to section 5.4.6 of the Observer's Manual, and to the release notes provided above.
Changed:
<
<
  • Spurs: Spurs are reported in HSpot when using the frequency editor and general information is also provided in the HIFI Observer's Manual (see above link). Spurs are also being checked in the pipeline and any detected feature is reported in the TrendAnalysis product of an observation context. The list of currently known spurs can be found in section 5.4.6 of the Observer's Manual. However not all possible spurs can be predicted or caught by the data processing, so that Users are invited to inform the Helpdesk about any features they found in their data and may not have been properly flagged by the system.
>
>
  • Spurs: Spurs are reported in HSpot when using the frequency editor and general information is also provided in the HIFI Observer's Manual (see above link). Spurs are also being checked in the pipeline and any detected feature is reported in the TrendAnalysis product of an observation context. The list of currently known spurs can be found in section 5.4.6 of the Observer's Manual. However not all possible spurs can be predicted or caught by the data processing, so that Users are invited to inform the Helpdesk about any features they found in their data and may not have been properly flagged by the system.
 

Interest Groups and Scripts

Line: 320 to 320
 
Changed:
<
<		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 11 February 2014
>
>		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 18 March 2014
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521151" name="freq_vel_3.PDF" path="freq_vel_3.PDF" size="312508" user="Main.DavidTeyssier" version="1"

Revision 632014-02-10 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 203 to 203
 
  • Side band ratio
    • Side band ratio technical note: This note describes the general calibration problem associated with the side-band gain imbalance existing in double-side-band heterodyne systems such as HIFI, and provides some recipes and calibration tables applicable to the particular case of the HIFI mixers.
  • Beam calibration
Changed:
<
<
>
>
 
Line: 320 to 320
 
Changed:
<
<		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 19 November 2013
>
>		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 11 February 2014
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521151" name="freq_vel_3.PDF" path="freq_vel_3.PDF" size="312508" user="Main.DavidTeyssier" version="1"
Line: 332 to 331
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1317389173" name="HifiObservingModesPerformance_110926a.pdf" path="HifiObservingModesPerformance_110926a.pdf" size="4428174" user="Main.JeanMatagne" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309104004" name="Sherry_et_al_AAS217_poster.pdf" path="Sherry_et_al_AAS217_poster.pdf" size="104086" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521138" name="freq_framework.pdf" path="freq_framework.pdf" size="185626" user="Main.DavidTeyssier" version="1"
Added:
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1392023495" name="spatial_response_framework_v1.9.pdf" path="spatial_response_framework_v1.9.pdf" size="603181" user="Main.DavidTeyssier" version="1"
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309104638" name="thesis_ronan_higgins_nuim.pdf" path="thesis_ronan_higgins_nuim.pdf" size="19376465" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1359637018" name="HIFI_Calibration_ReleaseNotes.pdf" path="HIFI_Calibration_ReleaseNotes.pdf" size="178716" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1313009238" name="calibration_hifi_stdw_2011_v0_1.pdf" path="calibration_hifi_stdw_2011_v0_1.pdf" size="4118561" user="Main.CarolynMcCoey" version="1"

Revision 622014-02-04 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 30 to 30
 

Recommended User release

Deleted:
<
<		Mac OS X and Java 7 Issues : please consult the HIPE Known Issues page regarding important information on issues with Java for different versions of HIPE.
 

Which data would most benefit from being reprocessed in HIPE 11.1?

If your data have been processed with the latest bulk reprocessing (10.3):

Revision 612014-01-31 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 30 to 30
 

Recommended User release

Added:
>
>		Mac OS X and Java 7 Issues : please consult the HIPE Known Issues page regarding important information on issues with Java for different versions of HIPE.
 

Which data would most benefit from being reprocessed in HIPE 11.1?

If your data have been processed with the latest bulk reprocessing (10.3):

Revision 602013-11-19 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Changed:
<
<		NEW NEW NEW Latest updates -- 23 October 2013 NEW NEW NEW
  • HIPE 11.0 has been released !
>
>		NEW NEW NEW Latest updates -- 19 November 2013 NEW NEW NEW
  • HIPE 11.1 has been released !
 
  • HIPI 2.1 (a library of Hifi Plug-In's) has been released - see Interest Groups and Scripts section
  • New matching technique script is available in the Expert Version of HIPE - see the "Outstanding calibration issues" section
  • A large amount of calibration data has been released ! - see the "Observing with HIFI" section
Line: 28 to 28
 

Reducing HIFI data

Recommended User release

Changed:
<
<
>
>
 
Changed:
<
<

Which data would most benefit from being reprocessed in HIPE 11.0?

>
>

Which data would most benefit from being reprocessed in HIPE 11.1?

  If your data have been processed with the latest bulk reprocessing (10.3):
Deleted:
<
<
  • DBS Observations
    • A new pipeline step, mkDbsReference, calculates the differences the chop positions in all DBS observations and after applying the band-pass correction stores them in a product in calibration->pipeline-out called ReferenceSpectra. This allows you to check for contamination in chop positions for all types of DBS observations.
 
  • Rotated maps
    • Rotated cubes are now produced in the Level 2.5 product by default for maps carried out with a non-zero position angle, a non-rotated cube is also produced.

  • Solar System Object ephemerides
    • A bug (see here) was fixed in the calculation of SSO ephemeris positions (ra_centre/dec_centre) that led to offsets of up to ~10". This bug affected users that use cubes in a co-moving frame (including the standard level 2.5 cubes), use the doOffset task, or make explicit use of ra_centre / dec_centre in their HIPE scripts. The offset is practically constant over a map and could, e.g., make the emission of a comet appear to be off-center when it really isn't. The fixed bug is in the level 0 pipeline, which users cannot run easily. The issue will be fixed by the HIPE 11 bulk reprocessing. In the meantime, you can use a HIPE snippet such as the one attached here.
Added:
>
>
  • Improved pointing reconstruction for observations taken before OD 320
  If your data have been processed with a version earlier than 10.3:
Added:
>
>
  • DBS Observations If you have data processed with < HIPE 10
    • A new pipeline step, mkDbsReference, calculates the differences the chop positions in all DBS observations and after applying the band-pass correction stores them in a product in calibration->pipeline-out called ReferenceSpectra. This allows you to check for contamination in chop positions for all types of DBS observations.
 
  • Improved pointing reconstruction for observations taken between OD 320 and 761 If you have data processed with < HIPE 9
    • The pointing information attached to the data for observations taken between OD 320 and OD761 did not use the most accurate representation of the star tracker focal length. This will be done for the bulk reprocessing with HIPE 9. As a consequence, some observations will experience a shift in astrometry, that can be as high as 8 arcsec. All the details about the consequences for a particular obsid, and recipes to reconstruct yourself the improved pointing can be found at http://herschel.esac.esa.int/twiki/bin/view/Public/HowToUseImprovedPointingProducts as well as in the level0 section of the Hifi Data Reduction Guide
Line: 58 to 60
 
    • Un-balanced side-band gains will apply for some ranges of these bands (see also the calibration section). Reprocessing data from band 2a is recommended if your data are still processed with HIPE 6. Reprocessing data from band 5a and 5b is recommended if your data are processed with HIPE 8.

  • Spectral Scan Data in bands 6 and 7 If you have data processed with < HIPE 8
Changed:
<
<
    • It is recommended to reprocess spectral scan data taken in band 6 and 7 if they are still processed with HIPE <8. This is not needed in data processed with HIPE 8.
    • Reprocessing with HIPE 9 will create a new level 2.5 product containing a deconvolved single-side-band spectrum for each applicable spectrometer
>
>
    • It is recommended to reprocess spectral scan data taken in band 6 and 7 if they are still processed with HIPE <8. This is not needed in data processed with HIPE 8 onward.
    • Reprocessing with HIPE 9 onward will create a new level 2.5 product containing a deconvolved single-side-band spectrum for each applicable spectrometer
 
  • Maps If you have data processed with < HIPE 8
    • Spectral cubes from mapping observations after OD 835 gridded using actual scan line and readout used in observation, earlier maps use a better approximation than in HIPE 7. These improvements are a consequence of changes in the level 0 pipeline to read in information about how the observation was performed and require the pipeline to be re-run after running the doUplink step:
Line: 79 to 81
  -->
Changed:
<
<
  • See the What's New page for more details of improvements in HIPE 11.0
>
>
  • See the What's New page for more details of improvements in HIPE 11.1
 

Changed:
<
<
>
>
 

Documentation and Cookbooks

  • The HIFI Launch Pad is intended to help you quickly off the ground with HIFI data reduction
Line: 146 to 148
 
    • Note that differences may be seen in H and V profiles, see the note on H and V positions. If you are particularly interested in the spatial structure of your source you may prefer not to average the H and V polarisations together.
Added:
>
>
 
Line: 195 to 199
 

Documentation

Deleted:
<
<
  • This FTP repository of system temperature plots across the IF for each HIFI subband (updated 24 Jan 2011) inform about the variation in system temperature (sensitivity) across the HIFI intermediate frequency band at various frequencies
 
Added:
>
>
  • Side band ratio
    • Side band ratio technical note: This note describes the general calibration problem associated with the side-band gain imbalance existing in double-side-band heterodyne systems such as HIFI, and provides some recipes and calibration tables applicable to the particular case of the HIFI mixers.
 
Line: 205 to 210
 
Changed:
<
<
  • Side band ratio
    • Side band ratio technical note: This note describes the general calibration problem associated with the side-band gain imbalance existing in double-side-band heterodyne systems such as HIFI, and provides some recipes and calibration tables applicable to the particular case of the HIFI mixers.
>
>
  • This FTP repository of system temperature plots across the IF for each HIFI subband (updated 24 Jan 2011) inform about the variation in system temperature (sensitivity) across the HIFI intermediate frequency band at various frequencies
 

Calibration error budget

Line: 282 to 286
 
      • Matching technique: for the strong IF standing waves seen in bands 6 and 7.
        • A script is available in the source directory of the 'expert' install of HIPE: scripts/hifi/scripts/users/engineering/HEBStWvCatalogCorrection.py It is extensively commented. Alternatively, the latest version can be found here
Changed:
<
<
        • At the HIPE Forum 2012, a demo of a pre-release version of the script provided in HIPE 11.0 was given. The presentation, that version of the script, a prepared input catalog for use by the script, and some result plots can be found on the twiki page:
>
>
        • At the HIPE Forum 2012, a demo of a pre-release version of the script provided in HIPE 11.1 was given. The presentation, that version of the script, a prepared input catalog for use by the script, and some result plots can be found on the twiki page:
 
Line: 316 to 320
 
Changed:
<
<		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 23 October 2013
>
>		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 19 November 2013
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521151" name="freq_vel_3.PDF" path="freq_vel_3.PDF" size="312508" user="Main.DavidTeyssier" version="1"

Revision 592013-10-23 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Changed:
<
<		NEW NEW NEW Latest updates -- 18 September 2013 NEW NEW NEW
>
>		NEW NEW NEW Latest updates -- 23 October 2013 NEW NEW NEW
 
  • HIPE 11.0 has been released !
  • HIPI 2.1 (a library of Hifi Plug-In's) has been released - see Interest Groups and Scripts section
  • New matching technique script is available in the Expert Version of HIPE - see the "Outstanding calibration issues" section
Line: 15 to 15
 
Deleted:
<
<
 
Deleted:
<
<
 
  • HIFI calibration data
    • A significant number of the data taken by HIFI in the framework of the Performance Validation (PV) and routine phase are now publicly available. There are particular caveats that apply to those data. They are described in this disclaimer note

Added:
>
>
 

Reducing HIFI data

Recommended User release

Line: 30 to 32
 

Which data would most benefit from being reprocessed in HIPE 11.0?

Added:
>
>		 If your data have been processed with the latest bulk reprocessing (10.3):
 
  • DBS Observations
    • A new pipeline step, mkDbsReference, calculates the differences the chop positions in all DBS observations and after applying the band-pass correction stores them in a product in calibration->pipeline-out called ReferenceSpectra. This allows you to check for contamination in chop positions for all types of DBS observations.

  • Rotated maps
    • Rotated cubes are now produced in the Level 2.5 product by default for maps carried out with a non-zero position angle, a non-rotated cube is also produced.

Added:
>
>
  • Solar System Object ephemerides
    • A bug (see here) was fixed in the calculation of SSO ephemeris positions (ra_centre/dec_centre) that led to offsets of up to ~10". This bug affected users that use cubes in a co-moving frame (including the standard level 2.5 cubes), use the doOffset task, or make explicit use of ra_centre / dec_centre in their HIPE scripts. The offset is practically constant over a map and could, e.g., make the emission of a comet appear to be off-center when it really isn't. The fixed bug is in the level 0 pipeline, which users cannot run easily. The issue will be fixed by the HIPE 11 bulk reprocessing. In the meantime, you can use a HIPE snippet such as the one attached here.

If your data have been processed with a version earlier than 10.3:

 
  • Improved pointing reconstruction for observations taken between OD 320 and 761 If you have data processed with < HIPE 9
    • The pointing information attached to the data for observations taken between OD 320 and OD761 did not use the most accurate representation of the star tracker focal length. This will be done for the bulk reprocessing with HIPE 9. As a consequence, some observations will experience a shift in astrometry, that can be as high as 8 arcsec. All the details about the consequences for a particular obsid, and recipes to reconstruct yourself the improved pointing can be found at http://herschel.esac.esa.int/twiki/bin/view/Public/HowToUseImprovedPointingProducts as well as in the level0 section of the Hifi Data Reduction Guide
Line: 45 to 54
 
  • Solar System Object maps If you have data processed with < HIPE 9
    • Cubes for moving targets are now created in the comoving frames in HIPE 9. In order to benefit from that, you should re-pipeline from level 0 up to level 2.5. Note the hifiPipeline task should be called with the option "Aux=True". Please check the Pipeline chapter of the HIFI Data Reduction Guide for more details about how to do that.
Deleted:
<
<
  • Solar System Object ephemerides If you have data processed with < HIPE 11
    • A bug (see here) was fixed in the calculation of SSO ephemeris positions (ra_centre/dec_centre) that led to offsets of up to ~10". This bug affected users that use cubes in a co-moving frame (including the standard level 2.5 cubes), use the doOffset task, or make explicit use of ra_centre / dec_centre in their HIPE scripts. The offset is practically constant over a map and could, e.g., make the emission of a comet appear to be off-center when it really isn't. The fixed bug is in the level 0 pipeline, which users cannot run easily. The issue will be fixed by the HIPE 11 bulk reprocessing. In the meantime, you can use a HIPE snippet such as the one attached here.

  • Strong continuum sources
    • An optional step in the pipeline can be used to remove standing waves arising in the loads, this is particularly effective for strong continuum sources. A report describing the technique can be found here. Instructions for using this modified passband technique are available in the Standing Wave Removal chapter of the HIFI Data Reduction Guide:
 

Revision 582013-09-18 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Changed:
<
<		NEW NEW NEW Latest updates -- 26 July 2013 NEW NEW NEW
>
>		NEW NEW NEW Latest updates -- 18 September 2013 NEW NEW NEW
 
  • HIPE 11.0 has been released !
Changed:
<
<
  • HIPI (a library of Hifi Plug-In's) - see Interest Groups and Scripts section
>
>
  • HIPI 2.1 (a library of Hifi Plug-In's) has been released - see Interest Groups and Scripts section
 
  • New matching technique script is available in the Expert Version of HIPE - see the "Outstanding calibration issues" section
  • A large amount of calibration data has been released ! - see the "Observing with HIFI" section

Revision 572013-07-29 - AnthonyMarston

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Changed:
<
<		NEW NEW NEW Latest updates -- 27 June 2013 NEW NEW NEW
  • HIPE 10.3 has been released !
>
>		NEW NEW NEW Latest updates -- 26 July 2013 NEW NEW NEW
  • HIPE 11.0 has been released !
 
  • HIPI (a library of Hifi Plug-In's) - see Interest Groups and Scripts section
  • New matching technique script is available in the Expert Version of HIPE - see the "Outstanding calibration issues" section
  • A large amount of calibration data has been released ! - see the "Observing with HIFI" section
Line: 25 to 26
 

Reducing HIFI data

Recommended User release

Changed:
<
<
>
>
 
Changed:
<
<

Which data would most benefit from being reprocessed in HIPE 10.3?

>
>

Which data would most benefit from being reprocessed in HIPE 11.0?

 
  • DBS Observations
    • A new pipeline step, mkDbsReference, calculates the differences the chop positions in all DBS observations and after applying the band-pass correction stores them in a product in calibration->pipeline-out called ReferenceSpectra. This allows you to check for contamination in chop positions for all types of DBS observations.
Line: 36 to 37
 
    • Rotated cubes are now produced in the Level 2.5 product by default for maps carried out with a non-zero position angle, a non-rotated cube is also produced.

  • Improved pointing reconstruction for observations taken between OD 320 and 761 If you have data processed with < HIPE 9
Changed:
<
<
    • The pointing information attached to the data for observations taken between OD 320 and OD761 did not use the most accurate representation of the star tracker focal length. This will be done for the bulk reprocessing with HIPE 9. As a consequence, some observations will experience a shift in astrometry, that can be as high as 8 arcsec. All the details about the consequences for a particular obsid, and recipes to reconstruct yourself the improved pointing can be found at http://herschel.esac.esa.int/twiki/bin/view/Public/HowToUseImprovedPointingProducts as well as in the level0 section of the Hifi Data Reduction Guide
>
>
    • The pointing information attached to the data for observations taken between OD 320 and OD761 did not use the most accurate representation of the star tracker focal length. This will be done for the bulk reprocessing with HIPE 9. As a consequence, some observations will experience a shift in astrometry, that can be as high as 8 arcsec. All the details about the consequences for a particular obsid, and recipes to reconstruct yourself the improved pointing can be found at http://herschel.esac.esa.int/twiki/bin/view/Public/HowToUseImprovedPointingProducts as well as in the level0 section of the Hifi Data Reduction Guide
 
  • Backfilling of observational parameters If you have data processed with < HIPE 9
    • From 9.1 onwards, most of the observational parameters that got optimised in HSpot will be propagated into the observation context, under a new product called HifiUplinkProduct (in the auxiliary product branch). Some of these parameters will be used in the pipeline to e.g. estimate the dimension of the cubes in the mapping and have them more representative of how the map was really obtained.

  • Solar System Object maps If you have data processed with < HIPE 9
Changed:
<
<
    • Cubes for moving targets are now created in the comoving frames in HIPE 9. In order to benefit from that, you should re-pipeline from level 0 up to level 2.5. Note the hifiPipeline task should be called with the option "Aux=True". Please check the Pipeline chapter of the HIFI Data Reduction Guide for more details about how to do that.
>
>
    • Cubes for moving targets are now created in the comoving frames in HIPE 9. In order to benefit from that, you should re-pipeline from level 0 up to level 2.5. Note the hifiPipeline task should be called with the option "Aux=True". Please check the Pipeline chapter of the HIFI Data Reduction Guide for more details about how to do that.

  • Solar System Object ephemerides If you have data processed with < HIPE 11
    • A bug (see here) was fixed in the calculation of SSO ephemeris positions (ra_centre/dec_centre) that led to offsets of up to ~10". This bug affected users that use cubes in a co-moving frame (including the standard level 2.5 cubes), use the doOffset task, or make explicit use of ra_centre / dec_centre in their HIPE scripts. The offset is practically constant over a map and could, e.g., make the emission of a comet appear to be off-center when it really isn't. The fixed bug is in the level 0 pipeline, which users cannot run easily. The issue will be fixed by the HIPE 11 bulk reprocessing. In the meantime, you can use a HIPE snippet such as the one attached here.
 
  • Strong continuum sources
    • An optional step in the pipeline can be used to remove standing waves arising in the loads, this is particularly effective for strong continuum sources. A report describing the technique can be found here. Instructions for using this modified passband technique are available in the Standing Wave Removal chapter of the HIFI Data Reduction Guide:
Changed:
<
<
>
>
 
Changed:
<
<
>
>
 
Changed:
<
<
>
>
 

Documentation and Cookbooks

  • The HIFI Launch Pad is intended to help you quickly off the ground with HIFI data reduction
Changed:
<
<
>
>
 
  • There is a cookbook for point mode DBS observations, other observing mode cookbooks are in progress
Changed:
<
<
>
>
 
  • The HIFI Data Reduction Guide and the HIFI AOT Observing Mode Release and Performance Notes contain information you need to know about HIFI data, regardless of the software you use for data reduction
Changed:
<
<
>
>
 

  • If you want to know more about the details of the HIFI pipeline, you should look at the HIFI Pipeline Specification Document
Changed:
<
<
>
>
 
Changed:
<
<
  • If you are working in HIPE, the HIFI Data Reduction Guide will be your prime resource for all things HIFI but you should also look at the Herschel Data Analysis Guide for information about general tools, such as those for viewing and manipulating spectra and spectral cubes, and for obtaining data from the archive.
>
>
  • If you are working in HIPE, the HIFI Data Reduction Guide will be your prime resource for all things HIFI but you should also look at the Herschel Data Analysis Guide for information about general tools, such as those for viewing and manipulating spectra and spectral cubes, and for obtaining data from the archive.
 
  • The full set of online documentation for the current user release also contains detailed information about the HIFI pipeline, information about scripting in HIPE, and reference manuals for command listings. New users of HIPE are recommended to read through the Quick Start Guide and the HIPE Owner's Guide
    • You can create pdfs of each document by clicking on the pdf symbol at the top of the main page.
Changed:
<
<
>
>
 
  • The full set of Herschel documentation for the developer track is also available. This documentation contains revisions and updates to the documentation associated with the current User Release but you should also be aware that it may describe functionality not available in the current User Release
Changed:
<
<
>
>
 
Line: 109 to 113
 

Typical Data Reduction Workflow

  • Download data from the HSA. If you are downloading several observations it is better to use the tar ball.
Changed:
<
<
>
>
 
  • Inspect data quality. Both point spectra and spectral cubes can be viewed in the SpectrumExplorer. In addition to inspecting the quality of your level 2 data, you should always check the level 1 data for any bad scans. If data at level 2 appears strange then looking at level 1 data may provide insight to the problem, see the Data Primer chapter of the HIFI Data Reduction Guide for more information about levels of data
Changed:
<
<
>
>
 
  • Assess whether data should be re-pipelined.
    • Look at the section above and also at the What's New in the latest User Release to decide if you should re-pipeline.
    • Note that the ICC recommends that data processing should not be allowed to fall more than one version behind the HIPE version being used.
    • Everything you need to re-pipeline data is available in the ObservationContext, and the instructions for using the pipeline, including tips on re-pipelining using new calibration data and customising the pipeline, can be found in the Pipeline chapter of the Hifi Data Reduction Guide
Changed:
<
<
>
>
 
  • Flag data. A new user-friendly task has been developed in order to easily flag your data: FlagTool. It can be used both as a GUI (similar concept as for the FitBaseline task) or in command line
Changed:
<
<
>
>
 
  • Remove standing waves. Standing waves are a common problem in HIFI data, particularly in bands 3, 4, 6, and 7. The HIFI Data Reduction Guide provides some information about typical standing waves in HIFI data and describes how to remove them in HIPE using fitHifiFringe and the modified band pass technique
Changed:
<
<
>
>
 
  • Correct baseline drift. Baselines can be flattened by subtracted, or division in the case of real continuum. This can be done in HIPE with the fitBaseline task
Changed:
<
<
>
>
 
  • Fold frequency switch data. Frequency switch data is not folded in the pipeline and can be done with the doFold task.
Changed:
<
<
>
>
 
  • Average together H and V polarisation.
    • The noise estimates given by HSpot assume that the H and V polarisations are averaged together. This can be done in HIPE with the PolarPair task, which resamples the spectra to the same frequency scale (they are frequency calibrated with different comb measurements) and then averages them. Alternatively, you can use the Accumulate task, which allows you to specify the resampling width if you desire.
Changed:
<
<
>
>
 
    • Note that differences may be seen in H and V profiles, see the note on H and V positions. If you are particularly interested in the spatial structure of your source you may prefer not to average the H and V polarisations together.

  • Spectral manipulation. The Spectral Toolbox in HIPE allows you to perform typical spectral arithmetic tasks on spectra and spectral cubes
Changed:
<
<
>
>
 
  • Fit lines. The SpectrumFitterGUI, which is also a part of the Spectral Toolbox, can be used to fit line profiles.
Changed:
<
<
>
>
 
  • Save modified data You can save data to a pool, or store your session in order to continue working in HIPE.
Changed:
<
<
>
>
 
  • You may wish to export data as FITS, ASCII or use the HiClass task to export data as CLASS readable FITs files.
Changed:
<
<
>
>
 
      • Warning, important WARNING! When converting to velocity, CLASS corrects the reference frequency (the frequency at the reference channel) for the Doppler shift, but not the channel width so there is a very small mismatch between the channel width before and after Doppler correction. Lines are shifted from the correct velocity as a consequence, with an increasing shift away from the reference channel.. The ICC recommends that line frequencies are checked in HIPE before exporting to CLASS.

Dedicated data reduction tips for Spectral Scan data

  • The data needs to be deconvolved to a single sideband solution. This can be done with the doDeconvolution task, which works with an ObservationContext.
Changed:
<
<
>
>
 
  • It is important to clean the data before deconvolving it, this means removing standing waves, correcting baselines and flagging out any spurs or bad data that the pipeline missed.
Line: 174 to 178
 
  • It is strongly recommended that you inspect your level 2 HTPs for baseline drift and residual standing waves and decide if some, or all, datasets need to be cleaned up before re-running the gridding task.
Changed:
<
<
  • Regridding in HIPE > 9 will automatically use the comoving frame for SSOs. See the section above about the benefits from reprocessing with HIPE 10 for more details on how to do that
>
>
  • Regridding in HIPE > 9 will automatically use the comoving frame for SSOs. See the section above about the benefits from reprocessing with HIPE 11 for more details on how to do that
 
  • You can then grid your cleaned data into a spectral cube using the doGridding task, you can also use this task to customise the cube creation to best suit your science goals.
Changed:
<
<
>
>
 

HIFI performance and calibration

Line: 234 to 238
 
  • Information about how to find what calibration version was used on your data, how to get the latest calibration and reprocess observations with it is found in the Pipeline chapter of the HIFI Data reduction Guide
Changed:
<
<
  • the latest calibration tree pool can be also retrieved directly here (16 May 2013)
>
>
  • the latest calibration tree pool can be also retrieved directly here (4 Jul 2013)
 
  • Updates to the HIFI calibration data are generally concurrent with the release of each major version of the HCSS-HIFI software. However, it is possible to have updates to the calibration data in between major releases of the software as the software and data are independent of each other, in the table below the HIPE or the OD from which the calibration updates apply are given. The calibration versions available since HIPE 5 are listed in the table below
Line: 252 to 256
 
10_0 08-11-12   9.1 Introduction of a new HifiUplink product to back-fill most the observational parameters as of HSpot optimisation back into the Uplink product, update of the list of corrupted data-frames for flagging at Level 0, update of a number of engineering threshold for more accurate quality flagging related to hardware housekeeping
11_0 22-01-13   10.0 Update of the list of corrupted data-frames for flagging at Level 0, Quality flags created for SEUs (Singe Event Upsets leading to on-board software corruption)
12_0 10-05-13   10.1 Update of the list of corrupted data-frames for flagging at Level 0, updates to quality flag meta-data and uplink information for mapping modes
Added:
>
>
13_0 30-05-13   11.0 Update of the list of corrupted data-frames for flagging at Level 0, updates to spur table to account for band 3B purification
14_0 04-07-13   11.0 Update of the list of corrupted data-frames for flagging at Level 0
 
  • The side-band ratio assumed in the pipeline as of the latest calibration tree pool are illustrated in this plot.
Line: 263 to 269
 
  • Standing waves:
    • there are several standing waves that can affect the HIFI data at various processing levels. Their nature and impact on the HIFI calibration are described in the Standing Wave technical note and, more briefly, in the Standing Wave removal chapter of the HIFI Data Reduction Guide
Changed:
<
<
>
>
 
    • There are various techniques currently offered to clean these baseline distortions:
Changed:
<
<
      • Remove sine waves, or combinations of sine waves. Check the bullet about usage of the fitHifiFringe task in the data reduction section above.
>
>
      • Remove sine waves, or combinations of sine waves. Check the bullet about usage of the fitHifiFringe task in the data reduction section above.
 
      • Matching technique: for the strong IF standing waves seen in bands 6 and 7.
        • A script is available in the source directory of the 'expert' install of HIPE: scripts/hifi/scripts/users/engineering/HEBStWvCatalogCorrection.py It is extensively commented. Alternatively, the latest version can be found here
Changed:
<
<
        • At the HIPE Forum 2012, a demo of a pre-release version of the script provided in HIPE 10.0 was given. The presentation, that version of the script, a prepared input catalog for use by the script, and some result plots can be found on the twiki page:
>
>
        • At the HIPE Forum 2012, a demo of a pre-release version of the script provided in HIPE 11.0 was given. The presentation, that version of the script, a prepared input catalog for use by the script, and some result plots can be found on the twiki page:
 
Line: 279 to 285
 
        • for more information please contact Ian Avruch (I.Avruch@sron.nl) at the HIFI ICC. You can also check the progress on this effort in the following internal ICC pages.
      • in case of strong source continuum some standing waves can be enhanced, as described in the Alternative Calibration Scheme report. Such standing waves can be significantly reduced in amplitude using an alternative pipeline algorithm known as the Modified Passband Technique , which is described in the Standing Wave Removal chapter of the HIFI Data Reduction Guide. Alternatively, you can run the Level1PipelineAlgo_hc_filtered_v1.py script. 
        <--Note that in 9.0, a bug still affects the doFilterLoad task in such a way that it will ignore the user-input parameters, resulting in the task always applying the "cubic_spline" method with default parameters - i.e. the "fft" method will not be applied despite being set in the task inputs. This will be fixed in 9.1.-->
Changed:
<
<
>
>
 
Changed:
<
<
  • Baseline removal: imperfect ON-OFF calibration scheme can result in residual baseline distortion (not necessarily standing waves). Check the bullet about usage of the fitBaseline task in the data reduction section above.
>
>
  • Baseline removal: imperfect ON-OFF calibration scheme can result in residual baseline distortion (not necessarily standing waves). Check the bullet about usage of the fitBaseline task in the data reduction section above.
 
  • Spectral purity: there are places over the HIFI frequency range where the Local Oscillator does not offer a single frequency tone, so that spurious spectral line may end up in the data, and the calibration of the targeted line gets affected. A dedicated cookbook on the matter is in preparation. In the meantime, we refer to section 5.4.6 of the Observer's Manual, and to the release notes provided above.
Changed:
<
<
  • Spurs: Spurs are reported in HSpot when using the frequency editor and general information is also provided in the HIFI Observer's Manual (see above link). Spurs are also being checked in the pipeline and any detected feature is reported in the TrendAnalysis product of an observation context. The list of currently known spurs can be found in section 5.4.6 of the Observer's Manual. However not all possible spurs can be predicted or caught by the data processing, so that Users are invited to inform the Helpdesk about any features they found in their data and may not have been properly flagged by the system.
>
>
  • Spurs: Spurs are reported in HSpot when using the frequency editor and general information is also provided in the HIFI Observer's Manual (see above link). Spurs are also being checked in the pipeline and any detected feature is reported in the TrendAnalysis product of an observation context. The list of currently known spurs can be found in section 5.4.6 of the Observer's Manual. However not all possible spurs can be predicted or caught by the data processing, so that Users are invited to inform the Helpdesk about any features they found in their data and may not have been properly flagged by the system.
 

Interest Groups and Scripts

Line: 303 to 309
 
Changed:
<
<		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 27 June 2013
>
>		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 26 July 2013
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521151" name="freq_vel_3.PDF" path="freq_vel_3.PDF" size="312508" user="Main.DavidTeyssier" version="1"

Revision 562013-06-27 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 194 to 194
 
Changed:
<
<
  • The following note describes the general calibration problem associated with the side-band gain imbalance existing in double-side-band heterodyne systems such as HIFI, and provides some recipes and calibration tables applicable to the particular case of the HIFI mixers.
>
>
  • Side band ratio
    • Side band ratio technical note: This note describes the general calibration problem associated with the side-band gain imbalance existing in double-side-band heterodyne systems such as HIFI, and provides some recipes and calibration tables applicable to the particular case of the HIFI mixers.
 

Calibration error budget

Revision 552013-06-27 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Changed:
<
<		NEW NEW NEW Latest updates -- 18 June 2013 NEW NEW NEW
>
>		NEW NEW NEW Latest updates -- 27 June 2013 NEW NEW NEW
 
  • HIPE 10.3 has been released !
  • HIPI (a library of Hifi Plug-In's) - see Interest Groups and Scripts section
  • New matching technique script is available in the Expert Version of HIPE - see the "Outstanding calibration issues" section
Line: 194 to 194
 
Added:
>
>
  • The following note describes the general calibration problem associated with the side-band gain imbalance existing in double-side-band heterodyne systems such as HIFI, and provides some recipes and calibration tables applicable to the particular case of the HIFI mixers.
 

Calibration error budget

Line: 301 to 302
 
Changed:
<
<		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 18 June 2013
>
>		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 27 June 2013
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521151" name="freq_vel_3.PDF" path="freq_vel_3.PDF" size="312508" user="Main.DavidTeyssier" version="1"
Line: 318 to 319
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1322760092" name="aa15120-10.pdf" path="aa15120-10.pdf" size="386925" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521098" name="calibframework.pdf" path="calibframework.pdf" size="310766" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1345720705" name="HIFI_SBR_Summary_V9.0.png" path="HIFI_SBR_Summary_V9.0.png" size="88066" user="Main.DavidTeyssier" version="1"
Added:
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="HIFI_SBR_Cookbook_V1.0" date="1372338964" name="HIFI_SBR_Cookbook_V1.0.pdf" path="HIFI_SBR_Cookbook_V1.0.pdf" size="1474786" user="Main.SylvieFBeaulieu" version="1"
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1309110936" name="freq_vel_1.1.pdf" path="freq_vel_1.1.pdf" size="261519" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1323853023" name="Alternative_Calibration_Schemes_v1.0.pdf" path="Alternative_Calibration_Schemes_v1.0.pdf" size="2883364" user="Main.DavidTeyssier" version="1"

Revision 542013-06-18 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Changed:
<
<		NEW NEW NEW Latest updates -- 4 June 2013 NEW NEW NEW
  • HIPE 10.2 has been released !
>
>		NEW NEW NEW Latest updates -- 18 June 2013 NEW NEW NEW
  • HIPE 10.3 has been released !
 
  • HIPI (a library of Hifi Plug-In's) - see Interest Groups and Scripts section
  • New matching technique script is available in the Expert Version of HIPE - see the "Outstanding calibration issues" section
  • A large amount of calibration data has been released ! - see the "Observing with HIFI" section
Line: 25 to 25
 

Reducing HIFI data

Recommended User release

Changed:
<
<
>
>
 
Changed:
<
<

Which data would most benefit from being reprocessed in HIPE 10.2?

>
>

Which data would most benefit from being reprocessed in HIPE 10.3?

 
  • DBS Observations
    • A new pipeline step, mkDbsReference, calculates the differences the chop positions in all DBS observations and after applying the band-pass correction stores them in a product in calibration->pipeline-out called ReferenceSpectra. This allows you to check for contamination in chop positions for all types of DBS observations.
Line: 73 to 73
  -->
Changed:
<
<
  • See the What's New page for more details of improvements in HIPE 10.2
>
>
  • See the What's New page for more details of improvements in HIPE 10.3
 

Changed:
<
<
>
>
 

Documentation and Cookbooks

  • The HIFI Launch Pad is intended to help you quickly off the ground with HIFI data reduction
Line: 301 to 301
 
Changed:
<
<		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 4 June 2013
>
>		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 18 June 2013
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521151" name="freq_vel_3.PDF" path="freq_vel_3.PDF" size="312508" user="Main.DavidTeyssier" version="1"

Revision 532013-06-05 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Changed:
<
<		NEW NEW NEW Latest updates -- 14 May 2013 NEW NEW NEW
  • HIPE 10.1 has been released !
>
>		NEW NEW NEW Latest updates -- 4 June 2013 NEW NEW NEW
  • HIPE 10.2 has been released !
 
  • HIPI (a library of Hifi Plug-In's) - see Interest Groups and Scripts section
  • New matching technique script is available in the Expert Version of HIPE - see the "Outstanding calibration issues" section
  • A large amount of calibration data has been released ! - see the "Observing with HIFI" section
Line: 25 to 25
 

Reducing HIFI data

Recommended User release

Changed:
<
<
>
>
 
Changed:
<
<

Which data would most benefit from being reprocessed in HIPE 10.1?

>
>

Which data would most benefit from being reprocessed in HIPE 10.2?

 
  • DBS Observations
    • A new pipeline step, mkDbsReference, calculates the differences the chop positions in all DBS observations and after applying the band-pass correction stores them in a product in calibration->pipeline-out called ReferenceSpectra. This allows you to check for contamination in chop positions for all types of DBS observations.
Line: 73 to 73
  -->
Changed:
<
<
  • See the What's New page for more details of improvements in HIPE 10.1
>
>
  • See the What's New page for more details of improvements in HIPE 10.2
 

Changed:
<
<
>
>
 

Documentation and Cookbooks

  • The HIFI Launch Pad is intended to help you quickly off the ground with HIFI data reduction
Line: 301 to 301
 
Changed:
<
<		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 16 May 2013
>
>		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 4 June 2013
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521151" name="freq_vel_3.PDF" path="freq_vel_3.PDF" size="312508" user="Main.DavidTeyssier" version="1"

Revision 522013-05-27 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 212 to 212
 
Pointing < 1 < 2 < 2 < 4
Opt. standing waves 4 4 3 3
Changed:
<
<		*Note that the "side-band gain ratio" convention used is the HIFI pipeline does not correspond to the ratio between the respective side-band gains (equal to 1 for a balanced system), but to the gain in one side-band normalised by the combined gain of the two side-band. Consequently, perfectly-balanced side-bands will feature a side-band ratio of 0.5 (see also section 5.3.1 of the Observers' Manual, and the Intensity Calibration Framework document).
>
>		*Note that the "side-band gain ratio" convention used in the HIFI pipeline does not correspond to the ratio between the respective side-band gains (equal to 1 for a balanced system), but to the gain in one side-band normalised by the combined gain of the two side-band. Consequently, perfectly-balanced side-bands will feature a side-band ratio of 0.5 (see also section 5.3.1 of the Observers' Manual, and the Intensity Calibration Framework document).
  Frequency Calibration Accuracy

Revision 512013-05-17 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Changed:
<
<		NEW NEW NEW Latest updates -- 31 January 2013 NEW NEW NEW
  • HIPE 10.0 has been released !
>
>		NEW NEW NEW Latest updates -- 14 May 2013 NEW NEW NEW
  • HIPE 10.1 has been released !
 
  • HIPI (a library of Hifi Plug-In's) - see Interest Groups and Scripts section
  • New matching technique script is available in the Expert Version of HIPE - see the "Outstanding calibration issues" section
  • A large amount of calibration data has been released ! - see the "Observing with HIFI" section
Line: 25 to 25
 

Reducing HIFI data

Recommended User release

Changed:
<
<
>
>
 
Changed:
<
<

Which data would most benefit from being reprocessed in HIPE 10.0?

>
>

Which data would most benefit from being reprocessed in HIPE 10.1?

 
  • DBS Observations
    • A new pipeline step, mkDbsReference, calculates the differences the chop positions in all DBS observations and after applying the band-pass correction stores them in a product in calibration->pipeline-out called ReferenceSpectra. This allows you to check for contamination in chop positions for all types of DBS observations.
Line: 73 to 73
  -->
Changed:
<
<
  • See the What's New page for more details of improvements in HIPE 10.0
>
>
  • See the What's New page for more details of improvements in HIPE 10.1
 

Changed:
<
<
>
>
 

Documentation and Cookbooks

  • The HIFI Launch Pad is intended to help you quickly off the ground with HIFI data reduction
Line: 92 to 92
 
Changed:
<
<
  • If you are working in HIPE, the HIFI Data Reduction Guide will be your prime resource for all things HIFI but you should also look at the Herschel Data Analysis Guide for information about general tools, such as those for viewing and manipulating spectra and spectral cubes, and for obtaining data from the archive.
>
>
  • If you are working in HIPE, the HIFI Data Reduction Guide will be your prime resource for all things HIFI but you should also look at the Herschel Data Analysis Guide for information about general tools, such as those for viewing and manipulating spectra and spectral cubes, and for obtaining data from the archive.
 

  • The full set of online documentation for the current user release also contains detailed information about the HIFI pipeline, information about scripting in HIPE, and reference manuals for command listings. New users of HIPE are recommended to read through the Quick Start Guide and the HIPE Owner's Guide
Line: 232 to 232
 
  • Information about how to find what calibration version was used on your data, how to get the latest calibration and reprocess observations with it is found in the Pipeline chapter of the HIFI Data reduction Guide
Changed:
<
<
  • the latest calibration tree pool can be also retrieved directly here (22 January 2013)
>
>
  • the latest calibration tree pool can be also retrieved directly here (16 May 2013)
 
  • Updates to the HIFI calibration data are generally concurrent with the release of each major version of the HCSS-HIFI software. However, it is possible to have updates to the calibration data in between major releases of the software as the software and data are independent of each other, in the table below the HIPE or the OD from which the calibration updates apply are given. The calibration versions available since HIPE 5 are listed in the table below
Line: 256 to 255
 
  • Beam efficiencies are derived from Mars maps. Deeper beam maps are currently being taken and analysed, and will result in a possible update of the coupling efficiencies. Those will be posted together with 2D beam map for each of the HIFI bands, which could be of particular interest for the interpretation of extended emission
Changed:
<
<

Outstanding calibration issues

>
>

Outstanding calibration issues

 

  • Standing waves:
    • there are several standing waves that can affect the HIFI data at various processing levels. Their nature and impact on the HIFI calibration are described in the Standing Wave technical note and, more briefly, in the Standing Wave removal chapter of the HIFI Data Reduction Guide
Changed:
<
<
>
>
 
    • There are various techniques currently offered to clean these baseline distortions:
      • Remove sine waves, or combinations of sine waves. Check the bullet about usage of the fitHifiFringe task in the data reduction section above.
Line: 302 to 301
 
Changed:
<
<		-- DavidTeyssier and AnthonyMarston - 10 May 2013
>
>		-- DavidTeyssier , CarolynMcCoey and SylvieFBeaulieu - 16 May 2013
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521151" name="freq_vel_3.PDF" path="freq_vel_3.PDF" size="312508" user="Main.DavidTeyssier" version="1"

Revision 502013-05-10 - AnthonyMarston

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 249 to 249
 
9_0 24-07-12   9.0 Sideband ratios in bands 5a and 5b, update of a number of engineering threshold for more accurate quality flagging related to hardware housekeeping, introduction of a list of a priori known corrupted data-frames for dedicated flagging in level0 data
10_0 08-11-12   9.1 Introduction of a new HifiUplink product to back-fill most the observational parameters as of HSpot optimisation back into the Uplink product, update of the list of corrupted data-frames for flagging at Level 0, update of a number of engineering threshold for more accurate quality flagging related to hardware housekeeping
11_0 22-01-13   10.0 Update of the list of corrupted data-frames for flagging at Level 0, Quality flags created for SEUs (Singe Event Upsets leading to on-board software corruption)
Added:
>
>
12_0 10-05-13   10.1 Update of the list of corrupted data-frames for flagging at Level 0, updates to quality flag meta-data and uplink information for mapping modes
 
  • The side-band ratio assumed in the pipeline as of the latest calibration tree pool are illustrated in this plot.
Line: 300 to 302
 
Changed:
<
<		-- DavidTeyssier and CarolynMcCoey - 31 January 2013
>
>		-- DavidTeyssier and AnthonyMarston - 10 May 2013
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307610505" name="Level1PipelineAlgo_hc_filtered_v1.py.txt" path="Level1PipelineAlgo_hc_filtered_v1.py.txt" size="10864" user="Main.DavidTeyssier" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1307521151" name="freq_vel_3.PDF" path="freq_vel_3.PDF" size="312508" user="Main.DavidTeyssier" version="1"

Revision 492013-03-01 - BrunoMerin

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

HIFI instrument and calibration web pages

Line: 254 to 254