Difference: HipeWhatsNew8x (61 vs. 62)

Revision 622011-12-02 - EdwardPolehampton

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 426 to 426
 

Calibration Products

Changed:
<
<
  • The Spectrometer Beam Parameters calibration product used by the Point Source Flux Conversion now contains the wavenumber-dependent point-source flux conversion factors for each spectrometer detector. Previously, this product contained factors only for each spectrometer array.
>
>
  • Photometer:
    • SCalPhotChanRelGain is a new product to improve extended emission calibration.
    • The SCalPhotChanNoise product has been updated to improve the flux calibration for MadMap (this is only used for MadMap).
  • Spectrometer:
    • The "extended" calibration in SCalSpecTeleRsrf is now derived from a mean of many observations, reducing the noise level in the RSRF. In addition, RSRFs have been added for the Bright Source mode, and the units have been updated to be in GHz.
    • The "point-source" calibration in _SCalSpecBeamParam! has been updated, Bright Source mode factors added and units updated to GHz. The internal structure of the product has been updated to allow the point-source conversion factors for other off-axis detectors to be added in future, and the "beam area" column has been removed (as this was misleading, and the beam area is better calculated from the diameter, assuming a Gaussian shape).
    • The Instrument RSRF in SCalSpecInstRsrf has been updated, Bright Source mode RSRFs added and units updated to GHz
    • The OPD limits in SCalSpecOpdLimits have been changed such that the interferograms for calibration observations (resolution CR) are cut to have the same limits as high resolution science observations (HR).
    • The spectral band edges in SCalSpecBandEdges have been made slightly narrower to avoid spikes at the edges. They have also been converted to GHz.
    • The spectral limits used for phase correction (used for the polynomial fit method) in !SCalSpecPhaseCorrLim have been converted to GHz.
    • Three new calibration products have been added to allow processing of Bright Source mode data:
      • SCalSpecNomPcal contains the weighted mean and standard deviation PCAL level separation for dark sky in Nominal Mode - this is used to compare with the PCAL level separation in Bright Source mode in order to calculate a bright mode gain factor for each detector.
      • SCalSpecBrightGain contains a frequency dependent correction to the PCAL gain factor (the gradient and intercept of a straight line). It is applied in the spectral domain to correct the spectral shape of Bright Mode observations.
      • SCalSpecBolPhase contains the optimum bolometer phases for each detector. These are used in the engineering conversion to correct the voltages for Bright Source mode, where the bolometers are dephased to reduce their responsivity.
 
Deleted:
<
<
  • The Spectrometer Nominal PCAL calibration product contains the weighted mean and standard deviation PCAL level separation for dark sky in Nominal Mode. This calibration product is used by the new PCAL Gain module.

  • The Spectrometer Bright Mode Gain calibration product contains the intercept and gradient for the wavenumber-dependent gain. This calibration product is used by the new Bright Mode Gain module.

 

Calibration Framework

Preprocessing

Engineering Conversion

Added:
>
>
An additional option to correct the bolometer voltages for phase gain has been added. This is used in the Spectrometer pipeline to process Bright Source mode observations.
 

Common Pipeline

The default value for reconstruction in the sigmaKappa deglitcher is changed to "DIRECT".
Added:
>
>
A new metadata item, calVersion, is added to all pipeline produced products to specity the calibration tree version used.
 

Photometer Pipeline

Extended Emission Calibration Using the Relative Detector Gains

Line: 457 to 466
 

Spectrometer Pipeline

Changed:
<
<

Averaging Spectra

The Average Spectra module no longer supports the bandEdge parameter. This parameter was used only for the module's defunct outlier rejection functionality, and so it is no longer needed. Please ensure that your scripts do not use this parameter.

>
>

Spectral Unit and Unit-Flexibility

 
Changed:
<
<

Spectrum Flux Calibration

>
>
The standard spectral unit used in the pipeline is now frequency in GHz rather than wavenumber in cm-1. In addition the band limits have been slightly reduced so that instrumental spikes at the edges are not included.
 
Changed:
<
<
  • The Flux Calibration module has been replaced by two modules, the Point Source Flux Calibration module and the Extended Source Flux Calibration module. References to SpecFluxConversionTask should be replaced with SpecPointFluxConversionTask or SpecExtendedFluxConversionTask. Read more in the pipeline specification manual at Spectrum Point Source Flux Calibration and Spectrum Extended Source Flux Calibration.
>
>
For interactive processing, it is now possible to specify the desired spectral unit in the Fourier Transform module. This accepts an optional spectralUnit string parameter to specify the output spectral unit. For example, if spectralUnit='GHz', then the output spectra will be in units of V/GHz versus GHz. If no unit is specified, the output spectra will be in units of V/(1/OPD_UNIT) versus 1/OPD_UNIT (the OPD unit is typically cm).
 
Changed:
<
<
  • The Point Source Flux Calibration module can now be applied to more than just the central detectors.
>
>
To accommodate this new feature, all spectral modules now accept spectra and calibration data products in any unit of wavenumber or frequency. For example, if a module is provided with spectra in units of V/GHz versus GHz and a calibration product in units of cm-1, then the cm-1 calibration product will be converted to GHz, and the output spectra will be in GHz.
 
Changed:
<
<

Spectral Unit and Unit-Flexibility

>
>

Averaging Spectra

 
Changed:
<
<
It is now possible to specify the spectral unit of the initial spectra. The Fourier Transform module now accepts an optional spectralUnit string parameter to specify the desired output spectral unit. For example, if spectralUnit='GHz', then the output spectra will be in units of V/GHz versus GHz. If no unit is specified, the output spectra will be in units of V/(1/OPD_UNIT) versus 1/OPD_UNIT (the OPD unit is typically cm).
>
>
The Average Spectra module no longer supports the bandEdge parameter. This parameter was used only for the module's defunct outlier rejection functionality, and so it is no longer needed. Please ensure that your scripts do not use this parameter.
 
Changed:
<
<
To accommodate this new feature, all spectral modules now accept spectra and calibration data products in any unit of wavenumber or frequency. For example, if a module is provided with spectra in units of V/GHz versus GHz and a calibration product in units of cm-1, then the cm-1 calibration product will be converted to GHz, and the output spectra will be in GHz.
>
>

Spectrum Flux Calibration

The Flux Calibration module has been replaced by two modules, the Point Source Flux Calibration module and the Extended Source Flux Calibration module. References to SpecFluxConversionTask should be replaced with SpecPointFluxConversionTask or SpecExtendedFluxConversionTask. Read more in the pipeline specification manual at Spectrum Point Source Flux Calibration and Spectrum Extended Source Flux Calibration.
 

Bright Source Mode

Changed:
<
<
Two modules have been created for processing bright source mode observations. Both of these modules are in the new herschel.spire.ia.pipeline.spec.gain package:
>
>
The pipeline has been adapted so that observations made using Bright Source mode can be calibrated. The processing sequence has several different steps from the Nominal mode: the bolometer voltages are corrected for a phase gain in the engineering conversion, they are converted to bolometer temperature, and two additional modules to correct for the remaining gain factors (due to the reduced responsivity in bright mode) are added:
 
  • The PCAL Gain module computes the ratio of a measured PCAL flash level separation to the mean PCAL flash separation measured on Dark Sky in Nominal Mode.
Changed:
<
<
  • The Bright Mode Gain module computes and then applies a wavenumber-dependent gain factor for observations which have been made using the Bright Source Mode.
>
>
  • The Bright Mode Gain module computes and then applies a frequency-dependent gain factor.
 

Phase Correction

Line: 502 to 509
 
sdi = inverseFourierTransform(sds = sds)
  • InverseFourierTransformTask is not the inverse of the FourierTransformTask because it does not remove zeropadding from the output interferograms.
Added:
>
>

Improved pre-processed cube product

The "preCube" product which contains the collection of spectra to be gridded into the final cube has been improved and now contains a dataset in Spectrum2d format, that can easily be viewed with Herschel viewers.
 

Convenience task for spectral cube creation

Changed:
<
<
  • SpireProjectionTask has been created to provide a convenient single API for creating spectral cubes.
  • The task can make spectral cubes using any of the following algorithms:
>
>
SpireProjectionTask has been created to provide a convenient single API for creating spectral cubes. The task can make spectral cubes using any of the following algorithms:
 
    • Naive projection
    • Nearest neighbour projection
    • Gridding projection

Interactive Analysis and Tools

Changed:
<
<

Photometer Destriper

>
>

Useful Scripts

A new menu item, Scripts has been added on the HIPE toolbar to contain "useful" scripts. Scripts have been added for the Photometer and Spectrometer (see below).

Photometer Destriper
 THe Destriper example use is now contained in a new Useful Script accessed from the top Scripts menu within HIPE The following new features are introduced in Hipe 8:
  • Level 2 deglitching is available through setting l2DeglitchRepeat parameter.
Line: 529 to 541
 
  • The input parameter name "Tod" changed to "tod".
  • The input parameter name "L2DeglitchRepeat" changed to "l2DeglitchRepeat".
Changed:
<
<

Photometer Baseline Removal

>
>
Photometer Baseline Removal
 THe Median and Polynomial Baseline Removal example use is now contained in a new Useful Script accessed from the top Scripts menu within HIPE
  • New "roi" (region of interest) parameter for baselineRemoval* tasks, specifying the region of the sky to be used in estimating the baseline
  • Removed "doInterpolation" task parameter
  • Minor changes and bug fixes (for example, the tasks now do ignore masked data samples when a TodBuffer is used internally)
Changed:
<
<

Photometer Moving Object Correction Script

>
>
Photometer Moving Object Correction Script
 A new Useful Script accessed from the top Scripts menu within HIPE:
  • Takes a SSO observation and corrects the map to be in the moving object frame
  • i.e. target is no longer a streak on a stable background.
Changed:
<
<

Photometer SPIRE Astrometry Alignment

>
>
Photometer SPIRE Astrometry Alignment
 A new Useful Script accessed from the top Scripts menu within HIPE:
  • Takes a reference SPIRE map and a SPIRE map to be corrected and aligns the astrometry of the latter to the former
  • Takes a reference SPIRE map and a list of SPIRE maps to be corrected and aligns the astrometry of the latter to the former
Line: 548 to 560
 
  • Takes a referencesource list (as a sourceListProduct) and a SPIRE map to be corrected and aligns the astrometry of the latter to the former
  • In all cases both the Level 2 map astrometry and Level 1 timelines are corrected
Added:
>
>
Spectrometer Array Footprint Plot
A new Useful Script accessed from the top Scripts menu within HIPE:
  • Plots the locations of the detectors on top of an image of the source.

Spectrometer Background Subtraction
A new Useful Script accessed from the top Scripts menu within HIPE:
  • Provides two methods to subtract residual background from the final spectrum:
    • using the daily dark observation
    • using off-axis detectors
This is particularly useful for faint sources.

Spectrometer Line Fitting
A new Useful Script accessed from the top Scripts menu within HIPE:
  • Allows simultaneous fitting of lines and continuum in the central detectors using the HIPE SpectrumFitter
  • CO lines are fitted with a polynomial baseline and results are printed for each line

Spectrometer Thumbnail Mosaic Plot
A new Useful Script accessed from the top Scripts menu within HIPE:
  • Gives several examples of plotting the spectra from each detector as a thumbnail arranged as the detectors appear in the array.

SDI/SDS Explorer

The SDI and SDS Explorer tools have been updated to:
  • Ability to plot errors as a shaded region on the spectrum added
  • Right click options added for each detector in the array footprint, including Open With, Send To and Create Variable.
  • Zooming is now permitted in the thumbnail view.
 

Documentation

Changed:
<
<
  • New SPIRE Launch Pads to allow Users an overview of the SPIRE processing, whats available and quick link jumps into the documentation.
>
>
  • New SPIRE Launch Pads to give Users an overview of the SPIRE processing, whats available, and quick link jumps into the documentation.
 
  • A major revision of the SPIRE Data Reduction Guide for both Photometer and Spectrometer Chapters.
  • SPIRE Data Reduction Guide restructured to be "recipe" orientated
Changed:
<
<
  • Includes description of new Useful Scripts accessed from the Scripts menu in HIPE 8
>
>
  • Includes description of new Useful Scripts accessed from the Scripts menu in HIPE
 
  • SPIRE Data Reduction Guide includes the manual for mapmaking with the SPIRE Photometer Interactive Analysis (SPIA) plug-in
 
This site is powered by the TWiki collaboration platform Powered by Perl