Difference: HipeWhatsNew8x (1 vs. 96)

Revision 962012-10-30 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in HIPE 8

Deleted:
<
<
For changes in the latest version (HIPE 9.0) see this page.
 
PDF Version Portrait Landscape
Added:
>
>
Help Interested in what's new for other versions of HIPE? See this page for links to all the What's New documents from HIPE 4.0 onwards.
 This document lists the changes in HIPE version 8.0 with respect to the 7.x series. Additional pages list changes between minor versions of the 8.x series.

Line: 667 to 667
  -->
Deleted:
<
<
META FILEATTACHMENT attr="h" autoattached="1" comment="Script for Google Analytics" date="1350656537" name="GoogleAnalytics.txt" path="GoogleAnalytics.txt" size="549" user="Main.DavideRizzo" version="1"
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1322831877" name="Screenshot-HIPE_-_historyExtract.png" path="Screenshot-HIPE_-_historyExtract.png" size="39416" user="Main.JeroenDeJong" version="1"
Added:
>
>
META FILEATTACHMENT attr="h" autoattached="1" comment="Script for Google Analytics" date="1350656537" name="GoogleAnalytics.txt" path="GoogleAnalytics.txt" size="549" user="Main.DavideRizzo" version="1"

Revision 952012-10-19 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
Added:
>
>
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->
 

What's new in HIPE 8

For changes in the latest version (HIPE 9.0) see this page.

Line: 664 to 667
  -->
Changed:
<
<
META FILEATTACHMENT attachment="Screenshot-HIPE_-_historyExtract.png" attr="" comment="" date="1322831874" name="Screenshot-HIPE_-_historyExtract.png" path="Screenshot-HIPE - historyExtract.png" size="39416" stream="Screenshot-HIPE - historyExtract.png" user="Main.JeroenDeJong" version="1"
>
>
META FILEATTACHMENT attr="h" autoattached="1" comment="Script for Google Analytics" date="1350656537" name="GoogleAnalytics.txt" path="GoogleAnalytics.txt" size="549" user="Main.DavideRizzo" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1322831877" name="Screenshot-HIPE_-_historyExtract.png" path="Screenshot-HIPE_-_historyExtract.png" size="39416" user="Main.JeroenDeJong" version="1"

Revision 942012-07-24 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Added:
>
>
For changes in the latest version (HIPE 9.0) see this page.
 
PDF Version Portrait Landscape

This document lists the changes in HIPE version 8.0 with respect to the 7.x series. Additional pages list changes between minor versions of the 8.x series.

Revision 932012-07-16 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Line: 37 to 37
  PACS
Changed:
<
<
  • PACS Spectroscopy: Several new or improved tasks: flux calibration now includes the calibration block; errors are calculated at the cube level; better transients correction for unchopped observations; improved the flatfielding tasks (line scan and range scan separately).
  • New pipeline scripts: To combine or bulk processes data, to compare off-source (sky) spectra to the on-source spectra, and to experiment with an alternative spectral response method.
>
>
  • PACS Spectroscopy: Several new and improved tasks: flux calibration now includes the calibration block; errors are calculated at the cube level; better transient correction for unchopped observations; improved the flatfielding tasks (line scan and range scan separately).
  • New Spectroscopy pipeline scripts: To combine or bulk process data; to compare off-source (sky) spectra to the on-source spectra; and to experiment with an alternative spectral response method.
 
  • PACS Photometry: Second-level deglitching more efficient; a companion task to photProject, which is much faster; new cross-talk correction task; updated calibration file for non-linearity.
Changed:
<
<
  • Reprocess PACS photometry with HIPE 8: Only if you have a very bright target of which you want to do the photometry with the non-linearity correction and the crosstalk artifacts removal.
>
>
  • Reprocess PACS photometry with HIPE 8: Only if you have a very bright target of which you want to do the photometry with the non-linearity correction and the crosstalk artefact removal.
  SPIRE

Revision 922012-07-03 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Line: 9 to 9
 
Added:
>
>
  See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.

Revision 912012-05-14 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Line: 640 to 640
 
  • Includes description of new scripts accessed from the Scripts menu in HIPE.
  • SPIRE Data Reduction Guide includes the manual for mapmaking with the SPIRE Photometer Interactive Analysis (SPIA) plug-in.
Changed:
<
<
>
>

<-- COMMENT BOX CODE - DO NOT EDIT -->
blog comments powered by Disqus
<-- END OF COMMENT BOX CODE -->

 

Revision 902012-04-10 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Line: 8 to 8
 
Changed:
<
<
<--   * Changes between HIPE 8.1 and 8.2 -->
>
>
  See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.

Revision 892012-03-23 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Line: 8 to 8
 
Added:
>
>
<--   * Changes between HIPE 8.1 and 8.2 -->
  See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.

Revision 882012-02-15 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Line: 7 to 7
 This document lists the changes in HIPE version 8.0 with respect to the 7.x series. Additional pages list changes between minor versions of the 8.x series.

Added:
>
>
  See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.

Revision 872012-01-24 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Line: 589 to 589
 
Photometer Baseline Removal
The Median and Polynomial Baseline Removal example use is now contained in a new script accessed from the top Scripts menu within HIPE.
  • New roi (region of interest) parameter for baselineRemoval* tasks, specifying the region of the sky to be used in estimating the baseline.
Changed:
<
<
  • Removed doInterpolation task parameter.
>
>
  • Removed doInterpolation task parameter. The reason is that this parameter is of such limited use that it is best left as a parameter in the madMapper task.
 
  • Minor changes and bug fixes (for example, the tasks now do ignore masked data samples when a TodBuffer is used internally).

Photometer Moving Object Correction Script

Revision 862012-01-20 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Line: 308 to 308
 

Level 2 Pipeline

  • Sideband gains coefficients added to metadata.
Changed:
<
<
  • doGridding now uses AOT parameters in the meta data (such as beam size, map size, and - for observations after OD 835 - readout and line separations) to create cubes that reflect how the observation was carried out. In addition a box filter (previously a Guassian) is automatically used for greater than Nyquist sampled DBS Raster maps.
>
>
  • doGridding now uses AOT parameters in the meta data (such as beam size, map size, and - for observations after OD 835 - readout and line separations) to create cubes that reflect how the observation was carried out. In addition a box filter (previously a Gaussian) is automatically used for greater than Nyquist sampled DBS Raster maps.
  Note that for the needed meta data to be present, the observation must be pipelined with HIPE 8 after the level 0 pipeline step doUplink has been run on the HTP.
Line: 369 to 369
 
  • GUI behaviour modified:
    • Picks up information from metadata to provide better default input parameters.
Changed:
<
<
    • Changes to values in the GUI form now only go in one direction, e.g. changing PixelSize changes the Beam and Filter Parameters but not vice versa.
>
>
    • Changes to values in the GUI form now only go in one direction, e.g. changing the PixelSize or the BeamSize results in an update of the Filter Parameters but not vice versa.
 

HiClass export tool

Revision 852012-01-04 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Line: 28 to 28
 HIFI

  • Improved spectral cubes: Spectral cubes from mapping observations after OD 835 gridded using actual scan line and readout used in observation, earlier maps use a better approximation.
Changed:
<
<
  • Improved band 6 and 7 data: Improvements in the level 1 pipeline give noticeably improved noise and baseline quality of all observations performed in bands 6 and 7.
>
>
  • Improved band 6 and 7 Spectral Scan data: Improvements in the level 1 pipeline give noticeably improved noise and baseline quality of Spectral Scan observations performed in bands 6 and 7.
 
  • Standing waves: For strong continuum sources an optional pipeline step, doFilterLoads, can reduce or eliminate standing waves from the hot and cold loads
  • Calibration version: The calibration version used by the pipeline is now included in the metadata of all calibrated products. You can also select the calibration version to be used by the pipeline (command line only).
Line: 303 to 303
 

Level 1 Pipeline

  • New (optional) pipeline step allowing smooth band pass to eliminate standing waves: doFilterLoads. This is an alternative calibration scheme for strong continuum sources.

Changed:
<
<
  • Substantial refactoring of CheckFrequencyGrid to correct assumptions made about the sequence of on-, off- and hc-datasets. This primarily affected HEB band data and resulted in unnecessary Quality Flags being raised. This correction produces a noticeable improvement in the noise and baseline quality of all observations performed in bands 6 and 7.
>
>
  • Substantial refactoring of CheckFrequencyGrid to correct assumptions made about the sequence of on-, off- and hc-datasets. This primarily affected HEB band data and resulted in unnecessary Quality Flags being raised. This correction produces a noticeable improvement in the noise and baseline quality of observations performed in bands 6 and 7.
 

Level 2 Pipeline

  • Sideband gains coefficients added to metadata.
Changed:
<
<
  • doGridding now uses AOT parameters in the meta data (such as beam size, map size, and - for observations after OD 835 - readout and line separations) to create cubes that reflect how the observation was carried out.
>
>
  • doGridding now uses AOT parameters in the meta data (such as beam size, map size, and - for observations after OD 835 - readout and line separations) to create cubes that reflect how the observation was carried out. In addition a box filter (previously a Guassian) is automatically used for greater than Nyquist sampled DBS Raster maps.
  Note that for the needed meta data to be present, the observation must be pipelined with HIPE 8 after the level 0 pipeline step doUplink has been run on the HTP.

Maps older than OD 835 will not have the readout and line spacings in the the meta data (this will happen with HIPE 9) instead doGridding calculates an approximate pixel size based on other meta data.

Deleted:
<
<
Other changes include:
    • the GUI now modifies filter parameters in connection with changes to pixel size entered in the GUI
    • introduction of a flyAngle parameter to allow the easy creation of rotated maps
    • a box filter is automatically used for greater than Nyquist sampled DBS Raster maps
 

HIFI Products

Revision 842012-01-04 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Line: 22 to 22
 

Highlights

Added:
>
>
HIPE
  • User pool names (pool ID) restrictions: Allowed characters in name: alphanumeric, dash ("-") and underscore ("_"). The pool ID is case-sensitive, must be with less than 255 characters, and "urn" is not allowed as pool ID, and finally the pool ID cannot start with a dash ("-"). Old pools with names that violate those restrictions need to be renamed (at OS level) in order to be used in HIPE.
 HIFI

  • Improved spectral cubes: Spectral cubes from mapping observations after OD 835 gridded using actual scan line and readout used in observation, earlier maps use a better approximation.

Revision 832011-12-19 - BrunoMerin

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Line: 42 to 42
 
  • Calibration files: Improvements in a number of Spectrometer calibration file
  • FTS spectra: The representation of FTS spectra is now in terms of Frequency in GHz and not in wavenumbers in cm-1.
  • SPIRE Photometry of extended emission: improved destriper; new module and calibration file to correct for relative gains correction for extended emission
Added:
>
>
  • Astrometry: new astrometry correction module. Given a reference image, it will correct the whole observation conext (level 1 and level 2)
 

Core system

Revision 822011-12-19 - LucaConversi

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Line: 38 to 38
  SPIRE
Changed:
<
<
  • Bright sources: Big improvements in the processing of bright source mode observations with the SPIRE Spectrometer.
  • Calibration files: Improvements in a number of Spectrometer calibration files.
>
>
  • SPIRE Spectroscopy of bright sources: Big improvements in the processing of bright source mode observations with the SPIRE Spectrometer.
  • Calibration files: Improvements in a number of Spectrometer calibration file
 
  • FTS spectra: The representation of FTS spectra is now in terms of Frequency in GHz and not in wavenumbers in cm-1.
Added:
>
>
  • SPIRE Photometry of extended emission: improved destriper; new module and calibration file to correct for relative gains correction for extended emission
 

Core system

Revision 812011-12-19 - BrunoMerin

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Line: 34 to 34
 
  • PACS Spectroscopy: Several new or improved tasks: flux calibration now includes the calibration block; errors are calculated at the cube level; better transients correction for unchopped observations; improved the flatfielding tasks (line scan and range scan separately).
  • New pipeline scripts: To combine or bulk processes data, to compare off-source (sky) spectra to the on-source spectra, and to experiment with an alternative spectral response method.
  • PACS Photometry: Second-level deglitching more efficient; a companion task to photProject, which is much faster; new cross-talk correction task; updated calibration file for non-linearity.
Changed:
<
<
  • Reprocess with HIPE 8: Only if you have a very bright target of which you want to do the photometry with the non-linearity correction and the crosstalk artifacts removal.
>
>
  • Reprocess PACS photometry with HIPE 8: Only if you have a very bright target of which you want to do the photometry with the non-linearity correction and the crosstalk artifacts removal.
  SPIRE

Revision 802011-12-14 - RolandVavrek

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Line: 445 to 445
 

Spectroscopy

  • There is a new GUI for viewing the footprint of the spectrometer upon an image with WCS: the tool pacsSpectralFootprint. It can be accessed through the Applicable tasks list in the Tasks view.
Added:
>
>
  • A tool specBaselineEstimator has been made available especially for long-range spectrum baseline removal. The wavelet-based multiresolution engine can also provide automatic line identification, line mask and continuum noise products.
 
  • The Spectrum Explorer GUI has a new panel for editing PACS masks.
  • Implementation of user friendly viewer for the PACS specific part of the ObservationContext:
    • Each spectrometer slice now shows after the slice number a short description of its contents.

Revision 782011-12-13 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

PDF Version Portrait Landscape
Added:
>
>
This document lists the changes in HIPE version 8.0 with respect to the 7.x series. Additional pages list changes between minor versions of the 8.x series.

 See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.

Revision 772011-12-13 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Changed:
<
<
PDF Version Click here
>
>
PDF Version Portrait Landscape
  See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.
Added:
>
>

 
Added:
>
>

Highlights

HIFI

  • Improved spectral cubes: Spectral cubes from mapping observations after OD 835 gridded using actual scan line and readout used in observation, earlier maps use a better approximation.
  • Improved band 6 and 7 data: Improvements in the level 1 pipeline give noticeably improved noise and baseline quality of all observations performed in bands 6 and 7.
  • Standing waves: For strong continuum sources an optional pipeline step, doFilterLoads, can reduce or eliminate standing waves from the hot and cold loads
  • Calibration version: The calibration version used by the pipeline is now included in the metadata of all calibrated products. You can also select the calibration version to be used by the pipeline (command line only).

PACS

  • PACS Spectroscopy: Several new or improved tasks: flux calibration now includes the calibration block; errors are calculated at the cube level; better transients correction for unchopped observations; improved the flatfielding tasks (line scan and range scan separately).
  • New pipeline scripts: To combine or bulk processes data, to compare off-source (sky) spectra to the on-source spectra, and to experiment with an alternative spectral response method.
  • PACS Photometry: Second-level deglitching more efficient; a companion task to photProject, which is much faster; new cross-talk correction task; updated calibration file for non-linearity.
  • Reprocess with HIPE 8: Only if you have a very bright target of which you want to do the photometry with the non-linearity correction and the crosstalk artifacts removal.

SPIRE

  • Bright sources: Big improvements in the processing of bright source mode observations with the SPIRE Spectrometer.
  • Calibration files: Improvements in a number of Spectrometer calibration files.
  • FTS spectra: The representation of FTS spectra is now in terms of Frequency in GHz and not in wavenumbers in cm-1.
 

Core system

Jython major upgrade. The version of Jython on which HIPE is based has been upgraded from 2.1 to 2.5.2. You may have to adapt your scripts for them to work with the new version.

Revision 762011-12-12 - TrevorFulton

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Line: 500 to 500
 
  • The Phase Correction module's interface has been simplified. As before, the module provides three ways of applying phase correction, but executing the module is simpler now:
    1. From the observation data:
Changed:
<
<
sdi = phaseCorrection(sdi=sdi, avgSdi=avgSdi, spectralUnit= GHz | cm) 
>
>
sdi = phaseCorrection(sdi=sdi, avgSdi=avgSdi, spectralUnit= GHz | cm-1) 
 
    1. From a calibration file:
      sdi = phaseCorrection(sdi=sdi, nonLinPhase=nonLinPhase)
    2. From a polynomial fit to the observation:

Revision 752011-12-12 - TrevorFulton

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Line: 500 to 500
 
  • The Phase Correction module's interface has been simplified. As before, the module provides three ways of applying phase correction, but executing the module is simpler now:
    1. From the observation data:
Changed:
<
<
sdi = phaseCorrection(sdi=sdi, avgSdi=avgSdi, spectralUnit= 
=GHz= | cm-1)
>
>
sdi = phaseCorrection(sdi=sdi, avgSdi=avgSdi, spectralUnit= GHz | cm) 
 
    1. From a calibration file:
      sdi = phaseCorrection(sdi=sdi, nonLinPhase=nonLinPhase)
    2. From a polynomial fit to the observation:

Revision 742011-12-12 - TrevorFulton

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Line: 500 to 500
 
  • The Phase Correction module's interface has been simplified. As before, the module provides three ways of applying phase correction, but executing the module is simpler now:
    1. From the observation data:
Changed:
<
<
sdi = phaseCorrection(sdi=sdi, avgSdi=avgSdi, spectralUnit= =GHz= | =cm<sup>-1</sup>=)
>
>
sdi = phaseCorrection(sdi=sdi, avgSdi=avgSdi, spectralUnit= 
=GHz= | cm-1)
 
    1. From a calibration file:
      sdi = phaseCorrection(sdi=sdi, nonLinPhase=nonLinPhase)
    2. From a polynomial fit to the observation:

Revision 732011-12-12 - TrevorFulton

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Line: 478 to 478
  The standard spectral unit used in the pipeline is now frequency in GHz rather than wavenumber in cm-1. In addition the band limits have been slightly reduced so that instrumental spikes at the edges are not included.
Changed:
<
<
For interactive processing, it is now possible to specify the desired spectral unit in the Fourier Transform module. This accepts an optional spectralUnit string parameter to specify the output spectral unit. For example, if spectralUnit='GHz', then the output spectra will be in units of V/GHz versus GHz. If no unit is specified, the output spectra will be in units of V/(1/OPD_UNIT) versus 1/OPD_UNIT (the OPD unit is typically cm).
>
>
For interactive processing, it is now possible to specify the desired spectral unit in the Fourier Transform and Phase Correction modules. Both tasks accept an optional spectralUnit string parameter to specify the output spectral unit. For example, if spectralUnit='GHz', then the output spectra will be in units of V/GHz versus GHz (or K/GHz versus GHz for bright mode observations). If no unit is specified, the output spectra will be in units of V/(1/OPD_UNIT) (or K/(1/OPD_UNIT)) versus 1/OPD_UNIT (the OPD unit is typically cm).
  To accommodate this new feature, all spectral modules now accept spectra and calibration data products in any unit of wavenumber or frequency. For example, if a module is provided with spectra in units of V/GHz versus GHz and a calibration product in units of cm-1, then the cm-1 calibration product will be converted to GHz, and the output spectra will be in GHz.
Line: 500 to 500
 
  • The Phase Correction module's interface has been simplified. As before, the module provides three ways of applying phase correction, but executing the module is simpler now:
    1. From the observation data:
Changed:
<
<
sdi = phaseCorrection(sdi=sdi, avgSdi=avgSdi)
>
>
sdi = phaseCorrection(sdi=sdi, avgSdi=avgSdi, spectralUnit= =GHz= | =cm<sup>-1</sup>=)
 
    1. From a calibration file:
      sdi = phaseCorrection(sdi=sdi, nonLinPhase=nonLinPhase)
    2. From a polynomial fit to the observation:

Revision 722011-12-09 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Line: 269 to 269
 

Level 2 Pipeline

  • Sideband gains coefficients added to metadata.
Changed:
<
<
  • doGridding now uses AOT parameters in the meta data (such as beam size, map size, and - for observations after OD 835) to create cubes that reflect how the observation was carried out. In the past gridding was done based only on a linear approximation of the beam size and for half beam or Nyquist spacing.
>
>
  • doGridding now uses AOT parameters in the meta data (such as beam size, map size, and - for observations after OD 835 - readout and line separations) to create cubes that reflect how the observation was carried out.

Note that for the needed meta data to be present, the observation must be pipelined with HIPE 8 after the level 0 pipeline step doUplink has been run on the HTP.

Maps older than OD 835 will not have the readout and line spacings in the the meta data (this will happen with HIPE 9) instead doGridding calculates an approximate pixel size based on other meta data.

  Other changes include:
    • the GUI now modifies filter parameters in connection with changes to pixel size entered in the GUI
    • introduction of a flyAngle parameter to allow the easy creation of rotated maps
Added:
>
>
    • a box filter is automatically used for greater than Nyquist sampled DBS Raster maps
 

HIFI Products

Line: 341 to 346
 
    • Addition of chapter, How to flag and remove flags from your data.
    • Addition of chapter, Unit Conversion.
    • Updates to chapters on Pipeline, Deconvolution, Creating Spectral Cubes, What was done to my data (clarifying units at each stage of the pipeline).
Changed:
<
<
  • Updates the examples in the HIFI Pipeline Specification Document.
>
>
  • Updates to the examples in the HIFI Pipeline Specification Document.
 
  • HIFI User Reference Manual (URM) improvements.
    • All HIFI tasks in URM now have a category and show up in the categorised view of commands.
    • The categorised view of commands now is grouped into headers that should be meaningful to astronomers.

Revision 712011-12-09 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Line: 264 to 264
 

Level 1 Pipeline

  • New (optional) pipeline step allowing smooth band pass to eliminate standing waves: doFilterLoads. This is an alternative calibration scheme for strong continuum sources.

Changed:
<
<
  • Substantial refactoring of CheckFrequencyGrid to correct assumptions made about the sequence of on-, off- and hc-datasets. This primarily affected HEB band data and resulted in unnecessary Quality Flags being raised.
>
>
  • Substantial refactoring of CheckFrequencyGrid to correct assumptions made about the sequence of on-, off- and hc-datasets. This primarily affected HEB band data and resulted in unnecessary Quality Flags being raised. This correction produces a noticeable improvement in the noise and baseline quality of all observations performed in bands 6 and 7.
 

Level 2 Pipeline

  • Sideband gains coefficients added to metadata.
Added:
>
>
  • doGridding now uses AOT parameters in the meta data (such as beam size, map size, and - for observations after OD 835) to create cubes that reflect how the observation was carried out. In the past gridding was done based only on a linear approximation of the beam size and for half beam or Nyquist spacing.

Other changes include:

    • the GUI now modifies filter parameters in connection with changes to pixel size entered in the GUI
    • introduction of a flyAngle parameter to allow the easy creation of rotated maps

 

HIFI Products

  • Meta data calVersion is applied to all calibrated products and has format HIFI_CAL_version_number

  • Updated browse image for point mode observations.
Deleted:
<
<
  • Noticeable improvement in the noise and baseline quality of the level2 products of Spectral Scans performed in bands 6 and 7. This is due to a fix to the procedure that did combine the ON and OFF phases, which was not done optimally in earlier versions of the pipeline.
 

HIFI Spectrum Toolbox

Spectrum tasks (HIFI specifics)

Revision 702011-12-07 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 8

Line: 28 to 28
 
  • Names of variables created from the Product Viewer are more consistent and meaningful.

Dataset Viewer
Changed:
<
<
  • Numeric values in metadata, array datasets or table datasets can be displayed in decimal or hexadecimal format, and dates can be shown in either UTC or TAI. These options are available in the contextual menu for individual cells.
>
>
  • Numeric values in metadata, array datasets or table datasets can be displayed in decimal or hexadecimal format, and dates can be shown in either UTC or TAI. These options are available in the table's contextual menu.
 
Product Viewer
  • The title (product's description) can be selected and copied.

Revision 692011-12-07 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
Changed:
<
<

What's coming up in HIPE 8

>
>

What's new in HIPE 8

 
PDF Version Click here
Line: 72 to 72
 
  • New task astrometryFix: adjusts the absolute pointing of map data to give the best possible agreement in overall astrometry with some ancillary data. This is done either by cross correlation of the map(s) with ancillary map data, or by stacking the map(s) at a list of known source positions (e.g., from radio observations).

Numeric routines

Deleted:
<
<

Data fitting

Data interpolation

Random numbers

Statistics

Trend analysis

Mapping

 

Wavelet transform

  • Continuous Wavelet Transform synthesis operation now adds low frequencies for a reliable reconstruction.
  • Continuous Wavelet Transform is now able to deal with Complex1d signal.
Deleted:
<
<

Other functions

 

Deprecations

Changed:
<
<
A number of features confuse the users and therefore they have been deprecated for a long time . This release will be the last time those features are still accessible.
>
>
A number of potentially confusing features have been deprecated for a long time. This release will be the last time these features are still accessible.
  As of HCSS 9 we will remove those deprecated functions:
deprecated alternative comment
Line: 120 to 113
 

Images

Display

  • The methods addCircle, addEllipse and addRectangle ae no longer dependent on the zoom factor of the image.
Deleted:
<
<

Analysis

 

Spectra

Deleted:
<
<

Display

 

Analysis

  • The SpectrumFitter now recognizes a CUBE such that you can give the x,y position of the spectrum. The script as written by the SpectrumFitterGUI also takes the correct spectrum.
Line: 132 to 123
 
  • Various changes when applying toolbox tasks to cubes, in particular avg, select and extract now allow you to work on several pixels/spaxels within the cube rather than only the entire cube.

  • The resampling task now allows alternative ways to specify the grid for resampling.
Deleted:
<
<

Data cubes

Display

Analysis

 

Herschel Science Archive

Line: 192 to 180
 

Miscellaneous tasks

  • OpenTask parameter names have changed capitalisation to comply with naming conventions.
Deleted:
<
<

Quality control

 

Data input-output

  • Improved entries in the User's Reference Manual.
  • Task importUFDirToPal renamed to importObservation
Line: 275 to 261
 
  • Suppression of unneeded Quality Flags: Quality check for mixer voltage now accounts for magnet tuning.

Deleted:
<
<

WBS Pipeline

HRS Pipeline

 

Level 1 Pipeline

  • New (optional) pipeline step allowing smooth band pass to eliminate standing waves: doFilterLoads. This is an alternative calibration scheme for strong continuum sources.

Line: 293 to 277
 
  • Noticeable improvement in the noise and baseline quality of the level2 products of Spectral Scans performed in bands 6 and 7. This is due to a fix to the procedure that did combine the ON and OFF phases, which was not done optimally in earlier versions of the pipeline.

HIFI Spectrum Toolbox

Deleted:
<
<

Spectrum Explorer (HIFI specifics)

 

Spectrum tasks (HIFI specifics)

  • Introduced the flagTool task, which allows you go through every subband of a spectrum dataset and interactively flag spurs, lines, bright lines, and data to ignore. Input data can be an Observation context, a HifiTimelineProduct, or a SpectrumDataset.
Line: 460 to 443
 

Calibration Framework

Deleted:
<
<

Preprocessing

 

Engineering Conversion

An additional option to correct the bolometer voltages for phase gain has been added. This is used in the Spectrometer pipeline to process Bright Source mode observations.

Revision 682011-12-06 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 365 to 365
 

Photometry

Added:
>
>
  • The HIPE menu Pipeline --> PACS has been reorganised and new interactive pipe scripts added.
  • Automatic pipeline scripts save the products at the end in the Observation Context. That mean the scripts can be executed in a sequence without interaction.
 
  • The less memory intensive task mapDeglitch is now used for deglitching instead of IIndLevelDeglitch
  • The second level glitch mask is now stored in level 1.
  • The new task filterOnScanSpeed now used to select frames at constant speed instead of the BBID selection.
Line: 407 to 408
 
  • The PhotProject task got a companion task miProject (MapIndex-Project). If you have a full MapIndex, you may use miProject instead of photProject. It needs frames and the MapIndex as parameters and speeds up projection time by a factor of at least 20. Use as follows: map = miProject(frames, mapindex)
  • A new task has been developed. It deglitches without MapIndex and runs with significantly less memory abuse than the IIndLevelDeglitch task. The name and package is herschel.pacs.spg.phot.deglitching.map.MapDeglitchTask. The HIPE alias is mapDeglitch. It is documented as javadoc, URM and on the online deglitching documentation page.
  • A new task has been implemented which removes crosstalk effects by masking (CrossTalkMask) the first rows of detector arrays: frames = photMaskCrosstalk(frames)
Deleted:
<
<
  • The HIPE menu Pipeline --> PACS has been reorganised and ipipe scripts added.
  • Automatic pipeline scripts save the products at the end in the Observation Context also in IA sessions. That mean the scripts can be executed in a sequence without interaction.
 
  • Pacs products (for example Frames, Ramps, PacsCube) got a method to rename a Mask: renameMask(String oldname, String newname). Up to now, a mask had to be removed and recreated, if the name had to be changed.
  • photProject, miProject, IIndLevelDeglitching and some other tasks now report their progress in the HIPE progress bar instead of the terminal output.
  • The effects of 2-bit rounding on second level deglitching have been reduced in the pipeline for blue parallel maps.
Line: 434 to 433
 
  • Red and blue camera spectra can now be combined in a single context in the level-2 processing, therefore, the lineId has changed for the red camera: 100 is added!

Documentation

Changed:
<
<
  • PACS Data Reduction Guide. Chapter 1 is now a collection of Launch Pads: on getting Data into HIPE, doing your PACS Spectroscopy and doing your PACS Photometry. These are quick-start guides to work with your PACS observations. Chapter 2 tells you how to load your data, your pipeline script, and your calibration tree into HIPE, for photometry and spectroscopy.
  • Spectroscopy: the pipeline chapters are being re-organised to follow the new pipeline scripts, and so for this 8.0 version of the PDRG only the LineScan Chop-Nod pipeline is described.
  • Photometry: the pipeline chapters are being re-organised to follow the new pipeline scripts, and so for this 8.0 version of the PDRG only the ScanMap pipeline scripts are described.
>
>
  • PACS Data Reduction Guide. Chapter 1 is now a collection of Launch Pads: on getting Data into HIPE, doing your PACS Spectroscopy and doing your PACS Photometry. These are quick-start guides to work with your PACS observations. Chapter 2 tells you how to load your data, your pipeline script, and your calibration tree into HIPE, for photometry and spectroscopy. An appendix describes the PACS products in detail and some PACS product viewers.
  • Spectroscopy: the pipeline chapters have been re-organised to follow the new pipeline scripts.
  • Photometry: the pipeline chapters have been re-organised to follow the new pipeline scripts.
 

SPIRE

Revision 672011-12-05 - JorgoBakker

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 82 to 82
 
  • Continuous Wavelet Transform synthesis operation now adds low frequencies for a reliable reconstruction.
  • Continuous Wavelet Transform is now able to deal with Complex1d signal.

Other functions

Added:
>
>

Deprecations

A number of features confuse the users and therefore they have been deprecated for a long time . This release will be the last time those features are still accessible.

As of HCSS 9 we will remove those deprecated functions:

deprecated alternative comment
Uniq.IS_UNIQ Uniq.COUNT_UNIQ_VALUES
UniqSorted.IS_UNIQ UniqSorted.COUNT_UNIQ_VALUES
Normalize.MEAN_FIDUTIAL Normalize.MEAN_FIDUCIAL
Normalize.PEAK_FIDUTIAL Normalize.PEAK_FIDUCIAL
RMS QRMS use as is explained in the User's Reference Manual
POLYNOMIAL Polynomial use as is explained in the User's Reference Manual
RESHAPE Reshape use as is explained in the User's Reference Manual
SHIFT Shift use as is explained in the User's Reference Manual
LOGN LogN use as is explained in the User's Reference Manual
EXPN ExpN use as is explained in the User's Reference Manual
POW Pow use as is explained in the User's Reference Manual
ANY_PRESENT AnyPresent use as is explained in the User's Reference Manual
ALL_PRESENT AllPresent use as is explained in the User's Reference Manual
NOT_PRESENT NotPresent use as is explained in the User's Reference Manual
WeightedMean.error WeightedMean.errorDistr
 

Source extraction

Revision 662011-12-05 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 425 to 425
 
    • The SCalPhotChanNoise product has been updated to improve the flux calibration for MadMap (this is only used for MadMap).
  • Spectrometer:
    • The "extended" calibration in SCalSpecTeleRsrf is now derived from a mean of many observations, reducing the noise level in the RSRF. In addition, RSRFs have been added for the Bright Source mode, and the units have been updated to be in GHz.
Changed:
<
<
    • The "point-source" calibration in _SCalSpecBeamParam! has been updated, Bright Source mode factors added and units updated to GHz. The internal structure of the product has been updated to allow the point-source conversion factors for other off-axis detectors to be added in future, and the "beam area" column has been removed (as this was misleading, and the beam area is better calculated from the diameter, assuming a Gaussian shape).
>
>
    • The "point-source" calibration in SCalSpecBeamParam has been updated, Bright Source mode factors added and units updated to GHz. The internal structure of the product has been updated to allow the point-source conversion factors for other off-axis detectors to be added in future, and the "beam area" column has been removed (as this was misleading, and the beam area is better calculated from the diameter, assuming a Gaussian shape).
 
    • The Instrument RSRF in SCalSpecInstRsrf has been updated, Bright Source mode RSRFs added and units updated to GHz
    • The OPD limits in SCalSpecOpdLimits have been changed such that the interferograms for calibration observations (resolution CR) are cut to have the same limits as high resolution science observations (HR).
    • The spectral band edges in SCalSpecBandEdges have been made slightly narrower to avoid spikes at the edges. They have also been converted to GHz.
Line: 451 to 451
 

Photometer Pipeline

Extended Emission Calibration Using the Relative Detector Gains

Changed:
<
<
A new optional module in the User Pipelines that takes into account the fact that the SPIRE detectors do not all have exactly the same beamshape
  • The Flag applyExtendedEmissionGains needs to be set to True
>
>
A new optional module in the User Pipelines that takes into account the fact that the SPIRE detectors do not all have exactly the same beamshape.
  • The Flag applyExtendedEmissionGains needs to be set to True.
 

Include Turnaround Data for Maps

Changed:
<
<
Extra area / coverage can be obtained by including the scan line turnarounds in the processing
  • The Flag includeTurnaround needs to be set to True
>
>
Extra area / coverage can be obtained by including the scan line turnarounds in the processing.
  • The Flag includeTurnaround needs to be set to True.
 

Spectrometer Pipeline

Spectral Unit and Unit-Flexibility

Changed:
<
<
The standard spectral unit used in the pipeline is now frequency in GHz rather than wavenumber in cm-1. In addition the band limits have been slightly reduced so that instrumental spikes at the edges are not included.
>
>
The standard spectral unit used in the pipeline is now frequency in GHz rather than wavenumber in cm-1. In addition the band limits have been slightly reduced so that instrumental spikes at the edges are not included.
  For interactive processing, it is now possible to specify the desired spectral unit in the Fourier Transform module. This accepts an optional spectralUnit string parameter to specify the output spectral unit. For example, if spectralUnit='GHz', then the output spectra will be in units of V/GHz versus GHz. If no unit is specified, the output spectra will be in units of V/(1/OPD_UNIT) versus 1/OPD_UNIT (the OPD unit is typically cm).
Line: 473 to 473
 The Average Spectra module no longer supports the bandEdge parameter. This parameter was used only for the module's defunct outlier rejection functionality, and so it is no longer needed. Please ensure that your scripts do not use this parameter.

Spectrum Flux Calibration

Changed:
<
<
The Flux Calibration module has been replaced by two modules, the Point Source Flux Calibration module and the Extended Source Flux Calibration module. References to SpecFluxConversionTask should be replaced with SpecPointFluxConversionTask or SpecExtendedFluxConversionTask. Read more in the pipeline specification manual at Spectrum Point Source Flux Calibration and Spectrum Extended Source Flux Calibration.
>
>
The Flux Calibration module has been replaced by two modules, the Point Source Flux Calibration module and the Extended Source Flux Calibration module. References to SpecFluxConversionTask should be replaced with SpecPointFluxConversionTask or SpecExtendedFluxConversionTask. Read more in the pipeline specification manual at Spectrum Point Source Flux Calibration and Spectrum Extended Source Flux Calibration.
 

Bright Source Mode

Line: 499 to 499
 
    • residualPhaseSLW/SSW gives the maximum absolute phase value (in units of degree) inside the phase correction limits in any of the nominal SLW/SSW detectors.

Create SDI from SDS

Changed:
<
<
  • InverseFourierTransformTask has been created to transform SDS objects into SDI objects by applying the inverse Fourier transform:
>
>
  • inverseFourierTransform has been created to transform SDS objects into SDI objects by applying the inverse Fourier transform:
 
sdi = inverseFourierTransform(sds = sds)
Changed:
<
<
  • InverseFourierTransformTask is not the inverse of the FourierTransformTask because it does not remove zeropadding from the output interferograms. The already existing makeSameOpds can be used to remove zero-padding.
>
>
  • inverseFourierTransform is not the inverse of the fourierTransform task because it does not remove zeropadding from the output interferograms. The already existing makeSameOpds can be used to remove zero-padding.
 

Improved pre-processed cube product

Changed:
<
<
The "preCube" product which contains the collection of spectra to be gridded into the final cube has been improved and now contains a dataset in Spectrum2d format, that can easily be viewed with Herschel viewers.
>
>
The "preCube" product which contains the collection of spectra to be gridded into the final cube has been improved and now contains a dataset in Spectrum2d format, that can easily be viewed with Herschel viewers.
 

Convenience task for spectral cube creation

SpireProjectionTask has been created to provide a convenient single API for creating spectral cubes. The task can make spectral cubes using any of the following algorithms:
Changed:
<
<
  • Naive projection (cf. SPIRE photometer)
  • Nearest neighbour projection
  • Gridding projection (cf. HIFI OTF gridding)
>
>
  • Naive projection (cf. SPIRE photometer).
  • Nearest neighbour projection.
  • Gridding projection (cf. HIFI OTF gridding).
 

Interactive Analysis and Tools

Useful Scripts

Changed:
<
<
A new menu item, Scripts has been added on the HIPE toolbar to contain "useful" scripts. Scripts have been added for the Photometer and Spectrometer (see below).
>
>
A new menu item, Scripts has been added to HIPE to contain "useful" scripts. Scripts have been added for the Photometer and Spectrometer (see below).
 
Photometer Destriper
Changed:
<
<
THe Destriper example use is now contained in a new Useful Script accessed from the top Scripts menu within HIPE The following new features are introduced in Hipe 8:
>
>
THe Destriper example use is now contained in a new Useful Script accessed from the top Scripts menu within HIPE. The following new features are introduced in HIPE 8:
 
  • Level 2 deglitching is available through setting l2DeglitchRepeat parameter.
  • The 5 individual output parameters are combined to one ArrayList parameter where each of 5 parameter can be retrieved base on its position.
  • The new diagnostic product is introduced which provides with useful diagnostic information.
Changed:
<
<
  • Both Tod and TodBuffer are acceptable input for Tod parameter.
  • The input parameter "array" now is MANDATORY.
  • Pixel size can be changed through setting pixelSize parameter.
  • brightSource parameter is changed to boolean.
  • A boolean parameter, "fitWithWeight" is introduced.
  • Interruption is added so that the destriper can be terminated at the user's request.
  • Tod files stored in temporary directory are available until Hipe is terminated.
  • The destriper speed is increased 30%.
  • The input parameter name "Level1" changed to "level1".
  • The input parameter name "Tod" changed to "tod".
  • The input parameter name "L2DeglitchRepeat" changed to "l2DeglitchRepeat".
>
>
  • Both Tod and TodBuffer are acceptable input for the tod parameter.
  • The input parameter array now is MANDATORY.
  • Pixel size can be changed through setting pixelSize parameter.
  • The brightSource parameter is changed to boolean.
  • The boolean fitWithWeight parameter was introduced.
  • Interruption is added so that the destriper can be terminated at your request.
  • Tod files stored in temporary directory are available until HIPE is terminated.
  • The destriper speed increased by 30%.
  • The input parameter name Level1 was changed to level1.
  • The input parameter name Tod was changed to tod.
  • The input parameter name L2DeglitchRepeat was changed to l2DeglitchRepeat.
 
Photometer Baseline Removal
Changed:
<
<
THe Median and Polynomial Baseline Removal example use is now contained in a new Useful Script accessed from the top Scripts menu within HIPE
  • New "roi" (region of interest) parameter for baselineRemoval* tasks, specifying the region of the sky to be used in estimating the baseline
  • Removed "doInterpolation" task parameter
  • Minor changes and bug fixes (for example, the tasks now do ignore masked data samples when a TodBuffer is used internally)
>
>
The Median and Polynomial Baseline Removal example use is now contained in a new script accessed from the top Scripts menu within HIPE.
  • New roi (region of interest) parameter for baselineRemoval* tasks, specifying the region of the sky to be used in estimating the baseline.
  • Removed doInterpolation task parameter.
  • Minor changes and bug fixes (for example, the tasks now do ignore masked data samples when a TodBuffer is used internally).
 
Photometer Moving Object Correction Script
Changed:
<
<
A new Useful Script accessed from the top Scripts menu within HIPE:
  • Takes a SSO observation and corrects the map to be in the moving object frame
  • i.e. target is no longer a streak on a stable background.
>
>
A new script accessed from the top Scripts menu within HIPE:
  • Takes a SSO observation and corrects the map to be in the moving object frame.
  • For example, target is no longer a streak on a stable background.
 
Photometer SPIRE Astrometry Alignment
Changed:
<
<
A new Useful Script accessed from the top Scripts menu within HIPE:
  • Takes a reference SPIRE map and a SPIRE map to be corrected and aligns the astrometry of the latter to the former
  • Takes a reference SPIRE map and a list of SPIRE maps to be corrected and aligns the astrometry of the latter to the former
  • Takes a reference ancillary map and a SPIRE map to be corrected and aligns the astrometry of the latter to the former
  • Takes a referencesource list (as a sourceListProduct) and a SPIRE map to be corrected and aligns the astrometry of the latter to the former
  • In all cases both the Level 2 map astrometry and Level 1 timelines are corrected
>
>
A new script accessed from the top Scripts menu within HIPE:
  • Takes a reference SPIRE map and a SPIRE map to be corrected and aligns the astrometry of the latter to the former.
  • Takes a reference SPIRE map and a list of SPIRE maps to be corrected and aligns the astrometry of the latter to the former.
  • Takes a reference ancillary map and a SPIRE map to be corrected and aligns the astrometry of the latter to the former .
  • Takes a reference source list (as a SourceListProduct) and a SPIRE map to be corrected and aligns the astrometry of the latter to the former.
  • In all cases both the Level 2 map astrometry and Level 1 timelines are corrected.
 
Spectrometer Array Footprint Plot
Changed:
<
<
A new Useful Script accessed from the top Scripts menu within HIPE:
>
>
A new script accessed from the top Scripts menu within HIPE:
 
  • Plots the locations of the detectors on top of an image of the source.

Spectrometer Background Subtraction
Changed:
<
<
A new Useful Script accessed from the top Scripts menu within HIPE:
>
>
A new script accessed from the top Scripts menu within HIPE:
 
  • Provides two methods to subtract residual background from the final spectrum:
Changed:
<
<
    • using the daily dark observation
    • using off-axis detectors
>
>
    • using the daily dark observation.
    • using off-axis detectors.
 This is particularly useful for faint sources.

Spectrometer Line Fitting
Changed:
<
<
A new Useful Script accessed from the top Scripts menu within HIPE:
  • Allows simultaneous fitting of lines and continuum in the central detectors using the HIPE SpectrumFitter
  • CO lines are fitted with a polynomial baseline and results are printed for each line, including the error estimate for the line flux
>
>
A new script accessed from the top Scripts menu within HIPE:
  • Allows simultaneous fitting of lines and continuum in the central detectors using the HIPE SpectrumFitter.
  • CO lines are fitted with a polynomial baseline and results are printed for each line, including the error estimate for the line flux.
 
Spectrometer Thumbnail Mosaic Plot
Changed:
<
<
A new Useful Script accessed from the top Scripts menu within HIPE:
>
>
A new script accessed from the top Scripts menu within HIPE:
 
  • Gives several examples of plotting the spectra from each detector as a thumbnail arranged as the detectors appear in the array.

SDI/SDS Explorer

Changed:
<
<
The SDI and SDS Explorer tools have been updated to:
  • Ability to plot errors as a shaded region on the spectrum added
>
>
The SDI and SDS Explorer tools have been updated with the following features:
  • Added ability to plot errors as a shaded region on the spectrum.
 
  • Right click options added for each detector in the array footprint, including Open With, Send To and Create Variable.
  • Zooming is now permitted in the thumbnail view.

Documentation

Changed:
<
<
  • New SPIRE Launch Pads to give Users an overview of the SPIRE processing, whats available, and quick link jumps into the documentation.
>
>
  • New SPIRE Launch Pads to give you an overview of the SPIRE processing, what's available, and quick link jumps into the documentation.
 
  • A major revision of the SPIRE Data Reduction Guide for both Photometer and Spectrometer Chapters.
Changed:
<
<
  • SPIRE Data Reduction Guide restructured to be "recipe" orientated
  • Includes description of new Useful Scripts accessed from the Scripts menu in HIPE
  • SPIRE Data Reduction Guide includes the manual for mapmaking with the SPIRE Photometer Interactive Analysis (SPIA) plug-in
>
>
  • SPIRE Data Reduction Guide restructured to be "recipe" orientated.
  • Includes description of new scripts accessed from the Scripts menu in HIPE.
  • SPIRE Data Reduction Guide includes the manual for mapmaking with the SPIRE Photometer Interactive Analysis (SPIA) plug-in.
 

Revision 652011-12-02 - PeterDavis

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 429 to 429
 
    • The Instrument RSRF in SCalSpecInstRsrf has been updated, Bright Source mode RSRFs added and units updated to GHz
    • The OPD limits in SCalSpecOpdLimits have been changed such that the interferograms for calibration observations (resolution CR) are cut to have the same limits as high resolution science observations (HR).
    • The spectral band edges in SCalSpecBandEdges have been made slightly narrower to avoid spikes at the edges. They have also been converted to GHz.
Changed:
<
<
    • The spectral limits used for phase correction (used for the polynomial fit method) in !SCalSpecPhaseCorrLim have been converted to GHz.
>
>
    • The spectral limits used for phase correction (used for the polynomial fit method) in SCalSpecPhaseCorrLim have been converted to GHz.
 
    • Three new calibration products have been added to allow processing of Bright Source mode data:
      • SCalSpecNomPcal contains the weighted mean and standard deviation PCAL level separation for dark sky in Nominal Mode - this is used to compare with the PCAL level separation in Bright Source mode in order to calculate a bright mode gain factor for each detector.
      • SCalSpecBrightGain contains a frequency dependent correction to the PCAL gain factor (the gradient and intercept of a straight line). It is applied in the spectral domain to correct the spectral shape of Bright Mode observations.
Line: 444 to 444
 An additional option to correct the bolometer voltages for phase gain has been added. This is used in the Spectrometer pipeline to process Bright Source mode observations.

Common Pipeline

Changed:
<
<
The default value for reconstruction in the sigmaKappa deglitcher is changed to "DIRECT".
>
>
The default value for the correctionMode parameter in the sigmaKappa deglitcher is changed to "DIRECT".
  A new metadata item, calVersion, is added to all pipeline produced products to specity the calibration tree version used.

Photometer Pipeline

Extended Emission Calibration Using the Relative Detector Gains

Changed:
<
<
A new optional module in the User Pipelines that takes into acount the fact that the SPIRE detectors dont all have exactly the same beamshape
>
>
A new optional module in the User Pipelines that takes into account the fact that the SPIRE detectors do not all have exactly the same beamshape
 
  • The Flag applyExtendedEmissionGains needs to be set to True

Include Turnaround Data for Maps

Line: 501 to 501
 

Create SDI from SDS

  • InverseFourierTransformTask has been created to transform SDS objects into SDI objects by applying the inverse Fourier transform:
    sdi = inverseFourierTransform(sds = sds)
Changed:
<
<
  • InverseFourierTransformTask is not the inverse of the FourierTransformTask because it does not remove zeropadding from the output interferograms.
>
>
  • InverseFourierTransformTask is not the inverse of the FourierTransformTask because it does not remove zeropadding from the output interferograms. The already existing makeSameOpds can be used to remove zero-padding.
 

Improved pre-processed cube product

The "preCube" product which contains the collection of spectra to be gridded into the final cube has been improved and now contains a dataset in Spectrum2d format, that can easily be viewed with Herschel viewers.

Convenience task for spectral cube creation

SpireProjectionTask has been created to provide a convenient single API for creating spectral cubes. The task can make spectral cubes using any of the following algorithms:
Changed:
<
<
  • Naive projection
>
>
  • Naive projection (cf. SPIRE photometer)
 
  • Nearest neighbour projection
Changed:
<
<
  • Gridding projection
>
>
  • Gridding projection (cf. HIFI OTF gridding)
 

Interactive Analysis and Tools

Line: 568 to 568
 
Spectrometer Line Fitting
A new Useful Script accessed from the top Scripts menu within HIPE:
  • Allows simultaneous fitting of lines and continuum in the central detectors using the HIPE SpectrumFitter
Changed:
<
<
  • CO lines are fitted with a polynomial baseline and results are printed for each line
>
>
  • CO lines are fitted with a polynomial baseline and results are printed for each line, including the error estimate for the line flux
 
Spectrometer Thumbnail Mosaic Plot
A new Useful Script accessed from the top Scripts menu within HIPE:

Revision 642011-12-02 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 18 to 18
 

Status bar

Changed:
<
<
The maximum memory level reached is indicated with a vertical line in the memory bar. The tooltip shows the actual reached value, and allows to reset it so that further computations can record its own maximum.
>
>
The maximum memory level reached is indicated with a vertical line in the memory bar. The tooltip shows the actual maximum value, and allows you to reset it.
 

Editors & Views

Variables View
Changed:
<
<
  • Added contextual menu action Show contents to show the contents of a variable in the console.
  • Added contextual menu action Send To -> Text File to save the contents of TableDatasets in files.
>
>
  • Added contextual menu entry Show contents to show the contents of a variable in the console.
  • Added contextual menu entry Send To --> Text File to save the contents of table datasets to files.
 
  • Names of variables created from the Product Viewer are more consistent and meaningful.

Dataset Viewer
Line: 34 to 34
 
  • The title (product's description) can be selected and copied.
  • The expanded sections can be resized, for example to show more metadata rows.
  • The summary of the Observation Viewer has been enriched with useful metadata of the observation being shown.
Changed:
<
<
  • Products can be saved directly from the tree with the Sent To action of the popup menu, without needing to create an intermediate variable.
>
>
  • Products can be saved directly from the tree with the Send To entry of the popup menu, without creating an intermediate variable.
 
Jython & Text Editors
  • The supported Jython language has been upgraded from version 2.1 to 2.5.2.
Changed:
<
<
  • Scripts and text files coming with the installation or with plug-ins are safe to edit without losing the original version.
>
>
  • Scripts and text files coming with the installation or with plug-ins are now safe to edit without losing the original version.
 
  • Improved the step by step execution of scripts. See the note on this page for more details.

Help & Debug

  • The Tip of the day contains information on the Console meta commands (those starting with !)
Changed:
<
<
  • If you get temporary (or permanent) freezes in the HIPE GUI, or you have created a custom task dialogue or view in HIPE, you can debug it by enabling the corresponding preference in Edit > Preferences > General > Debug. Any problem found with this debugging can be reported like previously, with the HSC Helpdesk or Give Feedback options of the Help menu.
>
>
  • If you get temporary (or permanent) freezes in the HIPE GUI, or you have created a custom task dialogue or view in HIPE, you can debug it by enabling the corresponding preference in Edit --> Preferences --> General --> Debug. Any problem found with this debugging can be reported like previously, with the HSC Helpdesk or Give Feedback options of the Help menu.
 

Plug-ins

Line: 57 to 57
 
  • Fixed a problem with the scanamorphos plug-in, which would not read correctly some scanamorphos output. The plug-in assumed that the PrimaryImage HDU contained a 3-dim array. Output containing a 2-dim array in the PrimaryImage HDU was not read correctly.

Upgrades

Changed:
<
<
If there is a new user release available, HIPE would notice it and offer to install the new version.
>
>
If there is a new user release available, HIPE will notify you and offer to install the new version.
 

Plotting

PlotXY is now able to display images (i.e. 2D data). Look at an advanced example here.

TablePlotter

Changed:
<
<
TablePlotter can display plot with Date/Time format.
>
>
TablePlotter can now display plot with Date/Time format.
 

OverPlotter

Changed:
<
<
OverPlotter can display plot with Date/Time format.
>
>
OverPlotter can now display plot with Date/Time format.
 

Astronomical utilities

Changed:
<
<
  • Planck class moved from spire_ia_pipeline_spec_util to ia_toolbox_astro. If you are doing an explicit import (it is not needed) you have to change it to (preferred) import herschel.ia.toolbox.astro.Planck or from herschel.ia.toolbox.astro import Planck
  • New task astrometryFix: adjusts the absolute pointing of map data to give the best possible agreement in overall astrometry with some ancillary data. This is done either by cross correlation of the map(s) with ancillary map data, or by stacking the map(s) at a list of known source positions (e.g., from radio observations).
>
>
  • Planck class moved from spire_ia_pipeline_spec_util to ia_toolbox_astro. If you are doing an explicit import (it is not needed) you have to change it to (preferred) import herschel.ia.toolbox.astro.Planck or from herschel.ia.toolbox.astro import Planck
  • New task astrometryFix: adjusts the absolute pointing of map data to give the best possible agreement in overall astrometry with some ancillary data. This is done either by cross correlation of the map(s) with ancillary map data, or by stacking the map(s) at a list of known source positions (e.g., from radio observations).
 

Numeric routines

Data fitting

Line: 79 to 79
 

Trend analysis

Mapping

Wavelet transform

Changed:
<
<
  • Continuous Wavelet Transform synthesis operation now adds low frequencies for a reliable reconstruction
  • Continuous Wavelet Transform is able to deal with Complex1d signal
>
>
  • Continuous Wavelet Transform synthesis operation now adds low frequencies for a reliable reconstruction.
  • Continuous Wavelet Transform is now able to deal with Complex1d signal.
 

Other functions

Source extraction

Changed:
<
<
  • Improvements to handling of pixel coordinates and region of interest in sourceExtractor* tasks:
    • Output source list now contains pixel coordinates of source (x, y) = (column, row)
    • Added "getWcs" and "setWcs" methods to the SourceListProduct/Dataset
    • Added "roi" parameter (region of interest) to tasks
    • Removed "corner*" and "returnPixelCoordinates" parameters from tasks
  • Non-standard columns are now accepted in a SourceListDataset, e.g., when converting from a TableDataset (an exception will be thrown if there are no standard columns at all)
  • Removed "subtractMedianBackground" parameter from sourceExtractorSussextractor (it didn't do anything)
  • Minor bug fixes, performance improvements and various internal changes
>
>
  • Improvements to handling of pixel coordinates and region of interest in sourceExtractor tasks:
    • Output source list now contains pixel coordinates of source (x, y) = (column, row).
    • Added getWcs and setWcs methods to the SourceListProduct/Dataset.
    • Added roi parameter (region of interest) to tasks.
    • Removed corner* and returnPixelCoordinates parameters from tasks.
  • Non-standard columns are now accepted in a SourceListDataset, e.g., when converting from a TableDataset (an exception will be thrown if there are no standard columns at all).
  • Removed subtractMedianBackground parameter from sourceExtractorSussextractor (it did not do anything).
  • Minor bug fixes and performance improvements.
 

Images

Display

Changed:
<
<
  • The methods addCircle, addEllipse and addRectangle is not longer dependent on the zoomfactor of the image.
>
>
  • The methods addCircle, addEllipse and addRectangle ae no longer dependent on the zoom factor of the image.
 

Analysis

Spectra

Display

Analysis

Changed:
<
<
  • The SpectrumFitter now recognizes a CUBE such that the user can give the x,y of the spectrum. The script as written by the SpectrumFitterGUI also takes the correct spectrum.
>
>
  • The SpectrumFitter now recognizes a CUBE such that you can give the x,y position of the spectrum. The script as written by the SpectrumFitterGUI also takes the correct spectrum.
 
Changed:
<
<
  • avg now warns if wavescales are not sufficiently aligned to get a good result
>
>
  • The avg task now warns if wavescales are not sufficiently aligned to get a good result.
 
Changed:
<
<
  • various changes when applying toolbox tasks to cubes, in particular avg, select and extract now allow you to work on several pixels/spaxels within the cube rather than only the entire cube.
>
>
  • Various changes when applying toolbox tasks to cubes, in particular avg, select and extract now allow you to work on several pixels/spaxels within the cube rather than only the entire cube.
 
Changed:
<
<
  • resampling task now allows alternative ways to specify the grid for resampling.
>
>
  • The resampling task now allows alternative ways to specify the grid for resampling.
 

Data cubes

Display

Analysis

Herschel Science Archive

Changed:
<
<
  • HIPE will now store username and password in encrypted form outside of properties files (if so requested -- one can also choose not to store the username and password). In any case, it is recommended to remove HSA username and password from all properties files.
>
>
  • HIPE can now store username and password in encrypted form outside of property files. It is recommended to remove HSA username and password from all property files.
 

Calibration sources

Line: 125 to 125
 

Products and datasets

All products

Changed:
<
<
  • Removed the legacy diff methods from Product and Dataset interface. All functionality moved into ProductDiff class which is available as part of the utility toolbox (herschel.ia.toolbox.util)
>
>
  • Removed the legacy diff methods from Product and Dataset interfaces. All functionality moved to the ProductDiff class in the utility toolbox (herschel.ia.toolbox.util)
 

History

Changed:
<
<
A new task historyExtract is available to facilitate retrieving the value of a certain parameter from the product history. E.g. one can use the following command line to find out which calibration file was used in a given task of the PACS spectrometer pipeline:
>
>
A new task historyExtract is available to facilitate retrieving the value of a certain parameter from the product history. For instance, you can use the following command to find out which calibration file was used in a given task of the PACS spectrometer pipeline:
 
value = historyExtract(product=frames, task='convertSignal2StandardCap', parameter='effectiveCapacitance')
Changed:
<
<
The GUI of this task guides you through the available tasks and parameters: The option lists of the 'task' and 'parameter' parameters only show what is available in the history of the specified product:
>
>
The GUI of this task guides you through the available tasks and parameters: The option lists of the task and parameter parameters only show what is available in the history of the specified product:
  Screenshot-HIPE_-_historyExtract.png
Line: 143 to 143
 

Product Access Layer

Changed:
<
<
  • A "Cache" entry was added to the HIPE Preferences. Under this entry, a panel is shown with an overview of all existing caches on the system. One can also clear out these caches from this panel.
  • Product Browser: A simplified Observations tab was added, that allows user to list all observations on local disk, just by hitting the Run button. The data source selector has also been re-designed to be more flexible and intuitive.
>
>
  • A Cache entry has been added to the HIPE Preferences, with an overview of all existing caches on the system. You can clear these caches from this panel.
  • Product Browser: A simplified Observations tab has been added, that allows you to list all observations on local disk, just by pressing the Run button. The data source selector has also been redesigned to be more flexible and intuitive.
 

Tasks

Line: 152 to 152
 
    • All mandatory parameters must be filled in by users: removed default values from mandatory parameters, warnings issued (detection of programming mistakes)
    • Improved parameter checking and messages on bad data: Validators in primary inputs, they return false on error and log a warning (detection of programming mistakes)
    • All task GUIs and nodes should now show tooltips
Changed:
<
<
  • Added some new user-friendly parameter editors (see For Task developers only)
>
>
  • Added some new user-friendly parameter editors (see For Task developers only below)
 
  • Default task layout:
Changed:
<
<
    • Parameter name tooltips now show the parameter name (for long names)
    • big modifiers can occupy a full row
    • groups (tabs) can be navigated with the mouse wheel
    • Parameters can be reset individually via contextual menu on parameter names (vs resetting all parameters with Clear button)
    • Parameters can list the variables they can accept via contextual menu on parameter names (to know before hand if we have the proper data to use a parameter)
>
>
    • Parameter name tooltips now show the parameter name (for long names).
    • Big modifiers can occupy a full row.
    • You can move through groups (tabs) with the mouse wheel.
    • Parameters can be reset individually via contextual menu on parameter names (in addition to resetting all parameters with the Clear button)
    • Parameters can list the variables they accept via contextual menu on parameter names, so you can check if you have the proper data to use in a parameter.
 
  • For Task developers only:
Changed:
<
<
    • Task registration will issue warnings if either the task was already registered or the taskname was already used or both.
>
>
    • Task registration will issue warnings if the task was already registered or the taskname was already used.
 
    • Command generation: added converters so that arbitrary types can be represented with Jython strings (part of better visibility for defaults)
    • New shared Modifiers (parameter editors):
      • Added Editor for selecting a subset of a list of values (JListModifier, N values of M possible, M >= N)
      • Added a variant of JOptionModifier (Combo box) that returns some values but shows others (example: parameter uses integers but we show strings to users)
Changed:
<
<

Misc Tasks

  • OpenTask parameter names have changed capitalization to comply with lower camel case
>
>

Miscellaneous tasks

  • OpenTask parameter names have changed capitalisation to comply with naming conventions.
 

Quality control

Data input-output

Changed:
<
<
  • URM entries (documentation) improved
>
>
  • Improved entries in the User's Reference Manual.
 
  • Task importUFDirToPal renamed to importObservation
  • Task exportPalToUFDir renamed to exportObservation
Changed:
<
<
  • Task simpleAsciiTableReader extended to support all options of asciiTableReader via Advanced tab.
>
>
  • Task simpleAsciiTableReader extended to support all options of asciiTableReader via the Advanced tab.
 
  • Improved type guessing for table reading (affects (simple)AsciiTableReader): for floating-point data it will detect precision loss and maintain representation. If a floating-point number will lose precision as a float (32-bits) a double (64-bits) will be used instead: up to 17 significant digits but in these case it will use twice the memory.
Changed:
<
<
  • Improved documentation and information provided by getObservation
  • saveObservation now reports the progress in the status bar and can be interrupted/stopped.
  • Added contextual menu option to Variables View to save tables with asciiTableWriter. Also added a validator to the task that will restrict its applicability to TableDatasets that can be fit into an ASCII table.
>
>
  • Improved documentation and information provided by getObservation.
  • saveObservation now reports the progress in the status bar and can be interrupted.
  • Added contextual menu option to the Variables view to save tables with asciiTableWriter. Also added a validator to the task that will restrict its applicability to table datasets that can be fit into an ASCII table.
 

Virtual Observatory

Line: 231 to 231
 

HIFI Calibration

  • Updates to Calibration tree (HICAL-30)
Changed:
<
<
    • Spur table (7b)
    • FPU HK Thresholds (5a) (Prevents unnecessary Quality Flags)
    • Units added to columns (Prevents unnecessary Quality Flags)
    • Wbs Bad Pixel (saturated regions) (Causes necessary Quality Flags)
    • QC flags for bad LOU currents (Causes necessary Quality Flags)
    • Uplink auxiliary product parameter filtering (First steps in ensuring information about the way the observation was carried out is put in the meta data)
>
>
    • Spur table (7b).
    • FPU HK Thresholds (5a) (Prevents unnecessary Quality Flags).
    • Units added to columns (Prevents unnecessary Quality Flags).
    • Wbs Bad Pixel (saturated regions) (Causes necessary Quality Flags).
    • QC flags for bad LOU currents (Causes necessary Quality Flags).
    • Uplink auxiliary product parameter filtering (First steps in ensuring information about the way the observation was carried out is put in the metadata).
 

HIFI Pipeline

hifiPipeline task

Changed:
<
<
  • Can now select calibration version to reprocess data with (command line only)
>
>
  • Can now select calibration version to reprocess data with (command line only).
 
Changed:
<
<
  • By default the pipeline now runs without saving products. Previously it always wrote output to .hcss/lstore/pipeline-out, which in theory allowed users to access processed observations from that location. This was rarely used and could cause disk space problems.
>
>
  • By default the pipeline now runs without saving products. Previously it always wrote output to .hcss/lstore/pipeline-out, which in theory allowed you to access processed observations from that location. This was rarely used and could cause disk space problems.
 
Changed:
<
<
  • Improvements to tooltips in the Pipeline GUI
>
>
  • Improvements to tooltips in the Pipeline GUI.
 

Level 0 Pipeline

Changed:
<
<
  • DoUplink now uses the new uplink auxiliary product to set some observation parameters in the meta data, e.g., mapReadoutSep and mapLineStep
>
>
  • DoUplink now uses the new uplink auxiliary product to set some observation parameters in the metadata, such as mapReadoutSep and mapLineStep.
 
  • Suppression of unneeded Quality Flags: Quality check for mixer voltage now accounts for magnet tuning.

WBS Pipeline

HRS Pipeline

Level 1 Pipeline

Changed:
<
<
  • New (optional) pipeline step allowing smooth band pass to eliminate standing waves: DoFilterLoadsTask. This is an alternative calibration scheme for strong continuum sources.
>
>
  • New (optional) pipeline step allowing smooth band pass to eliminate standing waves: doFilterLoads. This is an alternative calibration scheme for strong continuum sources.
 
Changed:
<
<
  • Substantial refactoring of CheckFrequencyGrid to correct assumptions made about the sequence of on-, off- and hc-datasets. This primarily affected HEB band data and resulted in unnecessary Quality Flags being raised.
>
>
  • Substantial refactoring of CheckFrequencyGrid to correct assumptions made about the sequence of on-, off- and hc-datasets. This primarily affected HEB band data and resulted in unnecessary Quality Flags being raised.
 

Level 2 Pipeline

Changed:
<
<
  • Sideband gains coefficients added to meta data
>
>
  • Sideband gains coefficients added to metadata.
 

HIFI Products

  • Meta data calVersion is applied to all calibrated products and has format HIFI_CAL_version_number
Line: 272 to 272
 

HIFI Spectrum Toolbox

Spectrum Explorer (HIFI specifics)

Spectrum tasks (HIFI specifics)

Changed:
<
<
  • Introduction of FlagToolTask, which allows you go through every subband of a SpectrumDataset and interactively flag spurs, lines, bright lines, and data to ignore. Input data can be an Observation context, a HifiTimelineProduct, or a SpectrumDataset
>
>
  • Introduced the flagTool task, which allows you go through every subband of a spectrum dataset and interactively flag spurs, lines, bright lines, and data to ignore. Input data can be an Observation context, a HifiTimelineProduct, or a SpectrumDataset.
 
Changed:
<
<
  • PolarPair is now a task

>
>
  • PolarPair is now a task and can be applied to spectrum datasets.
 

HIFI Data Processing Tools

Standing Wave Removal

Changed:
<
<
  • avermask option in FitHifiFringe introduced.
    • Allows a line mask to be calculated using the average of all scans. Makes fringe removal from maps much more efficient.
>
>
  • Introduced avermask option in FitHifiFringe. It allows a line mask to be calculated using the average of all scans. Makes fringe removal from maps much more efficient.
 
Changed:
<
<
  • Variable result_sds should no longer created if FitHifiFringe is run from the GUI with obs_or_htp input
>
>
  • Variable result_sds should no longer be created if FitHifiFringe is run from the GUI with obs_or_htp input.
 
  • When an Observation Context is supplied standing waves are now automatically removed from both upper and lower sidebands.
Line: 294 to 291
 
  • The back-end(s) to be processed can now be selected by clicking in a list.
Changed:
<
<
  • FitBaseline 'divide' option now handles negative signals robustly.
>
>
  • FitBaseline divide option now handles negative signals robustly.
 

Deconvolution

Changed:
<
<
  • Some output from doDeconvolution is now written to the console as it runs - helpful to know HIPE has not crashed!
>
>
  • Some output from doDeconvolution is now written to the console as it runs - helpful to know HIPE has not frozen!
 
Changed:
<
<
  • doDeconvolution now picks up the applied sideband gains from the meta data and "undoes" their effect on the incoming DSB data. Then the user running the deconvolution can then apply gains on or gains off. The pipeline supplied gains can in this way offer an optional starting point for the gain iteration inside the deconvolution.
>
>
  • doDeconvolution now picks up the applied sideband gains from the meta data and "undoes" their effect on the incoming DSB data. Then you can apply gains on or gains off. The pipeline supplied gains can in this way offer an optional starting point for the gain iteration inside the deconvolution.
 
    • This only has practical application for band 2 data at present.

  • New redundancy product allows you to overplot redundancy on the SSB solution. Helpful to identify where the SSB solution may not be reliable.
Changed:
<
<
  • SSB spectrum now has y-axis labels "Antenna Temperature (K)", x-axis label changed to "Frequency (GHz)"
>
>
  • SSB spectrum now has y-axis labels Antenna Temperature (K), x-axis label changed to Frequency (GHz).
 
Changed:
<
<
  • When diagnostic mode is turned on, doDeconvolution now produces an interim product in addition to the pop-up plots.
>
>
  • When diagnostic mode is turned on, doDeconvolution now produces an interim product in addition to the pop-up plots.
 
Changed:
<
<
  • The min chi squared and the number of iterations are now included in the meta data of the SSB solution
>
>
  • The min chi squared and the number of iterations are now included in the meta data of the SSB solution.
 
Changed:
<
<
  • Bug fix: In doDeconvolution an algorithmic error in binning the input data has been corrected. The fix has stopped the occurrence of two problems: Some band-edge channels had fluxes set to zero and some very narrow lines had their peaks shifted by as much 1/2 MHz. Both errors are fixed.
>
>
  • Bug fix: In doDeconvolution an algorithmic error in binning the input data has been corrected. The fix has stopped the occurrence of two problems: Some band-edge channels had fluxes set to zero and some very narrow lines had their peaks shifted by as much 1/2 MHz.
 

Spectral Cube creation

Changed:
<
<
  • BeamSize is now taken from the calibration tree if "hpbw" is in the metadata
    • If the "hpbw" is not in the meta data, the beam size is calculated in the same way as in convertK2Jy using the rfreq in the meta data
>
>
  • BeamSize is now taken from the calibration tree if hpbw is in the metadata.
    • If hpbw is not in the metadata, the beam size is calculated in the same way as in convertK2Jy using the rfreq in the metadata.
 
Changed:
<
<
* Can rotate maps using new parameter, FlyAngle
>
>
* Can rotate maps using new parameter, FlyAngle.
 
  • GUI behaviour modified:
Changed:
<
<
    • Picks up information from meta data to provide better default input parameters
>
>
    • Picks up information from metadata to provide better default input parameters.
 
    • Changes to values in the GUI form now only go in one direction, e.g. changing PixelSize changes the Beam and Filter Parameters but not vice versa.

HiClass export tool

  • You can now create keywords when exporting to CLASS fits format.

Documentation

Changed:
<
<
  • HIFI User's Manual changed name to HIFI Data Reduction Guide, chapters rearranged into more logical order, addition of a changelog
    • Addition of introductory chapter, "HIFI Launch Pad"
    • Addition of a DBS Point Mode cookbook
    • Addition of chapter, "How to flag and remove flags from your data"
    • Addition of chapter, "Unit Conversion"
    • Updates to chapters on "Pipeline", "Deconvolution", "Creating Spectral Cubes", "What was done to my data" (clarifying units at each stage of the pipeline),
  • Updates the examples in the HIFI Pipeline Specification Document
  • HIFI User Reference Manual (URM) improvements
    • All HIFI tasks in URM now have a category and show up in the categorised view of commands
    • The categorised view of commands now is grouped into headers that should be meaningful to astronomers
    • Many URM entries have been improved so that decriptions are clear and examples are useful
>
>
  • The HIFI User's Manual changed name to HIFI Data Reduction Guide, chapters rearranged into more logical order, addition of a changelog.
    • Addition of introductory chapter, HIFI Launch Pad.
    • Addition of a DBS Point Mode cookbook.
    • Addition of chapter, How to flag and remove flags from your data.
    • Addition of chapter, Unit Conversion.
    • Updates to chapters on Pipeline, Deconvolution, Creating Spectral Cubes, What was done to my data (clarifying units at each stage of the pipeline).
  • Updates the examples in the HIFI Pipeline Specification Document.
  • HIFI User Reference Manual (URM) improvements.
    • All HIFI tasks in URM now have a category and show up in the categorised view of commands.
    • The categorised view of commands now is grouped into headers that should be meaningful to astronomers.
    • Many URM entries have been improved so that descriptions are clear and examples are useful.
 

PACS

Line: 345 to 342
 

Photometry

Changed:
<
<
  • a new task is used for the deglitching less memory intensive MapDeglitch instead of IIndLevelDeglitch
  • the 2nd level glitch mask is now stored in level 1.
  • new task filterOnScanSpeed now used to select frames at constant speed instead of the BBID selection
  • new task photMaskCrosstalk, masking the column 0 of all matrices now used in the pipeline too. This is a bit of brute force to avoid negative or positive cross-talk artifact in teh level 2 maps, at the expense of getting rid of 1/16th of the data.
>
>
  • The less memory intensive task mapDeglitch is now used for deglitching instead of IIndLevelDeglitch
  • The second level glitch mask is now stored in level 1.
  • The new task filterOnScanSpeed now used to select frames at constant speed instead of the BBID selection.
  • The new task photMaskCrosstalk, masking the column 0 of all matrices, is now used in the pipeline too. This is a bit of brute force to avoid negative or positive cross-talk artifact in the level 2 maps, at the expense of getting rid of 1/16th of the data.
 

Spectroscopy

Changed:
<
<
  • interactive reduction scripts for all observing modes are now available
  • flatfielding is available in the interactive pipeline scripts
  • error propagation (and error bars) is implemented in the PACS rebinned cubes (STDDEV dataset)
  • spg pipeline: the wavelength switching pipeline is merged with the unchopped pipeline and removed
  • improved documentation and user friendliness of the pipeline tasks within hipe
  • The spectrometer level-2 tasks now produce the specialized sliced products SlicedPacsCube and SlicedPacsRebinnedCube, which offer the same functionality as the SlicedFrames product.
>
>
  • Interactive reduction scripts for all observing modes are now available.
  • Flatfielding is available in the interactive pipeline scripts.
  • Error propagation (and error bars) is implemented in the PACS rebinned cubes (STDDEV dataset).
  • SPG pipeline: the wavelength switching pipeline is merged with the unchopped pipeline and removed.
  • Improved documentation and user friendliness of the pipeline tasks within HIPE.
  • The spectrometer level-2 tasks now produce the specialized sliced products SlicedPacsCube and SlicedPacsRebinnedCube, which offer the same functionality as the SlicedFrames product.
 

Calibration

Changed:
<
<
  • The calibration sets that are installed on your local system can now be inspected with a 'Calibration Sets Viewer'. The viewer is located in the menu Window -> Show View -> Workbench -> Calibration Sets. Select a version from the available buttons, the you can inspect the release note for that version or the product list with the versions of the individual calibration products.
>
>
  • The calibration sets that are installed on your local system can now be inspected with a 'Calibration Sets Viewer'. The viewer is located in Window --> Show View --> Workbench --> Calibration Sets. Select a version from the available buttons, then you can inspect the release note for that version or the product list with the versions of the individual calibration products.
 

Photometry

Changed:
<
<
  • A new calibration file is available for non linearity correction required by the task photNonLinearityCorrection.
>
>
  • A new calibration file is available for non linearity correction required by the task photNonLinearityCorrection.
 

Spectroscopy

Changed:
<
<
  • major improvements of the flux calibration:
    • implementation of the background normalization method for chop/nod observations
    • it is now based on observations of astronomical standards and the calibration block attached to each observation. The pipeline task specDiffCs computes the detector response of the current observation.
    • implementation of a correction of transients for unchopped observations
>
>
  • Major improvements of the flux calibration:
    • Implementation of the background normalization method for chop/nod observations.
    • It is now based on observations of astronomical standards and the calibration block attached to each observation. The pipeline task specDiffCs computes the detector response of the current observation.
    • Implementation of a correction of transients for unchopped observations.
 
Changed:
<
<
  • the spectral (pixel) dependency of the spatial calibration is now available for each pixel
>
>
  • The spectral (pixel) dependency of the spatial calibration is now available for each pixel.
 

Analysis tools

Changed:
<
<
  • The Preferences now contain a Logging panel under the general tab which allows the user/developer to increase or decrease the number of logging messages send to the
terminal.
>
>
  • The Preferences windo now contain a Logging panel under the general tab which allows you to increase or decrease the number of logging messages sent to the terminal.
 
  • The calibration file version is now also shown in the product history.

Photometry

Changed:
<
<
MapMonitor got an improved interface and allows to flip north up-or downwards.

PhotProject got a companion task MiProject (MapIndex-Project). If you have a full MapIndex, you may use MiProject instead of PhotProject. It needs frames and the MapIndex as parameters and speeds up projection time by a factor of 20 (30, 40?). Whatever the number is: its much faster than PhotProject. Use as:

 map = miProject(frames, mapindex)  

A new Task has been developed. It deglitches without MapIndex and runs with significantly less memory abuse than the IIndLevelDeglitchTask. The name and package is

 herschel.pacs.spg.phot.deglitching.map.MapDeglitchTask 
. The hipe alias is mapDeglitch. It is documented as javadoc, URM and on the online deglitching documentation page:
http://herschel.esac.esa.int/twiki/bin/view/Public/SecondLevelDeglitching#MapDeglitchTask

A new Task has been implemented which removes crosstalk effects by masking (CrossTalkMask ) the first rows of detector arrays :

 frames = photMaskCrosstalk(frames) 

The HIPE Menu Pipeline -> PACS -> has been re organized and ipipe scripts added.

Automatic pipeline scripts save the Products at the end in the Observation Context also in IA sessions. That mean the scripts can be executed in a sequence without interaction.

Pacs Products (for example Frames, Ramps, PacsCube) got a method to rename a Mask :

 renameMask(String oldname, String newname) 

up to now, a mask had to be removed and created new, if the name had to be changed.

PhotProject, MiProject, IIndLevelDeglitching and some other tasks report their progress to the hipe progressbar instead of the terminal output.

The effects of 2-bit rounding on second level deglitching have been reduced in the pipeline for blue parallel maps.

The Pacs MaskViewer has been relocated to a new package:

 herschel.pacs.gui.maskviewer.MaskViewer 
. If you manually imported the old location (from herschel.pacs.signal import MaskViewer), the import will fail. Please just remove the import. Everything should work fine without further action. If there should be a problem nevertheless, please import the new location:
 from herschel.pacs.gui.maskviewer import MaskViewer  

  • A new tasks MaskFromCatalogue is available, very convenient to mask a list of point sources for high-pass filtering in deep fields.
>
>
  • MapMonitor got an improved interface and allows to flip north up-or downwards.
  • The PhotProject task got a companion task miProject (MapIndex-Project). If you have a full MapIndex, you may use miProject instead of photProject. It needs frames and the MapIndex as parameters and speeds up projection time by a factor of at least 20. Use as follows: map = miProject(frames, mapindex)
  • A new task has been developed. It deglitches without MapIndex and runs with significantly less memory abuse than the IIndLevelDeglitch task. The name and package is herschel.pacs.spg.phot.deglitching.map.MapDeglitchTask. The HIPE alias is mapDeglitch. It is documented as javadoc, URM and on the online deglitching documentation page.
  • A new task has been implemented which removes crosstalk effects by masking (CrossTalkMask) the first rows of detector arrays: frames = photMaskCrosstalk(frames)
  • The HIPE menu Pipeline --> PACS has been reorganised and ipipe scripts added.
  • Automatic pipeline scripts save the products at the end in the Observation Context also in IA sessions. That mean the scripts can be executed in a sequence without interaction.
  • Pacs products (for example Frames, Ramps, PacsCube) got a method to rename a Mask: renameMask(String oldname, String newname). Up to now, a mask had to be removed and recreated, if the name had to be changed.
  • photProject, miProject, IIndLevelDeglitching and some other tasks now report their progress in the HIPE progress bar instead of the terminal output.
  • The effects of 2-bit rounding on second level deglitching have been reduced in the pipeline for blue parallel maps.
  • The Pacs MaskViewer has been relocated to a new package: herschel.pacs.gui.maskviewer.MaskViewer. If you manually imported the old location (from herschel.pacs.signal import MaskViewer), the import will fail. Please remove the import, since everything should work automatically. If nevertheless there is a problem, please import the new location: from herschel.pacs.gui.maskviewer import MaskViewer
  • A new task maskFromCatalogue is available, very convenient to mask a list of point sources for high-pass filtering in deep fields.
 

Spectroscopy

Changed:
<
<
  • There is a new GUI for viewing the footprint of the spectrometer upon an image with WCS: the tool pacsSpectralFootprint. It can be accessed through the applicable tasks list in the Tasks View.
>
>
  • There is a new GUI for viewing the footprint of the spectrometer upon an image with WCS: the tool pacsSpectralFootprint. It can be accessed through the Applicable tasks list in the Tasks view.
 
  • The Spectrum Explorer GUI has a new panel for editing PACS masks.
Changed:
<
<
  • implementation of user friendly viewer for the PACS specific part of the ObservationContext:
    • Each spectrometer slice now shows after the slice number a short description of its contents
>
>
  • Implementation of user friendly viewer for the PACS specific part of the ObservationContext:
    • Each spectrometer slice now shows after the slice number a short description of its contents.
 
    • The tooltip of the slice shows the complete list of slice info keywords.

Product changes

Photometry

Changed:
<
<
  • a new level 2.5 product is available combining photProject scan and cross-scan maps.
>
>
  • A new level 2.5 product is available combining photProject scan and cross-scan maps.
 

Spectroscopy

Changed:
<
<
  • Browse products and browse image products are now available in spg
>
>
  • Browse products and browse image products are now available in SPG.
 
  • Red and blue camera spectra can now be combined in a single context in the level-2 processing, therefore, the lineId has changed for the red camera: 100 is added!

Documentation

Changed:
<
<
  • PACS Data Reduction Guide. Chp 1 is now a collection of Launch Pads: on getting Data into HIPE, doing your PACS Spectroscopy and doing your PACS Photometry. These are quick-start guides to working with your PACS observations. Chp2 tells you how to load your data, your pipeline script, and your calibration tree into HIPE, for photometry and spectroscopy both.
  • Spectroscopy: the pipeline chapters are being re-organised to follow the new pipeline scripts, and so for this 8.0 RC4 version of the PDRG only the LineScan Chop-Nod pipeline is described.
  • Photometry: the pipeline chapters are being re-organised to follow the new pipeline scripts, and so for this 8.0 RC4 version of the PDRG only the ScanMap pipeline scripts are described.
>
>
  • PACS Data Reduction Guide. Chapter 1 is now a collection of Launch Pads: on getting Data into HIPE, doing your PACS Spectroscopy and doing your PACS Photometry. These are quick-start guides to work with your PACS observations. Chapter 2 tells you how to load your data, your pipeline script, and your calibration tree into HIPE, for photometry and spectroscopy.
  • Spectroscopy: the pipeline chapters are being re-organised to follow the new pipeline scripts, and so for this 8.0 version of the PDRG only the LineScan Chop-Nod pipeline is described.
  • Photometry: the pipeline chapters are being re-organised to follow the new pipeline scripts, and so for this 8.0 version of the PDRG only the ScanMap pipeline scripts are described.
 

SPIRE

Revision 632011-12-02 - JeroenDeJong

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 126 to 127
 

All products

  • Removed the legacy diff methods from Product and Dataset interface. All functionality moved into ProductDiff class which is available as part of the utility toolbox (herschel.ia.toolbox.util)
Added:
>
>

History

A new task historyExtract is available to facilitate retrieving the value of a certain parameter from the product history. E.g. one can use the following command line to find out which calibration file was used in a given task of the PACS spectrometer pipeline:

value = historyExtract(product=frames, task='convertSignal2StandardCap', parameter='effectiveCapacitance')

The GUI of this task guides you through the available tasks and parameters: The option lists of the 'task' and 'parameter' parameters only show what is available in the history of the specified product:

Screenshot-HIPE_-_historyExtract.png

 

Observation context

Product Access Layer

Line: 344 to 357
 
  • error propagation (and error bars) is implemented in the PACS rebinned cubes (STDDEV dataset)
  • spg pipeline: the wavelength switching pipeline is merged with the unchopped pipeline and removed
  • improved documentation and user friendliness of the pipeline tasks within hipe
Added:
>
>
 

Calibration

Line: 401 to 415
 

Spectroscopy

Changed:
<
<
  • GUIs are now available to work on spectral data: SpectrumExplorer and spectral footprint
>
>
  • There is a new GUI for viewing the footprint of the spectrometer upon an image with WCS: the tool pacsSpectralFootprint. It can be accessed through the applicable tasks list in the Tasks View.
  • The Spectrum Explorer GUI has a new panel for editing PACS masks.
 
  • implementation of user friendly viewer for the PACS specific part of the ObservationContext:
    • Each spectrometer slice now shows after the slice number a short description of its contents
    • The tooltip of the slice shows the complete list of slice info keywords.
Line: 598 to 613
 
<--  
-->
Added:
>
>
META FILEATTACHMENT attachment="Screenshot-HIPE_-_historyExtract.png" attr="" comment="" date="1322831874" name="Screenshot-HIPE_-_historyExtract.png" path="Screenshot-HIPE - historyExtract.png" size="39416" stream="Screenshot-HIPE - historyExtract.png" user="Main.JeroenDeJong" version="1"

Revision 622011-12-02 - EdwardPolehampton

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 426 to 426
 

Calibration Products

Changed:
<
<
  • The Spectrometer Beam Parameters calibration product used by the Point Source Flux Conversion now contains the wavenumber-dependent point-source flux conversion factors for each spectrometer detector. Previously, this product contained factors only for each spectrometer array.
>
>
  • Photometer:
    • SCalPhotChanRelGain is a new product to improve extended emission calibration.
    • The SCalPhotChanNoise product has been updated to improve the flux calibration for MadMap (this is only used for MadMap).
  • Spectrometer:
    • The "extended" calibration in SCalSpecTeleRsrf is now derived from a mean of many observations, reducing the noise level in the RSRF. In addition, RSRFs have been added for the Bright Source mode, and the units have been updated to be in GHz.
    • The "point-source" calibration in _SCalSpecBeamParam! has been updated, Bright Source mode factors added and units updated to GHz. The internal structure of the product has been updated to allow the point-source conversion factors for other off-axis detectors to be added in future, and the "beam area" column has been removed (as this was misleading, and the beam area is better calculated from the diameter, assuming a Gaussian shape).
    • The Instrument RSRF in SCalSpecInstRsrf has been updated, Bright Source mode RSRFs added and units updated to GHz
    • The OPD limits in SCalSpecOpdLimits have been changed such that the interferograms for calibration observations (resolution CR) are cut to have the same limits as high resolution science observations (HR).
    • The spectral band edges in SCalSpecBandEdges have been made slightly narrower to avoid spikes at the edges. They have also been converted to GHz.
    • The spectral limits used for phase correction (used for the polynomial fit method) in !SCalSpecPhaseCorrLim have been converted to GHz.
    • Three new calibration products have been added to allow processing of Bright Source mode data:
      • SCalSpecNomPcal contains the weighted mean and standard deviation PCAL level separation for dark sky in Nominal Mode - this is used to compare with the PCAL level separation in Bright Source mode in order to calculate a bright mode gain factor for each detector.
      • SCalSpecBrightGain contains a frequency dependent correction to the PCAL gain factor (the gradient and intercept of a straight line). It is applied in the spectral domain to correct the spectral shape of Bright Mode observations.
      • SCalSpecBolPhase contains the optimum bolometer phases for each detector. These are used in the engineering conversion to correct the voltages for Bright Source mode, where the bolometers are dephased to reduce their responsivity.
 
Deleted:
<
<
  • The Spectrometer Nominal PCAL calibration product contains the weighted mean and standard deviation PCAL level separation for dark sky in Nominal Mode. This calibration product is used by the new PCAL Gain module.

  • The Spectrometer Bright Mode Gain calibration product contains the intercept and gradient for the wavenumber-dependent gain. This calibration product is used by the new Bright Mode Gain module.

 

Calibration Framework

Preprocessing

Engineering Conversion

Added:
>
>
An additional option to correct the bolometer voltages for phase gain has been added. This is used in the Spectrometer pipeline to process Bright Source mode observations.
 

Common Pipeline

The default value for reconstruction in the sigmaKappa deglitcher is changed to "DIRECT".
Added:
>
>
A new metadata item, calVersion, is added to all pipeline produced products to specity the calibration tree version used.
 

Photometer Pipeline

Extended Emission Calibration Using the Relative Detector Gains

Line: 457 to 466
 

Spectrometer Pipeline

Changed:
<
<

Averaging Spectra

The Average Spectra module no longer supports the bandEdge parameter. This parameter was used only for the module's defunct outlier rejection functionality, and so it is no longer needed. Please ensure that your scripts do not use this parameter.

>
>

Spectral Unit and Unit-Flexibility

 
Changed:
<
<

Spectrum Flux Calibration

>
>
The standard spectral unit used in the pipeline is now frequency in GHz rather than wavenumber in cm-1. In addition the band limits have been slightly reduced so that instrumental spikes at the edges are not included.
 
Changed:
<
<
  • The Flux Calibration module has been replaced by two modules, the Point Source Flux Calibration module and the Extended Source Flux Calibration module. References to SpecFluxConversionTask should be replaced with SpecPointFluxConversionTask or SpecExtendedFluxConversionTask. Read more in the pipeline specification manual at Spectrum Point Source Flux Calibration and Spectrum Extended Source Flux Calibration.
>
>
For interactive processing, it is now possible to specify the desired spectral unit in the Fourier Transform module. This accepts an optional spectralUnit string parameter to specify the output spectral unit. For example, if spectralUnit='GHz', then the output spectra will be in units of V/GHz versus GHz. If no unit is specified, the output spectra will be in units of V/(1/OPD_UNIT) versus 1/OPD_UNIT (the OPD unit is typically cm).
 
Changed:
<
<
  • The Point Source Flux Calibration module can now be applied to more than just the central detectors.
>
>
To accommodate this new feature, all spectral modules now accept spectra and calibration data products in any unit of wavenumber or frequency. For example, if a module is provided with spectra in units of V/GHz versus GHz and a calibration product in units of cm-1, then the cm-1 calibration product will be converted to GHz, and the output spectra will be in GHz.
 
Changed:
<
<

Spectral Unit and Unit-Flexibility

>
>

Averaging Spectra

 
Changed:
<
<
It is now possible to specify the spectral unit of the initial spectra. The Fourier Transform module now accepts an optional spectralUnit string parameter to specify the desired output spectral unit. For example, if spectralUnit='GHz', then the output spectra will be in units of V/GHz versus GHz. If no unit is specified, the output spectra will be in units of V/(1/OPD_UNIT) versus 1/OPD_UNIT (the OPD unit is typically cm).
>
>
The Average Spectra module no longer supports the bandEdge parameter. This parameter was used only for the module's defunct outlier rejection functionality, and so it is no longer needed. Please ensure that your scripts do not use this parameter.
 
Changed:
<
<
To accommodate this new feature, all spectral modules now accept spectra and calibration data products in any unit of wavenumber or frequency. For example, if a module is provided with spectra in units of V/GHz versus GHz and a calibration product in units of cm-1, then the cm-1 calibration product will be converted to GHz, and the output spectra will be in GHz.
>
>

Spectrum Flux Calibration

The Flux Calibration module has been replaced by two modules, the Point Source Flux Calibration module and the Extended Source Flux Calibration module. References to SpecFluxConversionTask should be replaced with SpecPointFluxConversionTask or SpecExtendedFluxConversionTask. Read more in the pipeline specification manual at Spectrum Point Source Flux Calibration and Spectrum Extended Source Flux Calibration.
 

Bright Source Mode

Changed:
<
<
Two modules have been created for processing bright source mode observations. Both of these modules are in the new herschel.spire.ia.pipeline.spec.gain package:
>
>
The pipeline has been adapted so that observations made using Bright Source mode can be calibrated. The processing sequence has several different steps from the Nominal mode: the bolometer voltages are corrected for a phase gain in the engineering conversion, they are converted to bolometer temperature, and two additional modules to correct for the remaining gain factors (due to the reduced responsivity in bright mode) are added:
 
  • The PCAL Gain module computes the ratio of a measured PCAL flash level separation to the mean PCAL flash separation measured on Dark Sky in Nominal Mode.
Changed:
<
<
  • The Bright Mode Gain module computes and then applies a wavenumber-dependent gain factor for observations which have been made using the Bright Source Mode.
>
>
  • The Bright Mode Gain module computes and then applies a frequency-dependent gain factor.
 

Phase Correction

Line: 502 to 509
 
sdi = inverseFourierTransform(sds = sds)
  • InverseFourierTransformTask is not the inverse of the FourierTransformTask because it does not remove zeropadding from the output interferograms.
Added:
>
>

Improved pre-processed cube product

The "preCube" product which contains the collection of spectra to be gridded into the final cube has been improved and now contains a dataset in Spectrum2d format, that can easily be viewed with Herschel viewers.
 

Convenience task for spectral cube creation

Changed:
<
<
  • SpireProjectionTask has been created to provide a convenient single API for creating spectral cubes.
  • The task can make spectral cubes using any of the following algorithms:
>
>
SpireProjectionTask has been created to provide a convenient single API for creating spectral cubes. The task can make spectral cubes using any of the following algorithms:
 
    • Naive projection
    • Nearest neighbour projection
    • Gridding projection

Interactive Analysis and Tools

Changed:
<
<

Photometer Destriper

>
>

Useful Scripts

A new menu item, Scripts has been added on the HIPE toolbar to contain "useful" scripts. Scripts have been added for the Photometer and Spectrometer (see below).

Photometer Destriper
 THe Destriper example use is now contained in a new Useful Script accessed from the top Scripts menu within HIPE The following new features are introduced in Hipe 8:
  • Level 2 deglitching is available through setting l2DeglitchRepeat parameter.
Line: 529 to 541
 
  • The input parameter name "Tod" changed to "tod".
  • The input parameter name "L2DeglitchRepeat" changed to "l2DeglitchRepeat".
Changed:
<
<

Photometer Baseline Removal

>
>
Photometer Baseline Removal
 THe Median and Polynomial Baseline Removal example use is now contained in a new Useful Script accessed from the top Scripts menu within HIPE
  • New "roi" (region of interest) parameter for baselineRemoval* tasks, specifying the region of the sky to be used in estimating the baseline
  • Removed "doInterpolation" task parameter
  • Minor changes and bug fixes (for example, the tasks now do ignore masked data samples when a TodBuffer is used internally)
Changed:
<
<

Photometer Moving Object Correction Script

>
>
Photometer Moving Object Correction Script
 A new Useful Script accessed from the top Scripts menu within HIPE:
  • Takes a SSO observation and corrects the map to be in the moving object frame
  • i.e. target is no longer a streak on a stable background.
Changed:
<
<

Photometer SPIRE Astrometry Alignment

>
>
Photometer SPIRE Astrometry Alignment
 A new Useful Script accessed from the top Scripts menu within HIPE:
  • Takes a reference SPIRE map and a SPIRE map to be corrected and aligns the astrometry of the latter to the former
  • Takes a reference SPIRE map and a list of SPIRE maps to be corrected and aligns the astrometry of the latter to the former
Line: 548 to 560
 
  • Takes a referencesource list (as a sourceListProduct) and a SPIRE map to be corrected and aligns the astrometry of the latter to the former
  • In all cases both the Level 2 map astrometry and Level 1 timelines are corrected
Added:
>
>
Spectrometer Array Footprint Plot
A new Useful Script accessed from the top Scripts menu within HIPE:
  • Plots the locations of the detectors on top of an image of the source.

Spectrometer Background Subtraction
A new Useful Script accessed from the top Scripts menu within HIPE:
  • Provides two methods to subtract residual background from the final spectrum:
    • using the daily dark observation
    • using off-axis detectors
This is particularly useful for faint sources.

Spectrometer Line Fitting
A new Useful Script accessed from the top Scripts menu within HIPE:
  • Allows simultaneous fitting of lines and continuum in the central detectors using the HIPE SpectrumFitter
  • CO lines are fitted with a polynomial baseline and results are printed for each line

Spectrometer Thumbnail Mosaic Plot
A new Useful Script accessed from the top Scripts menu within HIPE:
  • Gives several examples of plotting the spectra from each detector as a thumbnail arranged as the detectors appear in the array.

SDI/SDS Explorer

The SDI and SDS Explorer tools have been updated to:
  • Ability to plot errors as a shaded region on the spectrum added
  • Right click options added for each detector in the array footprint, including Open With, Send To and Create Variable.
  • Zooming is now permitted in the thumbnail view.
 

Documentation

Changed:
<
<
  • New SPIRE Launch Pads to allow Users an overview of the SPIRE processing, whats available and quick link jumps into the documentation.
>
>
  • New SPIRE Launch Pads to give Users an overview of the SPIRE processing, whats available, and quick link jumps into the documentation.
 
  • A major revision of the SPIRE Data Reduction Guide for both Photometer and Spectrometer Chapters.
  • SPIRE Data Reduction Guide restructured to be "recipe" orientated
Changed:
<
<
  • Includes description of new Useful Scripts accessed from the Scripts menu in HIPE 8
>
>
  • Includes description of new Useful Scripts accessed from the Scripts menu in HIPE
 
  • SPIRE Data Reduction Guide includes the manual for mapmaking with the SPIRE Photometer Interactive Analysis (SPIA) plug-in

Revision 612011-12-01 - AdwinBoogert

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 267 to 267
 

HIFI Data Processing Tools

Standing Wave Removal

Changed:
<
<
  • Average mask option in FitHifiFringe introduced.
    • Allows a line mask to be calculated based on the average of all scans. Makes fringe removal from maps much more efficient.
>
>
  • avermask option in FitHifiFringe introduced.
    • Allows a line mask to be calculated using the average of all scans. Makes fringe removal from maps much more efficient.
 
Changed:
<
<
  • Variable result_sds should no longer created if FitHifiFringe is run from the GUI with obs_or_htp input
>
>
  • Variable result_sds should no longer created if FitHifiFringe is run from the GUI with obs_or_htp input
 
  • When an Observation Context is supplied standing waves are now automatically removed from both upper and lower sidebands.

Baseline Removal

Changed:
<
<
  • FitBaseline now operates in interactive mode as default
>
>
  • FitBaseline now operates by default in interactive mode.
 
Changed:
<
<
  • The automask check box now only has "True" or "False" options, previously the grey box "ignore" was operational and could allow you to misunderstand what you were asking the GUI to do.

  • You can now enter the back-end to be corrected directly into the GUI.
>
>
  • The back-end(s) to be processed can now be selected by clicking in a list.
 
  • FitBaseline 'divide' option now handles negative signals robustly.

Revision 602011-12-01 - DavidShupe

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 555 to 555
 
  • A major revision of the SPIRE Data Reduction Guide for both Photometer and Spectrometer Chapters.
  • SPIRE Data Reduction Guide restructured to be "recipe" orientated
  • Includes description of new Useful Scripts accessed from the Scripts menu in HIPE 8
Added:
>
>
  • SPIRE Data Reduction Guide includes the manual for mapmaking with the SPIRE Photometer Interactive Analysis (SPIA) plug-in
 

Revision 592011-12-01 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 189 to 189
 
    • Completely revamped the organisation of the manual. Improved section titles and reduced the number of levels in the structure, for easier access.
    • Added note on Mac-PC key mapping.
    • Added link to a video tutorial to create a simple plug-in.
Added:
>
>
 
Added:
>
>
 
Added:
>
>
 
Added:
>
>
 
Added:
>
>
 

Revision 582011-12-01 - ChrisPearson

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 429 to 429
 
  • The Spectrometer Bright Mode Gain calibration product contains the intercept and gradient for the wavenumber-dependent gain. This calibration product is used by the new Bright Mode Gain module.
Added:
>
>

 

Calibration Framework

Preprocessing

Line: 440 to 444
 

Photometer Pipeline

Added:
>
>

Extended Emission Calibration Using the Relative Detector Gains

A new optional module in the User Pipelines that takes into acount the fact that the SPIRE detectors dont all have exactly the same beamshape
  • The Flag applyExtendedEmissionGains needs to be set to True

Include Turnaround Data for Maps

Extra area / coverage can be obtained by including the scan line turnarounds in the processing
  • The Flag includeTurnaround needs to be set to True
 

Spectrometer Pipeline

Averaging Spectra

Line: 496 to 508
 

Interactive Analysis and Tools

Changed:
<
<

Destriper

>
>

Photometer Destriper

THe Destriper example use is now contained in a new Useful Script accessed from the top Scripts menu within HIPE
 The following new features are introduced in Hipe 8:
  • Level 2 deglitching is available through setting l2DeglitchRepeat parameter.
  • The 5 individual output parameters are combined to one ArrayList parameter where each of 5 parameter can be retrieved base on its position.
Line: 513 to 526
 
  • The input parameter name "Tod" changed to "tod".
  • The input parameter name "L2DeglitchRepeat" changed to "l2DeglitchRepeat".
Changed:
<
<

Baseline Removal

>
>

Photometer Baseline Removal

THe Median and Polynomial Baseline Removal example use is now contained in a new Useful Script accessed from the top Scripts menu within HIPE
 
  • New "roi" (region of interest) parameter for baselineRemoval* tasks, specifying the region of the sky to be used in estimating the baseline
  • Removed "doInterpolation" task parameter
  • Minor changes and bug fixes (for example, the tasks now do ignore masked data samples when a TodBuffer is used internally)
Added:
>
>

Photometer Moving Object Correction Script

A new Useful Script accessed from the top Scripts menu within HIPE:
  • Takes a SSO observation and corrects the map to be in the moving object frame
  • i.e. target is no longer a streak on a stable background.

Photometer SPIRE Astrometry Alignment

A new Useful Script accessed from the top Scripts menu within HIPE:
  • Takes a reference SPIRE map and a SPIRE map to be corrected and aligns the astrometry of the latter to the former
  • Takes a reference SPIRE map and a list of SPIRE maps to be corrected and aligns the astrometry of the latter to the former
  • Takes a reference ancillary map and a SPIRE map to be corrected and aligns the astrometry of the latter to the former
  • Takes a referencesource list (as a sourceListProduct) and a SPIRE map to be corrected and aligns the astrometry of the latter to the former
  • In all cases both the Level 2 map astrometry and Level 1 timelines are corrected
 

Documentation

Added:
>
>
  • New SPIRE Launch Pads to allow Users an overview of the SPIRE processing, whats available and quick link jumps into the documentation.
  • A major revision of the SPIRE Data Reduction Guide for both Photometer and Spectrometer Chapters.
  • SPIRE Data Reduction Guide restructured to be "recipe" orientated
  • Includes description of new Useful Scripts accessed from the Scripts menu in HIPE 8
 

Revision 572011-12-01 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 177 to 177
 

Installer

Changed:
<
<
Sometimes, especially when a user has a shaky internet connection, an installation might fail in such a way that the local (in the user's machine) repository of hcss libraries might get corrupted. The workaround that existed so far can be found in the known issued page in: http://herschel.esac.esa.int/twiki/bin/view/Hcss/KnownIssuesHipeInstaller#Installer_fails_with_a_zip_error. The solution in short was to remove the local repository and all existing HIPE installation and start over. However a new solution that hopefully solves this issue has been implemented in 8.0. We know that bad proxy setup, hardware issues in routers can cause packet loss even in TCP. Such loss should always result in a file which is smaller than actually intended. Normally such a situation should result in an exception, but with packet loss in e.g. a router this will go unnoticed and a normal EOF completes the transmission. However, the resulting ZIP file will then always fail to unzip. The installer has been modified to extract the content-length from the HTTP header and validate the number of bytes actually received and raise an exception if such thing takes place. Moreover, a better exception reporting has been implemented to give more information on the basic cause of the exception. Finally the user installer reports the estimated time of completion while installing to give a rough feeling to the user.
>
>
  • Sometimes, especially with an unstable internet connection, an installation might fail in such a way that your local repository of hcss libraries gets corrupted. The installer has been modified to give an error if such thing takes place. Moreover, a better error reporting has been implemented to give more information on the cause of the error.
  • The user installer now reports the estimated time of completion.
 

Documentation

Revision 562011-12-01 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 66 to 66
 TablePlotter can display plot with Date/Time format.

OverPlotter

OverPlotter can display plot with Date/Time format.
Deleted:
<
<

Dataset viewer

 

Astronomical utilities

  • Planck class moved from spire_ia_pipeline_spec_util to ia_toolbox_astro. If you are doing an explicit import (it is not needed) you have to change it to (preferred) import herschel.ia.toolbox.astro.Planck or from herschel.ia.toolbox.astro import Planck
Line: 159 to 158
 

Quality control

Deleted:
<
<

Systematic product generation

 

Data input-output

  • URM entries (documentation) improved
  • Task importUFDirToPal renamed to importObservation

Revision 552011-11-24 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 251 to 251
 
  • Updated browse image for point mode observations.
Added:
>
>
  • Noticeable improvement in the noise and baseline quality of the level2 products of Spectral Scans performed in bands 6 and 7. This is due to a fix to the procedure that did combine the ON and OFF phases, which was not done optimally in earlier versions of the pipeline.
 

HIFI Spectrum Toolbox

Spectrum Explorer (HIFI specifics)

Spectrum tasks (HIFI specifics)

Revision 542011-11-18 - ZoltanBalog

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 416 to 416
 

Documentation

  • PACS Data Reduction Guide. Chp 1 is now a collection of Launch Pads: on getting Data into HIPE, doing your PACS Spectroscopy and doing your PACS Photometry. These are quick-start guides to working with your PACS observations. Chp2 tells you how to load your data, your pipeline script, and your calibration tree into HIPE, for photometry and spectroscopy both.
  • Spectroscopy: the pipeline chapters are being re-organised to follow the new pipeline scripts, and so for this 8.0 RC4 version of the PDRG only the LineScan Chop-Nod pipeline is described.
Added:
>
>
  • Photometry: the pipeline chapters are being re-organised to follow the new pipeline scripts, and so for this 8.0 RC4 version of the PDRG only the ScanMap pipeline scripts are described.
 

SPIRE

Revision 532011-11-15 - BrunoAltieri

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 394 to 394
 
 from herschel.pacs.gui.maskviewer import MaskViewer  
Added:
>
>
  • A new tasks MaskFromCatalogue is available, very convenient to mask a list of point sources for high-pass filtering in deep fields.
 

Spectroscopy

  • GUIs are now available to work on spectral data: SpectrumExplorer and spectral footprint

Revision 522011-11-15 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 165 to 165
 
  • URM entries (documentation) improved
  • Task importUFDirToPal renamed to importObservation
  • Task exportPalToUFDir renamed to exportObservation
Changed:
<
<
  • Task simpleAsciiTableReader extended to support all options of asciiTableReader via Advanced tab. (eventually, asciiTableReader will subsume simpleAsciiTableReader)
>
>
  • Task simpleAsciiTableReader extended to support all options of asciiTableReader via Advanced tab.
 
  • Improved type guessing for table reading (affects (simple)AsciiTableReader): for floating-point data it will detect precision loss and maintain representation. If a floating-point number will lose precision as a float (32-bits) a double (64-bits) will be used instead: up to 17 significant digits but in these case it will use twice the memory.
  • Improved documentation and information provided by getObservation
  • saveObservation now reports the progress in the status bar and can be interrupted/stopped.

Revision 512011-11-14 - BrunoAltieri

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 326 to 326
 

PACS

Pipeline

Added:
>
>
 

Photometry

Added:
>
>
  • a new task is used for the deglitching less memory intensive MapDeglitch instead of IIndLevelDeglitch
  • the 2nd level glitch mask is now stored in level 1.
  • new task filterOnScanSpeed now used to select frames at constant speed instead of the BBID selection
  • new task photMaskCrosstalk, masking the column 0 of all matrices now used in the pipeline too. This is a bit of brute force to avoid negative or positive cross-talk artifact in teh level 2 maps, at the expense of getting rid of 1/16th of the data.
 

Spectroscopy

  • interactive reduction scripts for all observing modes are now available
  • flatfielding is available in the interactive pipeline scripts
Line: 341 to 349
 

Photometry

Added:
>
>
  • A new calibration file is available for non linearity correction required by the task photNonLinearityCorrection.
 

Spectroscopy

  • major improvements of the flux calibration:
    • implementation of the background normalization method for chop/nod observations
Line: 392 to 402
 
    • The tooltip of the slice shows the complete list of slice info keywords.

Product changes

Added:
>
>
 

Photometry

Added:
>
>
  • a new level 2.5 product is available combining photProject scan and cross-scan maps.
 

Spectroscopy

  • Browse products and browse image products are now available in spg
  • Red and blue camera spectra can now be combined in a single context in the level-2 processing, therefore, the lineId has changed for the red camera: 100 is added!

Revision 502011-11-14 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 398 to 398
 
  • Red and blue camera spectra can now be combined in a single context in the level-2 processing, therefore, the lineId has changed for the red camera: 100 is added!

Documentation

Added:
>
>
  • PACS Data Reduction Guide. Chp 1 is now a collection of Launch Pads: on getting Data into HIPE, doing your PACS Spectroscopy and doing your PACS Photometry. These are quick-start guides to working with your PACS observations. Chp2 tells you how to load your data, your pipeline script, and your calibration tree into HIPE, for photometry and spectroscopy both.
  • Spectroscopy: the pipeline chapters are being re-organised to follow the new pipeline scripts, and so for this 8.0 RC4 version of the PDRG only the LineScan Chop-Nod pipeline is described.
 

SPIRE

Revision 492011-11-14 - JuergenSchreiber

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 328 to 328
 

Pipeline

Photometry

Spectroscopy

Added:
>
>
  • interactive reduction scripts for all observing modes are now available
  • flatfielding is available in the interactive pipeline scripts
  • error propagation (and error bars) is implemented in the PACS rebinned cubes (STDDEV dataset)
  • spg pipeline: the wavelength switching pipeline is merged with the unchopped pipeline and removed
  • improved documentation and user friendliness of the pipeline tasks within hipe
 

Calibration

Changed:
<
<
The calibration sets that are installed on your local system can now be inspected with a 'Calibration Sets Viewer'. The viewer is located in the menu Window -> Show View -> Workbench -> Calibration Sets. Select a version from the available buttons, the you can inspect the release note for that version or the product list with the versions of the individual calibration products.
>
>
  • The calibration sets that are installed on your local system can now be inspected with a 'Calibration Sets Viewer'. The viewer is located in the menu Window -> Show View -> Workbench -> Calibration Sets. Select a version from the available buttons, the you can inspect the release note for that version or the product list with the versions of the individual calibration products.
 

Photometry

Spectroscopy

Changed:
<
<
Flux calibration now based on observations of astronomical standards and the calibration block attached to each observation. The pipeline task specDiffCs computes the detector response of the current observation.
>
>
  • major improvements of the flux calibration:
    • implementation of the background normalization method for chop/nod observations
    • it is now based on observations of astronomical standards and the calibration block attached to each observation. The pipeline task specDiffCs computes the detector response of the current observation.
    • implementation of a correction of transients for unchopped observations

  • the spectral (pixel) dependency of the spatial calibration is now available for each pixel
 

Analysis tools

Changed:
<
<
The Preferences now contain a Logging panel under the general tab which allows the user/developer to increase or decrease the number of logging messages send to the
>
>
  • The Preferences now contain a Logging panel under the general tab which allows the user/developer to increase or decrease the number of logging messages send to the
 terminal.
Changed:
<
<
The calibration file version is now also shown in the product history.
>
>
  • The calibration file version is now also shown in the product history.
 

Photometry

Line: 376 to 386
 

Spectroscopy

Added:
>
>
  • GUIs are now available to work on spectral data: SpectrumExplorer and spectral footprint
  • implementation of user friendly viewer for the PACS specific part of the ObservationContext:
    • Each spectrometer slice now shows after the slice number a short description of its contents
    • The tooltip of the slice shows the complete list of slice info keywords.
 

Product changes

Photometry

Spectroscopy

Added:
>
>
  • Browse products and browse image products are now available in spg
  • Red and blue camera spectra can now be combined in a single context in the level-2 processing, therefore, the lineId has changed for the red camera: 100 is added!
 

Documentation

Revision 482011-11-14 - EkkehardWieprecht

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 334 to 334
 The calibration sets that are installed on your local system can now be inspected with a 'Calibration Sets Viewer'. The viewer is located in the menu Window -> Show View -> Workbench -> Calibration Sets. Select a version from the available buttons, the you can inspect the release note for that version or the product list with the versions of the individual calibration products.

Photometry

Deleted:
<
<

Spectroscopy

 
Added:
>
>

Spectroscopy

 Flux calibration now based on observations of astronomical standards and the calibration block attached to each observation. The pipeline task specDiffCs computes the detector response of the current observation.

Analysis tools

Changed:
<
<
The Preferences now contain a Logging panel under the general tab which allows the user/developer to increase or decrease the number of logging messages send to the terminal.
>
>
The Preferences now contain a Logging panel under the general tab which allows the user/developer to increase or decrease the number of logging messages send to the terminal.

The calibration file version is now also shown in the product history.

 

Photometry

Added:
>
>
MapMonitor got an improved interface and allows to flip north up-or downwards.

PhotProject got a companion task MiProject (MapIndex-Project). If you have a full MapIndex, you may use MiProject instead of PhotProject. It needs frames and the MapIndex as parameters and speeds up projection time by a factor of 20 (30, 40?). Whatever the number is: its much faster than PhotProject. Use as:

 map = miProject(frames, mapindex)  

A new Task has been developed. It deglitches without MapIndex and runs with significantly less memory abuse than the IIndLevelDeglitchTask. The name and package is

 herschel.pacs.spg.phot.deglitching.map.MapDeglitchTask 
. The hipe alias is mapDeglitch. It is documented as javadoc, URM and on the online deglitching documentation page:
http://herschel.esac.esa.int/twiki/bin/view/Public/SecondLevelDeglitching#MapDeglitchTask

A new Task has been implemented which removes crosstalk effects by masking (CrossTalkMask ) the first rows of detector arrays :

 frames = photMaskCrosstalk(frames) 

The HIPE Menu Pipeline -> PACS -> has been re organized and ipipe scripts added.

Automatic pipeline scripts save the Products at the end in the Observation Context also in IA sessions. That mean the scripts can be executed in a sequence without interaction.

Pacs Products (for example Frames, Ramps, PacsCube) got a method to rename a Mask :

 renameMask(String oldname, String newname) 

up to now, a mask had to be removed and created new, if the name had to be changed.

PhotProject, MiProject, IIndLevelDeglitching and some other tasks report their progress to the hipe progressbar instead of the terminal output.

The effects of 2-bit rounding on second level deglitching have been reduced in the pipeline for blue parallel maps.

The Pacs MaskViewer has been relocated to a new package:

 herschel.pacs.gui.maskviewer.MaskViewer 
. If you manually imported the old location (from herschel.pacs.signal import MaskViewer), the import will fail. Please just remove the import. Everything should work fine without further action. If there should be a problem nevertheless, please import the new location:
 from herschel.pacs.gui.maskviewer import MaskViewer  

 

Spectroscopy

Product changes

Revision 472011-11-14 - RikHuygen

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 168 to 168
 
  • Task simpleAsciiTableReader extended to support all options of asciiTableReader via Advanced tab. (eventually, asciiTableReader will subsume simpleAsciiTableReader)
  • Improved type guessing for table reading (affects (simple)AsciiTableReader): for floating-point data it will detect precision loss and maintain representation. If a floating-point number will lose precision as a float (32-bits) a double (64-bits) will be used instead: up to 17 significant digits but in these case it will use twice the memory.
  • Improved documentation and information provided by getObservation
Added:
>
>
  • saveObservation now reports the progress in the status bar and can be interrupted/stopped.
 
  • Added contextual menu option to Variables View to save tables with asciiTableWriter. Also added a validator to the task that will restrict its applicability to TableDatasets that can be fit into an ASCII table.

Virtual Observatory

Revision 462011-11-14 - SteveLord

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 293 to 293
 
  • The min chi squared and the number of iterations are now included in the meta data of the SSB solution
Changed:
<
<
  • Bug fix: potential lines shifts resulting from rounding errors in doDeconvolution.
>
>
  • Bug fix: In doDeconvolution an algorithmic error in binning the input data has been corrected. The fix has stopped the occurrence of two problems: Some band-edge channels had fluxes set to zero and some very narrow lines had their peaks shifted by as much 1/2 MHz. Both errors are fixed.
 

Spectral Cube creation

  • BeamSize is now taken from the calibration tree if "hpbw" is in the metadata

Revision 452011-11-14 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 168 to 168
 
  • Task simpleAsciiTableReader extended to support all options of asciiTableReader via Advanced tab. (eventually, asciiTableReader will subsume simpleAsciiTableReader)
  • Improved type guessing for table reading (affects (simple)AsciiTableReader): for floating-point data it will detect precision loss and maintain representation. If a floating-point number will lose precision as a float (32-bits) a double (64-bits) will be used instead: up to 17 significant digits but in these case it will use twice the memory.
  • Improved documentation and information provided by getObservation
Changed:
<
<
  • Added contextual menu option to Variables View to save tables with asciiTableWriter
>
>
  • Added contextual menu option to Variables View to save tables with asciiTableWriter. Also added a validator to the task that will restrict its applicability to TableDatasets that can be fit into an ASCII table.
 

Virtual Observatory

Revision 442011-11-11 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 39 to 39
 
Jython & Text Editors
  • The supported Jython language has been upgraded from version 2.1 to 2.5.2.
  • Scripts and text files coming with the installation or with plug-ins are safe to edit without losing the original version.
Added:
>
>
  • Improved the step by step execution of scripts. See the note on this page for more details.
 

Help & Debug

  • The Tip of the day contains information on the Console meta commands (those starting with !)

Revision 432011-11-07 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
Changed:
<
<

What's coming up in HIPE 8

>
>

What's coming up in HIPE 8

 
PDF Version Click here
Line: 10 to 10
 

Core system

Changed:
<
<
Jython major upgrade. The version of Jython on which HIPE is based has been upgraded from 2.1 to 2.5.2. You may have to adapt your scripts for them to work with the new version. See this page for the details.
>
>
Jython major upgrade. The version of Jython on which HIPE is based has been upgraded from 2.1 to 2.5.2. You may have to adapt your scripts for them to work with the new version.

ALERT! See this page for details on changes from Jython 2.1 to 2.5.2 that could affect your scripts.
 

HIPE

Revision 422011-11-04 - PeterKennedy

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 410 to 410
 
    • phaseWrapSLW/SSW counts the number of times that in-band phase changes by +/-1.5pi in any of the nominal SLW/SSW detectors.
    • residualPhaseSLW/SSW gives the maximum absolute phase value (in units of degree) inside the phase correction limits in any of the nominal SLW/SSW detectors.
Added:
>
>

Create SDI from SDS

  • InverseFourierTransformTask has been created to transform SDS objects into SDI objects by applying the inverse Fourier transform:
    sdi = inverseFourierTransform(sds = sds)
  • InverseFourierTransformTask is not the inverse of the FourierTransformTask because it does not remove zeropadding from the output interferograms.

Convenience task for spectral cube creation

  • SpireProjectionTask has been created to provide a convenient single API for creating spectral cubes.
  • The task can make spectral cubes using any of the following algorithms:
    • Naive projection
    • Nearest neighbour projection
    • Gridding projection
 

Interactive Analysis and Tools

Destriper

Revision 412011-11-04 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 180 to 180
 

Documentation

Added:
>
>
 

HIFI

HIFI Software Configuration and Properties

Revision 402011-11-03 - DavidSharpe

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 326 to 326
 

Calibration Products

Changed:
<
<
The Spectrometer Beam Parameters calibration product used by the Point Source Flux Conversion now contains the wavenumber-dependent point-source flux conversion factors for each spectrometer detector. Previously, this product contained factors only for each spectrometer array.
>
>
  • The Spectrometer Beam Parameters calibration product used by the Point Source Flux Conversion now contains the wavenumber-dependent point-source flux conversion factors for each spectrometer detector. Previously, this product contained factors only for each spectrometer array.
 
Changed:
<
<
The Spectrometer Nominal PCAL calibration product contains the weighted mean and standard deviation PCAL level separation for dark sky in Nominal Mode. This calibration product is used by the new PCAL Gain module.
>
>
  • The Spectrometer Nominal PCAL calibration product contains the weighted mean and standard deviation PCAL level separation for dark sky in Nominal Mode. This calibration product is used by the new PCAL Gain module.
 
Changed:
<
<
The Spectrometer Bright Mode Gain calibration product contains the intercept and gradient for the wavenumber-dependent gain. This calibration product is used by the new Bright Mode Gain module.
>
>
  • The Spectrometer Bright Mode Gain calibration product contains the intercept and gradient for the wavenumber-dependent gain. This calibration product is used by the new Bright Mode Gain module.
 

Calibration Framework

Line: 351 to 351
 

Spectrum Flux Calibration

Changed:
<
<
The Flux Calibration module has been replaced by two modules, the Point Source Flux Calibration module and the Extended Source Flux Calibration module. References to SpecFluxConversionTask should be replaced with SpecPointFluxConversionTask or SpecExtendedFluxConversionTask. Read more in the pipeline specification manual at Spectrum Point Source Flux Calibration and Spectrum Extended Source Flux Calibration.
>
>
  • The Flux Calibration module has been replaced by two modules, the Point Source Flux Calibration module and the Extended Source Flux Calibration module. References to SpecFluxConversionTask should be replaced with SpecPointFluxConversionTask or SpecExtendedFluxConversionTask. Read more in the pipeline specification manual at Spectrum Point Source Flux Calibration and Spectrum Extended Source Flux Calibration.
 
Changed:
<
<
The Point Source Flux Calibration module can now be applied to more than just the central detectors.
>
>
  • The Point Source Flux Calibration module can now be applied to more than just the central detectors.
 

Spectral Unit and Unit-Flexibility

Changed:
<
<
It is now possible to specify the spectral unit of the initial spectra. The Fourier Transform module (FourierTransformTask) now accepts an optional spectralUnit string parameter to specify the desired output spectral unit. For example, if spectralUnit='GHz', then the output spectra will be in units of V/GHz versus GHz. If no unit is specified, the output spectra will be in units of V/(1/OPD_UNIT) versus 1/OPD_UNIT (the OPD unit is typically cm).
>
>
It is now possible to specify the spectral unit of the initial spectra. The Fourier Transform module now accepts an optional spectralUnit string parameter to specify the desired output spectral unit. For example, if spectralUnit='GHz', then the output spectra will be in units of V/GHz versus GHz. If no unit is specified, the output spectra will be in units of V/(1/OPD_UNIT) versus 1/OPD_UNIT (the OPD unit is typically cm).
  To accommodate this new feature, all spectral modules now accept spectra and calibration data products in any unit of wavenumber or frequency. For example, if a module is provided with spectra in units of V/GHz versus GHz and a calibration product in units of cm-1, then the cm-1 calibration product will be converted to GHz, and the output spectra will be in GHz.

Bright Source Mode

Changed:
<
<
Two modules have been created for processing bright source mode observations: the PCAL Gain module (SpecApplyPcalGainTask) and the Bright Mode Gain module (SpecApplyBrightGainTask). Both of these modules are in the new herschel.spire.ia.pipeline.spec.gain package. The PCAL Gain module computes the ratio of a measured PCAL flash level separation to the mean PCAL flash separation measured on Dark Sky in Nominal Mode. The Bright Mode Gain module computes and then applies a wavenumber-dependent gain factor for observations which have been made using the Bright Source Mode.
>
>
Two modules have been created for processing bright source mode observations. Both of these modules are in the new herschel.spire.ia.pipeline.spec.gain package:
  • The PCAL Gain module computes the ratio of a measured PCAL flash level separation to the mean PCAL flash separation measured on Dark Sky in Nominal Mode.
  • The Bright Mode Gain module computes and then applies a wavenumber-dependent gain factor for observations which have been made using the Bright Source Mode.

Phase Correction

  • The Phase Correction module's interface has been simplified. As before, the module provides three ways of applying phase correction, but executing the module is simpler now:
    1. From the observation data:
      sdi = phaseCorrection(sdi=sdi, avgSdi=avgSdi)
    2. From a calibration file:
      sdi = phaseCorrection(sdi=sdi, nonLinPhase=nonLinPhase)
    3. From a polynomial fit to the observation:
      sdi = phaseCorrection(sdi=sdi, sds=dsds, phaseCorrLim=phaseCorrLim,\
            polyDegree=2, pcfSize=64, convolApodName="aNB_15")
            polynomial fit to the observation

  • The Phase Correction module creates four new quality control metadata on the phase-corrected interferogram. These values will be used by the Quality Control module to assess observations:
    • phaseWrapSLW/SSW counts the number of times that in-band phase changes by +/-1.5pi in any of the nominal SLW/SSW detectors.
    • residualPhaseSLW/SSW gives the maximum absolute phase value (in units of degree) inside the phase correction limits in any of the nominal SLW/SSW detectors.
 

Interactive Analysis and Tools

Revision 392011-11-03 - DavidSharpe

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 326 to 326
 

Calibration Products

Added:
>
>
The Spectrometer Beam Parameters calibration product used by the Point Source Flux Conversion now contains the wavenumber-dependent point-source flux conversion factors for each spectrometer detector. Previously, this product contained factors only for each spectrometer array.

The Spectrometer Nominal PCAL calibration product contains the weighted mean and standard deviation PCAL level separation for dark sky in Nominal Mode. This calibration product is used by the new PCAL Gain module.

The Spectrometer Bright Mode Gain calibration product contains the intercept and gradient for the wavenumber-dependent gain. This calibration product is used by the new Bright Mode Gain module.

 

Calibration Framework

Preprocessing

Line: 347 to 353
  The Flux Calibration module has been replaced by two modules, the Point Source Flux Calibration module and the Extended Source Flux Calibration module. References to SpecFluxConversionTask should be replaced with SpecPointFluxConversionTask or SpecExtendedFluxConversionTask. Read more in the pipeline specification manual at Spectrum Point Source Flux Calibration and Spectrum Extended Source Flux Calibration.
Added:
>
>
The Point Source Flux Calibration module can now be applied to more than just the central detectors.
 

Spectral Unit and Unit-Flexibility

It is now possible to specify the spectral unit of the initial spectra. The Fourier Transform module (FourierTransformTask) now accepts an optional spectralUnit string parameter to specify the desired output spectral unit. For example, if spectralUnit='GHz', then the output spectra will be in units of V/GHz versus GHz. If no unit is specified, the output spectra will be in units of V/(1/OPD_UNIT) versus 1/OPD_UNIT (the OPD unit is typically cm).

Revision 382011-11-02 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 100 to 100
 

Spectra

Display

Analysis

Added:
>
>
  • The SpectrumFitter now recognizes a CUBE such that the user can give the x,y of the spectrum. The script as written by the SpectrumFitterGUI also takes the correct spectrum.
 
Changed:
<
<
The SpectrumFitter now recognizes a CUBE such that the user can give the x,y of the spectrum. The script as written by the SpectrumFitterGUI also takes the correct spectrum.
>
>
  • avg now warns if wavescales are not sufficiently aligned to get a good result
 
Added:
>
>
  • various changes when applying toolbox tasks to cubes, in particular avg, select and extract now allow you to work on several pixels/spaxels within the cube rather than only the entire cube.

  • resampling task now allows alternative ways to specify the grid for resampling.
 

Data cubes

Display

Analysis

Line: 181 to 185
 

HIFI Software Configuration and Properties

HIFI Calibration

Added:
>
>
  • Updates to Calibration tree (HICAL-30)
    • Spur table (7b)
    • FPU HK Thresholds (5a) (Prevents unnecessary Quality Flags)
    • Units added to columns (Prevents unnecessary Quality Flags)
    • Wbs Bad Pixel (saturated regions) (Causes necessary Quality Flags)
    • QC flags for bad LOU currents (Causes necessary Quality Flags)
    • Uplink auxiliary product parameter filtering (First steps in ensuring information about the way the observation was carried out is put in the meta data)
 

HIFI Pipeline

hifiPipeline task

Added:
>
>
  • Can now select calibration version to reprocess data with (command line only)

  • By default the pipeline now runs without saving products. Previously it always wrote output to .hcss/lstore/pipeline-out, which in theory allowed users to access processed observations from that location. This was rarely used and could cause disk space problems.

  • Improvements to tooltips in the Pipeline GUI

 

Level 0 Pipeline

Added:
>
>
  • DoUplink now uses the new uplink auxiliary product to set some observation parameters in the meta data, e.g., mapReadoutSep and mapLineStep

  • Suppression of unneeded Quality Flags: Quality check for mixer voltage now accounts for magnet tuning.

 

WBS Pipeline

HRS Pipeline

Level 1 Pipeline

Added:
>
>
  • New (optional) pipeline step allowing smooth band pass to eliminate standing waves: DoFilterLoadsTask. This is an alternative calibration scheme for strong continuum sources.

  • Substantial refactoring of CheckFrequencyGrid to correct assumptions made about the sequence of on-, off- and hc-datasets. This primarily affected HEB band data and resulted in unnecessary Quality Flags being raised.
 

Level 2 Pipeline

Added:
>
>
  • Sideband gains coefficients added to meta data
 

HIFI Products

Added:
>
>
  • Meta data calVersion is applied to all calibrated products and has format HIFI_CAL_version_number

  • Updated browse image for point mode observations.
 

HIFI Spectrum Toolbox

Spectrum Explorer (HIFI specifics)

Spectrum tasks (HIFI specifics)

Added:
>
>
  • Introduction of FlagToolTask, which allows you go through every subband of a SpectrumDataset and interactively flag spurs, lines, bright lines, and data to ignore. Input data can be an Observation context, a HifiTimelineProduct, or a SpectrumDataset

  • PolarPair is now a task

 

HIFI Data Processing Tools

Standing Wave Removal

Added:
>
>
  • Average mask option in FitHifiFringe introduced.
    • Allows a line mask to be calculated based on the average of all scans. Makes fringe removal from maps much more efficient.

  • Variable result_sds should no longer created if FitHifiFringe is run from the GUI with obs_or_htp input

  • When an Observation Context is supplied standing waves are now automatically removed from both upper and lower sidebands.

 

Baseline Removal

Added:
>
>
  • FitBaseline now operates in interactive mode as default

  • The automask check box now only has "True" or "False" options, previously the grey box "ignore" was operational and could allow you to misunderstand what you were asking the GUI to do.

  • You can now enter the back-end to be corrected directly into the GUI.

  • FitBaseline 'divide' option now handles negative signals robustly.

 

Deconvolution

Added:
>
>
  • Some output from doDeconvolution is now written to the console as it runs - helpful to know HIPE has not crashed!

  • doDeconvolution now picks up the applied sideband gains from the meta data and "undoes" their effect on the incoming DSB data. Then the user running the deconvolution can then apply gains on or gains off. The pipeline supplied gains can in this way offer an optional starting point for the gain iteration inside the deconvolution.
    • This only has practical application for band 2 data at present.

  • New redundancy product allows you to overplot redundancy on the SSB solution. Helpful to identify where the SSB solution may not be reliable.

  • SSB spectrum now has y-axis labels "Antenna Temperature (K)", x-axis label changed to "Frequency (GHz)"

  • When diagnostic mode is turned on, doDeconvolution now produces an interim product in addition to the pop-up plots.

  • The min chi squared and the number of iterations are now included in the meta data of the SSB solution

  • Bug fix: potential lines shifts resulting from rounding errors in doDeconvolution.

 

Spectral Cube creation

Added:
>
>
  • BeamSize is now taken from the calibration tree if "hpbw" is in the metadata
    • If the "hpbw" is not in the meta data, the beam size is calculated in the same way as in convertK2Jy using the rfreq in the meta data

* Can rotate maps using new parameter, FlyAngle

  • GUI behaviour modified:
    • Picks up information from meta data to provide better default input parameters
    • Changes to values in the GUI form now only go in one direction, e.g. changing PixelSize changes the Beam and Filter Parameters but not vice versa.
 

HiClass export tool

Added:
>
>
  • You can now create keywords when exporting to CLASS fits format.
 

Documentation

Added:
>
>
  • HIFI User's Manual changed name to HIFI Data Reduction Guide, chapters rearranged into more logical order, addition of a changelog
    • Addition of introductory chapter, "HIFI Launch Pad"
    • Addition of a DBS Point Mode cookbook
    • Addition of chapter, "How to flag and remove flags from your data"
    • Addition of chapter, "Unit Conversion"
    • Updates to chapters on "Pipeline", "Deconvolution", "Creating Spectral Cubes", "What was done to my data" (clarifying units at each stage of the pipeline),
  • Updates the examples in the HIFI Pipeline Specification Document
  • HIFI User Reference Manual (URM) improvements
    • All HIFI tasks in URM now have a category and show up in the categorised view of commands
    • The categorised view of commands now is grouped into headers that should be meaningful to astronomers
    • Many URM entries have been improved so that decriptions are clear and examples are useful

 

PACS

Revision 372011-10-31 - PaulBalm

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 109 to 109
 

Herschel Science Archive

Added:
>
>
  • HIPE will now store username and password in encrypted form outside of properties files (if so requested -- one can also choose not to store the username and password). In any case, it is recommended to remove HSA username and password from all properties files.
 

Calibration sources

Added:
>
>
  • All SED models for asteroid observations have been added to the ESAC database.
  • The Calibrators data has been reformatted, so that spectra are now stored as SimpleSpectrum objects, allowing them to be viewed with standard HIPE spectrum tools.
 

Products and datasets

All products

  • Removed the legacy diff methods from Product and Dataset interface. All functionality moved into ProductDiff class which is available as part of the utility toolbox (herschel.ia.toolbox.util)
Line: 119 to 124
 

Product Access Layer

Added:
>
>
  • A "Cache" entry was added to the HIPE Preferences. Under this entry, a panel is shown with an overview of all existing caches on the system. One can also clear out these caches from this panel.
  • Product Browser: A simplified Observations tab was added, that allows user to list all observations on local disk, just by hitting the Run button. The data source selector has also been re-designed to be more flexible and intuitive.
 

Tasks

  • General clean up of tasks:
Line: 157 to 165
 

Virtual Observatory

Added:
>
>
  • Explicit support for DS9 was added. Right-clicking on a product will now show "SAOImage DS9" always, even if it is not running. In previous versions, DS9 had to be running for the menu item to appear. Requires DS9 version 5.3 or greater.
  • Upgrade to JSAMP 1.3, which targets version 1.3 of the SAMP protocol.
  • Improved GUI feedback using the HIPE progress bar when sending and receiving data.
  • The connection to the VO is now automatically established on HIPE start-up, to improve usability.
 

Installer

Sometimes, especially when a user has a shaky internet connection, an installation might fail in such a way that the local (in the user's machine) repository of hcss libraries might get corrupted. The workaround that existed so far can be found in the known issued page in: http://herschel.esac.esa.int/twiki/bin/view/Hcss/KnownIssuesHipeInstaller#Installer_fails_with_a_zip_error. The solution in short was to remove the local repository and all existing HIPE installation and start over. However a new solution that hopefully solves this issue has been implemented in 8.0. We know that bad proxy setup, hardware issues in routers can cause packet loss even in TCP. Such loss should always result in a file which is smaller than actually intended. Normally such a situation should result in an exception, but with packet loss in e.g. a router this will go unnoticed and a normal EOF completes the transmission. However, the resulting ZIP file will then always fail to unzip. The installer has been modified to extract the content-length from the HTTP header and validate the number of bytes actually received and raise an exception if such thing takes place. Moreover, a better exception reporting has been implemented to give more information on the basic cause of the exception. Finally the user installer reports the estimated time of completion while installing to give a rough feeling to the user.

Revision 362011-10-27 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 25 to 25
 
  • Added contextual menu action Send To -> Text File to save the contents of TableDatasets in files.
  • Names of variables created from the Product Viewer are more consistent and meaningful.
Added:
>
>
Dataset Viewer
  • Numeric values in metadata, array datasets or table datasets can be displayed in decimal or hexadecimal format, and dates can be shown in either UTC or TAI. These options are available in the contextual menu for individual cells.
 
Product Viewer
  • The title (product's description) can be selected and copied.
  • The expanded sections can be resized, for example to show more metadata rows.

Revision 352011-10-26 - DavidSharpe

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 239 to 239
  The Average Spectra module no longer supports the bandEdge parameter. This parameter was used only for the module's defunct outlier rejection functionality, and so it is no longer needed. Please ensure that your scripts do not use this parameter.
Added:
>
>

Spectrum Flux Calibration

The Flux Calibration module has been replaced by two modules, the Point Source Flux Calibration module and the Extended Source Flux Calibration module. References to SpecFluxConversionTask should be replaced with SpecPointFluxConversionTask or SpecExtendedFluxConversionTask. Read more in the pipeline specification manual at Spectrum Point Source Flux Calibration and Spectrum Extended Source Flux Calibration.

Spectral Unit and Unit-Flexibility

It is now possible to specify the spectral unit of the initial spectra. The Fourier Transform module (FourierTransformTask) now accepts an optional spectralUnit string parameter to specify the desired output spectral unit. For example, if spectralUnit='GHz', then the output spectra will be in units of V/GHz versus GHz. If no unit is specified, the output spectra will be in units of V/(1/OPD_UNIT) versus 1/OPD_UNIT (the OPD unit is typically cm).

To accommodate this new feature, all spectral modules now accept spectra and calibration data products in any unit of wavenumber or frequency. For example, if a module is provided with spectra in units of V/GHz versus GHz and a calibration product in units of cm-1, then the cm-1 calibration product will be converted to GHz, and the output spectra will be in GHz.

Bright Source Mode

Two modules have been created for processing bright source mode observations: the PCAL Gain module (SpecApplyPcalGainTask) and the Bright Mode Gain module (SpecApplyBrightGainTask). Both of these modules are in the new herschel.spire.ia.pipeline.spec.gain package. The PCAL Gain module computes the ratio of a measured PCAL flash level separation to the mean PCAL flash separation measured on Dark Sky in Nominal Mode. The Bright Mode Gain module computes and then applies a wavenumber-dependent gain factor for observations which have been made using the Bright Source Mode.

 

Interactive Analysis and Tools

Destriper

Revision 342011-10-25 - ThanosTsounis

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 156 to 156
 

Installer

Added:
>
>
Sometimes, especially when a user has a shaky internet connection, an installation might fail in such a way that the local (in the user's machine) repository of hcss libraries might get corrupted. The workaround that existed so far can be found in the known issued page in: http://herschel.esac.esa.int/twiki/bin/view/Hcss/KnownIssuesHipeInstaller#Installer_fails_with_a_zip_error. The solution in short was to remove the local repository and all existing HIPE installation and start over. However a new solution that hopefully solves this issue has been implemented in 8.0. We know that bad proxy setup, hardware issues in routers can cause packet loss even in TCP. Such loss should always result in a file which is smaller than actually intended. Normally such a situation should result in an exception, but with packet loss in e.g. a router this will go unnoticed and a normal EOF completes the transmission. However, the resulting ZIP file will then always fail to unzip. The installer has been modified to extract the content-length from the HTTP header and validate the number of bytes actually received and raise an exception if such thing takes place. Moreover, a better exception reporting has been implemented to give more information on the basic cause of the exception. Finally the user installer reports the estimated time of completion while installing to give a rough feeling to the user.
 

Documentation

HIFI

Revision 332011-10-24 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 124 to 124
 
    • All task GUIs and nodes should now show tooltips
  • Added some new user-friendly parameter editors (see For Task developers only)
  • Default task layout:
Changed:
<
<
    • Improving visibility of defaults: tooltip TODO
>
>
    • Parameter name tooltips now show the parameter name (for long names)
 
    • big modifiers can occupy a full row
    • groups (tabs) can be navigated with the mouse wheel
Deleted:
<
<
    • New Task Dialog Preference allows to generate task calls showing all non-empty parameters ("full calls").
 
    • Parameters can be reset individually via contextual menu on parameter names (vs resetting all parameters with Clear button)
    • Parameters can list the variables they can accept via contextual menu on parameter names (to know before hand if we have the proper data to use a parameter)
  • For Task developers only:
    • Task registration will issue warnings if either the task was already registered or the taskname was already used or both.
    • Command generation: added converters so that arbitrary types can be represented with Jython strings (part of better visibility for defaults)
    • New shared Modifiers (parameter editors):
Changed:
<
<
      • Added Editor for selecting a subset of a list of values (JListModifier, N values of M possible, M >= N)
      • Added a variant of JOptionModifier (Combo box) that returns some values but shows others (example: parameter uses integers but we show strings to users)
>
>
      • Added Editor for selecting a subset of a list of values (JListModifier, N values of M possible, M >= N)
      • Added a variant of JOptionModifier (Combo box) that returns some values but shows others (example: parameter uses integers but we show strings to users)
 

Misc Tasks

  • OpenTask parameter names have changed capitalization to comply with lower camel case
Line: 151 to 150
 
  • Task simpleAsciiTableReader extended to support all options of asciiTableReader via Advanced tab. (eventually, asciiTableReader will subsume simpleAsciiTableReader)
  • Improved type guessing for table reading (affects (simple)AsciiTableReader): for floating-point data it will detect precision loss and maintain representation. If a floating-point number will lose precision as a float (32-bits) a double (64-bits) will be used instead: up to 17 significant digits but in these case it will use twice the memory.
  • Improved documentation and information provided by getObservation
Changed:
<
<
  • Added contextual menu option to save tables with asciiTableWriter
>
>
  • Added contextual menu option to Variables View to save tables with asciiTableWriter
 

Virtual Observatory

Revision 322011-10-24 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 145 to 145
 

Systematic product generation

Data input-output

Changed:
<
<
  • Task importUFDirToPal renamed to importObservation
  • Task exportPalToUFDir renamed to exportObservation
  • Task simpleAsciiTableReader extended to support all options of asciiTableReader via Advanced tab. (eventually, AsciiTableReader will subsume SimpleAsciiTableReader)
  • Improved type guessing for table reading (affects (simple)AsciiTableReader): for floating-point data it will detect precision loss and maintain representation. If a floating-point number will lose precision as a float (32-bits) a double (64-bits) will be used instead: up to 17 significant digits but in these case it will use twice the memory.
>
>
  • URM entries (documentation) improved
  • Task importUFDirToPal renamed to importObservation
  • Task exportPalToUFDir renamed to exportObservation
  • Task simpleAsciiTableReader extended to support all options of asciiTableReader via Advanced tab. (eventually, asciiTableReader will subsume simpleAsciiTableReader)
  • Improved type guessing for table reading (affects (simple)AsciiTableReader): for floating-point data it will detect precision loss and maintain representation. If a floating-point number will lose precision as a float (32-bits) a double (64-bits) will be used instead: up to 17 significant digits but in these case it will use twice the memory.
  • Improved documentation and information provided by getObservation
  • Added contextual menu option to save tables with asciiTableWriter
 

Virtual Observatory

Revision 312011-10-21 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 40 to 40
 
  • If you get temporary (or permanent) freezes in the HIPE GUI, or you have created a custom task dialogue or view in HIPE, you can debug it by enabling the corresponding preference in Edit > Preferences > General > Debug. Any problem found with this debugging can be reported like previously, with the HSC Helpdesk or Give Feedback options of the Help menu.

Plug-ins

Deleted:
<
<
A utility is provided to disable all plug-ins without having to start HIPE. This can be convenient when you know or suspect that a plug-in causes HIPE not to start. The utility is called disable_plugins and is located in the bin directory of the installation.
 
Changed:
<
<
You can use the Plug-ins panel (from the Tools menu) to re-enable your plug-ins.
>
>
  • More protection against problematic plug-ins. You can now disable all plug-ins without having to start HIPE. This can be convenient when you know or suspect that a plug-in causes HIPE not to start. To disable all plug-ins, run the disable_plugins utility located in the bin directory of the installation. You can use the Plug-ins panel (from the Tools menu) to re-enable your plug-ins.
  • More flexible update mechanism. If multiple updates for a plug-in are found, they are now all shown. Previously only the most recent update was shown.
  • Improved user feedback. You now get popup messages when no updates are found for installed plug-ins, and when you install a plug-in incompatible with your version of HIPE.
  • Many improvements to plug-in installation:
    • The installation process is now much faster for large plug-ins, such as those containing data pools.
    • The HIPE interface is no longer unresponsive while installing a plug-in.
    • You can follow the progress of the installation in the progress bar at the bottom of the HIPE window.
  • Fixed a problem with the scanamorphos plug-in, which would not read correctly some scanamorphos output. The plug-in assumed that the PrimaryImage HDU contained a 3-dim array. Output containing a 2-dim array in the PrimaryImage HDU was not read correctly.
 

Upgrades

If there is a new user release available, HIPE would notice it and offer to install the new version.

Revision 302011-10-20 - ChristopheJean

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 193 to 193
 

Photometry

Spectroscopy

Added:
>
>
Flux calibration now based on observations of astronomical standards and the calibration block attached to each observation. The pipeline task specDiffCs computes the detector response of the current observation.
 

Analysis tools

The Preferences now contain a Logging panel under the general tab which allows the user/developer to increase or decrease the number of logging messages send to the terminal.

Revision 292011-10-20 - AnthonySmith

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 58 to 58
 

Astronomical utilities

  • Planck class moved from spire_ia_pipeline_spec_util to ia_toolbox_astro. If you are doing an explicit import (it is not needed) you have to change it to (preferred) import herschel.ia.toolbox.astro.Planck or from herschel.ia.toolbox.astro import Planck
Changed:
<
<
  • New task astrometryFix : takes a list of known source positions (e.g., from radio observations) and will adjust the absolute pointing of map data to give the best possible agreement in overall astrometry with the ancillary data (e.g., by stacking the map at the source positions).
>
>
  • New task astrometryFix: adjusts the absolute pointing of map data to give the best possible agreement in overall astrometry with some ancillary data. This is done either by cross correlation of the map(s) with ancillary map data, or by stacking the map(s) at a list of known source positions (e.g., from radio observations).
 

Numeric routines

Data fitting

Line: 74 to 74
 

Source extraction

Added:
>
>
  • Improvements to handling of pixel coordinates and region of interest in sourceExtractor* tasks:
    • Output source list now contains pixel coordinates of source (x, y) = (column, row)
    • Added "getWcs" and "setWcs" methods to the SourceListProduct/Dataset
    • Added "roi" parameter (region of interest) to tasks
    • Removed "corner*" and "returnPixelCoordinates" parameters from tasks
  • Non-standard columns are now accepted in a SourceListDataset, e.g., when converting from a TableDataset (an exception will be thrown if there are no standard columns at all)
  • Removed "subtractMedianBackground" parameter from sourceExtractorSussextractor (it didn't do anything)
  • Minor bug fixes, performance improvements and various internal changes
 

Images

Display

  • The methods addCircle, addEllipse and addRectangle is not longer dependent on the zoomfactor of the image.
Line: 219 to 228
 The Average Spectra module no longer supports the bandEdge parameter. This parameter was used only for the module's defunct outlier rejection functionality, and so it is no longer needed. Please ensure that your scripts do not use this parameter.

Interactive Analysis and Tools

Added:
>
>

Destriper

 The following new features are introduced in Hipe 8:
  • Level 2 deglitching is available through setting l2DeglitchRepeat parameter.
  • The 5 individual output parameters are combined to one ArrayList parameter where each of 5 parameter can be retrieved base on its position.
Line: 235 to 246
 
  • The input parameter name "Tod" changed to "tod".
  • The input parameter name "L2DeglitchRepeat" changed to "l2DeglitchRepeat".
Added:
>
>

Baseline Removal

 
Changed:
<
<
>
>
  • New "roi" (region of interest) parameter for baselineRemoval* tasks, specifying the region of the sky to be used in estimating the baseline
  • Removed "doInterpolation" task parameter
  • Minor changes and bug fixes (for example, the tasks now do ignore masked data samples when a TodBuffer is used internally)
 

Documentation

Revision 282011-10-19 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Changed:
<
<
>
>
PDF Version Click here

See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.

 

Core system

Revision 272011-10-18 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 33 to 33
 

Help & Debug

  • The Tip of the day contains information on the Console meta commands (those starting with !)
Changed:
<
<
  • If you get temporary (or permanent) freezes in the HIPE GUI, or you have created a custom task dialogue or view in HIPE, you can debug it by enabling the corresponding preference in Edit > Preferences > General > Debug. Any problem can be reported like previously, with the HSC Helpdesk or Give Feedback options of the Help menu.
>
>
  • If you get temporary (or permanent) freezes in the HIPE GUI, or you have created a custom task dialogue or view in HIPE, you can debug it by enabling the corresponding preference in Edit > Preferences > General > Debug. Any problem found with this debugging can be reported like previously, with the HSC Helpdesk or Give Feedback options of the Help menu.
 

Plug-ins

A utility is provided to disable all plug-ins without having to start HIPE. This can be convenient when you know or suspect that a plug-in causes HIPE not to start. The utility is called disable_plugins and is located in the bin directory of the installation.

Revision 262011-10-18 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 19 to 19
 
Variables View
  • Added contextual menu action Show contents to show the contents of a variable in the console.
  • Added contextual menu action Send To -> Text File to save the contents of TableDatasets in files.
Added:
>
>
  • Names of variables created from the Product Viewer are more consistent and meaningful.
 
Product Viewer
  • The title (product's description) can be selected and copied.
Line: 26 to 27
 
  • The summary of the Observation Viewer has been enriched with useful metadata of the observation being shown.
  • Products can be saved directly from the tree with the Sent To action of the popup menu, without needing to create an intermediate variable.
Changed:
<
<
Jython Editor
>
>
Jython & Text Editors
 
  • The supported Jython language has been upgraded from version 2.1 to 2.5.2.
Added:
>
>
  • Scripts and text files coming with the installation or with plug-ins are safe to edit without losing the original version.
 

Help & Debug

  • The Tip of the day contains information on the Console meta commands (those starting with !)

Revision 252011-10-18 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 9 to 9
 Jython major upgrade. The version of Jython on which HIPE is based has been upgraded from 2.1 to 2.5.2. You may have to adapt your scripts for them to work with the new version. See this page for the details.

HIPE

Deleted:
<
<

Variables View

  • Added contextual menu action "Show contents" to show the contents of a variable in the console.
  • Added contextual menu action "Send To -> Text File" to save the contents of TableDatasets in files.

Plug-ins

 
Added:
>
>

Status bar

The maximum memory level reached is indicated with a vertical line in the memory bar. The tooltip shows the actual reached value, and allows to reset it so that further computations can record its own maximum.

Editors & Views

Variables View
  • Added contextual menu action Show contents to show the contents of a variable in the console.
  • Added contextual menu action Send To -> Text File to save the contents of TableDatasets in files.

Product Viewer
  • The title (product's description) can be selected and copied.
  • The expanded sections can be resized, for example to show more metadata rows.
  • The summary of the Observation Viewer has been enriched with useful metadata of the observation being shown.
  • Products can be saved directly from the tree with the Sent To action of the popup menu, without needing to create an intermediate variable.

Jython Editor
  • The supported Jython language has been upgraded from version 2.1 to 2.5.2.

Help & Debug

  • The Tip of the day contains information on the Console meta commands (those starting with !)
  • If you get temporary (or permanent) freezes in the HIPE GUI, or you have created a custom task dialogue or view in HIPE, you can debug it by enabling the corresponding preference in Edit > Preferences > General > Debug. Any problem can be reported like previously, with the HSC Helpdesk or Give Feedback options of the Help menu.

Plug-ins

 A utility is provided to disable all plug-ins without having to start HIPE. This can be convenient when you know or suspect that a plug-in causes HIPE not to start. The utility is called disable_plugins and is located in the bin directory of the installation.

You can use the Plug-ins panel (from the Tools menu) to re-enable your plug-ins.

Added:
>
>

Upgrades

If there is a new user release available, HIPE would notice it and offer to install the new version.
 

Plotting

PlotXY is now able to display images (i.e. 2D data). Look at an advanced example here.

Revision 242011-10-17 - LijunZhang

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 22 to 22
 PlotXY is now able to display images (i.e. 2D data). Look at an advanced example here.

TablePlotter

Added:
>
>
TablePlotter can display plot with Date/Time format.
 

OverPlotter

Added:
>
>
OverPlotter can display plot with Date/Time format.
 

Dataset viewer

Astronomical utilities

Line: 177 to 179
 

Engineering Conversion

Common Pipeline

Added:
>
>
The default value for reconstruction in the sigmaKappa deglitcher is changed to "DIRECT".
 

Photometer Pipeline

Line: 187 to 190
 The Average Spectra module no longer supports the bandEdge parameter. This parameter was used only for the module's defunct outlier rejection functionality, and so it is no longer needed. Please ensure that your scripts do not use this parameter.

Interactive Analysis and Tools

Added:
>
>
The following new features are introduced in Hipe 8:
  • Level 2 deglitching is available through setting l2DeglitchRepeat parameter.
  • The 5 individual output parameters are combined to one ArrayList parameter where each of 5 parameter can be retrieved base on its position.
  • The new diagnostic product is introduced which provides with useful diagnostic information.
  • Both Tod and TodBuffer are acceptable input for Tod parameter.
  • The input parameter "array" now is MANDATORY.
  • Pixel size can be changed through setting pixelSize parameter.
  • brightSource parameter is changed to boolean.
  • A boolean parameter, "fitWithWeight" is introduced.
  • Interruption is added so that the destriper can be terminated at the user's request.
  • Tod files stored in temporary directory are available until Hipe is terminated.
  • The destriper speed is increased 30%.
  • The input parameter name "Level1" changed to "level1".
  • The input parameter name "Tod" changed to "tod".
  • The input parameter name "L2DeglitchRepeat" changed to "l2DeglitchRepeat".

 

Documentation

Revision 232011-09-22 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 102 to 102
 
  • Task importUFDirToPal renamed to importObservation
  • Task exportPalToUFDir renamed to exportObservation
  • Task simpleAsciiTableReader extended to support all options of asciiTableReader via Advanced tab. (eventually, AsciiTableReader will subsume SimpleAsciiTableReader)
Added:
>
>
  • Improved type guessing for table reading (affects (simple)AsciiTableReader): for floating-point data it will detect precision loss and maintain representation. If a floating-point number will lose precision as a float (32-bits) a double (64-bits) will be used instead: up to 17 significant digits but in these case it will use twice the memory.
 

Virtual Observatory

Revision 222011-09-21 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 11 to 11
 

HIPE

Variables View

  • Added contextual menu action "Show contents" to show the contents of a variable in the console.
Added:
>
>
  • Added contextual menu action "Send To -> Text File" to save the contents of TableDatasets in files.
 

Plug-ins

A utility is provided to disable all plug-ins without having to start HIPE. This can be convenient when you know or suspect that a plug-in causes HIPE not to start. The utility is called disable_plugins and is located in the bin directory of the installation.

Revision 212011-09-21 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 25 to 25
 

Dataset viewer

Astronomical utilities

Changed:
<
<
  • Planck class moved from spire_ia_pipeline_spec_util to ia_toolbox_astro.
>
>
  • Planck class moved from spire_ia_pipeline_spec_util to ia_toolbox_astro. If you are doing an explicit import (it is not needed) you have to change it to (preferred) import herschel.ia.toolbox.astro.Planck or from herschel.ia.toolbox.astro import Planck
  • New task astrometryFix : takes a list of known source positions (e.g., from radio observations) and will adjust the absolute pointing of map data to give the best possible agreement in overall astrometry with the ancillary data (e.g., by stacking the map at the source positions).
 

Numeric routines

Data fitting

Revision 202011-09-14 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 89 to 89
 
      • Added Editor for selecting a subset of a list of values (JListModifier, N values of M possible, M >= N)
      • Added a variant of JOptionModifier (Combo box) that returns some values but shows others (example: parameter uses integers but we show strings to users)
Added:
>
>

Misc Tasks

  • OpenTask parameter names have changed capitalization to comply with lower camel case
 

Quality control

Revision 192011-09-13 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 9 to 9
 Jython major upgrade. The version of Jython on which HIPE is based has been upgraded from 2.1 to 2.5.2. You may have to adapt your scripts for them to work with the new version. See this page for the details.

HIPE

Changed:
<
<
>
>

Variables View

  • Added contextual menu action "Show contents" to show the contents of a variable in the console.
 

Plug-ins

A utility is provided to disable all plug-ins without having to start HIPE. This can be convenient when you know or suspect that a plug-in causes HIPE not to start. The utility is called disable_plugins and is located in the bin directory of the installation.

Line: 24 to 25
 

Dataset viewer

Astronomical utilities

Added:
>
>
  • Planck class moved from spire_ia_pipeline_spec_util to ia_toolbox_astro.
 

Numeric routines

Data fitting

Revision 182011-08-24 - WimDeMeester

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 41 to 41
 

Images

Display

Added:
>
>
  • The methods addCircle, addEllipse and addRectangle is not longer dependent on the zoomfactor of the image.
 

Analysis

Spectra

Revision 172011-08-24 - RikHuygen

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 139 to 139
 

Spectroscopy

Calibration

Added:
>
>
The calibration sets that are installed on your local system can now be inspected with a 'Calibration Sets Viewer'. The viewer is located in the menu Window -> Show View -> Workbench -> Calibration Sets. Select a version from the available buttons, the you can inspect the release note for that version or the product list with the versions of the individual calibration products.
 

Photometry

Spectroscopy

Analysis tools

Added:
>
>
The Preferences now contain a Logging panel under the general tab which allows the user/developer to increase or decrease the number of logging messages send to the terminal.
 

Photometry

Spectroscopy

Revision 162011-08-02 - PasqualePanuzzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 17 to 17
 You can use the Plug-ins panel (from the Tools menu) to re-enable your plug-ins.

Plotting

Added:
>
>
PlotXY is now able to display images (i.e. 2D data). Look at an advanced example here.
 

TablePlotter

OverPlotter

Dataset viewer

Revision 152011-08-01 - PaulBalm

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 10 to 10
 

HIPE

Added:
>
>

Plug-ins

A utility is provided to disable all plug-ins without having to start HIPE. This can be convenient when you know or suspect that a plug-in causes HIPE not to start. The utility is called disable_plugins and is located in the bin directory of the installation.

You can use the Plug-ins panel (from the Tools menu) to re-enable your plug-ins.

 

Plotting

TablePlotter

OverPlotter

Revision 142011-07-27 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 6 to 6
 

Core system

Changed:
<
<
Jython major upgrade. The version of Jython on which HIPE is based has been upgraded from 2.1 to 2.5.2.
>
>
Jython major upgrade. The version of Jython on which HIPE is based has been upgraded from 2.1 to 2.5.2. You may have to adapt your scripts for them to work with the new version. See this page for the details.
 

HIPE

Revision 132011-07-27 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 6 to 6
 

Core system

Added:
>
>
Jython major upgrade. The version of Jython on which HIPE is based has been upgraded from 2.1 to 2.5.2.
 

HIPE

Plotting

Revision 122011-07-20 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 57 to 57
 

Tasks

Deleted:
<
<
  • Task registration will issue warnings if either the task was already registered or the taskname was already used or both.
 
  • General clean up of tasks:
Changed:
<
<
    • Removed default values from mandatory parameters, warnings issued (control of bad programming)
    • Validators in primary inputs (improved checking and messages on bad data), return false on error (control damage of developer mistakes)
  • Modifiers (parameter editors):
    • Added Editor for selecting a subset of a list of values (JListModifier)
    • Added a variant of JOptionModifier (Combobox) that returns some values but shows others (example: ints returned but strings shown)
>
>
    • All mandatory parameters must be filled in by users: removed default values from mandatory parameters, warnings issued (detection of programming mistakes)
    • Improved parameter checking and messages on bad data: Validators in primary inputs, they return false on error and log a warning (detection of programming mistakes)
    • All task GUIs and nodes should now show tooltips
  • Added some new user-friendly parameter editors (see For Task developers only)
 
  • Default task layout:
Changed:
<
<
    • Improving visibility of defaults: tooltip
>
>
    • Improving visibility of defaults: tooltip TODO
 
    • big modifiers can occupy a full row
    • groups (tabs) can be navigated with the mouse wheel
Changed:
<
<
    • Added checkboxes to parameters labels: they control which parameters will be used to generate the task call. By default:, optionals are unmarked. Defaults, when requested, will be shown (*)
    • Can reset parameters individually
    • Parameters can list the variables they can accept
    • Task command generation: added converters so that arbitrary types can be represented with Jython strings (*)
>
>
    • New Task Dialog Preference allows to generate task calls showing all non-empty parameters ("full calls").
    • Parameters can be reset individually via contextual menu on parameter names (vs resetting all parameters with Clear button)
    • Parameters can list the variables they can accept via contextual menu on parameter names (to know before hand if we have the proper data to use a parameter)
  • For Task developers only:
    • Task registration will issue warnings if either the task was already registered or the taskname was already used or both.
    • Command generation: added converters so that arbitrary types can be represented with Jython strings (part of better visibility for defaults)
    • New shared Modifiers (parameter editors):
      • Added Editor for selecting a subset of a list of values (JListModifier, N values of M possible, M >= N)
      • Added a variant of JOptionModifier (Combo box) that returns some values but shows others (example: parameter uses integers but we show strings to users)
 

Quality control

Line: 81 to 84
 

Data input-output

  • Task importUFDirToPal renamed to importObservation
  • Task exportPalToUFDir renamed to exportObservation
Changed:
<
<
  • Task simpleAsciiTableReader extended to support all options of asciiTableReader via Advanced tab.
>
>
 

Virtual Observatory

Revision 112011-07-19 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 57 to 57
 

Tasks

Changed:
<
<
  • Jython upgrade:
    • No need to extend JTask in Jython
>
>
  • Task registration will issue warnings if either the task was already registered or the taskname was already used or both.
 
  • General clean up of tasks:
    • Removed default values from mandatory parameters, warnings issued (control of bad programming)
    • Validators in primary inputs (improved checking and messages on bad data), return false on error (control damage of developer mistakes)

Revision 102011-07-14 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 82 to 82
 

Data input-output

  • Task importUFDirToPal renamed to importObservation
  • Task exportPalToUFDir renamed to exportObservation
Added:
>
>
  • Task simpleAsciiTableReader extended to support all options of asciiTableReader via Advanced tab.
 

Virtual Observatory

Revision 92011-07-11 - DavidSharpe

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 155 to 155
 

Spectrometer Pipeline

Added:
>
>

Averaging Spectra

The Average Spectra module no longer supports the bandEdge parameter. This parameter was used only for the module's defunct outlier rejection functionality, and so it is no longer needed. Please ensure that your scripts do not use this parameter.

 

Interactive Analysis and Tools

Documentation

Revision 82011-07-04 - BertrandMorin

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 23 to 23
 

Trend analysis

Mapping

Wavelet transform

Added:
>
>
  • Continuous Wavelet Transform synthesis operation now adds low frequencies for a reliable reconstruction
  • Continuous Wavelet Transform is able to deal with Complex1d signal
 

Other functions

Source extraction

Revision 72011-07-01 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 22 to 22
 

Statistics

Trend analysis

Mapping

Added:
>
>

Wavelet transform

 

Other functions

Source extraction

Revision 62011-06-17 - JorgoBakker

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 45 to 45
 

Calibration sources

Products and datasets

Added:
>
>

All products

  • Removed the legacy diff methods from Product and Dataset interface. All functionality moved into ProductDiff class which is available as part of the utility toolbox (herschel.ia.toolbox.util)
 

Observation context

Product Access Layer

Revision 52011-06-16 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
Changed:
<
<

What's New in HIPE 8

>
>

What's coming up in HIPE 8

 

Revision 42011-06-14 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 8

Line: 51 to 51
 

Tasks

Added:
>
>
  • Jython upgrade:
    • No need to extend JTask in Jython
  • General clean up of tasks:
    • Removed default values from mandatory parameters, warnings issued (control of bad programming)
    • Validators in primary inputs (improved checking and messages on bad data), return false on error (control damage of developer mistakes)
  • Modifiers (parameter editors):
    • Added Editor for selecting a subset of a list of values (JListModifier)
    • Added a variant of JOptionModifier (Combobox) that returns some values but shows others (example: ints returned but strings shown)
  • Default task layout:
    • Improving visibility of defaults: tooltip
    • big modifiers can occupy a full row
    • groups (tabs) can be navigated with the mouse wheel
    • Added checkboxes to parameters labels: they control which parameters will be used to generate the task call. By default:, optionals are unmarked. Defaults, when requested, will be shown (*)
    • Can reset parameters individually
    • Parameters can list the variables they can accept
    • Task command generation: added converters so that arbitrary types can be represented with Jython strings (*)
 

Quality control

Systematic product generation

Revision 32011-05-19 - RoBaS

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 8

Line: 34 to 34
 

Display

Analysis

Added:
>
>
The SpectrumFitter now recognizes a CUBE such that the user can give the x,y of the spectrum. The script as written by the SpectrumFitterGUI also takes the correct spectrum.
 

Data cubes

Display

Analysis

Revision 22011-04-29 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 8

Line: 54 to 54
 

Systematic product generation

Data input-output

Added:
>
>
  • Task importUFDirToPal renamed to importObservation
  • Task exportPalToUFDir renamed to exportObservation
 

Virtual Observatory

Revision 12011-04-12 - DavideRizzo

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 8

Core system

HIPE

Plotting

TablePlotter

OverPlotter

Dataset viewer

Astronomical utilities

Numeric routines

Data fitting

Data interpolation

Random numbers

Statistics

Trend analysis

Mapping

Other functions

Source extraction

Images

Display

Analysis

Spectra

Display

Analysis

Data cubes

Display

Analysis

Herschel Science Archive

Calibration sources

Products and datasets

Observation context

Product Access Layer

Tasks

Quality control

Systematic product generation

Data input-output

Virtual Observatory

Installer

Documentation

HIFI

HIFI Software Configuration and Properties

HIFI Calibration

HIFI Pipeline

hifiPipeline task

Level 0 Pipeline

WBS Pipeline

HRS Pipeline

Level 1 Pipeline

Level 2 Pipeline

HIFI Products

HIFI Spectrum Toolbox

Spectrum Explorer (HIFI specifics)

Spectrum tasks (HIFI specifics)

HIFI Data Processing Tools

Standing Wave Removal

Baseline Removal

Deconvolution

Spectral Cube creation

HiClass export tool

Documentation

PACS

Pipeline

Photometry

Spectroscopy

Calibration

Photometry

Spectroscopy

Analysis tools

Photometry

Spectroscopy

Product changes

Photometry

Spectroscopy

Documentation

SPIRE

Calibration Products

Calibration Framework

Preprocessing

Engineering Conversion

Common Pipeline

Photometer Pipeline

Spectrometer Pipeline

Interactive Analysis and Tools

Documentation

<--  
-->
 
This site is powered by the TWiki collaboration platform Powered by Perl