TWiki> Public Web>HipeWhatsNew>HipeWhatsNew14x (revision 13)EditAttach


What's coming up in HIPE 14

PDF Version Click here

Help Interested in what's new for other versions of HIPE? See this page for links to all the What's New documents from HIPE 4.0 onwards.
See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.

Planned work














  • FTS
    • Two new tasks and corresponding calibration products have been introduced for SPIRE FTS data. One task corrects for artefacts in low resolution point source calibrated spectra. The other corrects the level of all extended source calibrated data in or to get better agreement between the SPIRE Spectrometer and Photometer.

Core system


Editors & Viewers

Default editor

The default editor for every type of data, which is the one used when double clicking on a variable, is preserved regardless the editors opened through the Open With menu.

The previous behaviour, which is to use the last opened editor as the default, can still be set, by means of a new preference in the Editors & Viewers category.

Recent open files

The size of the recently opened files list, which can be accessed through the File menu, is now configurable through preferences.

Quick Analysis

There is a new perspective in HIPE, which is accessible by clicking on the magnifying glass at the top right toolbar, for quickly inspecting scientific data from the Herschel Science Archive.

Observations can be searched by target name, observation id, operational day or Ra/Dec.

When clicking on an observation, its standalone browse products are opened in the editor area for quick analysis, by means of the image, spectra or cube viewers.

Opening the observation context instead is also possible, either with the popup menu or by configuring so as the default behaviour, in the Quick Analysis preferences category.




Dataset viewer

Astronomical utilities

Numeric routines

Data fitting

Data interpolation

Random numbers


Trend analysis


Other functions

Source extraction


* When requesting the unit, the FITS compatible unit is returned. * The naxis keywords are not longer duplicated in the fits header. * The Wcs ia added to all datasets of the images (like the stddev, ...).


* Added shortcuts to some of the task directly in the image viewer.





Data cubes

* When requesting the unit, the FITS compatible unit is returned. * The naxis keywords are not longer duplicated in the fits header. * The Wcs ia added to all datasets of the images (like the stddev, ...).


* Added shortcuts to some of the task directly in the image viewer.


Herschel Science Archive

Calibration sources

Products and datasets

A new Auxiliary Observation Summary Product which can be retrieved using the tag LATEST_OBSERVATION_SUMMARY_PRODUCT. This product contains one dataset with a summary of information for each observation. Useful methods exist for querying the data as documented here: ObservationSummaryProduct

Observation context

A new Calibration Table Product exists under the Auxiliary Context in each Observation Context. This product contains calibration tables data for the given instrument including exactly how the hardware was configured, hard limits, soft limits etc. This provides valuable information for the long term completeness of an observation. Useful methods exist for querying the table data as documented here: SimpleCalTableProduct

Product Access Layer


Quality control

Systematic product generation

Data input-output

Virtual Observatory




HIFI Software Configuration and Properties

HIFI Calibration

HIFI Pipeline

hifiPipeline task

Level 0 Pipeline

WBS Pipeline

HRS Pipeline

Level 1 Pipeline

Level 2 Pipeline

Level 2.5 Pipeline

HIFI Products

HIFI Spectrum Toolbox

Spectrum Explorer (HIFI specifics)

Spectrum tasks (HIFI specifics)

HIFI Data Processing Tools

Standing Wave Removal

Baseline Removal


Spectral Cube creation


HiClass export tool


For HCSS 14, many figures were refreshed in both manuals (HIFI Data Reduction Guide and HIFI Pipeline Specification).

  • Updates to the HIFI Data Reduction Guide:
    • Updates to the chapters Standing wave Removal and HIFI Baseline Removal : adding definition of the new parameter addMedianContinuum
    • Updates to the chapter Flags in HIFI data, section Quality Flags (this chapter is being revamped, please check regularly)
    • Updates to the chapter How to make a spectral cube: added a definition for parameters ignoreOffs, outputNames and extrapolate
    • Updates to the chapter Sideband Deconvolution: the output deconvolution product ssb is now called dataset (please note that scripts prior to this change will break)
    • Updates to the chapter Standing wave Removal, ESW correction in HEB bands section: update to the scripts in the demonstration subsection
    • Updates to the chapter How to add and remove flags, section Setting and Clearing Flags with command line tools: improved the definition of the parameter selection for Level 2.5 cubes
    • A new chapter called Understanding the uncertainty table information in your data has been added
  • Updates to the HIFI Pipeline Specification manual
    • This manual has been restructured
    • Removal of the task mkSpur
    • Corrections to broken links pointing to the HIFI Data Reduction Guide
    • Correction due to a missing figure Level 0 Pipeline flow diagram
    • Added text to tasks which did not have text, or were poorly documented, including mkRms




  • Level2.5 products:
    • Unimap and Jscanam maps are generated for Solar System Objects, by combining scan and cross-scan observations.
    • A block detection and masking method is implemented to avoid strong signal gradients generated by the calibration blocks. The quality of Level2.5 is enhanced in particular for those observations acquired at the early stage of the mission (e.g. Science Demonstration Phase) where calibration blocks were acquired in the middle of the scans.
  • ipipe script:
    • The JScanam ipipe script scanmap_JScanam_multiplePairs is designed to process multiple pairs of obsIDs and it is upgraded by introducing the scanamorphosAlignFrames Task. This Task aligns the Frames signal levels, belonging to every observation, to a common offset level, by exploiting the overlapping sky regions. This new capability is particulary useful to recover a common background level over observations that are partially or slightly overlapping.
    • the script scanmap_Unimap_memory_optimized is added within the Pipelines menu for allowing to run the Unimap release located on the userís computer by using the minimum amount of memory. It is particularly useful for processing large observations or/and if several obsIDs should be processed all together in a single run.
    • the photextended_pipeline_L3 script is available within the Pipelines menu to generate mosaics of Unimpa and Jscanam Level2.5 maps
  • Unimap : Unimap 6.2.0 is spawned by the Standard Product Generator (SPG) to create Level2.5 products. See the Photometer Data Reduction Guide and the offical Unimap webpages ( to know what's new about the 6.2.0 Unimap release.




Analysis tools


  • The PhotometryConvolveResolution Kernel script was added within the PACS Useful Script section. This script convolves and regrids a map at a certain wavelength to the PSF of another map at longer wavelength, by using an appropriate kernel.


Product changes


  • The error layer within Unimap Level 2.5 products is renamed as stdDev, being the standard deviation computed at every sky pixel with respect to the samples that are falling on it. It can be properly assumed as the error map only if associated to the NaÔve map.




Calibration Products

  • Spectrometer
    • A new calibration product has been added that contains correction curves for point-source calibrated LR data.
    • A new calibration product has been added that contains polynomial coefficients that are used to generate correction curves for extended source calibrated data.

Calibration Framework


Engineering Conversion

Common Pipeline

Photometer Pipeline

Spectrometer Pipeline

  • A new task has been added to the point source pipeline that applies the LR correction.
  • A new task has been added to all pipelines that applies the extended calibration correction in order to get better agreement between FTS and Phototmeter photometry.

Interactive Analysis and Tools



<script type="text/javascript">
var disqus_shortname = 'herscheltwiki'; // required: replace example with your forum shortname

(function() {
var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;
dsq.src = 'http://' + disqus_shortname + '';
(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
</script> <noscript>Please enable JavaScript to view the comments powered by Disqus.</noscript> blog comments powered by Disqus <!-- END OF COMMENT BOX CODE -->

* Set ALLOWTOPICCHANGE = RegisteredUsersGroup, TWikiAdminGroup

-- AlvarGarcia - 24 Nov 2014

Editors & Viewers

Edit | Attach | Watch | Print version | History: r78 | r15 < r14 < r13 < r12 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r13 - 2015-10-06 - AlvarGarcia
This site is powered by the TWiki collaboration platform Powered by Perl