• The spectrometer bands are increased slightly in the overlap region and at the high frequency end of SSW:
  • Old bands: SLW: 446.7-989.4 GHz; SSW: 959.3-1544.0 GHz
  • New bands: SLW: 446.7-1018.0 GHz; SSW: 944.0-1568.0 GHz
• The extended calibration is not changed from spire_cal_12_2 (apart from wider bands).
• The Low resolution point-source calibration is changed by <5% (before OD1000) and <2.7% (after OD1000) to bring it into line with the High resolution calibration. This was achieved by correcting for the pointing offset of Uranus during the calibration observations. The effect is to reduce the flux density of the final spectra. Changes are greatest for SSWD4 at high frequency, and smallest for SLWC3 (~1% for SLW).
• The High resolution point-source calibration has also been adjusted using improved corrections for the pointing offset of Uranus in the calibration observations. The effect for SSWD4 is ~2.5% before OD1000, and 0.8% after OD 1000. For SLWC3, the effect is <0.8% before OD1000 and <0.1% after OD1000. The effect is to increase the flux density in the final spectra.
• A small correction has been made to the point-source calibration for off-axis detectors for observations made before OD1000.
• The frequency grid for the beam profile shape been updated to match exactly that used in Makiwa et al. (2013, Applied Optics, 42, 3864) - this gives slightly better coverage at the edges of the new bands. The shapes themselves do not change.

• JScanam maps will be available in level 2.5 and 3.

• All maps now have now the same pixel size in the SPG, so that they can be compared more easily.

• Browse images have been improved significantly (postcards in the HSA).

• The structure of the level 2.5/3 contexts in the photometer observation contexts have been improved. All the slots in these contexts have now consistent names for each type of map:
  • HPPJSMAP[R|B|G] for red, blue or green JScanamorphous maps.
  • HPPNAVMAP[R|B|G] for red, blue or green naive maps.
  • HPPCORMAP[R|B|G] for red, blue or green corrected maps.
  • HPPMADMAP[R|B|G] for red, blue or green MadMap maps.
  • HPPHPFMAP[R|B|G] for red, blue or green High Pass Filter maps.
• Fixed duplication of FITS file names when observations are exported from the HSA into a tar file.
• Made sure that the ra and dec in level 2 photometer maps correspond to the center of the map.

• plotCubesStddev: a new parameter wranges allows the user to select the continuum windows of his/her choice.

• One of the spectrometer's interactive reduction pipeline scripts is now offered in a simplified version (ChopNodLineScan.py). The processing itself is not affected, but the number of helper-plots ("verbose plots") has been reduced and the comments have been abridged. Saving intermediate or final results is also made easier (without necessarily editing the script). In absence of negative feedback, the same approach will be followed for all other scripts from hipe 13.0 on.

• We provide a new task (specExtendedToPointCorrection) for computing the correction factor for slightly extended sources relative to the point source correction, based on a model of the source flux distribution. This tool is described extensively in section 7.8 of the PACS Spectrometer Data Reduction Guide.
• The extractCentralSpectrum task has been improved to return both the central and central 3x3 spectra, and to handle the masks better. Also the syntax has been changed, since it returns now a list of spectra (central, SUM(central_3x3) and a scaled version of the central spaxel computed so that its continuum matches that of the Sum(central_3x3)). See the URM and section 7.6 of the PACS Spectrometer Data Reduction Guide for details.
• Improved the long-term transient correction for the unchopped line spectroscopy pipeline.

• Added more flags to the rebinned cubes produced by the specWaveRebin task: We now propagate besides the saturation masks also all other inactive masks into the Flag dataset of the rebinned cubes.

SpectrumFitterGUI

• Update to export panel to make more intuitive to use
• Generated scripts have been updated
• A ParameterCube can now be exported
• All implemented models now have their own GUI (previously an "abstract GUI" was used for all but the most common models)
• Can set lower and upper bounds limits in model GUIs

• Nonsense LO settings in found spectral scan data are corrected at Level 0 using a calibration table storing the correct values. This impacts only one observation: 342194735 (PSP2_HEXOS-oribar-hifiss-b6a-main)

• Level 2.5 pipeline can be run in parallel (command line only)
• More tasks, for example doDeconvolution and PolarPair, are better integrated into the pipeline.

• In the case of mapping observations, the level 2.5 products containing up until now only the cubes now also contain a copy of the level 2 individual spectra complemented with the necessary information in order to optimally compute the cube grid dimension.
• Information available in metadata has been generally improved with the intention that they can (eventually) be used when querying the HSA

• Channel weights are now calculated for the SSB output (calculated as 1/rms^2)
• Minor bug fixes
• Major bug in gain fitting discovered. Feature should not be used.

• A new task MergeHtpsTask is available in order to merge (not average) HTPs of H and V polarisation. This allows spectral cubes to be created using all of the pointing information in the observation
• DoGridding is adapted to use a 2d model

• Drastic improvement in memory consumption for certain pipeline and analysis tasks (due to the changes in how output parameters are handled, see below).

• • An automatic tool in HIPE will detect scripts with multiple outputs tasks: In case you open such a script then a dialog box pops up showing which lines need to be changed and asking your permission to convert it. Its current limitations are described in the Known Issues for HIPE 12 page. See Task parameters if your script contains any of the syntax features that are not currently implemented or if after running the conversion there are still errors that could be related to output parameters.

• If the Data section of the Observation, Product or Dataset viewers is collapsed (by clicking on its header), the available space is given to the Meta Data section.

• SimpleCube.copy() and SimpleImage.copy() create accurate copies now.

• Display does also work when image contains a TableDataset.

• SkyAperturePhotometry tasks can now be opened and filled from the command line. An example :

• ImageCeil, imageFloor and imageRound tasks allow the user to decide which decimal is to be rounded.
• ImageDivide also works with images without a unit.
• The output of the MosaicTask now has the metadata of the first image of the mosaic.
• Fixes in the GUI of SkyAperturePhotometry tasks.

• • If an observation is not found locally (and the main MyHSA is connected to the archive) the Archive will be tried.

• fitHifiFringe

• fitBaseline

• • works on a simpleSpectrum .

• hebCorrection

• • Added a new subsection called "Caveats" in chapter 11, "HIFI Baseline Removal" chapter, section "The FitBaseline Task".

• JScanam: We have introduced significant improvements in the JScanam pipeline. Most of JScanam task run now considerably faster and require less memory than in their previous HIPE 11 version, specially when the debug option is deactivated. The quality of the final products has also improved. In particular, the task scanamorphosIndividualDrifts is reducing the correlated noise in parallel observations in a better way.

• The pointing offset correction ipipe scripts for line (ChopNodPointingCorrectionLineScan.py) and range scans (ChopNodPointingCorrection.py) of point sources are made available in the hipe pipeline menu inclusive detailed documentation on the pointing offset correction algorithms.
• For the point source pointing offset correction pipeline 2 new tasks were developed to determine the pointing offset correction in case the

• • specDeterminePreCalculatedPointing: it calculates the pointing offset using the ra/dec/rollAngle information of the Frames.
  • specDeterminePointingOffsetFromPreCalculatedPointing: it determines the flux correction factors by using the pointing offset determined by specDeterminePreCalculatedPointing.

• A new task photTevCorrection() and related calibration file is available to correct for the dependence of the responsitivity to the evaporate temperature. The correction is typically below 1%. Only 1.2% of the observation will have a larger correction.

• Improved the browse product structure (numbers changed to meaningful names).
• Added a new observation summary dataset to the browse products. The obsSummary task shows the contents of this dataset and it can be viewed also in the Observation Context Viewer. Note that in order to update this summary to reflect changes in the observation context you have to run this task with forceUpdate=True.

• Added new meta keyword sliceNumber to slices. This number also appears in the FITS filenames in the pools, which makes them more readable.

• All spectra and spectral cubes are now corrected to the Local Standard of Rest. The frequency grid is interpolated back to the original scale so that all observations still have a consistent frequency grid. A metadata keyword, freqFrame (or SPECSYS in FITS), tells you whether the LSR correction has been applied or not.

• • Remake the image with the identified glitches masked out.

• A new SPIA task has been added to carry out background subtraction, called "spiaFtsBackgroundRemoval".

• • Sersic profile added.
  • The ability to select a Gaussian reference beam has been added.
  • A threshold parameter to exclude off-axis detectors from the output when they do not overlap with the source.

• Substantial updates to both the SPIRE Data Reduction Guide and SPIRE Pipeline Specification Manual.

• Drastic improvement in memory consumption for certain pipeline and analysis tasks (due to the changes in how output parameters are handled, see below).

• Updates to CleanDF tables (Level 0). These tables allow the pipeline to omit corrupted data frames.
• Nonsense LO settings in spectral scan (Level 0)
• Calibration tables for Electrical Standing Wave task hebCorrection: WBS-Splines (6aH, 6aV, 6bH, 6bV, 7aH, 7aV, 7bH, 7bV)

• FitHifiFringe
  • Input parameter data replaces the multiple inputs such as "obs_or_htp", "simple_spectrum", "sds", and "cube".
  • For the last (or single) spectrum, the move to next option has been removed from the pop-up dialogue window. The task exits more elegantly and leaves the two plot windows open (for the default plot=2 option).

• FitBaseline
  • Input parameter data replaces the multiple inputs such as "obs", "htp", "sds", and "simpleSpectrum".
  • now works from within SpectrumExplorer to create and manipulate line mask windows and to allow plot scaling with the right mouse button.
  • works on a simpleSpectrum
  • skips a spectrum if "empty".

• • The backends option now works correctly - only the ones selected are processed. This is true for obs, htp, and sds.
  • If data cannot be fitted fitBaseline will move onto the next dataframe. Previously it hung.

• hebCorrection
  • A new task is now available to help correct the Electronic Standing Waves in bands 6 and 7. See instructions for use in the HIFI Data Reduction Guide

• DoGridding uses a new command line syntax. This is a result of the general change in behaviour of Tasks with multiple outputs.

• New GUI figure to reflect the addition of an input parameter for htp

• It is now possible to open the hiClass task (to export in FITS) by using a right-click on a variable (Send to / CLASS FITS file)

• • Update to chapter 9, flagTool GUI figure (possible input parameters are now "obs" and "htp"). Adjusted text accordingly.
  • Update to chapter 10: "Standing Wave Removal, fitHifiFringe GUI figure to reflect change in input parameter named data. Adjusted text accordingly.

• • Update to chapter 11, fitBaseline GUI figure to reflect change in input parameter named data. Adjusted text accordingly.

• • Added tasks doMainBeamTemp, polarPair, doFold, and mkRms in General Pipeline, Level 2.5 Pipeline section

• • An automatic tool in HIPE will detect scripts with multiple outputs tasks: In case you open such a script then a dialog box pops up showing which lines need to be changed and asking your permission to convert it. See Task parameters if this fix did not work.

Analysis

Spectral Cubes

Display

• All cube tasks now propagate the description of the cubes, the instrument name and the units to the result.

• Tasks with multiple outputs now return a list (__list__ argument deprecated) to save memory.
  • Errors mentioning ArrayList suggest that your code is using tasks that return multiple outputs and you need to update it.
  • An automatic script in HIPE will detect scripts with multiple outputs tasks and will try to correct their syntax. See Task parameters if this fix did not work.

• There is no strong reason to reprocess HIFI data with this release. However for data taken in bands 6 and 7 (HEB bands) it is recommended to try and remove the electrical standing waves with the new dedicated task.

What's new in HIPE 12

• Tasks with multiple outputs return a list (__list__ argument deprecated). (This information is to be expanded by Pat Morris)
  • Errors mentioning ArrayList suggest that your code is using tasks that return multiple outputs and you need to update it. See Task parameters

• DoGridding task: new command line syntax in line with the general behavior of Tasks with multi output (see documentation update).

• • Added 3 figures to Introduction, How to get your data. Updated text.

• • Update to chapter 13, "How to make a spectral cube", section "...to change beam, pixel, map size". Introducing new syntax (for tasks with multiple results) to create cubes.
  • Update to chapter 20, Added a note for HIPE 12 on right-click on variable to open hiClass GUI. Improved method to export results of deconvolution to Class.

• A new task was introduced convertToFixedPixelSize() that performs the optical distortion correction. It is now applied in MADmap ipipe and will be used in the SPG12 processing, correcting for a 6% flux scale overestimate in the red channel and 2% underestimate in the blue channel.

• A new task photTevCorrection() is available and used for SPG, see section on calibration.

• There is a new task for chopped observations photCalPsToScan() that accounts for the difference in responsivity as compared to the scan map flux calibration scheme. When applying this task, the photometry of pint sources observed with the chopped photometry mode are in line with the results from scan map observations. For details of the method see the publication in Experimental Astronomy.

• A correction is applied on level 1 frames to correct for influences of the evaporator temperature.

• A correction is applied on level 1 frames of the chopped photometry data to correct for differences in the responsivity as compared to scan maps.

• There is a new task for chopped observations photCalPsToScan() that accounts for the difference in responsivity as compared to the scan map flux calibration scheme. When applying this task, the photometry of pint sources observed with the chopped photometry mode are in line with the results from scan map observations. For details of the method see the publication in Experimental Astronomy.

• a new task photTevCorrection() is available and used for SPG, see section on calibration.

• a correction is applied on level 1 frames for the evaporator temperature.

• A new task photTevCorrection() and related calibration file is available to correct for the dependence of the responsitivity to the evaporate temperature. The correction is typically below 1%. Only 1.2% of the observation will have a larger correction.

• The installer now bundles one specific version of the Java Runtime Environment (JRE) for Mac OS as it is already done with Linux and Windows. This will help isolate HIPE from the JRE installed at operating system level, but has the inevitable consequence of adding about 50 MB to the installer size.

Interactive Analysis and Tools

Photometer

• a new task was introduced convertToFixedPixelSize() that performs the optical distortion correction. It is now applied in MADmap ipipe and will be used in the SPG12 processing, correcting for a 6% flux scale overestimate in the red channel and 2% underestimate in the blue channel.

• JScanam: We have introduced significant improvements in the JScanam pipeline. Most of JScanam task run now considerably faster and require less memory than in their previous Hipe 11 version, specially when the debug option is deactivated. The quality of the final products has also improved. In particular, the task scanamorphosIndividualDrifts is reducing the correlated noise in parallel observations in a better way.

SPG Pipeline

Photometry

• JScanam maps will be available in level 2.5

• MADmap maps will be corrected for the flux scale due to the fact that optical distortion was not applied so far.

Spectrometer

• Point source calibration for off-axis detectors corrected to account for the pointing of Uranus in the calibration measurements. Only affects the off-axis detectors. Centre detector point source calibration changes by less than 0.15%.
• Unused Spatial Correction Coefficient product removed (was designed for use with the semi-extended correction tool, but never actually used).

• New SPIRE Calibration Automatic Updater introduced: it runs during HIPE startup and notifies the user if a new calibration tree is available. The tool allows also the installation of the new calibration tree.
• New SPIRE Calibration View introduced: this is a new HIPE view callable from HIPE view menu. The view shows all the available SPIRE calibration trees with their release notes, and allows installation/removal of each tree. Different colours are used to distinguish default/installed/not installed trees.
• New SPIRE Calibration Preferences panel added to the HIPE preferences: the preferences panel allows some configurations for the SPIRE Calibration View and the SPIRE Calibration Automatic Updater.

• All spectra and spectral cubes are now corrected to the Local Standard of Rest. The frequency grid is interpolated back to the original scale so that all observations still have a consistent frequency grid. A metadata keyword, freqFrame (or SPECSYS in FITS), tells you whether the LSR correction has been applied or not.
• Vignetted detectors are now included in spectral maps by default. The calibration of the vignetted outer ring of detectors was found to be reasonably consistent with the other detectors. This increases the area covered by the maps.
• The frequency axis for spectral cubes is now contained in the 3rd axis of the WCS (to match HIFI cubes), rather than in a separate dataset called ImageIndex.
• SpireWcsCreator has been renamed SpecWcsCreator to distinguish it from the photometer WCS task.
• Pipeline tasks with multiple outputs must now be specified explicitly in the pipeline - this affects 5 tasks: waveletDeglitcher, sigmaKappaDeglitcher, baselineSubtraction, phaseCorrection and spireProjection. Pipeline scripts have been updated with the new syntax, which is explained in the SPIRE Pipeline Specification Manual.
• The functionality of the preCube product was extended to make it easier to add or remove spectra.

• A new SPIA task has been added to carry out background subtraction, called "spiaFtsBackgroundRemoval"
• The Semi-extended Correction Tool has been updated:
  • Sersic profile added
  • The ability to select a Gaussian reference beam has been added
  • A threshold parameter to exclude off-axis detectors from the output when they do not overlap with the source
• A new tool is available to calculate FTS synthetic photometry called "specMatchPhot".
• Small additions/corrections to the FTS useful scripts.
• SpecWcsCreator allows a WCS to be calculated in instrument coordinates.

• Substantial updates to both the SPIRE Data Reduction Guide and SPIRE Pipeline Specification Manual
• Papers published in Experimental Astronomy:
  • Herschel SPIRE FTS Relative Spectral Response Calibration
  • Herschel SPIRE FTS telescope model correction
  • Herschel SPIRE Fourier Transform Spectrometer: Calibration of its Bright-source Mode
  • Relative pointing offset analysis of calibration targets with repeated observations with Herschel-SPIRE Fourier-Transform Spectrometer
  • Herschel SPIRE FTS Spectral Mapping Calibration
  • SPIRE Point Source Photometry

• • Added four new cookbooks to chapter 2: "HIFI Cookbooks": Point Mode Frequency Switch, Point Mode Position Switch, Point Mode Load Chop, and Spectral Scan.

• • Errors mentioning ArrayList suggest that your code is using tasks that return multiple outputs and you need to update it. See Task parameters

• Updates to CleanDF tables (Level 0)
• Non-sense LO settings in spectral scan (Level 0)
• Calibration tables for Electrical Standing Wave task *hebCorrection*: WBS-Splines (6aH, 6aV, 6bH, 6bV, 7aH, 7aV, 7bH, 7bV)

Electrical Standing Wave correction in HEB bands

• hebCorrection
  • A new task is now available to help correct the ESWs in bands 6 and 7. Details can be found in Chapter 10, section 10.4 (see documentation update section below)

• • Added three new cookbooks to chapter 2: "HIFI Cookbooks": Point Mode Frequency Switch, Point Mode Load Chop, and Spectral Scan.

• • Regular updates to chapter 8: "Flags in HIFI data", specifically to the Quality Flags Table.

• • Update to chapter 10: "Standing Wave Removal, fitHifiFringe GUI figure to reflect change in input parameter named data. Adjusted text accordingly.
  • Update to chapter 10: "Standing Wave Removal, specifically to the section "Electrical Standing Wave correction in HEB bands".

• • Updates to chapter 12: Sideband Deconvolution.

• • Added tasks doMainBeamTemp, polarPair, doFold, and mkRms in General Pipeline, Level 2.5 Pipeline section

• • Errors mentioning ArrayList suggest that you code is using tasks that return multiple outputs and you need to update it. See Task parameters

• The quality of some Level-3 mosaic maps has been improved by constructing them from Level-2.5 maps where available.

General

• A tool is now available to match the FTS continuum with the SPIRE photometry.

Photometer

• The Maximum Correlation Method (MCM) super-resolution algorithm developed for IRAS data can be applied to SPIRE photometer maps and is now available as the "hiresMapper" task. A useful script is also provided to help with using it.
• The new "Bolometer Finder Tool" provides a friendlier front-end to the "boloFinder" task, and functions as an interactive deglitching tool. In principle it makes the corresponding "useful script" obsolete. Its features include:
  • Identify a suspect pixel by image visualisation.
  • Plot the timeline data contributing to a selected pixel.
  • Identify glitches by clicking on the plots.
  • Remake the image with the identified glitched masked out.
  • Undockable tabs to allow side-by-side comparisons.
  • Save the modified masks back to the Level-1 and Level-0.5 timelines.
• SPIA (SPIRE Photometric Interactive Analysis), previously available as a plugin, is now integrated into HIPE.

Spectrometer

• Semi-Extended Correction Tool TBW

• Drastic improvement in memory consumption for certain pipeline and analysis tasks.
• Improved contents and documentation in the headers of FITS files generated with HCSS or HIPE.

• Special task to mitigate electronic standing waves in HIFI HEB bands.
• There is no strong reason to reprocess HIFI data for bands other than the HEB bands with this version.

• Systematic pipeline produces Level 2.5 and Level 3 mosaics.
• All observations should be reprocessed with this HIPE version.

• Super-resolution map-making algorithm for SPIRE-PHOT data.
• There is no strong reason to reprocess SPIRE-PHOT data with this version and for the SPIRE-SPEC data it depends on the science case.

• New GUI figure to reflect the addition of an input parameter for htp

• • Input parameter *data* replaces the multiple inputs such as "obs_or_htp", "simple_spectrum", "sds", and "cube". (New GUI figure to reflect the change)

• • Input parameter *data* replaces the multiple inputs such as "obs", "htp", "sds", and "simpleSpectrum". (New GUI figure to reflect the change)

• New GUI to reflect the addition of an input parameter for htp

• It is now possible to open the hiClass task (to export in FITS) by using a right-click on a variable (Sent to / CLASS FITS file)

• • Added a figure to Introduction, How to get your data.
  • Updates to the doDeconvolution chapter.
  • Regular updates to the Quality Flags Table in the "Flags in HIFI data" chapter.
  • Update to section 4.2 "What was done to my data" / "Level 0".
  • Update to section 5.4 "Re-pipelining with a new Auxiliary product".
  • Update to chapter 9, flagTool GUI figure (possible input parameters are now "obs" and "htp"). Adjusted text accordingly.
  • Update to chapter 10, fitHifiFringe GUI figure to reflect change in input parameter named data. Adjusted text accordingly.
  • Update to chapter 11, fitBaseline GUI figure to reflect change in input parameter named data. Adjusted text accordingly.

• • Update to chapter 13, "How to make a spectral cube", section "...to change beam, pixel, map size".
  • Update to chapter 20, Added a note for HIPE 12 on right-click on variable to open hiClass GUI.

• the pointing offset correction ipipe scripts for line (ChopNodPointingCorrectionLineScan.py) and range scans (ChopNodPointingCorrection.py) of point sources are made available in the hipe pipeline menu inclusive detailed documentation on the pointing offset correction algorithms
• for the point source pointing offset correction pipeline 2 new tasks were developed to determine the pointing offset correction in case the accurate pointing product is available for an observation.
  • specDeterminePreCalculatedPointing: it calculates the pointing offset using the ra/dec/rollAngle information of the Frames.
  • specDeterminePointingOffsetFromPreCalculatedPointing: it determines the flux correction factors by using the pointing offset determined by specDeterminePreCalculatedPointing

• For point source pointing offset flux calibration a time dependent correction to the current telescope background model is implemented. The calfiles for each spectral band are called TelBackCorR1, TelBackCorB2A, TelBackCorB2B, TelBackCorB3A and used by the corresponding task called specCorrectTelescopeBackground

• Detailed documentation of the point source pointing offset correction algorithm is delivered, which is a general description of the processing and a detailed description of the single tasks used including a flow chart.

• FitHifiFringe
  • Input parameter *data* replaces the multiple inputs such as "obs_or_htp", "simple_spectrum","sds", and "cube".
  • For the last (or single) spectrum, the *move to next* option has been removed from the pop-up dialog window. The task exit more elegantly and leaves the two plot windows open (for the default plot=2 option).

• FitBaseline
  • FitBaseline should now work from within SpectrumExplorer in handling line mask windows, draw 'curtains' and allow plot scaling with right mouse button.
  • FitBaseline should now work on a simpleSpectrum (a.k.a Spectrum1d).
  • FitBaseline will skip spectrum if "empty".
• Bug Fixes:
  • The backends option now works correctly - only the ones selected are processed. This is true for obs, htp, and sds.
  • If the task cannot fit the data, it no longer enters into a loop but the task nicely moves on to the next data-frame.

• Improved the browse product structure (numbers changed to meaningful names)
• Added a new observation summary dataset to the browse products. The obsSummary task shows the contents of this dataset and it can be viewed also in the Observation Context Viewer. Note that in order to update this summary to reflect changes in the observation context you have to run this task with 'forceUpdate=True'.
• Improved the names of many FITS keywords, which makes the headers of PACS products easier to read outside HIPE.
• Added new meta keyword sliceNumber to slices. This number also appears in the FITS filenames in the pools, which makes them better readable.
• Fixed some problems with copying products.

• Improved the meta keywords which describe the slices. The following keywords were added:
  • meanWave: the mean wavelength of the wavelength range of a slice.
  • prime: True if a slice belongs to the prime channel, false otherwise.
  • inLeak: Can be 'yes', 'no' or 'partial', depending on whether a slice is fully, not or partially in the leak region.
  • repetition: the repetition number of a slice.
  • gratingStep: the grating step size within a slice.
• Added more flags to the rebinned cubes: We now propagate besides the saturation masks also all other inactive masks into the Flag dataset of the rebinned cubes.

• Improved the table of contents of the PACS User Reference Manual.

• • Update to chapter 13, "How to make a spectral cube", section "...to change beam, pixel, map size"

• Improved the display of cubes in the Spectrum Explorer and added the panner with compass to the display panel.
• Improved loading large cubes.

• The cropCube task now also accepts ranges in wavelength units.
• The moment algorithm of computeVelocityMap has been significantly improved. Especially the error on the dispersion is now much smaller.
• Fixed a bug in integrateSpectralMap when using a cube with regular wavelength grid (in WCS).
• All cube tasks now propagate the description of the cubes to the result.

• Updates to the HIFI Data Reduction Guide:
  • Added a figure to Introduction, How to get your data
  • Updates to the doDeconvolution chapter
  • Regular updates to the Quality Flags Table in the "Flags in HIFI data" chapter
  • Update to section 4.2 "What was done to my data" / "Level 0"
  • Update to section 5.4 "Re-pipelining with a new Auxiliary product"
  • Update to chapter 13, "How to make a spectral cube", section "...to change beam, pixel, map size"
  • Added a new subsection called "Caveats" in chapter 11, "HIFI Baseline Removal" chapter, section "The FitBaseline Task".
• Updates to the HIFI Pipeline Specification manual
  • Added task doFold, in General Pipeline, Level 2 Pipeline section

• New Spire Calibration Automatic Updated introduced: it run during HIPE startup and notifies the user if a new tree is available. The tool allows also the installation of new calibration tree.
• New Spire Calibration View introduced: this is a new HIPE view callable from HIPE view menu. The view shows all the available calibrations trees, their release notes and allows installation/removal of each tree. Different colors are used to distinguish default/installed/not installed trees.
• New Sire Calibration Preferences added to HIPE preferences: the preferences panel allows some configurations for the Spire Calibration View and the Spire Calibration Automatic Updater.

• Improvements in user-friendliness (GUI layout, tooltips, harmonisation of Python syntax in HIPE)
• Improvement of SPG interface for DP TAs to ease future reprocessing exercises

• Performance issues for HIFI pipeline

• The internal name for the Data Analysis Guide was renamed from howtos to dag. Now when searching the documentation the results within this manual appear below the dag name.
• The internal name for the Scripting Guide was renamed from um to sg. Now when searching the documentation the results within this manual appear below the sg name.

• The raster panel allows to show coordinates in either sexagesimal or decimal format.
• Text of some popup menu options improved.

• Observation variables created by drag & drop from the Product Browser onto the Variables View have more sensible names.
• If a drop in the Variables View is not valid for certain variable (for example, trying to drop a non-observation in the Observations section), a forbidden icon will be shown to let the user know.

• The reduced browse image at the top-right corner of the Observation viewer has much better quality, so details can be distinguished even without having to enlarge it. The same applies for observations shown in the Outline.

• If the Data section of the Observation, Product and Dataset viewers is collapsed (by clicking on its header), the available space is given to the Meta Data section.
• A preference to restart Jython scripts when finished has been added. It is disabled by default, but can be enabled in the Editors & Viewers > Text Editor > Jython Editor preference panel.
• When autosave of text editors is on, a backup file is generated only when the original file is modified.

• Multiple result rows can be selected at once in the Product Browser and create variables for them in a single drag & drop operation to the Variables View.

• Minimized views are remembered as part of the persisted layout of perspectives.
• HIPE now asks to restart upon changes that require a new session in order to be applied.

Directories

From the General > Directories preferences panel, the user can:

• Select the default directory where to open file dialogues.
• Select the working directory, which will be used when reading or saving files with no absolute path specified.

Drag & Drop

• Multiple result rows can be selected at once in the Product Browser and create variables for them in a single drag & drop to the Variables View.
• Files can be dragged from the Navigator and dropped in different places:
  • In the Console, they echo their absolute path.
  • Within task dialogs, in parameters expecting a file path.

Editors & Viewers

• Map and List viewers have been improved. It is possible to open their elements in separate viewers, or to create variables for them.

Miscellany

• Perspectives: Minimized views are remembered as part of the persisted layout of perspectives.
• Restart: HIPE now asks to restart upon changes that require a new session to be applied.

To execute scripts in batch mode, run hipe with file path arguments.

Navigator

Files can be dragged from the Navigator and dropped in different places:

• In the Console, they echo their absolute path.
• Within task dialogs, in parameters expecting a file path.

Status bar

Any code can echo messages in the status bar. For example, from a Jython task:

from herschel.ia.gui.kernel.util import SiteUtil
SiteUtil.getSite().getStatusManager().submit(message)

• • Added tableType parameter that can generate output in fixed-width format (3 spaces between columns).

• convertAngles : (no longer a prototype) It converts from numeric to string formats and vice versa.

• SimpleCube.copy() and SimpleImage.copy() create accurate copies now.
• WCS 3rd axis does follow FITS convention for reference pixel now.
• The WCS metadata is copied to all planes from images (spectra).

• Display ignore flags when the description is prefixed with 'INFO:'.
• Colour table drawn in the axes is corrected.
• Display does also work when image contains a TableDataset.
• Zoom does work also on RGB images.

• SkyAperturePhotometry tasks can now be opened and filled from the command line. An example :

openTask("annularSkyAperturePhotometry",inputName="image,radiusArcsec,centerX,centerY,innerArcsec,outerArcsec",variableName="myImage,myRadiusArcsec,xCenter,yCenter,innerRadius,outerRadius")

• ImageCeil, imageFloor and imageRound tasks allow the user to decide which decimal is to be rounded.
• ImageDivide also works with images without a unit.
• The output of the MosaicTask now has the metadata of the first image of the mosaic.
• Fixes in the gui of SkyAperturePhotometry tasks.

• TemperatureDriftCorrectionTask now reacts to the "coolerBurpDetect" parameter defined in the timeline.

Release Notes

• Full list of tickets implemented: here.

• Tasks with multiple outputs return a list (__list__ argument deprecated).
  • Errors mentioning ArrayList suggest that you code is using tasks that return multiple outputs and you need to update it. See Task parameters
• Task GUIs: You can now drag and drop from the Navigator into 'file' arguments.

• • metadata and table descriptions are now read.

• • FixedWidthFormatter: can easily generate an appropriate formatter passing a table.

• • quality flags: (ongoing) now generate keywords.

• asciiTableWriter :
  • default value for warn has changed to false. By default the system will not ask for confirmation if the file to be written already exists.
  • By default, it will save meta-data.
  • Added tableType parameter that can generated output in fixed-width format (3 spaces between columns).
  • Simplified GUI and improved documentation.
• asciiTableReader :
  • Added null conversion parameters.
  • Simplified GUI and improved documentation.
• convertAngles : no longer a prototype. It converts from numeric to string formats and vice versa.
• convertUnits : several improvements.
• Export View : simplified GUI.
• getObservation:
  • improved GUI.
  • If an observation is not found locally (and the main MyHSA is connected to the archive) the Archive will be tried.
• help : Also provides access to Python help.
• openTask : you can now open a task attaching variables to any parameters.

• Sso: Added a variant that uses internal Horizons.
• radialV and RadialV.getProjection(...) : if Ephem is null the system will use its own if needed.

• asciiTableWriter :
  • default value for warn has changed to false. By default the system will not ask for confirmation if the file to be written already exists.
  • Added tableType parameter that can generated output in fixed-width format (3 spaces between columns)
• asciiTableReader :
• Table reading:
  • metadata and table descriptions are now read
  • nulls are converted to valid values (can be user supplied).
• Table writing:
  • FixedWidthFormatter: can easily generate an appropiate formatter passing a table.
• FITS:
  • BUNIT is now written
  • quality flags: (ongoing) now generate keywords

Batch mode

The hipe executable needs to be used to run scripts in batch mode, as jylaunch was removed

<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->

<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's coming up in HIPE 12

Interested in what's new for other versions of HIPE? See this page for links to all the What's New documents from HIPE 4.0 onwards.

See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.

Planned work

HIPE

• Gyro-propagation for pointing products
• List-return syntax for tasks (reduction of memory leaks)
• improvements in user-friendliness (GUI layout, tooltips, harmonisation of Python syntax in HIPE)
• improvement of SPG interface for DP TAs to ease future reprocessing exercises
• Framework for on-demand processing profiles
• Creation of a multi-observation context to reduce download time for users

HIFI

• performance issues for HIFI pipeline
• HEB standing wave mitigation

PACS

• Level 3 products
• Tailored level 2.5 products (extended vs. pointsource)

SPIRE

• Alternative Error maps for naïve mapper Provide the user with a set of alternative error maps including standard (no weighting), bolometer noise weighted and hybrid cases for low coverage maps
• Destriper improvements
  • Improved Level 2 deglitching algorithms
  • Destriper working on entire timelines
  • Implementation of weighting into destriping
• Automatic correction of data affected by Cooler burps
• Detection of bolometer jumps in Destriper
• Continued improvements on Zero Point Calibration with Planck
• Flux Calibration: New flux calibration using new Neptune fluxes from (ESA4?) models Will produce new flux calibration files and new Temp Drift Correction calibration files (and new Relative gain correction)
• Super Resolution Maps: Port the HiRES code into HIPE as a task for Super resolution mapping
• Iterative pipeline models to improve deglitching and jump detection.
• Quick pipeline without FFT modules up to the Level 2 deglitcher in the destriper that flags glitches in the Level 0.5 timelines and then we run the full pipeline again.
• Documentation
  • Continued updates to DRG
  • Continued updates to PSM
  • Homogenization and review of product metadata

Highlights

HIPE

HIFI

PACS

SPIRE

Core system

HIPE

Plotting

TablePlotter

OverPlotter

Dataset viewer

Astronomical utilities

Numeric routines

Data fitting

Data interpolation

Random numbers

Statistics

Trend analysis

Mapping

Other functions

Source extraction

Images

Display

Analysis

Spectra

Display

Analysis

Data cubes

Display

Analysis

Herschel Science Archive

Calibration sources

Products and datasets

Observation context

Product Access Layer

Tasks

Quality control

Systematic product generation

Data input-output

Virtual Observatory

Installer

Documentation

HIFI

HIFI Software Configuration and Properties

HIFI Calibration

HIFI Pipeline

hifiPipeline task

Level 0 Pipeline

WBS Pipeline

HRS Pipeline

Level 1 Pipeline

Level 2 Pipeline

Level 2.5 Pipeline

HIFI Products

HIFI Spectrum Toolbox

Spectrum Explorer (HIFI specifics)

Spectrum tasks (HIFI specifics)

HIFI Data Processing Tools

Standing Wave Removal

Baseline Removal

Deconvolution

Spectral Cube creation

flagTool

HiClass export tool

Documentation

PACS

Pipeline

Photometry

Spectroscopy

Calibration

Photometry

Spectroscopy

Analysis tools

Photometry

Spectroscopy

Product changes

Photometry

Spectroscopy

Documentation

SPIRE

Calibration Products

Calibration Framework

Preprocessing

Engineering Conversion

Common Pipeline

Photometer Pipeline

Spectrometer Pipeline

Interactive Analysis and Tools

Documentation

<-- COMMENT BOX CODE - DO NOT EDIT -->

<-- END OF COMMENT BOX CODE -->

<-- 

 Set ALLOWTOPICCHANGE = RegisteredUsersGroup, TWikiAdminGroup
 
-->

-- StephanOtt - 01 Mar 2013