Difference: HipeWhatsNew13x (1 vs. 68)

Revision 682015-09-29 - LucaCalzoletti

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in HIPE 13

Changed:
<
<
PDF Version Click here
>
>
PDF Version Click here
 
Help Interested in what's new for other versions of HIPE? See this page for links to all the What's New documents from HIPE 4.0 onwards.
See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.
Line: 15 to 14
 

Highlights

  • Core
Changed:
<
<
>
>
 
    • HIPE now opens ALMA files and identifies ALMA cubes as spectral cubes, displayed in the Spectrum Explorer.
    • A rescue mode has been added to HIPE for cases where intensive scripts or memory thrashing issues make the GUI unresponsive.
Line: 44 to 43
 
  • A new level 2.5 product has been generated for point mode observations. It is a stitched version of the level 2 spectra for each available spectrometer, in the form of a Spectrum1d.
  • All observations context will now contain a (Ta*) calibrated OFF position spectrum, in the same sky frequency scale as the ON position data.
Changed:
<
<
  * The new calibration tree contains updated beam models and efficiencies, as released initially as stand-alone products in September 2014. See for more details http://herschel.esac.esa.int/twiki/pub/Public/HifiCalibrationWeb/HifiBeamReleaseNote_Sep2014.pdf
>
>
 
  • In bands 6 and 7, the Electrical Standing Wave correction is now part of the standard pipeline, based on a data-base of optimised solutions stored in the calibration tree for each observation
  • For spectral scans, a significant fraction of the deconvolved level 2.5 products will benefit from flag tables optimised manually by the ICC, resulting in spur-free deconvolved spectra. This is not yet available for all spectral scan obsids though, but will be by HIPE 14. The list of optimised flags can be found in the calibration tree and further refined by the user using the flagTool if the User wishes so.
  • HIFI has a particular approach to the new gyro-based pointing reconstruction. The peculiarity of the approach and recommendation to the archive user are summarised in this memo.
Line: 53 to 52
  PACS
Changed:
<
<
   * TBD
>
>
  • TBD
  SPIRE
Changed:
<
<
   * TBD
>
>
  • TBD
  -->
Line: 72 to 71
 
import sys
Changed:
<
<
print sys.argv
>
>
print sys.argv
 

Then the output of this script would be:

Line: 81 to 80
 
> hipe test.py arg1 arg2 arg3
Changed:
<
<
['/home/user/test.py', 'arg1', 'arg2', 'arg3']
>
>
['/home/user/test.py', 'arg1', 'arg2', 'arg3']
 

Transitions

Line: 105 to 104
 
  • Backups for new files are automatically created, and reopened in a subsequent session upon crashes like a power cut. If the session terminates gracefully, the backup files are deleted.
  • Editing metadata and dataset cells in the Product or Observation viewers keeps the result in memory, and the table is refreshed accordingly with the new value.
  • Animations when maximizing or restoring parts of the viewer will be shown (see Transitions).
Changed:
<
<
  • The browse image of the Observation viewer can be opened with the Image viewer by means of Right click > Open in image viewer.
>
>
  • The browse image of the Observation viewer can be opened with the Image viewer by means of Right click > Open in image viewer.
 
  • The ArrayDataset viewer shows the unit when specified.
  • Column objects from a TableDataset have now a custom viewer.

Status bar

Changed:
<
<
From the Preferences > Data Access > MyHSA > Advanced panel, the user can:
>
>
From the Preferences > Data Access > MyHSA > Advanced panel, the user can:
 
  • Show the MyHSA on-line/off-line checkbox in the status bar.
  • Hide the MyHSA on-line/off-line checkbox in the status bar.
Line: 128 to 127
  If HIPE gets in a non responding state for a period of time, it is automatically detected so that a popup window asks to restart it in an ordered way. This means all unsaved editors will be saved, and a new fresh HIPE session with the state of the frozen one will be opened.
Changed:
<
<
The time intervals to run this check and to consider the application as not responding can be configured in Preferences > General > Rescue, where this mechanism can also be disabled altogether if you prefer so.
>
>
The time intervals to run this check and to consider the application as not responding can be configured in Preferences > General > Rescue, where this mechanism can also be disabled altogether if you prefer so.
 

Miscellany

Changed:
<
<
  • The Help > Check Java platform option shows the running Java version.
>
>
  • The Help > Check Java platform option shows the running Java version.
 
  • The Login panel is shown back at the top left corner of the Data Access perspective, and it has been simplified.
Changed:
<
<
  • HIFI is now providing a handful of useful scripts - see the menu Scripts > HIFI Useful Scripts.
>
>
  • HIFI is now providing a handful of useful scripts - see the menu Scripts > HIFI Useful Scripts.
 

Plotting

  • PlotXY no longer has an explicit way of forcing a batch of plotting statements. This is now done automatically and the script developer must not do anything special to take advantage of it.

Astronomical utilities

Changed:
<
<
>
>
 
  • Special corrections in pointing products to handle S/C operational events have been included in the auxiliary processors. Namely, the corrections are:
    • STR switch-overs, i.e. changes from the usually prime star-tracker (STR1) to the backup unit (STR2)
    • Reset of the Spacecraft Velocity Vector (SVV) that is used by the STR to compute the aberration correction of the coordinates of guide stars.
Changed:
<
<
For more  information please refer to the the dedicated section in the Spacecraft and Observatory pages.
>
>
For more information please refer to the the dedicated section in the Spacecraft and Observatory pages.
 

Numeric routines

Changed:
<
<

 Mapping

>
>

Mapping

 
  • Temporal files for mapping are now created in a subdirectory: each HIPE session running will use a different subdirectory.

Images

Changed:
<
<

 Display

>
>

Display

  The following changes are made to the Display class:
  • Replaced setCutLevels() by calcCutLevels()
Line: 170 to 169
 
  • The zoom in / out was by a factor of 4 instead of 2.
  • The Wcs viewer only shows the important wcs keywords now.
Changed:
<
<

 Analysis

>
>

Analysis

  Bugfixes :
  • Mosaics close to the north or south pole missed a part of the sky.
Line: 179 to 178
 
  • You can choose to add an input parameter to annularSkyAperturePhotometry for choosing the step size of curve-of-growth output.

Spectra

Changed:
<
<

 Display

>
>

Display

  * The mosaic view of the Spectrum Explorer shows axes coordinates in a more convenient way, with less digits.
Changed:
<
<

 Analysis

>
>

Analysis

 
  • multiFit task now accepts limits for the model parameters.

Data cubes

Changed:
<
<

 Display

>
>

Display

 
  • Disable the selection tools when some other data cube than the flux is shown.
  • Fixed bug in which the upper and right rows and columns were not included in the selection for the cropCube task.
Changed:
<
<

 Analysis

>
>

Analysis

 
  • Moment algorithm in computeVelocityMap improved with 'line strength' weighting. Especially dispersion is more accurate.
  • Show proper model cube when absorption line fitting is done in computeVelocityMap.
Line: 209 to 208
  ParameterFactory added to easily create meta data parameters.
Changed:
<
<
Several columns and meta-data keywords related to the gyro-based attitude reconstruction process have been added to the pointing product. See a brief description of the changes in the Spacecraft and Observatory pages.

Product Access Layer 

>
>
Several columns and meta-data keywords related to the gyro-based attitude reconstruction process have been added to the pointing product. See a brief description of the changes in the Spacecraft and Observatory pages.

Product Access Layer

 

Tasks

  • Conversion tool to adapt multi-output tasks to new style in off-line User scripts.
Changed:
<
<
  • Warning, important __list__ argument removed: if your task call still has it, remove it.
>
>
  • Warning, important __list__ argument removed: if your task call still has it, remove it.
 
  • Preference "Generate task calls that returns full set of outputs" removed: tasks GUIs now always generate a call with the expanded list of outputs assigned to.
  • Task status messages appear in the status bar
  • convertUnits : It can convert units from double parameters, array datasets and columns, plus units from table datasets or products, by allowing to select the target dataset or column.
Deleted:
<
<

 

Quality control

  • It is possible to create Quality Control summaries for several instruments in one go.
Line: 318 to 303
 

HIFI Data Processing Tools

Standing Wave Removal

  • Updates to default band-dependent period characteristics in task fitHifiFringe :
Changed:
<
<
    • Band 1 change nfringe 2->3, endPeriod 120->200
    • Band 3 change typicalPeriod 620->95
    • Band 4 change typicalPeriod 620->95
    • Band 6 change startPeriod 100->80, typicalPeriod 320->600
    • Band 7 change startPeriod 100->80, typicalPeriod 320->625
>
>
    • Band 1 change nfringe 2->3, endPeriod 120->200
    • Band 3 change typicalPeriod 620->95
    • Band 4 change typicalPeriod 620->95
    • Band 6 change startPeriod 100->80, typicalPeriod 320->600
    • Band 7 change startPeriod 100->80, typicalPeriod 320->625
 
    • Band 2,5 no change
  • swTable had wrong unit for amplitude (kHz instead of K). This is now fixed.
  • New output variable swTable was created to enable machine-readable output of fitted frequencies and amplitudes.
Line: 384 to 366
 
  • Level2 products: the quality of the High-pass Filtered Level2 products (HPPPMAP) is improved by adding the scanamorphosMaskLongTermGlitches() and the scanamorphosBaselinePreprocessing() tasks, which can detect jumps and correct for drifts on the detector timelines.
  • Level2.5 products: Unimap Level2.5 contexts contain the same types of maps generated by MADmap (naïve, GLS and Post-GLS), but they are arranged differently within the context (see Product changes).
  • Level3 products: High-pass Filtered maps (HPPHPFMAP) have been removed from Level3 products, leaving JScanam and Unimap maps. Level3 mosaics are created by using scanamorphosAlignImages(), a new task that links the flux by exploiting overlapping regions between images.
Changed:
<
<
  • Ipipe scripts: MADMap pipeline is still available as an interactive script in the Pipelines menu. The interactive script scanmap_Unimap allows users to run Unimap while tuning some parameters. It uses the new task runUnimap() that calls the Unimap release located on the user’s computer.
>
>
  • Ipipe scripts: MADMap pipeline is still available as an interactive script in the Pipelines menu. The interactive script scanmap_Unimap allows users to run Unimap while tuning some parameters. It uses the new task runUnimap() that calls the Unimap release located on the user’s computer.
 
  • Harmonisation of map pixel sizes: at any Level, maps have the following pixel sizes:
    • 3.2 arcsec for the red channel, Prime and Parallel Modes;
    • 1.6 arcsec for the blue channel, Prime Mode;
Line: 399 to 381
 
  • One of these three cubes is also provided with an equidistant wavelength grid, created with the new task specRegridWavelength.
  • Tables containing the data of the rebinned cubes are created produced with the new task pacsSpecCubeToTable (HPSTBRR and HPSTBRB). The spectral data of each spaxel of each raster position is included in a single table, and each wavelength range and camera is a separate table.
  • Flags are now also propagated by specProject and specInterpolate.
Changed:
<
<
 
>
>
 

Analysis tools

Spectroscopy

Line: 508 to 487
 

Interactive Analysis and Tools

Photometer

Changed:
<
<
  • Significant improvements to the bolometer finder tool have been made. 
>
>
  • Significant improvements to the bolometer finder tool have been made.
 

Spectrometer

Line: 517 to 496
 
  • New GUI interface for the script that determines the noise level in a spectrum.
  • Small corrections to the useful scripts.
Deleted:
<
<
 
 

Release Notes

Added:
>
>
* Full list of tickets implemented: here.
 
Added:
>
>

 
Deleted:
<
<
   * Full list of tickets implemented: here.
 
<-- COMMENT BOX CODE - DO NOT EDIT -->
Added:
>
>
 
Changed:
<
<
blog comments powered by Disqus
>
>
blog comments powered by Disqus
 
<-- END OF COMMENT BOX CODE -->
Deleted:
<
<

 
Added:
>
>

 
Deleted:
<
<
 -- AlvarGarcia - 15 Jan 2014

META FILEATTACHMENT attr="" autoattached="1" comment="pointing refinements" date="1420020588" name="Guide_gyro_recon_issue1.0.pdf" path="Guide_gyro_recon_issue1.0.pdf" size="857207" user="Main.JavierDiaz" version="1"

Revision 672015-04-13 - AlvarGarcia

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 200 to 200
 
  • Show proper model cube when absorption line fitting is done in computeVelocityMap.
  • More metadata in extractRegionSpectrum.
Deleted:
<
<

Herschel Science Archive

=HSAWritePool= now supports the removal of a list of tags.

 

Products and datasets

Changed:
<
<
A new Auxiliary Operational Day Summary Product which can be retrieved using the tag LATEST_OPERATIONAL_DAY_SUMMARY_PRODUCT. This product contains one dataset with the operational day number, start and end times. Useful methods exist for querying the data as documented here. A new Auxiliary Operational Day Summary Product which can be retrieved using the tag LATEST_OPERATIONAL_DAY_SUMMARY_PRODUCT. This product contains one dataset with the operational day number, start and end times. Useful methods exist for querying the data as documented here: OperationalDaySummaryProduct
>
>
A new Auxiliary Operational Day Summary Product which can be retrieved using the tag LATEST_OPERATIONAL_DAY_SUMMARY_PRODUCT. This product contains one dataset with the operational day number, start and end times. Useful methods exist for querying the data as documented here: OperationalDaySummaryProduct
  A new utility method in the Product class for querying a products duration.

Revision 662015-04-10 - MiguelSanchez

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 140 to 140
 
  • PlotXY no longer has an explicit way of forcing a batch of plotting statements. This is now done automatically and the script developer must not do anything special to take advantage of it.

Astronomical utilities

Changed:
<
<
  • Gyro-based attitude reconstruction, see pdf.
 
>
>
  • Gyro-based attitude reconstruction, see the dedicated section in the Spacecraft and Observatory pages.
  • Special corrections in pointing products to handle S/C operational events have been included in the auxiliary processors. Namely, the corrections are:
    • STR switch-overs, i.e. changes from the usually prime star-tracker (STR1) to the backup unit (STR2)
    • Reset of the Spacecraft Velocity Vector (SVV) that is used by the STR to compute the aberration correction of the coordinates of guide stars.

For more  information please refer to the the dedicated section in the Spacecraft and Observatory pages.

 

Numeric routines

 Mapping

  • Temporal files for mapping are now created in a subdirectory: each HIPE session running will use a different subdirectory.
Line: 211 to 216
  ParameterFactory added to easily create meta data parameters.
Changed:
<
<
>
>
Several columns and meta-data keywords related to the gyro-based attitude reconstruction process have been added to the pointing product. See a brief description of the changes in the Spacecraft and Observatory pages.
 

Product Access Layer 

Revision 652015-04-10 - MiguelSanchez

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 15 to 15
 

Highlights

  • Core
Changed:
<
<
>
>
 
    • HIPE now opens ALMA files and identifies ALMA cubes as spectral cubes, displayed in the Spectrum Explorer.
    • A rescue mode has been added to HIPE for cases where intensive scripts or memory thrashing issues make the GUI unresponsive.

Revision 642015-04-09 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 27 to 27
 
    • Many Spectral Scan observations benefit from flag tables optimised manually by the ICC, resulting in spur-free deconvolved spectra.
    • All observations now contain a (Ta*) calibrated OFF position spectrum, in the same sky frequency scale as the ON position data.
    • HIFI is using a particular approach in the usage of the new gyro-based pointing reconstruction. All details and impact on the HIFI data are explained in this memo.
Added:
>
>
    • New HIFI useful scripts
 
  • PACS
    • Photometer: New maps valid for point source and extended emission analysis: JScanam, UniMap (new!).
Line: 133 to 134
 
  • The Help > Check Java platform option shows the running Java version.
  • The Login panel is shown back at the top left corner of the Data Access perspective, and it has been simplified.
Changed:
<
<
>
>
  • HIFI is now providing a handful of useful scripts - see the menu Scripts > HIFI Useful Scripts.
 

Plotting

  • PlotXY no longer has an explicit way of forcing a batch of plotting statements. This is now done automatically and the script developer must not do anything special to take advantage of it.

Revision 632015-04-09 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 30 to 30
 
  • PACS
    • Photometer: New maps valid for point source and extended emission analysis: JScanam, UniMap (new!).
Changed:
<
<
    • Spectrometer: Cubes with evenly sampled spectral info. Improved mapping with additional 2D interpolation task.
>
>
    • Spectrometer: Cubes with evenly-sampled spectral grid. Rebinned cubes provided as tables for easier viewing outside HIPE. New cube interpolation task for undersampled mapping observations.
 
    • Spectrometer and photometer: many improvements to the structure of the observation context and names and descriptions of products.

  • SPIRE
Line: 381 to 381
 

Photometry

Changed:
<
<
  • Unimap: a new Generalized Least Square (GLS) mapmaker is introduced into the Standard Product Generator. Unimap is a MATLAB algorithm that is spawned from HCSS. Level2.5 and Level3 products generated by Unimap replace MadMap maps for SPG13.
  • Level2 products: the quality of the High-pass Filtered Level2 products (HPPPMAP) is improved by adding the scanamorphosMaskLongTermGlitches() and the scanamorphosBaselinePreprocessing() tasks, which are capable to detect jumps and to correct drift on the detector timelines.
  • Level2.5 products: Unimap Level2.5 contexts contain the same types of maps generated by Madmap (naïve, GLS and Post-GLS), nevertheless they are differently arranged within the context (see Product changes).
  • Level3 products: High-pass Filtered maps (HPPHPFMAP) are removed from Level3 products, having finally JScanam and Unimap maps. Level3 mosaics are created by using scanamorphosAlignImages(), a new task that links the flux by exploiting the overlapping regions between images.
  • Ipipe scripts: MadMap pipeline is still available as interactive script in the Pipelines menu. The interactive script scanmap_Unimap allows the users to run Unimap by tuning some parameters. It uses the new task runUnimap(), that calls the Unimap release available within the user’s computer.
>
>
  • Unimap: a new Generalized Least Square (GLS) mapmaker is introduced into the Standard Product Generator. Unimap is a MATLAB algorithm that is spawned from HCSS. Level2.5 and Level3 products generated by Unimap replace MADMap maps for SPG13.
  • Level2 products: the quality of the High-pass Filtered Level2 products (HPPPMAP) is improved by adding the scanamorphosMaskLongTermGlitches() and the scanamorphosBaselinePreprocessing() tasks, which can detect jumps and correct for drifts on the detector timelines.
  • Level2.5 products: Unimap Level2.5 contexts contain the same types of maps generated by MADmap (naïve, GLS and Post-GLS), but they are arranged differently within the context (see Product changes).
  • Level3 products: High-pass Filtered maps (HPPHPFMAP) have been removed from Level3 products, leaving JScanam and Unimap maps. Level3 mosaics are created by using scanamorphosAlignImages(), a new task that links the flux by exploiting overlapping regions between images.
  • Ipipe scripts: MADMap pipeline is still available as an interactive script in the Pipelines menu. The interactive script scanmap_Unimap allows users to run Unimap while tuning some parameters. It uses the new task runUnimap() that calls the Unimap release located on the user’s computer.
 
  • Harmonisation of map pixel sizes: at any Level, maps have the following pixel sizes:
    • 3.2 arcsec for the red channel, Prime and Parallel Modes;
    • 1.6 arcsec for the blue channel, Prime Mode;
    • 3.2 arcsec for the blue channel, Parallel Mode.
Changed:
<
<
  • Correction of timing shifts: frame pixels are not simultaneously read, but in SPG12 the frames are associated to a single readout time. That results in a time shift that can correspond up to 2 arcsec delay for fast scanning. This effect is corrected by setting the parameter correctReadoutTimes=True by default in every task that is affected by time offsets
>
>
  • Correction of timing shifts: frame pixels are not read simultaneously, so in SPG12 the frames are associated with a single readout time. That results in a time shift that can correspond to up to a 2 arcsec delay for fast scanning observations. This effect is corrected by setting the parameter correctReadoutTimes=True by default in every task that is affected by time offsets.
 

Spectroscopy

Changed:
<
<
  • Chop/nod SPG pipeline now uses the background normalisation method instead of the calibration blocks for the flux calibration.
  • Drizzle now also done for the unchopped line observations.
  • Interpolated cubes are computed for pointed observations with the new task specInterpolate.
  • Drizzled, projected or interpolated cubes are converted to an equidistant wavelength grid cube with complete standard WCS using the new task specRegridWavelength. These equidistant products are stored next to the non-regular wavelength grid products in the level-2 context.
  • Tables containing all the rebinned cubes per camera are produced with the new task pacsSpecCubeToTable.
>
>
  • The Chop/nod SPG pipeline now uses the background normalisation method instead of the calibration blocks for the flux calibration.
  • At Level 2/2.5 we now also produce drizzle cubes (HPS3DDR and HPS3DDB), and "interpolated" cubes (HSP3DIR and HPS3DIB) created with the new task specInterpolate.
  • Hence at Level 2/2.5 we now produce three cubes which have an equidistant spatial grid and which for mapping observations are also a mosaic: drizzled, projected, and interpolated cubes. Each observation will have two of these three cubes, chosen independently for each camera. Which two cubes you get depends on the AOT and the type of mapping raster: see the PACS Products Explained document (in HIPE) to learn more.
  • One of these three cubes is also provided with an equidistant wavelength grid, created with the new task specRegridWavelength.
  • Tables containing the data of the rebinned cubes are created produced with the new task pacsSpecCubeToTable (HPSTBRR and HPSTBRB). The spectral data of each spaxel of each raster position is included in a single table, and each wavelength range and camera is a separate table.
 
  • Flags are now also propagated by specProject and specInterpolate.
 

Analysis tools

Spectroscopy

Changed:
<
<
  • New long/short term Transient Correction algorithm and script.
  • Extra options to wavelengthGrid task to make regular grids with either the mean -, a fraction of the minimum -, or a specified bin size.
  • specSubtractOffPosition can now also combine off-positions using several algorithms and works with non-rebinned PacsCube products as well. The task specSubtractOffPositionNotRebinned is now superfluous and has been deprecated.
  • An image can be used as model for extended to point correction task.
  • Improvements to Pointing Offset Correction.
  • specDiffChop now also works on calibration blocks.
  • slicedSummary works now also on rebinned/projected cubes.
  • The footprint viewer is now a real viewer, which can be opened with the "Open with" menu on a product.
  • Interactive script improvements:
    • Made more user friendly, clear and with less verbose info.
    • Improvements of diagnostic plots.
    • New scripts for fitting mapping observations (from rebinned and projected cubes).
    • A new script to fit single pointings on interpolated cubes.
    • A new script to convolve 2 spectral images to the longer wavelength resolution.
    • Added a capability to overlay the footprint on an image.
>
>
  • New long/short term Transient Correction algorithms are included in a new pipeline script for the unchopped line and range scan modes.
  • Extra options have been added to the wavelengthGrid task to create a spectral grid with bins of the same size (equidistant grids). The bin size chosen can be the mean, minumum, a fraction of the minimum, or a specified value.
  • specSubtractOffPosition can now combine off-positions using different algorithms, and it works with non-rebinned PacsCube products as well. The task specSubtractOffPositionNotRebinned is now superfluous and has been deprecated.
  • An image can now be used as model in the extended to point correction task.
  • There are several improvements to the Pointing Offset Correction pipeline scripts (range and line scan, chopNod), including more useful plots.
  • specDiffChop now also works on a product containing only calibration block data.
  • slicedSummary works now also on all cubes.
  • The footprint viewer is now a real viewer, and can be opened with the "Open with" menu when clicking on an appropriate product.
  • General interactive pipeline script improvements:
    • Made more user-friendly, clear, and concise
    • Improvements to the diagnostic plots
    • Added a capability to overlay the IFU footprint on an image of the source in the pipeline helper task plotCubesRaDec
 
  • Fixed many problems with the mask editing functionality of the Spectrum Explorer on PACS frames and cubes.
Changed:
<
<
>
>
  • New/improved scripts have been added to the PACS Useful scripts menu:
    • Fitting mapping observations (from drizzled, interpolated, and projected cubes and secondly from rebinned cubes).
    • Fitting single pointings starting from interpolated cubes
    • A new script to convolve 2 spectral images to the longer wavelength resolution
    • A new script to re-create the standalone browse products
    • A script for point sources located in the central spaxel and a script for point sources located in another spaxel
    • A script summarising the post-processing tasks for extended sources
 

Product changes

Changed:
<
<
  • The meta keywords of all products have been improved. Especially every meta keyword has now a 8 character FITS equivalent, so that the META_XX keywords will not appear anymore in the FITS files for any of the important keywords.
  • The ra/dec of the level-1/2 products now reflect the actual average center of the particular cube/image containing these keywords. Only raNominal/decNominal contain the requested pointing coordinates.
  • Simplified the structure of the level 0-2 data in the observation context (fewer layers).
>
>
  • The meta keywords of all products have been improved. Most meta keywords now have an 8 character FITS equivalent, so fewer META_XX keywords will appear in the FITS files of the images and cubes.
  • The ra/dec of the Level 1/2 products now reflect the actual average centre of the particular cube/image containing these keywords. Only raNominal/decNominal contain the observer-requested pointing coordinates.
  • Simplified the structure of the Level 0--2 data in the observation context (fewer layers).
 
  • Moved the observation summary to the top level of the observation context and improved its contents.
Changed:
<
<
  • Improved browse images, replaced browse product by links to the level-2 products.
>
>
  • Improved the browse images, and replaced the browse products (the images and cubes) with links to the appropriate Level 2 products. Note that for spectroscopy the images are not necessarily created from the cubes that are the "browse products".
  • Removed redundant quality flags (for all SPG 13 products).
  • Improved the quality comments (will be visible for products of SPG 13 but only towards the end of 2015)
 

Photometry

Line: 444 to 449
 

Spectroscopy

Changed:
<
<
  • Drizzled cubes are also provided for unchopped observations.
  • We provide interpolated cubes for pointed observations (with 4.7" spaxel size) in the context under HPS3DIR and HPS3DIB.
>
>
  • Drizzled cubes are also provided in the context under HPS3DDR and HPS3DDB
  • We provide interpolated cubes with a 4.7" spaxel size in the context under HPS3DIR and HPS3DIB.
 
  • New level-2 products for easier handling outside of HIPE:
Changed:
<
<
    • Single table with all the rebinned cubes (PacsSpecTable product under HPSTBRR and HPSTBRB).
    • Many cubes are also regridded to an equidistant wavelength grid (store in the context under HPS3DEQ*), which are easier to view in viewers like DS9, since now also the wavelength is part of the standard WCS.
>
>
    • A single table (all spaxels, all raster position, but only one wavelength range per table) with all the spectral data from the rebinned cubes (PacsSpecTable product under HPSTBRR and HPSTBRB).
    • Cubes regridded on to an equidistant wavelength grid (stores in the context under HPS3DEQ*), which are easier to inspect in viewers such as DS9, since the wavelength grid is now part of the WCS.
  • We now provide Standalone Browse Products via the HSA, and these also can be found in the "browse product" part of any observation context. These products are: the new cubes we create with an equidistant spectral grid (HPS3DEQ[I|R|D][R|B], and the rebinned cubes tables (HPSTBR[R|B]).
 

Documentation

  • Improved the table of contents of the PACS User Reference Manual. Created a new content for the runUnimap() task.
Changed:
<
<
  • Detailed documentation of Unimap within the PACS Data Reduction.
  • Created PACS Products Explained document to describe and explain PACS products, from ObservationContext to the final maps and cubes
  • Cookbooks to guide users through certain common tasks done with PACS data in HIPE will be released, beginning in the spring of 2015
>
>
  • Detailed documentation of Unimap has been added to the PACS Data Reduction Guide for photometry.
  • Created PACS Products Explained document to describe and explain PACS products: from the observation context to the final maps and cubes
  • Improved the first two chapters of the PACS Data Reduction Guide for spectroscopy to help archive users decide what to do with their observations downloaded from the HSA: re-process or additionally process in HIPE or not? The description of the crucial pipeline tasks has also been improved.
  • Cookbooks to guide users through certain common tasks done with PACS data in HIPE will be released, beginning in the summer of 2015
 

SPIRE

Revision 622015-04-08 - AlvarGarcia

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 138 to 138
 

Plotting

  • PlotXY no longer has an explicit way of forcing a batch of plotting statements. This is now done automatically and the script developer must not do anything special to take advantage of it.
Deleted:
<
<

 TablePlotter

 OverPlotter

 

Astronomical utilities

  • Gyro-based attitude reconstruction, see pdf.
 

Numeric routines

Deleted:
<
<

 Data fitting

 Data interpolation

Random numbers

Statistics

 Trend analysis

 

 Mapping

  • Temporal files for mapping are now created in a subdirectory: each HIPE session running will use a different subdirectory.
Deleted:
<
<

Other functions

Source extraction

 

Images

 Display

Line: 224 to 199
 =HSAWritePool= now supports the removal of a list of tags.
Deleted:
<
<

Calibration sources

 

Products and datasets

Line: 239 to 210
  ParameterFactory added to easily create meta data parameters.
Deleted:
<
<

 Observation context

Pointing Product

 

Product Access Layer 

Line: 271 to 237
 
  • Specifying a ticket number for several observations is now possible.
Deleted:
<
<

Systematic product generation

 

Data input-output

  • getObservation (getObs task):
    • Now includes the possibility to retrieve observation using the SPG version that created the Observation (parameter spgVersion).
Line: 306 to 267
 

HIFI

Deleted:
<
<

HIFI Software Configuration and Properties

 

HIFI Calibration

The calibration tree released together with HIPE 13 contains the following updates:

Line: 444 to 403
  

Analysis tools

Deleted:
<
<

Photometry

 

Spectroscopy

  • New long/short term Transient Correction algorithm and script.
Line: 525 to 482
 
  • The PhaseCorrLim product was updated to correct the metadata - no effect on the data.
  • The TempDriftCorr product has been removed as it is not used by the pipeline.
Deleted:
<
<

Calibration Framework

Preprocessing

Engineering Conversion

Common Pipeline

 

Photometer Pipeline

  • The new 2-Pass Pipeline has replaced the standard pipelines for Large, Small and Parallel modes. The 2-Pass Pipeline runs iteratively on the data to produce optimally cleaned timelines. The first pass is run with minimal tasks (e.g. no FT tasks). Glitches are detected and flagged using 2nd level deglitching outlier detection. The full pipeline is then run using all the standard tasks.
  • New useful scripts to allow users to reconstruct the beam model and all calibration parameters (Photometer Calibration scripts) for both point, extended and semi-extended sources.
Line: 555 to 502
 

Interactive Analysis and Tools

Deleted:
<
<

General

 
 

Photometer

  • Significant improvements to the bolometer finder tool have been made. 
Line: 568 to 513
 
  • Small corrections to the useful scripts.

 

Deleted:
<
<

Documentation

 

Release Notes

Revision 612015-04-08 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 26 to 26
 
    • Level 2.5 for Maps contain HTPs of stitched spectra in addition to cubes created from the stitched spectra (HRS subbands are only stitched if they overlap in frequency).
    • Many Spectral Scan observations benefit from flag tables optimised manually by the ICC, resulting in spur-free deconvolved spectra.
    • All observations now contain a (Ta*) calibrated OFF position spectrum, in the same sky frequency scale as the ON position data.
Added:
>
>
    • HIFI is using a particular approach in the usage of the new gyro-based pointing reconstruction. All details and impact on the HIFI data are explained in this memo.
 
  • PACS
    • Photometer: New maps valid for point source and extended emission analysis: JScanam, UniMap (new!).
Line: 45 to 46
    * The new calibration tree contains updated beam models and efficiencies, as released initially as stand-alone products in September 2014. See for more details http://herschel.esac.esa.int/twiki/pub/Public/HifiCalibrationWeb/HifiBeamReleaseNote_Sep2014.pdf
  • In bands 6 and 7, the Electrical Standing Wave correction is now part of the standard pipeline, based on a data-base of optimised solutions stored in the calibration tree for each observation
  • For spectral scans, a significant fraction of the deconvolved level 2.5 products will benefit from flag tables optimised manually by the ICC, resulting in spur-free deconvolved spectra. This is not yet available for all spectral scan obsids though, but will be by HIPE 14. The list of optimised flags can be found in the calibration tree and further refined by the user using the flagTool if the User wishes so.
Added:
>
>
  • HIFI has a particular approach to the new gyro-based pointing reconstruction. The peculiarity of the approach and recommendation to the archive user are summarised in this memo.
 
Line: 316 to 318
 

HIFI Pipeline

hifiPipeline task

Level 0 Pipeline

Added:
>
>
  • The doPointing task is checking a quality figure in the pointing product in order to decide between the gyro-based or simple pointing reconstruction
 

WBS Pipeline

HRS Pipeline

Level 1 Pipeline

Line: 339 to 342
 
    • if it was not possible to apply the HEB electric standing wave correction
    • if the intended and computed pointing differ by more than three times the Absolute Pointing Error (APE)
    • if the APE check could not be performed
Added:
>
>
    • in case the gyro-based reconstructed pointing is considered to be under-performing, and the pointing product in used is reverted to the simple Pointing reconstruction from HIPE 12
 

HIFI Spectrum Toolbox

Line: 613 to 617
 
META FILEATTACHMENT attr="" autoattached="1" comment="pointing refinements" date="1420020588" name="Guide_gyro_recon_issue1.0.pdf" path="Guide_gyro_recon_issue1.0.pdf" size="857207" user="Main.JavierDiaz" version="1"
META FILEATTACHMENT attr="h" autoattached="1" comment="" date="1392627508" name="GoogleAnalytics.txt" path="GoogleAnalytics.txt" size="549" user="Main.AlvarGarcia" version="1"
Added:
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1428488821" name="HIFI_PointingAccuracyNote.pdf" path="HIFI_PointingAccuracyNote.pdf" size="381099" user="Main.DavidTeyssier" version="1"

Revision 602015-04-08 - LucaCalzoletti

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 418 to 418
 

Photometry

Changed:
<
<
  • Unimap: a new Generalized Least Square (GLS) mapmaker is introduced into the Standard Product Generator. Unimap is a MATLAB algorithm that is spawned from HCSS. Level2.5 and Level3 products generated by Unimap replace MadMap maps for PG13.
>
>
  • Unimap: a new Generalized Least Square (GLS) mapmaker is introduced into the Standard Product Generator. Unimap is a MATLAB algorithm that is spawned from HCSS. Level2.5 and Level3 products generated by Unimap replace MadMap maps for SPG13.
 
  • Level2 products: the quality of the High-pass Filtered Level2 products (HPPPMAP) is improved by adding the scanamorphosMaskLongTermGlitches() and the scanamorphosBaselinePreprocessing() tasks, which are capable to detect jumps and to correct drift on the detector timelines.
  • Level2.5 products: Unimap Level2.5 contexts contain the same types of maps generated by Madmap (naïve, GLS and Post-GLS), nevertheless they are differently arranged within the context (see Product changes).
  • Level3 products: High-pass Filtered maps (HPPHPFMAP) are removed from Level3 products, having finally JScanam and Unimap maps. Level3 mosaics are created by using scanamorphosAlignImages(), a new task that links the flux by exploiting the overlapping regions between images.
Changed:
<
<
  • Ipipe scripts: MadMap pipeline is still available as interactive script in the Pipelines menu. The interactive script scanmpa_Unimap allows the users to run Unimap by tuning some parameters. It uses the new task runUnimap(), that calls the Unimap release available within the user’s computer.
>
>
  • Ipipe scripts: MadMap pipeline is still available as interactive script in the Pipelines menu. The interactive script scanmap_Unimap allows the users to run Unimap by tuning some parameters. It uses the new task runUnimap(), that calls the Unimap release available within the user’s computer.
 
  • Harmonisation of map pixel sizes: at any Level, maps have the following pixel sizes:
    • 3.2 arcsec for the red channel, Prime and Parallel Modes;
    • 1.6 arcsec for the blue channel, Prime Mode;

Revision 592015-04-07 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 156 to 156
 

Statistics

 Trend analysis

 Mapping

Changed:
<
<
  • Mapping uses subdir per HIPE session for temp files
>
>
  • Temporal files for mapping are now created in a subdirectory: each HIPE session running will use a different subdirectory.
 

Other functions

Revision 582015-04-07 - AlvarGarcia

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->
Changed:
<
<

What's coming up in HIPE 13

>
>

What's new in HIPE 13

 
PDF Version Click here
Help Interested in what's new for other versions of HIPE? See this page for links to all the What's New documents from HIPE 4.0 onwards.
Line: 16 to 14
 

Highlights

Added:
>
>
  • Core
    • New auxiliary product OperationalDaySummaryProduct that helps filtering and handling data per Operational Day (OD) and new gyro-based pointing reconstruction algorithm.
    • HIPE now opens ALMA files and identifies ALMA cubes as spectral cubes, displayed in the Spectrum Explorer.
    • A rescue mode has been added to HIPE for cases where intensive scripts or memory thrashing issues make the GUI unresponsive.
 
Line: 299 to 298
 
  • When you send a table to Topcat it returns a table (before it returned a product)

Documentation

Added:
>
>
  • New chapter on converting units in the Data Analysis Guide.
  • Many more scripts in the manuals are tested periodically to ensure that they work correctly.
 

HIFI

Revision 572015-03-29 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 20 to 20
 
Changed:
<
<
    • Level 2.5 for Point mode observations contains spectra that are stitched (HRS subbands are only stitched if they overlap in frequency) and convert to simple spectrum.
>
>
    • Level 2.5 for Point mode observations contains spectra that are stitched (HRS spectra will contain NaNs between subbands, unless they overlap in frequency) and converted to simple spectrum. Frequency Switch observations are folded.
 
    • Level 2.5 for Maps contain HTPs of stitched spectra in addition to cubes created from the stitched spectra (HRS subbands are only stitched if they overlap in frequency).
    • Many Spectral Scan observations benefit from flag tables optimised manually by the ICC, resulting in spur-free deconvolved spectra.
    • All observations now contain a (Ta*) calibrated OFF position spectrum, in the same sky frequency scale as the ON position data.
Line: 327 to 327
 
  • The spur detection is now handed over to the direct application of an a priori spur table stored in the calibration tree. Obsids for which no entry exist will not have any spurs flags. This calibration tree is currently being populated and already covers hundreds of observations (mostly spectral scans) in HIPE 13. For those spectral scans, the users can expected improved quality of the deconvolved spectra at level 2.5

Level 2.5 Pipeline

  • Mapping mode: Level 2 HTPs are stitched and stored in Level 2.5. Cubes are then generated from the stitched HTP. HRS data are only stitched if the subbands are overlapping in frequency. Therefore you will find one cube for each WBS polarisation and backend and the number of cubes generated for the HRS depends on the spectral setup for the observation. In the case of frequency switch observations, spectra are folded after stitching and prior to gridding.
Changed:
<
<
  • Point mode: Level 2 HTPs are stitched. HRS subbands are stitched only if they overlap in frequency. If the data is frequency switched it is then folded. All spectra are converted to simpleSpectrum format.
>
>
  • Point mode: Level 2 HTPs are stitched. HRS spectra will contain NaNs between subbands unless the subbands overlap in frequency. If the data is frequency switched it is then folded. All spectra are converted to simpleSpectrum format.
 
  • A large number of statistics (primarily noise and mean) is performed both on the level 2 and level 2.5 products, as well as on an equivalent products converted to the Tmb scale with H and V merging (this latter is then removed from the observation context and only the statistics is stored). This computation is done by the =mkRms task and is controlled by the option computeRms. The calculation can be extremely time consuming for maps and spectral scans and is, by default, is disabled in HIPE (it is always done during bulk reprocessing at the HSA). You can turn it on in a HIPE session by setting computeRms=True

Revision 562015-03-23 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 326 to 326
 
  • A new task will create spectra for the OFF position data calibrated into the Ta* scale, on both USB and LSB frequency scales. For spectral scans, these can then be fed into deconvolution to create a deconvolved spectrum of the OFF position (option use_reference)
  • The spur detection is now handed over to the direct application of an a priori spur table stored in the calibration tree. Obsids for which no entry exist will not have any spurs flags. This calibration tree is currently being populated and already covers hundreds of observations (mostly spectral scans) in HIPE 13. For those spectral scans, the users can expected improved quality of the deconvolved spectra at level 2.5

Level 2.5 Pipeline

Changed:
<
<
  • Mapping mode: Level 2 HTPs are stitched and stored in Level 2.5. Cubes are then generated from the stitched HTP. HRS data are only stitched if the subbands are overlapping in frequency. Therefore you will find one cube for each WBS polarisation and backend and the number of cubes generated for the HRS depends on the spectral setup for the observation. In the case of frequency switch observations, spectra a folded after stitching and prior to gridding.
>
>
  • Mapping mode: Level 2 HTPs are stitched and stored in Level 2.5. Cubes are then generated from the stitched HTP. HRS data are only stitched if the subbands are overlapping in frequency. Therefore you will find one cube for each WBS polarisation and backend and the number of cubes generated for the HRS depends on the spectral setup for the observation. In the case of frequency switch observations, spectra are folded after stitching and prior to gridding.
 
  • Point mode: Level 2 HTPs are stitched. HRS subbands are stitched only if they overlap in frequency. If the data is frequency switched it is then folded. All spectra are converted to simpleSpectrum format.
  • A large number of statistics (primarily noise and mean) is performed both on the level 2 and level 2.5 products, as well as on an equivalent products converted to the Tmb scale with H and V merging (this latter is then removed from the observation context and only the statistics is stored). This computation is done by the =mkRms task and is controlled by the option computeRms. The calculation can be extremely time consuming for maps and spectral scans and is, by default, is disabled in HIPE (it is always done during bulk reprocessing at the HSA). You can turn it on in a HIPE session by setting computeRms=True
Line: 401 to 401
 

Documentation

  • Updates to the HIFI Data Reduction Guide:
    • A new chapter on The HIFI line identification tool (identifyLines task) is now available.
Added:
>
>
    • A new chapter describing tools to use the new HIFI beam model, Understanding and using HIFI beam information in your data is now available.
    • Updates to the chapter Running the HIFI pipeline for changes to the Interactive Level 2.5 pipeline, particularly the use of mkRms.
 
    • Updates to the chapter How to add and remove flags, section Setting flags interactively for many spectra (flagTool task).
    • Updates to the chapter Sideband deconvolution, section Basic strategy for running the deconvolution tool (doDeconvolution task).
    • Updates to the chapter Standing Wave Removal (fitHifiFringe task) - note also: new syntax for startPeriod, endPeriod, typicalPeriod, and subBase.
    • Updates to the chapter How to make a spectral cube to reflect changes in the level 2.5.
Added:
>
>
    • Updates to the chapter Unit conversions for changes to convertK2Jy
 
    • Updates to the chapter Exporting HIFI data to CLASS.
Changed:
<
<
>
>
    • Updates to all cookbooks to describe the OFF position spectra provided in the ObservationContext Calibration tree (ReferenceSpectra).
 

PACS

Revision 552015-03-10 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 405 to 405
 
    • Updates to the chapter Sideband deconvolution, section Basic strategy for running the deconvolution tool (doDeconvolution task).
    • Updates to the chapter Standing Wave Removal (fitHifiFringe task) - note also: new syntax for startPeriod, endPeriod, typicalPeriod, and subBase.
    • Updates to the chapter How to make a spectral cube to reflect changes in the level 2.5.
Added:
>
>
    • Updates to the chapter Exporting HIFI data to CLASS.
    • Updates to all cookbooks to introduce the new OFF position spectra now provided in the ObservationContext Calibration products (ReferenceSpectra).
 

PACS

Revision 542015-02-25 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 313 to 308
 

HIFI Calibration

The calibration tree released together with HIPE 13 contains the following updates:

Changed:
<
<
  • A list of optimised channel flags for spectral scans obsids, that will be honoured by the deconvolution task and result in spur-free level 2.5 data in this mode. This is not yet available for all spectral scan obsids though, but will be by HIPE 14.
>
>
  • A list of optimised channel flags for spectral scans obsids, that will be honoured by the deconvolution task and result in spur-free level 2.5 data in this mode. This is not yet available for all spectral scan observations but work will be completed by HIPE 14.
 
  • Solution tables for Electrical Standing Wave correction in the bands 6 and 7. This allows for a correction of this affect in the pipeline itself. Note that this is not a guarantee that the baseline artefact will be perfectly removed but it will provide the best correction available to data. It is then possible to further optimise the correction - see the dedicated sections in the DRG.
Changed:
<
<
  • Updated beam efficiencies and 2D beam models
>
>
 

HIFI Pipeline

hifiPipeline task

Line: 323 to 318
 

WBS Pipeline

HRS Pipeline

Level 1 Pipeline

Changed:
<
<
  • The Electrical Standing Wave correction is now automatically performed on the level 1 data, based on optimum correction solutions computed by the HIFI team and stored in the calibration tree.
>
>
  • The Electrical Standing Wave correction is now automatically performed on the level 1 data, by the doHebCorrection task. It is based on optimum correction solutions computed by the HIFI team and stored in the calibration tree.
  • A new task, mkUncertaintyTable creates a table from the uncertainty values present in the calibration tree and puts them into the calibration out part of the tree
 

Level 2 Pipeline

  • The sideband ratio correction is now taking into account the possible IF-dependency of the sideband ratio over the IF. It uses the IF-dependence information as given in the calibration tree. As of now, only the lower end of band 2a makes such an assumption. The presence of an IF-dependency will result in additional meta-data usbGain_n or lsbGain_n, which are the coefficients used to described the polynomial approximation of the gain dependence. Coefficients with index _0 simply correspond to the value of the constant, scalar, sideband ratio we used so far
  • A new task will automatically check for platforming in the WBS data and report any detection with a dedicated quality flag
  • A new task will create spectra for the OFF position data calibrated into the Ta* scale, on both USB and LSB frequency scales. For spectral scans, these can then be fed into deconvolution to create a deconvolved spectrum of the OFF position (option use_reference)
  • The spur detection is now handed over to the direct application of an a priori spur table stored in the calibration tree. Obsids for which no entry exist will not have any spurs flags. This calibration tree is currently being populated and already covers hundreds of observations (mostly spectral scans) in HIPE 13. For those spectral scans, the users can expected improved quality of the deconvolved spectra at level 2.5

Level 2.5 Pipeline

Changed:
<
<
  • Mapping mode: The cubes produced automatically by the pipeline (Level 2.5) (doGridding task), and consequently by hifiPipeline if you choose to run it again, will stitch the WBS subbands and create one final cube (with the index '_1'). You will not see individual cubes per subbands except for the HRS.
  • Point mode: Level 2 HTPs , stitched, are forming the Level 2.5 products for this mode.
  • A large number of statistics (primarily noise and mean) is performed both on the level 2 and level 2.5 products, as well as on an equivalent products converted to the Tmb scale with H and V merging (this latter is then removed from the observation context and only the statistics is stored). This computation is controlled by the option computeRms. It is only set to True when run for the bulk reprocessing, but ignored by default when the level 2.5 pipeline is called interactively (this computation can be extremely time consumming for maps and spectral scans)
>
>
  • Mapping mode: Level 2 HTPs are stitched and stored in Level 2.5. Cubes are then generated from the stitched HTP. HRS data are only stitched if the subbands are overlapping in frequency. Therefore you will find one cube for each WBS polarisation and backend and the number of cubes generated for the HRS depends on the spectral setup for the observation. In the case of frequency switch observations, spectra a folded after stitching and prior to gridding.
  • Point mode: Level 2 HTPs are stitched. HRS subbands are stitched only if they overlap in frequency. If the data is frequency switched it is then folded. All spectra are converted to simpleSpectrum format.
  • A large number of statistics (primarily noise and mean) is performed both on the level 2 and level 2.5 products, as well as on an equivalent products converted to the Tmb scale with H and V merging (this latter is then removed from the observation context and only the statistics is stored). This computation is done by the =mkRms task and is controlled by the option computeRms. The calculation can be extremely time consuming for maps and spectral scans and is, by default, is disabled in HIPE (it is always done during bulk reprocessing at the HSA). You can turn it on in a HIPE session by setting computeRms=True
 

HIFI Products

Changed:
<
<
>
>
  • New quality flags informing:
    • if it was not possible to apply the HEB electric standing wave correction
    • if the intended and computed pointing differ by more than three times the Absolute Pointing Error (APE)
    • if the APE check could not be performed
 

HIFI Spectrum Toolbox

Spectrum Explorer (HIFI specifics)

Added:
>
>
  • Introduction of HifiTool. This is an interface that allows you to conveniently display HIFI spectra with commonly used axes (velocity, USB, LSB, IF, Ta*, Jy). Click on the HIFI ICC icon to activate the tool.
 

Spectrum tasks (HIFI specifics)

  • The !PolarPair class has been removed. Instead the !polarPair task should be used. This should only affect scripts written prior to 2013.

Revision 532015-02-23 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 18 to 18
 

  • HIFI
Added:
>
>
 
    • Bands 6&7 corrected for electric standing waves.
Changed:
<
<
    • Level 2.5 for point spectra WBS subbands and convert to simple spectrum (the polarisations will still not be averaged).
    • Level 2.5 cubes for maps are now stitched
    • New HIFI beams.
>
>
    • Level 2.5 for Point mode observations contains spectra that are stitched (HRS subbands are only stitched if they overlap in frequency) and convert to simple spectrum.
    • Level 2.5 for Maps contain HTPs of stitched spectra in addition to cubes created from the stitched spectra (HRS subbands are only stitched if they overlap in frequency).
    • Many Spectral Scan observations benefit from flag tables optimised manually by the ICC, resulting in spur-free deconvolved spectra.
    • All observations now contain a (Ta*) calibrated OFF position spectrum, in the same sky frequency scale as the ON position data.
 
  • PACS
    • Photometer: New maps valid for point source and extended emission analysis: JScanam, UniMap (new!).
Line: 36 to 38
  HIFI
Changed:
<
<
  • A new level 2.5 products has been generated for point mode observations. It is a stitched version of the level 2 spectra for each available spectrometer, in the form of a Spectrum1d.
>
>
  • A new level 2.5 product has been generated for point mode observations. It is a stitched version of the level 2 spectra for each available spectrometer, in the form of a Spectrum1d.
 
  • All observations context will now contain a (Ta*) calibrated OFF position spectrum, in the same sky frequency scale as the ON position data.   * The new calibration tree contains updated beam models and efficiencies, as released initially as stand-alone products in September 2014. See for more details http://herschel.esac.esa.int/twiki/pub/Public/HifiCalibrationWeb/HifiBeamReleaseNote_Sep2014.pdf
  • In bands 6 and 7, the Electrical Standing Wave correction is now part of the standard pipeline, based on a data-base of optimised solutions stored in the calibration tree for each observation

Revision 522015-02-20 - JeroenDeJong

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 25 to 25
 
  • PACS
    • Photometer: New maps valid for point source and extended emission analysis: JScanam, UniMap (new!).
Changed:
<
<
    • Spectrometer: Cubes with evenly sampled spectral info.
>
>
    • Spectrometer: Cubes with evenly sampled spectral info. Improved mapping with additional 2D interpolation task.
    • Spectrometer and photometer: many improvements to the structure of the observation context and names and descriptions of products.
 
  • SPIRE
    • Photometer: Better deglitched maps (multi-pass pipeline).
Line: 426 to 427
 
  • Interpolated cubes are computed for pointed observations with the new task specInterpolate.
  • Drizzled, projected or interpolated cubes are converted to an equidistant wavelength grid cube with complete standard WCS using the new task specRegridWavelength. These equidistant products are stored next to the non-regular wavelength grid products in the level-2 context.
  • Tables containing all the rebinned cubes per camera are produced with the new task pacsSpecCubeToTable.
Added:
>
>
  • Flags are now also propagated by specProject and specInterpolate.
  

Analysis tools

Line: 435 to 437
 
  • New long/short term Transient Correction algorithm and script.
  • Extra options to wavelengthGrid task to make regular grids with either the mean -, a fraction of the minimum -, or a specified bin size.
Changed:
<
<
  • specSubtractOffPosition can now also combine off-positions using several algorithms.
>
>
  • specSubtractOffPosition can now also combine off-positions using several algorithms and works with non-rebinned PacsCube products as well. The task specSubtractOffPositionNotRebinned is now superfluous and has been deprecated.
 
  • An image can be used as model for extended to point correction task.
  • Improvements to Pointing Offset Correction.
  • specDiffChop now also works on calibration blocks.
Line: 448 to 450
 
    • A new script to fit single pointings on interpolated cubes.
    • A new script to convolve 2 spectral images to the longer wavelength resolution.
    • Added a capability to overlay the footprint on an image.
Added:
>
>
  • Fixed many problems with the mask editing functionality of the Spectrum Explorer on PACS frames and cubes.
 

Product changes

Revision 512015-02-06 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 373 to 373
 

Deconvolution

Changed:
<
<
  • By default, the option chosen for spur_rejection will be based on the availability of not of optimised flag tables for a given obsid (see also the calibration section). When no table exists, the option is enfored to REJECT_SCANS_WITH_SPURS. When tables exist, the option is enfored to DO_NOT_REJECT_SCANS_WITH_SPURS
>
>
  • By default, the option chosen for spur_rejection will be based on the availability of not of optimised flag tables for a given obsid (see also the calibration section). When no table exists, the option is enforced to REJECT_SCANS_WITH_SPURS. When tables exist, the option is enfored to DO_NOT_REJECT_SCANS_WITH_SPURS
 
  • A new option called enforce_spur_rejection is however made available to help control the existing spur_rejection option.
  • A new option use_reference indicates that the deconvolution should be made on the OFF position spectra computed with HIPE 13
Changed:
<
<
  • doDeconvolution has also been updated so it can take into account the IF-dependence of sideband ratio and use this a priori knowledge as preset to build the single sideband solution
>
>
  • doDeconvolution has also been updated so it can take into account the IF-dependence of sideband ratio and use this a priori knowledge as preset to build the single sideband solution
 

convertK2Jy

  • with the availability of the new beam efficiencies and beam model, the task has been upgraded in order to make best use of the detailed beam knowledge. A new option hifiBeam, set to True by default, will use the fully-detailled HIFI beam to compute the coupling factor involved in the temperature to flux conversion for sources of non-zero size.
  • for non-point-like sources, the task now offers disk or Gaussian source models (option shape)

Spectral Cube creation

Added:
>
>
  • It is now possible to use the mergeHtps task to combine HTPs prior to gridding. This can be used for combining any number of HTPs but is particularly useful for combining the H and V polarisation data of a given observation. Previously this could be done by following a script provided in the 'How to make a Spectral Cube' chapter of the HIFI Data Reduction Guide.

 

flagTool

  • Interactive (suitable for batch processing) and Expert modes now available.
Changed:
<
<
  • maskTables: a new variable to manage Linemasks and Rowmasks files.
>
>
  • maskTables: a new variable to manage Linemasks and Rowmasks files.
 

identifyLines

Changed:
<
<
  • IdentifyLines is a new task that allows you to identify lines in your spectrum and then run a comparison of known lines with a Linelist.
>
>
  • identifyLines is a new task that allows you to identify lines in your spectrum and then run a comparison of known lines with a Linelist.
 

HiClass export tool

Revision 502015-01-21 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 397 to 397
 
    • A new chapter on The HIFI line identification tool (identifyLines task) is now available.
    • Updates to the chapter How to add and remove flags, section Setting flags interactively for many spectra (flagTool task).
    • Updates to the chapter Sideband deconvolution, section Basic strategy for running the deconvolution tool (doDeconvolution task).
Changed:
<
<
    • Updates to the chapter Standing Wave Removal (fitHifiFringe task).
>
>
    • Updates to the chapter Standing Wave Removal (fitHifiFringe task) - note also: new syntax for startPeriod, endPeriod, typicalPeriod, and subBase.
 
    • Updates to the chapter How to make a spectral cube to reflect changes in the level 2.5.

PACS

Revision 492015-01-16 - AlvarGarcia

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 129 to 129
 

Plotting

Added:
>
>
  • PlotXY no longer has an explicit way of forcing a batch of plotting statements. This is now done automatically and the script developer must not do anything special to take advantage of it.
 

 TablePlotter

 OverPlotter

Revision 482015-01-16 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 418 to 418
 

Spectroscopy

Changed:
<
<
  • Chop/nod SPG pipeline now uses the background normalization method instead of the calibration blocks for the flux calibration.
>
>
  • Chop/nod SPG pipeline now uses the background normalisation method instead of the calibration blocks for the flux calibration.
 
  • Drizzle now also done for the unchopped line observations.
Changed:
<
<
  • Interpolated cubes are computed for pointed observations with the new task specIterpolate.
>
>
  • Interpolated cubes are computed for pointed observations with the new task specInterpolate.
 
  • Drizzled, projected or interpolated cubes are converted to an equidistant wavelength grid cube with complete standard WCS using the new task specRegridWavelength. These equidistant products are stored next to the non-regular wavelength grid products in the level-2 context.
  • Tables containing all the rebinned cubes per camera are produced with the new task pacsSpecCubeToTable.
 
Line: 477 to 478
 
  • Improved the table of contents of the PACS User Reference Manual. Created a new content for the runUnimap() task.
  • Detailed documentation of Unimap within the PACS Data Reduction.
Added:
>
>
  • Created PACS Products Explained document to describe and explain PACS products, from ObservationContext to the final maps and cubes
  • Cookbooks to guide users through certain common tasks done with PACS data in HIPE will be released, beginning in the spring of 2015
 

SPIRE

Revision 472015-01-08 - MiguelSanchez

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 233 to 233
 

 Observation context

Added:
>
>

Pointing Product

 

Revision 462014-12-31 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 141 to 141
 

Astronomical utilities

Added:
>
>
  • Gyro-based attitude reconstruction, see pdf.
  

Numeric routines

 Data fitting

Line: 591 to 592
  -- AlvarGarcia - 15 Jan 2014
Changed:
<
<
META FILEATTACHMENT attachment="GoogleAnalytics.txt" attr="h" comment="" date="1392627508" name="GoogleAnalytics.txt" path="GoogleAnalytics.txt" size="549" stream="GoogleAnalytics.txt" user="Main.AlvarGarcia" version="1"
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="pointing refinements" date="1420020588" name="Guide_gyro_recon_issue1.0.pdf" path="Guide_gyro_recon_issue1.0.pdf" size="857207" user="Main.JavierDiaz" version="1"
META FILEATTACHMENT attr="h" autoattached="1" comment="" date="1392627508" name="GoogleAnalytics.txt" path="GoogleAnalytics.txt" size="549" user="Main.AlvarGarcia" version="1"

Revision 452014-12-19 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 149 to 149
 

Statistics

 Trend analysis

 Mapping

Added:
>
>
  • Mapping uses subdir per HIPE session for temp files
 

Other functions

Line: 241 to 243
 

Tasks

  • Conversion tool to adapt multi-output tasks to new style in off-line User scripts.
  • Warning, important __list__ argument removed: if your task call still has it, remove it.
Added:
>
>
  • Preference "Generate task calls that returns full set of outputs" removed: tasks GUIs now always generate a call with the expanded list of outputs assigned to.
  • Task status messages appear in the status bar
 
  • convertUnits : It can convert units from double parameters, array datasets and columns, plus units from table datasets or products, by allowing to select the target dataset or column.
Line: 266 to 270
 
  • getObservation (getObs task):
    • Now includes the possibility to retrieve observation using the SPG version that created the Observation (parameter spgVersion).
    • The task path and poolLocation parameters have been consolidated. See up to date Task definition with examples.
Changed:
<
<
>
>
  • saveObservation:
    • Improved documentation
 
  • Reading of ALMA/CASA cubes directly into HIPE and the Spectrum Explorer: fitsReader and importSpectralCube
  • asciiTableReader:
    • Improved GUI
    • Improved error reporting for IPAC tables
Added:
>
>
  • fitsReader:
    • Identify velocity cubes (CTYPE3 = VELO-LSR) as spectralCubes
    • Identify ALMA/CASA cubes as spectralCubes
 
  • "Export Herschel data from HIPE" view:
    • automatically add the proper extension to output files
    • Improved GUI: changed default tab
Line: 280 to 288
 

Virtual Observatory

Changed:
<
<
The link to VOPlot in the External Tools sub-menu will be removed and substituted by Topcat.
>
>
  • The link to VOPlot in the External Tools sub-menu will be removed and substituted by Topcat.
  • Improved messages when failing to launch external applications
  • When you send a table to Topcat it returns a table (before it returned a product)
 

Documentation

Revision 442014-12-19 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 95 to 95
 

Editors & Viewers

  • Backups for new files are automatically created, and reopened in a subsequent session upon crashes like a power cut. If the session terminates gracefully, the backup files are deleted.
Added:
>
>
  • Editing metadata and dataset cells in the Product or Observation viewers keeps the result in memory, and the table is refreshed accordingly with the new value.
 
  • Animations when maximizing or restoring parts of the viewer will be shown (see Transitions).
  • The browse image of the Observation viewer can be opened with the Image viewer by means of Right click > Open in image viewer.
  • The ArrayDataset viewer shows the unit when specified.

Revision 432014-12-19 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 238 to 238
 

Tasks

Changed:
<
<
>
>
  • Conversion tool to adapt multi-output tasks to new style in off-line User scripts.
 
  • Warning, important __list__ argument removed: if your task call still has it, remove it.
  • convertUnits : It can convert units from double parameters, array datasets and columns, plus units from table datasets or products, by allowing to select the target dataset or column.
Line: 263 to 263
 

Data input-output

  • getObservation (getObs task):
Changed:
<
<
    • Now includes the possibility to retrieve observation using the SPG version that created the Observation.
>
>
    • Now includes the possibility to retrieve observation using the SPG version that created the Observation (parameter spgVersion).
 
    • The task path and poolLocation parameters have been consolidated. See up to date Task definition with examples.

  • Reading of ALMA/CASA cubes directly into HIPE and the Spectrum Explorer: fitsReader and importSpectralCube
Added:
>
>
  • asciiTableReader:
    • Improved GUI
    • Improved error reporting for IPAC tables
  • "Export Herschel data from HIPE" view:
    • automatically add the proper extension to output files
    • Improved GUI: changed default tab
 

Revision 422014-12-18 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 20 to 20
 
  • HIFI
    • Bands 6&7 corrected for electric standing waves.
    • Level 2.5 for point spectra WBS subbands and convert to simple spectrum (the polarisations will still not be averaged).
Added:
>
>
    • Level 2.5 cubes for maps are now stitched
 
    • New HIFI beams.

  • PACS
Line: 300 to 301
 

WBS Pipeline

HRS Pipeline

Level 1 Pipeline

Added:
>
>
  • The Electrical Standing Wave correction is now automatically performed on the level 1 data, based on optimum correction solutions computed by the HIFI team and stored in the calibration tree.
 

Level 2 Pipeline

  • The sideband ratio correction is now taking into account the possible IF-dependency of the sideband ratio over the IF. It uses the IF-dependence information as given in the calibration tree. As of now, only the lower end of band 2a makes such an assumption. The presence of an IF-dependency will result in additional meta-data usbGain_n or lsbGain_n, which are the coefficients used to described the polynomial approximation of the gain dependence. Coefficients with index _0 simply correspond to the value of the constant, scalar, sideband ratio we used so far
  • A new task will automatically check for platforming in the WBS data and report any detection with a dedicated quality flag
  • A new task will create spectra for the OFF position data calibrated into the Ta* scale, on both USB and LSB frequency scales. For spectral scans, these can then be fed into deconvolution to create a deconvolved spectrum of the OFF position (option use_reference)
Added:
>
>
  • The spur detection is now handed over to the direct application of an a priori spur table stored in the calibration tree. Obsids for which no entry exist will not have any spurs flags. This calibration tree is currently being populated and already covers hundreds of observations (mostly spectral scans) in HIPE 13. For those spectral scans, the users can expected improved quality of the deconvolved spectra at level 2.5
 

Level 2.5 Pipeline

  • Mapping mode: The cubes produced automatically by the pipeline (Level 2.5) (doGridding task), and consequently by hifiPipeline if you choose to run it again, will stitch the WBS subbands and create one final cube (with the index '_1'). You will not see individual cubes per subbands except for the HRS.
  • Point mode: Level 2 HTPs , stitched, are forming the Level 2.5 products for this mode.

Revision 412014-12-17 - AlvarGarcia

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 167 to 167
 
  • Renamed setCutLevels(double) to setCutLevelsPercentage(double)

  • The images can not only be flipped around the y-axis, but also around the x-, xy- and yx-axes. It is also possible to combine these flip-options : for example : y- and xy.
Changed:
<
<
  • The annotation in Display are totally reworked. All old annotation methods are deprectated, the new one end with Annotation. For example : addEllipse becomes addEllipseAnnotation.
>
>
  • The annotation in Display are totally reworked. All old annotation methods are deprecated, the new one end with Annotation. For example : addEllipse becomes addEllipseAnnotation.
 
Changed:
<
<
A lot of fixes in Display :
  • Display now displays images with some extremely high or low values correctly.
  • It is now possible to extract the current layer from a SimpleImage or from a SimpleCube with only one layer.
  • Display can now work with more than one extension.
>
>
A lot of fixes in Display:
  • Display now displays images with some extremely high or low values correctly.
  • It is now possible to extract the current layer from a SimpleImage or from a SimpleCube with only one layer.
  • Display can now work with more than one extension.
 
  • The zoom in / out was by a factor of 4 instead of 2.
  • The Wcs viewer only shows the important wcs keywords now.
Line: 182 to 182
 
  • Mosaics close to the north or south pole missed a part of the sky.
  • Aperture photometry tasks have clear description on how the background subtracted flux is calculated.
  • Bugfixes in aperture photometry tasks.
Changed:
<
<
  • You can choose to add an input parameter to annularSkyAperturePhotometry for choosing the step size of curve-of-growth output
>
>
  • You can choose to add an input parameter to annularSkyAperturePhotometry for choosing the step size of curve-of-growth output.
 

Spectra

 Display

Line: 191 to 191
 

 Analysis

Added:
>
>
  • multiFit task now accepts limits for the model parameters.
 

Data cubes

 Display

Changed:
<
<
  • Disable the selection tools when some other data cube then the flux is shown.
  • Fixed bug in which the upper and right rows and column were not included in the selection for the cropCube task.
>
>
  • Disable the selection tools when some other data cube than the flux is shown.
  • Fixed bug in which the upper and right rows and columns were not included in the selection for the cropCube task.
 

 Analysis

Changed:
<
<
  • Moment algorithm in computeVelocityMap improved with 'line strength' weighting. Especially dispersion more accurate.
  • Show proper model cube when absorption line fitting is done in computeVelocityMap.
  • More metadata in extractRegionSpectrum.
>
>
  • Moment algorithm in computeVelocityMap improved with 'line strength' weighting. Especially dispersion is more accurate.
  • Show proper model cube when absorption line fitting is done in computeVelocityMap.
  • More metadata in extractRegionSpectrum.
 

Herschel Science Archive

Line: 219 to 220
  A new Auxiliary Operational Day Summary Product which can be retrieved using the tag LATEST_OPERATIONAL_DAY_SUMMARY_PRODUCT. This product contains one dataset with the operational day number, start and end times. Useful methods exist for querying the data as documented here. A new Auxiliary Operational Day Summary Product which can be retrieved using the tag LATEST_OPERATIONAL_DAY_SUMMARY_PRODUCT. This product contains one dataset with the operational day number, start and end times. Useful methods exist for querying the data as documented here:
Changed:
<
<
http://herschel.esac.esa.int/hcss-doc-13.0/index.jsp#hcss_drm:herschel.ia.obs.auxiliary.od.OperationalDaySummaryProduct
>
>
OperationalDaySummaryProduct
  A new utility method in the Product class for querying a products duration.
Line: 289 to 290
 

HIFI Calibration

The calibration tree released together with HIPE 13 contains the following updates:

Changed:
<
<
  • it will contain a list of optimised channel flags for spectral scans obsids, that will be honoured by the deconvolution task and result in spur-free level 2.5 data in this mode. This is not yet available for all spectral scan obsids though, but will be by HIPE 14.
  • it will contain solution tables for Electrical Standing Wave correction in the bands 6 and 7. This allows for a correction of this affect in the pipeline itself. Note that this is not a guarantee that the baseline artefact will be perfectly removed but it will provide the best correction available to data. It is then possible to further optimise the correction - see the dedicated sections in the DRG.
  • it will contain the updated beam efficiencies and 2D beam models
>
>
  • A list of optimised channel flags for spectral scans obsids, that will be honoured by the deconvolution task and result in spur-free level 2.5 data in this mode. This is not yet available for all spectral scan obsids though, but will be by HIPE 14.
  • Solution tables for Electrical Standing Wave correction in the bands 6 and 7. This allows for a correction of this affect in the pipeline itself. Note that this is not a guarantee that the baseline artefact will be perfectly removed but it will provide the best correction available to data. It is then possible to further optimise the correction - see the dedicated sections in the DRG.
  • Updated beam efficiencies and 2D beam models
 

HIFI Pipeline

hifiPipeline task

Line: 300 to 301
 

HRS Pipeline

Level 1 Pipeline

Level 2 Pipeline

Changed:
<
<
  • the sideband ratio correction is now taking into account the possible IF-dependency of the sideband ratio over the IF. It uses the IF-dependence information as given in the calibration tree. As of now, only the lower end of band 2a makes such an assumption. The presence of an IF-dependency will result in additional meta-data usbGain_n or lsbGain_n, which are the coefficients used to described the polynomial approximation of the gain dependence. Coefficients with index _0 simply correspond to the value of the constant, scalar, sideband ratio we used so far
  • a new task will automatically check for platforming in the WBS data and report any detection with a dedicated quality flag
  • a new task will create spectra for the OFF position data calibrated into the Ta* scale, on both USB and LSB frequency scales. For spectral scans, these can then be fed into deconvolution to create a deconvolved spectrum of the OFF position (option use_reference)
>
>
  • The sideband ratio correction is now taking into account the possible IF-dependency of the sideband ratio over the IF. It uses the IF-dependence information as given in the calibration tree. As of now, only the lower end of band 2a makes such an assumption. The presence of an IF-dependency will result in additional meta-data usbGain_n or lsbGain_n, which are the coefficients used to described the polynomial approximation of the gain dependence. Coefficients with index _0 simply correspond to the value of the constant, scalar, sideband ratio we used so far
  • A new task will automatically check for platforming in the WBS data and report any detection with a dedicated quality flag
  • A new task will create spectra for the OFF position data calibrated into the Ta* scale, on both USB and LSB frequency scales. For spectral scans, these can then be fed into deconvolution to create a deconvolved spectrum of the OFF position (option use_reference)
 

Level 2.5 Pipeline

  • Mapping mode: The cubes produced automatically by the pipeline (Level 2.5) (doGridding task), and consequently by hifiPipeline if you choose to run it again, will stitch the WBS subbands and create one final cube (with the index '_1'). You will not see individual cubes per subbands except for the HRS.
  • Point mode: Level 2 HTPs , stitched, are forming the Level 2.5 products for this mode.
Line: 320 to 321
 

Spectrum Explorer (HIFI specifics)

Spectrum tasks (HIFI specifics)

Changed:
<
<
  • The PolarPair class has been removed. Instead the polarPair task should be used. This should only affect scripts written prior to 2013.
>
>
  • The !PolarPair class has been removed. Instead the !polarPair task should be used. This should only affect scripts written prior to 2013.
 
Line: 346 to 347
 

Electrical Standing Wave correction in HEB bands

Changed:
<
<
  • the task doHebCorrection is now part of the default pipeline, with several new options compared to its predecessor hebCorrection.
>
>
  • The task doHebCorrection is now part of the default pipeline, with several new options compared to its predecessor hebCorrection.
 

Deconvolution

Changed:
<
<
  • by default, the option chosen for spur_rejection will be based on the availability of not of optimised flag tables for a given obsid (see also the calibration section). When no table exists, the option is enfored to REJECT_SCANS_WITH_SPURS. When tables exist, the option is enfored to DO_NOT_REJECT_SCANS_WITH_SPURS
>
>
  • By default, the option chosen for spur_rejection will be based on the availability of not of optimised flag tables for a given obsid (see also the calibration section). When no table exists, the option is enfored to REJECT_SCANS_WITH_SPURS. When tables exist, the option is enfored to DO_NOT_REJECT_SCANS_WITH_SPURS
 
  • A new option called enforce_spur_rejection is however made available to help control the existing spur_rejection option.
  • A new option use_reference indicates that the deconvolution should be made on the OFF position spectra computed with HIPE 13
  • doDeconvolution has also been updated so it can take into account the IF-dependence of sideband ratio and use this a priori knowledge as preset to build the single sideband solution
Changed:
<
<

convertK2Jy

>
>

convertK2Jy

 
  • with the availability of the new beam efficiencies and beam model, the task has been upgraded in order to make best use of the detailed beam knowledge. A new option hifiBeam, set to True by default, will use the fully-detailled HIFI beam to compute the coupling factor involved in the temperature to flux conversion for sources of non-zero size.
  • for non-point-like sources, the task now offers disk or Gaussian source models (option shape)

Spectral Cube creation

Changed:
<
<

flagTool

>
>

flagTool

 
  • Interactive (suitable for batch processing) and Expert modes now available.
  • maskTables: a new variable to manage Linemasks and Rowmasks files.
Changed:
<
<

identifyLines

  • IdentifyLines is a new task that allows you to identify lines in your spectrum and then run a comparison of known lines with a Linelist.
>
>

identifyLines

  • IdentifyLines is a new task that allows you to identify lines in your spectrum and then run a comparison of known lines with a Linelist.
 

HiClass export tool

Line: 410 to 411
 
  • New long/short term Transient Correction algorithm and script.
  • Extra options to wavelengthGrid task to make regular grids with either the mean -, a fraction of the minimum -, or a specified bin size.
Changed:
<
<
  • specSubtractOffPosition can now also combine off-positions using several algorithms.
>
>
  • specSubtractOffPosition can now also combine off-positions using several algorithms.
 
  • An image can be used as model for extended to point correction task.
  • Improvements to Pointing Offset Correction.
Changed:
<
<
  • specDiffChop now also works on calibration blocks.
  • slicedSummary works now also on rebinned/projected cubes.
>
>
  • specDiffChop now also works on calibration blocks.
  • slicedSummary works now also on rebinned/projected cubes.
 
  • The footprint viewer is now a real viewer, which can be opened with the "Open with" menu on a product.
  • Interactive script improvements:
    • Made more user friendly, clear and with less verbose info.
Line: 494 to 495
 

Photometer Pipeline

  • The new 2-Pass Pipeline has replaced the standard pipelines for Large, Small and Parallel modes. The 2-Pass Pipeline runs iteratively on the data to produce optimally cleaned timelines. The first pass is run with minimal tasks (e.g. no FT tasks). Glitches are detected and flagged using 2nd level deglitching outlier detection. The full pipeline is then run using all the standard tasks.
  • New useful scripts to allow users to reconstruct the beam model and all calibration parameters (Photometer Calibration scripts) for both point, extended and semi-extended sources.
Changed:
<
<
  • New SPG map merging script utilising the 2-pass pipeline to produce Level 2.5 products. All parallel mode observations are reprocessed together as pairs at the Level 2.5 stage. After the level 2.5 stage all the Level 0.5, Level 1,Level 2, Level 2.5 data are updated with the impoved quality data.
>
>
  • New SPG map merging script utilising the 2-pass pipeline to produce Level 2.5 products. All parallel mode observations are reprocessed together as pairs at the Level 2.5 stage. After the level 2.5 stage all the Level 0.5, Level 1,Level 2, Level 2.5 data are updated with the improved quality data.
 
  • Optimized parameters for Timeline Fitter in photometry script. Background annulus changed for faster run time.
Line: 518 to 519
 

Spectrometer

Changed:
<
<
  • Minor updates, bug fixes, and improved documentation for the Semi-extended correction task
  • Updates to the task to take synthetic photometry of FTS spectra for comparison with the SPIRE photometer
  • New GUI interface for the script that determines the noise level in a spectrum
  • Small corrections to the useful scripts
>
>
  • Minor updates, bug fixes, and improved documentation for the Semi-extended correction task.
  • Updates to the task to take synthetic photometry of FTS spectra for comparison with the SPIRE photometer.
  • New GUI interface for the script that determines the noise level in a spectrum.
  • Small corrections to the useful scripts.
   

Revision 402014-12-17 - LucaCalzoletti

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 394 to 394
 
    • 3.2 arcsec for the blue channel, Parallel Mode.
  • Correction of timing shifts: frame pixels are not simultaneously read, but in SPG12 the frames are associated to a single readout time. That results in a time shift that can correspond up to 2 arcsec delay for fast scanning. This effect is corrected by setting the parameter correctReadoutTimes=True by default in every task that is affected by time offsets
Deleted:
<
<
What about gyro attitude?? What about qflags?
 

Spectroscopy

  • Chop/nod SPG pipeline now uses the background normalization method instead of the calibration blocks for the flux calibration.

Revision 392014-12-16 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 240 to 240
 
  • Warning, important __list__ argument removed: if your task call still has it, remove it.
  • convertUnits : It can convert units from double parameters, array datasets and columns, plus units from table datasets or products, by allowing to select the target dataset or column.
Deleted:
<
<
The GetObs Task:
  • Now includes the possibility to retrieve observation using the SPG version that created the Observation.
  • The task path and poolLocation parameters have been consolidated. See up to date Task definition with examples.
 
Deleted:
<
<
The FitsReaderTask and ImportSpectralCubeTask now support reading and loading of ALMA/CASA cubes directly into HIPE and the Spectrum Explorer.
 

Quality control

Line: 264 to 260
 

Data input-output

Added:
>
>
  • getObservation (getObs task):
    • Now includes the possibility to retrieve observation using the SPG version that created the Observation.
    • The task path and poolLocation parameters have been consolidated. See up to date Task definition with examples.

  • Reading of ALMA/CASA cubes directly into HIPE and the Spectrum Explorer: fitsReader and importSpectralCube
 

Revision 382014-12-15 - JeroenDeJong

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 193 to 193
 

Data cubes

Added:
>
>
 

 Display

Deleted:
<
<

 Analysis

 
Added:
>
>
  • Disable the selection tools when some other data cube then the flux is shown.
  • Fixed bug in which the upper and right rows and column were not included in the selection for the cropCube task.
 
Added:
>
>

 Analysis

 
Added:
>
>
  • Moment algorithm in computeVelocityMap improved with 'line strength' weighting. Especially dispersion more accurate.
  • Show proper model cube when absorption line fitting is done in computeVelocityMap.
  • More metadata in extractRegionSpectrum.
 

Herschel Science Archive

Revision 372014-12-15 - JeroenDeJong

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 12 to 12
 See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.
Changed:
<
<
>
>
 

Highlights

Line: 400 to 400
  

Analysis tools

Deleted:
<
<

Analysis tools

 

Photometry

Deleted:
<
<

Spectroscopy

 

Spectroscopy

Added:
>
>
  • New long/short term Transient Correction algorithm and script.
  • Extra options to wavelengthGrid task to make regular grids with either the mean -, a fraction of the minimum -, or a specified bin size.
  • specSubtractOffPosition can now also combine off-positions using several algorithms.
  • An image can be used as model for extended to point correction task.
  • Improvements to Pointing Offset Correction.
  • specDiffChop now also works on calibration blocks.
  • slicedSummary works now also on rebinned/projected cubes.
  • The footprint viewer is now a real viewer, which can be opened with the "Open with" menu on a product.
  • Interactive script improvements:
    • Made more user friendly, clear and with less verbose info.
    • Improvements of diagnostic plots.
    • New scripts for fitting mapping observations (from rebinned and projected cubes).
    • A new script to fit single pointings on interpolated cubes.
    • A new script to convolve 2 spectral images to the longer wavelength resolution.
    • Added a capability to overlay the footprint on an image.
 

Product changes

Added:
>
>
  • The meta keywords of all products have been improved. Especially every meta keyword has now a 8 character FITS equivalent, so that the META_XX keywords will not appear anymore in the FITS files for any of the important keywords.
  • The ra/dec of the level-1/2 products now reflect the actual average center of the particular cube/image containing these keywords. Only raNominal/decNominal contain the requested pointing coordinates.
  • Simplified the structure of the level 0-2 data in the observation context (fewer layers).
  • Moved the observation summary to the top level of the observation context and improved its contents.
  • Improved browse images, replaced browse product by links to the level-2 products.
 

Photometry

Revision 362014-12-15 - JeroenDeJong

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 201 to 201
 

Herschel Science Archive

Changed:
<
<
HSAWritePool now supports the removal of a list of tags.
>
>
=HSAWritePool= now supports the removal of a list of tags.
 

Calibration sources

Line: 211 to 211
 

Products and datasets

Changed:
<
<
A new Auxiliary Operational Day Summary Product which can be retrieved using the tag LATEST_OPERATIONAL_DAY_SUMMARY_PRODUCT. This product contains one dataset with the operational day number, start and end times. Useful methods exist for querying the data as documented here.
>
>
A new Auxiliary Operational Day Summary Product which can be retrieved using the tag LATEST_OPERATIONAL_DAY_SUMMARY_PRODUCT. This product contains one dataset with the operational day number, start and end times. Useful methods exist for querying the data as documented here. A new Auxiliary Operational Day Summary Product which can be retrieved using the tag LATEST_OPERATIONAL_DAY_SUMMARY_PRODUCT. This product contains one dataset with the operational day number, start and end times. Useful methods exist for querying the data as documented here: http://herschel.esac.esa.int/hcss-doc-13.0/index.jsp#hcss_drm:herschel.ia.obs.auxiliary.od.OperationalDaySummaryProduct
  A new utility method in the Product class for querying a products duration.
Line: 387 to 390
 What about gyro attitude?? What about qflags?
Deleted:
<
<
 

Spectroscopy

Added:
>
>
  • Chop/nod SPG pipeline now uses the background normalization method instead of the calibration blocks for the flux calibration.
  • Drizzle now also done for the unchopped line observations.
  • Interpolated cubes are computed for pointed observations with the new task specIterpolate.
  • Drizzled, projected or interpolated cubes are converted to an equidistant wavelength grid cube with complete standard WCS using the new task specRegridWavelength. These equidistant products are stored next to the non-regular wavelength grid products in the level-2 context.
  • Tables containing all the rebinned cubes per camera are produced with the new task pacsSpecCubeToTable.
  

Analysis tools

Changed:
<
<
>
>

Analysis tools

 

Photometry

Spectroscopy

Added:
>
>

Spectroscopy

 

Product changes

Line: 417 to 426
 

Spectroscopy

Added:
>
>
  • Drizzled cubes are also provided for unchopped observations.
  • We provide interpolated cubes for pointed observations (with 4.7" spaxel size) in the context under HPS3DIR and HPS3DIB.
  • New level-2 products for easier handling outside of HIPE:
    • Single table with all the rebinned cubes (PacsSpecTable product under HPSTBRR and HPSTBRB).
    • Many cubes are also regridded to an equidistant wavelength grid (store in the context under HPS3DEQ*), which are easier to view in viewers like DS9, since now also the wavelength is part of the standard WCS.
 

Documentation

Revision 352014-12-15 - JonathanCook

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 201 to 201
 

Herschel Science Archive

Changed:
<
<
>
>
HSAWritePool now supports the removal of a list of tags.
 

Calibration sources

Line: 211 to 211
 

Products and datasets

Changed:
<
<
A new Auxiliary Operational Day Summary Product which can be retrieved using the tag LATEST_OPERATIONAL_DAY_SUMMARY_PRODUCT. This product contains one dataset with the operational day number, start and end times. Useful methods exist for querying the data as documented here: http://herschel.esac.esa.int/hcss-doc-13.0/index.jsp#hcss_drm:herschel.ia.obs.auxiliary.od.OperationalDaySummaryProduct
>
>
A new Auxiliary Operational Day Summary Product which can be retrieved using the tag LATEST_OPERATIONAL_DAY_SUMMARY_PRODUCT. This product contains one dataset with the operational day number, start and end times. Useful methods exist for querying the data as documented here.
  A new utility method in the Product class for querying a products duration.

Revision 342014-12-15 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 214 to 214
 A new Auxiliary Operational Day Summary Product which can be retrieved using the tag LATEST_OPERATIONAL_DAY_SUMMARY_PRODUCT. This product contains one dataset with the operational day number, start and end times. Useful methods exist for querying the data as documented here: http://herschel.esac.esa.int/hcss-doc-13.0/index.jsp#hcss_drm:herschel.ia.obs.auxiliary.od.OperationalDaySummaryProduct
Changed:
<
<
A new utility method in the Product class for querying a products duration.
>
>
A new utility method in the Product class for querying a products duration.

ParameterFactory added to easily create meta data parameters.

 

 Observation context

Line: 231 to 233
 
  • Warning, important __list__ argument removed: if your task call still has it, remove it.
  • convertUnits : It can convert units from double parameters, array datasets and columns, plus units from table datasets or products, by allowing to select the target dataset or column.
Changed:
<
<
The GetObs Task now includes the possibility to retrieve observation using the spg version that created the Observation. See up to date Task definition with examples: (HCSS-19063) The GetObs Task path and poolLocation parameters have been consolidated. See up to date Task definition with examples: (HCSS-19241)

The FitsReaderTask and ImportSpectralCubeTask now support reading and loading of ALMA/CASA cubes directly into HIPE and the Spectrum Explorer - HCSS-18110

>
>
The GetObs Task:
  • Now includes the possibility to retrieve observation using the SPG version that created the Observation.
  • The task path and poolLocation parameters have been consolidated. See up to date Task definition with examples.
 
Added:
>
>
The FitsReaderTask and ImportSpectralCubeTask now support reading and loading of ALMA/CASA cubes directly into HIPE and the Spectrum Explorer.
 

Quality control

Changed:
<
<
>
>
  • It is possible to create Quality Control summaries for several instruments in one go.
  • The QCR Browser is more interactive, as it is no longer blocked when performing an operation. This way, tasks can be performed in parallel on different windows.
  • Panels in the QCR Display are ordered in a more sensible way.
  • The OD is shown in the QCR Display when available.
  • The operator can remove comments from several observations in a single operation.
  • The report showing the result of an ingestion or removal operation can be saved to a text file.
  • Specifying a ticket number for several observations is now possible.
 

Systematic product generation

Revision 332014-12-15 - JonathanCook

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 210 to 210
 

Products and datasets

Added:
>
>
A new Auxiliary Operational Day Summary Product which can be retrieved using the tag LATEST_OPERATIONAL_DAY_SUMMARY_PRODUCT. This product contains one dataset with the operational day number, start and end times. Useful methods exist for querying the data as documented here: http://herschel.esac.esa.int/hcss-doc-13.0/index.jsp#hcss_drm:herschel.ia.obs.auxiliary.od.OperationalDaySummaryProduct

A new utility method in the Product class for querying a products duration.

 

 Observation context

Revision 322014-12-14 - LucaCalzoletti

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 362 to 362
 

Pipeline

Photometry

Added:
>
>
  • Unimap: a new Generalized Least Square (GLS) mapmaker is introduced into the Standard Product Generator. Unimap is a MATLAB algorithm that is spawned from HCSS. Level2.5 and Level3 products generated by Unimap replace MadMap maps for PG13.
  • Level2 products: the quality of the High-pass Filtered Level2 products (HPPPMAP) is improved by adding the scanamorphosMaskLongTermGlitches() and the scanamorphosBaselinePreprocessing() tasks, which are capable to detect jumps and to correct drift on the detector timelines.
  • Level2.5 products: Unimap Level2.5 contexts contain the same types of maps generated by Madmap (naïve, GLS and Post-GLS), nevertheless they are differently arranged within the context (see Product changes).
  • Level3 products: High-pass Filtered maps (HPPHPFMAP) are removed from Level3 products, having finally JScanam and Unimap maps. Level3 mosaics are created by using scanamorphosAlignImages(), a new task that links the flux by exploiting the overlapping regions between images.
  • Ipipe scripts: MadMap pipeline is still available as interactive script in the Pipelines menu. The interactive script scanmpa_Unimap allows the users to run Unimap by tuning some parameters. It uses the new task runUnimap(), that calls the Unimap release available within the user’s computer.
  • Harmonisation of map pixel sizes: at any Level, maps have the following pixel sizes:
    • 3.2 arcsec for the red channel, Prime and Parallel Modes;
    • 1.6 arcsec for the blue channel, Prime Mode;
    • 3.2 arcsec for the blue channel, Parallel Mode.
  • Correction of timing shifts: frame pixels are not simultaneously read, but in SPG12 the frames are associated to a single readout time. That results in a time shift that can correspond up to 2 arcsec delay for fast scanning. This effect is corrected by setting the parameter correctReadoutTimes=True by default in every task that is affected by time offsets

What about gyro attitude?? What about qflags?

 

Spectroscopy

 

Analysis tools

Line: 380 to 395
 

Photometry

Changed:
<
<
>
>
  • The naming convention for Unimap Level2.5/3 is HPPUNIMAP[B][R], for the blue and red channel, respectively.
  • A single context is released for Unimap Level2.5. The Unimap context contains the following products:
    • image is the Weighted-GLS map. Generally, it is the most reliable product;
    • pgls is the Post-GLS map;
    • gls is the GLS map;
    • naive is the Naïve map;
    • error is that is the standard deviation map. It can be properly assumed as the error map only if associated to the Naïve map.
 

Spectroscopy

Line: 389 to 410
 

Documentation

Added:
>
>
  • Improved the table of contents of the PACS User Reference Manual. Created a new content for the runUnimap() task.
  • Detailed documentation of Unimap within the PACS Data Reduction.
 

SPIRE

Revision 312014-12-13 - JorgoBakker

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 254 to 254
 The link to VOPlot in the External Tools sub-menu will be removed and substituted by Topcat.
Deleted:
<
<

Installer

 

Documentation

HIFI

Revision 302014-12-11 - BrunoMerin

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 14 to 14
 
Deleted:
<
<

Planned work

HIPE

   * TBD

HIFI

   * TBD

PACS

 * TBD

SPIRE

   * TBD

 

Highlights

Deleted:
<
<
HIPE
 
Changed:
<
<
   * TBD
>
>
  • HIFI
    • Bands 6&7 corrected for electric standing waves.
    • Level 2.5 for point spectra WBS subbands and convert to simple spectrum (the polarisations will still not be averaged).
    • New HIFI beams.

  • PACS
    • Photometer: New maps valid for point source and extended emission analysis: JScanam, UniMap (new!).
    • Spectrometer: Cubes with evenly sampled spectral info.

  • SPIRE
    • Photometer: Better deglitched maps (multi-pass pipeline).
    • Spectrometer: Improved product structure inside observation contexts.

 

Core system

HIPE

Revision 292014-12-09 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 100 to 100
 
  • Backups for new files are automatically created, and reopened in a subsequent session upon crashes like a power cut. If the session terminates gracefully, the backup files are deleted.
  • Animations when maximizing or restoring parts of the viewer will be shown (see Transitions).
Added:
>
>
  • The browse image of the Observation viewer can be opened with the Image viewer by means of Right click > Open in image viewer.
 
  • The ArrayDataset viewer shows the unit when specified.
  • Column objects from a TableDataset have now a custom viewer.
Line: 126 to 127
 

Miscellany

Changed:
<
<
The Help > Check Java platform option shows the running Java version.
>
>
  • The Help > Check Java platform option shows the running Java version.
  • The Login panel is shown back at the top left corner of the Data Access perspective, and it has been simplified.
 

Plotting

Revision 282014-12-09 - ChrisPearson

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 436 to 436
 

Photometer Pipeline

  • The new 2-Pass Pipeline has replaced the standard pipelines for Large, Small and Parallel modes. The 2-Pass Pipeline runs iteratively on the data to produce optimally cleaned timelines. The first pass is run with minimal tasks (e.g. no FT tasks). Glitches are detected and flagged using 2nd level deglitching outlier detection. The full pipeline is then run using all the standard tasks.
  • New useful scripts to allow users to reconstruct the beam model and all calibration parameters (Photometer Calibration scripts) for both point, extended and semi-extended sources.
Changed:
<
<
  • New SPG map merging script utilising the 2-pass pipeline to produce Level 2.5 products.
>
>
  • New SPG map merging script utilising the 2-pass pipeline to produce Level 2.5 products. All parallel mode observations are reprocessed together as pairs at the Level 2.5 stage. After the level 2.5 stage all the Level 0.5, Level 1,Level 2, Level 2.5 data are updated with the impoved quality data.
 
  • Optimized parameters for Timeline Fitter in photometry script. Background annulus changed for faster run time.

Revision 272014-12-09 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 99 to 99
 

Editors & Viewers

  • Backups for new files are automatically created, and reopened in a subsequent session upon crashes like a power cut. If the session terminates gracefully, the backup files are deleted.
Changed:
<
<
  • Animations when maximizing or restoring parts of the viewer will be shown; see Transitions.
>
>
  • Animations when maximizing or restoring parts of the viewer will be shown (see Transitions).
 
  • The ArrayDataset viewer shows the unit when specified.
Added:
>
>
  • Column objects from a TableDataset have now a custom viewer.
 

Status bar

Revision 262014-12-09 - ChrisPearson

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 433 to 433
 

Common Pipeline

Photometer Pipeline

Added:
>
>
  • The new 2-Pass Pipeline has replaced the standard pipelines for Large, Small and Parallel modes. The 2-Pass Pipeline runs iteratively on the data to produce optimally cleaned timelines. The first pass is run with minimal tasks (e.g. no FT tasks). Glitches are detected and flagged using 2nd level deglitching outlier detection. The full pipeline is then run using all the standard tasks.
  • New useful scripts to allow users to reconstruct the beam model and all calibration parameters (Photometer Calibration scripts) for both point, extended and semi-extended sources.
  • New SPG map merging script utilising the 2-pass pipeline to produce Level 2.5 products.
  • Optimized parameters for Timeline Fitter in photometry script. Background annulus changed for faster run time.
 

Spectrometer Pipeline

Revision 252014-12-09 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 98 to 98
 

Editors & Viewers

Changed:
<
<
Backups for new files are automatically created, and reopened in a subsequent session upon crashes like a power cut. If the session terminates gracefully, the backup files are deleted.
>
>
  • Backups for new files are automatically created, and reopened in a subsequent session upon crashes like a power cut. If the session terminates gracefully, the backup files are deleted.
  • Animations when maximizing or restoring parts of the viewer will be shown; see Transitions.
  • The ArrayDataset viewer shows the unit when specified.
 

Status bar

Revision 242014-12-09 - SteveGuest

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 448 to 448
 

General

 

Photometer

Changed:
<
<
 
>
>
  • Significant improvements to the bolometer finder tool have been made. 
 

Spectrometer

  • Minor updates, bug fixes, and improved documentation for the Semi-extended correction task

Revision 232014-12-09 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 68 to 68
 
import sys
Changed:
<
<
print str(sys.argv)
>
>
print sys.argv
 
Line: 77 to 77
 
> hipe test.py arg1 arg2 arg3
Changed:
<
<
['test.py', 'arg1', 'arg2', 'arg3']
>
>
['/home/user/test.py', 'arg1', 'arg2', 'arg3']
 
Line: 90 to 90
 
  • Maximizing or restoring the data viewer within the observation context viewer is shown with an animation.
  • Maximizing or restoring the browse image of the observation context viewer is animated too.
Added:
>
>

Outine

  • The Outline will show a small image for applicable variables like spectra, simple images, etc. when selected in the Variables view.
  • When selecting String variables, both the string contents and a literal to use it in a script will be shown in the Outline.
  • When selecting Unit variables, specific information about the unit is shown.
 

Editors & Viewers

Added:
>
>
Backups for new files are automatically created, and reopened in a subsequent session upon crashes like a power cut. If the session terminates gracefully, the backup files are deleted.
 

Status bar

Line: 105 to 112
 

File handling

Changed:
<
<
When saving a file of a known type with the file dialogue, the extension is automatically added to the file name if not explicitly done.
>
>
  • When saving a file of a known type with the file dialogue, the extension is automatically added to the file name if not explicitly done.
  • The filter for file types in the Navigator view is now more visible. In addition, it includes all kinds of defined file types.
 

Rescue mechanism

Line: 113 to 121
  The time intervals to run this check and to consider the application as not responding can be configured in Preferences > General > Rescue, where this mechanism can also be disabled altogether if you prefer so.
Added:
>
>

Miscellany

The Help > Check Java platform option shows the running Java version.

 

Plotting

Revision 222014-12-09 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 63 to 63
 

Batch mode

Added:
>
>
In batch mode, you can use sys.argv to get the input arguments. For example:
 
Changed:
<
<

Status bar

>
>
import sys
print str(sys.argv)

Then the output of this script would be:

> hipe test.py arg1 arg2 arg3
['test.py', 'arg1', 'arg2', 'arg3']

Transitions

In order to give better user feedback on GUI operations, screen transitions have been introduced in the following cases:

  • Switching perspective is performed by fading to the new perspective.
  • Navigation between pages in the welcome perspective is also done by fading to the new page.
  • Maximizing or restoring the data viewer within the observation context viewer is shown with an animation.
  • Maximizing or restoring the browse image of the observation context viewer is animated too.

Editors & Viewers

 
Deleted:
<
<
From the _Preferences > Data Access > MyHSA > Advanced panel, the user can:
 
Changed:
<
<
  • Show the MyHSA on-line/off-line checkbox in the status bar.
  • Hide the MyHSA on-line/off-line checkbox in the status bar.
>
>

Status bar

 
Changed:
<
<
This label provides a more prominent location reflecting the current connection status of MyHSA, irrespective of what view the user is currently using. It also offers the ability to update this status, the status is synced with that shown in the product browser and vice versa. (HCSS-18451)
>
>
From the Preferences > Data Access > MyHSA > Advanced panel, the user can:
 
Changed:
<
<

Directories

>
>
  • Show the MyHSA on-line/off-line checkbox in the status bar.
  • Hide the MyHSA on-line/off-line checkbox in the status bar.
 
Added:
>
>
This label provides a more prominent location reflecting the current connection status of MyHSA, irrespective of what view the user is currently using. It also offers the ability to update this status, the status is synced with that shown in the product browser and vice versa.
 
Changed:
<
<

Drag & Drop

>
>

File handling

 
Added:
>
>
When saving a file of a known type with the file dialogue, the extension is automatically added to the file name if not explicitly done.
 
Changed:
<
<

Editors & Viewers

>
>

Rescue mechanism

 
Added:
>
>
If HIPE gets in a non responding state for a period of time, it is automatically detected so that a popup window asks to restart it in an ordered way. This means all unsaved editors will be saved, and a new fresh HIPE session with the state of the frozen one will be opened.
 
Changed:
<
<

Miscellany

>
>
The time intervals to run this check and to consider the application as not responding can be configured in Preferences > General > Rescue, where this mechanism can also be disabled altogether if you prefer so.
 

Plotting

Changed:
<
<

 !TablePlotter

 !OverPlotter

>
>

 TablePlotter

 OverPlotter

 
Line: 147 to 175
 

Spectra

 Display

Added:
>
>
* The mosaic view of the Spectrum Explorer shows axes coordinates in a more convenient way, with less digits.
 

 Analysis

Line: 181 to 211
 

Tasks

  • Warning, important __list__ argument removed: if your task call still has it, remove it.
Added:
>
>
  • convertUnits : It can convert units from double parameters, array datasets and columns, plus units from table datasets or products, by allowing to select the target dataset or column.
  The GetObs Task now includes the possibility to retrieve observation using the spg version that created the Observation. See up to date Task definition with examples: (HCSS-19063)

Revision 212014-12-05 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 284 to 284
 

Baseline Removal

Changed:
<
<
  • In order to correct fitBaseline from crashing when 'useWbs' is set to 'True', especially when doglue is also set to 'True', the task will automatically override doglue to 'False'.
>
>
  • In order to prevent fitBaseline from crashing when 'useWbs' is set to 'True', especially when doglue is also set to 'True', the task will automatically override doglue to 'False'.
 

Electrical Standing Wave correction in HEB bands

Revision 202014-12-02 - EdwardPolehampton

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 356 to 356
 

Calibration Products

Added:
>
>
Note that the size of the calibration tree has increased by 120 MB, mainly due to the addition of the finely sampled (1") photometer beam maps.

Photometer

  • The beam profile maps are improved using the shadow observations of Neptune (i.e. better characterising and removing the background).
  • The corresponding beam areas given in the metadata of the beam products are updated accordingly.
  • The radial beam calibration product has been updated to match the new beams.
  • All the colour correction products have been updated following the beam updates.
  • In order to improve the operation of the source extractors, the beam profiles are adjusted to be relevant for a 1 Jy source - previously the peak of the profile was fixed to 1 Jy, but the new profiles integrate to 1 Jy. In addition, the map consists of an odd number of pixels so that the beam centre is precisely at the centre of the central pixel.
  • A new "fine" beam map is provided for each band with 1" pixels. The "nominal" maps are also available with pixel sizes of 6", 10" and 14" for PSW, PMW and PLW.
  • The K4 parameters in the Flux Conversion product metadata are updated (very small change to K4E)
  • The Planck HFI colour correction table has been updated following the beam updates (very small change).
  • The parameters in the temperature drift calibration product that are used to perform cooler burp correction have been updated to improve the correction.
 

Spectrometer

Changed:
<
<
  • Detector non-linearity correction factors updated. Does not affect the majority of observations, but is very important for faint targets observed at the beginning of each FTS observing day (when detector temperatures were low).
>
>
  • The non-linearity product has been updated to better fit observations made with low subktemp at the beginning of each pair of FTS observing days (i.e. the observations following the cooler recycle). This results in a much improved SLWC3 spectral shape.
  • The other RSRF, telescope model and flux conversion products were updated following the non-linearity update.
  • The PhaseCorrLim product was updated to correct the metadata - no effect on the data.
  • The TempDriftCorr product has been removed as it is not used by the pipeline.
 

Calibration Framework

Revision 192014-12-01 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 280 to 280
 
  • New output variable swTable was created to enable machine-readable output of fitted frequencies and amplitudes.
  • Output for fitHifiFringe now contains corrected data and summary table.
  • New input parameters "cal" and "saveInput" was created in fitHifiFringe to extract user input parameters into the calTree for later use on demand.
Added:
>
>
  • When a large portion of channels is flagged as invalid, i.e. when not enough channels can be fitted, fitHifiFringe will raise a comment that it is skipping the spectrum and then it will continue to the next one.
 

Baseline Removal

Changed:
<
<
  • In order to correct fitBaseline from crashing when 'useWbs' is set to 'True', especially when doGlue is also set to 'True', the task will automatically override doGlue to 'False'.
>
>
  • In order to correct fitBaseline from crashing when 'useWbs' is set to 'True', especially when doglue is also set to 'True', the task will automatically override doglue to 'False'.
 

Electrical Standing Wave correction in HEB bands

Revision 182014-12-01 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 41 to 41
  HIFI
Changed:
<
<
  * TBD
>
>
  • A new level 2.5 products has been generated for point mode observations. It is a stitched version of the level 2 spectra for each available spectrometer, in the form of a Spectrum1d.
  • All observations context will now contain a (Ta*) calibrated OFF position spectrum, in the same sky frequency scale as the ON position data.   * The new calibration tree contains updated beam models and efficiencies, as released initially as stand-alone products in September 2014. See for more details http://herschel.esac.esa.int/twiki/pub/Public/HifiCalibrationWeb/HifiBeamReleaseNote_Sep2014.pdf
  • In bands 6 and 7, the Electrical Standing Wave correction is now part of the standard pipeline, based on a data-base of optimised solutions stored in the calibration tree for each observation
  • For spectral scans, a significant fraction of the deconvolved level 2.5 products will benefit from flag tables optimised manually by the ICC, resulting in spur-free deconvolved spectra. This is not yet available for all spectral scan obsids though, but will be by HIPE 14. The list of optimised flags can be found in the calibration tree and further refined by the user using the flagTool if the User wishes so.
 
Line: 226 to 230
 

HIFI Calibration

Added:
>
>
The calibration tree released together with HIPE 13 contains the following updates:
  • it will contain a list of optimised channel flags for spectral scans obsids, that will be honoured by the deconvolution task and result in spur-free level 2.5 data in this mode. This is not yet available for all spectral scan obsids though, but will be by HIPE 14.
  • it will contain solution tables for Electrical Standing Wave correction in the bands 6 and 7. This allows for a correction of this affect in the pipeline itself. Note that this is not a guarantee that the baseline artefact will be perfectly removed but it will provide the best correction available to data. It is then possible to further optimise the correction - see the dedicated sections in the DRG.
  • it will contain the updated beam efficiencies and 2D beam models
 

HIFI Pipeline

hifiPipeline task

Level 0 Pipeline

Line: 233 to 242
 

HRS Pipeline

Level 1 Pipeline

Level 2 Pipeline

Added:
>
>
  • the sideband ratio correction is now taking into account the possible IF-dependency of the sideband ratio over the IF. It uses the IF-dependence information as given in the calibration tree. As of now, only the lower end of band 2a makes such an assumption. The presence of an IF-dependency will result in additional meta-data usbGain_n or lsbGain_n, which are the coefficients used to described the polynomial approximation of the gain dependence. Coefficients with index _0 simply correspond to the value of the constant, scalar, sideband ratio we used so far
  • a new task will automatically check for platforming in the WBS data and report any detection with a dedicated quality flag
  • a new task will create spectra for the OFF position data calibrated into the Ta* scale, on both USB and LSB frequency scales. For spectral scans, these can then be fed into deconvolution to create a deconvolved spectrum of the OFF position (option use_reference)
 

Level 2.5 Pipeline

  • Mapping mode: The cubes produced automatically by the pipeline (Level 2.5) (doGridding task), and consequently by hifiPipeline if you choose to run it again, will stitch the WBS subbands and create one final cube (with the index '_1'). You will not see individual cubes per subbands except for the HRS.
Changed:
<
<
  • Mapping mode: Level 2 HTPs , stitched, are now part of the Level 2.5.
>
>
  • Point mode: Level 2 HTPs , stitched, are forming the Level 2.5 products for this mode.
  • A large number of statistics (primarily noise and mean) is performed both on the level 2 and level 2.5 products, as well as on an equivalent products converted to the Tmb scale with H and V merging (this latter is then removed from the observation context and only the statistics is stored). This computation is controlled by the option computeRms. It is only set to True when run for the bulk reprocessing, but ignored by default when the level 2.5 pipeline is called interactively (this computation can be extremely time consumming for maps and spectral scans)
 
Line: 274 to 287
 

Electrical Standing Wave correction in HEB bands

Changed:
<
<
>
>
  • the task doHebCorrection is now part of the default pipeline, with several new options compared to its predecessor hebCorrection.
 

Deconvolution

Changed:
<
<
  • New option called enforce_spur_rejection to help control the existing spur_rejection option.
>
>
  • by default, the option chosen for spur_rejection will be based on the availability of not of optimised flag tables for a given obsid (see also the calibration section). When no table exists, the option is enfored to REJECT_SCANS_WITH_SPURS. When tables exist, the option is enfored to DO_NOT_REJECT_SCANS_WITH_SPURS
  • A new option called enforce_spur_rejection is however made available to help control the existing spur_rejection option.
  • A new option use_reference indicates that the deconvolution should be made on the OFF position spectra computed with HIPE 13
  • doDeconvolution has also been updated so it can take into account the IF-dependence of sideband ratio and use this a priori knowledge as preset to build the single sideband solution

convertK2Jy

  • with the availability of the new beam efficiencies and beam model, the task has been upgraded in order to make best use of the detailed beam knowledge. A new option hifiBeam, set to True by default, will use the fully-detailled HIFI beam to compute the coupling factor involved in the temperature to flux conversion for sources of non-zero size.
  • for non-point-like sources, the task now offers disk or Gaussian source models (option shape)
 

Spectral Cube creation

flagTool

Revision 172014-11-25 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 296 to 296
 
    • Updates to the chapter How to add and remove flags, section Setting flags interactively for many spectra (flagTool task).
    • Updates to the chapter Sideband deconvolution, section Basic strategy for running the deconvolution tool (doDeconvolution task).
    • Updates to the chapter Standing Wave Removal (fitHifiFringe task).
Added:
>
>
    • Updates to the chapter How to make a spectral cube to reflect changes in the level 2.5.
 

PACS

Revision 162014-11-25 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 263 to 263
 
    • Band 6 change startPeriod 100->80, typicalPeriod 320->600
    • Band 7 change startPeriod 100->80, typicalPeriod 320->625
    • Band 2,5 no change
Deleted:
<
<
  • Will now handle the combined flags NOT_CALIBRATED and SATURATED.
 
  • swTable had wrong unit for amplitude (kHz instead of K). This is now fixed.
  • New output variable swTable was created to enable machine-readable output of fitted frequencies and amplitudes.
  • Output for fitHifiFringe now contains corrected data and summary table.
Changed:
<
<
  • New input parameters "cal" and "saveInput" was created fitHifiFringe to extract user input parameters into the calTree for later use on demand.
>
>
  • New input parameters "cal" and "saveInput" was created in fitHifiFringe to extract user input parameters into the calTree for later use on demand.
 

Baseline Removal

Line: 296 to 295
 
    • A new chapter on The HIFI line identification tool (identifyLines task) is now available.
    • Updates to the chapter How to add and remove flags, section Setting flags interactively for many spectra (flagTool task).
    • Updates to the chapter Sideband deconvolution, section Basic strategy for running the deconvolution tool (doDeconvolution task).
Added:
>
>
    • Updates to the chapter Standing Wave Removal (fitHifiFringe task).
 

PACS

Revision 152014-11-25 - WimDeMeester

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 83 to 83
 

Plotting

Deleted:
<
<
The following changes are made to the Display class:
  • Replaced setCutLevels() by calcCutLevels()
  • Removed setCutLevels(double,double)
  • Renamed setCutLevels(double) to setCutLevelsPercentage(double)
 

 !TablePlotter

 !OverPlotter

Line: 121 to 117
 

 Display

Added:
>
>
The following changes are made to the Display class:
  • Replaced setCutLevels() by calcCutLevels()
  • Removed setCutLevels(double,double)
  • Renamed setCutLevels(double) to setCutLevelsPercentage(double)

  • The images can not only be flipped around the y-axis, but also around the x-, xy- and yx-axes. It is also possible to combine these flip-options : for example : y- and xy.
  • The annotation in Display are totally reworked. All old annotation methods are deprectated, the new one end with Annotation. For example : addEllipse becomes addEllipseAnnotation.

A lot of fixes in Display :

  • Display now displays images with some extremely high or low values correctly.
  • It is now possible to extract the current layer from a SimpleImage or from a SimpleCube with only one layer.
  • Display can now work with more than one extension.
  • The zoom in / out was by a factor of 4 instead of 2.
  • The Wcs viewer only shows the important wcs keywords now.
 

 Analysis

Added:
>
>
Bugfixes :
  • Mosaics close to the north or south pole missed a part of the sky.
  • Aperture photometry tasks have clear description on how the background subtracted flux is calculated.
  • Bugfixes in aperture photometry tasks.
  • You can choose to add an input parameter to annularSkyAperturePhotometry for choosing the step size of curve-of-growth output
 

Spectra

 Display

Revision 142014-11-24 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 239 to 239
 

HIFI Data Processing Tools

Standing Wave Removal

Changed:
<
<

Baseline Removal

>
>
  • Updates to default band-dependent period characteristics in task fitHifiFringe :
    • Band 1 change nfringe 2->3, endPeriod 120->200
    • Band 3 change typicalPeriod 620->95
    • Band 4 change typicalPeriod 620->95
    • Band 6 change startPeriod 100->80, typicalPeriod 320->600
    • Band 7 change startPeriod 100->80, typicalPeriod 320->625
    • Band 2,5 no change
  • Will now handle the combined flags NOT_CALIBRATED and SATURATED.
  • swTable had wrong unit for amplitude (kHz instead of K). This is now fixed.
  • New output variable swTable was created to enable machine-readable output of fitted frequencies and amplitudes.
  • Output for fitHifiFringe now contains corrected data and summary table.
  • New input parameters "cal" and "saveInput" was created fitHifiFringe to extract user input parameters into the calTree for later use on demand.
 
Added:
>
>

Baseline Removal

  • In order to correct fitBaseline from crashing when 'useWbs' is set to 'True', especially when doGlue is also set to 'True', the task will automatically override doGlue to 'False'.
 

Electrical Standing Wave correction in HEB bands

Revision 132014-11-24 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 217 to 217
 

Level 1 Pipeline

Level 2 Pipeline

Level 2.5 Pipeline

Added:
>
>
  • Mapping mode: The cubes produced automatically by the pipeline (Level 2.5) (doGridding task), and consequently by hifiPipeline if you choose to run it again, will stitch the WBS subbands and create one final cube (with the index '_1'). You will not see individual cubes per subbands except for the HRS.
  • Mapping mode: Level 2 HTPs , stitched, are now part of the Level 2.5.
 
Line: 248 to 250
 

Deconvolution

Added:
>
>
  • New option called enforce_spur_rejection to help control the existing spur_rejection option.
 

Spectral Cube creation

flagTool

Added:
>
>
  • Interactive (suitable for batch processing) and Expert modes now available.
  • maskTables: a new variable to manage Linemasks and Rowmasks files.
 

identifyLines

  • IdentifyLines is a new task that allows you to identify lines in your spectrum and then run a comparison of known lines with a Linelist.
Line: 259 to 265
 

Documentation

  • Updates to the HIFI Data Reduction Guide:
    • A new chapter on The HIFI line identification tool (identifyLines task) is now available.
Added:
>
>
    • Updates to the chapter How to add and remove flags, section Setting flags interactively for many spectra (flagTool task).
    • Updates to the chapter Sideband deconvolution, section Basic strategy for running the deconvolution tool (doDeconvolution task).
 

PACS

Revision 122014-11-19 - EdwardPolehampton

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 295 to 295
 

SPIRE

Deleted:
<
<
 

Calibration Products

Added:
>
>

Spectrometer

 
Added:
>
>
  • Detector non-linearity correction factors updated. Does not affect the majority of observations, but is very important for faint targets observed at the beginning of each FTS observing day (when detector temperatures were low).
 

Calibration Framework

Line: 325 to 315
 

Spectrometer Pipeline

Deleted:
<
<
 

Interactive Analysis and Tools

 
Added:
>
>
  • Update to the naming within the Observation Context to make it clearer which products are spectra/cubes and pointsource/extended calibrated.
  • Moved all of the final spectral products into the Level-2 context. Previously the extended calibrated spectra were in the Level-1 context, but now both point source and extended calibrated data are in the Level-2 context.
  • Introduced a new spectrum2d product for mapping observations to contain all of the spectra before they are gridded into a regular spectral cube. Previously these spectra were saved in individual products for each jiggle/raster position. The new product is added to the Level-2 context.
  • A new task is added to perform spectral apodization. This step was previously carried out with several tasks to take the inverse Fourier transform and apply the apodization on the interferogram. The new task works directly on the spectra.
  • Quality control parameters related to the unaveraged FTS scans are now written back into the interferogram products in the Level-1 context. Previously these metadata items were lost after averaging the spectra over all of the scans.
  • New metadata parameters to contain known pointing offsets from the commanded position have been added to the Level-2 spectra.
  • A limit has been placed on the spectral resolution allowed as input when a spectral mapping observation is gridded into a cube. This eliminates spectra that were clipped, and improves the quality of the maps.
 
Added:
>
>

Interactive Analysis and Tools

 

General

 

Photometer

 

Spectrometer

Added:
>
>
  • Minor updates, bug fixes, and improved documentation for the Semi-extended correction task
  • Updates to the task to take synthetic photometry of FTS spectra for comparison with the SPIRE photometer
  • New GUI interface for the script that determines the noise level in a spectrum
  • Small corrections to the useful scripts
  

Revision 112014-11-12 - AlvarGarcia

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 83 to 83
 

Plotting

Added:
>
>
The following changes are made to the Display class:
  • Replaced setCutLevels() by calcCutLevels()
  • Removed setCutLevels(double,double)
  • Renamed setCutLevels(double) to setCutLevelsPercentage(double)
 

 !TablePlotter

 !OverPlotter

Revision 102014-10-09 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 246 to 246
 

Spectral Cube creation

flagTool

Added:
>
>

identifyLines

  • IdentifyLines is a new task that allows you to identify lines in your spectrum and then run a comparison of known lines with a Linelist.
 

HiClass export tool

Documentation

Added:
>
>
  • Updates to the HIFI Data Reduction Guide:
    • A new chapter on The HIFI line identification tool (identifyLines task) is now available.
 

PACS

Revision 92014-09-19 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 225 to 225
 

Spectrum Explorer (HIFI specifics)

Spectrum tasks (HIFI specifics)

Added:
>
>
  • The PolarPair class has been removed. Instead the polarPair task should be used. This should only affect scripts written prior to 2013.
 

Revision 82014-08-06 - JonathanCook

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 161 to 161
 The GetObs Task path and poolLocation parameters have been consolidated. See up to date Task definition with examples: (HCSS-19241)
Added:
>
>
The FitsReaderTask and ImportSpectralCubeTask now support reading and loading of ALMA/CASA cubes directly into HIPE and the Spectrum Explorer - HCSS-18110
 

Quality control

Revision 72014-06-16 - AlvarGarcia

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 180 to 180
 

Virtual Observatory

Changed:
<
<
>
>
The link to VOPlot in the External Tools sub-menu will be removed and substituted by Topcat.
 

Installer

Revision 62014-04-30 - JonathanCook

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 158 to 158
  The GetObs Task now includes the possibility to retrieve observation using the spg version that created the Observation. See up to date Task definition with examples: (HCSS-19063)
Added:
>
>
The GetObs Task path and poolLocation parameters have been consolidated. See up to date Task definition with examples: (HCSS-19241)
 

Quality control

Revision 52014-03-21 - JonathanCook

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 156 to 156
 
  • Warning, important __list__ argument removed: if your task call still has it, remove it.
Added:
>
>
The GetObs Task now includes the possibility to retrieve observation using the spg version that created the Observation. See up to date Task definition with examples: (HCSS-19063)

 

Quality control

Revision 42014-03-17 - JonathanCook

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 62 to 62
 

Status bar

Added:
>
>
From the _Preferences > Data Access > MyHSA > Advanced panel, the user can:

  • Show the MyHSA on-line/off-line checkbox in the status bar.
  • Hide the MyHSA on-line/off-line checkbox in the status bar.

This label provides a more prominent location reflecting the current connection status of MyHSA, irrespective of what view the user is currently using. It also offers the ability to update this status, the status is synced with that shown in the product browser and vice versa. (HCSS-18451)

 

Directories

Revision 32014-02-17 - AlvarGarcia

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 355 to 355
 

-- AlvarGarcia - 15 Jan 2014 \ No newline at end of file

Added:
>
>
META FILEATTACHMENT attachment="GoogleAnalytics.txt" attr="h" comment="" date="1392627508" name="GoogleAnalytics.txt" path="GoogleAnalytics.txt" size="549" stream="GoogleAnalytics.txt" user="Main.AlvarGarcia" version="1"

Revision 22014-01-15 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
Deleted:
<
<
 
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->
Line: 147 to 146
 

Tasks

Added:
>
>
  • Warning, important __list__ argument removed: if your task call still has it, remove it.
 

Quality control

Revision 12014-01-15 - AlvarGarcia

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="HipeWhatsNew"

<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's coming up in HIPE 13

PDF Version Click here
Help Interested in what's new for other versions of HIPE? See this page for links to all the What's New documents from HIPE 4.0 onwards.
See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.

Planned work

HIPE

   * TBD

HIFI

   * TBD

PACS

 * TBD

SPIRE

   * TBD

Highlights

HIPE

   * TBD

HIFI

  * TBD

PACS

   * TBD

SPIRE

   * TBD

Core system

HIPE

Batch mode

Status bar

Directories

Drag & Drop

Editors & Viewers

Miscellany

Plotting

 !TablePlotter

 !OverPlotter

Astronomical utilities

 

Numeric routines

 Data fitting

 Data interpolation

Random numbers

Statistics

 Trend analysis

 Mapping

Other functions

Source extraction

Images

 Display

 Analysis

Spectra

 Display

 Analysis

Data cubes

 Display

 Analysis

Herschel Science Archive

Calibration sources

Products and datasets

 Observation context

Product Access Layer 

Tasks

Quality control

Systematic product generation

Data input-output

Virtual Observatory

Installer

Documentation

HIFI

HIFI Software Configuration and Properties

HIFI Calibration

HIFI Pipeline

hifiPipeline task

Level 0 Pipeline

WBS Pipeline

HRS Pipeline

Level 1 Pipeline

Level 2 Pipeline

Level 2.5 Pipeline

HIFI Products

HIFI Spectrum Toolbox

Spectrum Explorer (HIFI specifics)

Spectrum tasks (HIFI specifics)

HIFI Data Processing Tools

Standing Wave Removal

Baseline Removal

Electrical Standing Wave correction in HEB bands

Deconvolution

Spectral Cube creation

flagTool

HiClass export tool

Documentation

PACS

Pipeline

Photometry

Spectroscopy

 

Analysis tools

Photometry

Spectroscopy

Product changes

Photometry

Spectroscopy

Documentation

SPIRE

Calibration Products

Calibration Framework

Preprocessing

Engineering Conversion

Common Pipeline

Photometer Pipeline

Spectrometer Pipeline

 

Interactive Analysis and Tools

General

 

Photometer

 

Spectrometer

 

Documentation

Release Notes

   * Full list of tickets implemented: here.


<-- COMMENT BOX CODE - DO NOT EDIT -->
blog comments powered by Disqus
<-- END OF COMMENT BOX CODE -->

<--
   * Set ALLOWTOPICCHANGE = RegisteredUsersGroup, TWikiAdminGroup, MarkRKidger
-->

-- AlvarGarcia - 15 Jan 2014

 
This site is powered by the TWiki collaboration platform Powered by Perl