Tags:
create new tag
view all tags

What's New for HIPE 15.x

PDF Version Click here

Help Interested in what's new for other versions of HIPE? See this page for links to all the What's New documents from HIPE 4.0 onwards.
See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.


Highlights

Core

  • The FitsArchive task will now write FITS files with header entries organised in categories. See the Data input-output section below for more details.
  • A new General Useful Script named Astrometry Thermoelastic Drift Correction has been introduced, allowing to perform a tentative astrometry correction for those observations taken under 'warm' attitude conditions (Star Tracker 2 temperature > -15 C) resulting in small pointing drifts.
  • 15.0.1: The documentation has been updated to reflect the changes to the Herschel Science Archive interface, which is now a web application instead of a Java application.
HIFI
  • The task convertK2Jy has been upgraded in order to take as a new input parameter a model for the source brightness distribution shape.
  • A new task identifyLinesCatalog has been introduced, allowing to assign spectral species and transitions to a catalogue of spectral features passed as input.
  • The documentation has been revised to ensure a consolidated and conherent set of legacy manuals.
  • The masking of lines has been improved in the task mkRms.
  • 15.0.1: a fix was applied to fitBaseline so that it again takes into account flags added by the user interactively (there was a bug in 15.0.0)
PACS
  • An update to the aperture corrections calibration file for photometry
  • A minor addition to the spectroscopy pipeline tasks (update of the Meta data "ra" and "dec" to all Level 2 cubes)
  • Information about extracting uncertainties from scan maps has been added to the PDRG for photometry (chp. 4)
  • 15.0.1: Minor updates to the useful scripts to link to the places where advice for each scripts' topic can be found
  • 15.0.1: Minor updates to the PDRG and PPE to link to the new documentation placed on the Herschel Legacy Library pages, and to reflect recent advice provided in the PACS Handbook there.

SPIRE

  • 15.0.1: sourceTimelineFitter now ignores serendipity scans

Core system

HIPE

Editors & Viewers

  • Editor for XML / HTML files added.
  • Internal browser for HTML files provided.
  • External browser for HTML files available through Open With.
  • Open votable files directly through Open With VO Table Reader

Console

Improved auto-completion mechanism.

Plug-ins

  • The plug-ins dialogue has been made more user friendly.
  • Plug-ins can not only add sub-panels in the Variables view, but also remove existing ones.

Plotting

TablePlotter

OverPlotter

Astronomical utilities

A new script named Astrometry Thermoelastic Drift Correction has been incorporated to the General Useful Scripts category. This is aimed at estimating astrometry shifts induced by thermoelastic drifts. These appear when an observation is performed at 'warm' attitude conditions (Star Tracker 2 temperature > -15 C). The script calculates the average temperature of STR2 sensors during the mid-point of the observation and, if that is > -15 C, it uses an established STR2-Z-axis offset correlation function to get the Z-axis offset and derive RA and DEC offsets. Two possible outputs are provided.

  • A new pointing product that takes into account the updated rotation matrix/quaternion and corrects the filtered attitude. This product can be used by the user to reprocess the observation.
  • New Level 2 products (only applicable to Photometry) where the offsets in RA y DEC (derived from the Z-axis offset) have been used to adjust the WCS.

Numeric routines

Data fitting

Data interpolation

Random numbers

Statistics

Trend analysis

Mapping

Other functions

Source extraction

Images

  • Save browse image products directly to jpeg via the new saveAsJpeg method and also via right-click Send to options from Data Panel

Display

Analysis

Spectra

Display

  • Coordination between the selected spectra and the spectrum explorer made more rational.
  • Option to choose the target spectra for the selected task added in preferences.
  • Former Create variable option in the spectrum plot is split into Create plot variable (referring to the SE plot) and Create plot copy (creating a copy of the spectrum plot, outside the SE).

Analysis

  • Spectrum task extract is faster than in previous versions.
  • Parameters showFlags, showWeight, showGrid and displayRows have been added to the openSE task.

Data cubes

Display

  • The cube explorer opens the preview panel from start. This solves the bothersome behaviour of previous versions where it appeared when hovering the mouse on a spaxel, thus moving the spaxels display from below the mouse cursor.
  • The cube toolbox drop-down list shows tasks in a more logical order.
  • Ranges in tasks of the cube toolbox have to be defined afresh when a new task is selected.

Analysis

Spectrum task extract is much faster when applied on spectral cubes than in previous versions.

Herschel Science Archive

Calibration sources

Products and datasets

Observation context

Product Access Layer

Tasks

  • The Applicable node of the Tasks view is opened by default, so tasks applicable to the current selection are more directly visible.
  • The automatic multiple output task checker that was introduced with the transition from HIPE 12.0 to HIPE 13.0 has been removed.
  • Trying to change the task name or prime input, once the task has been constructed, throws an exception.
  • openVariable is able to open a variable directly, in addition to opening it by name. For example:

p = Product()
openVariable("p")  # already available with previous versions of HIPE
openVariable(p)    # also available since HIPE 15.0

  • Instrument must now be specified when querying parallel observations via the getObservation task.
  • The getObservation task now supports interruption meaning the Stop button in HIPE can be used to halt the process when loading an observation.
  • Created a new task for writing TableDataset to VO table XML files.
  • Created a new task for reading VO table XML files into a TableDataset.
  • It is now possible to read FITS files without modifying their FITS header using the RawFitsReader functionality which is provided by FitsArchive and also available via the simpleFitsReader Task

archive = FitsArchive()
archive.setReader(FitsArchive.RAW_READER)
product = archive.load('file_path')

#Alternatively use the raw option provided with simpleFitsReader
product = simpleFitsReader(file='file_path', reader='raw')

Quality control

Systematic product generation

Data input-output

The FitsArchive task has been upgraded in order to re-organise keywords in the products FITS headers in a much more structured fashion. Keywords will not be grouped and ordered according to categories, in the same spirit as what was done e.g. for products from the Hubble Space Telescope or Spitzer. The following snapshots illustrate this re-structuration for the first part of a given FITS header in one HIFI product. The top image shows the product header as of 14.1.0. The bottom image shows the product header that would be obtained by saving with the FitsArchive task under 15.

Virtual Observatory

Installer

Documentation

External libraries

HIPE uses a rich set of external libraries, of which the following have been upgraded:

  15.0 14.0
Java 1.8.0_101 1.8.0_60
Jetty 9.2.12-2 6.1.26
Jython 2.7.0 2.5.2
JSAMP 1.3.5 1.3.0
nom.tam.fits 1.14.2 1.6.0
fop 2.1 1.1

HIFI

HIFI Software Configuration and Properties

HIFI Calibration

HIFI Pipeline

hifiPipeline task

Level 0 Pipeline

WBS Pipeline

HRS Pipeline

Level 1 Pipeline

Level 2 Pipeline

Level 2.5 Pipeline

HIFI Products

HIFI Spectrum Toolbox

Spectrum Explorer (HIFI specifics)

Spectrum tasks (HIFI specifics)

The task mkRms has been improved in order to better handle line masking when computing the statistics in spectra containing spectral lines. This will lead to very slight differences with the noise estimates reported in the pipeline products from 14.1.

On top of that, the task convertK2Jy has been upgraded so that it can take as source model an input file in the form of an image. This allows more complex and ad hoc source filling factor estimate in case the Gaussian or Disk approximation are not appropriate. See the Data Reduction Guide and task URM for more details.

HIFI Data Processing Tools

Standing Wave Removal

Baseline Removal

15.0.1: a fix was applied to fitBaseline so that it again takes into account flags added by the user interactively (there was a bug in 15.0.0)

Deconvolution

Spectral Cube creation

flagTool

identifyLinesCatalog

See also examples in the task URM.

HiClass export tool

The useful script explaining how to export HIFI data into Class with the hiClass task (SaveObsContextToClass.py) has been removed in 15. This is because HIFI FITS files as processed by HIPE 12 and later can now be directly read by Class, if one uses a Gildas installation later than October 2015 - see the following Manual. For HCSS products older than HIPE 12, we recommend to either use more recent products, or fall back on the hiClass export, using the documentation of the task as provided in its URM.

Documentation

Update of DRG and some URMs

PACS

Pipeline

Photometry

Spectroscopy

  • Addition of centerRaDecMetaData to the interactive pipeline scripts, with the result that the Meta data "ra" and "dec" are updated to be the actual centre of each cube, separately. This affects the Meta data only, not the actual spatial grid of the cubes.

Calibration

Photometry

  • An update to the aperture correction factors has been added to the final calibration tree (version 78). In the previous version, only the medium scan speed prime mode aperture corrections were provided in the table. However, the corrections actually depend on the scan speed and are different for the prime/parallel mode. Therefore, the medium, fast, prime, and parallel values are all provided in this final calibration file release. The differences depend on the aperture one uses when performing the aperture photometry. For smaller aperture sizes the difference can be quite large, by about 2xFWHM (of the beam) they becomes negligible. To take advantage of the new corrections it is necessary to re-run the aperture correction task providing two optional parameters: parallelObservation (True/False) and scanSpeed (20 or 60).

Spectroscopy

Analysis tools

Photometry

Spectroscopy

  • extractSpaxelSpectrum and the pointSourceLossCorrection tasks now propagate the spectral errors
  • Minor updates to the useful scripts to link to the places where advice for each scripts' topic can be found

Product changes

Photometry

Spectroscopy

Documentation

  • A new section in chp. 4 of the PDRG for photometry includes a guide to getting the uncertainties for scan maps
  • A basic explanation of the forward modelling tool has been added to the PDRG for spectroscopy (chps 8, 9) and to the PACS documentation page (accessible via the HSC main pages)
  • Minor updates to the PDRG and PPE to link to the new documentation placed on the Herschel Legacy Library pages, and to reflect recent advice provided in the PACS Handbook there.

SPIRE

The following are the only changes or new addtions in HIPE v15 for SPIRE:

Interactive Analysis and Tools

  • Using the final, legacy URLs, to the calibration tree and some auxiliary files (e.g. Planck all sky maps) in the interactive useful scripts.
  • sourceTimelineFitter now ignores samples from serendipity scans.

Documentation

Final updates in SPIRE Data reduction Guide.


blog comments powered by Disqus


Topic attachments
I Attachment History Action SizeSorted ascending Date Who Comment
PNGpng header2.png r1 manage 15.7 K 2016-12-02 - 10:22 DavidTeyssier  
PNGpng header1.png r1 manage 17.3 K 2016-12-02 - 10:22 DavidTeyssier  
Edit | Attach | Watch | Print version | History: r27 < r26 < r25 < r24 < r23 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r27 - 2017-04-24 - DavidTeyssier
 
This site is powered by the TWiki collaboration platform Powered by Perl