TWiki> Public Web>HipeWhatsNew>HipeWhatsNew6x (revision 20)EditAttach

What's New in HIPE 6

PDF Version Click here

Core system

The Java Virtual Machine included in Windows and Linux downloads has been updated to version 1.6.0_22.

HIPE

Plotting

  • A small subplot can be put inside a bigger plot.

TablePlotter

OverPlotter

Dataset viewer

Astronomical utilities

Numeric routines

Data fitting

Data interpolation

Random numbers

Statistics

Trend analysis

Mapping

Other functions

Source extraction

Images

Display

  • Major speedup when zooming in on images. Less memory is used when zooming in.
  • Fixed some memory issues. The display does not longer keep a lot of memory in use.
  • It is possible to hide or show the panel with the panner and the zoom

Analysis

Spectra

Display

Analysis

Data cubes

Display

Analysis

Herschel Science Archive

Calibration sources

Products and datasets

Observation context

Product Access Layer

Tasks

  • Improved documentation
  • Added descriptions to tasks. All tasks can provide a high level description (shown in tooltips in Task View).
  • Simplified localStoreWriter task

Quality control

  • QCR Browser allow to perform bulk actions on several qc reports simultaneously
    • This includes: Generate QC Summary directly as a menu option
  • Show metadata proCaveats in the browser
  • Operator can assign different colors to the quality flags

Systematic product generation

  • SPG GUI can now launch OD preprocessing ( Missing TM , Events Log , HK products )
  • Connect to database only when needed, releasing this resource otherwise
  • Extended process:
    • Re-process feature for SYSTEMATIC and ON_DEMAND when the original data is obsolete
    • Cancel the OC ingestion if the process throws an Error or Exception.
    • Limit the number of obs. being processed
    • Memory management in operation's environment for subordinate jobs
  • On-demand pipelines must be able to process any observation with the latest auxiliary and calibration data

Bulk specific tool:

  • Operator can now re-execute individual jobs
  • Ability to launch one of 2 parallel mode obsids

Data input-output

Virtual Observatory

Installer

Documentation

Help viewer

  • Advanced search functionality
  • Added information about currently viewed document and current version of HIPE
  • New functionality to jump to the index page of the currently open manual

HIFI

HIFI Software Configuration and Properties

HIFI Calibration

HIFI Pipeline

hifiPipeline task

Level 0 Pipeline

WBS Pipeline

HRS Pipeline

Level 1 Pipeline

Level 2 Pipeline

HIFI Products

HIFI Spectrum Toolbox

Spectrum Explorer (HIFI specifics)

Spectrum tasks (HIFI specifics)

HIFI Data Processing Tools

Standing Wave Removal

Baseline Removal

Deconvolution

Spectral Cube creation

HiClass export tool

Documentation

PACS

Pipeline

  • Full functional slicing framework
    • used for spectrometer processing, only partly used for photometer processing
  • Significant cleanup and re organisation of interactive processing scripts.
  • New structure of Menu -> Pipeline -> PACS
  • A number of instrument mode specific user provided IA scripts are available now

Photometry

  • Proposal for a two-stage photometer pipeline with masked highpass filtering implemented
  • MMT Deglitching may mask a glitch, although the signal would be unmodified
  • Improvements on data filtering (usage of BBID in removeCalBlocks, usage of scan legs etc.)
  • Significant improvements in the SSO astrometry (correctRaDec4Sso)
  • Full functional MadMap processing implementation inclusive of cross scans

Spectroscopy

  • Linearity correction removed from pipeline scripts
  • Usage of elements of the Spectrum Explorer
  • The fitter tool can now be used for all PACS products.

Calibration

Photometry

Spectroscopy

  • New wavelength calibration applied by using the cal file wavePolynomes
  • The NominalResponse cal file is active, including the 1.3 and 1.1 ILT to flight conversion factors.
    • This means, if you process data you shall not apply these correction factors manually.
  • Also the pixel-pixel flatfield improvement described in PICC-KL-TN-045 (available in the central file) is implemented in this NominalResponse calfile

Analysis tools

  • Calibration framework : As of build 6.0.1452 the calibration framework will read the calibration products no longer from the distribution, but from a directory on your local machine.
    • You will be presented with a popup dialog asking to install the calibration. The dialog looks like this.
      • Screen_shot_2011-01-13_at_12.28.15.png *Please go through the process of installing the calibration products on your local machine.
    • During the HIPE startup process, the calibration framework will check for new updates in the background and notify you of any updates.
    • If you want to start the update manually, go to the Tools>pacs-cal menu and select Run Updater...
    • You can read more about the PACS Calibration Framework in the comprehensive document available in the public part of the Wiki.

  • Data Access : A new keyword is introduced to getObservation which allows the user to disable the Cache when using the HSA.
    • obs = getObservation(123456789, useHsa=True, useCache=False)
    • This should solve many problems when the user just wants to download and save an observation from the HSA on his/her local machine, as the data will not be saved twice (once in the cache and once in the local store pool) anymore.

Photometry

Spectroscopy

Product changes

Photometry

Spectroscopy

Documentation

SPIRE

Calibration Products

Calibration Framework

Preprocessing

Engineering Conversion

Common Pipeline

Photometer Pipeline

Spectrometer Pipeline

Level 0 to Level 0.5

See Engineering Conversion

Level 0.5 to Level 1

Processing Steps

Phase Correction
A new algorithm has been implemented that corrects for asymmetries using the low-resolution portion of the measured interferograms.

Data Products
Level-1 spectral products now contain all measured extended flux-calibrated spectra rather than the average spectra per detector.

Level 1 to Level 2

Processing Steps
Spectral Averaging
  1. The spectral averaging step is now only apply to Sparse, Point-source observations.
  2. The error column in the output data product is now populated with the standard error of the mean (Standard Deviation / SQRT(Number of Samples)) of the input spectral samples.

Spectral Maps
Level2 spectral cubes are now created by averaging flux values within each pixel in the output map. In addition, the execution speed of Spectral Mapping processing step has been greatly improved - a set of high resolution spectra is now processed in approximately 10 seconds.

Data Products
Level-1 spectral products now contain all measured extended flux-calibrated spectra rather than the average spectra per detector.

New Utitlities

A new set of utility functions and tasks have been implemented for HIPE version 6.

  1. filterChannels. A task that allows one to remove a selection of channels from any type of SPRIE FTS data product.
  2. removeOutOfBand A utility task that allows one to remove a from a SPIRE FTS Spectral data product spectral samples outside a specified band.
  3. makeSameOpds A utility task that allows one to remove a from a SPIRE FTS Interferogram data product samples outside a specified OPD range.
  4. mergeFtsBuildingBlocks A utility task that merges Level-1 SPIRE FTS Interferogram and Nominal Housekeeping products.
  5. specMergeHighLowSds A utility task that is used for HR=+=LR SPIRE FTS observations. This task appends the LR portion of the interferograms from the HR buildling block to the interferograms from the LR building block at each Point and Jiggle position of the observation.
  6. observationLayout A utility task that inspects the Level-0.5 context of a SPIRE FTS observation and returns a table linking the Building Block ID numbers with Point numbers, Jiggle numbers, and the commanded resolution.

Interactive Analysis and Tools

Mosaic plotting utility
A new plotting utility, Mosaic, has been developed to allow one to plot the contents of a SPIRE FTS Interferogram or Spectral Data Product in a grid that corresponds to the physical layout of the SPIRE SLW and SSW detector arrays.

Documentation

Edit | Attach | Watch | Print version | History: r49 | r22 < r21 < r20 < r19 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r20 - 2011-01-17 - EkkehardWieprecht
 
This site is powered by the TWiki collaboration platform Powered by Perl