Difference: HipeWhatsNew8x (63 vs. 64)

Revision 642011-12-02 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's coming up in HIPE 8

Line: 18 to 18
 

Status bar

Changed:
<
<
The maximum memory level reached is indicated with a vertical line in the memory bar. The tooltip shows the actual reached value, and allows to reset it so that further computations can record its own maximum.
>
>
The maximum memory level reached is indicated with a vertical line in the memory bar. The tooltip shows the actual maximum value, and allows you to reset it.
 

Editors & Views

Variables View
Changed:
<
<
  • Added contextual menu action Show contents to show the contents of a variable in the console.
  • Added contextual menu action Send To -> Text File to save the contents of TableDatasets in files.
>
>
  • Added contextual menu entry Show contents to show the contents of a variable in the console.
  • Added contextual menu entry Send To --> Text File to save the contents of table datasets to files.
 
  • Names of variables created from the Product Viewer are more consistent and meaningful.

Dataset Viewer
Line: 34 to 34
 
  • The title (product's description) can be selected and copied.
  • The expanded sections can be resized, for example to show more metadata rows.
  • The summary of the Observation Viewer has been enriched with useful metadata of the observation being shown.
Changed:
<
<
  • Products can be saved directly from the tree with the Sent To action of the popup menu, without needing to create an intermediate variable.
>
>
  • Products can be saved directly from the tree with the Send To entry of the popup menu, without creating an intermediate variable.
 
Jython & Text Editors
  • The supported Jython language has been upgraded from version 2.1 to 2.5.2.
Changed:
<
<
  • Scripts and text files coming with the installation or with plug-ins are safe to edit without losing the original version.
>
>
  • Scripts and text files coming with the installation or with plug-ins are now safe to edit without losing the original version.
 
  • Improved the step by step execution of scripts. See the note on this page for more details.

Help & Debug

  • The Tip of the day contains information on the Console meta commands (those starting with !)
Changed:
<
<
  • If you get temporary (or permanent) freezes in the HIPE GUI, or you have created a custom task dialogue or view in HIPE, you can debug it by enabling the corresponding preference in Edit > Preferences > General > Debug. Any problem found with this debugging can be reported like previously, with the HSC Helpdesk or Give Feedback options of the Help menu.
>
>
  • If you get temporary (or permanent) freezes in the HIPE GUI, or you have created a custom task dialogue or view in HIPE, you can debug it by enabling the corresponding preference in Edit --> Preferences --> General --> Debug. Any problem found with this debugging can be reported like previously, with the HSC Helpdesk or Give Feedback options of the Help menu.
 

Plug-ins

Line: 57 to 57
 
  • Fixed a problem with the scanamorphos plug-in, which would not read correctly some scanamorphos output. The plug-in assumed that the PrimaryImage HDU contained a 3-dim array. Output containing a 2-dim array in the PrimaryImage HDU was not read correctly.

Upgrades

Changed:
<
<
If there is a new user release available, HIPE would notice it and offer to install the new version.
>
>
If there is a new user release available, HIPE will notify you and offer to install the new version.
 

Plotting

PlotXY is now able to display images (i.e. 2D data). Look at an advanced example here.

TablePlotter

Changed:
<
<
TablePlotter can display plot with Date/Time format.
>
>
TablePlotter can now display plot with Date/Time format.
 

OverPlotter

Changed:
<
<
OverPlotter can display plot with Date/Time format.
>
>
OverPlotter can now display plot with Date/Time format.
 

Astronomical utilities

Changed:
<
<
  • Planck class moved from spire_ia_pipeline_spec_util to ia_toolbox_astro. If you are doing an explicit import (it is not needed) you have to change it to (preferred) import herschel.ia.toolbox.astro.Planck or from herschel.ia.toolbox.astro import Planck
  • New task astrometryFix: adjusts the absolute pointing of map data to give the best possible agreement in overall astrometry with some ancillary data. This is done either by cross correlation of the map(s) with ancillary map data, or by stacking the map(s) at a list of known source positions (e.g., from radio observations).
>
>
  • Planck class moved from spire_ia_pipeline_spec_util to ia_toolbox_astro. If you are doing an explicit import (it is not needed) you have to change it to (preferred) import herschel.ia.toolbox.astro.Planck or from herschel.ia.toolbox.astro import Planck
  • New task astrometryFix: adjusts the absolute pointing of map data to give the best possible agreement in overall astrometry with some ancillary data. This is done either by cross correlation of the map(s) with ancillary map data, or by stacking the map(s) at a list of known source positions (e.g., from radio observations).
 

Numeric routines

Data fitting

Line: 79 to 79
 

Trend analysis

Mapping

Wavelet transform

Changed:
<
<
  • Continuous Wavelet Transform synthesis operation now adds low frequencies for a reliable reconstruction
  • Continuous Wavelet Transform is able to deal with Complex1d signal
>
>
  • Continuous Wavelet Transform synthesis operation now adds low frequencies for a reliable reconstruction.
  • Continuous Wavelet Transform is now able to deal with Complex1d signal.
 

Other functions

Source extraction

Changed:
<
<
  • Improvements to handling of pixel coordinates and region of interest in sourceExtractor* tasks:
    • Output source list now contains pixel coordinates of source (x, y) = (column, row)
    • Added "getWcs" and "setWcs" methods to the SourceListProduct/Dataset
    • Added "roi" parameter (region of interest) to tasks
    • Removed "corner*" and "returnPixelCoordinates" parameters from tasks
  • Non-standard columns are now accepted in a SourceListDataset, e.g., when converting from a TableDataset (an exception will be thrown if there are no standard columns at all)
  • Removed "subtractMedianBackground" parameter from sourceExtractorSussextractor (it didn't do anything)
  • Minor bug fixes, performance improvements and various internal changes
>
>
  • Improvements to handling of pixel coordinates and region of interest in sourceExtractor tasks:
    • Output source list now contains pixel coordinates of source (x, y) = (column, row).
    • Added getWcs and setWcs methods to the SourceListProduct/Dataset.
    • Added roi parameter (region of interest) to tasks.
    • Removed corner* and returnPixelCoordinates parameters from tasks.
  • Non-standard columns are now accepted in a SourceListDataset, e.g., when converting from a TableDataset (an exception will be thrown if there are no standard columns at all).
  • Removed subtractMedianBackground parameter from sourceExtractorSussextractor (it did not do anything).
  • Minor bug fixes and performance improvements.
 

Images

Display

Changed:
<
<
  • The methods addCircle, addEllipse and addRectangle is not longer dependent on the zoomfactor of the image.
>
>
  • The methods addCircle, addEllipse and addRectangle ae no longer dependent on the zoom factor of the image.
 

Analysis

Spectra

Display

Analysis

Changed:
<
<
  • The SpectrumFitter now recognizes a CUBE such that the user can give the x,y of the spectrum. The script as written by the SpectrumFitterGUI also takes the correct spectrum.
>
>
  • The SpectrumFitter now recognizes a CUBE such that you can give the x,y position of the spectrum. The script as written by the SpectrumFitterGUI also takes the correct spectrum.
 
Changed:
<
<
  • avg now warns if wavescales are not sufficiently aligned to get a good result
>
>
  • The avg task now warns if wavescales are not sufficiently aligned to get a good result.
 
Changed:
<
<
  • various changes when applying toolbox tasks to cubes, in particular avg, select and extract now allow you to work on several pixels/spaxels within the cube rather than only the entire cube.
>
>
  • Various changes when applying toolbox tasks to cubes, in particular avg, select and extract now allow you to work on several pixels/spaxels within the cube rather than only the entire cube.
 
Changed:
<
<
  • resampling task now allows alternative ways to specify the grid for resampling.
>
>
  • The resampling task now allows alternative ways to specify the grid for resampling.
 

Data cubes

Display

Analysis

Herschel Science Archive

Changed:
<
<
  • HIPE will now store username and password in encrypted form outside of properties files (if so requested -- one can also choose not to store the username and password). In any case, it is recommended to remove HSA username and password from all properties files.
>
>
  • HIPE can now store username and password in encrypted form outside of property files. It is recommended to remove HSA username and password from all property files.
 

Calibration sources

Line: 125 to 125
 

Products and datasets

All products

Changed:
<
<
  • Removed the legacy diff methods from Product and Dataset interface. All functionality moved into ProductDiff class which is available as part of the utility toolbox (herschel.ia.toolbox.util)
>
>
  • Removed the legacy diff methods from Product and Dataset interfaces. All functionality moved to the ProductDiff class in the utility toolbox (herschel.ia.toolbox.util)
 

History

Changed:
<
<
A new task historyExtract is available to facilitate retrieving the value of a certain parameter from the product history. E.g. one can use the following command line to find out which calibration file was used in a given task of the PACS spectrometer pipeline:
>
>
A new task historyExtract is available to facilitate retrieving the value of a certain parameter from the product history. For instance, you can use the following command to find out which calibration file was used in a given task of the PACS spectrometer pipeline:
 
value = historyExtract(product=frames, task='convertSignal2StandardCap', parameter='effectiveCapacitance')
Changed:
<
<
The GUI of this task guides you through the available tasks and parameters: The option lists of the 'task' and 'parameter' parameters only show what is available in the history of the specified product:
>
>
The GUI of this task guides you through the available tasks and parameters: The option lists of the task and parameter parameters only show what is available in the history of the specified product:
  Screenshot-HIPE_-_historyExtract.png
Line: 143 to 143
 

Product Access Layer

Changed:
<
<
  • A "Cache" entry was added to the HIPE Preferences. Under this entry, a panel is shown with an overview of all existing caches on the system. One can also clear out these caches from this panel.
  • Product Browser: A simplified Observations tab was added, that allows user to list all observations on local disk, just by hitting the Run button. The data source selector has also been re-designed to be more flexible and intuitive.
>
>
  • A Cache entry has been added to the HIPE Preferences, with an overview of all existing caches on the system. You can clear these caches from this panel.
  • Product Browser: A simplified Observations tab has been added, that allows you to list all observations on local disk, just by pressing the Run button. The data source selector has also been redesigned to be more flexible and intuitive.
 

Tasks

Line: 152 to 152
 
    • All mandatory parameters must be filled in by users: removed default values from mandatory parameters, warnings issued (detection of programming mistakes)
    • Improved parameter checking and messages on bad data: Validators in primary inputs, they return false on error and log a warning (detection of programming mistakes)
    • All task GUIs and nodes should now show tooltips
Changed:
<
<
  • Added some new user-friendly parameter editors (see For Task developers only)
>
>
  • Added some new user-friendly parameter editors (see For Task developers only below)
 
  • Default task layout:
Changed:
<
<
    • Parameter name tooltips now show the parameter name (for long names)
    • big modifiers can occupy a full row
    • groups (tabs) can be navigated with the mouse wheel
    • Parameters can be reset individually via contextual menu on parameter names (vs resetting all parameters with Clear button)
    • Parameters can list the variables they can accept via contextual menu on parameter names (to know before hand if we have the proper data to use a parameter)
>
>
    • Parameter name tooltips now show the parameter name (for long names).
    • Big modifiers can occupy a full row.
    • You can move through groups (tabs) with the mouse wheel.
    • Parameters can be reset individually via contextual menu on parameter names (in addition to resetting all parameters with the Clear button)
    • Parameters can list the variables they accept via contextual menu on parameter names, so you can check if you have the proper data to use in a parameter.
 
  • For Task developers only:
Changed:
<
<
    • Task registration will issue warnings if either the task was already registered or the taskname was already used or both.
>
>
    • Task registration will issue warnings if the task was already registered or the taskname was already used.
 
    • Command generation: added converters so that arbitrary types can be represented with Jython strings (part of better visibility for defaults)
    • New shared Modifiers (parameter editors):
      • Added Editor for selecting a subset of a list of values (JListModifier, N values of M possible, M >= N)
      • Added a variant of JOptionModifier (Combo box) that returns some values but shows others (example: parameter uses integers but we show strings to users)
Changed:
<
<

Misc Tasks

  • OpenTask parameter names have changed capitalization to comply with lower camel case
>
>

Miscellaneous tasks

  • OpenTask parameter names have changed capitalisation to comply with naming conventions.
 

Quality control

Data input-output

Changed:
<
<
  • URM entries (documentation) improved
>
>
  • Improved entries in the User's Reference Manual.
 
  • Task importUFDirToPal renamed to importObservation
  • Task exportPalToUFDir renamed to exportObservation
Changed:
<
<
  • Task simpleAsciiTableReader extended to support all options of asciiTableReader via Advanced tab.
>
>
  • Task simpleAsciiTableReader extended to support all options of asciiTableReader via the Advanced tab.
 
  • Improved type guessing for table reading (affects (simple)AsciiTableReader): for floating-point data it will detect precision loss and maintain representation. If a floating-point number will lose precision as a float (32-bits) a double (64-bits) will be used instead: up to 17 significant digits but in these case it will use twice the memory.
Changed:
<
<
  • Improved documentation and information provided by getObservation
  • saveObservation now reports the progress in the status bar and can be interrupted/stopped.
  • Added contextual menu option to Variables View to save tables with asciiTableWriter. Also added a validator to the task that will restrict its applicability to TableDatasets that can be fit into an ASCII table.
>
>
  • Improved documentation and information provided by getObservation.
  • saveObservation now reports the progress in the status bar and can be interrupted.
  • Added contextual menu option to the Variables view to save tables with asciiTableWriter. Also added a validator to the task that will restrict its applicability to table datasets that can be fit into an ASCII table.
 

Virtual Observatory

Line: 231 to 231
 

HIFI Calibration

  • Updates to Calibration tree (HICAL-30)
Changed:
<
<
    • Spur table (7b)
    • FPU HK Thresholds (5a) (Prevents unnecessary Quality Flags)
    • Units added to columns (Prevents unnecessary Quality Flags)
    • Wbs Bad Pixel (saturated regions) (Causes necessary Quality Flags)
    • QC flags for bad LOU currents (Causes necessary Quality Flags)
    • Uplink auxiliary product parameter filtering (First steps in ensuring information about the way the observation was carried out is put in the meta data)
>
>
    • Spur table (7b).
    • FPU HK Thresholds (5a) (Prevents unnecessary Quality Flags).
    • Units added to columns (Prevents unnecessary Quality Flags).
    • Wbs Bad Pixel (saturated regions) (Causes necessary Quality Flags).
    • QC flags for bad LOU currents (Causes necessary Quality Flags).
    • Uplink auxiliary product parameter filtering (First steps in ensuring information about the way the observation was carried out is put in the metadata).
 

HIFI Pipeline

hifiPipeline task

Changed:
<
<
  • Can now select calibration version to reprocess data with (command line only)
>
>
  • Can now select calibration version to reprocess data with (command line only).
 
Changed:
<
<
  • By default the pipeline now runs without saving products. Previously it always wrote output to .hcss/lstore/pipeline-out, which in theory allowed users to access processed observations from that location. This was rarely used and could cause disk space problems.
>
>
  • By default the pipeline now runs without saving products. Previously it always wrote output to .hcss/lstore/pipeline-out, which in theory allowed you to access processed observations from that location. This was rarely used and could cause disk space problems.
 
Changed:
<
<
  • Improvements to tooltips in the Pipeline GUI
>
>
  • Improvements to tooltips in the Pipeline GUI.
 

Level 0 Pipeline

Changed:
<
<
  • DoUplink now uses the new uplink auxiliary product to set some observation parameters in the meta data, e.g., mapReadoutSep and mapLineStep
>
>
  • DoUplink now uses the new uplink auxiliary product to set some observation parameters in the metadata, such as mapReadoutSep and mapLineStep.
 
  • Suppression of unneeded Quality Flags: Quality check for mixer voltage now accounts for magnet tuning.

WBS Pipeline

HRS Pipeline

Level 1 Pipeline

Changed:
<
<
  • New (optional) pipeline step allowing smooth band pass to eliminate standing waves: DoFilterLoadsTask. This is an alternative calibration scheme for strong continuum sources.
>
>
  • New (optional) pipeline step allowing smooth band pass to eliminate standing waves: doFilterLoads. This is an alternative calibration scheme for strong continuum sources.
 
Changed:
<
<
  • Substantial refactoring of CheckFrequencyGrid to correct assumptions made about the sequence of on-, off- and hc-datasets. This primarily affected HEB band data and resulted in unnecessary Quality Flags being raised.
>
>
  • Substantial refactoring of CheckFrequencyGrid to correct assumptions made about the sequence of on-, off- and hc-datasets. This primarily affected HEB band data and resulted in unnecessary Quality Flags being raised.
 

Level 2 Pipeline

Changed:
<
<
  • Sideband gains coefficients added to meta data
>
>
  • Sideband gains coefficients added to metadata.
 

HIFI Products

  • Meta data calVersion is applied to all calibrated products and has format HIFI_CAL_version_number
Line: 272 to 272
 

HIFI Spectrum Toolbox

Spectrum Explorer (HIFI specifics)

Spectrum tasks (HIFI specifics)

Changed:
<
<
  • Introduction of FlagToolTask, which allows you go through every subband of a SpectrumDataset and interactively flag spurs, lines, bright lines, and data to ignore. Input data can be an Observation context, a HifiTimelineProduct, or a SpectrumDataset
>
>
  • Introduced the flagTool task, which allows you go through every subband of a spectrum dataset and interactively flag spurs, lines, bright lines, and data to ignore. Input data can be an Observation context, a HifiTimelineProduct, or a SpectrumDataset.
 
Changed:
<
<
  • PolarPair is now a task

>
>
  • PolarPair is now a task and can be applied to spectrum datasets.
 

HIFI Data Processing Tools

Standing Wave Removal

Changed:
<
<
  • avermask option in FitHifiFringe introduced.
    • Allows a line mask to be calculated using the average of all scans. Makes fringe removal from maps much more efficient.
>
>
  • Introduced avermask option in FitHifiFringe. It allows a line mask to be calculated using the average of all scans. Makes fringe removal from maps much more efficient.
 
Changed:
<
<
  • Variable result_sds should no longer created if FitHifiFringe is run from the GUI with obs_or_htp input
>
>
  • Variable result_sds should no longer be created if FitHifiFringe is run from the GUI with obs_or_htp input.
 
  • When an Observation Context is supplied standing waves are now automatically removed from both upper and lower sidebands.
Line: 294 to 291
 
  • The back-end(s) to be processed can now be selected by clicking in a list.
Changed:
<
<
  • FitBaseline 'divide' option now handles negative signals robustly.
>
>
  • FitBaseline divide option now handles negative signals robustly.
 

Deconvolution

Changed:
<
<
  • Some output from doDeconvolution is now written to the console as it runs - helpful to know HIPE has not crashed!
>
>
  • Some output from doDeconvolution is now written to the console as it runs - helpful to know HIPE has not frozen!
 
Changed:
<
<
  • doDeconvolution now picks up the applied sideband gains from the meta data and "undoes" their effect on the incoming DSB data. Then the user running the deconvolution can then apply gains on or gains off. The pipeline supplied gains can in this way offer an optional starting point for the gain iteration inside the deconvolution.
>
>
  • doDeconvolution now picks up the applied sideband gains from the meta data and "undoes" their effect on the incoming DSB data. Then you can apply gains on or gains off. The pipeline supplied gains can in this way offer an optional starting point for the gain iteration inside the deconvolution.
 
    • This only has practical application for band 2 data at present.

  • New redundancy product allows you to overplot redundancy on the SSB solution. Helpful to identify where the SSB solution may not be reliable.
Changed:
<
<
  • SSB spectrum now has y-axis labels "Antenna Temperature (K)", x-axis label changed to "Frequency (GHz)"
>
>
  • SSB spectrum now has y-axis labels Antenna Temperature (K), x-axis label changed to Frequency (GHz).
 
Changed:
<
<
  • When diagnostic mode is turned on, doDeconvolution now produces an interim product in addition to the pop-up plots.
>
>
  • When diagnostic mode is turned on, doDeconvolution now produces an interim product in addition to the pop-up plots.
 
Changed:
<
<
  • The min chi squared and the number of iterations are now included in the meta data of the SSB solution
>
>
  • The min chi squared and the number of iterations are now included in the meta data of the SSB solution.
 
Changed:
<
<
  • Bug fix: In doDeconvolution an algorithmic error in binning the input data has been corrected. The fix has stopped the occurrence of two problems: Some band-edge channels had fluxes set to zero and some very narrow lines had their peaks shifted by as much 1/2 MHz. Both errors are fixed.
>
>
  • Bug fix: In doDeconvolution an algorithmic error in binning the input data has been corrected. The fix has stopped the occurrence of two problems: Some band-edge channels had fluxes set to zero and some very narrow lines had their peaks shifted by as much 1/2 MHz.
 

Spectral Cube creation

Changed:
<
<
  • BeamSize is now taken from the calibration tree if "hpbw" is in the metadata
    • If the "hpbw" is not in the meta data, the beam size is calculated in the same way as in convertK2Jy using the rfreq in the meta data
>
>
  • BeamSize is now taken from the calibration tree if hpbw is in the metadata.
    • If hpbw is not in the metadata, the beam size is calculated in the same way as in convertK2Jy using the rfreq in the metadata.
 
Changed:
<
<
* Can rotate maps using new parameter, FlyAngle
>
>
* Can rotate maps using new parameter, FlyAngle.
 
  • GUI behaviour modified:
Changed:
<
<
    • Picks up information from meta data to provide better default input parameters
>
>
    • Picks up information from metadata to provide better default input parameters.
 
    • Changes to values in the GUI form now only go in one direction, e.g. changing PixelSize changes the Beam and Filter Parameters but not vice versa.

HiClass export tool

  • You can now create keywords when exporting to CLASS fits format.

Documentation

Changed:
<
<
  • HIFI User's Manual changed name to HIFI Data Reduction Guide, chapters rearranged into more logical order, addition of a changelog
    • Addition of introductory chapter, "HIFI Launch Pad"
    • Addition of a DBS Point Mode cookbook
    • Addition of chapter, "How to flag and remove flags from your data"
    • Addition of chapter, "Unit Conversion"
    • Updates to chapters on "Pipeline", "Deconvolution", "Creating Spectral Cubes", "What was done to my data" (clarifying units at each stage of the pipeline),
  • Updates the examples in the HIFI Pipeline Specification Document
  • HIFI User Reference Manual (URM) improvements
    • All HIFI tasks in URM now have a category and show up in the categorised view of commands
    • The categorised view of commands now is grouped into headers that should be meaningful to astronomers
    • Many URM entries have been improved so that decriptions are clear and examples are useful
>
>
  • The HIFI User's Manual changed name to HIFI Data Reduction Guide, chapters rearranged into more logical order, addition of a changelog.
    • Addition of introductory chapter, HIFI Launch Pad.
    • Addition of a DBS Point Mode cookbook.
    • Addition of chapter, How to flag and remove flags from your data.
    • Addition of chapter, Unit Conversion.
    • Updates to chapters on Pipeline, Deconvolution, Creating Spectral Cubes, What was done to my data (clarifying units at each stage of the pipeline).
  • Updates the examples in the HIFI Pipeline Specification Document.
  • HIFI User Reference Manual (URM) improvements.
    • All HIFI tasks in URM now have a category and show up in the categorised view of commands.
    • The categorised view of commands now is grouped into headers that should be meaningful to astronomers.
    • Many URM entries have been improved so that descriptions are clear and examples are useful.
 

PACS

Line: 345 to 342
 

Photometry

Changed:
<
<
  • a new task is used for the deglitching less memory intensive MapDeglitch instead of IIndLevelDeglitch
  • the 2nd level glitch mask is now stored in level 1.
  • new task filterOnScanSpeed now used to select frames at constant speed instead of the BBID selection
  • new task photMaskCrosstalk, masking the column 0 of all matrices now used in the pipeline too. This is a bit of brute force to avoid negative or positive cross-talk artifact in teh level 2 maps, at the expense of getting rid of 1/16th of the data.
>
>
  • The less memory intensive task mapDeglitch is now used for deglitching instead of IIndLevelDeglitch
  • The second level glitch mask is now stored in level 1.
  • The new task filterOnScanSpeed now used to select frames at constant speed instead of the BBID selection.
  • The new task photMaskCrosstalk, masking the column 0 of all matrices, is now used in the pipeline too. This is a bit of brute force to avoid negative or positive cross-talk artifact in the level 2 maps, at the expense of getting rid of 1/16th of the data.
 

Spectroscopy

Changed:
<
<
  • interactive reduction scripts for all observing modes are now available
  • flatfielding is available in the interactive pipeline scripts
  • error propagation (and error bars) is implemented in the PACS rebinned cubes (STDDEV dataset)
  • spg pipeline: the wavelength switching pipeline is merged with the unchopped pipeline and removed
  • improved documentation and user friendliness of the pipeline tasks within hipe
  • The spectrometer level-2 tasks now produce the specialized sliced products SlicedPacsCube and SlicedPacsRebinnedCube, which offer the same functionality as the SlicedFrames product.
>
>
  • Interactive reduction scripts for all observing modes are now available.
  • Flatfielding is available in the interactive pipeline scripts.
  • Error propagation (and error bars) is implemented in the PACS rebinned cubes (STDDEV dataset).
  • SPG pipeline: the wavelength switching pipeline is merged with the unchopped pipeline and removed.
  • Improved documentation and user friendliness of the pipeline tasks within HIPE.
  • The spectrometer level-2 tasks now produce the specialized sliced products SlicedPacsCube and SlicedPacsRebinnedCube, which offer the same functionality as the SlicedFrames product.
 

Calibration

Changed:
<
<
  • The calibration sets that are installed on your local system can now be inspected with a 'Calibration Sets Viewer'. The viewer is located in the menu Window -> Show View -> Workbench -> Calibration Sets. Select a version from the available buttons, the you can inspect the release note for that version or the product list with the versions of the individual calibration products.
>
>
  • The calibration sets that are installed on your local system can now be inspected with a 'Calibration Sets Viewer'. The viewer is located in Window --> Show View --> Workbench --> Calibration Sets. Select a version from the available buttons, then you can inspect the release note for that version or the product list with the versions of the individual calibration products.
 

Photometry

Changed:
<
<
  • A new calibration file is available for non linearity correction required by the task photNonLinearityCorrection.
>
>
  • A new calibration file is available for non linearity correction required by the task photNonLinearityCorrection.
 

Spectroscopy

Changed:
<
<
  • major improvements of the flux calibration:
    • implementation of the background normalization method for chop/nod observations
    • it is now based on observations of astronomical standards and the calibration block attached to each observation. The pipeline task specDiffCs computes the detector response of the current observation.
    • implementation of a correction of transients for unchopped observations
>
>
  • Major improvements of the flux calibration:
    • Implementation of the background normalization method for chop/nod observations.
    • It is now based on observations of astronomical standards and the calibration block attached to each observation. The pipeline task specDiffCs computes the detector response of the current observation.
    • Implementation of a correction of transients for unchopped observations.
 
Changed:
<
<
  • the spectral (pixel) dependency of the spatial calibration is now available for each pixel
>
>
  • The spectral (pixel) dependency of the spatial calibration is now available for each pixel.
 

Analysis tools

Changed:
<
<
  • The Preferences now contain a Logging panel under the general tab which allows the user/developer to increase or decrease the number of logging messages send to the
terminal.
>
>
  • The Preferences windo now contain a Logging panel under the general tab which allows you to increase or decrease the number of logging messages sent to the terminal.
 
  • The calibration file version is now also shown in the product history.

Photometry

Changed:
<
<
MapMonitor got an improved interface and allows to flip north up-or downwards.

PhotProject got a companion task MiProject (MapIndex-Project). If you have a full MapIndex, you may use MiProject instead of PhotProject. It needs frames and the MapIndex as parameters and speeds up projection time by a factor of 20 (30, 40?). Whatever the number is: its much faster than PhotProject. Use as:

 map = miProject(frames, mapindex)  

A new Task has been developed. It deglitches without MapIndex and runs with significantly less memory abuse than the IIndLevelDeglitchTask. The name and package is

 herschel.pacs.spg.phot.deglitching.map.MapDeglitchTask 
. The hipe alias is mapDeglitch. It is documented as javadoc, URM and on the online deglitching documentation page:
http://herschel.esac.esa.int/twiki/bin/view/Public/SecondLevelDeglitching#MapDeglitchTask

A new Task has been implemented which removes crosstalk effects by masking (CrossTalkMask ) the first rows of detector arrays :

 frames = photMaskCrosstalk(frames) 

The HIPE Menu Pipeline -> PACS -> has been re organized and ipipe scripts added.

Automatic pipeline scripts save the Products at the end in the Observation Context also in IA sessions. That mean the scripts can be executed in a sequence without interaction.

Pacs Products (for example Frames, Ramps, PacsCube) got a method to rename a Mask :

 renameMask(String oldname, String newname) 

up to now, a mask had to be removed and created new, if the name had to be changed.

PhotProject, MiProject, IIndLevelDeglitching and some other tasks report their progress to the hipe progressbar instead of the terminal output.

The effects of 2-bit rounding on second level deglitching have been reduced in the pipeline for blue parallel maps.

The Pacs MaskViewer has been relocated to a new package:

 herschel.pacs.gui.maskviewer.MaskViewer 
. If you manually imported the old location (from herschel.pacs.signal import MaskViewer), the import will fail. Please just remove the import. Everything should work fine without further action. If there should be a problem nevertheless, please import the new location:
 from herschel.pacs.gui.maskviewer import MaskViewer  

  • A new tasks MaskFromCatalogue is available, very convenient to mask a list of point sources for high-pass filtering in deep fields.
>
>
  • MapMonitor got an improved interface and allows to flip north up-or downwards.
  • The PhotProject task got a companion task miProject (MapIndex-Project). If you have a full MapIndex, you may use miProject instead of photProject. It needs frames and the MapIndex as parameters and speeds up projection time by a factor of at least 20. Use as follows: map = miProject(frames, mapindex)
  • A new task has been developed. It deglitches without MapIndex and runs with significantly less memory abuse than the IIndLevelDeglitch task. The name and package is herschel.pacs.spg.phot.deglitching.map.MapDeglitchTask. The HIPE alias is mapDeglitch. It is documented as javadoc, URM and on the online deglitching documentation page.
  • A new task has been implemented which removes crosstalk effects by masking (CrossTalkMask) the first rows of detector arrays: frames = photMaskCrosstalk(frames)
  • The HIPE menu Pipeline --> PACS has been reorganised and ipipe scripts added.
  • Automatic pipeline scripts save the products at the end in the Observation Context also in IA sessions. That mean the scripts can be executed in a sequence without interaction.
  • Pacs products (for example Frames, Ramps, PacsCube) got a method to rename a Mask: renameMask(String oldname, String newname). Up to now, a mask had to be removed and recreated, if the name had to be changed.
  • photProject, miProject, IIndLevelDeglitching and some other tasks now report their progress in the HIPE progress bar instead of the terminal output.
  • The effects of 2-bit rounding on second level deglitching have been reduced in the pipeline for blue parallel maps.
  • The Pacs MaskViewer has been relocated to a new package: herschel.pacs.gui.maskviewer.MaskViewer. If you manually imported the old location (from herschel.pacs.signal import MaskViewer), the import will fail. Please remove the import, since everything should work automatically. If nevertheless there is a problem, please import the new location: from herschel.pacs.gui.maskviewer import MaskViewer
  • A new task maskFromCatalogue is available, very convenient to mask a list of point sources for high-pass filtering in deep fields.
 

Spectroscopy

Changed:
<
<
  • There is a new GUI for viewing the footprint of the spectrometer upon an image with WCS: the tool pacsSpectralFootprint. It can be accessed through the applicable tasks list in the Tasks View.
>
>
  • There is a new GUI for viewing the footprint of the spectrometer upon an image with WCS: the tool pacsSpectralFootprint. It can be accessed through the Applicable tasks list in the Tasks view.
 
  • The Spectrum Explorer GUI has a new panel for editing PACS masks.
Changed:
<
<
  • implementation of user friendly viewer for the PACS specific part of the ObservationContext:
    • Each spectrometer slice now shows after the slice number a short description of its contents
>
>
  • Implementation of user friendly viewer for the PACS specific part of the ObservationContext:
    • Each spectrometer slice now shows after the slice number a short description of its contents.
 
    • The tooltip of the slice shows the complete list of slice info keywords.

Product changes

Photometry

Changed:
<
<
  • a new level 2.5 product is available combining photProject scan and cross-scan maps.
>
>
  • A new level 2.5 product is available combining photProject scan and cross-scan maps.
 

Spectroscopy

Changed:
<
<
  • Browse products and browse image products are now available in spg
>
>
  • Browse products and browse image products are now available in SPG.
 
  • Red and blue camera spectra can now be combined in a single context in the level-2 processing, therefore, the lineId has changed for the red camera: 100 is added!

Documentation

Changed:
<
<
  • PACS Data Reduction Guide. Chp 1 is now a collection of Launch Pads: on getting Data into HIPE, doing your PACS Spectroscopy and doing your PACS Photometry. These are quick-start guides to working with your PACS observations. Chp2 tells you how to load your data, your pipeline script, and your calibration tree into HIPE, for photometry and spectroscopy both.
  • Spectroscopy: the pipeline chapters are being re-organised to follow the new pipeline scripts, and so for this 8.0 RC4 version of the PDRG only the LineScan Chop-Nod pipeline is described.
  • Photometry: the pipeline chapters are being re-organised to follow the new pipeline scripts, and so for this 8.0 RC4 version of the PDRG only the ScanMap pipeline scripts are described.
>
>
  • PACS Data Reduction Guide. Chapter 1 is now a collection of Launch Pads: on getting Data into HIPE, doing your PACS Spectroscopy and doing your PACS Photometry. These are quick-start guides to work with your PACS observations. Chapter 2 tells you how to load your data, your pipeline script, and your calibration tree into HIPE, for photometry and spectroscopy.
  • Spectroscopy: the pipeline chapters are being re-organised to follow the new pipeline scripts, and so for this 8.0 version of the PDRG only the LineScan Chop-Nod pipeline is described.
  • Photometry: the pipeline chapters are being re-organised to follow the new pipeline scripts, and so for this 8.0 version of the PDRG only the ScanMap pipeline scripts are described.
 

SPIRE

 
This site is powered by the TWiki collaboration platform Powered by Perl