Difference: HipeWhatsNew9x (1 vs. 75)

Revision 752012-11-28 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 12 to 12
  This document lists the changes in HIPE version 9.0 with respect to the 8.x series. Additional pages list changes between minor versions of the 9.x series.

Changed:
<
<
<--   * Changes between HIPE 9.1 and 9.2 -->
>
>
  See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.
Line: 491 to 491
 

META FILEATTACHMENT attr="" autoattached="1" comment="HSA download times" date="1333576243" name="MyHSA_downloadTimes.png" path="MyHSA_downloadTimes.png" size="51431" user="Main.CarolynMcCoey" version="1"
Deleted:
<
<
META FILEATTACHMENT attr="" autoattached="1" comment="HSA disk usage" date="1333576193" name="MyHSA_diskUsage.png" path="MyHSA_diskUsage.png" size="85841" user="Main.CarolynMcCoey" version="1"
 
META FILEATTACHMENT attr="h" autoattached="1" comment="Script for Google Analytics" date="1350652571" name="GoogleAnalytics.txt" path="GoogleAnalytics.txt" size="549" user="Main.DavideRizzo" version="1"
Added:
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="HSA disk usage" date="1333576193" name="MyHSA_diskUsage.png" path="MyHSA_diskUsage.png" size="85841" user="Main.CarolynMcCoey" version="1"

Revision 742012-11-28 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 12 to 12
  This document lists the changes in HIPE version 9.0 with respect to the 8.x series. Additional pages list changes between minor versions of the 9.x series.

Changed:
<
<
>
>
<--   * Changes between HIPE 9.1 and 9.2 -->
  See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.

Revision 732012-11-23 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 12 to 12
  This document lists the changes in HIPE version 9.0 with respect to the 8.x series. Additional pages list changes between minor versions of the 9.x series.

Added:
>
>
  See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.

Revision 722012-10-30 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 7 to 7
 
PDF Version Click here
Added:
>
>
Help Interested in what's new for other versions of HIPE? See this page for links to all the What's New documents from HIPE 4.0 onwards.
  This document lists the changes in HIPE version 9.0 with respect to the 8.x series. Additional pages list changes between minor versions of the 9.x series.

Revision 712012-10-29 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 212 to 212
 

HIFI Calibration

Changed:
<
<
  • New calibration database HIFI_CAL_9_0, containing the following updates
    • Side-band ratio correction over the whole of bands 5a and 5b
>
>
  • New calibration database HIFI_CAL_10_0, containing the following updates
    • Introduction of a new HifiUplink? product to back-fill most the observational parameters as of HSpot optimisation back into the Uplink product, and used at particular stages of the pipeline (e.g. cube making)
 
    • Update of a number of engineering threshold for more accurate quality flagging related to hardware housekeeping
Changed:
<
<
    • introduction of a list of a priori known corrupted data-frames for dedicated flagging in level0 data
>
>
    • Update of the list of a priori known corrupted data-frames for dedicated flagging in level0 data
 

HIFI Pipeline

Revision 702012-10-29 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 7 to 7
 
PDF Version Click here
Added:
>
>
This document lists the changes in HIPE version 9.0 with respect to the 8.x series. Additional pages list changes between minor versions of the 9.x series.

 See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.

Revision 692012-10-19 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
Added:
>
>
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->
 

What's new in HIPE 9

PDF Version Click here
Line: 482 to 485
 
META FILEATTACHMENT attr="" autoattached="1" comment="HSA download times" date="1333576243" name="MyHSA_downloadTimes.png" path="MyHSA_downloadTimes.png" size="51431" user="Main.CarolynMcCoey" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="HSA disk usage" date="1333576193" name="MyHSA_diskUsage.png" path="MyHSA_diskUsage.png" size="85841" user="Main.CarolynMcCoey" version="1"
Added:
>
>
META FILEATTACHMENT attr="h" autoattached="1" comment="Script for Google Analytics" date="1350652571" name="GoogleAnalytics.txt" path="GoogleAnalytics.txt" size="549" user="Main.DavideRizzo" version="1"

Revision 682012-10-04 - LucaConversi

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 9

Line: 36 to 36
  SPIRE
Added:
>
>
  • Destriper. Now this is the default task for baseline removal in both SPG and User pipelines.
 
  • Faster processing. Improved processing speed using multi-threading.
  • Cooler burps. Improved task for correction of cooler burps.
  • Photometer pipeline processing. Improved error maps and baseline correction.

Revision 672012-10-04 - BrunoMerin

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 9

Revision 662012-09-12 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 9

Line: 10 to 10
 

Highlights

Deleted:
<
<
See also the New in HIPE 9 series on Twitter.
 HIPE

  • New cube toolbox. The nex toolbox replaces the Cube Spectrum Analysis Toolbox (CSAT). Common look and feel with the Spectrum Explorer and improved algorithms.

Revision 652012-09-03 - RolandVavrek

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 9

Line: 332 to 332
 
  • test versions of pointing correction tasks
  • improved error propagation
  • correction of SSO pointing task correctRaDec4Sso introduced in spg pipeline
Added:
>
>
  • level 2.5 product generation for unchopped range spectroscopy multi-observation mode (off-subtracted spectrum)
 

Calibration

Photometry

Revision 642012-08-08 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 9

Line: 176 to 176
 

Data input-output

  • Deprecation of duplicated functionalities: poolDataWriter, localStoreWriter, poolDataReader. Use getObservation and saveProduct.
  • Deprecation of MyHSA prototype duplicated functionalities: loadObs and downObs. Use getObservation
Changed:
<
<
  • getObservation promoted to a Task (currently getObs).
>
>
  • getObservation is now a proper task.
 
  • simpleAsciiTableReader deprecated, use asciiTableReader.

Documentation

Line: 479 to 479
 

Deleted:
<
<
META FILEATTACHMENT attr="" autoattached="1" comment="HSA disk usage" date="1333576193" name="MyHSA_diskUsage.png" path="MyHSA_diskUsage.png" size="85841" user="Main.CarolynMcCoey" version="1"
 
META FILEATTACHMENT attr="" autoattached="1" comment="HSA download times" date="1333576243" name="MyHSA_downloadTimes.png" path="MyHSA_downloadTimes.png" size="51431" user="Main.CarolynMcCoey" version="1"
Added:
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="HSA disk usage" date="1333576193" name="MyHSA_diskUsage.png" path="MyHSA_diskUsage.png" size="85841" user="Main.CarolynMcCoey" version="1"

Revision 632012-08-02 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 9

Line: 10 to 10
 

Highlights

Added:
>
>
See also the New in HIPE 9 series on Twitter.
 HIPE

  • New cube toolbox. The nex toolbox replaces the Cube Spectrum Analysis Toolbox (CSAT). Common look and feel with the Spectrum Explorer and improved algorithms.
Line: 477 to 479
 

Deleted:
<
<
META FILEATTACHMENT attr="" autoattached="1" comment="HSA download times" date="1333576243" name="MyHSA_downloadTimes.png" path="MyHSA_downloadTimes.png" size="51431" user="Main.CarolynMcCoey" version="1"
 
META FILEATTACHMENT attr="" autoattached="1" comment="HSA disk usage" date="1333576193" name="MyHSA_diskUsage.png" path="MyHSA_diskUsage.png" size="85841" user="Main.CarolynMcCoey" version="1"
Added:
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="HSA download times" date="1333576243" name="MyHSA_downloadTimes.png" path="MyHSA_downloadTimes.png" size="51431" user="Main.CarolynMcCoey" version="1"

Revision 622012-07-25 - EdwardPolehampton

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 9

Line: 365 to 365
 

Calibration Products

Changed:
<
<
These changes are in the SPIRE_CAL_9_0 calibration tree.
>
>
These changes are in the SPIRE_CAL_9_1 calibration tree.
  Photometer:
  • A column containing frequency in GHz was added to the Photometer filter profile product, SCalPhotRsrf. This now matches Spectrometer data which are in frequency and not wavenumber.

Revision 612012-07-25 - RolandVavrek

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 9

Line: 30 to 30
 
  • Photometer. Addressing additional specific cases in new interactive pipeline scripts.
  • Spectrometer pipeline processing. Improved pipeline processing for better reliability of broadband features.
  • Spectral flatfielding. Improved spectral flatfielding for wide spectral ranges (e.g. SED).
Changed:
<
<
  • Flux losses. Correction for flux losses due to pointing jitter.
>
>
  • Flux losses. Correction for flux losses due to pointing jitter and pointing offset
  • Flux extraction for point-sources. Use of central 3x3 spaxel fluxes for a more reliable beam correction
 
  • Convolution kernels. For photometer and spectrometer maps, aiding the interpretation of extended and point sources.

SPIRE

Revision 602012-07-24 - BrunoAltieri

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 9

Line: 336 to 336
 
    • flat-field (flatField, version 4)
    • responsivity (version 7): the change in photometry for extended sources is about +3% in the red channel (160 microns), + 1.5% at 100 microns and none at 70 microns. For point-sources the change of photometry is minimal, about 2% in at 160 microns and less than 1% for usual photometry apertures at 70 and 100 microns.
    • aperture correction (apertureCorrection v3)
Changed:
<
<
    • *non-linearity correction ( nonLinearCoef v2)
>
>
    • non-linearity correction ( nonLinearCoef v2)
 

Spectroscopy

Revision 592012-07-20 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 9

Line: 21 to 21
  HIFI
Changed:
<
<
  • HIFI level 2.5 product introduced: for Spectral Scans it holds the deconvolved single sideband soluions; for Maps it holds the cubes; for all cases it holds a copy of the level 2 HTP.
  • Addition of an interactive pipeline between level 2 and 2.5, which allows the insertion of tasks like fitHifiFringe and fitBaseline into the pipeline between level 2 and 2.5.
  • Browse products for spectral scans and maps added
>
>
  • HIFI level 2.5 product introduced. For Spectral Scans it holds the deconvolved single sideband solutions; for Maps it holds the cubes; for all cases it holds a copy of the level 2 HTP.
  • New interactive pipeline between level 2 and 2.5. This allows the insertion of tasks like fitHifiFringe and fitBaseline into the pipeline between level 2 and 2.5.
  • Browse products added. For spectral scans and maps.
  PACS
Changed:
<
<
  • Photometer: Addressing additional specific cases in new interactive pipeline scripts
  • Spectrometer
    • Improved pipeline processing for better reliability of broadband features
    • Improved spectral flatfielding for wide spectral ranges (e.g. SED)
    • Correction for flux losses due to pointing jitter
  • Convolution kernels for photometer and spectrometer maps, aiding the interpretation of extended and point sources
  • Reduction in memory consumption (TBS) when pipeline processing
>
>
  • Photometer. Addressing additional specific cases in new interactive pipeline scripts.
  • Spectrometer pipeline processing. Improved pipeline processing for better reliability of broadband features.
  • Spectral flatfielding. Improved spectral flatfielding for wide spectral ranges (e.g. SED).
  • Flux losses. Correction for flux losses due to pointing jitter.
  • Convolution kernels. For photometer and spectrometer maps, aiding the interpretation of extended and point sources.
  SPIRE
Changed:
<
<
  • common (both Photometer and Spectrometer)
    • improved processing speed using multi-threading
    • improved task for correction of "cooler burps"
    • PcalTask can now remove noise due to pointing (prevalent in really bright sources) and process Brigh mode Pcal flashes
  • Photometer
    • Pipeline Processing
      • improved error maps (possibly as an option)
      • improved baseline correction using "destriper“
    • Interactive processing
      • source timeline fitter task
      • improvements to bolometer finder tool
      • selection of turnaround data in map creation
  • Spectrometer
    • Pipeline Processing: smaller size for level 1 products
    • Improved calibration and data processing
      • updated point source calibration (improved Uranus model)
      • updated bright mode calibration (better PCAL module)
      • improved calibration for Low Resolution observations (hopefully improved calibration, but at minimum, updated advice)
      • more straightforward calling sequence for phase-correction task
      • corrected spectral averaging - more flexible usage
    • Interactive processing
      • drag and drop from SDS Explorer detector layout to variables tab
      • new and updated useful scripts for background subtraction and spectral noise
>
>
  • Faster processing. Improved processing speed using multi-threading.
  • Cooler burps. Improved task for correction of cooler burps.
  • Photometer pipeline processing. Improved error maps and baseline correction.
  • Spectrometer calibration. Improved point source and bright mode calibration. Better calibration for low resolution observations.
  • Spectrometer scripts. New and updated useful scripts for background subtraction and spectral noise.
 

Java version

Revision 582012-07-17 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 9

Line: 12 to 12
  HIPE
Changed:
<
<
  • New cube toolbox, which replaces the Cube Spectrum Analysis Toolbox (CSAT)
    • Common look and feel with the Spectrum Explorer
    • Improved algorithms
  • Improvements to Spectrum Explorer to make it more stable, faster and easier to customise plots
  • Easing the data access via MyHSA
  • Improvement of handling of ASCII tables
  • Mechanism to notify users on removed names (tasks, functions...), why they were removed, and their replacement
  • Speed-up of HIPE start-up
  • Human-made index added to three core manuals (HIPE Owner's Guide, Data Analysis Guide and Scripting and Data Mining Guide)
  • You can leave comments at the bottom of any page in the HIPE Help System
  • Support to OSX Lion
    <--   
    
  • Easy access to calibration data, including version and updates (TBS)-->
  • Improvements of search capability of documentation system
    <--
    
  • Publication ready plots (TBS)-->
>
>
  • New cube toolbox. The nex toolbox replaces the Cube Spectrum Analysis Toolbox (CSAT). Common look and feel with the Spectrum Explorer and improved algorithms.
  • Easier data access. Handling Herschel data is now done via the new MyHSA tool and the improved getObservation task.
  • Human-made index. Added to three core manuals: HIPE Owner's Guide, Data Analysis Guide and Scripting and Data Mining Guide.
  • Comments on documentation. You can now leave comments at the bottom of any page in the HIPE Help System.
  • Support to OSX Lion. HIPE is now officially supported on version 10.7 of Mac OS X.
 

HIFI

Line: 506 to 497
 

Deleted:
<
<
META FILEATTACHMENT attr="" autoattached="1" comment="HSA disk usage" date="1333576193" name="MyHSA_diskUsage.png" path="MyHSA_diskUsage.png" size="85841" user="Main.CarolynMcCoey" version="1"
 
META FILEATTACHMENT attr="" autoattached="1" comment="HSA download times" date="1333576243" name="MyHSA_downloadTimes.png" path="MyHSA_downloadTimes.png" size="51431" user="Main.CarolynMcCoey" version="1"
Added:
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="HSA disk usage" date="1333576193" name="MyHSA_diskUsage.png" path="MyHSA_diskUsage.png" size="85841" user="Main.CarolynMcCoey" version="1"

Revision 572012-07-16 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 9

Line: 36 to 36
  PACS
Changed:
<
<
  • Photometer: Addressing further specific cases in IPipe scripts
>
>
  • Photometer: Addressing additional specific cases in new interactive pipeline scripts
 
  • Spectrometer
Changed:
<
<
    • Broadband reliability
    • Improved spectral pixel flatfield broad ranges (SED)
    • Correction Flux loss due to pointing
  • Convolution kernels for photometer and spectrometer maps aiding the interpretation of extended and point sources
  • Reduction in memory consumption (TBS)
>
>
    • Improved pipeline processing for better reliability of broadband features
    • Improved spectral flatfielding for wide spectral ranges (e.g. SED)
    • Correction for flux losses due to pointing jitter
  • Convolution kernels for photometer and spectrometer maps, aiding the interpretation of extended and point sources
  • Reduction in memory consumption (TBS) when pipeline processing
  SPIRE
Line: 506 to 506
 

Deleted:
<
<
META FILEATTACHMENT attr="" autoattached="1" comment="HSA download times" date="1333576243" name="MyHSA_downloadTimes.png" path="MyHSA_downloadTimes.png" size="51431" user="Main.CarolynMcCoey" version="1"
 
META FILEATTACHMENT attr="" autoattached="1" comment="HSA disk usage" date="1333576193" name="MyHSA_diskUsage.png" path="MyHSA_diskUsage.png" size="85841" user="Main.CarolynMcCoey" version="1"
Added:
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="HSA download times" date="1333576243" name="MyHSA_downloadTimes.png" path="MyHSA_downloadTimes.png" size="51431" user="Main.CarolynMcCoey" version="1"

Revision 562012-07-16 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's new in HIPE 9

Line: 19 to 19
 
  • Easing the data access via MyHSA
  • Improvement of handling of ASCII tables
  • Mechanism to notify users on removed names (tasks, functions...), why they were removed, and their replacement
Deleted:
<
<
  • Smaller memory footprint for HIPE editor
 
  • Speed-up of HIPE start-up
Deleted:
<
<
  • Progress bar informs on progress of for-loops
 
  • Human-made index added to three core manuals (HIPE Owner's Guide, Data Analysis Guide and Scripting and Data Mining Guide)
  • You can leave comments at the bottom of any page in the HIPE Help System
  • Support to OSX Lion

    Revision 552012-07-12 - CarolynMcCoey

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 12 to 12
      HIPE
    Changed:
    <
    <
    • Common look and feel for the spectrum explorer and cube analysis toolbox
    • Easing the data access via MyHSA
    >
    >
    • New cube toolbox, which replaces the Cube Spectrum Analysis Toolbox (CSAT)
      • Common look and feel with the Spectrum Explorer
      • Improved algorithms
    • Improvements to Spectrum Explorer to make it more stable, faster and easier to customise plots
    • Easing the data access via MyHSA
     
    • Improvement of handling of ASCII tables
    • Mechanism to notify users on removed names (tasks, functions...), why they were removed, and their replacement
    • Smaller memory footprint for HIPE editor
    Line: 21 to 24
     
    • Progress bar informs on progress of for-loops
    • Human-made index added to three core manuals (HIPE Owner's Guide, Data Analysis Guide and Scripting and Data Mining Guide)
    • You can leave comments at the bottom of any page in the HIPE Help System
    Changed:
    <
    <
    • Support to OSX Lion
    • Easy access to calibration data, including version and updates (TBS)
    • Improvements of search capability of documentation system
    • Publication ready plots (TBS)
    >
    >
    • Support to OSX Lion
      <--   
      
    • Easy access to calibration data, including version and updates (TBS)-->
    • Improvements of search capability of documentation system
      <--
      
    • Publication ready plots (TBS)-->
     

    HIFI

    Line: 156 to 159
     

    Spectra

    Display

    • Major updates to Spectrum Explorer (SE) to improve performance and provide more direct control over plot appearance.
    Changed:
    <
    <
    • The Cube Spectrum Analysis Toolbox (CSAT) has been integrated into the Spectrum Explorer:
      • The CSAT tasks can now be found under a icon in the SE toolbox.
      • All CSAT tasks use now the standard HIPE Task GUI interface, with some additions to receive spectral and spatial selections from the SE. Users can e.g. either fill in a spectral range, or select the range in the spectrum plot.
    >
    >
    • The Cube Spectrum Analysis Toolbox (CSAT) has been integrated into the Spectrum Explorer and is now called the Cube Toolbox:
      • The cube toolbox tasks can now be found under a icon in the SE toolbox.
      • All cube toolbox tasks use the standard HIPE Task GUI interface, with some additions to receive spectral and spatial selections from the SE. You can e.g. either fill in a spectral range, or select the range in the spectrum plot.
     
      • Each task has it own specialized selection tab, so that the original spectra selection do not disappear when one opens a task.

    Analysis

    • The Smooth task is corrected for a bug in the ZEROS option.
      • As a consequence of this fix, which is more mathematically correct, it is found that smoothed spectra for SPIRE are quite changed. It is recommended instead to use the REPEAT option.
    • Introduction of a new "Accumulate" task allowing to automatically combine data taken on different frequency grids, with or without frequency overlap.
    Added:
    >
    >
    • Improvements to the weights/error handling of the tasks in the Spectral Toolbox so that errors are now propagated correctly for the simple arithmetics tasks.
     

    Herschel data access and download

    • Access to data in the Herschel Science Archive is now handled via the MyHSA utility, which dramatically improves performance and allows working with partial observations.

    Revision 542012-07-09 - BrunoAltieri

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 362 to 362
     

    Photometry

    • new calibration files for :
      • flat-field (flatField, version 4)
    Changed:
    <
    <
      • responsivity (version 7): the change in photometry for extended sources is about +3% in the red channel (160 microns), + 1.5% at 100 microns and none at 70 microns. For point-sources the change of photometry is minimal, typically less than 1% for usual photometry apertures.
    >
    >
      • responsivity (version 7): the change in photometry for extended sources is about +3% in the red channel (160 microns), + 1.5% at 100 microns and none at 70 microns. For point-sources the change of photometry is minimal, about 2% in at 160 microns and less than 1% for usual photometry apertures at 70 and 100 microns.
     
      • aperture correction (apertureCorrection v3)
      • *non-linearity correction ( nonLinearCoef v2)

    Revision 532012-07-09 - DavideRizzo

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 166 to 166
     
      • As a consequence of this fix, which is more mathematically correct, it is found that smoothed spectra for SPIRE are quite changed. It is recommended instead to use the REPEAT option.
    • Introduction of a new "Accumulate" task allowing to automatically combine data taken on different frequency grids, with or without frequency overlap.
    Changed:
    <
    <

    Herschel Science Archive

    • Improvements in download speed and memory requirements for data downloaded from HSA using a behind-the-scenes utility called MyHSA
      • Testing with 5 HIFI, 9 PACS and 10 SPIRE representative observations of all major observing modes simultaneously with getObservation/saveObservation and with getObs/MyHSA with the same network connection on two different machines show 20 times faster download speed and that 66% less hard disk space is needed to hold the same data:
    >
    >

    Herschel data access and download

    • Access to data in the Herschel Science Archive is now handled via the MyHSA utility, which dramatically improves performance and allows working with partial observations.
    • The old Import HSA data to HIPE task is no longer needed. MyHSA is mainly used through the improved getObservation task.
    • For more details see Chapter 1 of the Data Analysis Guide.
    • Results of testing performance improvements with 5 HIFI, 9 PACS and 10 SPIRE observations of all major observing modes. Tests were made with the old getObservation/saveObservation and with the new getObservation/MyHSA, with the same network connection on two different machines. Results show that downloads are 20 times faster and that 66% less hard disk space is needed to store the same data:
      MyHSA_downloadTimes.png MyHSA_diskUsage.png
    Added:
    >
    >

    On-demand data reprocessing

    • All data levels are now regenerated (from level0 to level2).
    • All observation files are delivered, including auxiliary, calibration and level0 products.
    • The zip file with the results contains now an HSA hierarchical directory structure instead of a local pool structure.
    • For more details see Section 1.15 of the Data Analysis Guide.
     

    Product Access Layer

    When storing pools on shared NFS-mounted disks, HIPE now uses a more robust mechanism to prevent corruption. The default locking mechanism for Local Pools was changed to a mechanism that works more reliably on remote disks.

    Revision 522012-07-09 - BrunoAltieri

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 352 to 352
     

    Calibration

    Photometry

    Changed:
    <
    <
    • new calibration products for :
    >
    >
    • new calibration files for :
     
      • flat-field (flatField, version 4)
      • responsivity (version 7): the change in photometry for extended sources is about +3% in the red channel (160 microns), + 1.5% at 100 microns and none at 70 microns. For point-sources the change of photometry is minimal, typically less than 1% for usual photometry apertures.
      • aperture correction (apertureCorrection v3)

    Revision 512012-07-08 - BrunoAltieri

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 353 to 353
     

    Calibration

    Photometry

    • new calibration products for :
    Changed:
    <
    <
      • the flat-field (flatField, v4)
      • responsivity change of ~2% only at 100 and 160 microns (responsivity v7)
    >
    >
      • flat-field (flatField, version 4)
      • responsivity (version 7): the change in photometry for extended sources is about +3% in the red channel (160 microns), + 1.5% at 100 microns and none at 70 microns. For point-sources the change of photometry is minimal, typically less than 1% for usual photometry apertures.
     
      • aperture correction (apertureCorrection v3)
      • *non-linearity correction ( nonLinearCoef v2)

    Revision 502012-07-04 - DavideRizzo

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 19 to 19
     
    • Smaller memory footprint for HIPE editor
    • Speed-up of HIPE start-up
    • Progress bar informs on progress of for-loops
    Changed:
    <
    <
    • Human-made indexes added to two core manuals (HIPE Owner's Guide and Scripting and Data Mining Guide)
    >
    >
    • Human-made index added to three core manuals (HIPE Owner's Guide, Data Analysis Guide and Scripting and Data Mining Guide)
     
    • You can leave comments at the bottom of any page in the HIPE Help System
    • Support to OSX Lion
    • Easy access to calibration data, including version and updates (TBS)
    Line: 70 to 70
     
        • new and updated useful scripts for background subtraction and spectral noise
    Added:
    >
    >

    Java version

     
    Added:
    >
    >
    • HIPE 9.0 was built and tested against the Oracle Java Runtime Environment (JRE) version 1.6.0_33.
    • User releases for Windows and Linux already include the correct Java version.
    • Developer releases and user releases for Mac do not include Java. You must ensure that you have a compatible version of Java installed on your system.
    • If the minor version of your installed Java is different from the recommended one (for instance, 1.6.0_31 instead of 1.6.0_33) HIPE prints a warning in the command line window from which it was started. Minor version differences are unlikely to cause problems, but this cannot be guaranteed.
    • If the major version of your installed Java is different (for instance, 1.5.0 instead of 1.6.0) HIPE shows a warning in a dialogue window at startup. Major version differences are very likely to cause problems. We do not recommend that you try to run HIPE 9.0 with a Java version other than 1.6.0.
     

    Core system

    Line: 206 to 212
     
    Added:
    >
    >
     
    • Scripting and Data Mining
    Added:
    >
    >
     

    HIFI

    Revision 492012-07-02 - JavierDiaz

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 178 to 178
     
      • Preference to check input and inout parameter modification/replacement by tasks
    • Deprecated and removed tasks (names and parameters) documented in HIPE.
    • Task Viewer:
    Changed:
    <
    <
      • Drag and Drop of the (full) signature of tasks from Task Viewer to Editor & Console
    >
    >
      • Drag and Drop of the (full) signature of tasks from Task Viewer to Editor & Console: whether to drop only mandatory parameters or the full signature is governed by the preference to generate full task calls
     
      • Reordering of nodes and reduced depth
      • Tooltips include the Categories and deprecated tasks appear crossed out
    • Task Outline:

    Revision 482012-06-06 - DavideRizzo

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 126 to 126
     
    • TablePlotter now supports the axis.Date functionality.

    OverPlotter

    • OverPlotter now supports the axis.Date functionality.
    Deleted:
    <
    <

    Dataset viewer

    Astronomical utilities

    Numeric routines

    Data fitting

    Data interpolation

    Random numbers

    Statistics

    Trend analysis

    Mapping

    Other functions

     

    Source extraction

    • SUSSEXtractor algorithm implemented in a fully consistent manner, with detailed mathematical documentation provided
    Line: 159 to 147
     
    • There is a preference for image display to always show the image with the 'match wcs' orientation.
    • A lot of bugfixes.
    Deleted:
    <
    <

    Analysis

     

    Spectra

    Display

    • Major updates to Spectrum Explorer (SE) to improve performance and provide more direct control over plot appearance.
    Line: 174 to 160
     
      • As a consequence of this fix, which is more mathematically correct, it is found that smoothed spectra for SPIRE are quite changed. It is recommended instead to use the REPEAT option.
    • Introduction of a new "Accumulate" task allowing to automatically combine data taken on different frequency grids, with or without frequency overlap.
    Deleted:
    <
    <

    Data cubes

    Display

    Analysis

     

    Herschel Science Archive

    • Improvements in download speed and memory requirements for data downloaded from HSA using a behind-the-scenes utility called MyHSA
      • Testing with 5 HIFI, 9 PACS and 10 SPIRE representative observations of all major observing modes simultaneously with getObservation/saveObservation and with getObs/MyHSA with the same network connection on two different machines show 20 times faster download speed and that 66% less hard disk space is needed to hold the same data:
        MyHSA_downloadTimes.png MyHSA_diskUsage.png
    Deleted:
    <
    <

    Calibration sources

    Products and datasets

    Observation context

     

    Product Access Layer

    When storing pools on shared NFS-mounted disks, HIPE now uses a more robust mechanism to prevent corruption. The default locking mechanism for Local Pools was changed to a mechanism that works more reliably on remote disks.

    Line: 207 to 184
     
    • Task Outline:
      • Deprecated tasks & parameters marked

    Deleted:
    <
    <

    Quality control

    Systematic product generation

     

    Data input-output

    • Deprecation of duplicated functionalities: poolDataWriter, localStoreWriter, poolDataReader. Use getObservation and saveProduct.
    • Deprecation of MyHSA prototype duplicated functionalities: loadObs and downObs. Use getObservation
    • getObservation promoted to a Task (currently getObs).
    • simpleAsciiTableReader deprecated, use asciiTableReader.
    Deleted:
    <
    <

    Virtual Observatory

    Installer

     

    Documentation

    • Quick Start Guide
    Line: 244 to 212
     

    HIFI

    Deleted:
    <
    <

    HIFI Software Configuration and Properties

     

    HIFI Calibration

    • New calibration database HIFI_CAL_9_0, containing the following updates
    Line: 393 to 359
     
    • new viewer for the calibration set release notes

    Analysis tools

    Deleted:
    <
    <

    Photometry

     

    Spectroscopy

    • refactoring of the spectrum explorer
    • improved GUI for depiction of the spectrometer footprint

    Product changes

    Deleted:
    <
    <

    Photometry

     

    Spectroscopy

    • complete browse product generation
    • quality flag "SLOWSAMP" added in case of low sampled spectra (level 2 products)
    Line: 442 to 406
     
      • SCalSpecTeleModel - this contains an empirical correction to the emissivity used in the telescope model - it leads to lower residuals in very faint observations
      • SCalSpecPcalModel - this contains a model PCAL flash profile, used in the new PCAL processing algorithm
    Deleted:
    <
    <

    Calibration Framework

    Preprocessing

    Engineering Conversion

    Common Pipeline

     

    Photometer Pipeline

    • Electrical crosstalk correction has been reinstated in the photometer pipeline.
    Line: 509 to 464
     
      • Useful script for line fitting improved
      • Implemented drag and drop from SDS Explorer detector layout to variables tab
    Deleted:
    <
    <

    Documentation

     
    <-- COMMENT BOX CODE - DO NOT EDIT -->

    Revision 472012-05-25 - KatrinaExter

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 406 to 406
     

    Documentation

    • improvements of URM
    Changed:
    <
    <
    • improvements of the PDRG
    >
    >
    • split of the PDRG into two: Spectroscopy and Photometry
     

    SPIRE

    Revision 462012-05-23 - EdwardPolehampton

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 414 to 414
      These changes are in the SPIRE_CAL_9_0 calibration tree.
    Changed:
    <
    <

    Photometer

    >
    >
    Photometer:
     
    • A column containing frequency in GHz was added to the Photometer filter profile product, SCalPhotRsrf. This now matches Spectrometer data which are in frequency and not wavenumber.
    • Electrical crosstalk coefficients introduced for bolometer-thermistor crosstalk in the SCalPhotElecCross product. This means the electrical crosstalk module can be included in the pipeline - it affects the thermistor timelines but not the bolometer timelines.
    • A new product was added:
      • SCalSpecPcalModel - this contains a model PCAL flash profile, used in the new PCAL processing algorithm
    Changed:
    <
    <

    Spectrometer

    >
    >
    Spectrometer:
     
    • Point source calibration updated in the SCalSpecBeamParam product to:
      • use a new Uranus model (ESA4) derived by Glenn Orton. This is much more rigorous and traceable than the previous model used. It only makes a small difference to the overall calibration: SSW +0.5%, SLW up to -5% at the low frequency end
      • to separate the Uranus observation used to produce the LR and HR versions of the file - this improves the LR calibration
    Line: 455 to 453
     

    Photometer Pipeline

    Changed:
    <
    <
    • Electrical crosstalk correction has been reinstated in the pipeline.
    >
    >
    • Electrical crosstalk correction has been reinstated in the photometer pipeline.
     
      • Implemented crosstalk correction for bolometer-thermistor crosstalk only.
      • Bright Source Calibration will follow in HCSS 10
    Line: 474 to 472
     

    Spectrometer Pipeline

    Changed:
    <
    <
    • Multi-threading implemented in the most time consuming modules - speeds up processing time
    • Un-used columns removed from interferogram product, reducing size of final Observation Context
    • Average Spectra task updated to correctly propagate existing errors - now saves the number of scans averaged in "numScans" column
    • PCAL flash processing improved to take better account of PCAL flash baseline - leads to improvements in Bright mode processing
    >
    >

    Improved calibration accuracy: updated telescope model

    The telescope model has been updated to include an OD dependent multiplicative correction factor to the emissivity of the primary mirror. This leads to a more accurate model, and better calibration, particularly noticable for faint sources. The residual additive uncertainty in the pipeline products was previously 1-3 Jy, and this has now been reduced to 0.5 Jy.

    Low resolution processing

    Low resolution observations are now processed using calibration products based only on Low resolution input data. This leads to a much improved spectral shape for Low resolution observations.

    Faster and slimmer pipeline processing

    Multi-threading has been implemented in the most time consuming modules - speeds up processing time.

    Un-used columns have been removed from interferogram product - this can reduce the size of the interferogram products by 44%. This leads to a small reduction in the total size of the final Observation Context because some other products increased in size slightly.

    Average Spectra

    The Average Spectra task has been updated to correctly propagate existing errors - now saves the number of scans averaged in "numScans" column

    PCAL processing

    The PCAL flash processing has been improved to take better account of the PCAL flash baseline. This is important for Bright mode observations of very bright point sources. In these cases there can be a large baseline caused by pointing jitter. This is correctly fitted and removed by the new PCAL algorithm and leads to improvements in spectral shape for very bright targets.

     

    Interactive Analysis and Tools

    • Photometer:

    Revision 452012-05-23 - EdwardPolehampton

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 416 to 416
     

    Photometer

    Changed:
    <
    <
    • The photometer filter profiles are contained in SCalPhotRsrf. This adds a frequency column to match Spectrometer data which are now in frequency and not wavenumber. Only affects IA.
    • The electrical crosstalk updates mean that the electrical crosstalk correction can be included in the SPG pipeline. These crosstalk files change the thermistor timelines but not the bolometer timelines.
    • The PCAL profile allows the SPG pipeline to use the new PCAL processing algorithm that takes the PCAL baseline into account. This fills the photPcal product which is just for information (it does not affect data in the photometer pipeline).
    >
    >
    • A column containing frequency in GHz was added to the Photometer filter profile product, SCalPhotRsrf. This now matches Spectrometer data which are in frequency and not wavenumber.
    • Electrical crosstalk coefficients introduced for bolometer-thermistor crosstalk in the SCalPhotElecCross product. This means the electrical crosstalk module can be included in the pipeline - it affects the thermistor timelines but not the bolometer timelines.
    • A new product was added:
      • SCalSpecPcalModel - this contains a model PCAL flash profile, used in the new PCAL processing algorithm
     

    Spectrometer

    Changed:
    <
    <
    • Fixed units in the bright gain file meaning that correct units can be propagated by the task in the pipeline. Affects bright mode observations only.
    • The nominal PCAL values needed to be updated to match the new PCAL processing algorithm. Affects bright mode observations only.
    • Updated the beamParam product to:
      • allow using the new Uranus model - small effect on the pipeline result, but model better justified. Biggest effect at longest wavelength.
      • include off-axis point source conversion. No effect on SPG pipeline. For information and IA only.
      • restructure product to remove unnecessary columns and include errors. Makes it easier to understand for IA (no change to pipeline)
    • The PCAL profile allows the SPG pipeline to use the new PCAL processing algorithm that takes the PCAL baseline into account. This is used for processing of bright mode observations only and leads to an improved PCAL gain esp. for very bright sources.
    • The beam prof product is not used in the pipeline - purely informational. Currently has pre-launch model data. Resetting to NaN makes it clear that this should not be used. Update planned for v10.
    • Errors for smecStepFactor are purely informational and not used by the pipeline.
    • Errors for bolPhase are purely informational and not used by the pipeline.
    • RSRF files must be updated to match software updates in the pipeline - this makes them consistent with the final pipeline.
    • The RSRF and beamParam for LR mode are now created only from LR data, giving a better shape to processed low resolution data.
    • The new teleModel allows a correction to be made to the telescope model and leads to a substantial improvement in data for faint sources.
    • A change the OPD limits is necessary for the new phase correction algorithm, and also for consistency to include the real limits for CR observations. CR observations are now processed as HR within the pipeline rather than relying on setting the values in the opdLimits product.
    >
    >
    • Point source calibration updated in the SCalSpecBeamParam product to:
      • use a new Uranus model (ESA4) derived by Glenn Orton. This is much more rigorous and traceable than the previous model used. It only makes a small difference to the overall calibration: SSW +0.5%, SLW up to -5% at the low frequency end
      • to separate the Uranus observation used to produce the LR and HR versions of the file - this improves the LR calibration
      • update the beam FWHM to more recent numbers - affects SLW more than SSW
      • point source conversion factors included for most off-axis detectors (although not used in the standard pipeline)
      • forwardRsrf and reverseRsrf columns removed as these were not used
    • SCalSpecTeleRsrf and SCalSpecInstRsrf updated:
      • to be consistent with other pipeline and calibration changes
      • to use a separate processing to fill LR and HR products separately - this leads to better LR calibration
    • Added errors to the following calibration products:
      • SCalSpecBeamParam
      • SCalSpecSmecStepFactor
      • SCalSpecBolPhase
    • SCalSpecNomPcal product updated to be consistent with the new PCAL processing algorithm
    • Units were corrected in the SCalSpecBrightGain product
    • The interferogram limits were adjusted in the SCalSpecOpdLimits product to be consistent with the new phase correction algorithm and to allow correct processing of CR observations as HR in the pipeline
    • The SCalBeamProf product is not used in the pipeline and contained model data from before launch. In order to avoid confusion it was filled with NaNs until we have enough information to fill it properly.
    • Two new products were added:
      • SCalSpecTeleModel - this contains an empirical correction to the emissivity used in the telescope model - it leads to lower residuals in very faint observations
      • SCalSpecPcalModel - this contains a model PCAL flash profile, used in the new PCAL processing algorithm
     

    Calibration Framework

    Line: 445 to 452
     

    Common Pipeline

    Added:
    >
    >

    Photometer Pipeline

     
    • Electrical crosstalk correction has been reinstated in the pipeline.
      • Implemented crosstalk correction for bolometer-thermistor crosstalk only.
      • Bright Source Calibration will follow in HCSS 10
    Deleted:
    <
    <

    Photometer Pipeline

     
    • DestriperTask
      • Now the default task for baseline removal in both SPG and User pipelines.
      • Added the "useSink" task parameter. The default value is False. It if it is set to True, the ProductSink on disk is used for the Level 1 output. This allows larger observations to be processed with a limited amount of RAM, but makes the task slower.
    Line: 465 to 474
     

    Spectrometer Pipeline

    Added:
    >
    >
    • Multi-threading implemented in the most time consuming modules - speeds up processing time
    • Un-used columns removed from interferogram product, reducing size of final Observation Context
    • Average Spectra task updated to correctly propagate existing errors - now saves the number of scans averaged in "numScans" column
    • PCAL flash processing improved to take better account of PCAL flash baseline - leads to improvements in Bright mode processing
     

    Interactive Analysis and Tools

    Added:
    >
    >
    • Photometer:
     
    • End of scan line turnaround data can now be used in processing and mapmaking.
    • The naive mapper can now create an alternative error map using bolometer weights.
    • SpireMaskEditor
      • Level0_5 SpireMaskEditor was added to the right click menu on an observation context. It is used for displaying and editing the mask data of a Level0_5 Context.
      • Level1 SpireMaskEditor was added to the right click menu on an observation context. It is used for displaying and editing the mask data of a Level1Context.
      • These two additions effectively replace the SpireMaskEditorComponent which was deprecated.
    Added:
    >
    >
    • Spectrometer:
      • Useful scripts added for estimating spectral noise and convolving a high resolution spectrum to the SPIRE resolution
      • Useful script for background subtraction updated to include smoothing of off-axis detectors, and interactive selection
      • Useful script for line fitting improved
      • Implemented drag and drop from SDS Explorer detector layout to variables tab
     

    Documentation

    Revision 442012-05-16 - DavidTeyssier

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 293 to 293
     
    • Level 2.5 product introduced: for Spectral Scans it holds the deconvolved single sideband soluions; for Maps it holds the cubes; for all cases it holds a copy of the level 2 HTP.
    • Addition of browse products for spectral scans and maps. The spectral scan browse product is the deconvolved single sideband solution for H and V. The mapping browse product shows the map averaged spectra and the integrated map over the range of the WBS for the H- and V-polarisations.
    • Point mode browse product axis labels corrected.
    Added:
    >
    >
    • Cubes for Solar System Objects are now created in the comoving frame of the target
     
    • Addition of a flag array to cubes, allowing pixels to be flagged in the same way as points in spectra can be.
    • HTP default view is changed from SpectrumExplorer to the ContextViewer
    • Observing band is added to HTP metadata

    Revision 432012-05-14 - DavideRizzo

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 475 to 475
     

    Documentation

    Added:
    >
    >

    <-- COMMENT BOX CODE - DO NOT EDIT -->
    blog comments powered by Disqus
    <-- END OF COMMENT BOX CODE -->

     
    <--  
    -->

    Revision 422012-05-09 - AndreasPapageorgiou

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 48 to 48
     
    • common (both Photometer and Spectrometer)
      • improved processing speed using multi-threading
      • improved task for correction of "cooler burps"
    Added:
    >
    >
      • PcalTask can now remove noise due to pointing (prevalent in really bright sources) and process Brigh mode Pcal flashes
     
    • Photometer
      • Pipeline Processing
        • improved error maps (possibly as an option)

    Revision 412012-05-08 - AdwinBoogert

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 310 to 310
     

    HIFI Data Processing Tools

    Standing Wave Removal

    • fitHifiFringe now accept SimpleSpectrum data so that, for example, spectra extracted from cubes can be processed.
    Changed:
    <
    <
    • All flag values are represented by 0.5 in the fitHifiFringe plots (masked lines by 1.0, as before). This avoids huge y-axis range.
    >
    >
    • All flag values are represented by 0.5 in the fitHifiFringe plots (masked lines by 1.0, as before). This avoids huge y-axis ranges.
     

    Baseline Removal

    Changed:
    <
    <
    • fitBaseline has new input parameter replace_weights. If set, the pipeline weight columns in the datasets are replaced by 1/(measured RMS noise)^2. This empirical weight is picked up by doDeconvolution to improve the weighting of the different channels.
    >
    >
    • fitBaseline has a new input parameter replace_weights. If set, the pipeline weight columns in the datasets are replaced by 1/(measured RMS noise)^2. This empirical weight is picked up by doDeconvolution to improve the weighting of the different channels.
      *The backends input parameter of fitBaseline allows for individual sideband products to be processed (USB,LSB) in the GUI (it was already possible on the command line).
    Changed:
    <
    <
    *The polynomial order used in a previous run of fitBaseline is now automatically applied if an obsContext is re-processed. Thus the order=-1 option is now obsolete.
    >
    >
    • The polynomial order used in a previous run of fitBaseline and stored in the fitResult tables of an obsContext is automatically applied if an observation is reprocessed with the obsContext input parameter. Thus the order=-1 option is now obsolete.
     

    Deconvolution

    • When multiple observations are passed to doDeconvolution the metadata are now concatenated and put into the decon_result

    Revision 402012-05-08 - AdwinBoogert

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 309 to 309
     

    HIFI Data Processing Tools

    Standing Wave Removal

    Changed:
    <
    <
    • Improved handling of pipeline flags in fitHifiFringe
    • fitHifiFringe now accept SimpleSpectrum data
    >
    >
    • fitHifiFringe now accept SimpleSpectrum data so that, for example, spectra extracted from cubes can be processed.
    • All flag values are represented by 0.5 in the fitHifiFringe plots (masked lines by 1.0, as before). This avoids huge y-axis range.
     

    Baseline Removal

    Changed:
    <
    <
    >
    >
    • fitBaseline has new input parameter replace_weights. If set, the pipeline weight columns in the datasets are replaced by 1/(measured RMS noise)^2. This empirical weight is picked up by doDeconvolution to improve the weighting of the different channels. *The backends input parameter of fitBaseline allows for individual sideband products to be processed (USB,LSB) in the GUI (it was already possible on the command line). *The polynomial order used in a previous run of fitBaseline is now automatically applied if an obsContext is re-processed. Thus the order=-1 option is now obsolete.
     

    Deconvolution

    • When multiple observations are passed to doDeconvolution the metadata are now concatenated and put into the decon_result

    Revision 392012-05-08 - SteveGuest

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 408 to 408
     

    Calibration Products

    Added:
    >
    >
    These changes are in the SPIRE_CAL_9_0 calibration tree.

    Photometer

    • The photometer filter profiles are contained in SCalPhotRsrf. This adds a frequency column to match Spectrometer data which are now in frequency and not wavenumber. Only affects IA.
    • The electrical crosstalk updates mean that the electrical crosstalk correction can be included in the SPG pipeline. These crosstalk files change the thermistor timelines but not the bolometer timelines.
    • The PCAL profile allows the SPG pipeline to use the new PCAL processing algorithm that takes the PCAL baseline into account. This fills the photPcal product which is just for information (it does not affect data in the photometer pipeline).

    Spectrometer

    • Fixed units in the bright gain file meaning that correct units can be propagated by the task in the pipeline. Affects bright mode observations only.
    • The nominal PCAL values needed to be updated to match the new PCAL processing algorithm. Affects bright mode observations only.
    • Updated the beamParam product to:
      • allow using the new Uranus model - small effect on the pipeline result, but model better justified. Biggest effect at longest wavelength.
      • include off-axis point source conversion. No effect on SPG pipeline. For information and IA only.
      • restructure product to remove unnecessary columns and include errors. Makes it easier to understand for IA (no change to pipeline)
    • The PCAL profile allows the SPG pipeline to use the new PCAL processing algorithm that takes the PCAL baseline into account. This is used for processing of bright mode observations only and leads to an improved PCAL gain esp. for very bright sources.
    • The beam prof product is not used in the pipeline - purely informational. Currently has pre-launch model data. Resetting to NaN makes it clear that this should not be used. Update planned for v10.
    • Errors for smecStepFactor are purely informational and not used by the pipeline.
    • Errors for bolPhase are purely informational and not used by the pipeline.
    • RSRF files must be updated to match software updates in the pipeline - this makes them consistent with the final pipeline.
    • The RSRF and beamParam for LR mode are now created only from LR data, giving a better shape to processed low resolution data.
    • The new teleModel allows a correction to be made to the telescope model and leads to a substantial improvement in data for faint sources.
    • A change the OPD limits is necessary for the new phase correction algorithm, and also for consistency to include the real limits for CR observations. CR observations are now processed as HR within the pipeline rather than relying on setting the values in the opdLimits product.
     

    Calibration Framework

    Preprocessing

    Line: 416 to 441
     

    Common Pipeline

    Added:
    >
    >
    • Electrical crosstalk correction has been reinstated in the pipeline.
      • Implemented crosstalk correction for bolometer-thermistor crosstalk only.
      • Bright Source Calibration will follow in HCSS 10
     

    Photometer Pipeline

    Changed:
    <
    <
    • DestriperTask
    >
    >
    • DestriperTask
      • Now the default task for baseline removal in both SPG and User pipelines.
     
      • Added the "useSink" task parameter. The default value is False. It if it is set to True, the ProductSink on disk is used for the Level 1 output. This allows larger observations to be processed with a limited amount of RAM, but makes the task slower.
      • Added the "nThreads" task parameter. The default value is 1. The parameter specifies the number of threads to use, increasing speed significantly on multi-core architectures.
      • Implemented Level2 deglitching. The new "l2DeglichAlgorithm" task parameter specifies the algorithm to be used and can be set to "sigmaKappa" (default) or "twoSigmas".
    Line: 425 to 455
     
      • Added the "startParameters" task parameter. This input parameter accepts a diagnostic product that was provided at the end of a prior destriper run. It contains a table dataset with the last fit parameters and a flag column that determines which scans were excluded. Manual changes to these flags will be honored by a second destriper run in an interactive analysis session. If provided, the contents of the table are used as start parameters for a new destriper run, enabling faster convergence, and improvements to the map by deselecting "bad" scans. The default value is null.
      • Added the "storeTod" task parameter. The default value is False. In this case no TOD files will be stored and the returned TOD parameter is null. Setting the parameter to True, returns a TOD product and leaves the associated files (pixel, signal and weight) stored in the temporary directory until the end of the HIPE session.
    Changed:
    <
    <
    • Cooler Burp
    >
    >
    • Cooler Burp
     
      • A new task called RecalcTempDriftTask to deal with the cooler burp problem was introduced in the build. During the cooler burp the drifts of bolometers is unusual and for this reason the temperature drift correction leaves residuals. The proposed solution is a empirical solution that assume that the problem is due a change in temperature drift coefficients A used in TempDriftCorrectionTask. These parameters are normally read from calibration table and are necessary to link bolometers' signal and voltage thermistors. The task recalculates the A parameters via linear fitting of uncorrected signal vs thermistor voltages and the new parameters are then used in TempDriftCorrectionTask.
      • The user photometer pipeline scripts for parallel and large map observations was modified introducing the possibility to check for cooler burp and to use this new task to correct it. To do this, a new boolean parameter was introduced at the start of the user scripts: coolerBurpDetection. The default value is false, the user that wants check for the cooler burp he must set this parameter value to true.
    Line: 432 to 463
     

    Spectrometer Pipeline

    Interactive Analysis and Tools

    Changed:
    <
    <
    • SpireMaskEditor
    >
    >
    • End of scan line turnaround data can now be used in processing and mapmaking.
    • The naive mapper can now create an alternative error map using bolometer weights.
    • SpireMaskEditor
     
      • Level0_5 SpireMaskEditor was added to the right click menu on an observation context. It is used for displaying and editing the mask data of a Level0_5 Context.
      • Level1 SpireMaskEditor was added to the right click menu on an observation context. It is used for displaying and editing the mask data of a Level1Context.
      • These two additions effectively replace the SpireMaskEditorComponent which was deprecated.

    Revision 382012-05-03 - JeroenDeJong

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 162 to 162
     

    Spectra

    Display

    Changed:
    <
    <
    • Major updates to Spectrum Explorer to improve performance and provide more direct control over plot appearance
    >
    >
    • Major updates to Spectrum Explorer (SE) to improve performance and provide more direct control over plot appearance.
    • The Cube Spectrum Analysis Toolbox (CSAT) has been integrated into the Spectrum Explorer:
      • The CSAT tasks can now be found under a icon in the SE toolbox.
      • All CSAT tasks use now the standard HIPE Task GUI interface, with some additions to receive spectral and spatial selections from the SE. Users can e.g. either fill in a spectral range, or select the range in the spectrum plot.
      • Each task has it own specialized selection tab, so that the original spectra selection do not disappear when one opens a task.
     

    Analysis

    • The Smooth task is corrected for a bug in the ZEROS option.

    Revision 372012-05-03 - CarolynMcCoey

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 320 to 320
     
    • Default output of doGridding changed from a cubes array to a MapContext, this will affect scripts in which a cube is extracted from the output of doGridding. See the HIFI Data Reduction Guide "How to make a Spectral Cube" chapter.
    • The comoving option is now automatically turned on for mapping observations of Solar System Objects
    • Fix to a bug in the way the pixel size was calculated for strip maps
    Changed:
    <
    <
    • Data marked as BAD are ignored during the cube creation.
    >
    >
    • Data flagged as bad_DATA or IGNORE_DATA are ignored during the cube creation.
     

    flagTool

    • The flagTool task introduced in HIPE 8 is now fully functional and can process both spectrometers, both polarizations, and both subbands in one go.

    Revision 362012-05-02 - CarolynMcCoey

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 162 to 162
     

    Spectra

    Display

    Added:
    >
    >
    • Major updates to Spectrum Explorer to improve performance and provide more direct control over plot appearance
     

    Analysis

    • The Smooth task is corrected for a bug in the ZEROS option.
      • As a consequence of this fix, which is more mathematically correct, it is found that smoothed spectra for SPIRE are quite changed. It is recommended instead to use the REPEAT option.
    Added:
    >
    >
    • Introduction of a new "Accumulate" task allowing to automatically combine data taken on different frequency grids, with or without frequency overlap.
     

    Data cubes

    Display

    Line: 273 to 277
     

    Level 2 Pipeline

    • doAvg now honours user-defined flags, RowMask.IGNORE_DATA and HifiMask.IGNORE_DATA
    Added:
    >
    >

    Level 2.5 Pipeline

    • Introduction of a level 2.5 pipeline
      • cube creation is moved from the level 2 pipeline to the level 2.5 pipeline for mapping observations (see Cube Creation section for updates to gridding done in the pipeline)
      • deconvolution is done for spectral scan observations (see Data Processing Tools -> Deconvolution section for updates to deconvolution done in the pipeline)
      • the browse product for all observing modes is done (see HIFI Products section for updates to browse products)
     

    HIFI Products

    • Level 2.5 product introduced: for Spectral Scans it holds the deconvolved single sideband soluions; for Maps it holds the cubes; for all cases it holds a copy of the level 2 HTP.
    Line: 283 to 293
     
    • Observing band is added to HTP metadata

    HIFI Spectrum Toolbox

    Added:
    >
    >
     

    Spectrum Explorer (HIFI specifics)

    Added:
    >
    >
    • Introduction of a HIFI task panel in the Spectrum Explorer. Still in development
     

    Spectrum tasks (HIFI specifics)

    • ConvertK2Jy now uses the same scheme as doGridding to select frequency when interpolating the efficiency values from the calibration tree: the task searches first for the LO Frequency in the metadata, if it is not found the task looks in the metadata for the value inserted in HSpot. If neither are found then the middle frequency of the observing band (e.g. band 1a) is used.
    • Bug fix in ConvertK2Jy: non-zero source size data on frequency scale can now be converted to Jy
    Changed:
    <
    <
    • introduction of a new "accumulate" task allowing to automatically combine data taken on different frequency grids, with or without frequency overlap.
    >
    >
    • ConvertK2Jy will accept data with any wavescale
     

    HIFI Data Processing Tools

    Standing Wave Removal

    Line: 300 to 314
     

    Deconvolution

    • When multiple observations are passed to doDeconvolution the metadata are now concatenated and put into the decon_result
    • doDeconvolution made more robust against failure when there are many bad datasets in the HTP.
    Added:
    >
    >
    • A median subtraction is possible prior to running doDeconvolution.
     

    Spectral Cube creation

    • Default output of doGridding changed from a cubes array to a MapContext, this will affect scripts in which a cube is extracted from the output of doGridding. See the HIFI Data Reduction Guide "How to make a Spectral Cube" chapter.
    Changed:
    <
    <
    >
    >
    • The comoving option is now automatically turned on for mapping observations of Solar System Objects
    • Fix to a bug in the way the pixel size was calculated for strip maps
    • Data marked as BAD are ignored during the cube creation.
     

    flagTool

    • The flagTool task introduced in HIPE 8 is now fully functional and can process both spectrometers, both polarizations, and both subbands in one go.

    Revision 352012-05-02 - CarolynMcCoey

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 250 to 250
     

    hifiPipeline task

    • Addition of an interactive pipeline between level 2 and 2.5, which allows the insertion of tasks like fitHifiFringe and fitBaseline into the pipeline between level 2 and 2.5.
    • Inclusion in the pipeline GUI of a drop-down box to select calibration version
    Added:
    >
    >
    • Spectrometer checkboxes in the pipeline GUI are now only checked if data for that spectrometer are available at level 0.
    • The DoBadLo step of the level 0 pipeline is now run when reprocessing WBS data from level 0 with new calibration. This is to allow you to take advantage of flags indicating bad LO settings that are raised by this pipeline step without having to wait for a new reduction from raw data to be available in the HSA.
     

    Level 0 Pipeline

    • flagging of rare corrupted data-frames according to a list of pre-defined data-frames indices. The difference will only be noticed after the bulk reprocessing has taken place
    Added:
    >
    >
    • revision of quality checks on diplexer out of limits, the effect is to reduce the number of benign quality flags.
     

    WBS Pipeline

    • Introduction of quality flag to warn about a bad zero.
    Added:
    >
    >
    • Introduction of a flag for a wrong scan count
    • Improvements to flagging of saturated channels
    • Improvements to algorithm to fit to COMB lines

     

    HRS Pipeline

    Level 1 Pipeline

    Line: 294 to 302
     
    • doDeconvolution made more robust against failure when there are many bad datasets in the HTP.

    Spectral Cube creation

    Added:
    >
    >
    • Default output of doGridding changed from a cubes array to a MapContext, this will affect scripts in which a cube is extracted from the output of doGridding. See the HIFI Data Reduction Guide "How to make a Spectral Cube" chapter.

    flagTool

    • The flagTool task introduced in HIPE 8 is now fully functional and can process both spectrometers, both polarizations, and both subbands in one go.
    • Capabilities are:
      • choice of 4 flag types (LINE, BRIGHT_LINE, SPUR_CANDIDATE, IGNORE_DATA)
      • automatic flag propagation
      • natural zooming enabled in the plot window
      • easy flag switching
      • automatic merging and splitting of the flags
      • copying of flag from one polarization to the other for a same sideband
      • subband gluing (keyword doglue)
      • embedded use of fitHifiFringe, and embedded use of fitBaseline
     

    HiClass export tool

    • Sideband gain information in the metadata are now exported to CLASS.

    Revision 342012-04-27 - DavideRizzo

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 20 to 20
     
    • Speed-up of HIPE start-up
    • Progress bar informs on progress of for-loops
    • Human-made indexes added to two core manuals (HIPE Owner's Guide and Scripting and Data Mining Guide)
    Changed:
    <
    <
    • Possibility to leave comments at the bottom of any page in the HIPE Help System
    >
    >
    • You can leave comments at the bottom of any page in the HIPE Help System
     
    • Support to OSX Lion
    • Easy access to calibration data, including version and updates (TBS)
    • Improvements of search capability of documentation system
    Line: 53 to 53
     
        • improved error maps (possibly as an option)
        • improved baseline correction using "destriper“
      • Interactive processing
    Changed:
    <
    <
    * sourceExtractorTimeline task, for measuring source photometry using timeline data * source timeline fitter task * improvements to bolometer finder tool
    >
    >
        • source timeline fitter task
        • improvements to bolometer finder tool
     
        • selection of turnaround data in map creation
    • Spectrometer
      • Pipeline Processing: smaller size for level 1 products
    Line: 113 to 112
     
    • Video Tutorials entry has been added to the Help menu.
    • Context sensitive help on the focused view has been added. It can be accessed through Help > Help on <name> view or with F2.
    • If debug is enabled (see preferences, General > Debug), a threads dump can be obtained with the menu option Tools > Debug > Dump thread stacks. It can be helpful when informing the Helpdesk on software problems.
    Added:
    >
    >
    • A new tab with a glossary of terms has been added to the Help System.
    • You can leave comments at the bottom of any help page. You must be connected to the Internet to read and leave comments.
    • You can get the full address of any help page, for sharing or adding to your browser's bookmarks. Click on the icon shaped like a chain in the Help System toolbar.
    • Many improvements to the search interface of the Help System:
      • You can now filter searches on any manual or group of manuals.
      • After each search, a feedback form asks you whether you found what you were looking for. Your feedback will be used to further improve the search engine.
      • Glossary terms are always at the top of search results.
     

    Plotting

    TablePlotter

    Line: 211 to 217
     

    Documentation

    • Quick Start Guide
    Changed:
    <
    <
      • Updated to reflect the latest changes in data access (MyHSA and eHSA).
    >
    >
      • Updated to reflect the latest changes in data access (MyHSA and eHSA).
     
    • HIPE Owner's Guide
    Changed:
    <
    <
      • Updated throughout to reflect the latest changes in HIPE.
      • In particular, the latest improvements in the help search engine are now described.
    >
    >
      • Updated throughout to reflect the latest changes in HIPE.
      • In particular, the latest improvements in the help search engine are now described.
      • Added man-made index.
     
    • Data Analysis Guide
    Changed:
    <
    <
      • Updated the data input/output chapter to reflect the latest changes in data access (MyHSA and eHSA).
      • Updated and reorganised the image analysis chapter.
      • Updated and reorganised the spectral analysis chapter.
      • Added new spectral cubes chapter.
    >
    >

    • Scripting and Data Mining
     

    HIFI

    Revision 332012-04-25 - DavideRizzo

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 210 to 210
     

    Documentation

    Added:
    >
    >
    • Quick Start Guide
      • Updated to reflect the latest changes in data access (MyHSA and eHSA).

    • HIPE Owner's Guide
      • Updated throughout to reflect the latest changes in HIPE.
      • In particular, the latest improvements in the help search engine are now described.
     
    • Data Analysis Guide
    Changed:
    <
    <
      • Updated the data input/output chapter to reflect the latest updates (MyHSA and eHSA).
    >
    >
      • Updated the data input/output chapter to reflect the latest changes in data access (MyHSA and eHSA).
     
      • Updated and reorganised the image analysis chapter.
      • Updated and reorganised the spectral analysis chapter.
      • Added new spectral cubes chapter.

    Revision 322012-04-24 - DavideRizzo

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 19 to 19
     
    • Smaller memory footprint for HIPE editor
    • Speed-up of HIPE start-up
    • Progress bar informs on progress of for-loops
    Changed:
    <
    <
    • Human-made indexes added to the core manual (HIPE Owner's Guide, Scripting Guide, Data Analysis Guide)
    >
    >
    • Human-made indexes added to two core manuals (HIPE Owner's Guide and Scripting and Data Mining Guide)
     
    • Possibility to leave comments at the bottom of any page in the HIPE Help System
    • Support to OSX Lion
    • Easy access to calibration data, including version and updates (TBS)
    Changed:
    <
    <
    • Improvements of search capability of documentation system (TBS)
    >
    >
    • Improvements of search capability of documentation system
     
    • Publication ready plots (TBS)
    Line: 210 to 210
     

    Documentation

    Added:
    >
    >
    • Data Analysis Guide
      • Updated the data input/output chapter to reflect the latest updates (MyHSA and eHSA).
      • Updated and reorganised the image analysis chapter.
      • Updated and reorganised the spectral analysis chapter.
      • Added new spectral cubes chapter.
     

    HIFI

    HIFI Software Configuration and Properties

    Revision 312012-04-23 - JaimeSaiz

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 76 to 76
     

    HIPE

    Changed:
    <
    <

    Main window

    >
    >

    Status bar

     
    Changed:
    <
    <
    The state of the main window (size, location, maximized state) is remembered among HIPE sessions, so it is not necessary to set it after every start-up.
    >
    >
    Execution of for loops is reflected in the progress bar by being increased along with the percentage of executed iterations.
     

    Startup & shutdown

    Changed:
    <
    <
    Check of local pool locks (probably left from a previous session that was abnormally terminated) is executed at startup. This check is configurable in preferences.
    >
    >
    • It is possible to set a number of py files to be executed at startup, in the new Imports preferences panel.
    • The main window's state (size, location, maximized state) is remembered among HIPE sessions, so it is not necessary to set it after every startup.
    • Check of local pool locks (probably left from a previous session that was abnormally terminated) is executed at startup. This check is configurable in preferences.
    • The running Java version is compared with the supported one; if they are different, a warning is popped up. This check can be enabled or disabled at will, in preferences.
     
    Changed:
    <
    <

    Editor area & Viewers

    >
    >

    Editors & Viewers

     
    • The tabs within the Editor area show the close button even if not selected, so they can be closed with 1 click instead of 2.
    Changed:
    <
    <
    • The list of viewers shown in the Open With menu for the selected variable is more specific.
    • The viewer of lists shows the data values instead of just their type.
    >
    >
    • The viewers shown in the Open With menu are the most specific ones for the selected variable.
    • The viewer for lists shows data values instead of just their type.
    • Autosave of text editors and Jython script editors is turned on by default.
    • The Jython Editor preferences now appear under the Text Editor preferences in the preferences window.
    • Boolean values in tables are shown as true / false instead of the previous less clear 1 / 0.
    • Tooltips in table headers include more information on the columns.
    • The load of a node in the product tree, when selected, can be cancelled with the Stop button.

    Console & Log

    • Obsolete names (renamed or removed tasks, functions, etc.) can be registered by developers so meaningful messages, explaining the reason of the name removal, can be shown instead of a plain NameError.
    • Incremental search (with Ctrl+I) is now available in the Log view.

    Navigator

    • The refresh period of the Navigator view can be configured in preferences.
    • User areas are included in the Open File dialogue.
     

    Help

    Changed:
    <
    <
    Context sensitive help on the focused view added. It can be accessed through Help > Help on <name> view or with F2.
    >
    >
    • Video Tutorials entry has been added to the Help menu.
    • Context sensitive help on the focused view has been added. It can be accessed through Help > Help on <name> view or with F2.
    • If debug is enabled (see preferences, General > Debug), a threads dump can be obtained with the menu option Tools > Debug > Dump thread stacks. It can be helpful when informing the Helpdesk on software problems.
     

    Plotting

    TablePlotter

    Revision 302012-04-20 - BrunoAltieri

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 290 to 290
     
      • the flat-field (flatField, v4)
      • responsivity change of ~2% only at 100 and 160 microns (responsivity v7)
      • aperture correction (apertureCorrection v3)
    Added:
    >
    >
      • *non-linearity correction ( nonLinearCoef v2)
     

    Spectroscopy

    Revision 292012-04-20 - JaimeSaiz

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 76 to 76
     

    HIPE

    Added:
    >
    >

    Main window

    The state of the main window (size, location, maximized state) is remembered among HIPE sessions, so it is not necessary to set it after every start-up.

    Startup & shutdown

    Check of local pool locks (probably left from a previous session that was abnormally terminated) is executed at startup. This check is configurable in preferences.

    Editor area & Viewers

    • The tabs within the Editor area show the close button even if not selected, so they can be closed with 1 click instead of 2.
    • The list of viewers shown in the Open With menu for the selected variable is more specific.
    • The viewer of lists shows the data values instead of just their type.

    Help

    Context sensitive help on the focused view added. It can be accessed through Help > Help on <name> view or with F2.

     

    Plotting

    TablePlotter

    • TablePlotter now supports the axis.Date functionality.

    Revision 282012-04-20 - DavidTeyssier

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 53 to 53
     
        • improved error maps (possibly as an option)
        • improved baseline correction using "destriper“
      • Interactive processing
    Changed:
    <
    <
        • sourceExtractorTimeline task, for measuring source photometry using timeline data
        • improvements to bolometer finder tool
    >
    >
    * sourceExtractorTimeline task, for measuring source photometry using timeline data * source timeline fitter task * improvements to bolometer finder tool
     
        • selection of turnaround data in map creation
    • Spectrometer
      • Pipeline Processing: smaller size for level 1 products
    Line: 177 to 178
     

    HIFI Calibration

    Added:
    >
    >
    • New calibration database HIFI_CAL_9_0, containing the following updates
      • Side-band ratio correction over the whole of bands 5a and 5b
      • Update of a number of engineering threshold for more accurate quality flagging related to hardware housekeeping
      • introduction of a list of a priori known corrupted data-frames for dedicated flagging in level0 data
     

    HIFI Pipeline

    hifiPipeline task

    Line: 184 to 190
     
    • Inclusion in the pipeline GUI of a drop-down box to select calibration version

    Level 0 Pipeline

    Added:
    >
    >
    • flagging of rare corrupted data-frames according to a list of pre-defined data-frames indices. The difference will only be noticed after the bulk reprocessing has taken place
     

    WBS Pipeline

    • Introduction of quality flag to warn about a bad zero.
    Line: 191 to 198
     

    HRS Pipeline

    Level 1 Pipeline

    • Addition of parameters in the doFilterLoads to allow configuration of the filtering.
    Added:
    >
    >
    • Addition of a new quality products "flagsSummary" compiling the list of all rowflags set to the data at this level
     

    Level 2 Pipeline

    • doAvg now honours user-defined flags, RowMask.IGNORE_DATA and HifiMask.IGNORE_DATA
    Line: 209 to 217
     

    Spectrum tasks (HIFI specifics)

    • ConvertK2Jy now uses the same scheme as doGridding to select frequency when interpolating the efficiency values from the calibration tree: the task searches first for the LO Frequency in the metadata, if it is not found the task looks in the metadata for the value inserted in HSpot. If neither are found then the middle frequency of the observing band (e.g. band 1a) is used.
    • Bug fix in ConvertK2Jy: non-zero source size data on frequency scale can now be converted to Jy
    Changed:
    <
    <
    >
    >
    • introduction of a new "accumulate" task allowing to automatically combine data taken on different frequency grids, with or without frequency overlap.
     

    HIFI Data Processing Tools

    Standing Wave Removal

    Revision 272012-04-20 - AnthonySmith

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 53 to 53
     
        • improved error maps (possibly as an option)
        • improved baseline correction using "destriper“
      • Interactive processing
    Changed:
    <
    <
        • source timeline fitter task
    >
    >
        • sourceExtractorTimeline task, for measuring source photometry using timeline data
     
        • improvements to bolometer finder tool
        • selection of turnaround data in map creation
    • Spectrometer
    Line: 94 to 94
     

    Other functions

    Source extraction

    Added:
    >
    >
    • SUSSEXtractor algorithm implemented in a fully consistent manner, with detailed mathematical documentation provided
    • Defaults for sourceExtractorSussextractor now: useSignalToNoise=False, fitBackground=True, fluxPriorsLambda=0.0
    • "beamArea" parameter for sourceExtractorSussextractor deprecated
    • "inputSourceList" parameter now accepts [ra, dec] and other lists of values as input, as well as a SourceListProduct.
    • Many internal changes to the tasks, improving speed and consistency in applying the algorithms
    • Multi-threading: increasing speed of tasks significantly on multi-core machines
     

    Images

    Display

    Revision 262012-04-19 - AnthonySmith

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 50 to 50
     
      • improved task for correction of "cooler burps"
    • Photometer
      • Pipeline Processing
    Deleted:
    <
    <
        • kernel-smoothed maps (aka "drizzling")
     
        • improved error maps (possibly as an option)
        • improved baseline correction using "destriper“
      • Interactive processing

    Revision 252012-04-19 - GabrieleMainetti

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 305 to 305
     
      • Added the "startParameters" task parameter. This input parameter accepts a diagnostic product that was provided at the end of a prior destriper run. It contains a table dataset with the last fit parameters and a flag column that determines which scans were excluded. Manual changes to these flags will be honored by a second destriper run in an interactive analysis session. If provided, the contents of the table are used as start parameters for a new destriper run, enabling faster convergence, and improvements to the map by deselecting "bad" scans. The default value is null.
      • Added the "storeTod" task parameter. The default value is False. In this case no TOD files will be stored and the returned TOD parameter is null. Setting the parameter to True, returns a TOD product and leaves the associated files (pixel, signal and weight) stored in the temporary directory until the end of the HIPE session.
    Added:
    >
    >
    • Cooler Burp
      • A new task called RecalcTempDriftTask to deal with the cooler burp problem was introduced in the build. During the cooler burp the drifts of bolometers is unusual and for this reason the temperature drift correction leaves residuals. The proposed solution is a empirical solution that assume that the problem is due a change in temperature drift coefficients A used in TempDriftCorrectionTask. These parameters are normally read from calibration table and are necessary to link bolometers' signal and voltage thermistors. The task recalculates the A parameters via linear fitting of uncorrected signal vs thermistor voltages and the new parameters are then used in TempDriftCorrectionTask.
      • The user photometer pipeline scripts for parallel and large map observations was modified introducing the possibility to check for cooler burp and to use this new task to correct it. To do this, a new boolean parameter was introduced at the start of the user scripts: coolerBurpDetection. The default value is false, the user that wants check for the cooler burp he must set this parameter value to true.
     

    Spectrometer Pipeline

    Revision 242012-04-18 - JavierDiaz

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Revision 232012-04-18 - BrunoAltieri

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 233 to 233
     

    Pipeline

    Photometry

    Added:
    >
    >
    • updated ipipe scripts with:
      • optimized second order deglitching with the MapDeglitch task,
      • optimized mask generation for the highpass filtering
      • now using the photMaskCrosstalk and filterSlew tasks
      • improved MADmap scripts with a new pre-processing task photBaselineDriftCorrection and a post-processing task to remove bright point-source artifacts photCorrMadmapArtifacts
    • the filterOnScanSpeed task now only masks the turnover frames instead of deleting them.
     

    Spectroscopy

    • estimate of continuum RMS is included in plotting function of ipipe scripts
    • new task that extracts the central spaxel spectrum with a 3x3/central correction
    Line: 248 to 255
     

    Calibration

    Photometry

    Added:
    >
    >
    • new calibration products for :
      • the flat-field (flatField, v4)
      • responsivity change of ~2% only at 100 and 160 microns (responsivity v7)
      • aperture correction (apertureCorrection v3)
     

    Spectroscopy

    • new calibration product offRatio for all bands and chopper throws
    • new calibration product beams (raster beams of neptune) for all bands

    Revision 222012-04-17 - BernhardSchulz

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 285 to 285
     

    Photometer Pipeline

    • DestriperTask
    Changed:
    <
    <
      • The "useSink" task parameter is added. The default value is False. It if it True, the ProductSink is used to save the output.
      • The "nThreads" task parameter is added. The default value is 1. Multi-threads will be used when nThreads>1.
      • The Level2 deglitching is implemented. The "l2DeglichAlgorithm" task parameter was added. The default value is sigmaKappa.
      • The "l2IterMax" task parameter is added. The default value is 5. It specifies how many iterations the Level2 deglitcher will run.
      • The "starParameters" task parameter is added. The default value is null. It can be used to de-selected some detectors or scans with known issues. It can also be used as a "check state" to restart from the previous run.
      • The "storeTod" task parameter is added. The default value is False and no tod files will be stored and returned tod is null. When it is True, the tod associated files (pixel, signal and weight) will be stored in the temporary directory.
    >
    >
      • Added the "useSink" task parameter. The default value is False. It if it is set to True, the ProductSink on disk is used for the Level 1 output. This allows larger observations to be processed with a limited amount of RAM, but makes the task slower.
      • Added the "nThreads" task parameter. The default value is 1. The parameter specifies the number of threads to use, increasing speed significantly on multi-core architectures.
      • Implemented Level2 deglitching. The new "l2DeglichAlgorithm" task parameter specifies the algorithm to be used and can be set to "sigmaKappa" (default) or "twoSigmas".
      • Added the "l2IterMax" task parameter. It limits the number of iterations of the Level2 deglitcher within a destriper iteration. The default value is 5.
      • Added the "startParameters" task parameter. This input parameter accepts a diagnostic product that was provided at the end of a prior destriper run. It contains a table dataset with the last fit parameters and a flag column that determines which scans were excluded. Manual changes to these flags will be honored by a second destriper run in an interactive analysis session. If provided, the contents of the table are used as start parameters for a new destriper run, enabling faster convergence, and improvements to the map by deselecting "bad" scans. The default value is null.
      • Added the "storeTod" task parameter. The default value is False. In this case no TOD files will be stored and the returned TOD parameter is null. Setting the parameter to True, returns a TOD product and leaves the associated files (pixel, signal and weight) stored in the temporary directory until the end of the HIPE session.
     

    Spectrometer Pipeline

    Interactive Analysis and Tools

    • SpireMaskEditor
    Changed:
    <
    <
      • Level0_5 SpireMaskEditor is introduced. It can only be used for editing and displaying Level0_5 Context.
      • Level1 SpireMaskEditor is introduced. It can only be used for editing and displaying Level1Context.
      • SpireMaskEditorComponent is deprecated.
    >
    >
      • Level0_5 SpireMaskEditor was added to the right click menu on an observation context. It is used for displaying and editing the mask data of a Level0_5 Context.
      • Level1 SpireMaskEditor was added to the right click menu on an observation context. It is used for displaying and editing the mask data of a Level1Context.
      • These two additions effectively replace the SpireMaskEditorComponent which was deprecated.
     

    Documentation

    Revision 212012-04-17 - PierreRoyer

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 238 to 238
     
    • new task that extracts the central spaxel spectrum with a 3x3/central correction
    • the task specFlagGlitchFramesQTest can optionally be threaded
    • background normalization ipipe script introduced also for long range SED
    Changed:
    <
    <
    • improved flatfielding task
    >
    >
    • improved flatfielding task, for line-scan mode
     
    • improved drizzling routines
      • available for chopped and unchopped mode
      • uses observed beam size for the calculation of the pixel scaling of the spatial grid

    Revision 202012-04-17 - JuergenSchreiber

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 244 to 244
     
      • uses observed beam size for the calculation of the pixel scaling of the spatial grid
    • test versions of pointing correction tasks
    • improved error propagation
    Added:
    >
    >
    • correction of SSO pointing task correctRaDec4Sso introduced in spg pipeline
     

    Calibration

    Photometry

    Line: 252 to 253
     
    • new calibration product beams (raster beams of neptune) for all bands
    • update of the pointSourceLoss calibration product
    • new calibration product beamSize
    Added:
    >
    >
    • new viewer for the calibration set release notes
     

    Analysis tools

    Photometry

    Spectroscopy

    • refactoring of the spectrum explorer
    Changed:
    <
    <
    • Gui for depiction of the spectrometer footprint
    >
    >
    • improved GUI for depiction of the spectrometer footprint
     

    Product changes

    Photometry

    Revision 192012-04-17 - SaraRegibo

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 240 to 240
     
    • background normalization ipipe script introduced also for long range SED
    • improved flatfielding task
    • improved drizzling routines
    Added:
    >
    >
      • available for chopped and unchopped mode
      • uses observed beam size for the calculation of the pixel scaling of the spatial grid
     
    • test versions of pointing correction tasks
    • improved error propagation

    Revision 182012-04-17 - JuergenSchreiber

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 238 to 238
     
    • new task that extracts the central spaxel spectrum with a 3x3/central correction
    • the task specFlagGlitchFramesQTest can optionally be threaded
    • background normalization ipipe script introduced also for long range SED
    Added:
    >
    >
    • improved flatfielding task
    • improved drizzling routines
    • test versions of pointing correction tasks
    • improved error propagation
     

    Calibration

    Photometry

    Spectroscopy

    Added:
    >
    >
    • new calibration product offRatio for all bands and chopper throws
    • new calibration product beams (raster beams of neptune) for all bands
    • update of the pointSourceLoss calibration product
    • new calibration product beamSize
     

    Analysis tools

    Photometry

    Spectroscopy

    Added:
    >
    >
    • refactoring of the spectrum explorer
    • Gui for depiction of the spectrometer footprint
     

    Product changes

    Photometry

    Spectroscopy

    Added:
    >
    >
    • complete browse product generation
    • quality flag "SLOWSAMP" added in case of low sampled spectra (level 2 products)
     

    Documentation

    Added:
    >
    >
    • improvements of URM
    • improvements of the PDRG
     

    SPIRE

    Revision 172012-04-17 - WimDeMeester

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 13 to 13
     HIPE

    • Common look and feel for the spectrum explorer and cube analysis toolbox
    Deleted:
    <
    <
    • Image analysis perspective
     
    • Easing the data access via MyHSA
    • Improvement of handling of ASCII tables
    • Mechanism to notify users on removed names (tasks, functions...), why they were removed, and their replacement
    Line: 104 to 103
     
      • saveCurrentView(filename), saveCurrentViewAsEPS(filename), saveCurrentViewAsPNG(filename), saveCurrentViewAsJPG(filename) and saveCurrentViewAsPDF(filename) : These methods will save the current view of the image, including annotations and axes.
    • The coordinate axes on the image panner are always visible now
    • There is a preference for image display to turn of the echoing of commands to the command line. It is possible to turn all these commands off, or only the zoom commands.
    Added:
    >
    >
    • Flagged out pixels and pixels with a value of NaN will be shown in the color of the background.
    • Image can be matched with the Wcs. This means that the images can be oriented so that north is shown at the top and east to the left.
    • There is a preference for image display to always show the image with the 'match wcs' orientation.
    • A lot of bugfixes.
     

    Analysis

    Revision 162012-04-17 - JuergenSchreiber

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 231 to 231
     

    Pipeline

    Photometry

    Spectroscopy

    Added:
    >
    >
    • estimate of continuum RMS is included in plotting function of ipipe scripts
    • new task that extracts the central spaxel spectrum with a 3x3/central correction
    • the task specFlagGlitchFramesQTest can optionally be threaded
    • background normalization ipipe script introduced also for long range SED
     

    Calibration

    Photometry

    Revision 152012-04-17 - JavierDiaz

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 138 to 138
     
    • Task Preferences
    • Preference to generate full task call from the task GUIs
    • Preference to check input and inout parameter modification/replacement by tasks
    Changed:
    <
    <
    • Deprecated Tasks (names and parameters) appear documented in HIPE.
    • Drag and Drop of the (full) signature of tasks: from Task Viewer to Editor & Console
    >
    >
    • Deprecated and removed tasks (names and parameters) documented in HIPE.
    • Task Viewer:
      • Drag and Drop of the (full) signature of tasks from Task Viewer to Editor & Console
      • Reordering of nodes and reduced depth
      • Tooltips include the Categories and deprecated tasks appear crossed out
    • Task Outline:
      • Deprecated tasks & parameters marked

     

    Quality control

    Revision 142012-04-17 - JavierDiaz

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 135 to 135
     Usability improvements for the Product Browser and for working with tags. One can now update existing tags (for example to fix a typo).

    Tasks

    Added:
    >
    >
    • Task Preferences
    • Preference to generate full task call from the task GUIs
    • Preference to check input and inout parameter modification/replacement by tasks
     
    • Deprecated Tasks (names and parameters) appear documented in HIPE.
    Added:
    >
    >
    • Drag and Drop of the (full) signature of tasks: from Task Viewer to Editor & Console
     

    Quality control

    Revision 132012-04-16 - LijunZhang

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 79 to 79
     

    Plotting

    TablePlotter

    Added:
    >
    >
    • TablePlotter now supports the axis.Date functionality.
     

    OverPlotter

    Added:
    >
    >
    • OverPlotter now supports the axis.Date functionality.
     

    Dataset viewer

    Astronomical utilities

    Line: 247 to 249
     

    Common Pipeline

    Photometer Pipeline

    Added:
    >
    >
    • DestriperTask
      • The "useSink" task parameter is added. The default value is False. It if it True, the ProductSink is used to save the output.
      • The "nThreads" task parameter is added. The default value is 1. Multi-threads will be used when nThreads>1.
      • The Level2 deglitching is implemented. The "l2DeglichAlgorithm" task parameter was added. The default value is sigmaKappa.
      • The "l2IterMax" task parameter is added. The default value is 5. It specifies how many iterations the Level2 deglitcher will run.
      • The "starParameters" task parameter is added. The default value is null. It can be used to de-selected some detectors or scans with known issues. It can also be used as a "check state" to restart from the previous run.
      • The "storeTod" task parameter is added. The default value is False and no tod files will be stored and returned tod is null. When it is True, the tod associated files (pixel, signal and weight) will be stored in the temporary directory.
     

    Spectrometer Pipeline

    Interactive Analysis and Tools

    Added:
    >
    >
    • SpireMaskEditor
      • Level0_5 SpireMaskEditor is introduced. It can only be used for editing and displaying Level0_5 Context.
      • Level1 SpireMaskEditor is introduced. It can only be used for editing and displaying Level1Context.
      • SpireMaskEditorComponent is deprecated.
     

    Documentation

    Revision 122012-04-16 - PaulBalm

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's new in HIPE 9

    Line: 128 to 128
     

    Product Access Layer

    Added:
    >
    >
    When storing pools on shared NFS-mounted disks, HIPE now uses a more robust mechanism to prevent corruption. The default locking mechanism for Local Pools was changed to a mechanism that works more reliably on remote disks.

    Usability improvements for the Product Browser and for working with tags. One can now update existing tags (for example to fix a typo).

     

    Tasks

    • Deprecated Tasks (names and parameters) appear documented in HIPE.

    Revision 112012-04-16 - DavideRizzo

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"
    Changed:
    <
    <

    What's coming up in HIPE 9

    >
    >

    What's new in HIPE 9

     
    PDF Version Click here

    Revision 102012-04-04 - CarolynMcCoey

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's coming up in HIPE 9

    Line: 116 to 116
     

    Analysis

    Herschel Science Archive

    Added:
    >
    >
    • Improvements in download speed and memory requirements for data downloaded from HSA using a behind-the-scenes utility called MyHSA
      • Testing with 5 HIFI, 9 PACS and 10 SPIRE representative observations of all major observing modes simultaneously with getObservation/saveObservation and with getObs/MyHSA with the same network connection on two different machines show 20 times faster download speed and that 66% less hard disk space is needed to hold the same data:
        MyHSA_downloadTimes.png MyHSA_diskUsage.png
     

    Calibration sources

    Line: 249 to 253
     
    <--  
    -->
    Added:
    >
    >

    META FILEATTACHMENT attr="" autoattached="1" comment="HSA download times" date="1333576243" name="MyHSA_downloadTimes.png" path="MyHSA_downloadTimes.png" size="51431" user="Main.CarolynMcCoey" version="1"
    META FILEATTACHMENT attr="" autoattached="1" comment="HSA disk usage" date="1333576193" name="MyHSA_diskUsage.png" path="MyHSA_diskUsage.png" size="85841" user="Main.CarolynMcCoey" version="1"

    Revision 92012-03-28 - CarolynMcCoey

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's coming up in HIPE 9

    Line: 108 to 108
     

    Spectra

    Display

    Analysis

    Added:
    >
    >
    • The Smooth task is corrected for a bug in the ZEROS option.
      • As a consequence of this fix, which is more mathematically correct, it is found that smoothed spectra for SPIRE are quite changed. It is recommended instead to use the REPEAT option.
     

    Data cubes

    Display

    Line: 169 to 171
     
    • Level 2.5 product introduced: for Spectral Scans it holds the deconvolved single sideband soluions; for Maps it holds the cubes; for all cases it holds a copy of the level 2 HTP.
    • Addition of browse products for spectral scans and maps. The spectral scan browse product is the deconvolved single sideband solution for H and V. The mapping browse product shows the map averaged spectra and the integrated map over the range of the WBS for the H- and V-polarisations.
    Added:
    >
    >
    • Point mode browse product axis labels corrected.
     
    • Addition of a flag array to cubes, allowing pixels to be flagged in the same way as points in spectra can be.
    • HTP default view is changed from SpectrumExplorer to the ContextViewer
    • Observing band is added to HTP metadata
    Line: 190 to 193
     

    Deconvolution

    • When multiple observations are passed to doDeconvolution the metadata are now concatenated and put into the decon_result
    Added:
    >
    >
    • doDeconvolution made more robust against failure when there are many bad datasets in the HTP.
     

    Spectral Cube creation

    Line: 198 to 202
     

    Documentation

    • Updates to DBS Cookbook
    Added:
    >
    >
    • Updates to HIFI Data Reduction Guide for changes to:
      • fitBaseline
      • fitHifiFringe
     

    PACS

    Revision 82012-03-28 - CarolynMcCoey

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's coming up in HIPE 9

    Line: 154 to 154
     
    • Inclusion in the pipeline GUI of a drop-down box to select calibration version

    Level 0 Pipeline

    Added:
    >
    >
     

    WBS Pipeline

    Added:
    >
    >
    • Introduction of quality flag to warn about a bad zero.

     

    HRS Pipeline

    Level 1 Pipeline

    Added:
    >
    >
    • Addition of parameters in the doFilterLoads to allow configuration of the filtering.
     

    Level 2 Pipeline

    Added:
    >
    >
    • doAvg now honours user-defined flags, RowMask.IGNORE_DATA and HifiMask.IGNORE_DATA
     

    HIFI Products

    Line: 165 to 171
     
    • Addition of browse products for spectral scans and maps. The spectral scan browse product is the deconvolved single sideband solution for H and V. The mapping browse product shows the map averaged spectra and the integrated map over the range of the WBS for the H- and V-polarisations.
    • Addition of a flag array to cubes, allowing pixels to be flagged in the same way as points in spectra can be.
    • HTP default view is changed from SpectrumExplorer to the ContextViewer
    Added:
    >
    >
    • Observing band is added to HTP metadata
     

    HIFI Spectrum Toolbox

    Spectrum Explorer (HIFI specifics)

    Spectrum tasks (HIFI specifics)

    Added:
    >
    >
    • ConvertK2Jy now uses the same scheme as doGridding to select frequency when interpolating the efficiency values from the calibration tree: the task searches first for the LO Frequency in the metadata, if it is not found the task looks in the metadata for the value inserted in HSpot. If neither are found then the middle frequency of the observing band (e.g. band 1a) is used.
    • Bug fix in ConvertK2Jy: non-zero source size data on frequency scale can now be converted to Jy
     

    HIFI Data Processing Tools

    Standing Wave Removal

    Line: 179 to 189
     

    Deconvolution

    Added:
    >
    >
    • When multiple observations are passed to doDeconvolution the metadata are now concatenated and put into the decon_result
     

    Spectral Cube creation

    HiClass export tool

    Added:
    >
    >
    • Sideband gain information in the metadata are now exported to CLASS.
     

    Documentation

    Added:
    >
    >
    • Updates to DBS Cookbook
     

    PACS

    Revision 72012-03-28 - CarolynMcCoey

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's coming up in HIPE 9

    Line: 30 to 30
      HIFI
    Changed:
    <
    <
    • Scripts to permit fringe removal / current matching
    • HIFI level 2.5 product to hold data of Spectral Surveys
    • Browse products for spectral surveys in maps (TBS)
    >
    >
    • HIFI level 2.5 product introduced: for Spectral Scans it holds the deconvolved single sideband soluions; for Maps it holds the cubes; for all cases it holds a copy of the level 2 HTP.
    • Addition of an interactive pipeline between level 2 and 2.5, which allows the insertion of tasks like fitHifiFringe and fitBaseline into the pipeline between level 2 and 2.5.
    • Browse products for spectral scans and maps added
      PACS
    Line: 148 to 148
     

    HIFI Calibration

    HIFI Pipeline

    Added:
    >
    >
     

    hifiPipeline task

    Added:
    >
    >
    • Addition of an interactive pipeline between level 2 and 2.5, which allows the insertion of tasks like fitHifiFringe and fitBaseline into the pipeline between level 2 and 2.5.
    • Inclusion in the pipeline GUI of a drop-down box to select calibration version
     

    Level 0 Pipeline

    WBS Pipeline

    HRS Pipeline

    Line: 157 to 161
     

    HIFI Products

    Added:
    >
    >
    • Level 2.5 product introduced: for Spectral Scans it holds the deconvolved single sideband soluions; for Maps it holds the cubes; for all cases it holds a copy of the level 2 HTP.
    • Addition of browse products for spectral scans and maps. The spectral scan browse product is the deconvolved single sideband solution for H and V. The mapping browse product shows the map averaged spectra and the integrated map over the range of the WBS for the H- and V-polarisations.
    • Addition of a flag array to cubes, allowing pixels to be flagged in the same way as points in spectra can be.
    • HTP default view is changed from SpectrumExplorer to the ContextViewer
     

    HIFI Spectrum Toolbox

    Spectrum Explorer (HIFI specifics)

    Spectrum tasks (HIFI specifics)

    HIFI Data Processing Tools

    Standing Wave Removal

    Added:
    >
    >
    • Improved handling of pipeline flags in fitHifiFringe
    • fitHifiFringe now accept SimpleSpectrum data
     

    Baseline Removal

    Added:
    >
    >
     

    Deconvolution

    Added:
    >
    >
     

    Spectral Cube creation

    HiClass export tool

    Revision 62012-03-12 - JavierDiaz

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's coming up in HIPE 9

    Line: 123 to 123
     

    Product Access Layer

    Tasks

    Added:
    >
    >
    • Deprecated Tasks (names and parameters) appear documented in HIPE.
     

    Quality control

    Systematic product generation

    Data input-output

    Added:
    >
    >
    • Deprecation of duplicated functionalities: poolDataWriter, localStoreWriter, poolDataReader. Use getObservation and saveProduct.
    • Deprecation of MyHSA prototype duplicated functionalities: loadObs and downObs. Use getObservation
    • getObservation promoted to a Task (currently getObs).
    • simpleAsciiTableReader deprecated, use asciiTableReader.
     

    Virtual Observatory

    Revision 52012-03-07 - WimDeMeester

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's coming up in HIPE 9

    Line: 101 to 101
     
      • save(filename), saveAsEPS(filename), saveAsPNG(filename), saveAsJPG(filename) and saveAsPDF(filename) : These methods will save the full image, including annotations.
      • saveCurrentView(filename), saveCurrentViewAsEPS(filename), saveCurrentViewAsPNG(filename), saveCurrentViewAsJPG(filename) and saveCurrentViewAsPDF(filename) : These methods will save the current view of the image, including annotations and axes.
    • The coordinate axes on the image panner are always visible now
    Added:
    >
    >
    • There is a preference for image display to turn of the echoing of commands to the command line. It is possible to turn all these commands off, or only the zoom commands.
     

    Analysis

    Spectra

    Revision 42012-03-02 - StephanOtt

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's coming up in HIPE 9

    Line: 8 to 8
     
    Added:
    >
    >

    Highlights

    HIPE

    • Common look and feel for the spectrum explorer and cube analysis toolbox
    • Image analysis perspective
    • Easing the data access via MyHSA
    • Improvement of handling of ASCII tables
    • Mechanism to notify users on removed names (tasks, functions...), why they were removed, and their replacement
    • Smaller memory footprint for HIPE editor
    • Speed-up of HIPE start-up
    • Progress bar informs on progress of for-loops
    • Human-made indexes added to the core manual (HIPE Owner's Guide, Scripting Guide, Data Analysis Guide)
    • Possibility to leave comments at the bottom of any page in the HIPE Help System
    • Support to OSX Lion
    • Easy access to calibration data, including version and updates (TBS)
    • Improvements of search capability of documentation system (TBS)
    • Publication ready plots (TBS)

    HIFI

    • Scripts to permit fringe removal / current matching
    • HIFI level 2.5 product to hold data of Spectral Surveys
    • Browse products for spectral surveys in maps (TBS)

    PACS

    • Photometer: Addressing further specific cases in IPipe scripts
    • Spectrometer
      • Broadband reliability
      • Improved spectral pixel flatfield broad ranges (SED)
      • Correction Flux loss due to pointing
    • Convolution kernels for photometer and spectrometer maps aiding the interpretation of extended and point sources
    • Reduction in memory consumption (TBS)

    SPIRE

    • common (both Photometer and Spectrometer)
      • improved processing speed using multi-threading
      • improved task for correction of "cooler burps"
    • Photometer
      • Pipeline Processing
        • kernel-smoothed maps (aka "drizzling")
        • improved error maps (possibly as an option)
        • improved baseline correction using "destriper“
      • Interactive processing
        • source timeline fitter task
        • improvements to bolometer finder tool
        • selection of turnaround data in map creation
    • Spectrometer
      • Pipeline Processing: smaller size for level 1 products
      • Improved calibration and data processing
        • updated point source calibration (improved Uranus model)
        • updated bright mode calibration (better PCAL module)
        • improved calibration for Low Resolution observations (hopefully improved calibration, but at minimum, updated advice)
        • more straightforward calling sequence for phase-correction task
        • corrected spectral averaging - more flexible usage
      • Interactive processing
        • drag and drop from SDS Explorer detector layout to variables tab
        • new and updated useful scripts for background subtraction and spectral noise

     

    Core system

    HIPE

    Revision 32012-01-04 - WimDeMeester

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's coming up in HIPE 9

    Line: 35 to 35
     
    • Reworked the save methods on Display. You can now save to jpg, png, eps or pdf files. The methods to use are :
      • save(filename), saveAsEPS(filename), saveAsPNG(filename), saveAsJPG(filename) and saveAsPDF(filename) : These methods will save the full image, including annotations.
      • saveCurrentView(filename), saveCurrentViewAsEPS(filename), saveCurrentViewAsPNG(filename), saveCurrentViewAsJPG(filename) and saveCurrentViewAsPDF(filename) : These methods will save the current view of the image, including annotations and axes.
    Added:
    >
    >
    • The coordinate axes on the image panner are always visible now
     

    Analysis

    Spectra

    Revision 22011-11-23 - WimDeMeester

    Line: 1 to 1
     
    META TOPICPARENT name="HipeWhatsNew"

    What's coming up in HIPE 9

    Line: 32 to 32
     

    Images

    Display

    Added:
    >
    >
    • Reworked the save methods on Display. You can now save to jpg, png, eps or pdf files. The methods to use are :
      • save(filename), saveAsEPS(filename), saveAsPNG(filename), saveAsJPG(filename) and saveAsPDF(filename) : These methods will save the full image, including annotations.
      • saveCurrentView(filename), saveCurrentViewAsEPS(filename), saveCurrentViewAsPNG(filename), saveCurrentViewAsJPG(filename) and saveCurrentViewAsPDF(filename) : These methods will save the current view of the image, including annotations and axes.
     

    Analysis

    Spectra

    Revision 12011-10-19 - DavideRizzo

    Line: 1 to 1
    Added:
    >
    >
    META TOPICPARENT name="HipeWhatsNew"

    What's coming up in HIPE 9

    PDF Version Click here

    See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.

    Core system

    HIPE

    Plotting

    TablePlotter

    OverPlotter

    Dataset viewer

    Astronomical utilities

    Numeric routines

    Data fitting

    Data interpolation

    Random numbers

    Statistics

    Trend analysis

    Mapping

    Other functions

    Source extraction

    Images

    Display

    Analysis

    Spectra

    Display

    Analysis

    Data cubes

    Display

    Analysis

    Herschel Science Archive

    Calibration sources

    Products and datasets

    Observation context

    Product Access Layer

    Tasks

    Quality control

    Systematic product generation

    Data input-output

    Virtual Observatory

    Installer

    Documentation

    HIFI

    HIFI Software Configuration and Properties

    HIFI Calibration

    HIFI Pipeline

    hifiPipeline task

    Level 0 Pipeline

    WBS Pipeline

    HRS Pipeline

    Level 1 Pipeline

    Level 2 Pipeline

    HIFI Products

    HIFI Spectrum Toolbox

    Spectrum Explorer (HIFI specifics)

    Spectrum tasks (HIFI specifics)

    HIFI Data Processing Tools

    Standing Wave Removal

    Baseline Removal

    Deconvolution

    Spectral Cube creation

    HiClass export tool

    Documentation

    PACS

    Pipeline

    Photometry

    Spectroscopy

    Calibration

    Photometry

    Spectroscopy

    Analysis tools

    Photometry

    Spectroscopy

    Product changes

    Photometry

    Spectroscopy

    Documentation

    SPIRE

    Calibration Products

    Calibration Framework

    Preprocessing

    Engineering Conversion

    Common Pipeline

    Photometer Pipeline

    Spectrometer Pipeline

    Interactive Analysis and Tools

    Documentation

    <--  
    -->
     
    This site is powered by the TWiki collaboration platform Powered by Perl