Difference: HipeWhatsNew5x (1 vs. 66)

Revision 662012-10-30 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 7 to 7
 
PDF Version Click here
Added:
>
>
Help Interested in what's new for other versions of HIPE? See this page for links to all the What's New documents from HIPE 4.0 onwards.
 This document lists the changes in HIPE version 5.0 with respect to the 4.x series. Additional pages list changes between minor versions of the 5.x series.

Revision 652012-10-19 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
Added:
>
>
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->
 

What's New in HIPE 5

PDF Version Click here
Line: 838 to 841
  -->
Changed:
<
<
META FILEATTACHMENT attachment="toolbox.gif" attr="" comment="New spectrum toolbox icon" date="1292538464" name="toolbox.gif" path="toolbox.gif" size="973" stream="toolbox.gif" user="Main.CarolynMcCoey" version="1"
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="New spectrum toolbox icon" date="1292538464" name="toolbox.gif" path="toolbox.gif" size="973" user="Main.CarolynMcCoey" version="1"
META FILEATTACHMENT attr="h" autoattached="1" comment="Script for Google Analytics" date="1350656328" name="GoogleAnalytics.txt" path="GoogleAnalytics.txt" size="549" user="Main.DavideRizzo" version="1"

Revision 642012-05-14 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5

Line: 660 to 660
  Other small changes:
    • getCalTree(), getCalProduct() now understand the 'verbose=True' keyword and will print information on the location of the calibration products.
    • getCalProduct() the third argument, 'version', is no longer mandatory, if not given, the last version of the calibration product will be returned.
Changed:
<
<
  • A new Pacs Documentation page has been opened on the public wiki.
>
>
 

Photometry

  • MadMap
Line: 817 to 817
 
  • Changed SpecExplorer's detector display so that it automatically re-sizes to fit the enclosing frame. Also, SpecExplorer's detectors are now enclosed in a splitter pane. This enables you to maximize or hide the detector display with a single click.
  • A new SpireMultiObs task to process several observations through the pipeline at once.
Added:
>
>

<-- COMMENT BOX CODE - DO NOT EDIT -->
blog comments powered by Disqus
<-- END OF COMMENT BOX CODE -->

 
<--  
-->

Revision 632011-02-23 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5

Line: 8 to 8
 
Added:
>
>
  See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.

Revision 622011-02-07 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5

PDF Version Click here
Deleted:
<
<
See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.

 This document lists the changes in HIPE version 5.0 with respect to the 4.x series. Additional pages list changes between minor versions of the 5.x series.

Added:
>
>

See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.

 

Core system

Revision 612011-01-11 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5

Revision 602011-01-11 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5

Line: 100 to 100
 
  • Added relativistic frequency to velocity transform: When changing the unit of the wave axis to velocity the transform is now relativistic.

Analysis

  • Gaussian resampling introduced.
Added:
>
>
 

Herschel Science Archive

  • Support for exporting data in packed HSA format.

Revision 592011-01-10 - KevinEdwards

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5

Line: 559 to 559
 
  • hifiPipeline will make an ObservationContext even if creation of trend products fails
  • hifiPipeline GUI layout improved to make more efficient use of space
Added:
>
>
  • When reprocessing with the hifiPipeline task it is not recommend to set aux=True when using the HSA to retrieve your data as not all Auxiliary data products will be updated correctly. The default option of aux=False is valid for all observations created with SPG v4.0+ (which should be all HIFI observations in the HSA).
 

Level 0 Pipeline

  • H and V spectrometer astrometry

Revision 582010-12-21 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5

Line: 269 to 269
 
Aperture Efficiency (H)  
Band Frequency (GHz) Eta_A
Changed:
<
<
1a 491 0.692
1b 610 0.693
2a 648 0.689
2b 770 0.678
3a 828 0.665
3b 948 0.676
4a 1012 0.664
4b 1112 0.695
5a 1127 0.618
5b 1243 0.561
6a 1483 0.663
6b 1625 0.642
7a 1703 0.613
7b 1807 0.627
7b 1893 0.604
>
>
1a 489.0 0.676
1a 512.0 0.675
1a 548.0 0.675
1b 564.0 0.675
1b 595.0 0.674
1b 627.0 0.673
2a 641.0 0.673
2a 677.0 0.672
2a 710.0 0.671
2b 724.0 0.671
2b 756.0 0.670
2b 792.0 0.669
3a 809.0 0.669
3a 830.0 0.668
3a 850.0 0.668
3b 868.0 0.667
3b 910.0 0.666
3b 945.0 0.665
4a 959.0 0.664
4a 1006.0 0.663
4a 1052.0 0.661
4b 1064.0 0.661
4b 1086.0 0.660
4b 1112.0 0.659
5a 1118.0 0.562
5a 1179.0 0.560
5a 1240.0 0.558
5b 1148.0 0.561
5b 1207.0 0.559
5b 1268.0 0.557
6a 1431.0 0.646
6a 1503.0 0.642
6a 1574.0 0.639
6b 1579.0 0.638
6b 1637.0 0.635
6b 1694.0 0.632
7a 1708.0 0.631
7a 1768.0 0.628
7a 1813.0 0.626
7b 1720.0 0.631
7b 1815.0 0.625
7b 1897.0 0.621
 
Aperture Efficiency (V)
Band Frequency (GHz) Eta_A
Changed:
<
<
1a 491 0.656
1b 610 0.668
2a 648 0.706
2b 770 0.694
3a 828 0.725
3b 948 0.734
4a 1012 0.724
4b 1112 0.744
5a 1127 0.588
5b 1243 0.588
6a 1483 0.675
6b 1625 0.597
7a 1703 0.582
7b 1807 0.650
7b 1893 0.674
>
>
1a 489.0 0.676
1a 512.0 0.675
1a 548.0 0.675
1b 564.0 0.675
1b 595.0 0.674
1b 627.0 0.673
2a 641.0 0.673
2a 677.0 0.672
2a 710.0 0.671
2b 724.0 0.671
2b 756.0 0.670
2b 792.0 0.669
3a 809.0 0.669
3a 830.0 0.668
3a 850.0 0.668
3b 868.0 0.667
3b 910.0 0.666
3b 945.0 0.665
4a 959.0 0.664
4a 1006.0 0.663
4a 1052.0 0.661
4b 1064.0 0.661
4b 1086.0 0.660
4b 1112.0 0.659
5a 1118.0 0.562
5a 1179.0 0.560
5a 1240.0 0.558
5b 1148.0 0.561
5b 1207.0 0.559
5b 1268.0 0.557
6a 1431.0 0.646
6a 1503.0 0.642
6a 1574.0 0.639
6b 1579.0 0.638
6b 1637.0 0.635
6b 1694.0 0.632
7a 1708.0 0.631
7a 1768.0 0.628
7a 1813.0 0.626
7b 1720.0 0.631
7b 1815.0 0.625
7b 1897.0 0.621
 
Beam Efficiency (H)
Band frequency (GHz) eta_mb
Changed:
<
<
1a 491 0.719
1b 610 0.708
2a 648 0.752
2b 770 0.719
3a 828 0.716
3b 948 0.739
4a 1012 0.712
4b 1112 0.764
5a 1127 0.682
5b 1243 0.677
6a 1483 0.707
6b 1625 0.691
7a 1703 0.620
7b 1807 0.664
7b 1893 0.650
>
>
1a 489.0 0.755
1a 512.0 0.755
1a 548.0 0.754
1b 564.0 0.754
1b 595.0 0.753
1b 627.0 0.752
2a 641.0 0.752
2a 677.0 0.751
2a 710.0 0.750
2b 724.0 0.750
2b 756.0 0.749
2b 792.0 0.748
3a 809.0 0.747
3a 830.0 0.747
3a 850.0 0.746
3b 868.0 0.746
3b 910.0 0.744
3b 945.0 0.743
4a 959.0 0.742
4a 1006.0 0.741
4a 1052.0 0.739
4b 1064.0 0.738
4b 1086.0 0.738
4b 1112.0 0.737
5a 1118.0 0.639
5a 1179.0 0.637
5a 1240.0 0.635
5b 1148.0 0.638
5b 1207.0 0.636
5b 1268.0 0.634
6a 1431.0 0.722
6a 1503.0 0.718
6a 1574.0 0.714
6b 1579.0 0.713
6b 1637.0 0.710
6b 1694.0 0.707
7a 1708.0 0.706
7a 1768.0 0.702
7a 1813.0 0.699
7b 1720.0 0.705
7b 1815.0 0.699
7b 1897.0 0.694
 

Beam Efficiency (V)
Band Frequency (GHz) eta_mb
Changed:
<
<
1a 491 0.71
1b 610 0.730
2a 648 0.767
2b 770 0.739
3a 828 0.716
3b 948 0.740
4a 1012 0.697
4b 1112 0.747
5a 1127 0.705
5b 1243 0.721
6a 1483 0.738
6b 1625 0.652
7a 1703 0.720
7b 1807 0.747
7b 1893 0.716
>
>
1a 489.0 0.755
1a 512.0 0.755
1a 548.0 0.754
1b 564.0 0.754
1b 595.0 0.753
1b 627.0 0.752
2a 641.0 0.752
2a 677.0 0.751
2a 710.0 0.750
2b 724.0 0.750
2b 756.0 0.749
2b 792.0 0.748
3a 809.0 0.747
3a 830.0 0.747
3a 850.0 0.746
3b 868.0 0.746
3b 910.0 0.744
3b 945.0 0.743
4a 959.0 0.742
4a 1006.0 0.741
4a 1052.0 0.739
4b 1064.0 0.738
4b 1086.0 0.738
4b 1112.0 0.737
5a 1118.0 0.639
5a 1179.0 0.637
5a 1240.0 0.635
5b 1148.0 0.638
5b 1207.0 0.636
5b 1268.0 0.634
6a 1431.0 0.722
6a 1503.0 0.718
6a 1574.0 0.714
6b 1579.0 0.713
6b 1637.0 0.710
6b 1694.0 0.707
7a 1708.0 0.706
7a 1768.0 0.702
7a 1813.0 0.699
7b 1720.0 0.705
7b 1815.0 0.699
7b 1897.0 0.694
 
Beam Widths (H)
Band Frequency (GHz) HPBW
Changed:
<
<
1a 491 43.22154
1b 610 34.49101
2a 648 33.54672
2b 770 27.83073
3a 828 26.07131
3b 948 22.95481
4a 1012 21.29627
4b 1112 19.62355
5a 1127 19.38483
5b 1243 18.38567
6a 1483 14.48873
6b 1625 13.29430
7a 1703 12.28931
7b 1807 11.83705
7b 1893 11.39468
>
>
1a 489.0 43.362
1a 512.0 41.414
1a 548.0 38.694
1b 564.0 37.596
1b 595.0 35.637
1b 627.0 33.818
2a 641.0 33.080
2a 677.0 31.321
2a 710.0 29.865
2b 724.0 29.287
2b 756.0 28.048
2b 792.0 26.773
3a 809.0 26.210
3a 830.0 25.547
3a 850.0 24.946
3b 868.0 24.429
3b 910.0 23.301
3b 945.0 22.438
4a 959.0 22.111
4a 1006.0 21.078
4a 1052.0 20.156
4b 1064.0 19.929
4b 1086.0 19.525
4b 1112.0 19.068
5a 1118.0 18.966
5a 1179.0 17.985
5a 1240.0 17.100
5b 1148.0 18.470
5b 1207.0 17.568
5b 1268.0 16.722
6a 1431.0 14.818
6a 1503.0 14.108
6a 1574.0 13.471
6b 1579.0 13.429
6b 1637.0 12.953
6b 1694.0 12.517
7a 1708.0 12.415
7a 1768.0 11.993
7a 1813.0 11.696
7b 1720.0 12.328
7b 1815.0 11.683
7b 1897.0 11.178
 
Beam Widths (V)
Band Frequency (GHz) HPBW
Changed:
<
<
1a 491 44.30458
1b 610 35.66471
2a 648 33.47924
2b 770 27.89113
3a 828 24.98089
3b 948 22.03684
4a 1012 20.16164
4b 1112 18.74398
5a 1127 20.21393
5b 1243 18.54717
6a 1483 14.66947
6b 1625 13.38736
7a 1703 13.59736
7b 1807 12.34126
7b 1893 11.32418

Note these are still initial results, measurements from ODs 390, 391, 406 and 407 still need to be added. One can use the developer part of calibration tree, where newer values will be tested first, by setting the following property: Configuration.setProperty("hcss.ia.obs.cal.release", "developer")

>
>
1a 489.0 43.362
1a 512.0 41.414
1a 548.0 38.694
1b 564.0 37.596
1b 595.0 35.637
1b 627.0 33.818
2a 641.0 33.080
2a 677.0 31.321
2a 710.0 29.865
2b 724.0 29.287
2b 756.0 28.048
2b 792.0 26.773
3a 809.0 26.210
3a 830.0 25.547
3a 850.0 24.946
3b 868.0 24.429
3b 910.0 23.301
3b 945.0 22.438
4a 959.0 22.111
4a 1006.0 21.078
4a 1052.0 20.156
4b 1064.0 19.929
4b 1086.0 19.525
4b 1112.0 19.068
5a 1118.0 18.966
5a 1179.0 17.985
5a 1240.0 17.100
5b 1148.0 18.470
5b 1207.0 17.568
5b 1268.0 16.722
6a 1431.0 14.818
6a 1503.0 14.108
6a 1574.0 13.471
6b 1579.0 13.429
6b 1637.0 12.953
6b 1694.0 12.517
7a 1708.0 12.415
7a 1768.0 11.993
7a 1813.0 11.696
7b 1720.0 12.328
7b 1815.0 11.683
7b 1897.0 11.178
 

HIFI Pipeline

Line: 424 to 584
 

HIFI Data Processing Tools

  • PolarPairs now has a tolerance of 100. Previously it as 5 and this sometimes caused the task to fail resulting in the output being identical to the first input product.
Added:
>
>
  • A task to allow conversion of Kelvin to Jansky, ConverK2Jy, is introduced
 

Standing Wave Removal

Line: 438 to 599
 
  • Introduction of a new task, FitBaseline, to interactively fit and subtract or divide out baselines in HIFI spectra.
  • A bug that appeared in early versions of HIPE 5 where domask=2 in did not work in FitBaseline (preventing automatic line masking) now fixed.
  • A bug that appeared in early versions of HIPE 5 in which fitBaseline would not save on an intermediate quit is resolved.
Added:
>
>
  • The fitting routine in fitBaseline has a condition set such that no polynomials of order n are fitted if less than n+1 channels are available. This prevents crashes of the task in the case of heavily flagged data.
 

Deconvolution

Line: 449 to 611
 
  • HiClass will now appear an an applicable task for HTP as well as ObservationContexts
  • HiClass now exports the correct frequencies for bands 6 and 7
Changed:
<
<
  • Bug that appeared in early versions of HIPE 5 preventing export of Spectral Scans now fixed
>
>
  • A bug that appeared in early versions of HIPE 5 preventing export of Spectral Scans now fixed
  • HiClass now exports the source velocity as provided by the astronomer in the AOR into a new FITS header keyword. At this time, the CLASS software will not do anything with this information, but the requirement to provide ALL the information is satisfied by having the velocity information present in the FITS header somewhere. The keywords used, which are slightly different than the usual ones used to reflect the fact that only the velocity in the proposal is exported, are tabulated below.

reference frame usual keyword proposed keyword
topocentric VELO-OBS PVEL-OBS
geocentric VELO-GEO PVEL-GEO
heliocentric VELO-HEL PVEL-HEL
LSRk VELO-LSR PVEL-LSR
LSRd VELO-LSR PVEL-LSR
LSR VELO-LSR PVEL-LSR
galactocentric VELO-GAL PVEL-GAL
source VELOREST PVELREST
unknown VELOCITY PVELCITY

These keyword takes care of the value, the unit and the reference frame. The PVELTYPE keyword will identify the type of 'velocity' (velocity, redshift, radio or optical convention).

 

HIFI Documentation

  • New chapters in the HIFI User's Manual: What was done to my data? and HIFI Baseline Removal.
Changed:
<
<
  • Updates to the HIFI Users Manual: HIFI cookbook, Data Primer, Running the HIFI Pipeline, HIFI Standing Wave Removal Tool and Exporting HIFI data to CLASS.
>
>
  • Updates to the HIFI Users Manual: HIFI cookbook, Data Primer, Running the HIFI Pipeline, HIFI Standing Wave Removal Tool , How to make Spectral Cubes and Exporting HIFI data to CLASS.
 
  • Updates to HIFI Pipeline Specification document: improved pipeline flow diagrams, improved information about Euler resampling used in DoFreq grid task of pipeline.
  • Creation of FitHifiFringe URM entry.
  • The HIFI cookbook can now be selected from the Help menu in the HIPE toolbar under Help -> Data Reduction -> HIFI

Revision 572010-12-16 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5

Line: 53 to 53
 

Numeric routines

Data fitting

Changed:
<
<
  • A complete new GUI for the SpectrumFitter was added. Start it from the SpectrumExplorer right mouse menu, select SpectrumFitterGUI. Besides the functionality that was available in the SpectrumFitterTool, the GUI has an export tab that enables you to export the residual, total model, and individual models as an object with the type of the input object and as text.
>
>
  • A complete new GUI for the SpectrumFitter was added. Start it from the spectrum toolbox menu, select SpectrumFitterGUI. Besides the functionality that was available in the SpectrumFitterTool, the GUI has an export tab that enables you to export the residual, total model, and individual models as an object with the type of the input object and as text.
 
  • The SpectrumFitter (commandline version) accepts Container with PointSpectrum and SpectralSegment index.
  • The SpectrumFitter (commandline version) has methods to export the data as the input type and as text.
  • The SpectrumFitter (commandline version) has method that returns a Jython script.
Line: 92 to 92
 

Spectra

Display

  • Task & Tools Toolbox: All task and tool panels are displayed in the same toolbox panel and selectable through a drop-down list. The task GUI components have become clearer as well.
Changed:
<
<
Instead of dragging a selection of spectra to the dataset parameter, the task is applied on either the displayed or the selected spectra within the plot.
>
>
  • The new spectrum toolbox panel is opened by clicking on toolbox.gif in the toolbar
  • Instead of dragging a selection of spectra to the dataset parameter, the task is applied on either the displayed or the selected spectra within the plot.
 
  • Load selectable datasets from a product: When opening the spectrum explorer on a product, a selection table is displayed showing all datasets that are stored in this product along with certain characteristics. Just as within the original spectrum selection panel, these datasets can be sorted by clicking on the header columns. To load a set of datasets instead of just one or all, select each row and right click to load the selected datasets.
  • Realtime statistics information: Selecting the Statistics task within the Toolbox displays realtime statistics while dragging ranges in the plot.
Line: 410 to 411
 

Level 2 Pipeline

  • DoMainBeamTemp now multiplies by the forward efficiency. In earlier versions of HIPE this was not an issue because a value of Feff=1 was assumed. This update was required as a consequence of the adoption of Feff=0.96 in the calibration tree.
Added:
>
>
  • The ra and dec in cube meta data is now set to be the centre of the cube. Previously the ra and dec in the first dataset of the HTP had been used.
 

HIFI Products

  • Modifications to HifiSpectrumDatasets are now reflected in SpectrumExplorer when the HTP is viewed there.
Added:
>
>
  • It is now possible to create a SimpleSpectrum from a HIFI product or dataset containing only one spectrum. A SimpleSpectrum is easily written to FITS and maintains metadata. The conversion is done as:
simpleSpectrum = convertSingleHifiSpectrum(spectra=data)
 
Deleted:
<
<

HIFI Data Processing Tools

 
Added:
>
>

HIFI Data Processing Tools

  • PolarPairs now has a tolerance of 100. Previously it as 5 and this sometimes caused the task to fail resulting in the output being identical to the first input product.
 

Standing Wave Removal

Line: 430 to 436
 

Baseline removal

  • Introduction of a new task, FitBaseline, to interactively fit and subtract or divide out baselines in HIFI spectra.
Changed:
<
<
  • Bug that appeared in early versions of HIPE 5 where domask=2 in did not work in FitBaseline (preventing automatic line masking) now fixed.
>
>
  • A bug that appeared in early versions of HIPE 5 where domask=2 in did not work in FitBaseline (preventing automatic line masking) now fixed.
  • A bug that appeared in early versions of HIPE 5 in which fitBaseline would not save on an intermediate quit is resolved.
 

Deconvolution

Line: 450 to 457
 
  • Updates to the HIFI Users Manual: HIFI cookbook, Data Primer, Running the HIFI Pipeline, HIFI Standing Wave Removal Tool and Exporting HIFI data to CLASS.
  • Updates to HIFI Pipeline Specification document: improved pipeline flow diagrams, improved information about Euler resampling used in DoFreq grid task of pipeline.
  • Creation of FitHifiFringe URM entry.
Added:
>
>
  • The HIFI cookbook can now be selected from the Help menu in the HIPE toolbar under Help -> Data Reduction -> HIFI
 

PACS

Line: 631 to 639
 
<--  
-->
\ No newline at end of file
Added:
>
>
META FILEATTACHMENT attachment="toolbox.gif" attr="" comment="New spectrum toolbox icon" date="1292538464" name="toolbox.gif" path="toolbox.gif" size="973" stream="toolbox.gif" user="Main.CarolynMcCoey" version="1"

Revision 562010-12-15 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5

Line: 8 to 8
 
Added:
>
>
This document lists the changes in HIPE version 5.0 with respect to the 4.x series. Additional pages list changes between minor versions of the 5.x series.

 

Core system

HIPE

Revision 552010-12-01 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5

Line: 232 to 232
 
  • Unnecessary and deprecated properties files have been removed from the system

HIFI Calibration

Added:
>
>
  • The Spur table has been updated to reflect the improvements in band 1a. For data taken after OD 516 the spur table now only shows three entries:
    • 1a 537500.00 3000.0 LO_settling 1,2,3,4
    • 1a 540929.25 1000.0 Strong_spur 1
    • 1a 542306.25 1000.0 Weak_spur 1
 
  • The minimum and maximum chopper threshold values have been changed.
    • HkThresholdsContext_H fpuChopperMin -> -0.04
    • HkThresholdsContext_V fpuChopperMin -> -0.04
Line: 370 to 375
 Note these are still initial results, measurements from ODs 390, 391, 406 and 407 still need to be added. One can use the developer part of calibration tree, where newer values will be tested first, by setting the following property: Configuration.setProperty("hcss.ia.obs.cal.release", "developer")
Deleted:
<
<

Level 0 Pipeline

  • H and V spectrometer astrometry
SpectrumDatasets contain columns, in decimal degrees, for the commanded pointing (a position between the H and V beams known as the synthesized aperture), labelled longitude_cmd and latitude_cmd, while the actual coordinates for the spectrometer beam are found in the longitude and latitude columns.
  • DoPointing no longer fails when reapplied

WBS Pipeline

 
Deleted:
<
<
  • Fixes for frequency calibration in the case that line centres fall precisely in the middle of two pixels
 
Changed:
<
<

hifiPipelineTask

>
>

HIFI Pipeline

hifiPipelineTask

 
  • The path/filename for a user defined/edited pipeline algorithm can now be passed to the hifiPipeline task via the command line, previously this was only possible in the GUI.
Line: 396 to 394
 
  • hifiPipeline will make an ObservationContext even if creation of trend products fails
  • hifiPipeline GUI layout improved to make more efficient use of space
Changed:
<
<

DoDeconvolution

>
>

Level 0 Pipeline

 
Changed:
<
<
  • More understandable error message when deconvolution fails (usual due to poor data input).
  • Plotting of DSB solution as deconvolutions runs now possible when diag_mode_on selected.
>
>
  • H and V spectrometer astrometry
SpectrumDatasets contain columns, in decimal degrees, for the commanded pointing (a position between the H and V beams known as the synthesized aperture), labelled longitude_cmd and latitude_cmd, while the actual coordinates for the spectrometer beam are found in the longitude and latitude columns. Note that HIPE 5 RC 9 and later versions only should be used for this.

WBS Pipeline

  • Fixes for frequency calibration in the case that line centres fall precisely in the middle of two pixels

Level 2 Pipeline

  • DoMainBeamTemp now multiplies by the forward efficiency. In earlier versions of HIPE this was not an issue because a value of Feff=1 was assumed. This update was required as a consequence of the adoption of Feff=0.96 in the calibration tree.

HIFI Products

  • Modifications to HifiSpectrumDatasets are now reflected in SpectrumExplorer when the HTP is viewed there.
 
Changed:
<
<

FitHifiFringe

>
>

HIFI Data Processing Tools

Standing Wave Removal

 
  • It is now possible to remove the smoothed baseline calculated by FitHifiFringe from the spectrum, in addition to the fitted sine waves. This option (sub_base) should be used with care as real spectral features can be removed this way. For interactive fitting of polynomial baselines, the new task FitBaseline is recommended.
Changed:
<
<
  • Tooltips implemented
  • Improved error messaging
>
>
  • Tooltips implemented in the FitHifiFringe GUI
  • Improved error messaging from FitHifiFringe
  • Bug that appeared in early versions of HIPE 5 where usermask max/min boxes in FitHifiFringe GUI disappeared fixed.

  • A Deep Fourier Transform (DFT) module is available (not a part of FitHifiFringe)

Baseline removal

  • Introduction of a new task, FitBaseline, to interactively fit and subtract or divide out baselines in HIFI spectra.
  • Bug that appeared in early versions of HIPE 5 where domask=2 in did not work in FitBaseline (preventing automatic line masking) now fixed.

Deconvolution

  • More understandable error message when deconvolution fails (usual due to poor data input).
  • Plotting of DSB solution as deconvolutions runs now possible when diag_mode_on selected.
 

HiClass export tool

  • HiClass will now appear an an applicable task for HTP as well as ObservationContexts
  • HiClass now exports the correct frequencies for bands 6 and 7
Added:
>
>
  • Bug that appeared in early versions of HIPE 5 preventing export of Spectral Scans now fixed
 
Changed:
<
<

HIFI DP scripts (User tool scripts)

  • Introduction of a new task, FitBaseline, to interactively fit and subtract or divide out baselines in HIFI spectra.

Documentation

>
>

HIFI Documentation

 
  • New chapters in the HIFI User's Manual: What was done to my data? and HIFI Baseline Removal.
  • Updates to the HIFI Users Manual: HIFI cookbook, Data Primer, Running the HIFI Pipeline, HIFI Standing Wave Removal Tool and Exporting HIFI data to CLASS.

Revision 542010-12-01 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5

Line: 410 to 410
 

HiClass export tool

  • HiClass will now appear an an applicable task for HTP as well as ObservationContexts
Added:
>
>
  • HiClass now exports the correct frequencies for bands 6 and 7
 

HIFI DP scripts (User tool scripts)

Revision 532010-11-25 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5

PDF Version Click here
Added:
>
>
See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.
 

Core system

Revision 522010-11-22 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
Changed:
<
<

What's New in HIPE 5.0

>
>

What's New in HIPE 5

 
PDF Version Click here
Line: 40 to 40
 
  • Title and label can now accept the LaTeX commands \textrm \textit \textbf \mathrm \mathit \mathbf. For example: V$_{\textrm{LSR}}$
Deleted:
<
<

TablePlotter

OverPlotter

Dataset viewer

 

Astronomical utilities

  • RadialVelocity now uses TimeCorrProduct to select pointing data.
  • SSO now uses TimeCorrProduct to select pointing data.
Line: 70 to 66
 
  • Covariance, which returns the covariance between two random variables/vectors x and y with finite second moments. If one vector is longer than the other, only the values up to the length of the shorter vector will be taken into account.
  • CovarianceMatrix, which returns the covariance matrix of the input M x N matrix. The result is a N x N matrix with each i, j value equal to the covariance of the ith and jth columns of the original matrix.
Deleted:
<
<

Trend analysis

Mapping

 

Other functions

  • SigClip
    • Median mode uses median absolute deviation instead of standard deviation.
Line: 100 to 94
 

Analysis

  • Gaussian resampling introduced.
Deleted:
<
<

Data cubes

Display

Analysis

 

Herschel Science Archive

  • Support for exporting data in packed HSA format.
  • Import and Export Views improvements:
Line: 207 to 197
 
    • HeaderWriter exceptions have been modified to include more information.
    • Documentation updates.
Deleted:
<
<

Virtual Observatory

 

Installer

There was a major restructuring of the pre-installation steps:
  • The Basic/Advanced selection page has been removed.
Line: 386 to 374
 SpectrumDatasets contain columns, in decimal degrees, for the commanded pointing (a position between the H and V beams known as the synthesized aperture), labelled longitude_cmd and latitude_cmd, while the actual coordinates for the spectrometer beam are found in the longitude and latitude columns.
  • DoPointing no longer fails when reapplied
Deleted:
<
<

HRS Pipeline

 

WBS Pipeline

  • Fixes for frequency calibration in the case that line centres fall precisely in the middle of two pixels
Deleted:
<
<

Level 1 Pipeline

Level 2 Pipeline

 

hifiPipelineTask

  • The path/filename for a user defined/edited pipeline algorithm can now be passed to the hifiPipeline task via the command line, previously this was only possible in the GUI.
Line: 412 to 394
 
  • hifiPipeline will make an ObservationContext even if creation of trend products fails
  • hifiPipeline GUI layout improved to make more efficient use of space
Deleted:
<
<

Products

 

DoDeconvolution

  • More understandable error message when deconvolution fails (usual due to poor data input).
  • Plotting of DSB solution as deconvolutions runs now possible when diag_mode_on selected.
Deleted:
<
<

DoGridding

 

FitHifiFringe

  • It is now possible to remove the smoothed baseline calculated by FitHifiFringe from the spectrum, in addition to the fitted sine waves. This option (sub_base) should be used with care as real spectral features can be removed this way. For interactive fitting of polynomial baselines, the new task FitBaseline is recommended.
Line: 431 to 409
 
  • HiClass will now appear an an applicable task for HTP as well as ObservationContexts
Deleted:
<
<

Flags

SpectrumExplorer (HIFI specifics)

 

HIFI DP scripts (User tool scripts)

  • Introduction of a new task, FitBaseline, to interactively fit and subtract or divide out baselines in HIFI spectra.
Line: 446 to 420
 
  • Updates to HIFI Pipeline Specification document: improved pipeline flow diagrams, improved information about Euler resampling used in DoFreq grid task of pipeline.
  • Creation of FitHifiFringe URM entry.
Deleted:
<
<

Other

 

PACS

Pipeline

Line: 548 to 520
 

Analysis tools

Deleted:
<
<

Photometry

 

Spectrometry

  • Compatibility of PACS products with HCSS tools improved: you can now properly select data points and use the fitter tool on PACS frames and cubes.
Deleted:
<
<

Product changes

Photometry

Spectrometry

Documentation

Other

 

SPIRE

Calibration Products

Line: 590 to 552
 
  • Updated to match the new products and dependencies.
  • The spireCal task now has an option to read from a local pool.
Deleted:
<
<

Preprocessing

Engineering Conversion

 

Common Pipeline

  • Minor changes in logs and documentation.
Line: 639 to 597
 
  • Changed SpecExplorer's detector display so that it automatically re-sizes to fit the enclosing frame. Also, SpecExplorer's detectors are now enclosed in a splitter pane. This enables you to maximize or hide the detector display with a single click.
  • A new SpireMultiObs task to process several observations through the pipeline at once.
Deleted:
<
<

Documentation

Other

 
<--  
-->

Revision 512010-11-16 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 131 to 131
 

Observation context

Changed:
<
<
  • Include new metadata slewTime returning the scheduled start time of the slew as a FineTime.
>
>
  • Included new metadata slewTime returning the scheduled start time of the slew as a FineTime.
 
    • Specific accessors, getSlewStartTime() and setSlewStartTime(), where added to the ObservationContext class.
    • For old observations in the HSA without this value, the method will return a null value indicating that this field was never set before.

History

Changed:
<
<
A new history dataset viewer is available: When you click on the History dataset in your product then two tables appear above another:
>
>
A new history dataset viewer is available: When you click on the History dataset in your product two tables appear:
 
  • An overview table which shows all the tasks used to generate the product. For each task, the name, execution time, build number and used calibration files are shown.
  • When you click on a row in this overview table then a second table is shown with details for the selected task. This table contains the details of this task: For each parameter its simple value and some product information is shown (when available).
Line: 145 to 145
 

Cache

Changed:
<
<
Several bugfixes to improve the effectiveness of caching and the robustness has been introduced. One of these is a check to verify that the default LocalStore directory is not modified at any time during the cache's existence, which is a common source of corruption of the cache.
>
>
Several bugfixes to improve the effectiveness and robustness of caching have been introduced. One of these is a check to verify that the default LocalStore directory is not modified at any time during the cache's existence, which is a common source of corruption of the cache.
 

HsaPool

  • HsaReadPool/HsaXmlPool can work with compressed data.
Line: 156 to 156
 Product storages and pools can be managed in the new Data Access > Storages & Pools preferences panel.

Tasks

Changed:
<
<
  • Updated task sample code
>
>
  • Updated task sample code.
 
  • Updated documentation:
Changed:
<
<
    • wiki chapters
>
>
    • Wiki chapters
 
    • Javadoc of general tasks
Changed:
<
<
    • Standardised _ doc _ of general tasks (signature, description, parameter descriptions)
>
>
    • Standardised __ doc __ of general tasks (signature, description, parameter descriptions)
 
  • Tasks View provides tooltips with the signature of the tasks
  • Examples of general tasks in the URM are now tested automatically
Changed:
<
<
  • Compress and Decompress tasks: they support zip, tar, gzip, tgz in a portable way and decompress can guess the compression of a file.
>
>
  • Compress and Decompress tasks: they support zip, tar, gzip, tgz in a portable way. Decompress can guess the compression of a file.
 
  • FitsReader supports compressed FITS files.

Quality control

Line: 173 to 173
 
FITS keyword Value
PCAVEATS Please refer to http://herschel.esac.esa.int/DpKnownIssues.shtml for known problems in products
Changed:
<
<
  • Perform bulk actions on several quality control reports simultaneously
  • Flags colour codes depending on their importance
>
>
  • Perform bulk actions on several quality control reports simultaneously.
  • Flags colour codes depending on their importance.
 

Systematic product generation

Changed:
<
<
  • New framework to process/combine data from several observations in the same pipeline
    • This will allow, for instances, better results using MADmap algorithm with cross-scan data from two or more observations
>
>
  • New framework to process/combine data from several observations in the same pipeline.
    • This will allow, for instances, better results using MADmap algorithm with cross-scan data from two or more observations.
 
  • Automatic processing:
Changed:
<
<
    • New start-time algorithm based on the DTCP time of the OD being processed instead of fixed timestamps
    • New check on TM data based on the gap checker reports. Only those observations with complete tm data will be processed at this point
>
>
    • New start-time algorithm based on the DTCP time of the OD being processed instead of fixed timestamps.
    • New check on TM data based on the gap checker reports. Only those observations with complete tm data will be processed at this point.
 
  • Uplink plugin is executed without problems in different conditions, even for manual commanding and no uplink data at all.
Changed:
<
<
  • Obsids shown also in decimal format in all the application tables
  • Include proper motion data into the ObservationContext
  • SPG stand-alone application can now run on MAC O.S. with no versant libraries available.
  • On-demand request are now recorded into a report file
>
>
  • Obsids shown also in decimal format in all the application tables.
  • Include proper motion data into the ObservationContext.
  • SPG stand-alone application can now run on Mac OS with no Versant libraries available.
  • On-demand request are now recorded into a report file.
 

Data input-output

Changed:
<
<
  • Imporved error reporting in AsciiTableTool
  • FitsArchive
>
>
  • Improved error reporting in AsciiTableTool.
  • FitsArchive:
 
    • FITS Metadata direct access.
    • ESO Hierarch keyword implemented when reading FITS files.
    • FITS translation dictionary updated:
Line: 206 to 206
 
    • FitsArchive support reading gzipped FITS files and compressed stream headers.
    • HeaderWriter exceptions have been modified to include more information.
    • Documentation updates.
Deleted:
<
<
    • JExamples automatic testing support.
 

Virtual Observatory

Installer

Changed:
<
<
There was a major restructuring of the pre-installation steps
  • The Basic/Advance selection page has been removed.
  • The memory, proxy and versant settings were restructured. There are no separate panels asking if the user wants to set a proxy or a versant. There is a dedicated page for the proxy setting and one for the versant settings. In the top of the panel there is the question with a check box. Then there are the text fields not editable by default which will become editable upon user action on the above check box.
  • Instead of the old instrument selection page, the user will now be offered the following list with check boxes for selection:
>
>
There was a major restructuring of the pre-installation steps:
  • The Basic/Advanced selection page has been removed.
  • The memory, proxy and Versant settings were restructured. There are no separate panels asking if you want to set a proxy or a Versant database. There is a dedicated page for the proxy setting and one for the Versant settings.
  • Instead of the old instrument selection page, you are now offered the following list with check boxes for selection:
 
Option Description
PACS (preselected by default) PACS pipelines and analysis tools for astronomers.
PACS expert Extra instrument-related applications for Calibration Scientists.
Line: 224 to 222
 
HIFI (preselected by default) HIFI pipelines and analysis tools for astronomers.
HIFI expert Extra instrument-related applications for Calibration Scientists.
Changed:
<
<
  • The instrument selection window should allow to click in any combination of items to install those components.
  • It should also allow the deselection of all instruments to get HIPE framework only for FITS i/o and numerics analysis.
  • If possible, the installer should inform the user of the amount of data to download for the selected combination of modules.
  • The Proxy selection page should contain the Yes/No button and the entry fields, which are kept in grey until "Yes" is selected. The default is kept to "No".
  • The Versant selection page should contain the Yes/No button and the entry fields, which are kept in grey until "Yes" is selected. The default is kept to "No".
  • The rest of the pages should stay as they are.
>
>
  • The instrument selection window allows you to click in any combination of items to install those components.
  • You can also deselect all the instruments to get a barebone HIPE framework only for FITS I/O and numeric analysis.
  • If possible, the installer informs you of the amount of data to download for the selected combination of modules.
 

Documentation

Changed:
<
<
To improve the quality of the examples, we have introduced automatic testing of (many of) the source code examples occurring in manuals such as the Data Analysis Guide and the HIPE Owner's Guide. We have also introduced automatic testing of the examples occurring in the User's Reference Manual.
>
>
To improve the quality of the examples, we have introduced automatic testing of (many of) the source code examples occurring in manuals such as the Data Analysis Guide. We have also introduced automatic testing of the examples occurring in the User's Reference Manual.
 

HIFI

HIFI Software Configuration and Properties

  • Generally most HIFI properties have moved from the hcss.dp.hifi build to the hcss.icc.hifi (expert) area and it is recommended that all internal ICC users always install the expert build in order to be able to run the pipeline from level -1.0.
Changed:
<
<
  • Those using the expert build and wishing to run the pipeline from level zero will need to set the property var.hcss.instrument = hifi ' this so the system knows to use the HIFI binstruct settings for reading the packets. Note those using the legacy hifi-new.props file will already have this property set.
>
>
  • Those using the expert build and wishing to run the pipeline from level zero will need to set the property var.hcss.instrument = hifi ' this so the system knows to use the HIFI binstruct settings for reading the packets. Those using the legacy hifi-new.props file will already have this property set.
 
  • The hcss.dp.hifi (non-expert) build has pre-configured pools to access the HSA and the local pool via the Product Browser and PAL Storage Manager
Line: 267 to 261
 
    • beamWidths-V

The approach so far is to strictly use observed numbers (see tables below with initial results), without involving fitting results or empirical formulas. In particular this means:

Changed:
<
<
    • for beam widths the fitted widths of circular Gaussians, corrected for the convolution with the Mars disk. See Raphael Moreno's report for details. Open question: is the fit of a circular Gaussian correct? Need to check by how much Mars moved during an integration.
    • for main beam efficiencies the values derived using the observed beam widths for the beam coupling. It is not yet decided if an average value or a linear interpolation between (and beyond) frequencies produces the best results
    • for aperture efficiencies the values are derived, again, using observed beam widths for the coupling of the beam to the source.
>
>
    • For beam widths the fitted widths of circular Gaussians, corrected for the convolution with the Mars disk. Open question: is the fit of a circular Gaussian correct? Need to check by how much Mars moved during an integration.
    • For main beam efficiencies the values derived using the observed beam widths for the beam coupling. It is not yet decided if an average value or a linear interpolation between (and beyond) frequencies produces the best results
    • For aperture efficiencies the values are derived, again, using observed beam widths for the coupling of the beam to the source.
  In addition, the already available entries for the forward efficiencies (H and V) are updated from 1.0 to 0.96 (in all bands, all frequencies) to be compatible with above results. This requires the change of constant 1.0 to 0.96 .
Line: 390 to 384
 
  • H and V spectrometer astrometry
SpectrumDatasets contain columns, in decimal degrees, for the commanded pointing (a position between the H and V beams known as the synthesized aperture), labelled longitude_cmd and latitude_cmd, while the actual coordinates for the spectrometer beam are found in the longitude and latitude columns.
Changed:
<
<
>
>
  • DoPointing no longer fails when reapplied
 

HRS Pipeline

Line: 422 to 416
 

DoDeconvolution

Changed:
<
<
  • More understandable error message when deconvolution fails (usual due to poor data input)
  • Plotting of DSB solution as deconvolutions runs now possible when diag_mode_on selected
>
>
  • More understandable error message when deconvolution fails (usual due to poor data input).
  • Plotting of DSB solution as deconvolutions runs now possible when diag_mode_on selected.
 

DoGridding

Line: 445 to 439
 
  • Introduction of a new task, FitBaseline, to interactively fit and subtract or divide out baselines in HIFI spectra.
Deleted:
<
<
 

Documentation

Changed:
<
<
  • New chapters in the HIFI User's Manual: "What was done to my data?" and "HIFI Baseline Removal"
  • Updates to the HIFI Users Manual: "HIFI cookbook", "Data Primer", "Running the HIFI Pipeline", "HIFI Standing Wave Removal Tool", and "Exporting HIFI data to CLASS"
  • Updates to HIFI Pipeline Specification document: improved pipeline flow diagrams, improved information about Euler resampling used in DoFreq grid task of pipeline
  • Creation of FitHifiFringe URM entry
>
>
  • New chapters in the HIFI User's Manual: What was done to my data? and HIFI Baseline Removal.
  • Updates to the HIFI Users Manual: HIFI cookbook, Data Primer, Running the HIFI Pipeline, HIFI Standing Wave Removal Tool and Exporting HIFI data to CLASS.
  • Updates to HIFI Pipeline Specification document: improved pipeline flow diagrams, improved information about Euler resampling used in DoFreq grid task of pipeline.
  • Creation of FitHifiFringe URM entry.
 

Other

PACS

Pipeline

Changed:
<
<
  • PACS "New Style Sliced" pipeline implementation is now activated
    • Phot and spec ipipe script names were changed to be more logic and intuitive
    • Pipeline Script access
>
>
  • PACS "New Style Sliced" pipeline implementation is now activated.
    • Phot and spec ipipe script names were changed to be more logic and intuitive.
    • Pipeline Script access:
 
      • HIPE -> Menu -> pipeline -> PACS
Changed:
<
<
  • Calibration framework
>
>
  • Calibration framework:
  Calibration products are no longer delivered with the build in a local store.
Calibration products are now delivered as a directory of FITS files.
This directory with calibration products can be found in the data/pcal sub-directory of your installation directory.
Line: 471 to 464
  CalTree read from: /Software/Herschel/hcss.dp.pacs-5.0.500/data/pcal The calibration framework is adapted and will read these FITS files by default.
Changed:
<
<
The getCalTree() command and all access to calibration data has not changed, under the hood, the local store has been replaced by the directory of FITS files.
>
>
The getCalTree() command and all access to calibration data has not changed, under the hood, the local store has been replaced by the directory of FITS files.
  Other small changes:
    • getCalTree(), getCalProduct() now understand the 'verbose=True' keyword and will print information on the location of the calibration products.
    • getCalProduct() the third argument, 'version', is no longer mandatory, if not given, the last version of the calibration product will be returned.
Changed:
<
<
>
>
  • A new Pacs Documentation page has been opened on the public wiki.
 

Photometry

  • MadMap
Line: 491 to 484
 
  • New script for scan Map pipeline
    • Two-stage photometer pipeline with masked highpass filtering
Changed:
<
<
      • Now THE standard processing script for scan maps
>
>
      • Now the standard processing script for scan maps
 
  • Photometer Mapping
    • Photometer mapping accepts a variable input pixelsize (parameter pixfrac in the PhotProjectTask and the MapIndexTask).
    • The MapIndexViewer has a Sigclip gui to display the effect of different Sigclip parameters on the second level deglitching. Now it is not necessary any more to guess Sigclip parameters, run the deglitching and check the quality afterwards. The new option allows to find the best parameter quickly
Changed:
<
<
BEFORE deglitching has been applied.
>
>
before deglitching has been applied.
  The MapIndexViewer now also displays the signal vectors in the "timeordered" mode.
    • The MapIndexViewer shows mean/median +/- error permanently in the signal plot
    • The Full MapIndex needs less memory than before
    • Wcs4mapTask got a crota option to manually set the requested rotation angle of a map.
Changed:
<
<
    • CorrectRaDec4SsoTask is now also accepting the new Horizons class definition. An example how to use it can be found in the attached
    • photProject error map has been checked for NaN
    • Exposure map has been removed in favour of the coverage map
  • Noise treatmement
    • A new module herschel.pacs.spg.phot.PhotAddNoisePerPixelTask provides 4 ways to calculate the initial photometer noise.
>
>
    • CorrectRaDec4SsoTask is now also accepting the new Horizons class definition. An example how to use it can be found in the attached.
    • photProject error map has been checked for NaN.
    • Exposure map has been removed in favour of the coverage map.
  • Noise treatment
    • A new module herschel.pacs.spg.phot.PhotAddNoisePerPixelTask provides four ways to calculate the initial photometer noise.
  *Noise propagation is also implemented in the HighpassFilter, Mean- and MedianFilters.
  • Pointing
Changed:
<
<
    • New Horizons Product used in PACS pipeline (SSO observations)
    • Residual spatial offset PACS photometer blue/green vs red
    • Corners shall follow changes in PhotAssignRaDec
    • Separate blue and green in ArrayInstrument calfile and the code using it
>
>
    • New Horizons Product used in PACS pipeline (SSO observations).
    • Residual spatial offset PACS photometer blue/green vs red.
    • Corners shall follow changes in PhotAssignRaDec.
    • Separate blue and green in ArrayInstrument calfile and the code using it.
 
    • Astrometric offset between two scan maps in parallel mode on the same field.
  • Others
Changed:
<
<
    • PACS EDP: phot pipeline adaptation to cope with variable scan velocity
>
>
    • PACS EDP: phot pipeline adaptation to cope with variable scan velocity.
 
    • A new module photMaskFrames offer masking :
    • framesOut = photMaskFrames(framesIn [,beforeFirstScanLeg] [,noScanData] [,afterCalblock=0] )
      • beforeFirstScanLeg : Mask data befor the first scan leg
Changed:
<
<
      • noScanData : mask data where the ScanBBID is not set (includes turn arounds)
>
>
      • noScanData: mask data where the ScanBBID is not set (includes turn arounds)
 
      • afterCalblock : Mask n data after calibration block
    • Sigclip can now be applied n-times in one go.

Spectrometry

  • new AOR mode "unchopped" including improved spatial calibration:
Changed:
<
<
    • First draft of the unchopped mode pipeline added to the build
      • A "newstyle" unchopped mode pipeline is in place which can be run automatically or interactively
    • New task specSubtractOffPosition implemented for unchopped pipeline
  • new interactive tool SpectrumExplorer:
  • improved wavelength calibration:
    • New wavelength calibration applied in waveCalc task using the wavePolynomes cal file
>
>
    • First draft of the unchopped mode pipeline added to the build.
      • A "newstyle" unchopped mode pipeline is in place which can be run automatically or interactively.
    • New task specSubtractOffPosition implemented for unchopped pipeline .
  • New interactive tool SpectrumExplorer:
    • PACS Product interface to SpectrumExplorer
  • Improved wavelength calibration:
    • New wavelength calibration applied in waveCalc task using the wavePolynomes cal file.
 
  • spectrometer trend analysis product for the calibration blocks:
Changed:
<
<
    • Calibration block trendAnalysis product added to the observation context
  • Extra options to specDiffChop (note these are EXPERT options for the analysis of special observations!)
>
>
    • Calibration block trendAnalysis product added to the observation context.
  • Extra options to specDiffChop (note these are expert options for the analysis of special observations!).
 
    • Extra option 'extend' added which allows for other pairwise differencing schemes. These schemes will subtract more datapoints from each other than the standard scheme.
    • Extra option 'chopNodScheme' which provides different scheme for determining which label is on-source and which is off-source for a given nodding position. Needed for some special test AORs.
  • Different bands in a range scan are now stored in their own blocks in the block table. This means that it is now possible to slice SEDs per band.
Changed:
<
<
  • Several fixes to specWaveRebin, etc. to reduce the amount of NaNs considerably in the rebinned data.
  • SpecProject sets the flux to Jy.
>
>
  • Several fixes to specWaveRebin, etc. to reduce the amount of NaNs considerably in the rebinned data.
  • SpecProject sets the flux to Jy.
 

Calibration

Photometry

Changed:
<
<
  • Residual spatial offset PACS photometer blue/green vs red --> PCalPhotometer_ArrayInstrument_FM_v6.fits
>
>
  • Residual spatial offset PACS photometer blue/green vs red --> PCalPhotometer_ArrayInstrument_FM_v6.fits.
 

Spectrometry

Changed:
<
<
  • New Wavelength Calibration Product
>
>
  • New Wavelength Calibration Product.
 
  • Version 8 of the Pacs Spectrometer Array to Instrument and version 4 of the Module2Array cal file are inserted in the system. This generates correct values also for center chopper positions (unchopped and waveswitch mode).
Changed:
<
<
  • Updated spectrometer spatial cal file --> PCalSpectrometer_ArrayInstrument_FM_v6.fits
  • The RSRF calibration tables contain "NaN" which are then introduced into the signal by rsrfCal
>
>
  • Updated spectrometer spatial cal file --> PCalSpectrometer_ArrayInstrument_FM_v6.fits.
  • The RSRF calibration tables contain "NaN" which are then introduced into the signal by rsrfCal.
 

Analysis tools

Line: 559 to 552
 

Spectrometry

Changed:
<
<
  • Compatibility of PACS products with HCSS tools improved: One can now properly select data points and use the fitter tool on PACS frames and cubes.
>
>
  • Compatibility of PACS products with HCSS tools improved: you can now properly select data points and use the fitter tool on PACS frames and cubes.
 

Product changes

Photometry

Line: 602 to 595
 

Engineering Conversion

Common Pipeline

Changed:
<
<
  • Only minor changes in logs and documentation.
>
>
  • Minor changes in logs and documentation.
 

Photometer Pipeline

  • Pipeline scripts:
Changed:
<
<
    • Added time correlation task and concurrent glitch deglitcher task in scan map pipelines
    • Comment out optical and electrical crosstalk corrections
    • Removed plotting blocks
>
>
    • Added time correlation task and concurrent glitch deglitcher task in scan map pipelines.
    • Comment out optical and electrical crosstalk corrections.
    • Removed plotting blocks.
 
  • Pipeline tasks:

Spectrometer Pipeline

The spectrometer pipelines have undergone major changes for version 5. It is HIGHLY recommended that users with data processed with older versions update to version 5 to reprocess their data.

Level-0 -> Level 0.5

Changed:
<
<
No change
>
>
No change.
 

Level-0.5 -> Level 1 (IFGM)

  • Electrical crosstalk correction has been commented out of the SPG pipelines.
Line: 636 to 629
 
    • Added NaiveProjectionTask, a new task for creating spectral simple cubes from preprocessed spectral data. This task creates spectral cubes by averaging the flux of samples within each pixel. Pixels without samples are given a value of NaN.

Interactive Analysis and Tools

Changed:
<
<
  • Added the new DTMosaic class that allows the user to plot data recorded by many detectors during one observation on a single page to compare data more effectively (mosaic plot). This features is available from HIPE command line or within DetectorTimelineExplorer
  • DetectorTimelineExplorer and DetectorTimelineExplorerComponent :
    • Added a new features that allows the creation of mosaic plot of the detectors via right-click on the desired array
    • Added a new Color&Mask Preferences panel
    • Improved the color scale (using the Color&Mask Preferences it is now possible to select the Min/Max value for the color scale)
    • Improved the mask visualization (using the Color&Mask Preferences it's now possible to visualize any specific mask value directly on the array display)
>
>
  • Added the new DTMosaic class that allows the user to plot data recorded by many detectors during one observation on a single page to compare data more effectively (mosaic plot). This features is available from HIPE command line or within DetectorTimelineExplorer.
  • DetectorTimelineExplorer and DetectorTimelineExplorerComponent:
    • Added a new features that allows the creation of mosaic plot of the detectors via right-click on the desired array.
    • Added a new Color&Mask Preferences panel.
    • Improved the color scale (using the Color&Mask Preferences it is now possible to select the Min/Max value for the color scale).
    • Improved the mask visualization (using the Color&Mask Preferences it's now possible to visualize any specific mask value directly on the array display).
 
  • Made available the Mosaic class which creates a mosaic of plots in the shape of the SLW, SSW or coaligned detectors of the SPIRE FTS. The Mosaic class is intended to provide a means for viewing (at a glance) the spectra or interferograms associated with each detector.
  • Changed SpecExplorer's detector display so that it automatically re-sizes to fit the enclosing frame. Also, SpecExplorer's detectors are now enclosed in a splitter pane. This enables you to maximize or hide the detector display with a single click.
  • A new SpireMultiObs task to process several observations through the pipeline at once.

Revision 502010-11-16 - DavideRizzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 16 to 16
 
  • The tooltip shown when the mouse cursor is over a task in the Task view is now more informative.
  • Autocompletion in the Console has been improved.
  • A variable value can be copied by right clicking on the variable in the Variables view and selecting Copy value; it can be pasted in the Console or Jython editor afterwards. This works only for strings and numbers.
Changed:
<
<
  • Drag & drop a variable from the Variables view to the Console writes the variable name. The same behaviour can be achieved when dropping it to a Jython editor.
  • Drag & drop selected text from a Jython editor to the Console or vice-versa copy & paste that text.
  • The small level-2 preview shown in the top-right corner of the Observation Context viewer can be enlarged by just clicking on it.
>
>
  • Drag & drop a variable from the Variables view to the Console to write the variable name. The same behaviour can be achieved when dropping it to a Jython editor.
  • Drag & drop selected text from a Jython editor to the Console or vice-versa to copy & paste that text.
  • The small level-2 preview shown in the top-right corner of the Observation Context viewer can be enlarged by clicking on it.
 

Startup & Shutdown

Changed:
<
<
If the running Java platform is not supported, a warning is popped up.
>
>
If the running Java platform is not supported, a warning is displayed.
 

Feedback

Line: 29 to 28
  A feedback button has been added for documentation and HIPE functionality.
Deleted:
<
<
 

Look & Feel

Changed:
<
<
Aqua look & feel has been replaced in Mac OS X by the HIPE look & feel that is used among all platforms.
>
>
Aqua look & feel has been replaced in Mac OS X by the HIPE look & feel used on the other platforms.
 

Plug-ins

Changed:
<
<
HIPE now supports plug-ins, with are small packages that anyone can easily create and distribute to share code (Java or Jython) and data (LocalStores). Plug-ins can be downloaded and installed with a single-click.
>
>
HIPE now supports plug-ins, small packages that anyone can easily create and distribute to share code (Java or Jython) and data (LocalStores). Plug-ins can be downloaded and installed with a single-click.
 

Plotting

Changed:
<
<
  • Title and label can accept LaTeX commands \textrm \textit \textbf \mathrm \mathit \mathbf. for example, V$_{\textrm{LSR}}$
>
>
  • Title and label can now accept the LaTeX commands \textrm \textit \textbf \mathrm \mathit \mathbf. For example: V$_{\textrm{LSR}}$
 

TablePlotter

OverPlotter

Dataset viewer

Astronomical utilities

Changed:
<
<
  • RadialVelocity now uses TimeCorrProduct to select pointing data
  • SSO now uses TimeCorrProduct to select pointing data
  • Improved documentation
>
>
  • RadialVelocity now uses TimeCorrProduct to select pointing data.
  • SSO now uses TimeCorrProduct to select pointing data.
  • Improved documentation.
 

Numeric routines

Data fitting

Changed:
<
<
  • A complete new GUI for the SpectrumFitter is added. Start it from the SpectrumExplorer right mouse menu, select SpectrumFitterGUI. Besides the functionality that was available in the SpectrumFitterTool, the GUI has an export TAB that enables the user to export the residual, total model, and individual models as an object with the type of the input object and as ASCII.
>
>
  • A complete new GUI for the SpectrumFitter was added. Start it from the SpectrumExplorer right mouse menu, select SpectrumFitterGUI. Besides the functionality that was available in the SpectrumFitterTool, the GUI has an export tab that enables you to export the residual, total model, and individual models as an object with the type of the input object and as text.
 
  • The SpectrumFitter (commandline version) accepts Container with PointSpectrum and SpectralSegment index.
Changed:
<
<
  • The SpectrumFitter (commandline version) has methods to export the data as the input type and as ASCII.
  • The SpectrumFitter (commandline version) has method that returns a Jython script
>
>
  • The SpectrumFitter (commandline version) has methods to export the data as the input type and as text.
  • The SpectrumFitter (commandline version) has method that returns a Jython script.
 

Data interpolation

Random Numbers

  • New random numbers generators from different distribution: Uniform, Gaussian, Poisson, Gamma, Cauchy (Lorentz), Exponential and Jeffreys.

Statistics functions

Changed:
<
<
  • RMS has been renamed to QRMS
  • RMS implements a new functionality (see URM)
>
>
  • RMS has been renamed to QRMS.
  • RMS implements a new functionality (see the User Reference Manual).
 
  • Improved StdDev and Variance algorithms.
  • Improved CorrelateMatrix algorithm.

New functions
Changed:
<
<
  • GeoMean which calculates the geometric mean of a array of numeric data types with 1 to 5 dimensions.
  • Mode which returns the mode(s), or the most common element(s), of an array of numeric data with 1 to 5 dimensions.
  • Covariance which yields the covariance between two random variables/vectors x and y with finite second moments. If one vector is longer than the other, only the values up to the length of the shorter vector will be taken into account.
  • CovarianceMatrix which returns the covariance matrix of the input M x N matrix. The result is a N x N matrix with each i, j value equal to the covariance of the ith and jth columns of the original matrix.
>
>
  • GeoMean, which calculates the geometric mean of a array of numeric data types with 1 to 5 dimensions.
  • Mode, which returns the mode(s), or the most common element(s), of an array of numeric data with 1 to 5 dimensions.
  • Covariance, which returns the covariance between two random variables/vectors x and y with finite second moments. If one vector is longer than the other, only the values up to the length of the shorter vector will be taken into account.
  • CovarianceMatrix, which returns the covariance matrix of the input M x N matrix. The result is a N x N matrix with each i, j value equal to the covariance of the ith and jth columns of the original matrix.
 

Trend analysis

Mapping

Other functions

  • SigClip
Changed:
<
<
    • Median mode uses Median Absolute Deviation instead of Standard deviation.
>
>
    • Median mode uses median absolute deviation instead of standard deviation.
 
    • Iterative use is allowed.
Deleted:
<
<

Source extraction

  • No significant changes to functionality.
 

Images

Display

Changed:
<
<
  • Images can be opened as RgbImage directly from the Navigator in HIPE.
  • Regular image files (jpg, gif, png, ...) are shown as preview in the outline when being clicked on in the navigator view of hipe.
  • 2 images can be compared by setting the opacity of the image. This can be done using the setOpacity(float) method or using a slider in the Image Display.
>
>
  • Images can be opened as RgbImage directly from the Navigator in HIPE.
  • Regular image files (jpg, gif, png, ...) are shown as preview in the outline when being clicked on in the navigator view of HIPE.
  • Two images can be compared by setting the opacity of the image. This can be done using the setOpacity(float) method or using a slider in the Image Display.
 

Analysis

Changed:
<
<
  • Added the methods containsNorthCelestialPole and containsSouthCelestialPole on the Wcs.
  • Regridding an image on the grid of another image using RegridTask
  • Speed improvement of createRgbImageTask
  • Update of URM documentation
  • Cropping an image by drawing a rectangle on it
  • Correction of the calculation of the dimensions of a mosaic
>
>
  • Added the methods containsNorthCelestialPole and containsSouthCelestialPole to the Wcs.
  • Regridding an image on the grid of another image using RegridTask.
  • Speed improvement of createRgbImageTask.
  • Update of documentation in the User Reference Manual.
  • You can now crop an image by drawing a rectangle on it
  • Correction of the calculation of the dimensions of a mosaic.
 

Spectra

Display

Changed:
<
<
  • Task & Tools Toolbox All task and tool panels are displayed in the same toolbox panel and selectable through a drop-down list. The task gui components have become more clear as well.
>
>
  • Task & Tools Toolbox: All task and tool panels are displayed in the same toolbox panel and selectable through a drop-down list. The task GUI components have become clearer as well.
  Instead of dragging a selection of spectra to the dataset parameter, the task is applied on either the displayed or the selected spectra within the plot.
Changed:
<
<
  • Load selectable datasets from a product When opening the spectrum explorer on a Product, a selection table is displayed showing all datasets that are stored in this Product along with certain characteristics.
>
>
  • Load selectable datasets from a product: When opening the spectrum explorer on a product, a selection table is displayed showing all datasets that are stored in this product along with certain characteristics.
  Just as within the original spectrum selection panel, these datasets can be sorted by clicking on the headercolumns. To load a set of datasets instead of just one or all, select each row and right click to load the selected datasets.
Changed:
<
<
  • Realtime statistics information Selecting the Statistics task within the Toolbox displays realtime statistics while dragging ranges in the plot.
  • Added relativistic frequency to velocity transform When changing the unit of the wave axis to velocity the transform is now relativistic
>
>
  • Realtime statistics information: Selecting the Statistics task within the Toolbox displays realtime statistics while dragging ranges in the plot.
  • Added relativistic frequency to velocity transform: When changing the unit of the wave axis to velocity the transform is now relativistic.
 

Analysis

Changed:
<
<
  • Gaussian resampling introduced
>
>
  • Gaussian resampling introduced.
 

Data cubes

Display

Line: 117 to 107
 

Herschel Science Archive

  • Support for exporting data in packed HSA format.
  • Import and Export Views improvements:
Changed:
<
<
    • better error reporting
    • file dialogs can show hidden files
    • versions of observations inside pools shown
>
>
    • Better error reporting
    • File dialogue windows can show hidden files
    • Versions of observations inside pools are now shown
 

Pipeline processing

  • Crated a daemon dispatcher mechanism.
  • Aliases set tool has been modified for performing the required verifications before a tag is ingested.
  • HsaPoolDaemon
Changed:
<
<
    • releases memory calling Java garbage collector if it is needed.
    • can write compressed files.
    • can save a binary representation of a product if the FITS save operation fails.
>
>
    • Releases memory calling Java garbage collector if it is needed.
    • Can write compressed files.
    • Can save a binary representation of a product if the FITS save operation fails.
 
  • HsaStagePool can export products into a LocalStore.
  • HSA ingestion procedure can generate an XML ingestion request file even if a previous version does not contain its own XML.
Deleted:
<
<
  • Improved test-harnesses to include an HAIO simulated server framework.
 

Calibration sources

Line: 138 to 127
 

Products and datasets

Changed:
<
<
  • Product comparison API is available in ia_toolbox_util module.
>
>
  • Product comparison API is available in the ia_toolbox_util module.
 

Observation context

Changed:
<
<
  • Include new metadata "slewTime" returning the scheduled start time of the slew as a FineTime
    • Specific accessors, getSlewStartTime() and setSlewStartTime(), where added to the ObservationContext class
    • For old observations in the HSA without this value, the method will return a null value indicating that this field was never set before
>
>
  • Include new metadata slewTime returning the scheduled start time of the slew as a FineTime.
    • Specific accessors, getSlewStartTime() and setSlewStartTime(), where added to the ObservationContext class.
    • For old observations in the HSA without this value, the method will return a null value indicating that this field was never set before.
 

History

Line: 470 to 459
 

Pipeline

  • PACS "New Style Sliced" pipeline implementation is now activated
Changed:
<
<
    • PacsSpr:2647 : phot and spec ipipe script names are changed to be more logic and intuitive
>
>
    • Phot and spec ipipe script names were changed to be more logic and intuitive
 
    • Pipeline Script access
      • HIPE -> Menu -> pipeline -> PACS
  • Calibration framework
Line: 493 to 482
 
    • Extended Madmap (using scan and cross scan) for interactive usage
      • HIPE -> Menu -> pipeline -> PACS -> L2_scanMap.py
Changed:
<
<
Based on HCSS-9361, a prototype for the extended Madmap pipeline is now available in HIPE for interactive usage. The extended Madmap pipeline is designed to combine scans and cross-scans in order to produce optional map. Unlike DP pipelines (which normally starts with one ObservationContext), this pipeline takes an obsList (an ArrayList) which is a list of ObservationContext. The pipeline will run even there is only one obs in the obsList. Below is an example creating an obsList:
>
>
A prototype for the extended Madmap pipeline is now available in HIPE for interactive usage. The extended Madmap pipeline is designed to combine scans and cross-scans in order to produce optional map. Unlike DP pipelines (which normally starts with one ObservationContext), this pipeline takes an obsList (an ArrayList) which is a list of ObservationContext. The pipeline will run even there is only one obs in the obsList. Below is an example creating an obsList:
 
HIPE> obsList=ArrayList()
HIPE> obsList.add(obs1)
Line: 501 to 490
 

  • New script for scan Map pipeline
Changed:
<
<
    • PacsSpr:2801 Proposal for a two-stage photometer pipeline with masked highpass filtering
>
>
    • Two-stage photometer pipeline with masked highpass filtering
 
      • Now THE standard processing script for scan maps
  • Photometer Mapping
Changed:
<
<
    • PacsSpr:2853: Photometer mapping accepts a variable input pixelsize (parameter pixfrac in the PhotProjectTask and the MapIndexTask).
    • PacsSpr:2789: The MapIndexViewer has a Sigclip gui to display the effect of different Sigclip parameters on the second level deglitching. Now it is not necessary any
>
>
    • Photometer mapping accepts a variable input pixelsize (parameter pixfrac in the PhotProjectTask and the MapIndexTask).
    • The MapIndexViewer has a Sigclip gui to display the effect of different Sigclip parameters on the second level deglitching. Now it is not necessary any
  more to guess Sigclip parameters, run the deglitching and check the quality afterwards. The new option allows to find the best parameter quickly BEFORE deglitching has been applied.
The MapIndexViewer now also displays the signal vectors in the "timeordered" mode.
Changed:
<
<
    • PacsSpr:2921: the MapIndexViewer shows mean/median +/- error permanently in the signal plot
    • PacsSpr:2807: the Full MapIndex needs less memory than before
    • PacsSpr:2859: Wcs4mapTask got a crota option to manually set the requested rotation angle of a map.
    • PacsSpr:2925: CorrectRaDec4SsoTask is now also accepting the new Horizons class definition. An example how to use it can be found in the attached
    • PacsSpr:2751 : photProject error map has been checked for NaN
    • PacsSpr:2543 : exposure map has been removed in favour of the coverage map
>
>
    • The MapIndexViewer shows mean/median +/- error permanently in the signal plot
    • The Full MapIndex needs less memory than before
    • Wcs4mapTask got a crota option to manually set the requested rotation angle of a map.
    • CorrectRaDec4SsoTask is now also accepting the new Horizons class definition. An example how to use it can be found in the attached
    • photProject error map has been checked for NaN
    • Exposure map has been removed in favour of the coverage map
 
  • Noise treatmement
Changed:
<
<
    • PacsSpr:2752: A new modul herschel.pacs.spg.phot.PhotAddNoisePerPixelTask provides 4 ways to calculate the initial photometer noise.
>
>
    • A new module herschel.pacs.spg.phot.PhotAddNoisePerPixelTask provides 4 ways to calculate the initial photometer noise.
  *Noise propagation is also implemented in the HighpassFilter, Mean- and MedianFilters.
  • Pointing
    • New Horizons Product used in PACS pipeline (SSO observations)
Changed:
<
<
>
>
    • Residual spatial offset PACS photometer blue/green vs red
    • Corners shall follow changes in PhotAssignRaDec
    • Separate blue and green in ArrayInstrument calfile and the code using it
    • Astrometric offset between two scan maps in parallel mode on the same field.
 
  • Others
Changed:
<
<
    • PacsSpr:2761 PACS EDP: phot pipeline adaptation to cope with variable scan velocity
>
>
    • PACS EDP: phot pipeline adaptation to cope with variable scan velocity
 
    • A new module photMaskFrames offer masking :
    • framesOut = photMaskFrames(framesIn [,beforeFirstScanLeg] [,noScanData] [,afterCalblock=0] )
      • beforeFirstScanLeg : Mask data befor the first scan leg
      • noScanData : mask data where the ScanBBID is not set (includes turn arounds)
      • afterCalblock : Mask n data after calibration block
Changed:
<
<
>
>
    • Sigclip can now be applied n-times in one go.
 

Spectrometry

  • new AOR mode "unchopped" including improved spatial calibration:
    • First draft of the unchopped mode pipeline added to the build
Changed:
<
<
      • PacsScr:2779: a "newstyle" unchopped mode pipeline is in place which can be run automatically or interactively
    • PacsScr:1048 New task specSubtractOffPosition implemented for unchopped pipeline
>
>
      • A "newstyle" unchopped mode pipeline is in place which can be run automatically or interactively
    • New task specSubtractOffPosition implemented for unchopped pipeline
 
Changed:
<
<
>
>
 
  • improved wavelength calibration:
Changed:
<
<
    • PacsScr:2964: new wavelength calibration applied in waveCalc task using the wavePolynomes cal file
>
>
    • New wavelength calibration applied in waveCalc task using the wavePolynomes cal file
 
  • spectrometer trend analysis product for the calibration blocks:
Changed:
<
<
    • PacsScr:2777: calibration block trendAnalysis product added to the observation context
>
>
    • Calibration block trendAnalysis product added to the observation context
 
  • Extra options to specDiffChop (note these are EXPERT options for the analysis of special observations!)
Changed:
<
<
    • PacsScr:2717: extra option 'extend' added which allows for other pairwise differencing schemes. These schemes will subtract more datapoints from each other than the standard scheme.
    • PacsScr:2427: extra option 'chopNodScheme' which provides different scheme for determining which label is on-source and which is off-source for a given nodding position. Needed for some special test AORs.
  • Different bands in a range scan are now stored in their own blocks in the block table. This means that it is now possible to slice SEDs per band (PacsScr:2963).
>
>
    • Extra option 'extend' added which allows for other pairwise differencing schemes. These schemes will subtract more datapoints from each other than the standard scheme.
    • Extra option 'chopNodScheme' which provides different scheme for determining which label is on-source and which is off-source for a given nodding position. Needed for some special test AORs.
  • Different bands in a range scan are now stored in their own blocks in the block table. This means that it is now possible to slice SEDs per band.
 
  • Several fixes to specWaveRebin, etc. to reduce the amount of NaNs considerably in the rebinned data.
  • SpecProject sets the flux to Jy.

Line: 558 to 546
 

Photometry

Changed:
<
<
>
>
  • Residual spatial offset PACS photometer blue/green vs red --> PCalPhotometer_ArrayInstrument_FM_v6.fits
 

Spectrometry

Changed:
<
<
  • PacsSpr:2488 : New Wavelength Calibration Product
  • PacsSpr:2855 v8 of the Pacs Spectrometer Array to Intrument and v4 of the Module2Array cal file is inserted in the system. This generates correct values also for center chopper positions (unchopped and waveswitch mode).
  • PacsSpr:2715 : updated spectrometer spatial cal file --> PCalSpectrometer_ArrayInstrument_FM_v6.fits
  • PacsSpr:2607 The RSRF calibration tables contain "NaN" which are then introduced into the signal by rsrfCal
>
>
  • New Wavelength Calibration Product
  • Version 8 of the Pacs Spectrometer Array to Instrument and version 4 of the Module2Array cal file are inserted in the system. This generates correct values also for center chopper positions (unchopped and waveswitch mode).
  • Updated spectrometer spatial cal file --> PCalSpectrometer_ArrayInstrument_FM_v6.fits
  • The RSRF calibration tables contain "NaN" which are then introduced into the signal by rsrfCal
 

Analysis tools

Revision 492010-10-14 - SteveGuest

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 586 to 586
 

Calibration Products

Photometer

Added:
>
>
Updated
  1. Flux Calibration updated for the nominal voltages.
  2. Temperature Drift Correction to match the new flux conversion.
 

Spectrometer

New
  1. Non-linear phase
Line: 602 to 606
 

Calibration Framework

Added:
>
>
  • Updated to match the new products and dependencies.
  • The spireCal task now has an option to read from a local pool.
 

Preprocessing

Engineering Conversion

Line: 649 to 656
 
    • Improved the mask visualization (using the Color&Mask Preferences it's now possible to visualize any specific mask value directly on the array display)
  • Made available the Mosaic class which creates a mosaic of plots in the shape of the SLW, SSW or coaligned detectors of the SPIRE FTS. The Mosaic class is intended to provide a means for viewing (at a glance) the spectra or interferograms associated with each detector.
  • Changed SpecExplorer's detector display so that it automatically re-sizes to fit the enclosing frame. Also, SpecExplorer's detectors are now enclosed in a splitter pane. This enables you to maximize or hide the detector display with a single click.
Added:
>
>
  • A new SpireMultiObs task to process several observations through the pipeline at once.
 

Documentation

Revision 482010-10-07 - StephanOtt

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 24 to 24
  If the running Java platform is not supported, a warning is popped up.
Added:
>
>

Feedback

A feedback button has been added for documentation and HIPE functionality.

 

Look & Feel

Aqua look & feel has been replaced in Mac OS X by the HIPE look & feel that is used among all platforms.

Line: 42 to 48
 

Astronomical utilities

  • RadialVelocity now uses TimeCorrProduct to select pointing data
Changed:
<
<
  • SSo now uses TimeCorrProduct to select pointing data
>
>
  • SSO now uses TimeCorrProduct to select pointing data
 
  • Improved documentation

Numeric routines

Line: 138 to 144
 
  • Include new metadata "slewTime" returning the scheduled start time of the slew as a FineTime
    • Specific accessors, getSlewStartTime() and setSlewStartTime(), where added to the ObservationContext class
Changed:
<
<
    • For old observations in the hsa without this value, the method will return a null value indicating that this field was never set before
>
>
    • For old observations in the HSA without this value, the method will return a null value indicating that this field was never set before
 

History

Line: 165 to 171
 
  • Updated documentation:
    • wiki chapters
    • Javadoc of general tasks
Changed:
<
<
    • Standarized _ doc _ of general tasks (signature, description, parameter descriptions)
>
>
    • Standardised _ doc _ of general tasks (signature, description, parameter descriptions)
 
  • Tasks View provides tooltips with the signature of the tasks
  • Examples of general tasks in the URM are now tested automatically
  • Compress and Decompress tasks: they support zip, tar, gzip, tgz in a portable way and decompress can guess the compression of a file.
Line: 173 to 179
 

Quality control

Changed:
<
<
  • New metadata into the QualityContext product. The field is fixed and the main interest is to include this information into the FITs file associated to the product.
>
>
  • New metadata into the QualityContext product. The field is fixed and the main interest is to include this information into the FITS file associated to the product.
 
FITS keyword Value
PCAVEATS Please refer to http://herschel.esac.esa.int/DpKnownIssues.shtml for known problems in products
Changed:
<
<
  • Perform bulk actions on several qc reports simultaneously
  • Flags color codes depending on their importance
>
>
  • Perform bulk actions on several quality control reports simultaneously
  • Flags colour codes depending on their importance
 

Systematic product generation

Line: 393 to 399
 

Level 0 Pipeline

Changed:
<
<
  • H and V spectrometer astronometry
>
>
  • H and V spectrometer astrometry
 SpectrumDatasets contain columns, in decimal degrees, for the commanded pointing (a position between the H and V beams known as the synthesized aperture), labelled longitude_cmd and latitude_cmd, while the actual coordinates for the spectrometer beam are found in the longitude and latitude columns.

Revision 472010-10-07 - WernerSalomons

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 91 to 91
 
  • Correction of the calculation of the dimensions of a mosaic

Spectra

Display

Added:
>
>
  • Task & Tools Toolbox All task and tool panels are displayed in the same toolbox panel and selectable through a drop-down list. The task gui components have become more clear as well. Instead of dragging a selection of spectra to the dataset parameter, the task is applied on either the displayed or the selected spectra within the plot.
  • Load selectable datasets from a product When opening the spectrum explorer on a Product, a selection table is displayed showing all datasets that are stored in this Product along with certain characteristics. Just as within the original spectrum selection panel, these datasets can be sorted by clicking on the headercolumns. To load a set of datasets instead of just one or all, select each row and right click to load the selected datasets.
  • Realtime statistics information Selecting the Statistics task within the Toolbox displays realtime statistics while dragging ranges in the plot.
  • Added relativistic frequency to velocity transform When changing the unit of the wave axis to velocity the transform is now relativistic
 

Analysis

  • Gaussian resampling introduced

Revision 462010-10-06 - AdwinBoogert

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 424 to 424
 

FitHifiFringe

Changed:
<
<
  • Now possible to remove from the spectrum the smoothed baseline calculated by FitHIFIFringe. This option should be used with care as real spectral features can be removed this way
>
>
  • It is now possible to remove the smoothed baseline calculated by FitHifiFringe from the spectrum, in addition to the fitted sine waves. This option (sub_base) should be used with care as real spectral features can be removed this way. For interactive fitting of polynomial baselines, the new task FitBaseline is recommended.
 
  • Tooltips implemented
  • Improved error messaging

Revision 452010-10-05 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 426 to 426
 
  • Now possible to remove from the spectrum the smoothed baseline calculated by FitHIFIFringe. This option should be used with care as real spectral features can be removed this way
  • Tooltips implemented
Added:
>
>
  • Improved error messaging
 

HiClass export tool

Revision 442010-10-04 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 385 to 385
 
  • H and V spectrometer astronometry
SpectrumDatasets contain columns, in decimal degrees, for the commanded pointing (a position between the H and V beams known as the synthesized aperture), labelled longitude_cmd and latitude_cmd, while the actual coordinates for the spectrometer beam are found in the longitude and latitude columns.
Added:
>
>
 

HRS Pipeline

WBS Pipeline

Added:
>
>
  • Fixes for frequency calibration in the case that line centres fall precisely in the middle of two pixels
 

Level 1 Pipeline

Level 2 Pipeline

Line: 408 to 411
 

  • hifiPipeline will make an ObservationContext even if creation of trend products fails
Added:
>
>
  • hifiPipeline GUI layout improved to make more efficient use of space
 

Products

Line: 423 to 427
 
  • Now possible to remove from the spectrum the smoothed baseline calculated by FitHIFIFringe. This option should be used with care as real spectral features can be removed this way
  • Tooltips implemented
Deleted:
<
<

FitBaseline

  • New task introduced to interactively fit and subtract or divide out baselines in HIFI spectra.
 

HiClass export tool

Added:
>
>
  • HiClass will now appear an an applicable task for HTP as well as ObservationContexts
 

Flags

SpectrumExplorer (HIFI specifics)

HIFI DP scripts (User tool scripts)

Added:
>
>
  • Introduction of a new task, FitBaseline, to interactively fit and subtract or divide out baselines in HIFI spectra.
 

Documentation

  • New chapters in the HIFI User's Manual: "What was done to my data?" and "HIFI Baseline Removal"

Revision 432010-10-04 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 92 to 92
 

Spectra

Display

Analysis

Added:
>
>
  • Gaussian resampling introduced
 

Data cubes

Display

Line: 239 to 240
 
  • The hcss.dp.hifi (non-expert) build has pre-configured pools to access the HSA and the local pool via the Product Browser and PAL Storage Manager
Added:
>
>
  • Unnecessary and deprecated properties files have been removed from the system
 

HIFI Calibration

  • The minimum and maximum chopper threshold values have been changed.
    • HkThresholdsContext_H fpuChopperMin -> -0.04
Line: 258 to 261
 
    • beamWidths-H
    • beamWidths-V

Changed:
<
<
The approach so far is to strictly use observed numbers (see tables below), without involving fitting results or empirical formulas. In particular this means:
>
>
The approach so far is to strictly use observed numbers (see tables below with initial results), without involving fitting results or empirical formulas. In particular this means:
  * for beam widths the fitted widths of circular Gaussians, corrected for the convolution with the Mars disk. See Raphael Moreno's report for details. Open question: is the fit of a circular Gaussian correct? Need to check by how much Mars moved during an integration.
Changed:
<
<
* for main beam efficiencies the values derived by Raphael (case c, i.e. using the observed beam widths for the beam coupling). Do we want to interpolate linearly between (and beyond) frequencies or do we want to use an average value per band? * for aperture efficiencies the values derived by me, again using observed beam widths for the coupling of the beam to the source.
>
>
    • for main beam efficiencies the values derived using the observed beam widths for the beam coupling. It is not yet decided if an average value or a linear interpolation between (and beyond) frequencies produces the best results
    • for aperture efficiencies the values are derived, again, using observed beam widths for the coupling of the beam to the source.
  In addition, the already available entries for the forward efficiencies (H and V) are updated from 1.0 to 0.96 (in all bands, all frequencies) to be compatible with above results. This requires the change of constant 1.0 to 0.96 .
Changed:
<
<
Aperture Efficiency (H)
>
>
Aperture Efficiency (H)  
 
Band Frequency (GHz) Eta_A
1a 491 0.692
1b 610 0.693
Line: 283 to 286
 
7b 1807 0.627
7b 1893 0.604
Added:
>
>
Aperture Efficiency (V)
Band Frequency (GHz) Eta_A
1a 491 0.656
1b 610 0.668
2a 648 0.706
2b 770 0.694
3a 828 0.725
3b 948 0.734
4a 1012 0.724
4b 1112 0.744
5a 1127 0.588
5b 1243 0.588
6a 1483 0.675
6b 1625 0.597
7a 1703 0.582
7b 1807 0.650
7b 1893 0.674

Beam Efficiency (H)
Band frequency (GHz) eta_mb
1a 491 0.719
1b 610 0.708
2a 648 0.752
2b 770 0.719
3a 828 0.716
3b 948 0.739
4a 1012 0.712
4b 1112 0.764
5a 1127 0.682
5b 1243 0.677
6a 1483 0.707
6b 1625 0.691
7a 1703 0.620
7b 1807 0.664
7b 1893 0.650

Beam Efficiency (V)
Band Frequency (GHz) eta_mb
1a 491 0.71
1b 610 0.730
2a 648 0.767
2b 770 0.739
3a 828 0.716
3b 948 0.740
4a 1012 0.697
4b 1112 0.747
5a 1127 0.705
5b 1243 0.721
6a 1483 0.738
6b 1625 0.652
7a 1703 0.720
7b 1807 0.747
7b 1893 0.716

Beam Widths (H)
Band Frequency (GHz) HPBW
1a 491 43.22154
1b 610 34.49101
2a 648 33.54672
2b 770 27.83073
3a 828 26.07131
3b 948 22.95481
4a 1012 21.29627
4b 1112 19.62355
5a 1127 19.38483
5b 1243 18.38567
6a 1483 14.48873
6b 1625 13.29430
7a 1703 12.28931
7b 1807 11.83705
7b 1893 11.39468

Beam Widths (V)
Band Frequency (GHz) HPBW
1a 491 44.30458
1b 610 35.66471
2a 648 33.47924
2b 770 27.89113
3a 828 24.98089
3b 948 22.03684
4a 1012 20.16164
4b 1112 18.74398
5a 1127 20.21393
5b 1243 18.54717
6a 1483 14.66947
6b 1625 13.38736
7a 1703 13.59736
7b 1807 12.34126
7b 1893 11.32418
 

Note these are still initial results, measurements from ODs 390, 391, 406 and 407 still need to be added. One can use the developer part of calibration tree, where newer values will be tested first, by setting the following property:

Line: 290 to 383
 

Level 0 Pipeline

Added:
>
>
  • H and V spectrometer astronometry
SpectrumDatasets contain columns, in decimal degrees, for the commanded pointing (a position between the H and V beams known as the synthesized aperture), labelled longitude_cmd and latitude_cmd, while the actual coordinates for the spectrometer beam are found in the longitude and latitude columns.
 

HRS Pipeline

WBS Pipeline

Line: 300 to 396
 

hifiPipelineTask

Added:
>
>
  • The path/filename for a user defined/edited pipeline algorithm can now be passed to the hifiPipeline task via the command line, previously this was only possible in the GUI.

obs =hifiPipeline (obs=obs, level2AlgoPath="/home/Me/myLevel2Algo.py")

The older method of defining a function with the def keyword in Jython and passing that function to the hifiPipeline task still exists

obs =hifiPipeline (obs=obs, level2Algo=level2PipelineAlgo)

  • hifiPipeline will make an ObservationContext even if creation of trend products fails
 

Products

DoDeconvolution

Added:
>
>
  • More understandable error message when deconvolution fails (usual due to poor data input)
  • Plotting of DSB solution as deconvolutions runs now possible when diag_mode_on selected
 

DoGridding

FitHifiFringe

Added:
>
>
  • Now possible to remove from the spectrum the smoothed baseline calculated by FitHIFIFringe. This option should be used with care as real spectral features can be removed this way
  • Tooltips implemented

FitBaseline

  • New task introduced to interactively fit and subtract or divide out baselines in HIFI spectra.
 

HiClass export tool

Flags

Line: 318 to 437
 

Documentation

Added:
>
>
  • New chapters in the HIFI User's Manual: "What was done to my data?" and "HIFI Baseline Removal"
  • Updates to the HIFI Users Manual: "HIFI cookbook", "Data Primer", "Running the HIFI Pipeline", "HIFI Standing Wave Removal Tool", and "Exporting HIFI data to CLASS"
  • Updates to HIFI Pipeline Specification document: improved pipeline flow diagrams, improved information about Euler resampling used in DoFreq grid task of pipeline
  • Creation of FitHifiFringe URM entry
 

Other

PACS

Revision 422010-10-04 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 233 to 233
 

HIFI

HIFI Software Configuration and Properties

Added:
>
>
  • Generally most HIFI properties have moved from the hcss.dp.hifi build to the hcss.icc.hifi (expert) area and it is recommended that all internal ICC users always install the expert build in order to be able to run the pipeline from level -1.0.

  • Those using the expert build and wishing to run the pipeline from level zero will need to set the property var.hcss.instrument = hifi ' this so the system knows to use the HIFI binstruct settings for reading the packets. Note those using the legacy hifi-new.props file will already have this property set.

  • The hcss.dp.hifi (non-expert) build has pre-configured pools to access the HSA and the local pool via the Product Browser and PAL Storage Manager
 

HIFI Calibration

Added:
>
>
  • The minimum and maximum chopper threshold values have been changed.
    • HkThresholdsContext_H fpuChopperMin -> -0.04
    • HkThresholdsContext_V fpuChopperMin -> -0.04
    • HkThresholdsContext_H fpuChopperMax -> 0.06
    • HkThresholdsContext_H fpuChopperMax -> 0.06
Previous values used were those appropriate for the Hot and Cold black bodies, which are not representative of the full, planned range of the chopper. As a result, the quality control flag "FPU Check: Chopper measured values differ from the commanded" had been frequently raised more than necessary.

  • The upper limit of the mixer current quality control check for HEBs (bands 6 and 7) have been decreased from 0.08 mA to 0.055 mA, where tests have shown the mixers start to be under-pumped.

  • Update to calibration tree for results of Mars beam measurements
The following items have been added to the HIFI Calibration tree for the aperture efficiencies (H and V), main beam efficiencies (H and V), and the half power beam widths (H and V) :
    • apertureEfficiency-H
    • apertureEfficiency-V
    • beamEfficiency-H
    • beamEfficiency-V
    • beamWidths-H
    • beamWidths-V

The approach so far is to strictly use observed numbers (see tables below), without involving fitting results or empirical formulas. In particular this means: * for beam widths the fitted widths of circular Gaussians, corrected for the convolution with the Mars disk. See Raphael Moreno's report for details. Open question: is the fit of a circular Gaussian correct? Need to check by how much Mars moved during an integration. * for main beam efficiencies the values derived by Raphael (case c, i.e. using the observed beam widths for the beam coupling). Do we want to interpolate linearly between (and beyond) frequencies or do we want to use an average value per band? * for aperture efficiencies the values derived by me, again using observed beam widths for the coupling of the beam to the source.

In addition, the already available entries for the forward efficiencies (H and V) are updated from 1.0 to 0.96 (in all bands, all frequencies) to be compatible with above results. This requires the change of constant 1.0 to 0.96 .

Aperture Efficiency (H)
Band Frequency (GHz) Eta_A
1a 491 0.692
1b 610 0.693
2a 648 0.689
2b 770 0.678
3a 828 0.665
3b 948 0.676
4a 1012 0.664
4b 1112 0.695
5a 1127 0.618
5b 1243 0.561
6a 1483 0.663
6b 1625 0.642
7a 1703 0.613
7b 1807 0.627
7b 1893 0.604

Note these are still initial results, measurements from ODs 390, 391, 406 and 407 still need to be added. One can use the developer part of calibration tree, where newer values will be tested first, by setting the following property: Configuration.setProperty("hcss.ia.obs.cal.release", "developer")

 

Level 0 Pipeline

Revision 412010-10-04 - CarolynMcCoey

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 232 to 232
 

HIFI

Changed:
<
<

hifiPipelineTask

>
>

HIFI Software Configuration and Properties

HIFI Calibration

Level 0 Pipeline

 

HRS Pipeline

Line: 242 to 246
 

Level 2 Pipeline

Added:
>
>

hifiPipelineTask

 

Products

DoDeconvolution

Revision 402010-09-30 - TrevorFulton

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 380 to 380
 

SPIRE

Calibration Products

Added:
>
>

Photometer

Spectrometer

New
  1. Non-linear phase
    • This calibration product is used by the phase correction module to make the data symmetric prior to application of the Fourier transform.
  2. Spectrometer Instrument RSRF
    • This calibration product is used, along with the instrument temperatures from the housekeeping telemetry, to compute a model for the contribution to the measured spectra from the SPIRE instrument.
  3. Telescope RSRF
    • This calibration product is used, along with the Telescope temperatures from the housekeeping telemetry, to compute a model for the contribution to the measured spectra from the Herschel Telescope.
    • This calibration product is also used for extended source flux calibration.

Updated

  1. Point-source RSRF
    • This calibration product has been updated to use a higher signal-to-noise observation of Uranus.
 

Calibration Framework

Line: 398 to 413
 
  • Pipeline tasks:

Spectrometer Pipeline

Added:
>
>
The spectrometer pipelines have undergone major changes for version 5. It is HIGHLY recommended that users with data processed with older versions update to version 5 to reprocess their data.

Level-0 -> Level 0.5

No change

Level-0.5 -> Level 1 (IFGM)

  • Electrical crosstalk correction has been commented out of the SPG pipelines.

Level 1 (IFGM) -> Single-sided Fourier transform

  • Subtraction of the Instrument/Telescope contribution has been moved to the spectral domain.
  • Phase correction now uses a Jiggle dependent empirically-derived Non-linear phase cal product.

Single-sided Fourier transform -> Level 1 (SPEC)

  • Correction for the contribution of the SPIRE instrument. A new processing step has been added to remove from the measured spectrum the contribution from the SPIRE instrument.
  • Correction for the contribution of the Herschel Telescope. A new processing step has been added to remove from the measured spectrum the contribution from the Herschel Telescope.
  • Optical crosstalk correction has been commented out of the SPG pipelines.

Level 1 (SPEC) -> Level 2

  1. SOF1 Sparse spatial sampling mode
    • A minor change in that the point source calibration product is now tailored to spectral resolution and apodization.
  2. Mapping Modes
 
  • Added NaiveProjectionTask, a new task for creating spectral simple cubes from preprocessed spectral data. This task creates spectral cubes by averaging the flux of samples within each pixel. Pixels without samples are given a value of NaN.

Interactive Analysis and Tools

Line: 405 to 441
 
Changed:
<
<
    • Improved the color scale (using the Color&Mask Preferences it's now possible to select the Min/Max value fot the color scale)
>
>
    • Improved the color scale (using the Color&Mask Preferences it is now possible to select the Min/Max value for the color scale)
 
    • Improved the mask visualization (using the Color&Mask Preferences it's now possible to visualize any specific mask value directly on the array display)
  • Made available the Mosaic class which creates a mosaic of plots in the shape of the SLW, SSW or coaligned detectors of the SPIRE FTS. The Mosaic class is intended to provide a means for viewing (at a glance) the spectra or interferograms associated with each detector.
Changed:
<
<
  • Changed SpecExplorer's detector display so that it automatically resizes to fit the enclosing frame. Also, SpecExplorer's detectors are now enclosed in a splitter pane. This enables you to maximize or hide the detector display with a single click.
>
>
  • Changed SpecExplorer's detector display so that it automatically re-sizes to fit the enclosing frame. Also, SpecExplorer's detectors are now enclosed in a splitter pane. This enables you to maximize or hide the detector display with a single click.
 

Documentation

Revision 392010-09-29 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 10 to 10
 

HIPE

Changed:
<
<

Navigator view

>
>

Editors & Views

 
Changed:
<
<
Files ending with extension ".csv" are recognized as tables, and opened as such with double click.

Editors & Viewers

The small level-2 preview shown in the top-right corner of the Observation Context viewer can be enlarged by just clicking on it.

>
>
  • Files ending with extension ".csv" are recognized as tables, and opened as such with double click in the Navigator view or with File > Open.
  • The tooltip shown when the mouse cursor is over a task in the Task view is now more informative.
  • Autocompletion in the Console has been improved.
  • A variable value can be copied by right clicking on the variable in the Variables view and selecting Copy value; it can be pasted in the Console or Jython editor afterwards. This works only for strings and numbers.
  • Drag & drop a variable from the Variables view to the Console writes the variable name. The same behaviour can be achieved when dropping it to a Jython editor.
  • Drag & drop selected text from a Jython editor to the Console or vice-versa copy & paste that text.
  • The small level-2 preview shown in the top-right corner of the Observation Context viewer can be enlarged by just clicking on it.
 

Startup & Shutdown

Line: 30 to 32
  HIPE now supports plug-ins, with are small packages that anyone can easily create and distribute to share code (Java or Jython) and data (LocalStores). Plug-ins can be downloaded and installed with a single-click.
Deleted:
<
<

Miscellanea

  • For writing a variable's name in the Jython editor or the Console, just drag & drop it there from the Variables view.
  • Drag & drop selected text from a Jython editor to the Console or vice-versa copy & paste that text.
 

Plotting

  • Title and label can accept LaTeX commands \textrm \textit \textbf \mathrm \mathit \mathbf. for example, V$_{\textrm{LSR}}$
Line: 50 to 47
 

Numeric routines

Data fitting

Changed:
<
<
>
>
  • A complete new GUI for the SpectrumFitter is added. Start it from the SpectrumExplorer right mouse menu, select SpectrumFitterGUI. Besides the functionality that was available in the SpectrumFitterTool, the GUI has an export TAB that enables the user to export the residual, total model, and individual models as an object with the type of the input object and as ASCII.
  • The SpectrumFitter (commandline version) accepts Container with PointSpectrum and SpectralSegment index.
  • The SpectrumFitter (commandline version) has methods to export the data as the input type and as ASCII.
  • The SpectrumFitter (commandline version) has method that returns a Jython script
 

Data interpolation

Random Numbers

  • New random numbers generators from different distribution: Uniform, Gaussian, Poisson, Gamma, Cauchy (Lorentz), Exponential and Jeffreys.

Revision 382010-09-28 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 12 to 12
 

Navigator view

Changed:
<
<
  • Files ending with extension ".csv" are recognized as tables, and opened as such with double click.
>
>
Files ending with extension ".csv" are recognized as tables, and opened as such with double click.
 

Editors & Viewers

Changed:
<
<
  • The small level-2 preview shown in the top-right corner of the Observation Context viewer can be enlarged by just clicking on it.
>
>
The small level-2 preview shown in the top-right corner of the Observation Context viewer can be enlarged by just clicking on it.
 

Startup & Shutdown

Changed:
<
<
  • If the running Java platform is not supported, a warning is popped up.
>
>
If the running Java platform is not supported, a warning is popped up.
 

Look & Feel

Changed:
<
<
  • Aqua look & feel has been replaced in Mac OS X by the HIPE look & feel that is used among all platforms.
>
>
Aqua look & feel has been replaced in Mac OS X by the HIPE look & feel that is used among all platforms.
 

Plug-ins

HIPE now supports plug-ins, with are small packages that anyone can easily create and distribute to share code (Java or Jython) and data (LocalStores). Plug-ins can be downloaded and installed with a single-click.

Added:
>
>

Miscellanea

  • For writing a variable's name in the Jython editor or the Console, just drag & drop it there from the Variables view.
  • Drag & drop selected text from a Jython editor to the Console or vice-versa copy & paste that text.
 

Plotting

Added:
>
>
 
  • Title and label can accept LaTeX commands \textrm \textit \textbf \mathrm \mathit \mathbf. for example, V$_{\textrm{LSR}}$
Added:
>
>
 

TablePlotter

OverPlotter

Dataset viewer

Line: 141 to 148
 
  • HsaReadPool/HsaXmlPool can work with compressed data.
  • HsaReadPool can retrieve product metadata directly.
Added:
>
>

Preferences

Product storages and pools can be managed in the new Data Access > Storages & Pools preferences panel.

 

Tasks

  • Updated task sample code
  • Updated documentation:

Revision 372010-09-24 - PasqualePanuzzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 384 to 384
 

Photometer Pipeline

  • Pipeline scripts:
Changed:
<
<
    • Added time correlation task in scan map pipelines
>
>
    • Added time correlation task and concurrent glitch deglitcher task in scan map pipelines
 
    • Comment out optical and electrical crosstalk corrections
    • Removed plotting blocks
  • Pipeline tasks:

Revision 362010-09-24 - ThanosTsounis

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 198 to 198
 

Virtual Observatory

Installer

Added:
>
>
There was a major restructuring of the pre-installation steps
  • The Basic/Advance selection page has been removed.
  • The memory, proxy and versant settings were restructured. There are no separate panels asking if the user wants to set a proxy or a versant. There is a dedicated page for the proxy setting and one for the versant settings. In the top of the panel there is the question with a check box. Then there are the text fields not editable by default which will become editable upon user action on the above check box.
  • Instead of the old instrument selection page, the user will now be offered the following list with check boxes for selection:
    Option Description
    PACS (preselected by default) PACS pipelines and analysis tools for astronomers.
    PACS expert Extra instrument-related applications for Calibration Scientists.
    SPIRE (preselected by default) SPIRE pipelines and analysis tools for astronomers.
    SPIRE expert Extra instrument-related applications for Calibration Scientists.
    HIFI (preselected by default) HIFI pipelines and analysis tools for astronomers.
    HIFI expert Extra instrument-related applications for Calibration Scientists.

  • The instrument selection window should allow to click in any combination of items to install those components.
  • It should also allow the deselection of all instruments to get HIPE framework only for FITS i/o and numerics analysis.
  • If possible, the installer should inform the user of the amount of data to download for the selected combination of modules.
  • The Proxy selection page should contain the Yes/No button and the entry fields, which are kept in grey until "Yes" is selected. The default is kept to "No".
  • The Versant selection page should contain the Yes/No button and the entry fields, which are kept in grey until "Yes" is selected. The default is kept to "No".
  • The rest of the pages should stay as they are.
 

Documentation

Revision 352010-09-21 - PeterKennedy

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 371 to 371
 
  • Pipeline tasks:

Spectrometer Pipeline

Added:
>
>
  • Added NaiveProjectionTask, a new task for creating spectral simple cubes from preprocessed spectral data. This task creates spectral cubes by averaging the flux of samples within each pixel. Pixels without samples are given a value of NaN.
 

Interactive Analysis and Tools

  • Added the new DTMosaic class that allows the user to plot data recorded by many detectors during one observation on a single page to compare data more effectively (mosaic plot). This features is available from HIPE command line or within DetectorTimelineExplorer

Revision 342010-09-17 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 22 to 22
 
  • If the running Java platform is not supported, a warning is popped up.
Changed:
<
<

Plug-ins

>
>

Look & Feel

 
Changed:
<
<
HIPE now supports plug-ins, with are small packages that anyone can easily create and distribute to share code (Java or Jython) and data (LocalStores). Plug-ins can be downloaded and installed with a single-click.
>
>
  • Aqua look & feel has been replaced in Mac OS X by the HIPE look & feel that is used among all platforms.
 
Changed:
<
<

View to access Calibration Sources Database

>
>

Plug-ins

 
Changed:
<
<
A new view has been added to HIPE for easy access to the Calibration Sources Database (Calsdb).
>
>
HIPE now supports plug-ins, with are small packages that anyone can easily create and distribute to share code (Java or Jython) and data (LocalStores). Plug-ins can be downloaded and installed with a single-click.
 

Plotting

  • Title and label can accept LaTeX commands \textrm \textit \textbf \mathrm \mathit \mathbf. for example, V$_{\textrm{LSR}}$
Line: 113 to 113
 

Calibration sources

Added:
>
>
A new view has been added to HIPE for easy access to the Calibration Sources Database (Calsdb).
 

Products and datasets

  • Product comparison API is available in ia_toolbox_util module.

Revision 332010-09-17 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 147 to 147
 
    • Standarized _ doc _ of general tasks (signature, description, parameter descriptions)
  • Tasks View provides tooltips with the signature of the tasks
  • Examples of general tasks in the URM are now tested automatically
Changed:
<
<
  • Compress and Decompress tasks: they support zip, tar, gzip, tgz in a portable way and compress can guess the compression of a file.
>
>
  • Compress and Decompress tasks: they support zip, tar, gzip, tgz in a portable way and decompress can guess the compression of a file.
 
  • FitsReader supports compressed FITS files.

Quality control

Revision 322010-09-17 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 10 to 10
 

HIPE

Added:
>
>

Navigator view

  • Files ending with extension ".csv" are recognized as tables, and opened as such with double click.
 

Editors & Viewers

  • The small level-2 preview shown in the top-right corner of the Observation Context viewer can be enlarged by just clicking on it.

Revision 312010-09-16 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 10 to 10
 

HIPE

Added:
>
>

Editors & Viewers

  • The small level-2 preview shown in the top-right corner of the Observation Context viewer can be enlarged by just clicking on it.
 

Startup & Shutdown

  • If the running Java platform is not supported, a warning is popped up.

Revision 302010-09-16 - JuanCarlosSegovia

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 107 to 107
 

Products and datasets

Added:
>
>
  • Product comparison API is available in ia_toolbox_util module.
 

Observation context

  • Include new metadata "slewTime" returning the scheduled start time of the slew as a FineTime

Revision 292010-09-16 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 29 to 29
 

Dataset viewer

Astronomical utilities

Added:
>
>
  • RadialVelocity now uses TimeCorrProduct to select pointing data
  • SSo now uses TimeCorrProduct to select pointing data
  • Improved documentation
 

Numeric routines

Data fitting

Line: 83 to 86
 

Analysis

Herschel Science Archive

Added:
>
>
  • Support for exporting data in packed HSA format.
  • Import and Export Views improvements:
    • better error reporting
    • file dialogs can show hidden files
    • versions of observations inside pools shown
 

Pipeline processing

  • Crated a daemon dispatcher mechanism.
  • Aliases set tool has been modified for performing the required verifications before a tag is ingested.
Line: 121 to 130
 
  • HsaReadPool can retrieve product metadata directly.

Tasks

Added:
>
>
  • Updated task sample code
  • Updated documentation:
    • wiki chapters
    • Javadoc of general tasks
    • Standarized _ doc _ of general tasks (signature, description, parameter descriptions)
  • Tasks View provides tooltips with the signature of the tasks
  • Examples of general tasks in the URM are now tested automatically
  • Compress and Decompress tasks: they support zip, tar, gzip, tgz in a portable way and compress can guess the compression of a file.
  • FitsReader supports compressed FITS files.
 

Quality control

Line: 146 to 164
 
  • On-demand request are now recorded into a report file

Data input-output

Added:
>
>
  • Imporved error reporting in AsciiTableTool
 
  • FitsArchive
    • FITS Metadata direct access.
    • ESO Hierarch keyword implemented when reading FITS files.
Line: 158 to 177
 
slewTime SLEWTIME
proCaveats PCAVEATS
    • FitsArchive allow flag for compression when saving FITS files.
Added:
>
>
    • FitsArchive support reading gzipped FITS files and compressed stream headers.
 
    • HeaderWriter exceptions have been modified to include more information.
    • Documentation updates.
    • JExamples automatic testing support.

Revision 282010-09-16 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 10 to 10
 

HIPE

Added:
>
>

Startup & Shutdown

  • If the running Java platform is not supported, a warning is popped up.
 

Plug-ins

HIPE now supports plug-ins, with are small packages that anyone can easily create and distribute to share code (Java or Jython) and data (LocalStores). Plug-ins can be downloaded and installed with a single-click.

View to access Calibration Sources Database

Changed:
<
<
A new view has been to added to HIPE for easy access to the Calibration Sources Database (Calsdb).
>
>
A new view has been added to HIPE for easy access to the Calibration Sources Database (Calsdb).
 

Plotting

* Title and label can accept LaTeX commands \textrm \textit \textbf \mathrm \mathit \mathbf. for example, V$_{\textrm{LSR}}$

Revision 272010-09-16 - JuanCarlosSegovia

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 36 to 36
 

Random Numbers

  • New random numbers generators from different distribution: Uniform, Gaussian, Poisson, Gamma, Cauchy (Lorentz), Exponential and Jeffreys.

Statistics functions

Added:
>
>
  • RMS has been renamed to QRMS
  • RMS implements a new functionality (see URM)
  • Improved StdDev and Variance algorithms.
  • Improved CorrelateMatrix algorithm.
 
New functions
  • GeoMean which calculates the geometric mean of a array of numeric data types with 1 to 5 dimensions.
  • Mode which returns the mode(s), or the most common element(s), of an array of numeric data with 1 to 5 dimensions.
Line: 45 to 50
 

Trend analysis

Mapping

Added:
>
>

Other functions

  • SigClip
    • Median mode uses Median Absolute Deviation instead of Standard deviation.
    • Iterative use is allowed.
 

Source extraction

  • No significant changes to functionality.
Line: 70 to 79
 

Analysis

Herschel Science Archive

Added:
>
>

Pipeline processing

  • Crated a daemon dispatcher mechanism.
  • Aliases set tool has been modified for performing the required verifications before a tag is ingested.
  • HsaPoolDaemon
    • releases memory calling Java garbage collector if it is needed.
    • can write compressed files.
    • can save a binary representation of a product if the FITS save operation fails.
  • HsaStagePool can export products into a LocalStore.
  • HSA ingestion procedure can generate an XML ingestion request file even if a previous version does not contain its own XML.
  • Improved test-harnesses to include an HAIO simulated server framework.
 

Calibration sources

Line: 93 to 112
  Several bugfixes to improve the effectiveness of caching and the robustness has been introduced. One of these is a check to verify that the default LocalStore directory is not modified at any time during the cache's existence, which is a common source of corruption of the cache.
Added:
>
>

HsaPool

  • HsaReadPool/HsaXmlPool can work with compressed data.
  • HsaReadPool can retrieve product metadata directly.
 

Tasks

Quality control

Line: 119 to 142
 
  • On-demand request are now recorded into a report file

Data input-output

Added:
>
>
  • FitsArchive
    • FITS Metadata direct access.
    • ESO Hierarch keyword implemented when reading FITS files.
    • FITS translation dictionary updated:
      Herschel keyword FITS keyword
      pmRA PMRA
      pmDEC PMDEC
      state STATUS
      action ACTION
      slewTime SLEWTIME
      proCaveats PCAVEATS
    • FitsArchive allow flag for compression when saving FITS files.
    • HeaderWriter exceptions have been modified to include more information.
    • Documentation updates.
    • JExamples automatic testing support.
 

Virtual Observatory

Revision 262010-09-16 - JoseOsinde

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 77 to 77
 

Observation context

Added:
>
>
  • Include new metadata "slewTime" returning the scheduled start time of the slew as a FineTime
    • Specific accessors, getSlewStartTime() and setSlewStartTime(), where added to the ObservationContext class
    • For old observations in the hsa without this value, the method will return a null value indicating that this field was never set before
 

History

A new history dataset viewer is available: When you click on the History dataset in your product then two tables appear above another:

Revision 252010-09-15 - AnthonySmith

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 47 to 47
 

Mapping

Source extraction

Added:
>
>
  • No significant changes to functionality.
 

Images

Display

Revision 242010-09-15 - JoseOsinde

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 92 to 92
 

Quality control

Added:
>
>
  • New metadata into the QualityContext product. The field is fixed and the main interest is to include this information into the FITs file associated to the product.

FITS keyword Value
PCAVEATS Please refer to http://herschel.esac.esa.int/DpKnownIssues.shtml for known problems in products

  • Perform bulk actions on several qc reports simultaneously
  • Flags color codes depending on their importance
 

Systematic product generation

  • New framework to process/combine data from several observations in the same pipeline
    • This will allow, for instances, better results using MADmap algorithm with cross-scan data from two or more observations
Added:
>
>
  • Automatic processing:
    • New start-time algorithm based on the DTCP time of the OD being processed instead of fixed timestamps
    • New check on TM data based on the gap checker reports. Only those observations with complete tm data will be processed at this point
 
  • Uplink plugin is executed without problems in different conditions, even for manual commanding and no uplink data at all.
Deleted:
<
<
  • New start-time algorithm for automatic processing based on the DTCP time of the OD being processed instead of fixed timestamps
  • SPG stand-alone application can now run on MAC O.S. with no versant libraries available.
  • Include proper motion data into the ObservationContext
 
  • Obsids shown also in decimal format in all the application tables
Added:
>
>
  • Include proper motion data into the ObservationContext
  • SPG stand-alone application can now run on MAC O.S. with no versant libraries available.
  • On-demand request are now recorded into a report file
 

Data input-output

Revision 232010-09-15 - JoseOsinde

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 94 to 94
 

Systematic product generation

Added:
>
>
  • New framework to process/combine data from several observations in the same pipeline
    • This will allow, for instances, better results using MADmap algorithm with cross-scan data from two or more observations
  • Uplink plugin is executed without problems in different conditions, even for manual commanding and no uplink data at all.
  • New start-time algorithm for automatic processing based on the DTCP time of the OD being processed instead of fixed timestamps
  • SPG stand-alone application can now run on MAC O.S. with no versant libraries available.
  • Include proper motion data into the ObservationContext
  • Obsids shown also in decimal format in all the application tables
 

Data input-output

Virtual Observatory

Revision 222010-09-15 - PaulBalm

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 10 to 10
 

HIPE

Added:
>
>

Plug-ins

HIPE now supports plug-ins, with are small packages that anyone can easily create and distribute to share code (Java or Jython) and data (LocalStores). Plug-ins can be downloaded and installed with a single-click.

View to access Calibration Sources Database

A new view has been to added to HIPE for easy access to the Calibration Sources Database (Calsdb).

 

Plotting

Changed:
<
<
* Title and label can accept LaTeX commands \textrm \textit \textbf \mathrm \mathit \mathbf. for example, V$_{\textrm{LSR}}$
>
>
* Title and label can accept LaTeX commands \textrm \textit \textbf \mathrm \mathit \mathbf. for example, V$_{\textrm{LSR}}$
 

TablePlotter

OverPlotter

Dataset viewer

Line: 76 to 84
 

Product Access Layer

Added:
>
>

Cache

Several bugfixes to improve the effectiveness of caching and the robustness has been introduced. One of these is a check to verify that the default LocalStore directory is not modified at any time during the cache's existence, which is a common source of corruption of the cache.

 

Tasks

Quality control

Line: 90 to 102
 

Documentation

Added:
>
>
To improve the quality of the examples, we have introduced automatic testing of (many of) the source code examples occurring in manuals such as the Data Analysis Guide and the HIPE Owner's Guide. We have also introduced automatic testing of the examples occurring in the User's Reference Manual.
 

HIFI

hifiPipelineTask

Revision 212010-09-14 - PeterKennedy

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 266 to 266
 
    • Added a new Color&Mask Preferences panel
    • Improved the color scale (using the Color&Mask Preferences it's now possible to select the Min/Max value fot the color scale)
    • Improved the mask visualization (using the Color&Mask Preferences it's now possible to visualize any specific mask value directly on the array display)
Changed:
<
<
>
>
  • Made available the Mosaic class which creates a mosaic of plots in the shape of the SLW, SSW or coaligned detectors of the SPIRE FTS. The Mosaic class is intended to provide a means for viewing (at a glance) the spectra or interferograms associated with each detector.
  • Changed SpecExplorer's detector display so that it automatically resizes to fit the enclosing frame. Also, SpecExplorer's detectors are now enclosed in a splitter pane. This enables you to maximize or hide the detector display with a single click.
 

Documentation

Revision 202010-09-14 - JeroenDeJong

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 65 to 65
 

Calibration sources

Products and datasets

Added:
>
>
 

Observation context

Added:
>
>

History

A new history dataset viewer is available: When you click on the History dataset in your product then two tables appear above another:

  • An overview table which shows all the tasks used to generate the product. For each task, the name, execution time, build number and used calibration files are shown.
  • When you click on a row in this overview table then a second table is shown with details for the selected task. This table contains the details of this task: For each parameter its simple value and some product information is shown (when available).
 

Product Access Layer

Tasks

Line: 193 to 201
 
    • PacsScr:2964: new wavelength calibration applied in waveCalc task using the wavePolynomes cal file
  • spectrometer trend analysis product for the calibration blocks:
    • PacsScr:2777: calibration block trendAnalysis product added to the observation context
Added:
>
>
  • Extra options to specDiffChop (note these are EXPERT options for the analysis of special observations!)
    • PacsScr:2717: extra option 'extend' added which allows for other pairwise differencing schemes. These schemes will subtract more datapoints from each other than the standard scheme.
    • PacsScr:2427: extra option 'chopNodScheme' which provides different scheme for determining which label is on-source and which is off-source for a given nodding position. Needed for some special test AORs.
  • Different bands in a range scan are now stored in their own blocks in the block table. This means that it is now possible to slice SEDs per band (PacsScr:2963).
  • Several fixes to specWaveRebin, etc. to reduce the amount of NaNs considerably in the rebinned data.
  • SpecProject sets the flux to Jy.

 

Calibration

Line: 206 to 222
 
  • PacsSpr:2607 The RSRF calibration tables contain "NaN" which are then introduced into the signal by rsrfCal

Analysis tools

Added:
>
>
 

Photometry

Added:
>
>
 

Spectrometry

Added:
>
>
  • Compatibility of PACS products with HCSS tools improved: One can now properly select data points and use the fitter tool on PACS frames and cubes.
 

Product changes

Photometry

Spectrometry

Revision 192010-09-14 - GabrieleMainetti

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 240 to 240
 

Spectrometer Pipeline

Interactive Analysis and Tools

Added:
>
>
  • Added the new DTMosaic class that allows the user to plot data recorded by many detectors during one observation on a single page to compare data more effectively (mosaic plot). This features is available from HIPE command line or within DetectorTimelineExplorer
 
Changed:
<
<
    • Add a new Color&Mask Preferences panel
>
>
    • Added a new features that allows the creation of mosaic plot of the detectors via right-click on the desired array
    • Added a new Color&Mask Preferences panel
 
    • Improved the color scale (using the Color&Mask Preferences it's now possible to select the Min/Max value fot the color scale)
    • Improved the mask visualization (using the Color&Mask Preferences it's now possible to visualize any specific mask value directly on the array display)

Revision 182010-09-13 - CateLiu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 142 to 142
 
    • Extended Madmap (using scan and cross scan) for interactive usage
      • HIPE -> Menu -> pipeline -> PACS -> L2_scanMap.py
Changed:
<
<
Based on HCSS-9361, a prototype for the extended Madmap pipeline is now available in HIPE for interactive usage. The extended Madmap pipeline is designed to combine scans and cross-scans in order to produce optional map. Different from DP pipelines (which normally starts with one ObservationContext), this pipeline takes an obsList which is a list of ObservationContext.
>
>
Based on HCSS-9361, a prototype for the extended Madmap pipeline is now available in HIPE for interactive usage. The extended Madmap pipeline is designed to combine scans and cross-scans in order to produce optional map. Unlike DP pipelines (which normally starts with one ObservationContext), this pipeline takes an obsList (an ArrayList) which is a list of ObservationContext. The pipeline will run even there is only one obs in the obsList. Below is an example creating an obsList:
HIPE> obsList=ArrayList()
HIPE> obsList.add(obs1)
HIPE> obsList.add(obs2)
 
  • New script for scan Map pipeline
    • PacsSpr:2801 Proposal for a two-stage photometer pipeline with masked highpass filtering

Revision 172010-09-13 - CateLiu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 138 to 138
 

Photometry

Changed:
<
<
>
>
  • MadMap
 
    • Extended Madmap (using scan and cross scan) for interactive usage
      • HIPE -> Menu -> pipeline -> PACS -> L2_scanMap.py
Added:
>
>

Based on HCSS-9361, a prototype for the extended Madmap pipeline is now available in HIPE for interactive usage. The extended Madmap pipeline is designed to combine scans and cross-scans in order to produce optional map. Different from DP pipelines (which normally starts with one ObservationContext), this pipeline takes an obsList which is a list of ObservationContext.
 
  • New script for scan Map pipeline
    • PacsSpr:2801 Proposal for a two-stage photometer pipeline with masked highpass filtering
      • Now THE standard processing script for scan maps

Revision 162010-09-13 - CateLiu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Revision 152010-09-13 - CateLiu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 25 to 25
 
  • The SpectrumFitter (commandline version) has methods to export the data as the input type and as ASCII.
  • The SpectrumFitter (commandline version) has method that returns a jython script

Data interpolation

Added:
>
>

Random Numbers

  • New random numbers generators from different distribution: Uniform, Gaussian, Poisson, Gamma, Cauchy (Lorentz), Exponential and Jeffreys.
 

Statistics functions

New functions
  • GeoMean which calculates the geometric mean of a array of numeric data types with 1 to 5 dimensions.

Revision 142010-09-13 - MichaelWetzstein

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 133 to 133
  Other small changes:
    • getCalTree(), getCalProduct() now understand the 'verbose=True' keyword and will print information on the location of the calibration products.
    • getCalProduct() the third argument, 'version', is no longer mandatory, if not given, the last version of the calibration product will be returned.
Changed:
<
<
>
>
 

Photometry

Line: 148 to 148
  more to guess Sigclip parameters, run the deglitching and check the quality afterwards. The new option allows to find the best parameter quickly BEFORE deglitching has been applied.
The MapIndexViewer now also displays the signal vectors in the "timeordered" mode.
Changed:
<
<
>
>
    • PacsSpr:2921: the MapIndexViewer shows mean/median +/- error permanently in the signal plot
    • PacsSpr:2807: the Full MapIndex needs less memory than before
    • PacsSpr:2859: Wcs4mapTask got a crota option to manually set the requested rotation angle of a map.
    • PacsSpr:2925: CorrectRaDec4SsoTask is now also accepting the new Horizons class definition. An example how to use it can be found in the attached
    • PacsSpr:2751 : photProject error map has been checked for NaN
    • PacsSpr:2543 : exposure map has been removed in favour of the coverage map
 
  • Noise treatmement
    • PacsSpr:2752: A new modul herschel.pacs.spg.phot.PhotAddNoisePerPixelTask provides 4 ways to calculate the initial photometer noise. *Noise propagation is also implemented in the HighpassFilter, Mean- and MedianFilters.

Revision 132010-09-13 - JuergenSchreiber

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 172 to 172
 

Spectrometry

Added:
>
>
  • new AOR mode "unchopped" including improved spatial calibration:
 
  • First draft of the unchopped mode pipeline added to the build
    • PacsScr:2779: a "newstyle" unchopped mode pipeline is in place which can be run automatically or interactively
  • PacsScr:1048 New task specSubtractOffPosition implemented for unchopped pipeline
Added:
>
>
 
Added:
>
>
  • improved wavelength calibration:
 
  • PacsScr:2964: new wavelength calibration applied in waveCalc task using the wavePolynomes cal file
Added:
>
>
  • spectrometer trend analysis product for the calibration blocks:
 
  • PacsScr:2777: calibration block trendAnalysis product added to the observation context

Calibration

Revision 122010-09-13 - GabrieleMainetti

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 225 to 225
 

Spectrometer Pipeline

Interactive Analysis and Tools

Added:
>
>
  • DetectorTimelineExplorer and DetectorTimelineExplorerComponent :
    • Add a new Color&Mask Preferences panel
    • Improved the color scale (using the Color&Mask Preferences it's now possible to select the Min/Max value fot the color scale)
    • Improved the mask visualization (using the Color&Mask Preferences it's now possible to visualize any specific mask value directly on the array display)
 

Documentation

Revision 112010-09-13 - JuergenSchreiber

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 176 to 176
 
Added:
>
>
  • PacsScr:2964: new wavelength calibration applied in waveCalc task using the wavePolynomes cal file
  • PacsScr:2777: calibration block trendAnalysis product added to the observation context
 

Calibration

Revision 102010-09-13 - EkkehardWieprecht

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 116 to 116
 

PACS

Pipeline

Added:
>
>
  • PACS "New Style Sliced" pipeline implementation is now activated
    • PacsSpr:2647 : phot and spec ipipe script names are changed to be more logic and intuitive
    • Pipeline Script access
      • HIPE -> Menu -> pipeline -> PACS
  • Calibration framework Calibration products are no longer delivered with the build in a local store.
    Calibration products are now delivered as a directory of FITS files.
    This directory with calibration products can be found in the data/pcal sub-directory of your installation directory.
    HIPE> fm = getCalTree(verbose=1)
    CalTree read from: /Software/Herschel/hcss.dp.pacs-5.0.500/data/pcal
    The calibration framework is adapted and will read these FITS files by default.
    The getCalTree() command and all access to calibration data has not changed, under the hood, the local store has been replaced by the directory of FITS files. Other small changes:
    • getCalTree(), getCalProduct() now understand the 'verbose=True' keyword and will print information on the location of the calibration products.
    • getCalProduct() the third argument, 'version', is no longer mandatory, if not given, the last version of the calibration product will be returned.

 

Photometry

Added:
>
>
  • MadMap
    • Extended Madmap (using scan and cross scan) for interactive usage
      • HIPE -> Menu -> pipeline -> PACS -> L2_scanMap.py
  • New script for scan Map pipeline
    • PacsSpr:2801 Proposal for a two-stage photometer pipeline with masked highpass filtering
      • Now THE standard processing script for scan maps
  • Photometer Mapping
    • PacsSpr:2853: Photometer mapping accepts a variable input pixelsize (parameter pixfrac in the PhotProjectTask and the MapIndexTask).
    • PacsSpr:2789: The MapIndexViewer has a Sigclip gui to display the effect of different Sigclip parameters on the second level deglitching. Now it is not necessary any more to guess Sigclip parameters, run the deglitching and check the quality afterwards. The new option allows to find the best parameter quickly BEFORE deglitching has been applied.
      The MapIndexViewer now also displays the signal vectors in the "timeordered" mode.
    • PacsSpr:2859: Wcs4mapTask got a crota option to manually set the requested rotation angle of a map.
    • PacsSpr:2925: CorrectRaDec4SsoTask is now also accepting the new Horizons class definition. An example how to use it can be found in the attached
    • PacsSpr:2751 : photProject error map not useable
    • PacsSpr:2543 : Clarify mapping exposure vs coverage
    • PacsSpr:2751 : photProject error map not useable
  • Noise treatmement
    • PacsSpr:2752: A new modul herschel.pacs.spg.phot.PhotAddNoisePerPixelTask provides 4 ways to calculate the initial photometer noise. *Noise propagation is also implemented in the HighpassFilter, Mean- and MedianFilters.
  • Pointing
  • Others
    • PacsSpr:2761 PACS EDP: phot pipeline adaptation to cope with variable scan velocity
    • A new module photMaskFrames offer masking :
    • framesOut = photMaskFrames(framesIn [,beforeFirstScanLeg] [,noScanData] [,afterCalblock=0] )
      • beforeFirstScanLeg : Mask data befor the first scan leg
      • noScanData : mask data where the ScanBBID is not set (includes turn arounds)
      • afterCalblock : Mask n data after calibration block
    • HcssSpr:6681: Sigclip can be applied n-times in one go.
 

Spectrometry

Added:
>
>
  • First draft of the unchopped mode pipeline added to the build
    • PacsScr:2779: a "newstyle" unchopped mode pipeline is in place which can be run automatically or interactively
  • PacsScr:1048 New task specSubtractOffPosition implemented for unchopped pipeline
  • PACS Product interface to SpectrumExplorer ( e.g. PacsSpr:2843 )
 

Calibration

Added:
>
>
 

Photometry

Added:
>
>
 

Spectrometry

Added:
>
>
  • PacsSpr:2488 : New Wavelength Calibration Product
  • PacsSpr:2855 v8 of the Pacs Spectrometer Array to Intrument and v4 of the Module2Array cal file is inserted in the system. This generates correct values also for center chopper positions (unchopped and waveswitch mode).
  • PacsSpr:2715 : updated spectrometer spatial cal file --> PCalSpectrometer_ArrayInstrument_FM_v6.fits
  • PacsSpr:2607 The RSRF calibration tables contain "NaN" which are then introduced into the signal by rsrfCal
 

Analysis tools

Photometry

Revision 92010-09-10 - YiMei

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 25 to 25
 
  • The SpectrumFitter (commandline version) has methods to export the data as the input type and as ASCII.
  • The SpectrumFitter (commandline version) has method that returns a jython script

Data interpolation

Added:
>
>

Statistics functions

New functions
  • GeoMean which calculates the geometric mean of a array of numeric data types with 1 to 5 dimensions.
  • Mode which returns the mode(s), or the most common element(s), of an array of numeric data with 1 to 5 dimensions.
  • Covariance which yields the covariance between two random variables/vectors x and y with finite second moments. If one vector is longer than the other, only the values up to the length of the shorter vector will be taken into account.
  • CovarianceMatrix which returns the covariance matrix of the input M x N matrix. The result is a N x N matrix with each i, j value equal to the covariance of the ith and jth columns of the original matrix.
 

Trend analysis

Mapping

Revision 82010-09-10 - SaraRegibo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 41 to 41
 
  • Speed improvement of createRgbImageTask
  • Update of URM documentation
  • Cropping an image by drawing a rectangle on it
Added:
>
>
  • Correction of the calculation of the dimensions of a mosaic
 

Spectra

Display

Analysis

Revision 72010-09-10 - WimDeMeester

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 33 to 33
 

Images

Display

  • Images can be opened as RgbImage directly from the Navigator in HIPE.
Added:
>
>
  • Regular image files (jpg, gif, png, ...) are shown as preview in the outline when being clicked on in the navigator view of hipe.
 
  • 2 images can be compared by setting the opacity of the image. This can be done using the setOpacity(float) method or using a slider in the Image Display.

Analysis

  • Added the methods containsNorthCelestialPole and containsSouthCelestialPole on the Wcs.
Changed:
<
<
>
>
  • Regridding an image on the grid of another image using RegridTask
  • Speed improvement of createRgbImageTask
  • Update of URM documentation
  • Cropping an image by drawing a rectangle on it
 

Spectra

Display

Analysis

Revision 62010-09-10 - PasqualePanuzzo

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 133 to 133
 

Engineering Conversion

Common Pipeline

Added:
>
>
  • Only minor changes in logs and documentation.
 

Photometer Pipeline

Added:
>
>
  • Pipeline scripts:
    • Added time correlation task in scan map pipelines
    • Comment out optical and electrical crosstalk corrections
    • Removed plotting blocks
  • Pipeline tasks:
 

Spectrometer Pipeline

Revision 52010-09-09 - RoBaS

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 20 to 20
 

Numeric routines

Data fitting

Added:
>
>
 

Data interpolation

Trend analysis

Mapping

Revision 42010-07-20 - JingjingLi

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 11 to 11
 

HIPE

Plotting

Added:
>
>
* Title and label can accept LaTeX commands \textrm \textit \textbf \mathrm \mathit \mathbf. for example, V$_{\textrm{LSR}}$
 

TablePlotter

OverPlotter

Dataset viewer

Revision 32010-07-14 - WimDeMeester

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 30 to 30
 
  • Images can be opened as RgbImage directly from the Navigator in HIPE.
  • 2 images can be compared by setting the opacity of the image. This can be done using the setOpacity(float) method or using a slider in the Image Display.

Analysis

Added:
>
>
  • Added the methods containsNorthCelestialPole and containsSouthCelestialPole on the Wcs.
 

Spectra

Display

Revision 22010-06-28 - WimDeMeester

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

Line: 27 to 27
 

Images

Display

Added:
>
>
  • Images can be opened as RgbImage directly from the Navigator in HIPE.
  • 2 images can be compared by setting the opacity of the image. This can be done using the setOpacity(float) method or using a slider in the Image Display.
 

Analysis

Spectra

Revision 12010-06-28 - DavideRizzo

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="HipeWhatsNew"

What's New in HIPE 5.0

PDF Version Click here

Core system

HIPE

Plotting

TablePlotter

OverPlotter

Dataset viewer

Astronomical utilities

Numeric routines

Data fitting

Data interpolation

Trend analysis

Mapping

Source extraction

Images

Display

Analysis

Spectra

Display

Analysis

Data cubes

Display

Analysis

Herschel Science Archive

Calibration sources

Products and datasets

Observation context

Product Access Layer

Tasks

Quality control

Systematic product generation

Data input-output

Virtual Observatory

Installer

Documentation

HIFI

hifiPipelineTask

HRS Pipeline

WBS Pipeline

Level 1 Pipeline

Level 2 Pipeline

Products

DoDeconvolution

DoGridding

FitHifiFringe

HiClass export tool

Flags

SpectrumExplorer (HIFI specifics)

HIFI DP scripts (User tool scripts)

Documentation

Other

PACS

Pipeline

Photometry

Spectrometry

Calibration

Photometry

Spectrometry

Analysis tools

Photometry

Spectrometry

Product changes

Photometry

Spectrometry

Documentation

Other

SPIRE

Calibration Products

Calibration Framework

Preprocessing

Engineering Conversion

Common Pipeline

Photometer Pipeline

Spectrometer Pipeline

Interactive Analysis and Tools

Documentation

Other

<--  
-->
 
This site is powered by the TWiki collaboration platform Powered by Perl