Difference: HipeWhatsNew14x (1 vs. 78)

Revision 782017-02-01 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in version 14.x

Line: 315 to 315
 

Spectroscopy

Changed:
<
<
  • Extended source correction. The extended-source calibrated rebinned cubes (level2) are subjected to a correction that accounts for the non-uniform illumination of the PACS IFU field-of-view. It stems from the fact that the 9.4"x9.4" spaxel edges are less responsive than their centres, translating into a spaxel active area of approximately 8"x8" where the response is flat. This effect is more severe at shorter wavelengths, and affects (very) extended emission, as well as off-centred point sources and any source that has a size between a point and a very extended. All Level 2/2.5 cubes for these types of sources will be affected, whether pointed or mapping.
    Hence, for centred point source extracted with the task extractCentralSpectrum, no change to the spectra are expected in this version of HIPE because of this effect. For very extended sources, we have developed a new Extended Source Correction calibration file (ExtendedSourceLoss, applied in the pipeline tasks specRespCal and specRespCalToTelescope), and this incorporates the necessary correction as part of the spatial flatfielding for the PACS spectrometer. For these sources, the fluxes of the spectra will differ to what was obtained in previous versions of HIPE. For intermediate sources, a more complex analysis is necessary to estimate the flux loss, and a task to do this will be provided in HIPE 15.
    The fluxes of the spaxels of all cubes will be increased by ~30% in the blue to ~5% in the red. The calibration file can be inspected to extract the curve:
    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data
    p=PlotXY(w, 1./f)
    After such a correction, a source of flat surface brightness within the field of view of a single pointing will be well calibrated at the level of the rebinned cubes (i.e. level2). The flux loss between the detectors was already accounted for in the point-source calibration correction in the correspondent calibration file, therefore, point-source calibrated spectra of centred point sources are not affected. For this reason, in HIPE 14, the extractCentralSpectrum task checks and removes the extendedSourceLoss correction from the level2 cubes, in case it has been applied, before extracting the final point source corrected spectrum. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: this task can be found in the "Point source loss correction (any spaxel)" script in the Scripts->PACS useful menu.
>
>
  • Extended source correction. The extended-source calibrated rebinned cubes (level2) are subjected to a correction that accounts for the non-uniform illumination of the PACS IFU field-of-view. It stems from the fact that the 9.4"x9.4" spaxel edges are less responsive than their centres, translating into a spaxel active area of approximately 8"x8" where the response is flat. This effect is more severe at shorter wavelengths, and affects (very) extended emission, as well as off-centred point sources and any source that has a size between a point and a very extended. All Level 2/2.5 cubes for these types of sources will be affected, whether pointed or mapping.
    Hence, for centred point source extracted with the task extractCentralSpectrum, no change to the spectra are expected in this version of HIPE because of this effect. For very extended sources, we have developed a new Extended Source Correction calibration file (ExtendedSourceLoss, applied in the pipeline tasks specRespCal and specRespCalToTelescope), and this incorporates the necessary correction as part of the spatial flatfielding for the PACS spectrometer. For these sources, the fluxes of the spectra will differ to what was obtained in previous versions of HIPE. For intermediate sources, a more complex analysis is necessary to estimate the flux loss, and a task to do this will be provided in HIPE 15.
    The fluxes of the spaxels of all cubes will be changed by ~+30% in the blue to ~-5% in the red. The calibration file can be inspected to extract the curve:
    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data
    p=PlotXY(w, 1./f)
    After such a correction, a source of flat surface brightness within the field of view of a single pointing will be well calibrated at the level of the rebinned cubes (i.e. level2). The flux loss between the detectors was already accounted for in the point-source calibration correction in the correspondent calibration file, therefore, point-source calibrated spectra of centred point sources are not affected. For this reason, in HIPE 14, the extractCentralSpectrum task checks and removes the extendedSourceLoss correction from the level2 cubes, in case it has been applied, before extracting the final point source corrected spectrum. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: this task can be found in the "Point source loss correction (any spaxel)" script in the Scripts->PACS useful menu.
 
  • New beam calibration files have been delivered, differing from the previous files only in the way they are normalised. These will be used in a new task being developed in HIPE 15, and do not impact on HIPE 14 or SPG 14 in any way.
    • all the 25 beams were divided by the peak value of the central spaxel beam;
    • then, they are multiplied to match the Point Source Correction calibration file (pointSourceLoss FM_4).

Revision 772017-01-11 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in version 14.x

Line: 334 to 334
 

Spectroscopy

  • A new script has been provided to allow users to plot the telescope background spectrum model. The code and necessary files are provided on http://herschel.esac.esa.int/twiki/bin/view/Public/PacsCalibrationWeb?template=viewprint (PACS Spectrometer Calibration section).
  • PacsCubes and PacsRebinnedCubes created by the pipeline will now display in HIPE cube viewers with a flip over the diagonal, as compared to previously (X: to the right, Y: upwards). This change allows the visualisation of PACS cubes into the PACS footprint orientation (note the axes display with X: downwards, Y: to the left). The displayed indices correspond to the PACS spaxel indices (see more in Sect. 6.3 of the DAG). The standard HIPE cube visualisation can be recovered in two steps, using the flip menu in the cube display control bar: no flip + vertical flip. When exporting the cube data outside HIPE, note that the data will still be organised according to standard cube indices.The new orientation makes it easier to compare the cubes in cube viewers (the Standard Cube Viewer or the Spectrum Explorer) to the display of the PacsSpectralFootprint viewer.
Added:
>
>
  • Two new Useful scripts: to create mosaic cubes with any spaxel size (prime use-case: wanting to give drizzled cubes the same spaxel size for all wavelengths for a single observation) starting from the Level 2 products; to mosaic together unrelated observations that overlap spatially and spectrally, starting from Level 1 products
 

Product changes

Line: 345 to 346
 
  • The error layer within Unimap Level 2.5 products is renamed as stdDev, being the standard deviation computed at every sky pixel with respect to the samples that are falling on it. It can be properly assumed as the error map only if associated to the NaÔve map.

Spectroscopy

Added:
>
>
  • The following spectroscopy obsids will receive a final SPG processing in 14.2.2: 1342247513, 1342246558,1342225987, 1342225145, 1342225789, 1342247790
 
  • Unchopped range scan observations in 14.2
    • For unchopped range scan observations, the on-source and off-source are separate obsids and each is reduced to Level2. Then the off-source data are subtracted from the on-source data and the "background subtracted" products added to Level 2.5. This has been the case for a long time. The difference now is that these products have a slightly different name: "BS" is appended to the product names as viewed in HIPE and to the FITS file names on disc.
Changed:
<
<
  • Level 2/2.5 extracted spectra tables in 14.0, 14.2
>
>
  • Level 2/2.5 extracted spectral tables in 14.0, 14.2
 
    • A new product is provided at Level 2 for all (line, range, chopNod, unchopped) pointed observations, and also at Level 2.5 for the subset of unchopped, rangeScan observations. This table, HPSSPEC[BS][R|B], contains the extracted spectrum of the central spaxel, and the point source calibrated spectra "c1" and "c129" from the pipeline task extractCentralSpectrum. Since the point-source calibrated spectra are taken from the centre of the cube, it is necessary that the source is located within spaxel 2,2 (the central one). However, we make no determination that the target of the observation actually is a point source - that is up to the end-user to establish. These tables are also provided as SBPs.
    • Added in 14.2.0: the output spectrum "c9" from extractCentralSpectrum is also provided.
  • Level 3 extracted spectra table in 14.0, 14.2

Revision 762016-12-22 - AlvarGarcia

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in version 14.x

Line: 13 to 13
 

Highlights

Core

Changed:
<
<
  • Now you can pass arguments to scripts in batch mode (former jylaunch).
>
>
  • Now you can pass arguments to scripts in batch mode (former jylaunch).
 
  • New Quick Analysis perspective allows GUI download of products and processing all within HIPE.
  • New Auxiliary Observation Summary Product with information about the observation in a simple dataset.
  • Quality flags have been reorganised.

Revision 752016-12-19 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in version 14.x

Line: 34 to 34
  PACS
  • Spectroscopy calibration
Changed:
<
<
    • 14.0: a new Extended Source Calibration (ESCF) is released: fully extended sources are now better calibrated, and for point and semi-extended sources treated with their dedicated tasks there is no change. Any source between these two extremes will still suffer from intra-spaxel flux losses, but less so than before.
>
>
    • 14.0: a new Extended Source Calibration (ESCF) is released: fully extended sources are now better calibrated, and for point and semi-extended sources treated with their dedicated tasks there is no change as the ESCF is taken out before the point/semi-extended correction is applied. Sources that lie between fully extended and a point, and point or semi-extended sources that are not processed with the dedicated tasks, will have slightly incorrect fluxes--at most wavelengths the fluxes will in fact be _over_corrected by the ESC. A "forward modelling" tool will be release in HIPE 15 to help users estimate the correct fluxes.
 
    • 14.0: drizzled cubes have a dedicated calibration and pipeline, and produce correct fluxes (a vast improvement on previous releases).
    • 14.2: a new spatial calibration (spaxel sizes and layout) has been released. There are slight differences from the one used previously. The flux distribution in mosaic cubes (interpolated, projected, drizzled) will change as a result.
    • The calibration tree update released for 14.2 includes a new RSRFR1 calibration file that can be used with the interactive pipelines to produce data in the red leak (>190 micron) region.
Line: 47 to 47
 
    • 14.2: for all observations, data below 55 microns and above 190 microns are excluded.
  • Photometer products
    • Level 2.5 maps of solar system objects have been created for 14.0 and 14.2.
Changed:
<
<
    • 14.2: Unimap mapper has been upgraded to the version 6.4.4.
>
>
    • 14.2: Unimap mapper has been upgraded to version 6.4.4.
 SPIRE
  • New in 14.1 Major impact: SPIRE Spectrometer extended source calibrated spectra are corrected for the far-field feedhorn efficiency. The correction leads to systematic ~40% higher intensities for SSW and ~40-100% for SLW. All extended source calibration products - spectra or spectral cubes - are affected by this significant effect. With this correction the extended source calibrated SPIRE Photometer and Spectrometer products are in good agreement.
  • Correction for artefacts in SPIRE Spectrometer Low Resolution (LR) point source calibrated spectra.

Revision 742016-12-02 - AlvarGarcia

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in version 14.x

Line: 38 to 38
 
    • 14.0: drizzled cubes have a dedicated calibration and pipeline, and produce correct fluxes (a vast improvement on previous releases).
    • 14.2: a new spatial calibration (spaxel sizes and layout) has been released. There are slight differences from the one used previously. The flux distribution in mosaic cubes (interpolated, projected, drizzled) will change as a result.
    • The calibration tree update released for 14.2 includes a new RSRFR1 calibration file that can be used with the interactive pipelines to produce data in the red leak (>190 micron) region.
Added:
>
>
  • Spectroscopy and photometer products
    • Note that in some cases the level2.5 and level3 products have been produced with a higher version of 14 than the level0 to level2 processing.
 
  • Spectroscopy products
    • 14.0: spectrum tables have been added to the higher levels and the standalone browse products.
    • 14.2: for unchopped range scans, the on-source and the background subtracted on-source products (i.e. on-off) both are provided in the standalone browse products.
    • 14.2: interpolated cubes have an error array.
    • 14.2: for all observations, data below 55 microns and above 190 microns are excluded.
Changed:
<
<
  • Photometer
>
>
  • Photometer products
 
    • Level 2.5 maps of solar system objects have been created for 14.0 and 14.2.
    • 14.2: Unimap mapper has been upgraded to the version 6.4.4.
SPIRE
Line: 52 to 54
 
  • The default SPIRE Spectrometer spectral cubes are now produced with the convolution projection (CP) method: the browse images/thumbnails and standalone browse products are now the CP cubes.
  • In HIPE 14.1, the SPIRE Spectrometer bright mode calibration was fixed.
  • In HIPE 14.2.1, the mosaicked SPIRE Photometer Level-3 maps are correctly calculated.
Added:
>
>
  • Note that in some cases the level2.5 and level3 products have been produced with a higher version of 14 than the level0 to level2 processing.
  NOTE: Highlights with a version number in bold represent a change from version 14.0.
Line: 336 to 342
 
  • Many improvements to the descriptions and/or units of the header meta keywords.
  • All keywords have now a corresponding 8 character FITS keyword, so that the headers are easier to read with other software.
Added:
>
>
  • Note that in some cases the level2.5 and level3 products have been produced with a higher version of 14 than the level0 to level2 processing.
 

Photometry

  • The error layer within Unimap Level 2.5 products is renamed as stdDev, being the standard deviation computed at every sky pixel with respect to the samples that are falling on it. It can be properly assumed as the error map only if associated to the NaÔve map.
Line: 382 to 390
 

Engineering Conversion

Common Pipeline

Added:
>
>
  • Note that in some cases the level2.5 and level3 products have been produced with a higher version of 14 than the level0 to level2 processing.
 

Photometer Pipeline

  • The User Pipelines have been updated with an option to create not just the point source maps but also the absolute calibtated maps in MJy/sr by setting the makeExtendMaps option

Revision 732016-10-11 - LucaCalzoletti

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in version 14.x

Line: 296 to 294
 
    • The script scanmap_Unimap_memory_optimized has been added to the Pipelines menu. This allows one to run the Unimap release located on the userís computer while using the minimum amount of memory. It is particularly useful for processing large observations or/and if several obsIDs should be processed all together in a single run.
    • The Unimap interactive scripts (scanmap_Unimap_memory and scanmap_Unimap_memory_optimized) are updated in HIPE14.2 in order to be compatible with any Unimap release (up to 6.5.3).
    • The script photextended_pipeline_L3 is now available in the Pipelines menu. This can be used to generate mosaics of Unimap and Jscanam Level2.5 maps.
Changed:
<
<
  • Unimap : In SPG14.0.1, Unimap 6.2.0 is spawned to create Level 2.5 products, while in SPG14.2 Level 2.5 products are generated by using Unimap 6.4.4. The main differences between the two Unimap releases are the introduction of the pixel noise compensation and the generation of the proper error maps in Unimap 6.4.4 (See the Photometer Data Reduction Guide and the offical Unimap webpages (http://infocom.uniroma1.it/unimap/) for details).
>
>
  • Unimap : In SPG14.0.1, Unimap 6.2.0 is spawned to create Level 2.5 products, while in SPG14.2 Level 2.5 products are generated by using Unimap 6.4.4. The main differences between the two Unimap releases are the introduction of the pixel noise compensation and the generation of the proper error maps in Unimap 6.4.4 (See the Photometer Data Reduction Guide and the offical Unimap webpages (http://infocom.uniroma1.it/unimap/) for details). For SPG14.2.1, the values of some Unimap parameters were modified for a small set of observations, because of a bug in the 6.4.4 Unimap release (see PACS Photometer section at the Data Products Known Issues wikipage)
 

Spectroscopy

  • Interactive pipeline scripts in 14.0, 14.2

Revision 722016-09-29 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in version 14.x

Line: 349 to 351
 
  • Level 3 extracted spectra table in 14.0, 14.2
    • The new spectrum tables from Level 2 for the pointed, chopNod, full SED observations, are concatenated into a single table which is placed in Level 3. This table, HPSSPEC, contains the extracted spectra for all the observations that were taken to cover the full spectral range (the full SED) of PACS for that target (this will be usually two, sometimes three, separate observations). Moreover, the red and blue camera, and all slices, are included this single table. This table is also provided as a standalone browse product.
    • The list of obisds that are combined into a Level 3 can be found on the data products overview page (http://www.cosmos.esa.int/web/herschel/data-products-overview).
Changed:
<
<
    • Added in 14.2.1: due to an oversight in 14.2.0, the Level 3 was only added to the first observation (the first obsid) in each set of obsids that are combined into a unique Level 3. In 14.2.1 this has been corrected, and now every obsid that is included in creating a Level 3 will have a Level 3 with its appropriate spectrum table in it.
>
>
    • Added in 14.2.1: due to an oversight in 14.2.0, the Level 3 was only added to the first observation (the first obsid) in each set of (2 or 3) obsids in the list linked above. In SPG 14.2.1 we reprocessed the Level 3 only to correct this oversight. Now, every obsid in the list will have its appropriate Level 3.
 
  • Interpolated cubes in 14.0 and 14.2
    • These cubes now have a spaxel size of 3" rather than 4.7".
    • They are no longer provided for oversampled mapping observations.

Revision 712016-09-29 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in version 14.x

Line: 314 to 312
 

Spectroscopy

Changed:
<
<
  • Extended source correction. The extended-source calibrated rebinned cubes (level2) are subjected to a correction that accounts for the non-uniform illumination of the PACS IFU field-of-view. It stems from the fact that the 9.4"x9.4" spaxel edges are less responsive than their centres, translating into a spaxel active area of approximately 8"x8" where the response is flat. This effect is more severe at shorter wavelengths, and affects (very) extended emission, as well as off-centred point sources and any source that has a size between a point and a very extended. All Level 2/2.5 cubes for these types of sources will be affected, whether pointed or mapping.
    Hence, for centred point source extracted with the task extractCentralSpectrum, no change to the spectra are expected in this version of HIPE because of this effect. For very extended sources, we have developed a new Extended Source Correction calibration file (ExtendedSourceLoss, applied in the pipeline tasks specRespCal and specRespCalToTelescope), and this incorporates the necessary correction as part of the spatial flatfielding for the PACS spectrometer. For these sources, the fluxes of the spectra will differ to what was obtained in previous versions of HIPE. For intermediate sources, a more complex analysis is necessary to estimate the flux loss, and a task to do this will be provided in HIPE 15.
    The fluxes of the spaxels of all cubes will be increased by ~30% in the blue to ~5% in the red. The calibration file can be inspected to extract the curve:
    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data
    p=PlotXY(w, 1./f)
    After such a correction, a source of flat surface brightness within the field of view of a single pointing will be well calibrated at the level of the rebinned cubes (i.e. level2).The flux loss between the detectors was already accounted for in the point-source calibration correction in the correspondent calibration file, therefore, point-source calibrated spectra of centred point sources are not affected. For this reason, in HIPE 14, the extractCentralSpectrum task checks and removes the extendedSourceLoss correction from the level2 cubes, in case it has been applied, before extracting the final point source corrected spectrum. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: to get this, raise a helpdesk ticket.
>
>
  • Extended source correction. The extended-source calibrated rebinned cubes (level2) are subjected to a correction that accounts for the non-uniform illumination of the PACS IFU field-of-view. It stems from the fact that the 9.4"x9.4" spaxel edges are less responsive than their centres, translating into a spaxel active area of approximately 8"x8" where the response is flat. This effect is more severe at shorter wavelengths, and affects (very) extended emission, as well as off-centred point sources and any source that has a size between a point and a very extended. All Level 2/2.5 cubes for these types of sources will be affected, whether pointed or mapping.
    Hence, for centred point source extracted with the task extractCentralSpectrum, no change to the spectra are expected in this version of HIPE because of this effect. For very extended sources, we have developed a new Extended Source Correction calibration file (ExtendedSourceLoss, applied in the pipeline tasks specRespCal and specRespCalToTelescope), and this incorporates the necessary correction as part of the spatial flatfielding for the PACS spectrometer. For these sources, the fluxes of the spectra will differ to what was obtained in previous versions of HIPE. For intermediate sources, a more complex analysis is necessary to estimate the flux loss, and a task to do this will be provided in HIPE 15.
    The fluxes of the spaxels of all cubes will be increased by ~30% in the blue to ~5% in the red. The calibration file can be inspected to extract the curve:
    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data
    p=PlotXY(w, 1./f)
    After such a correction, a source of flat surface brightness within the field of view of a single pointing will be well calibrated at the level of the rebinned cubes (i.e. level2). The flux loss between the detectors was already accounted for in the point-source calibration correction in the correspondent calibration file, therefore, point-source calibrated spectra of centred point sources are not affected. For this reason, in HIPE 14, the extractCentralSpectrum task checks and removes the extendedSourceLoss correction from the level2 cubes, in case it has been applied, before extracting the final point source corrected spectrum. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: this task can be found in the "Point source loss correction (any spaxel)" script in the Scripts->PACS useful menu.
 
  • New beam calibration files have been delivered, differing from the previous files only in the way they are normalised. These will be used in a new task being developed in HIPE 15, and do not impact on HIPE 14 or SPG 14 in any way.
    • all the 25 beams were divided by the peak value of the central spaxel beam;
    • then, they are multiplied to match the Point Source Correction calibration file (pointSourceLoss FM_4).
Line: 347 to 345
 
    • For unchopped range scan observations, the on-source and off-source are separate obsids and each is reduced to Level2. Then the off-source data are subtracted from the on-source data and the "background subtracted" products added to Level 2.5. This has been the case for a long time. The difference now is that these products have a slightly different name: "BS" is appended to the product names as viewed in HIPE and to the FITS file names on disc.
  • Level 2/2.5 extracted spectra tables in 14.0, 14.2
    • A new product is provided at Level 2 for all (line, range, chopNod, unchopped) pointed observations, and also at Level 2.5 for the subset of unchopped, rangeScan observations. This table, HPSSPEC[BS][R|B], contains the extracted spectrum of the central spaxel, and the point source calibrated spectra "c1" and "c129" from the pipeline task extractCentralSpectrum. Since the point-source calibrated spectra are taken from the centre of the cube, it is necessary that the source is located within spaxel 2,2 (the central one). However, we make no determination that the target of the observation actually is a point source - that is up to the end-user to establish. These tables are also provided as SBPs.
Changed:
<
<
    • Added in 14.2: the output spectrum "c9" from extractCentralSpectrum is also provided.
>
>
    • Added in 14.2.0: the output spectrum "c9" from extractCentralSpectrum is also provided.
 
  • Level 3 extracted spectra table in 14.0, 14.2
    • The new spectrum tables from Level 2 for the pointed, chopNod, full SED observations, are concatenated into a single table which is placed in Level 3. This table, HPSSPEC, contains the extracted spectra for all the observations that were taken to cover the full spectral range (the full SED) of PACS for that target (this will be usually two, sometimes three, separate observations). Moreover, the red and blue camera, and all slices, are included this single table. This table is also provided as a standalone browse product.
Added:
>
>
    • The list of obisds that are combined into a Level 3 can be found on the data products overview page (http://www.cosmos.esa.int/web/herschel/data-products-overview).
    • Added in 14.2.1: due to an oversight in 14.2.0, the Level 3 was only added to the first observation (the first obsid) in each set of obsids that are combined into a unique Level 3. In 14.2.1 this has been corrected, and now every obsid that is included in creating a Level 3 will have a Level 3 with its appropriate spectrum table in it.
 
  • Interpolated cubes in 14.0 and 14.2
    • These cubes now have a spaxel size of 3" rather than 4.7".
    • They are no longer provided for oversampled mapping observations.

Revision 702016-09-27 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in version 14.x

Line: 51 to 51
 
  • Correction for artefacts in SPIRE Spectrometer Low Resolution (LR) point source calibrated spectra.
  • The default SPIRE Spectrometer spectral cubes are now produced with the convolution projection (CP) method: the browse images/thumbnails and standalone browse products are now the CP cubes.
  • In HIPE 14.1, the SPIRE Spectrometer bright mode calibration was fixed.
Added:
>
>
  • In HIPE 14.2.1, the mosaicked SPIRE Photometer Level-3 maps are correctly calculated.
  NOTE: Highlights with a version number in bold represent a change from version 14.0.
Line: 387 to 388
 
  • Access to the Planck maps in both the User pipelines and Useful Scripts is now handled automatically (see the zero point correction script for details on initial set up)
  • The serendipity data (when the telescope slews to a target) is now included all the way to Level 1 in the Observation. Note that the serendipity data is the final building block and is NOT destriped.
  • The SPIRE Level 2.5 data products have been extended to include, not only parallel pairs, but also sets of 2,3,4+ observations that would benefit from merging at the Level 2.5 stage. See the SPIRE Data Reduction Guide for a list of these observations.
Added:
>
>
  • In HIPE 14.2.1: the image and error extensions of the SPIRE level-3 maps, produced with the mosaic() task, are properly calculated.
 

Spectrometer Pipeline

  • A new task has been added to the point source pipeline that applies a correction to remove significant artefacts from SLW spectra of LR observations (and the LR part of H+LR observations)
  • A new task has been added to all pipelines that applies the extended calibration correction in order to incorporate the far field feedhorn efficiency for extended source. As a consequence we get better agreement between FTS and Phototmeter for fully extended sources.

Revision 692016-08-03 - MiguelSanchez

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in version 14.x

Revision 682016-08-03 - MiguelSanchez

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in version 14.x

Line: 115 to 115
 
    • Reset of the Spacecraft Velocity Vector (SVV) that is used by the STR to compute the aberration correction of the coordinates of guide stars. For more information please refer to the the dedicated section in the Spacecraft and Observatory pages.
    • It has been found that the STR misalignment was not properly handled in early ODs (<= 113) and has been fixed. (see SPR HCSS-20990). However, some issues still exist for a small set of SPIRE parallel observations in OD 111 (investigation on-going).
  • Now up to 18 stars are now included in the computation of the gyro-based attitude reconstruction, whenever available in TM (in the previous release, only up to 9 stars were included).
Changed:
<
<
* HCSS 142.: the gyro-based attitude reconstruction algorithm has been modified in order to in order to allow switching the logic used when estimating the star vector measurement error during the star tracker attitude calculations
>
>
  • New in HCSS 14.2: the gyro-based attitude reconstruction algorithm has been modified in order to in order to allow switching the logic used when estimating the star vector measurement error during the star tracker attitude calculations. As a general rule, the estimate of error used in QUEST measurement model should be as realistic as possible (for each observation -- see SPR HCSS-19267). However, in some cases this leads to wrong attitude estimates so it has been enabled a feature to switch between the variable error estimate and the original assumption of a constant error value for all star vector measurements.
 

Numeric routines

Data fitting

Revision 672016-08-02 - MiguelSanchez

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in version 14.x

Line: 117 to 115
 
    • Reset of the Spacecraft Velocity Vector (SVV) that is used by the STR to compute the aberration correction of the coordinates of guide stars. For more information please refer to the the dedicated section in the Spacecraft and Observatory pages.
    • It has been found that the STR misalignment was not properly handled in early ODs (<= 113) and has been fixed. (see SPR HCSS-20990). However, some issues still exist for a small set of SPIRE parallel observations in OD 111 (investigation on-going).
  • Now up to 18 stars are now included in the computation of the gyro-based attitude reconstruction, whenever available in TM (in the previous release, only up to 9 stars were included).
Added:
>
>
* HCSS 142.: the gyro-based attitude reconstruction algorithm has been modified in order to in order to allow switching the logic used when estimating the star vector measurement error during the star tracker attitude calculations
 

Numeric routines

Data fitting

Revision 662016-07-18 - AlvarGarcia

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->
Changed:
<
<

What's new in HIPE 14

>
>

What's new in version 14.x

 
PDF Version Click here
Line: 17 to 17
 
  • New Quick Analysis perspective allows GUI download of products and processing all within HIPE.
  • New Auxiliary Observation Summary Product with information about the observation in a simple dataset.
  • Quality flags have been reorganised.
Changed:
<
<
  • SVV resets and STR switch-overs handled by auxiliary processors in the generation of the pointing product. Fixes for early ODs (<= 113) added in HIPE 14.1.
>
>
  • SVV resets and STR switch-overs are now handled by auxiliary processors in the generation of the pointing product. Fixes for early ODs (<= 113) added in HIPE 14.1.
  HIFI
Changed:
<
<
  • New in 14.1 Update of sideband ratio in bands 1 to 4 - note that this will imply changes in the intensities in all concerned bands
  • New in 14.1 Introduction of a new calibration output providing a frequency-dependent intensity calibration uncertainty budget
  • Provision of spur channel flags for all modes (was limited to spectral scan up to HIPE 13)
>
>
  • New in 14.1 Update of sideband ratio in bands 1 to 4 - note that this will imply changes in the intensities in all concerned bands.
  • New in 14.1 Introduction of a new calibration output providing a frequency-dependent intensity calibration uncertainty budget.
  • Provision of spur channel flags for all modes (was limited to spectral scan up to HIPE 13).
 
  • Changes to the output of the Deconvolution task.
Changed:
<
<
  • New parameters for the removal tasks and updated functionality for *identifyLines.
>
>
  • New parameters for the removal tasks and updated functionality for identifyLines.
 
  • Legacy quality documentation.
Changed:
<
<
  • From 14.1 onwards:
    • FitBaseline has been upgraded to properly deal with channels holding negative fluxes in the masking, and to not honour the informative SPUR_WARNING flags
    • mkRms has been upgraded to not honour the informative SPUR_WARNING flags
    • The issues with the 14.0.1 products whereby the mean and median statistics products were erroneously set to 0 has been fixed.
    • Some relevant meta-data that were missing in cubes generated with 14.0.1 have now been added to the product headers
>
>
  • From 14.1 onwards:
    • FitBaseline has been upgraded to properly deal with channels holding negative fluxes in the masking, and to not honour the informative SPUR_WARNING flags.
    • mkRms has been upgraded to not honour the informative SPUR_WARNING flags.
    • The issues with the 14.0.1 products whereby the mean and median statistics products were erroneously set to 0 has been fixed.
    • Some relevant meta-data that were missing in cubes generated with 14.0.1 have now been added to the product headers.
  PACS
  • Spectroscopy calibration
Changed:
<
<
    • 14.0: a new Extended Source Calibration (ESCF) is released: fully extended sources are now better calibrated, and for point and semi-extended sources treated with their dedicated tasks there is no change. Any source between these two extremes will still suffer from intra-spaxel flux losses, but less so that before.
>
>
    • 14.0: a new Extended Source Calibration (ESCF) is released: fully extended sources are now better calibrated, and for point and semi-extended sources treated with their dedicated tasks there is no change. Any source between these two extremes will still suffer from intra-spaxel flux losses, but less so than before.
 
    • 14.0: drizzled cubes have a dedicated calibration and pipeline, and produce correct fluxes (a vast improvement on previous releases).
Changed:
<
<
    • 14.2: a new spatial calibration (spaxel sizes and layout) has been released. There are slight differences from that used previously. The flux distribution in mosaic cubes (interpolated, projected, drizzled) will change as a result.
    • The calibration tree update released for 14.2 includes a new RSRFR1 calibration file that can be used with the interactive pipelines to produce data in the red leak (>190 micron) region.
>
>
    • 14.2: a new spatial calibration (spaxel sizes and layout) has been released. There are slight differences from the one used previously. The flux distribution in mosaic cubes (interpolated, projected, drizzled) will change as a result.
    • The calibration tree update released for 14.2 includes a new RSRFR1 calibration file that can be used with the interactive pipelines to produce data in the red leak (>190 micron) region.
 
  • Spectroscopy products
    • 14.0: spectrum tables have been added to the higher levels and the standalone browse products.
Changed:
<
<
    • 14.2: for unchopped range scans, the on-source and the background subtracted on-source products (i.e. on-off) both are provided in the standalone browse products.
    • 14.2: interpolated cubes have a error array.
    • 14.2: for all observations, data below 55 microns and above 190 microns are excluded.
>
>
    • 14.2: for unchopped range scans, the on-source and the background subtracted on-source products (i.e. on-off) both are provided in the standalone browse products.
    • 14.2: interpolated cubes have an error array.
    • 14.2: for all observations, data below 55 microns and above 190 microns are excluded.
 
  • Photometer
Changed:
<
<
    • Level 2.5 maps of solar system objects have been created for 14.0 and 14.2.
    • 14.2: Unimap mapper has been upgraded to the version 6.4.4.
>
>
    • Level 2.5 maps of solar system objects have been created for 14.0 and 14.2.
    • 14.2: Unimap mapper has been upgraded to the version 6.4.4.
 SPIRE
  • New in 14.1 Major impact: SPIRE Spectrometer extended source calibrated spectra are corrected for the far-field feedhorn efficiency. The correction leads to systematic ~40% higher intensities for SSW and ~40-100% for SLW. All extended source calibration products - spectra or spectral cubes - are affected by this significant effect. With this correction the extended source calibrated SPIRE Photometer and Spectrometer products are in good agreement.
  • Correction for artefacts in SPIRE Spectrometer Low Resolution (LR) point source calibrated spectra.
  • The default SPIRE Spectrometer spectral cubes are now produced with the convolution projection (CP) method: the browse images/thumbnails and standalone browse products are now the CP cubes.
Changed:
<
<
  • In HIPE v14.1, the SPIRE Spectrometer bright mode calibration was fixed.
>
>
  • In HIPE 14.1, the SPIRE Spectrometer bright mode calibration was fixed.

NOTE: Highlights with a version number in bold represent a change from version 14.0.

 

Core system

Revision 652016-07-14 - ElenaPuga

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in HIPE 14

Line: 34 to 34
  PACS
  • Spectroscopy calibration
Changed:
<
<
    • 14.0: a new Extended Source Calibration (ESC) is released: fully extended sources are now better calibrated, and for point and semi-extended sources treated with their dedicated tasks there is no change. Any source between these two extremes will still suffer from intra-spaxel flux losses, but less so that before.
>
>
    • 14.0: a new Extended Source Calibration (ESCF) is released: fully extended sources are now better calibrated, and for point and semi-extended sources treated with their dedicated tasks there is no change. Any source between these two extremes will still suffer from intra-spaxel flux losses, but less so that before.
 
    • 14.0: drizzled cubes have a dedicated calibration and pipeline, and produce correct fluxes (a vast improvement on previous releases).
    • 14.2: a new spatial calibration (spaxel sizes and layout) has been released. There are slight differences from that used previously. The flux distribution in mosaic cubes (interpolated, projected, drizzled) will change as a result.
Changed:
<
<
    • The calibration tree update released for 14.2 includes a new RSRFR1 calibration file that can be used with the interactive pipelines to produce data in the red leak (>190 micron) region.
>
>
    • The calibration tree update released for 14.2 includes a new RSRFR1 calibration file that can be used with the interactive pipelines to produce data in the red leak (>190 micron) region.
 
  • Spectroscopy products
    • 14.0: spectrum tables have been added to the higher levels and the standalone browse products.
    • 14.2: for unchopped range scans, the on-source and the background subtracted on-source products (i.e. on-off) both are provided in the standalone browse products.
Line: 315 to 313
 
    • all the 25 beams were divided by the peak value of the central spaxel beam;
    • then, they are multiplied to match the Point Source Correction calibration file (pointSourceLoss FM_4).
  • A slightly updated telescope background model for use in the Pointing Offset Correction pipeline script for point sources has been developed. This update is to allow better consistency between the "POC" script and the default pipeline script "Telescope normaliation", for chopNod AOTs. The changes are at the percent level and largest in band B3A.
Changed:
<
<
  • A new RSRF for band R1, which includes the correct response correction for the red leak region (>190 micron) is used in HIPE/SPG 14.2. The red region is not actually calibrated in the SPG products, since a different pipeline to that run by the SPG is necessary, but these spectral ranges can be reduced interactively using appropriate the HIPE pipeline scripts and the advice in the PACS Data Reduction Guide.
>
>
  • A new RSRF for band R1, which includes the correct response correction for the red leak region (>190 micron) is used in HIPE/SPG 14.2. The red region is not actually calibrated in the SPG products, since a different pipeline to that run by the SPG is necessary, but these spectral ranges can be reduced interactively using appropriate the HIPE pipeline scripts and the advice in the PACS Data Reduction Guide.
 
  • New spatial calibration in 14.2. The footprint of the IFU of PACS (the spaxel sizes and distributions) has been recalibrated.
    • The main change is that the spaxel sizes are set at 9.4" in two of the the tasks that create mosaic cubes (specInterpolate, specProject), rather than being calculated uniquely for each observation: the sizes now are slightly larger that before. Drizzle already assumed a fixed spaxel size of 9.4". The spaxel themselves have not changed, only the way they are treated in these two mosaic tasks.
    • The layout of the spaxels has changed slightly, and this affects the spectra in all mosaic cubes (specInterpolate, specProject, and drizzle).

Revision 642016-07-07 - LucaCalzoletti

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in HIPE 14

Line: 44 to 44
 
    • 14.2: interpolated cubes have a error array.
    • 14.2: for all observations, data below 55 microns and above 190 microns are excluded.
  • Photometer
Changed:
<
<
    • Updates to high level products: Photometry level 2.5
    • Photometer: Maps of solar system objects have been created for this version.
    • Spectrometer: Tables of spectra in level 2.5 and 3, as mentioned above.
    • Unimap mapper has been upgraded to the latest version.
    • There are new scripts making use of JScanam. There is also a script to generate drizzled cubes.
    • New in 14.1 New spectroscopy calibration for extended sources. New beam calibration and background model.
    • New useful and analysis scripts to account for these calibration changes (beam and background related).
>
>
    • Level 2.5 maps of solar system objects have been created for 14.0 and 14.2.
    • 14.2: Unimap mapper has been upgraded to the version 6.4.4.
 SPIRE
  • New in 14.1 Major impact: SPIRE Spectrometer extended source calibrated spectra are corrected for the far-field feedhorn efficiency. The correction leads to systematic ~40% higher intensities for SSW and ~40-100% for SLW. All extended source calibration products - spectra or spectral cubes - are affected by this significant effect. With this correction the extended source calibrated SPIRE Photometer and Spectrometer products are in good agreement.
  • Correction for artefacts in SPIRE Spectrometer Low Resolution (LR) point source calibrated spectra.
Line: 296 to 290
 
  • Interactive pipeline scripts:
    • The JScanam ipipe script scanmap_JScanam_multiplePairs is designed to process multiple pairs of obsIDs and it has been upgraded by introducing the scanamorphosAlignFrames Task. This task aligns the Frames signal levels, belonging to every observation, to a common offset level, by exploiting the overlapping sky regions. This new capability is particulary useful for recovering a common background level over observations that are partially or slightly overlapping.
    • The script scanmap_Unimap_memory_optimized has been added to the Pipelines menu. This allows one to run the Unimap release located on the userís computer while using the minimum amount of memory. It is particularly useful for processing large observations or/and if several obsIDs should be processed all together in a single run.
Added:
>
>
    • The Unimap interactive scripts (scanmap_Unimap_memory and scanmap_Unimap_memory_optimized) are updated in HIPE14.2 in order to be compatible with any Unimap release (up to 6.5.3).
 
    • The script photextended_pipeline_L3 is now available in the Pipelines menu. This can be used to generate mosaics of Unimap and Jscanam Level2.5 maps.
Changed:
<
<
  • Unimap : Unimap 6.2.0 is spawned by the Standard Product Generator (SPG) to create Level2.5 products. See the Photometer Data Reduction Guide and the offical Unimap webpages (http://infocom.uniroma1.it/unimap/) to know What's New about the 6.2.0 Unimap release.
>
>
  • Unimap : In SPG14.0.1, Unimap 6.2.0 is spawned to create Level 2.5 products, while in SPG14.2 Level 2.5 products are generated by using Unimap 6.4.4. The main differences between the two Unimap releases are the introduction of the pixel noise compensation and the generation of the proper error maps in Unimap 6.4.4 (See the Photometer Data Reduction Guide and the offical Unimap webpages (http://infocom.uniroma1.it/unimap/) for details).
 

Spectroscopy

  • Interactive pipeline scripts in 14.0, 14.2

Revision 632016-07-07 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in HIPE 14

Line: 30 to 30
 
    • FitBaseline has been upgraded to properly deal with channels holding negative fluxes in the masking, and to not honour the informative SPUR_WARNING flags
    • mkRms has been upgraded to not honour the informative SPUR_WARNING flags
    • The issues with the 14.0.1 products whereby the mean and median statistics products were erroneously set to 0 has been fixed.
Changed:
<
<
    • Some relevand meta-data that were missing in cubes generated with 14.0.1 have now been added to the product headers
>
>
    • Some relevant meta-data that were missing in cubes generated with 14.0.1 have now been added to the product headers
  PACS
Changed:
<
<
  • Updates to high level products: Photometry level 2.5 and the creation of the Spectroscopy levels 2.5 and 3 (not really products but tables of spectra).
>
>
  • Spectroscopy calibration
    • 14.0: a new Extended Source Calibration (ESC) is released: fully extended sources are now better calibrated, and for point and semi-extended sources treated with their dedicated tasks there is no change. Any source between these two extremes will still suffer from intra-spaxel flux losses, but less so that before.
    • 14.0: drizzled cubes have a dedicated calibration and pipeline, and produce correct fluxes (a vast improvement on previous releases).
    • 14.2: a new spatial calibration (spaxel sizes and layout) has been released. There are slight differences from that used previously. The flux distribution in mosaic cubes (interpolated, projected, drizzled) will change as a result.
    • The calibration tree update released for 14.2 includes a new RSRFR1 calibration file that can be used with the interactive pipelines to produce data in the red leak (>190 micron) region.
  • Spectroscopy products
    • 14.0: spectrum tables have been added to the higher levels and the standalone browse products.
    • 14.2: for unchopped range scans, the on-source and the background subtracted on-source products (i.e. on-off) both are provided in the standalone browse products.
    • 14.2: interpolated cubes have a error array.
    • 14.2: for all observations, data below 55 microns and above 190 microns are excluded.
  • Photometer
    • Updates to high level products: Photometry level 2.5
 
    • Photometer: Maps of solar system objects have been created for this version.
    • Spectrometer: Tables of spectra in level 2.5 and 3, as mentioned above.
  • Unimap mapper has been upgraded to the latest version.
  • There are new scripts making use of JScanam. There is also a script to generate drizzled cubes.
  • New in 14.1 New spectroscopy calibration for extended sources. New beam calibration and background model.
  • New useful and analysis scripts to account for these calibration changes (beam and background related).
Deleted:
<
<
  • A new task runPacsSpg has been introduced, with which users can run the full PACS SPG pipeline from within HIPE.
  SPIRE
  • New in 14.1 Major impact: SPIRE Spectrometer extended source calibrated spectra are corrected for the far-field feedhorn efficiency. The correction leads to systematic ~40% higher intensities for SSW and ~40-100% for SLW. All extended source calibration products - spectra or spectral cubes - are affected by this significant effect. With this correction the extended source calibrated SPIRE Photometer and Spectrometer products are in good agreement.
Line: 305 to 315
 

Spectroscopy

Changed:
<
<
  • Extended source correction. The extended-source calibrated rebinned cubes (level2) contain now a correction that accounts for the non-uniform illumination of the PACS IFU field-of-view. It stems from the fact that the 9.4"x9.4" spaxel edges are less responsive than their centres, translating into a spaxel active area of approximately 8"x8" where the response is flat. This effect is more severe at shorter wavelengths, and affects (very) extended emission, as well as off-centred point sources and any source that has a size between a point and a very extended. All Level 2/2.5 cubes for these types of sources will be affected, whether pointed or mapping.
    Hence, for centred point source extracted with the task extractCentralSpectrum, no change to the spectra are expected in this version of HIPE because of this effect. For very extended sources, we have developed a new Extended Source Correction calibration file (ExtendedSourceLoss, applied in the pipeline tasks specRespCal and specRespCalToTelescope), and this incorporates the necessary correction as part of the spatial flatfielding for the PACS spectrometer. For these sources, the fluxes of the spectra will differ to what was obtained in previous versions of HIPE. For intermediate sources, a more complex analysis is necessary to estimate the flux loss, and a task to do this will be provided in HIPE 15.
    The fluxes of the spaxels of all cubes will be increased by ~30% in the blue to ~5% in the red. The calibration file can be inspected to extract the curve:
    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data
    p=PlotXY(w, 1./f)
    After such a correction, a source of flat surface brightness within the field of view of a single pointing will be well calibrated at the level of the rebinned cubes (i.e. level2).The flux loss between the detectors was already accounted for in the point-source calibration correction in the correspondent calibration file, therefore, point-source calibrated spectra of centred point sources are not affected. For this reason, in HIPE 14, the extractCentralSpectrum task checks and removes the extendedSourceLoss correction from the level2 cubes, in case it has been applied, before extracting the final point source corrected spectrum. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: to get this, raise a helpdesk ticket.
>
>
  • Extended source correction. The extended-source calibrated rebinned cubes (level2) are subjected to a correction that accounts for the non-uniform illumination of the PACS IFU field-of-view. It stems from the fact that the 9.4"x9.4" spaxel edges are less responsive than their centres, translating into a spaxel active area of approximately 8"x8" where the response is flat. This effect is more severe at shorter wavelengths, and affects (very) extended emission, as well as off-centred point sources and any source that has a size between a point and a very extended. All Level 2/2.5 cubes for these types of sources will be affected, whether pointed or mapping.
    Hence, for centred point source extracted with the task extractCentralSpectrum, no change to the spectra are expected in this version of HIPE because of this effect. For very extended sources, we have developed a new Extended Source Correction calibration file (ExtendedSourceLoss, applied in the pipeline tasks specRespCal and specRespCalToTelescope), and this incorporates the necessary correction as part of the spatial flatfielding for the PACS spectrometer. For these sources, the fluxes of the spectra will differ to what was obtained in previous versions of HIPE. For intermediate sources, a more complex analysis is necessary to estimate the flux loss, and a task to do this will be provided in HIPE 15.
    The fluxes of the spaxels of all cubes will be increased by ~30% in the blue to ~5% in the red. The calibration file can be inspected to extract the curve:
    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data
    p=PlotXY(w, 1./f)
    After such a correction, a source of flat surface brightness within the field of view of a single pointing will be well calibrated at the level of the rebinned cubes (i.e. level2).The flux loss between the detectors was already accounted for in the point-source calibration correction in the correspondent calibration file, therefore, point-source calibrated spectra of centred point sources are not affected. For this reason, in HIPE 14, the extractCentralSpectrum task checks and removes the extendedSourceLoss correction from the level2 cubes, in case it has been applied, before extracting the final point source corrected spectrum. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: to get this, raise a helpdesk ticket.
 
  • New beam calibration files have been delivered, differing from the previous files only in the way they are normalised. These will be used in a new task being developed in HIPE 15, and do not impact on HIPE 14 or SPG 14 in any way.
    • all the 25 beams were divided by the peak value of the central spaxel beam;
    • then, they are multiplied to match the Point Source Correction calibration file (pointSourceLoss FM_4).

Revision 622016-07-07 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in HIPE 14

Line: 298 to 298
 
  • SPG scripts in 14.0. 14.2
    • The range scan observations are now flatfielded: using the cubic spline fitting method for ranges longer than 5 microns, and using the line scan flatfielding for the shorter ranges. The decision as to which flatfielding method to use is based on the wavelength range of the prime camera, and the parallel camera data use the same task as the prime camera uses (regardless of its own wavelength range).
    • We exclude the spectral regions affected by leakage as the calibration is incorrect for those ranges. In 14.0 this applied to range scans only, but in 14.2 no observation will have data below 55 microns or above 190 microns.
Deleted:
<
<
    • The interpolation algorithm in specInterpolate produces now NaN values in case one of the input spaxels has a NaN value, instead of producing skewed values due to a missing flux value. This results in more NaN values in the interpolated cubes, but avoids non-reliable values instead.
 

Calibration

Line: 344 to 343
 
  • Level 3 extracted spectra table in 14.0, 14.2
    • The new spectrum tables from Level 2 for the pointed, chopNod, full SED observations, are concatenated into a single table which is placed in Level 3. This table, HPSSPEC, contains the extracted spectra for all the observations that were taken to cover the full spectral range (the full SED) of PACS for that target (this will be usually two, sometimes three, separate observations). Moreover, the red and blue camera, and all slices, are included this single table. This table is also provided as a standalone browse product.
  • Interpolated cubes in 14.0 and 14.2
Changed:
<
<
    • Cubes produced by the task specInterpolate now have a spaxel size of 3" rather than 4.7".
    • Are no longer provided for oversampled mapping observations.
    • Have an error dataset. This is not a full error, which would be the combination of the interpolation of the original errors and the error in the interpolation process. We have provided the first only, that is the stddev array of the input cubes are interpolated to create an error array in the output cubes.
    • The way the task specInterpolate handles NaNs as it creates these cubes has changed. This will primarily affect cubes with saturated data. Saturated data are masked out and replaced with NaNs when rebinned cubes are built. Interpolated cubes are then created from the rebinned cubes. NaNs are now filtered out as the interpolation task does its work on the input cubes, and all affected and immediately surrounding spaxels of the output interpolated cube will also be NaN. Hence, at the affected wavelengths, the NaN area will be larger than before...but this is also more correct than before.
>
>
    • These cubes now have a spaxel size of 3" rather than 4.7".
    • They are no longer provided for oversampled mapping observations.
    • In 14.2 they have an error dataset. This is not a full error, which would be the combination of the interpolation of the original errors and the error in the interpolation process. We have provided the first only, that is the stddev array of the input cubes are interpolated to create an error array in the output cubes.
    • The way the task specInterpolate handles NaNs as it creates these cubes has changed. This will primarily affect cubes with saturated data. Saturated data are masked out and replaced with NaNs when rebinned cubes are created. The interpolated cubes are then created from these rebinned cubes. The interpolation algorithm filters out NaNs rather than interpolating over them, and as a consequence the affected and the immediately surrounding spaxels will also be set to NaN for that wavelength point. This is a more correct way to handle NaNs, especially those due to saturation.
 
  • Standalone browse products (SBPs) in 14.0 and 14.2
    • We now provide the new extracted spectra tables, HPSSPEC[BS][R|B] and HPSSPEC (see above)
    • The background subtracted unchopped range ("BS") products have been added: so for unchopped range scans where the observer requested an off-source, the SBPs for the on-source obsids will include Level2 cubes and tables and the background subtracted ones ("BS")

Revision 612016-07-06 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in HIPE 14

Line: 277 to 277
 

Pipeline

Changed:
<
<
* A new task runPacsSpg has been introduced, with which a user can run a full SPG pipeline within HIPE, starting from the level-0 data in an observation context from the HSA, or even from the raw telemetry files when they are available.
>
>
* A new task runPacsSpg was introduced in 14.0, with which a user can run a full SPG pipeline within HIPE, starting from the level-0 data in an observation context from the HSA, or even from the raw telemetry files when they are available.
 

Photometry

  • Level2.5 products:
Line: 288 to 288
 
    • The script scanmap_Unimap_memory_optimized has been added to the Pipelines menu. This allows one to run the Unimap release located on the userís computer while using the minimum amount of memory. It is particularly useful for processing large observations or/and if several obsIDs should be processed all together in a single run.
    • The script photextended_pipeline_L3 is now available in the Pipelines menu. This can be used to generate mosaics of Unimap and Jscanam Level2.5 maps.
  • Unimap : Unimap 6.2.0 is spawned by the Standard Product Generator (SPG) to create Level2.5 products. See the Photometer Data Reduction Guide and the offical Unimap webpages (http://infocom.uniroma1.it/unimap/) to know What's New about the 6.2.0 Unimap release.
Added:
>
>
 

Spectroscopy

Changed:
<
<
  • Level 2/2.5 extracted spectra tables:
    • A new product is provided at Level 2 for all (line, range, chopNod, unchopped) pointed observations, and also at Level 2.5 for the subset of unchopped, rangeScan observations. This table, HPSSPEC[R|B], contains the extracted spectrum of the central spaxel, and the point source calibrated spectra "c1" and "c129" from the pipeline task extractCentralSpectrum. Since the point-source calibrated spectra are taken from the centre of the cube, it is necessary that the source is located within spaxel 2,2 (the central one). However, we make no determination that the target of the observation actually is a point source - that is up to the end-user to establish. These tables are also provided as standalone browse products.
  • Level 3 extracted spectra table:
    • The new spectrum tables from Level 2 for the pointed, chopNod, full SED observations, are concatenated into a single table which is placed in Level 3. This table, HPSSPEC, contains the extracted spectra for all the observations that were taken to cover the full spectral range (the full SED) of PACS for that target (this will be usually two, sometimes three, separate observations). Moreoever, the red and blue camera, and all slices, are included this single table. This table is also provided as a standalone browse product.
  • Interactive pipeline scripts:
>
>
* Interactive pipeline scripts in 14.0, 14.2
 
    • The pipeline menu has been reworded, so that the names reflect better what the individual scripts do. The first script offered in each menu is also the preferred one, and that on which the SPG scripts are based.
Changed:
<
<
    • A new pipeline script has been written to create drizzled cubes, based on (and almost hardly different to) the 'Telescope normalisation' pipeline script for chopNod line scans. With this script it is now possible to create drizzle cubes using this preferred pipeline method (in Track 13 it was not possible).
>
>
    • A new pipeline script has been written to create drizzled cubes, based on (and only slightly different to) the 'Telescope normalisation' pipeline script for chopNod line scans. With this script it is now possible to create drizzle cubes using this preferred pipeline method (in Track 13 it was not possible).
 
    • There have been small, but potentially significant changes to some of the tasks in the 'Pointing offset correction' scripts (chopNod) and the '...with transients" scripts (unchopped). These are, respectively, to the tasks specDetermineChiSquare and specLongTermTransient. The improvements are "under the hood" so there is no change in the user scripts themselves.
    • The range scan flatfielding task now performs a cubic spline fitting to the continuum of the spectra. This produces the same results as previously with the polynomial fit for shorter ranges, but improved results for longer ranges (especially SEDs) and is more robust against curvature changes in the spectra.
Changed:
<
<
  • SPG scripts:
    • The range scan observations are now flatfielded: using the spline fitting method for ranges longer than 5 microns, and using the line scan flatfielding for the shorter ranges
    • The range scan observations are now flatfielded: using the line scan flatfielding (specFlatFieldLine) for shorter ranges than 5 microns, and using specFlatFieldRange with the cubic spline fitting method for ranges longer than 5 microns. In the latter case, we exclude the spectral regions affected by leakeage in the correspondent bands during the fit, as the calibration is incorrect for those ranges. Additionally, these spectral ranges are trimmed out via the 'NOTFFED' mask.
    • The interpolated cubes are no longer produced for oversample range scan mapping observations, and they also have a smaller spaxel size of 3" (rather than 4.7" before).
>
>
  • SPG scripts in 14.0. 14.2
    • The range scan observations are now flatfielded: using the cubic spline fitting method for ranges longer than 5 microns, and using the line scan flatfielding for the shorter ranges. The decision as to which flatfielding method to use is based on the wavelength range of the prime camera, and the parallel camera data use the same task as the prime camera uses (regardless of its own wavelength range).
    • We exclude the spectral regions affected by leakage as the calibration is incorrect for those ranges. In 14.0 this applied to range scans only, but in 14.2 no observation will have data below 55 microns or above 190 microns.
 
    • The interpolation algorithm in specInterpolate produces now NaN values in case one of the input spaxels has a NaN value, instead of producing skewed values due to a missing flux value. This results in more NaN values in the interpolated cubes, but avoids non-reliable values instead.

Calibration

Line: 310 to 306
 

Spectroscopy

Changed:
<
<
  • The extended-source calibrated rebinned cubes (level2) contain now a correction that accounts for the non-uniform illumination of the PACS IFU field-of-view. It stems from the fact that the 9.4"x9.4" spaxel edges are less responsive than their centres, translating into a spaxel active area of approximately 8"x8" where the response is flat. This effect is more severe at shorter wavelengths, and affects (very) extended emission, as well as off-centred point sources and any source that has a size between a point and a very extended. All Level 2/2.5 cubes for these types of sources will be affected, whether pointed or mapping.
    Hence, for centred point source extracted with the task extractCentralSpectrum, no change to the spectra are expected in this version of HIPE because of this effect. For very extended sources, we have developed a new Extended Source Correction calibration file (ExtendedSourceLoss, applied in the pipeline tasks specRespCal and specRespCalToTelescope), and this incorporates the necessary correction as part of the spatial flatfielding for the PACS spectrometer. For these sources, the fluxes of the spectra will differ to what was obtained in previous versions of HIPE. For intermediate sources, a more complex analysis is necessary to estimate the flux loss, and a task to do this will be provided in HIPE 15.
    The fluxes of the spaxels of all cubes will be increased by ~30% in the blue to ~5% in the red. The calibration file can be inspected to extract the curve:
    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data
    p=PlotXY(w, 1./f)
    After such a correction, a source of flat surface brightness within the field of view of a single pointing will be well calibrated at the level of the rebinned cubes (i.e. level2).The flux loss between the detectors was already accounted for in the point-source calibration correction in the correspondent calibration file, therefore, point-source calibrated spectra of centred point sources are not affected. For this reason, in HIPE 14, the extractCentralSpectrum task checks and removes the extendedSourceLoss correction from the level2 cubes, in case it has been applied, before extracting the final point source corrected spectrum. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: to get this, raise a helpdesk ticket.

  • New beam calibration files have been delivered, differing from the previous files only in the way they are normalised. These will be used in a new task being developed in HIPE 15, and do not impact on HIPE 14 or SPG 14 in any way.
>
>
  • Extended source correction. The extended-source calibrated rebinned cubes (level2) contain now a correction that accounts for the non-uniform illumination of the PACS IFU field-of-view. It stems from the fact that the 9.4"x9.4" spaxel edges are less responsive than their centres, translating into a spaxel active area of approximately 8"x8" where the response is flat. This effect is more severe at shorter wavelengths, and affects (very) extended emission, as well as off-centred point sources and any source that has a size between a point and a very extended. All Level 2/2.5 cubes for these types of sources will be affected, whether pointed or mapping.
    Hence, for centred point source extracted with the task extractCentralSpectrum, no change to the spectra are expected in this version of HIPE because of this effect. For very extended sources, we have developed a new Extended Source Correction calibration file (ExtendedSourceLoss, applied in the pipeline tasks specRespCal and specRespCalToTelescope), and this incorporates the necessary correction as part of the spatial flatfielding for the PACS spectrometer. For these sources, the fluxes of the spectra will differ to what was obtained in previous versions of HIPE. For intermediate sources, a more complex analysis is necessary to estimate the flux loss, and a task to do this will be provided in HIPE 15.
    The fluxes of the spaxels of all cubes will be increased by ~30% in the blue to ~5% in the red. The calibration file can be inspected to extract the curve:
    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data
    p=PlotXY(w, 1./f)
    After such a correction, a source of flat surface brightness within the field of view of a single pointing will be well calibrated at the level of the rebinned cubes (i.e. level2).The flux loss between the detectors was already accounted for in the point-source calibration correction in the correspondent calibration file, therefore, point-source calibrated spectra of centred point sources are not affected. For this reason, in HIPE 14, the extractCentralSpectrum task checks and removes the extendedSourceLoss correction from the level2 cubes, in case it has been applied, before extracting the final point source corrected spectrum. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: to get this, raise a helpdesk ticket.
  • New beam calibration files have been delivered, differing from the previous files only in the way they are normalised. These will be used in a new task being developed in HIPE 15, and do not impact on HIPE 14 or SPG 14 in any way.
 
    • all the 25 beams were divided by the peak value of the central spaxel beam;
    • then, they are multiplied to match the Point Source Correction calibration file (pointSourceLoss FM_4).
Changed:
<
<
  • A slightly updated telescope background model for use in the Pointing Offset Correction pipeline script for point sources has been developed. This update is to allow better consistency between the "POC" script and the default pipeline script "Telescope normaliation", for chopNod AOTs. The changes are at the percent level and largest in band B3A.
>
>
  • A slightly updated telescope background model for use in the Pointing Offset Correction pipeline script for point sources has been developed. This update is to allow better consistency between the "POC" script and the default pipeline script "Telescope normaliation", for chopNod AOTs. The changes are at the percent level and largest in band B3A.
  • A new RSRF for band R1, which includes the correct response correction for the red leak region (>190 micron) is used in HIPE/SPG 14.2. The red region is not actually calibrated in the SPG products, since a different pipeline to that run by the SPG is necessary, but these spectral ranges can be reduced interactively using appropriate the HIPE pipeline scripts and the advice in the PACS Data Reduction Guide.
  • New spatial calibration in 14.2. The footprint of the IFU of PACS (the spaxel sizes and distributions) has been recalibrated.
    • The main change is that the spaxel sizes are set at 9.4" in two of the the tasks that create mosaic cubes (specInterpolate, specProject), rather than being calculated uniquely for each observation: the sizes now are slightly larger that before. Drizzle already assumed a fixed spaxel size of 9.4". The spaxel themselves have not changed, only the way they are treated in these two mosaic tasks.
    • The layout of the spaxels has changed slightly, and this affects the spectra in all mosaic cubes (specInterpolate, specProject, and drizzle).
    • There is no difference in the spectral fluxes due to this new spatial calibration in the rebinned cubes. For the mosaic cubes you will notice a difference to the flux distribution. The degree of this effect will be reported on when the SPG processing for 14.2 has been completed. It depends on the mapping pattern, the source brightness, and the source morphology.
 

Analysis tools

Line: 324 to 325
 

Spectroscopy

Added:
>
>
  • PacsCubes and PacsRebinnedCubes created by the pipeline will now display in HIPE cube viewers with a flip over the diagonal, as compared to previously (X: to the right, Y: upwards). This change allows the visualisation of PACS cubes into the PACS footprint orientation (note the axes display with X: downwards, Y: to the left). The displayed indices correspond to the PACS spaxel indices (see more in Sect. 6.3 of the DAG). The standard HIPE cube visualisation can be recovered in two steps, using the flip menu in the cube display control bar: no flip + vertical flip. When exporting the cube data outside HIPE, note that the data will still be organised according to standard cube indices.The new orientation makes it easier to compare the cubes in cube viewers (the Standard Cube Viewer or the Spectrum Explorer) to the display of the PacsSpectralFootprint viewer.
 

Product changes

Deleted:
<
<
  • A bug fix in the waveCalc step for 14.0 causes that some slices redward of the outofband limits, that were previously non-correctly showing, are now properly masked out.
 
  • Many improvements to the descriptions and/or units of the header meta keywords.
  • All keywords have now a corresponding 8 character FITS keyword, so that the headers are easier to read with other software.
Line: 335 to 336
 
  • The error layer within Unimap Level 2.5 products is renamed as stdDev, being the standard deviation computed at every sky pixel with respect to the samples that are falling on it. It can be properly assumed as the error map only if associated to the NaÔve map.

Spectroscopy

Changed:
<
<
  • Interpolated cubes: Cubes produced by the task specInterpolate now have a spaxel size of 3".
  • Interpolated cubes: Are no longer provided for oversampled mapping observations.
  • Interpolated cubes: The way the task specInterpolate handles NaNs as it creates these cubes has changed. This will primarily affect cubes with saturated data. Saturated data are masked out and replaced with NaNs when rebinned cubes are built. Interpolated cubes are then created from the rebinned cubes. NaNs are now filtered out as the interpolation task does its work on the input cubes, and all affected and immediately surrounding spaxels of the output interpolated cube will also be NaN. Hence, at the affected wavelengths, the NaN area will be larger than before...but this is also more correct than before.
  • Standalone browse products: we now provide the new extracted spectra tables, HPSSPEC[R|B] and HPSSPEC (see above), as SBPs.
  • Drizzle cubes: in SPG 13 the drizzle cube fluxes were incorrect. In SPG 14 they are now correct.
  • PacsCubes and PacsRebinnedCubes created by the pipeline (HPS3D [R|B] and HPS3DR [R|B]) will now display in HIPE cube viewers with a flip over the diagonal, as compared to previously (X: to the right, Y: upwards). This change allows the visualization of PACS cubes into the PACS footprint orientation (note the axes display with X: downwards, Y: to the left). The displayed indices correspond to the PACS spaxel indices (see more in Sect. 6.3 of the DAG). The standard HIPE cube visualization can be recovered in two steps, using the flip menu in the cube display control bar: no flip + vertical flip. When exporting the cube data outside HIPE, note that the data will still be organized according to standard cube indices.The new orientation makes it easier to compare the cubes in cube viewers (the Standard Cube Viewer or the Spectrum Explorer) to the display of the PacsSpectralFootprint viewer.
>
>
  • Unchopped range scan observations in 14.2
    • For unchopped range scan observations, the on-source and off-source are separate obsids and each is reduced to Level2. Then the off-source data are subtracted from the on-source data and the "background subtracted" products added to Level 2.5. This has been the case for a long time. The difference now is that these products have a slightly different name: "BS" is appended to the product names as viewed in HIPE and to the FITS file names on disc.
  • Level 2/2.5 extracted spectra tables in 14.0, 14.2
    • A new product is provided at Level 2 for all (line, range, chopNod, unchopped) pointed observations, and also at Level 2.5 for the subset of unchopped, rangeScan observations. This table, HPSSPEC[BS][R|B], contains the extracted spectrum of the central spaxel, and the point source calibrated spectra "c1" and "c129" from the pipeline task extractCentralSpectrum. Since the point-source calibrated spectra are taken from the centre of the cube, it is necessary that the source is located within spaxel 2,2 (the central one). However, we make no determination that the target of the observation actually is a point source - that is up to the end-user to establish. These tables are also provided as SBPs.
    • Added in 14.2: the output spectrum "c9" from extractCentralSpectrum is also provided.
  • Level 3 extracted spectra table in 14.0, 14.2
    • The new spectrum tables from Level 2 for the pointed, chopNod, full SED observations, are concatenated into a single table which is placed in Level 3. This table, HPSSPEC, contains the extracted spectra for all the observations that were taken to cover the full spectral range (the full SED) of PACS for that target (this will be usually two, sometimes three, separate observations). Moreover, the red and blue camera, and all slices, are included this single table. This table is also provided as a standalone browse product.
  • Interpolated cubes in 14.0 and 14.2
    • Cubes produced by the task specInterpolate now have a spaxel size of 3" rather than 4.7".
    • Are no longer provided for oversampled mapping observations.
    • Have an error dataset. This is not a full error, which would be the combination of the interpolation of the original errors and the error in the interpolation process. We have provided the first only, that is the stddev array of the input cubes are interpolated to create an error array in the output cubes.
    • The way the task specInterpolate handles NaNs as it creates these cubes has changed. This will primarily affect cubes with saturated data. Saturated data are masked out and replaced with NaNs when rebinned cubes are built. Interpolated cubes are then created from the rebinned cubes. NaNs are now filtered out as the interpolation task does its work on the input cubes, and all affected and immediately surrounding spaxels of the output interpolated cube will also be NaN. Hence, at the affected wavelengths, the NaN area will be larger than before...but this is also more correct than before.
  • Standalone browse products (SBPs) in 14.0 and 14.2
    • We now provide the new extracted spectra tables, HPSSPEC[BS][R|B] and HPSSPEC (see above)
    • The background subtracted unchopped range ("BS") products have been added: so for unchopped range scans where the observer requested an off-source, the SBPs for the on-source obsids will include Level2 cubes and tables and the background subtracted ones ("BS")
  • Wavelength cutoff: no observation processed by SPG 14.2 has data blue of 55 microns or red of 190 microns.
  • Drizzle cubes: in SPG 13 the drizzle cube fluxes were incorrect. In SPG 14 they are correct.
 

Documentation

Added:
>
>
  • The PACS Products Explained includes information about the organisation and contents of the FITS files from a tarball download from the HSA. It also has a short section summarising which products to use for general science cases.

 

SPIRE

Revision 602016-04-15 - AlvarGarcia

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in HIPE 14

Line: 17 to 17
 
  • New Quick Analysis perspective allows GUI download of products and processing all within HIPE.
  • New Auxiliary Observation Summary Product with information about the observation in a simple dataset.
  • Quality flags have been reorganised.
Changed:
<
<
  • SVV resets and STR switch-overs handled by auxiliary processors in the generation of the pointing product. Fixes for early ODs (<= 113) added in HIPE 14.1.
>
>
  • SVV resets and STR switch-overs handled by auxiliary processors in the generation of the pointing product. Fixes for early ODs (<= 113) added in HIPE 14.1.
  HIFI
Changed:
<
<
  • Update of sideband ratio in bands 1 to 4 - note that this will imply changes in the intensities in all concerned bands
  • Introduction of a new calibration output providing a frequency-dependent intensity calibration uncertainty budget
>
>
  • New in 14.1 Update of sideband ratio in bands 1 to 4 - note that this will imply changes in the intensities in all concerned bands
  • New in 14.1 Introduction of a new calibration output providing a frequency-dependent intensity calibration uncertainty budget
 
  • Provision of spur channel flags for all modes (was limited to spectral scan up to HIPE 13)
  • Changes to the output of the Deconvolution task.
  • New parameters for the removal tasks and updated functionality for *identifyLines.
Line: 38 to 38
 
    • Spectrometer: Tables of spectra in level 2.5 and 3, as mentioned above.
  • Unimap mapper has been upgraded to the latest version.
  • There are new scripts making use of JScanam. There is also a script to generate drizzled cubes.
Changed:
<
<
  • New spectroscopy calibration for extended sources. New beam calibration and background model.
>
>
  • New in 14.1 New spectroscopy calibration for extended sources. New beam calibration and background model.
 
  • New useful and analysis scripts to account for these calibration changes (beam and background related).
  • A new task runPacsSpg has been introduced, with which users can run the full PACS SPG pipeline from within HIPE.

SPIRE

Changed:
<
<
  • Major impact: SPIRE Spectrometer extended source calibrated spectra are corrected for the far-field feedhorn efficiency. The correction leads to systematic ~40% higher intensities for SSW and ~40-100% for SLW. All extended source calibration products - spectra or spectral cubes - are affected by this significant effect. With this correction the extended source calibrated SPIRE Photometer and Spectrometer products are in good agreement.
>
>
  • New in 14.1 Major impact: SPIRE Spectrometer extended source calibrated spectra are corrected for the far-field feedhorn efficiency. The correction leads to systematic ~40% higher intensities for SSW and ~40-100% for SLW. All extended source calibration products - spectra or spectral cubes - are affected by this significant effect. With this correction the extended source calibrated SPIRE Photometer and Spectrometer products are in good agreement.
 
  • Correction for artefacts in SPIRE Spectrometer Low Resolution (LR) point source calibrated spectra.
  • The default SPIRE Spectrometer spectral cubes are now produced with the convolution projection (CP) method: the browse images/thumbnails and standalone browse products are now the CP cubes.
  • In HIPE v14.1, the SPIRE Spectrometer bright mode calibration was fixed.
Line: 107 to 109
 
  • Special corrections in pointing products to handle S/C operational events have been included in the auxiliary processors. Namely, the corrections are:
    • STR switch-overs, i.e. changes from the usually prime star-tracker (STR1) to the backup unit (STR2)
    • Reset of the Spacecraft Velocity Vector (SVV) that is used by the STR to compute the aberration correction of the coordinates of guide stars. For more information please refer to the the dedicated section in the Spacecraft and Observatory pages.
Changed:
<
<
    • It has been found that the STR misalignment was not properly handled in early ODs (<= 113) and has been fixed. (see SPR HCSS-20990). However, some issues still exist for a small set of SPIRE parallel observations in OD 111 (investigation on-going).
>
>
    • It has been found that the STR misalignment was not properly handled in early ODs (<= 113) and has been fixed. (see SPR HCSS-20990). However, some issues still exist for a small set of SPIRE parallel observations in OD 111 (investigation on-going).
 
  • Now up to 18 stars are now included in the computation of the gyro-based attitude reconstruction, whenever available in TM (in the previous release, only up to 9 stars were included).

Numeric routines

Line: 378 to 380
 

Documentation


Changed:
<
<
<!-- COMMENT BOX CODE - DO NOT EDIT -->
>
>
<-- COMMENT BOX CODE - DO NOT EDIT -->
 
Changed:
<
<
<script type="text/javascript">
<br />var disqus_shortname = 'herscheltwiki'; // required: replace example with your forum shortname<br /><br />(function() {<br />var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;<br />dsq.src = 'http://' + disqus_shortname + '.disqus.com/embed.js';<br />(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);<br />})();<br /></script> <noscript>Please enable JavaScript to view the comments powered by Disqus.</noscript> blog comments powered by Disqus <!-- END OF COMMENT BOX CODE -->
>
>
blog comments powered by Disqus
<-- END< OF COMMENT BOX CODE -->
 
Changed:
<
<
<!--<br />* Set ALLOWTOPICCHANGE = RegisteredUsersGroup, TWikiAdminGroup <br />-->
>
>
<--
* Set ALLOWTOPICCHANGE = RegisteredUsersGroup, TWikiAdminGroup
-->
  -- AlvarGarcia - 24 Nov 2014

Editors & Viewers

Revision 592016-04-12 - MiguelSanchez

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in HIPE 14

Line: 17 to 17
 
  • New Quick Analysis perspective allows GUI download of products and processing all within HIPE.
  • New Auxiliary Observation Summary Product with information about the observation in a simple dataset.
  • Quality flags have been reorganised.
Changed:
<
<
  • SVV resets and STR switch-overs handled by auxiliary processors in the generation of the pointing product.
>
>
  • SVV resets and STR switch-overs handled by auxiliary processors in the generation of the pointing product. Fixes for early ODs (<= 113) added in HIPE 14.1.
  HIFI
  • Update of sideband ratio in bands 1 to 4 - note that this will imply changes in the intensities in all concerned bands
Line: 107 to 107
 
  • Special corrections in pointing products to handle S/C operational events have been included in the auxiliary processors. Namely, the corrections are:
    • STR switch-overs, i.e. changes from the usually prime star-tracker (STR1) to the backup unit (STR2)
    • Reset of the Spacecraft Velocity Vector (SVV) that is used by the STR to compute the aberration correction of the coordinates of guide stars. For more information please refer to the the dedicated section in the Spacecraft and Observatory pages.
Changed:
<
<
    • Caveat: large offsets (compatible with STR misalignment) have been reported in two observations, namely 1342183075 and 1342183076. See SPR HCSS-20990.
>
>
    • It has been found that the STR misalignment was not properly handled in early ODs (<= 113) and has been fixed. (see SPR HCSS-20990). However, some issues still exist for a small set of SPIRE parallel observations in OD 111 (investigation on-going).
 
  • Now up to 18 stars are now included in the computation of the gyro-based attitude reconstruction, whenever available in TM (in the previous release, only up to 9 stars were included).

Numeric routines

Revision 582016-04-08 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in HIPE 14

Line: 46 to 46
 
  • Major impact: SPIRE Spectrometer extended source calibrated spectra are corrected for the far-field feedhorn efficiency. The correction leads to systematic ~40% higher intensities for SSW and ~40-100% for SLW. All extended source calibration products - spectra or spectral cubes - are affected by this significant effect. With this correction the extended source calibrated SPIRE Photometer and Spectrometer products are in good agreement.
  • Correction for artefacts in SPIRE Spectrometer Low Resolution (LR) point source calibrated spectra.
  • The default SPIRE Spectrometer spectral cubes are now produced with the convolution projection (CP) method: the browse images/thumbnails and standalone browse products are now the CP cubes.
Added:
>
>
  • In HIPE v14.1, the SPIRE Spectrometer bright mode calibration was fixed.
 

Core system

Line: 348 to 349
 
    • A new calibration product SCalSpecLrCorr has been added, containing correction curves for point-source calibrated LR data.
    • A new calibration product SCalSpecExtCorr has been added, containing polynomial coefficients that are used to generate correction curves for extended source calibrated data.
    • A new calibration product SCalSpecOpdCorr has been added, it is currently neither used in the pipelines nor in user scripts.
Added:
>
>
    • Updated calibration product SCalSpecBrightGain with the correct gains for the bright source mode
 

Revision 572016-04-08 - AlvarGarcia

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in HIPE 14

Line: 23 to 23
 
  • Update of sideband ratio in bands 1 to 4 - note that this will imply changes in the intensities in all concerned bands
  • Introduction of a new calibration output providing a frequency-dependent intensity calibration uncertainty budget
  • Provision of spur channel flags for all modes (was limited to spectral scan up to HIPE 13)
Changed:
<
<
  • Changes to the output of the Deconvolution task.
>
>
  • Changes to the output of the Deconvolution task.
 
  • New parameters for the removal tasks and updated functionality for *identifyLines.
  • Legacy quality documentation.
  • From 14.1 onwards:
Changed:
<
<
    • FitBaseline has been upgraded to properly deal with channels holding negtive fluxes in the masking, and to not honour the informative SPUR_WARNING flags
    • mkRms has been upgraded to not honour the informative SPUR_WARNING flags
>
>
    • FitBaseline has been upgraded to properly deal with channels holding negative fluxes in the masking, and to not honour the informative SPUR_WARNING flags
    • mkRms has been upgraded to not honour the informative SPUR_WARNING flags
 
    • The issues with the 14.0.1 products whereby the mean and median statistics products were erroneously set to 0 has been fixed.
    • Some relevand meta-data that were missing in cubes generated with 14.0.1 have now been added to the product headers
Line: 40 to 40
 
  • There are new scripts making use of JScanam. There is also a script to generate drizzled cubes.
  • New spectroscopy calibration for extended sources. New beam calibration and background model.
  • New useful and analysis scripts to account for these calibration changes (beam and background related).
Changed:
<
<
  • A new task runPacsSpg has been introduced, with which users can run the full PACS SPG pipeline from within HIPE.
>
>
  • A new task runPacsSpg has been introduced, with which users can run the full PACS SPG pipeline from within HIPE.
  SPIRE
  • Major impact: SPIRE Spectrometer extended source calibrated spectra are corrected for the far-field feedhorn efficiency. The correction leads to systematic ~40% higher intensities for SSW and ~40-100% for SLW. All extended source calibration products - spectra or spectral cubes - are affected by this significant effect. With this correction the extended source calibrated SPIRE Photometer and Spectrometer products are in good agreement.

Revision 562016-04-01 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in HIPE 14

Line: 26 to 26
 
  • Changes to the output of the Deconvolution task.
  • New parameters for the removal tasks and updated functionality for *identifyLines.
  • Legacy quality documentation.
Added:
>
>
  • From 14.1 onwards:
    • FitBaseline has been upgraded to properly deal with channels holding negtive fluxes in the masking, and to not honour the informative SPUR_WARNING flags
    • mkRms has been upgraded to not honour the informative SPUR_WARNING flags
    • The issues with the 14.0.1 products whereby the mean and median statistics products were erroneously set to 0 has been fixed.
    • Some relevand meta-data that were missing in cubes generated with 14.0.1 have now been added to the product headers
  PACS
  • Updates to high level products: Photometry level 2.5 and the creation of the Spectroscopy levels 2.5 and 3 (not really products but tables of spectra).
Line: 215 to 220
 
  • Flags are now applied to OFF spectra as well
  • BrowseProduct: generation of a browse image for the reference spectra, when the option useReferenceSpectra is set to true
  • The FITS header keywords have been revamped in order to provide proper nicknames to the parameters featured in those headers
Added:
>
>
  • From 14.1 onwards:
    • Some relevand meta-data that were missing in cubes generated with 14.0.1 have now been added to the product headers
    • The issues with the 14.0.1 products whereby the mean and median statistics products were erroneously set to 0 has been fixed
 

HIFI Spectrum Toolbox

Spectrum Explorer (HIFI specifics)

Line: 225 to 233
 
  • New parameter addMedianContinuum: allow the median continuum to be added back into the baseline fit flux [DEFAULT: False]

Baseline Removal

  • New parameter addMedianContinuum: allow the median continuum to be added back into the baseline fit flux but only performs well for basemode='sub' and not basemode='div' [DEFAULT: False]
Added:
>
>
  • In 14.1, the above option will benefit from a fix whereby channels holding negtive fluxes will now be properly handled in the masking. This was not done correctly until 14.0.1.
  • FitBaseline has also been updated so that it deos not honour the informative SPUR_WARNING flags
 

Deconvolution

  • The output deconvolution product ssb is now called dataset (please note that scripts prior to this change will break)

Spectral Cube creation

Line: 240 to 250
 
    • New parameter snrType: the signal to noise ratio calculation algorithm (Intensity or Flux)
    • Implementation of the exportLines task

HiClass export tool

Changed:
<
<
  • Although this is not an HCSS development, it should be noted that GILDAS/Class now reads the HCSS-generated FITS without needing any prior conversion as it used to be the case using the HiClass task.
>
>
  • Although this is not an HCSS development, it should be noted that GILDAS/Class now reads the HCSS-generated FITS without needing any prior conversion as it used to be the case using the HiClass task.
 

Documentation

For HCSS 14, many figures were refreshed in both manuals (HIFI Data Reduction Guide and HIFI Pipeline Specification).

Line: 253 to 263
 
    • Updates to the chapter How to add and remove flags, section Setting and Clearing Flags with command line tools: improved the definition of the parameter selection for Level 2.5 cubes
    • A new chapter called Understanding the uncertainty table information in your data has been added
    • Updates to the chapter The HIFI line identification tool, adding a section called The exportLines Task
Added:
>
>
    • From 14.1 onwards: updated the table describing the content of the HIFI Calibratioin Tree
 
  • Updates to the HIFI Pipeline Specification manual
    • This manual has been restructured
    • Removal of the task mkSpur

Revision 552016-03-24 - RosalindHopwood

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in HIPE 14

Line: 356 to 358
 

Spectrometer Pipeline

  • A new task has been added to the point source pipeline that applies a correction to remove significant artefacts from SLW spectra of LR observations (and the LR part of H+LR observations)
  • A new task has been added to all pipelines that applies the extended calibration correction in order to incorporate the far field feedhorn efficiency for extended source. As a consequence we get better agreement between FTS and Phototmeter for fully extended sources.
Added:
>
>
  • The browse product image for intermediate and fully sampled mapping observations, and sparse raster observations, has been updated to provide more information.
 

Interactive Analysis and Tools

  • There is a new useful script, Spectrometer Pointing Offset Corrector, for estimating and correcting pointing offset in sparse point observations. The scripts assesses the difference between the SLW and SSW spectra of the centre detectors, over the overlapping frequency range, uses this to estimate any pointing offset and corrects the data using SECT.
Added:
>
>
  • The Spectrometer Background Subtraction useful script will now automatically omit outlying off-axis detectors from the average used to subtract the background. The smoothed off-axis spectra used for the average, any off-axis spectra that were omitted and the mean sum are plotted at the end of the script. If interactive selection is turned off, these plots make it possible to check the outliers selected by the script, and if necessary, repeat the process so that outliers can be selected by eye.
 

Documentation


Revision 542016-03-10 - AlvarGarcia

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
Changed:
<
<
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

>
>
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->
 

What's new in HIPE 14

PDF Version Click here

Revision 532016-01-08 - ElenaPuga

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's new in HIPE 14

Line: 33 to 33
 
    • Spectrometer: Tables of spectra in level 2.5 and 3, as mentioned above.
  • Unimap mapper has been upgraded to the latest version.
  • There are new scripts making use of JScanam. There is also a script to generate drizzled cubes.
Changed:
<
<
  • New spectroscopy calibration for extended sources which also works for off-centre point sources. New beam calibration and background model.
>
>
  • New spectroscopy calibration for extended sources. New beam calibration and background model.
 
  • New useful and analysis scripts to account for these calibration changes (beam and background related).
  • A new task runPacsSpg has been introduced, with which users can run the full PACS SPG pipeline from within HIPE.
Line: 298 to 298
 
  • The extended-source calibrated rebinned cubes (level2) contain now a correction that accounts for the non-uniform illumination of the PACS IFU field-of-view. It stems from the fact that the 9.4"x9.4" spaxel edges are less responsive than their centres, translating into a spaxel active area of approximately 8"x8" where the response is flat. This effect is more severe at shorter wavelengths, and affects (very) extended emission, as well as off-centred point sources and any source that has a size between a point and a very extended. All Level 2/2.5 cubes for these types of sources will be affected, whether pointed or mapping.
    Hence, for centred point source extracted with the task extractCentralSpectrum, no change to the spectra are expected in this version of HIPE because of this effect. For very extended sources, we have developed a new Extended Source Correction calibration file (ExtendedSourceLoss, applied in the pipeline tasks specRespCal and specRespCalToTelescope), and this incorporates the necessary correction as part of the spatial flatfielding for the PACS spectrometer. For these sources, the fluxes of the spectra will differ to what was obtained in previous versions of HIPE. For intermediate sources, a more complex analysis is necessary to estimate the flux loss, and a task to do this will be provided in HIPE 15.
    The fluxes of the spaxels of all cubes will be increased by ~30% in the blue to ~5% in the red. The calibration file can be inspected to extract the curve:
    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data
    p=PlotXY(w, 1./f)
    After such a correction, a source of flat surface brightness within the field of view of a single pointing will be well calibrated at the level of the rebinned cubes (i.e. level2).The flux loss between the detectors was already accounted for in the point-source calibration correction in the correspondent calibration file, therefore, point-source calibrated spectra of centred point sources are not affected. For this reason, in HIPE 14, the extractCentralSpectrum task checks and removes the extendedSourceLoss correction from the level2 cubes, in case it has been applied, before extracting the final point source corrected spectrum. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: to get this, raise a helpdesk ticket.
Changed:
<
<
  • New beam calibration files have been delivered, differing from the previous files only in the way they are normalised. These will be used in a new task being developed in HIPE 15, and do not impact on HIPE 14 or SPG 14 in any way.
    • In the spatial domain, they are normalised such that the sum of all spaxels is 1 when a point source of flux 1 is observed at the peak position of the central spaxel;
    • In the wavelength domain, the central spaxel is normalised to match the Point Source Correction calibration file.
>
>
  • New beam calibration files have been delivered, differing from the previous files only in the way they are normalised. These will be used in a new task being developed in HIPE 15, and do not impact on HIPE 14 or SPG 14 in any way.
    • all the 25 beams were divided by the peak value of the central spaxel beam;
    • then, they are multiplied to match the Point Source Correction calibration file (pointSourceLoss FM_4).
 
  • A slightly updated telescope background model for use in the Pointing Offset Correction pipeline script for point sources has been developed. This update is to allow better consistency between the "POC" script and the default pipeline script "Telescope normaliation", for chopNod AOTs. The changes are at the percent level and largest in band B3A.

Analysis tools

Revision 522015-12-22 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's new in HIPE 14

Line: 15 to 15
 Core
  • Now you can pass arguments to scripts in batch mode (former jylaunch).
  • New Quick Analysis perspective allows GUI download of products and processing all within HIPE.
Changed:
<
<
  • New product Auxiliary Observation Summary Product with information about the observation in a simple dataset.
>
>
  • New Auxiliary Observation Summary Product with information about the observation in a simple dataset.
 
  • Quality flags have been reorganised.
  • SVV resets and STR switch-overs handled by auxiliary processors in the generation of the pointing product.

Revision 512015-12-21 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's new in HIPE 14

Line: 20 to 20
 
  • SVV resets and STR switch-overs handled by auxiliary processors in the generation of the pointing product.

HIFI

Changed:
<
<
  • Update of sideband ratio in bands 1 to 4 - note that this will imply changes in the intensities in all concerned bands
  • Introduction of a new calibration output providing a frequency-dependent intensity calibration uncertainty budget
>
>
  • Update of sideband ratio in bands 1 to 4 - note that this will imply changes in the intensities in all concerned bands
  • Introduction of a new calibration output providing a frequency-dependent intensity calibration uncertainty budget
 
  • Provision of spur channel flags for all modes (was limited to spectral scan up to HIPE 13)
  • Changes to the output of the Deconvolution task.
  • New parameters for the removal tasks and updated functionality for *identifyLines.
Line: 38 to 38
 
  • A new task runPacsSpg has been introduced, with which users can run the full PACS SPG pipeline from within HIPE.

SPIRE

Changed:
<
<
  • Major impact: SPIRE Spectrometer extended source calibrated spectra are corrected for the far-field feedhorn efficiency. The correction leads to systematic ~40% higher intensities for SSW and ~40-100% for SLW. All extended source calibration products - spectra or spectral cubes - are affected by this significant effect. With this correction the extended source calibrated SPIRE Photometer and Spectrometer products are in good agreement.
>
>
  • Major impact: SPIRE Spectrometer extended source calibrated spectra are corrected for the far-field feedhorn efficiency. The correction leads to systematic ~40% higher intensities for SSW and ~40-100% for SLW. All extended source calibration products - spectra or spectral cubes - are affected by this significant effect. With this correction the extended source calibrated SPIRE Photometer and Spectrometer products are in good agreement.
 
  • Correction for artefacts in SPIRE Spectrometer Low Resolution (LR) point source calibrated spectra.
  • The default SPIRE Spectrometer spectral cubes are now produced with the convolution projection (CP) method: the browse images/thumbnails and standalone browse products are now the CP cubes.

Revision 502015-12-21 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
Changed:
<
<
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->
>
>
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

 

What's new in HIPE 14

PDF Version Click here
Changed:
<
<
Help Interested in what's new for other versions of HIPE? See this page for links to all the What's New documents from HIPE 4.0 onwards.
>
>
Help Interested in what's new for other versions of HIPE? See this page for links to all the What's New documents from HIPE 4.0 onwards.
 See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.
Changed:
<
<

>
>

 

Highlights

Line: 20 to 20
 
  • SVV resets and STR switch-overs handled by auxiliary processors in the generation of the pointing product.

HIFI

Changed:
<
<
  • Update of sideband ratio in bands 1 to 4 - note that this will imply changes in the intensities in all concerned bands
  • Introduction of a new calibration output providing a frequency-dependent intensity calibration uncertainty budget
>
>
  • Update of sideband ratio in bands 1 to 4 - note that this will imply changes in the intensities in all concerned bands
  • Introduction of a new calibration output providing a frequency-dependent intensity calibration uncertainty budget
 
  • Provision of spur channel flags for all modes (was limited to spectral scan up to HIPE 13)
  • Changes to the output of the Deconvolution task.
  • New parameters for the removal tasks and updated functionality for *identifyLines.
Line: 38 to 38
 
  • A new task runPacsSpg has been introduced, with which users can run the full PACS SPG pipeline from within HIPE.

SPIRE

Changed:
<
<
  • FTS
    • Two new tasks and corresponding calibration products have been introduced for SPIRE FTS data. One task corrects for artefacts in low resolution point source calibrated spectra. The other corrects the level of all extended source calibrated data in or to get better agreement between the SPIRE Spectrometer and Photometer.
>
>
  • Major impact: SPIRE Spectrometer extended source calibrated spectra are corrected for the far-field feedhorn efficiency. The correction leads to systematic ~40% higher intensities for SSW and ~40-100% for SLW. All extended source calibration products - spectra or spectral cubes - are affected by this significant effect. With this correction the extended source calibrated SPIRE Photometer and Spectrometer products are in good agreement.
  • Correction for artefacts in SPIRE Spectrometer Low Resolution (LR) point source calibrated spectra.
  • The default SPIRE Spectrometer spectral cubes are now produced with the convolution projection (CP) method: the browse images/thumbnails and standalone browse products are now the CP cubes.
 

Core system

Line: 79 to 80
  would open HIPE with three viewers: one for the image, a spectrum explorer for the cube, and a dataset viewer for the CSV table.
Changed:
<
<

ALERT! Note:
>
>

ALERT! Note:
  Passing a Jython script as the first argument will still behave as running HIPE in batch mode:
bash> hipe myScript.py arg1 arg2
Line: 272 to 273
 
    • The JScanam ipipe script scanmap_JScanam_multiplePairs is designed to process multiple pairs of obsIDs and it has been upgraded by introducing the scanamorphosAlignFrames Task. This task aligns the Frames signal levels, belonging to every observation, to a common offset level, by exploiting the overlapping sky regions. This new capability is particulary useful for recovering a common background level over observations that are partially or slightly overlapping.
    • The script scanmap_Unimap_memory_optimized has been added to the Pipelines menu. This allows one to run the Unimap release located on the userís computer while using the minimum amount of memory. It is particularly useful for processing large observations or/and if several obsIDs should be processed all together in a single run.
    • The script photextended_pipeline_L3 is now available in the Pipelines menu. This can be used to generate mosaics of Unimap and Jscanam Level2.5 maps.
Changed:
<
<
  • Unimap : Unimap 6.2.0 is spawned by the Standard Product Generator (SPG) to create Level2.5 products. See the Photometer Data Reduction Guide and the offical Unimap webpages (http://infocom.uniroma1.it/unimap/) to know What's New about the 6.2.0 Unimap release.
>
>
  • Unimap : Unimap 6.2.0 is spawned by the Standard Product Generator (SPG) to create Level2.5 products. See the Photometer Data Reduction Guide and the offical Unimap webpages (http://infocom.uniroma1.it/unimap/) to know What's New about the 6.2.0 Unimap release.
 

Spectroscopy

  • Level 2/2.5 extracted spectra tables:
    • A new product is provided at Level 2 for all (line, range, chopNod, unchopped) pointed observations, and also at Level 2.5 for the subset of unchopped, rangeScan observations. This table, HPSSPEC[R|B], contains the extracted spectrum of the central spaxel, and the point source calibrated spectra "c1" and "c129" from the pipeline task extractCentralSpectrum. Since the point-source calibrated spectra are taken from the centre of the cube, it is necessary that the source is located within spaxel 2,2 (the central one). However, we make no determination that the target of the observation actually is a point source - that is up to the end-user to establish. These tables are also provided as standalone browse products.
Line: 295 to 296
 

Spectroscopy

Changed:
<
<
  • The extended-source calibrated rebinned cubes (level2) contain now a correction that accounts for the non-uniform illumination of the PACS IFU field-of-view. It stems from the fact that the 9.4"x9.4" spaxel edges are less responsive than their centres, translating into a spaxel active area of approximately 8"x8" where the response is flat. This effect is more severe at shorter wavelengths, and affects (very) extended emission, as well as off-centred point sources and any source that has a size between a point and a very extended. All Level 2/2.5 cubes for these types of sources will be affected, whether pointed or mapping.
    Hence, for centred point source extracted with the task extractCentralSpectrum, no change to the spectra are expected in this version of HIPE because of this effect. For very extended sources, we have developed a new Extended Source Correction calibration file (ExtendedSourceLoss, applied in the pipeline tasks specRespCal and specRespCalToTelescope), and this incorporates the necessary correction as part of the spatial flatfielding for the PACS spectrometer. For these sources, the fluxes of the spectra will differ to what was obtained in previous versions of HIPE. For intermediate sources, a more complex analysis is necessary to estimate the flux loss, and a task to do this will be provided in HIPE 15.
    The fluxes of the spaxels of all cubes will be increased by ~30% in the blue to ~5% in the red. The calibration file can be inspected to extract the curve:
    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data
    p=PlotXY(w, 1./f)
    After such a correction, a source of flat surface brightness within the field of view of a single pointing will be well calibrated at the level of the rebinned cubes (i.e. level2).The flux loss between the detectors was already accounted for in the point-source calibration correction in the correspondent calibration file, therefore, point-source calibrated spectra of centred point sources are not affected. For this reason, in HIPE 14, the extractCentralSpectrum task checks and removes the extendedSourceLoss correction from the level2 cubes, in case it has been applied, before extracting the final point source corrected spectrum. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: to get this, raise a helpdesk ticket.
>
>
  • The extended-source calibrated rebinned cubes (level2) contain now a correction that accounts for the non-uniform illumination of the PACS IFU field-of-view. It stems from the fact that the 9.4"x9.4" spaxel edges are less responsive than their centres, translating into a spaxel active area of approximately 8"x8" where the response is flat. This effect is more severe at shorter wavelengths, and affects (very) extended emission, as well as off-centred point sources and any source that has a size between a point and a very extended. All Level 2/2.5 cubes for these types of sources will be affected, whether pointed or mapping.
    Hence, for centred point source extracted with the task extractCentralSpectrum, no change to the spectra are expected in this version of HIPE because of this effect. For very extended sources, we have developed a new Extended Source Correction calibration file (ExtendedSourceLoss, applied in the pipeline tasks specRespCal and specRespCalToTelescope), and this incorporates the necessary correction as part of the spatial flatfielding for the PACS spectrometer. For these sources, the fluxes of the spectra will differ to what was obtained in previous versions of HIPE. For intermediate sources, a more complex analysis is necessary to estimate the flux loss, and a task to do this will be provided in HIPE 15.
    The fluxes of the spaxels of all cubes will be increased by ~30% in the blue to ~5% in the red. The calibration file can be inspected to extract the curve:
    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data
    p=PlotXY(w, 1./f)
    After such a correction, a source of flat surface brightness within the field of view of a single pointing will be well calibrated at the level of the rebinned cubes (i.e. level2).The flux loss between the detectors was already accounted for in the point-source calibration correction in the correspondent calibration file, therefore, point-source calibrated spectra of centred point sources are not affected. For this reason, in HIPE 14, the extractCentralSpectrum task checks and removes the extendedSourceLoss correction from the level2 cubes, in case it has been applied, before extracting the final point source corrected spectrum. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: to get this, raise a helpdesk ticket.
 
  • New beam calibration files have been delivered, differing from the previous files only in the way they are normalised. These will be used in a new task being developed in HIPE 15, and do not impact on HIPE 14 or SPG 14 in any way.
    • In the spatial domain, they are normalised such that the sum of all spaxels is 1 when a point source of flux 1 is observed at the peak position of the central spaxel;
Line: 333 to 334
 

Calibration Products

  • Spectrometer
Changed:
<
<
    • A new calibration product SCalSpecLrCorr has been added, containing correction curves for point-source calibrated LR data.
    • A new calibration product SCalSpecExtCorr has been added, containing polynomial coefficients that are used to generate correction curves for extended source calibrated data.
    • A new calibration product SCalSpecOpdCorr has been added, it is currently neither used in the pipelines nor in user scripts.
>
>
    • A new calibration product SCalSpecLrCorr has been added, containing correction curves for point-source calibrated LR data.
    • A new calibration product SCalSpecExtCorr has been added, containing polynomial coefficients that are used to generate correction curves for extended source calibrated data.
    • A new calibration product SCalSpecOpdCorr has been added, it is currently neither used in the pipelines nor in user scripts.
 
  • Photometer
Changed:
<
<
    • Updates to the photometer beams: SCalPhotBeamProf and SCalPhotRadialCorrBeam affecting most colour correction parameters.
>
>
 

Calibration Framework

Line: 362 to 363
 

Documentation


Changed:
<
<
<-- COMMENT BOX CODE - DO NOT EDIT -->
>
>
<!-- COMMENT BOX CODE - DO NOT EDIT -->
 
Changed:
<
<
blog comments powered by Disqus
<-- END OF COMMENT BOX CODE -->
>
>
<script type="text/javascript">
<br />var disqus_shortname = 'herscheltwiki'; // required: replace example with your forum shortname<br /><br />(function() {<br />var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;<br />dsq.src = 'http://' + disqus_shortname + '.disqus.com/embed.js';<br />(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);<br />})();<br /></script> <noscript>Please enable JavaScript to view the comments powered by Disqus.</noscript> blog comments powered by Disqus <!-- END OF COMMENT BOX CODE -->
 
Changed:
<
<
<--
* Set ALLOWTOPICCHANGE = RegisteredUsersGroup, TWikiAdminGroup
-->
>
>
<!--<br />* Set ALLOWTOPICCHANGE = RegisteredUsersGroup, TWikiAdminGroup <br />-->
  -- AlvarGarcia - 24 Nov 2014

Editors & Viewers

Revision 492015-12-21 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in HIPE 14

Line: 20 to 20
 
  • SVV resets and STR switch-overs handled by auxiliary processors in the generation of the pointing product.

HIFI

Changed:
<
<
  • Introduction of a new calibration output providing a frequency-dependent intensity calibration uncertainty budget
  • Update of sideband ratio in bands 1 to 4
>
>
  • Update of sideband ratio in bands 1 to 4 - note that this will imply changes in the intensities in all concerned bands
  • Introduction of a new calibration output providing a frequency-dependent intensity calibration uncertainty budget
 
  • Provision of spur channel flags for all modes (was limited to spectral scan up to HIPE 13)
  • Changes to the output of the Deconvolution task.
  • New parameters for the removal tasks and updated functionality for *identifyLines.

Revision 482015-12-15 - MiguelSanchez

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
Changed:
<
<
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

>
>
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->
 

What's new in HIPE 14

PDF Version Click here
Line: 17 to 17
 
  • New Quick Analysis perspective allows GUI download of products and processing all within HIPE.
  • New product Auxiliary Observation Summary Product with information about the observation in a simple dataset.
  • Quality flags have been reorganised.
Added:
>
>
  • SVV resets and STR switch-overs handled by auxiliary processors in the generation of the pointing product.
  HIFI
  • Introduction of a new calibration output providing a frequency-dependent intensity calibration uncertainty budget
Line: 96 to 99
 

Dataset viewer

Astronomical utilities

Deleted:
<
<
 
  • Special corrections in pointing products to handle S/C operational events have been included in the auxiliary processors. Namely, the corrections are:
    • STR switch-overs, i.e. changes from the usually prime star-tracker (STR1) to the backup unit (STR2)
Changed:
<
<
    • Reset of the Spacecraft Velocity Vector (SVV) that is used by the STR to compute the aberration correction of the coordinates of guide stars. For more information please refer to the the dedicated section in the Spacecraft and Observatory pages.
>
>
    • Reset of the Spacecraft Velocity Vector (SVV) that is used by the STR to compute the aberration correction of the coordinates of guide stars. For more information please refer to the the dedicated section in the Spacecraft and Observatory pages.
    • Caveat: large offsets (compatible with STR misalignment) have been reported in two observations, namely 1342183075 and 1342183076. See SPR HCSS-20990.
  • Now up to 18 stars are now included in the computation of the gyro-based attitude reconstruction, whenever available in TM (in the previous release, only up to 9 stars were included).
 

Numeric routines

Data fitting

Line: 361 to 364
 

Documentation


Changed:
<
<
<!-- COMMENT BOX CODE - DO NOT EDIT -->
>
>
<-- COMMENT BOX CODE - DO NOT EDIT -->
 
Changed:
<
<
<script type="text/javascript"><br />var disqus_shortname = 'herscheltwiki'; // required: replace example with your forum shortname<br /><br />(function() {<br />var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;<br />dsq.src = 'http://' + disqus_shortname + '.disqus.com/embed.js';<br />(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);<br />})();<br /></script> <noscript>Please enable JavaScript to view the comments powered by Disqus.</noscript> blog comments powered by Disqus <!-- END OF COMMENT BOX CODE -->
>
>
blog comments powered by Disqus
<-- END OF COMMENT BOX CODE -->
 
Changed:
<
<
<!--<br />* Set ALLOWTOPICCHANGE = RegisteredUsersGroup, TWikiAdminGroup <br />-->
>
>
<--
* Set ALLOWTOPICCHANGE = RegisteredUsersGroup, TWikiAdminGroup
-->
  -- AlvarGarcia - 24 Nov 2014

Editors & Viewers

Revision 472015-12-15 - MiguelSanchez

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's new in HIPE 14

Line: 98 to 96
 

Dataset viewer

Astronomical utilities

Added:
>
>
  • Gyro-based attitude reconstruction, see the dedicated section in the Spacecraft and Observatory pages.
  • Special corrections in pointing products to handle S/C operational events have been included in the auxiliary processors. Namely, the corrections are:
    • STR switch-overs, i.e. changes from the usually prime star-tracker (STR1) to the backup unit (STR2)
    • Reset of the Spacecraft Velocity Vector (SVV) that is used by the STR to compute the aberration correction of the coordinates of guide stars. For more information please refer to the the dedicated section in the Spacecraft and Observatory pages.
 

Numeric routines

Data fitting

Revision 462015-12-14 - RosalindHopwood

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's new in HIPE 14

Line: 347 to 349
 
  • The serendipity data (when the telescope slews to a target) is now included all the way to Level 1 in the Observation. Note that the serendipity data is the final building block and is NOT destriped.
  • The SPIRE Level 2.5 data products have been extended to include, not only parallel pairs, but also sets of 2,3,4+ observations that would benefit from merging at the Level 2.5 stage. See the SPIRE Data Reduction Guide for a list of these observations.

Spectrometer Pipeline

Changed:
<
<
  • A new task has been added to the point source pipeline that applies a correction to remove significant artefacts from SLW spectra of LR observations.
>
>
  • A new task has been added to the point source pipeline that applies a correction to remove significant artefacts from SLW spectra of LR observations (and the LR part of H+LR observations)
 
  • A new task has been added to all pipelines that applies the extended calibration correction in order to incorporate the far field feedhorn efficiency for extended source. As a consequence we get better agreement between FTS and Phototmeter for fully extended sources.

Interactive Analysis and Tools

Revision 452015-12-14 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's new in HIPE 14

Line: 182 to 182
 

HIFI Software Configuration and Properties

HIFI Calibration

Changed:
<
<
  • A new calibration tree has been implemented in HIPE 14.0. It contains substantial changes in the sideband ratio tables, as well as a complete intensity calibration uncertainty component model. The corresponding UncertaintyTable products are available in calibration -> Downlink -> Generic were updated with the following uncertaintyType HotLoadTemp, ColdLoadTemp, HotLoadCoupling, ColdLoadCoupling, SidebandRatio, OpticalStandingWavesLoads, OpticalStandingWavesDiplexer
  • The new calibration tree also provides tables of spur warnign channel flags for point and mapping modes. These flags are assigned based on a knowledgebase built out of the spur flags manually identified for spectral scans and populated in the calibration tree in HIPE 13.0
>
>
  • A new calibration tree has been implemented in HIPE 14.0. It contains substantial changes in the sideband ratio tables, as well as a complete intensity calibration uncertainty component model. The corresponding UncertaintyTable products are available in calibration -> Downlink -> Generic and were updated with the following uncertaintyType HotLoadTemp, ColdLoadTemp, HotLoadCoupling, ColdLoadCoupling, SidebandRatio, OpticalStandingWavesLoads, OpticalStandingWavesDiplexer
  • The new calibration tree also provides tables of spur warning channel flags for point and mapping modes. These flags are assigned based on a knowledgebase built out of the spur flags manually identified for spectral scans and populated in the calibration tree in HIPE 13.0
 
  • The sideband ratio updates will imply changes in intensity at all frequencies in bands 1 to 4. No changes are expected in bands 5 to 7.

HIFI Pipeline

hifiPipeline task

Line: 194 to 194
 

Level 2 Pipeline

  • For DBS Raster maps, the Level 2 spectra are no longer averaged - this was already the case for OTF maps
  • Introduction of a new calibration output providing a frequency-dependent intensity calibration uncertainty budget
Changed:
<
<
  • Assignement of spur warning channel flag for all point and mapping modes
>
>
  • Assignment of spur warning channel flag for all point and mapping modes
 

Level 2.5 Pipeline

  • Changes to the output of the Deconvolution task: the single-sideband spectrum is now contained in a Spectrum1d called "dataset" (it was called "ssb" up to HIPE 13).

HIFI Products

Revision 442015-12-14 - RosalindHopwood

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's new in HIPE 14

Line: 347 to 347
 
  • The serendipity data (when the telescope slews to a target) is now included all the way to Level 1 in the Observation. Note that the serendipity data is the final building block and is NOT destriped.
  • The SPIRE Level 2.5 data products have been extended to include, not only parallel pairs, but also sets of 2,3,4+ observations that would benefit from merging at the Level 2.5 stage. See the SPIRE Data Reduction Guide for a list of these observations.

Spectrometer Pipeline

Changed:
<
<
  • A new task has been added to the point source pipeline that applies the LR correction. It is only done for SLW and only in a subset of observations.
>
>
  • A new task has been added to the point source pipeline that applies a correction to remove significant artefacts from SLW spectra of LR observations.
 
  • A new task has been added to all pipelines that applies the extended calibration correction in order to incorporate the far field feedhorn efficiency for extended source. As a consequence we get better agreement between FTS and Phototmeter for fully extended sources.

Interactive Analysis and Tools

Revision 432015-12-14 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's new in HIPE 14

Line: 73 to 73
 Opening the observation context instead is also possible, either with the popup menu or by configuring so as the default behaviour, in the Quick Analysis preferences category.

Command line

Changed:
<
<
It is now possible to pass arguments to HIPE from the command line, which will be tried to be interpreted as files to be opened with the proper viewers. For example:
>
>
It is now possible to pass arguments to HIPE from the command line, which will be tried to be interpreted as files to be opened with the proper viewers. For example:
 
bash> hipe myImage.jpg cube.fits resultTable.csv

would open HIPE with three viewers: one for the image, a spectrum explorer for the cube, and a dataset viewer for the CSV table.


ALERT! Note:

Changed:
<
<
Passing a Jython script as the first argument will still behave as running HIPE in batch mode:
>
>
Passing a Jython script as the first argument will still behave as running HIPE in batch mode:
 
bash> hipe myScript.py arg1 arg2

would not open the HIPE window, but execute the script with the given arguments passed to it.

Line: 289 to 289
 

Spectroscopy

Changed:
<
<
  • The extended-source calibrated rebinned cubes (level2) contain now a correction that accounts for the non-uniform illumination of the PACS IFU field-of-view. It stems from the fact that the 9.4"x9.4" spaxel edges are less responsive than their centres, translating into a spaxel active area of approximately 8"x8" where the response is flat. This effect is more severe at shorter wavelengths, and affects (very) extended emission, as well as off-centred point sources and any source that has a size between a point and a very extended. All Level 2/2.5 cubes for these types of sources will be affected, whether pointed or mapping.
    Hence, for centred point source extracted with the task extractCentralSpectrum, no change to the spectra are expected in this version of HIPE because of this effect. For very extended sources, we have developed a new Extended Source Correction calibration file (ExtendedSourceLoss, applied in the pipeline tasks specRespCal and specRespCalToTelescope), and this incorporates the necessary correction as part of the spatial flatfielding for the PACS spectrometer. For these sources, the fluxes of the spectra will differ to what was obtained in previous versions of HIPE. For intermediate sources, a more complex analysis is necessary to estimate the flux loss, and a task to do this will be provided in HIPE 15.
    The fluxes of the spaxels of all cubes will be increased by ~30% in the blue to ~5% in the red. The calibration file can be inspected to extract the curve:
    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data
    p=PlotXY(w, 1./f)
    After such a correction, a source of flat surface brightness within the field of view of a single pointing will be well calibrated at the level of the rebinned cubes (i.e. level2).The flux loss between the detectors was already accounted for in the point-source calibration correction in the correspondent calibration file, therefore, point-source calibrated spectra of centred point sources are not affected. For this reason, in HIPE 14, the extractCentralSpectrum task checks and removes the extendedSourceLoss correction from the level2 cubes, in case it has been applied, before extracting the final point source corrected spectrum. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: to get this, raise a helpdesk ticket.
>
>
  • The extended-source calibrated rebinned cubes (level2) contain now a correction that accounts for the non-uniform illumination of the PACS IFU field-of-view. It stems from the fact that the 9.4"x9.4" spaxel edges are less responsive than their centres, translating into a spaxel active area of approximately 8"x8" where the response is flat. This effect is more severe at shorter wavelengths, and affects (very) extended emission, as well as off-centred point sources and any source that has a size between a point and a very extended. All Level 2/2.5 cubes for these types of sources will be affected, whether pointed or mapping.
    Hence, for centred point source extracted with the task extractCentralSpectrum, no change to the spectra are expected in this version of HIPE because of this effect. For very extended sources, we have developed a new Extended Source Correction calibration file (ExtendedSourceLoss, applied in the pipeline tasks specRespCal and specRespCalToTelescope), and this incorporates the necessary correction as part of the spatial flatfielding for the PACS spectrometer. For these sources, the fluxes of the spectra will differ to what was obtained in previous versions of HIPE. For intermediate sources, a more complex analysis is necessary to estimate the flux loss, and a task to do this will be provided in HIPE 15.
    The fluxes of the spaxels of all cubes will be increased by ~30% in the blue to ~5% in the red. The calibration file can be inspected to extract the curve:
    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data
    p=PlotXY(w, 1./f)
    After such a correction, a source of flat surface brightness within the field of view of a single pointing will be well calibrated at the level of the rebinned cubes (i.e. level2).The flux loss between the detectors was already accounted for in the point-source calibration correction in the correspondent calibration file, therefore, point-source calibrated spectra of centred point sources are not affected. For this reason, in HIPE 14, the extractCentralSpectrum task checks and removes the extendedSourceLoss correction from the level2 cubes, in case it has been applied, before extracting the final point source corrected spectrum. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: to get this, raise a helpdesk ticket.
 
  • New beam calibration files have been delivered, differing from the previous files only in the way they are normalised. These will be used in a new task being developed in HIPE 15, and do not impact on HIPE 14 or SPG 14 in any way.
    • In the spatial domain, they are normalised such that the sum of all spaxels is 1 when a point source of flux 1 is observed at the peak position of the central spaxel;
Line: 327 to 327
 

Calibration Products

  • Spectrometer
Changed:
<
<
    • A new calibration product has been added that contains correction curves for point-source calibrated LR data.
    • A new calibration product has been added that contains polynomial coefficients that are used to generate correction curves for extended source calibrated data.
>
>
    • A new calibration product SCalSpecLrCorr has been added, containing correction curves for point-source calibrated LR data.
    • A new calibration product SCalSpecExtCorr has been added, containing polynomial coefficients that are used to generate correction curves for extended source calibrated data.
    • A new calibration product SCalSpecOpdCorr has been added, it is currently neither used in the pipelines nor in user scripts.
  • Photometer
    • Updates to the photometer beams: SCalPhotBeamProf and SCalPhotRadialCorrBeam affecting most colour correction parameters.
 

Calibration Framework

Line: 340 to 343
 

Photometer Pipeline

  • The User Pipelines have been updated with an option to create not just the point source maps but also the absolute calibtated maps in MJy/sr by setting the makeExtendMaps option
Changed:
<
<
  • Access to the Planck maps in boter the User pipelines and Useful Scripts is now handled automatically (see the zero point correction script for details on initial set up)
>
>
  • Access to the Planck maps in both the User pipelines and Useful Scripts is now handled automatically (see the zero point correction script for details on initial set up)
 
  • The serendipity data (when the telescope slews to a target) is now included all the way to Level 1 in the Observation. Note that the serendipity data is the final building block and is NOT destriped.
  • The SPIRE Level 2.5 data products have been extended to include, not only parallel pairs, but also sets of 2,3,4+ observations that would benefit from merging at the Level 2.5 stage. See the SPIRE Data Reduction Guide for a list of these observations.

Spectrometer Pipeline

Changed:
<
<
  • A new task has been added to the point source pipeline that applies the LR correction.
  • A new task has been added to all pipelines that applies the extended calibration correction in order to get better agreement between FTS and Phototmeter photometry.
>
>
  • A new task has been added to the point source pipeline that applies the LR correction. It is only done for SLW and only in a subset of observations.
  • A new task has been added to all pipelines that applies the extended calibration correction in order to incorporate the far field feedhorn efficiency for extended source. As a consequence we get better agreement between FTS and Phototmeter for fully extended sources.
 

Interactive Analysis and Tools

  • There is a new useful script, Spectrometer Pointing Offset Corrector, for estimating and correcting pointing offset in sparse point observations. The scripts assesses the difference between the SLW and SSW spectra of the centre detectors, over the overlapping frequency range, uses this to estimate any pointing offset and corrects the data using SECT.

Revision 422015-12-11 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
Changed:
<
<
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

>
>
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

 

What's new in HIPE 14

PDF Version Click here
Changed:
<
<
Help Interested in what's new for other versions of HIPE? See this page for links to all the What's New documents from HIPE 4.0 onwards.
>
>
Help Interested in what's new for other versions of HIPE? See this page for links to all the What's New documents from HIPE 4.0 onwards.
 See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.
Changed:
<
<

>
>

 

Highlights

Line: 73 to 73
 Opening the observation context instead is also possible, either with the popup menu or by configuring so as the default behaviour, in the Quick Analysis preferences category.

Command line

Changed:
<
<
It is now possible to pass arguments to HIPE from the command line, which will be tried to be interpreted as files to be opened with the proper viewers. For example:
>
>
It is now possible to pass arguments to HIPE from the command line, which will be tried to be interpreted as files to be opened with the proper viewers. For example:
 
bash> hipe myImage.jpg cube.fits resultTable.csv

would open HIPE with three viewers: one for the image, a spectrum explorer for the cube, and a dataset viewer for the CSV table.

Changed:
<
<

ALERT! Note:

Passing a Jython script as the first argument will still behave as running HIPE in batch mode:

>
>

ALERT! Note:
 
Added:
>
>
Passing a Jython script as the first argument will still behave as running HIPE in batch mode:
 
bash> hipe myScript.py arg1 arg2

would not open the HIPE window, but execute the script with the given arguments passed to it.

Line: 268 to 266
 
    • The JScanam ipipe script scanmap_JScanam_multiplePairs is designed to process multiple pairs of obsIDs and it has been upgraded by introducing the scanamorphosAlignFrames Task. This task aligns the Frames signal levels, belonging to every observation, to a common offset level, by exploiting the overlapping sky regions. This new capability is particulary useful for recovering a common background level over observations that are partially or slightly overlapping.
    • The script scanmap_Unimap_memory_optimized has been added to the Pipelines menu. This allows one to run the Unimap release located on the userís computer while using the minimum amount of memory. It is particularly useful for processing large observations or/and if several obsIDs should be processed all together in a single run.
    • The script photextended_pipeline_L3 is now available in the Pipelines menu. This can be used to generate mosaics of Unimap and Jscanam Level2.5 maps.
Changed:
<
<
  • Unimap : Unimap 6.2.0 is spawned by the Standard Product Generator (SPG) to create Level2.5 products. See the Photometer Data Reduction Guide and the offical Unimap webpages (http://infocom.uniroma1.it/unimap/) to know What's New about the 6.2.0 Unimap release.
>
>
  • Unimap : Unimap 6.2.0 is spawned by the Standard Product Generator (SPG) to create Level2.5 products. See the Photometer Data Reduction Guide and the offical Unimap webpages (http://infocom.uniroma1.it/unimap/) to know What's New about the 6.2.0 Unimap release.
 

Spectroscopy

  • Level 2/2.5 extracted spectra tables:
    • A new product is provided at Level 2 for all (line, range, chopNod, unchopped) pointed observations, and also at Level 2.5 for the subset of unchopped, rangeScan observations. This table, HPSSPEC[R|B], contains the extracted spectrum of the central spaxel, and the point source calibrated spectra "c1" and "c129" from the pipeline task extractCentralSpectrum. Since the point-source calibrated spectra are taken from the centre of the cube, it is necessary that the source is located within spaxel 2,2 (the central one). However, we make no determination that the target of the observation actually is a point source - that is up to the end-user to establish. These tables are also provided as standalone browse products.
Line: 291 to 289
 

Spectroscopy

Changed:
<
<
  • The extended-source calibrated rebinned cubes (level2) contain now a correction that accounts for the non-uniform illumination of the PACS IFU field-of-view. It stems from the fact that the 9.4"x9.4" spaxel edges are less responsive than their centres, translating into a spaxel active area of 8"x8" where the response is flat. This effect is more severe at shorter wavelengths, and affects (very) extended emission, as well as off-centred point sources and any source that has a size between a point and a very extended. All Level 2/2.5 cubes for these types of sources will be affected.
    The new Extended Source Correction calibration file (ExtendedSourceLoss, applied in the pipeline tasks specRespCal and specRespCalToTelescope) incorporates the correction as part of the spatial flatfielding for the PACS spectrometer.
    The fluxes of the spaxels of all cubes will be increased by ~30% in the blue to ~5% in the red. The calibration file can be inspected to extract the curve:

    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data
    p=PlotXY(w, 1./f)

    After such a correction, a source of flat surface brightness will be well calibrated at the level of the rebinned cubes (i.e. level2).
    The flux loss between the detectors was already accounted for in the point-source calibration correction in the correspondent calibration file, therefore, point-source calibrated spectra of centred point sources are not affected. For this reason, in HIPE 14, the extractCentralSpectrum task checks and removes the extendedSourceLoss correction from the level2 cubes, in case it has been applied, before extracting the final point source corrected spectrum. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: to get this, raise a helpdesk ticket.
>
>
  • The extended-source calibrated rebinned cubes (level2) contain now a correction that accounts for the non-uniform illumination of the PACS IFU field-of-view. It stems from the fact that the 9.4"x9.4" spaxel edges are less responsive than their centres, translating into a spaxel active area of approximately 8"x8" where the response is flat. This effect is more severe at shorter wavelengths, and affects (very) extended emission, as well as off-centred point sources and any source that has a size between a point and a very extended. All Level 2/2.5 cubes for these types of sources will be affected, whether pointed or mapping.
    Hence, for centred point source extracted with the task extractCentralSpectrum, no change to the spectra are expected in this version of HIPE because of this effect. For very extended sources, we have developed a new Extended Source Correction calibration file (ExtendedSourceLoss, applied in the pipeline tasks specRespCal and specRespCalToTelescope), and this incorporates the necessary correction as part of the spatial flatfielding for the PACS spectrometer. For these sources, the fluxes of the spectra will differ to what was obtained in previous versions of HIPE. For intermediate sources, a more complex analysis is necessary to estimate the flux loss, and a task to do this will be provided in HIPE 15.
    The fluxes of the spaxels of all cubes will be increased by ~30% in the blue to ~5% in the red. The calibration file can be inspected to extract the curve:
    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data
    p=PlotXY(w, 1./f)
    After such a correction, a source of flat surface brightness within the field of view of a single pointing will be well calibrated at the level of the rebinned cubes (i.e. level2).The flux loss between the detectors was already accounted for in the point-source calibration correction in the correspondent calibration file, therefore, point-source calibrated spectra of centred point sources are not affected. For this reason, in HIPE 14, the extractCentralSpectrum task checks and removes the extendedSourceLoss correction from the level2 cubes, in case it has been applied, before extracting the final point source corrected spectrum. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: to get this, raise a helpdesk ticket.
 
  • New beam calibration files have been delivered, differing from the previous files only in the way they are normalised. These will be used in a new task being developed in HIPE 15, and do not impact on HIPE 14 or SPG 14 in any way.
    • In the spatial domain, they are normalised such that the sum of all spaxels is 1 when a point source of flux 1 is observed at the peak position of the central spaxel;

Revision 412015-12-11 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's new in HIPE 14

Line: 289 to 291
 

Spectroscopy

Changed:
<
<
  • The extended-source calibrated rebinned cubes (level2) contain now a correction that accounts for the non-uniform illumination of the PACS IFU field-of-view. It stems from the fact that the 9.4"x9.4" spaxel edges are less responsive than their centres, translating into a spaxel active area of 8"x8" where the response is flat. This effect is more severe at shorter wavelengths, and affects (very) extended emission as well as off-centred point sources and drizzle/projected/interpolated maps.
    The new Extended Source Correction calibration file (ExtendedSourceLoss, applied in the pipeline tasks specRespCal and specRespCalToTelescope) incorporates the correction as part of the spatial flatfielding for the PACS spectrometer.
    The fluxes of the spaxels of all cubes will be increased by ~30% in the blue to ~5% in the red. The calibration file can be inspected to extract the curve:

    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data
    p=PlotXY(w, 1./f)

    After such a correction, a source of flat surface brightness will be well calibrated at the level of the rebinned cubes (i.e. level2).
    The flux loss between the detectors was already accounted for in the point-source calibration correction in the correspondent calibration file, therefore, point-source calibrated spectra of centred point sources are not affected. For this reason, in HIPE 14, the extractCentralSpectrum task checks and removes the extendedSourceLoss correction from the level2 cubes, in case it has been applied, before extracting the final point source corrected spectrum. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: to get this, raise a helpdesk ticket.
>
>
  • The extended-source calibrated rebinned cubes (level2) contain now a correction that accounts for the non-uniform illumination of the PACS IFU field-of-view. It stems from the fact that the 9.4"x9.4" spaxel edges are less responsive than their centres, translating into a spaxel active area of 8"x8" where the response is flat. This effect is more severe at shorter wavelengths, and affects (very) extended emission, as well as off-centred point sources and any source that has a size between a point and a very extended. All Level 2/2.5 cubes for these types of sources will be affected.
    The new Extended Source Correction calibration file (ExtendedSourceLoss, applied in the pipeline tasks specRespCal and specRespCalToTelescope) incorporates the correction as part of the spatial flatfielding for the PACS spectrometer.
    The fluxes of the spaxels of all cubes will be increased by ~30% in the blue to ~5% in the red. The calibration file can be inspected to extract the curve:

    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data
    p=PlotXY(w, 1./f)

    After such a correction, a source of flat surface brightness will be well calibrated at the level of the rebinned cubes (i.e. level2).
    The flux loss between the detectors was already accounted for in the point-source calibration correction in the correspondent calibration file, therefore, point-source calibrated spectra of centred point sources are not affected. For this reason, in HIPE 14, the extractCentralSpectrum task checks and removes the extendedSourceLoss correction from the level2 cubes, in case it has been applied, before extracting the final point source corrected spectrum. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: to get this, raise a helpdesk ticket.
 
  • New beam calibration files have been delivered, differing from the previous files only in the way they are normalised. These will be used in a new task being developed in HIPE 15, and do not impact on HIPE 14 or SPG 14 in any way.
    • In the spatial domain, they are normalised such that the sum of all spaxels is 1 when a point source of flux 1 is observed at the peak position of the central spaxel;

Revision 402015-12-11 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
Changed:
<
<
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->
>
>
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

 

What's new in HIPE 14

PDF Version Click here
Line: 22 to 22
 
  • Introduction of a new calibration output providing a frequency-dependent intensity calibration uncertainty budget
  • Update of sideband ratio in bands 1 to 4
  • Provision of spur channel flags for all modes (was limited to spectral scan up to HIPE 13)
Changed:
<
<
  • Changes to the output of the Deconvolution task.
  • New parameters for the *removal tasks and updated functionality for identifyLines.
>
>
  • Changes to the output of the Deconvolution task.
  • New parameters for the removal tasks and updated functionality for *identifyLines.
 
  • Legacy quality documentation.

PACS

Line: 219 to 217
 

Baseline Removal

  • New parameter addMedianContinuum: allow the median continuum to be added back into the baseline fit flux but only performs well for basemode='sub' and not basemode='div' [DEFAULT: False]

Deconvolution

Changed:
<
<
  • The output deconvolution product ssb is now called dataset (please note that scripts prior to this change will break)
>
>
  • The output deconvolution product ssb is now called dataset (please note that scripts prior to this change will break)
 

Spectral Cube creation

flagTool

identifyLines

Line: 229 to 227
 
    • Changes on the Herschel Spectral Line List columns names
    • Implementation of the image band line identification for pointed and mapping observations
    • Fix to the rms computation
Changed:
<
<
    • Parameters change: toleranceEmission and toleranceAbsorption has become toleranceFactorEmi and toleranceFactorAbs (may break scripts prior to this change)
>
>
    • Parameters change: toleranceEmission and toleranceAbsorption has become toleranceFactorEmi and toleranceFactorAbs (may break scripts prior to this change)
 
    • New parameter snrType: the signal to noise ratio calculation algorithm (Intensity or Flux)
    • Implementation of the exportLines task

HiClass export tool

Line: 241 to 239
 
    • Updates to the chapters Standing wave Removal and HIFI Baseline Removal : adding definition of the new parameter addMedianContinuum
    • Updates to the chapter Flags in HIFI data, section Quality Flags (this chapter is being revamped, please check regularly)
    • Updates to the chapter How to make a spectral cube: added a definition for parameters ignoreOffs, outputNames and extrapolate
Changed:
<
<
    • Updates to the chapter Sideband Deconvolution: the output deconvolution product ssb is now called dataset (please note that scripts prior to this change will break)
>
>
    • Updates to the chapter Sideband Deconvolution: the output deconvolution product ssb is now called dataset (please note that scripts prior to this change will break)
 
    • Updates to the chapter Standing wave Removal, ESW correction in HEB bands section: update to the scripts in the demonstration subsection
    • Updates to the chapter How to add and remove flags, section Setting and Clearing Flags with command line tools: improved the definition of the parameter selection for Level 2.5 cubes
    • A new chapter called Understanding the uncertainty table information in your data has been added
Line: 291 to 289
 

Spectroscopy

Changed:
<
<
  • The extended-source calibrated rebinned cubes (level2) contain now a correction that accounts for the non-uniform illumination of the PACS IFU field-of-view. It stems in the fact that the 9.4"x9.4" spaxel edges are less responsive, translating into a spaxel active area of 8"x8" where this response is flat. This effect is more severe at shorter wavelengths, and affects (very) extended emission as well as off-centred point sources and drizzle maps.
    The new Extended Source Correction calibration file (ExtendedSourceLoss, applied in the pipeline tasks specRespCal and specRespCalToTelescope) incorporates the correction as part of the spatial flatfield for the PACS spectrometer.
    The fluxes of the spaxels of all cubes will be increased by ~30% in the blue to ~5% in the red. The calibration file can be inspected to extract the curve:

    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data
    p=PlotXY(w, 1./f)

    After such a correction, a source of flat surface brightness will be well calibrated at the level of the rebinned cubes (i.e. level2).
    The flux loss between the detectors was already accounted for in the point-source calibration correction in the correspondent calibration file, therefore, point-source calibrated spectra of centred point sources are not affected. For this reason, in HIPE 14, the extractCentralSpectrum task checks and removes the extendedSourceLoss correction from the level2 cubes, in case it has been applied, before extracting the final point source corrected spectrum. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: to get this, raise a helpdesk ticket.
  • New beam calibration files have been delivered, differing from the previous files only in the way they are normalised.
    • In the spatial domain, they are normalized such that the sum of all spaxels is 1 when a point source of flux 1 is observed at the peak position of the central spaxel;
    • In the wavelength domain, the central spaxel is normalized to match the Point Source Correction calibration file.
These will be used in a new task being developed in HIPE 15, and do not impact on HIPE 14 or SPG 14 in any way.
>
>
  • The extended-source calibrated rebinned cubes (level2) contain now a correction that accounts for the non-uniform illumination of the PACS IFU field-of-view. It stems from the fact that the 9.4"x9.4" spaxel edges are less responsive than their centres, translating into a spaxel active area of 8"x8" where the response is flat. This effect is more severe at shorter wavelengths, and affects (very) extended emission as well as off-centred point sources and drizzle/projected/interpolated maps.
    The new Extended Source Correction calibration file (ExtendedSourceLoss, applied in the pipeline tasks specRespCal and specRespCalToTelescope) incorporates the correction as part of the spatial flatfielding for the PACS spectrometer.
    The fluxes of the spaxels of all cubes will be increased by ~30% in the blue to ~5% in the red. The calibration file can be inspected to extract the curve:

    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data
    p=PlotXY(w, 1./f)

    After such a correction, a source of flat surface brightness will be well calibrated at the level of the rebinned cubes (i.e. level2).
    The flux loss between the detectors was already accounted for in the point-source calibration correction in the correspondent calibration file, therefore, point-source calibrated spectra of centred point sources are not affected. For this reason, in HIPE 14, the extractCentralSpectrum task checks and removes the extendedSourceLoss correction from the level2 cubes, in case it has been applied, before extracting the final point source corrected spectrum. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: to get this, raise a helpdesk ticket.

  • New beam calibration files have been delivered, differing from the previous files only in the way they are normalised. These will be used in a new task being developed in HIPE 15, and do not impact on HIPE 14 or SPG 14 in any way.
    • In the spatial domain, they are normalised such that the sum of all spaxels is 1 when a point source of flux 1 is observed at the peak position of the central spaxel;
    • In the wavelength domain, the central spaxel is normalised to match the Point Source Correction calibration file.
 
  • A slightly updated telescope background model for use in the Pointing Offset Correction pipeline script for point sources has been developed. This update is to allow better consistency between the "POC" script and the default pipeline script "Telescope normaliation", for chopNod AOTs. The changes are at the percent level and largest in band B3A.

Analysis tools

Line: 355 to 353
 

Documentation


Changed:
<
<
<-- COMMENT BOX CODE - DO NOT EDIT -->
>
>
<!-- COMMENT BOX CODE - DO NOT EDIT -->
 
Changed:
<
<
blog comments powered by Disqus
<-- END OF COMMENT BOX CODE -->
>
>
<script type="text/javascript"><br />var disqus_shortname = 'herscheltwiki'; // required: replace example with your forum shortname<br /><br />(function() {<br />var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;<br />dsq.src = 'http://' + disqus_shortname + '.disqus.com/embed.js';<br />(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);<br />})();<br /></script> <noscript>Please enable JavaScript to view the comments powered by Disqus.</noscript> blog comments powered by Disqus <!-- END OF COMMENT BOX CODE -->
 
Changed:
<
<
<--
* Set ALLOWTOPICCHANGE = RegisteredUsersGroup, TWikiAdminGroup
-->
>
>
<!--<br />* Set ALLOWTOPICCHANGE = RegisteredUsersGroup, TWikiAdminGroup <br />-->
  -- AlvarGarcia - 24 Nov 2014

Editors & Viewers

Revision 392015-12-10 - ElenaPuga

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's new in HIPE 14

Line: 276 to 278
 
    • The pipeline menu has been reworded, so that the names reflect better what the individual scripts do. The first script offered in each menu is also the preferred one, and that on which the SPG scripts are based.
    • A new pipeline script has been written to create drizzled cubes, based on (and almost hardly different to) the 'Telescope normalisation' pipeline script for chopNod line scans. With this script it is now possible to create drizzle cubes using this preferred pipeline method (in Track 13 it was not possible).
    • There have been small, but potentially significant changes to some of the tasks in the 'Pointing offset correction' scripts (chopNod) and the '...with transients" scripts (unchopped). These are, respectively, to the tasks specDetermineChiSquare and specLongTermTransient. The improvements are "under the hood" so there is no change in the user scripts themselves.
Changed:
<
<
    • The range scan flatfielding task now incorporates a spline fitting to the continuum of the spectra. This produces the same results for shorter ranges but improved results for longer ranges (especially SEDs) and where there are strong curvatures in the spectra.
>
>
    • The range scan flatfielding task now performs a cubic spline fitting to the continuum of the spectra. This produces the same results as previously with the polynomial fit for shorter ranges, but improved results for longer ranges (especially SEDs) and is more robust against curvature changes in the spectra.
 
  • SPG scripts:
    • The range scan observations are now flatfielded: using the spline fitting method for ranges longer than 5 microns, and using the line scan flatfielding for the shorter ranges
Added:
>
>
    • The range scan observations are now flatfielded: using the line scan flatfielding (specFlatFieldLine) for shorter ranges than 5 microns, and using specFlatFieldRange with the cubic spline fitting method for ranges longer than 5 microns. In the latter case, we exclude the spectral regions affected by leakeage in the correspondent bands during the fit, as the calibration is incorrect for those ranges. Additionally, these spectral ranges are trimmed out via the 'NOTFFED' mask.
 
    • The interpolated cubes are no longer produced for oversample range scan mapping observations, and they also have a smaller spaxel size of 3" (rather than 4.7" before).
    • The interpolation algorithm in specInterpolate produces now NaN values in case one of the input spaxels has a NaN value, instead of producing skewed values due to a missing flux value. This results in more NaN values in the interpolated cubes, but avoids non-reliable values instead.
Line: 288 to 291
 

Spectroscopy

Changed:
<
<
  • There is a new Extended Source Correction calibration file (ExtendedSourceLoss, used in the pipeline tasks specRespCal and specRespCalToTelescope) which produces a better spatial flatfield for the PACS spectrometer. The fluxes of the spaxels of all cubes will be increased by ~30% in the blue to a decrease of ~5% in the red. The calibration file can be inspected to extract the curve:

    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data

  • For point sources calibrated with extractCentralSpectrum, this extended source correction is taken out before the point source corrections are computed. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: to get this, raise a helpdesk ticket.
  • New Beam calibration files have been delivered, differing from the previous files only in the way they are normalised. These will be used in a new task being developed in HIPE 15, and do not impact on HIPE 14 or SPG 14 in any way.
>
>
  • The extended-source calibrated rebinned cubes (level2) contain now a correction that accounts for the non-uniform illumination of the PACS IFU field-of-view. It stems in the fact that the 9.4"x9.4" spaxel edges are less responsive, translating into a spaxel active area of 8"x8" where this response is flat. This effect is more severe at shorter wavelengths, and affects (very) extended emission as well as off-centred point sources and drizzle maps.
    The new Extended Source Correction calibration file (ExtendedSourceLoss, applied in the pipeline tasks specRespCal and specRespCalToTelescope) incorporates the correction as part of the spatial flatfield for the PACS spectrometer.
    The fluxes of the spaxels of all cubes will be increased by ~30% in the blue to ~5% in the red. The calibration file can be inspected to extract the curve:

    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data
    p=PlotXY(w, 1./f)

    After such a correction, a source of flat surface brightness will be well calibrated at the level of the rebinned cubes (i.e. level2).
    The flux loss between the detectors was already accounted for in the point-source calibration correction in the correspondent calibration file, therefore, point-source calibrated spectra of centred point sources are not affected. For this reason, in HIPE 14, the extractCentralSpectrum task checks and removes the extendedSourceLoss correction from the level2 cubes, in case it has been applied, before extracting the final point source corrected spectrum. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: to get this, raise a helpdesk ticket.
  • New beam calibration files have been delivered, differing from the previous files only in the way they are normalised.
    • In the spatial domain, they are normalized such that the sum of all spaxels is 1 when a point source of flux 1 is observed at the peak position of the central spaxel;
    • In the wavelength domain, the central spaxel is normalized to match the Point Source Correction calibration file.
These will be used in a new task being developed in HIPE 15, and do not impact on HIPE 14 or SPG 14 in any way.
 
  • A slightly updated telescope background model for use in the Pointing Offset Correction pipeline script for point sources has been developed. This update is to allow better consistency between the "POC" script and the default pipeline script "Telescope normaliation", for chopNod AOTs. The changes are at the percent level and largest in band B3A.

Analysis tools

Line: 303 to 308
 

Product changes

Added:
>
>
  • A bug fix in the waveCalc step for 14.0 causes that some slices redward of the outofband limits, that were previously non-correctly showing, are now properly masked out.
 
  • Many improvements to the descriptions and/or units of the header meta keywords.
  • All keywords have now a corresponding 8 character FITS keyword, so that the headers are easier to read with other software.
Line: 315 to 321
 
  • Interpolated cubes: The way the task specInterpolate handles NaNs as it creates these cubes has changed. This will primarily affect cubes with saturated data. Saturated data are masked out and replaced with NaNs when rebinned cubes are built. Interpolated cubes are then created from the rebinned cubes. NaNs are now filtered out as the interpolation task does its work on the input cubes, and all affected and immediately surrounding spaxels of the output interpolated cube will also be NaN. Hence, at the affected wavelengths, the NaN area will be larger than before...but this is also more correct than before.
  • Standalone browse products: we now provide the new extracted spectra tables, HPSSPEC[R|B] and HPSSPEC (see above), as SBPs.
  • Drizzle cubes: in SPG 13 the drizzle cube fluxes were incorrect. In SPG 14 they are now correct.
Changed:
<
<
  • PacsCubes and PacsRebinnedCubes created by the pipeline (HPS3D [R|B] and HPS3DR [R|B]) will now display in cube viewers with a flip over the diagonal, as compared to previously. The new orientation makes it easier to compare the cubes in cube viewers (the Standard Cube Viewer or the Spectrum Explorer) to the display of the PacsSpectralFootprint viewer.
>
>
  • PacsCubes and PacsRebinnedCubes created by the pipeline (HPS3D [R|B] and HPS3DR [R|B]) will now display in HIPE cube viewers with a flip over the diagonal, as compared to previously (X: to the right, Y: upwards). This change allows the visualization of PACS cubes into the PACS footprint orientation (note the axes display with X: downwards, Y: to the left). The displayed indices correspond to the PACS spaxel indices (see more in Sect. 6.3 of the DAG). The standard HIPE cube visualization can be recovered in two steps, using the flip menu in the cube display control bar: no flip + vertical flip. When exporting the cube data outside HIPE, note that the data will still be organized according to standard cube indices.The new orientation makes it easier to compare the cubes in cube viewers (the Standard Cube Viewer or the Spectrum Explorer) to the display of the PacsSpectralFootprint viewer.
 

Documentation

Revision 382015-12-10 - AlvarGarcia

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->
Changed:
<
<

What's coming up in HIPE 14

>
>

What's new in HIPE 14

 
PDF Version Click here

Revision 372015-12-10 - ChrisPearson

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 335 to 337
 

Common Pipeline

Photometer Pipeline

Changed:
<
<
>
>
  • The User Pipelines have been updated with an option to create not just the point source maps but also the absolute calibtated maps in MJy/sr by setting the makeExtendMaps option
  • Access to the Planck maps in boter the User pipelines and Useful Scripts is now handled automatically (see the zero point correction script for details on initial set up)
  • The serendipity data (when the telescope slews to a target) is now included all the way to Level 1 in the Observation. Note that the serendipity data is the final building block and is NOT destriped.
  • The SPIRE Level 2.5 data products have been extended to include, not only parallel pairs, but also sets of 2,3,4+ observations that would benefit from merging at the Level 2.5 stage. See the SPIRE Data Reduction Guide for a list of these observations.
 

Spectrometer Pipeline

  • A new task has been added to the point source pipeline that applies the LR correction.
  • A new task has been added to all pipelines that applies the extended calibration correction in order to get better agreement between FTS and Phototmeter photometry.

Revision 362015-12-09 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 19 to 19
 
  • Quality flags have been reorganised.

HIFI

Added:
>
>
  • Introduction of a new calibration output providing a frequency-dependent intensity calibration uncertainty budget
  • Update of sideband ratio in bands 1 to 4
  • Provision of spur channel flags for all modes (was limited to spectral scan up to HIPE 13)
 
  • Changes to the output of the Deconvolution task.
  • New parameters for the *removal tasks and updated functionality for identifyLines.
  • Legacy quality documentation.
Line: 181 to 182
 

HIFI Software Configuration and Properties

HIFI Calibration

Changed:
<
<
>
>
  • A new calibration tree has been implemented in HIPE 14.0. It contains substantial changes in the sideband ratio tables, as well as a complete intensity calibration uncertainty component model. The corresponding UncertaintyTable products are available in calibration -> Downlink -> Generic were updated with the following uncertaintyType HotLoadTemp, ColdLoadTemp, HotLoadCoupling, ColdLoadCoupling, SidebandRatio, OpticalStandingWavesLoads, OpticalStandingWavesDiplexer
  • The new calibration tree also provides tables of spur warnign channel flags for point and mapping modes. These flags are assigned based on a knowledgebase built out of the spur flags manually identified for spectral scans and populated in the calibration tree in HIPE 13.0
  • The sideband ratio updates will imply changes in intensity at all frequencies in bands 1 to 4. No changes are expected in bands 5 to 7.
 

HIFI Pipeline

hifiPipeline task

Deleted:
<
<
 

Level 0 Pipeline

WBS Pipeline

HRS Pipeline

Level 1 Pipeline

Level 2 Pipeline

Changed:
<
<
  • For DBS Raster, Level 2 spectra are not averaged
>
>
  • For DBS Raster maps, the Level 2 spectra are no longer averaged - this was already the case for OTF maps
  • Introduction of a new calibration output providing a frequency-dependent intensity calibration uncertainty budget
  • Assignement of spur warning channel flag for all point and mapping modes
 

Level 2.5 Pipeline

Changed:
<
<
>
>
  • Changes to the output of the Deconvolution task: the single-sideband spectrum is now contained in a Spectrum1d called "dataset" (it was called "ssb" up to HIPE 13).
 

HIFI Products

  • New quality flags information:
    • updates of a few QFs names/descriptions (which were obsolete), deleting a few obsolete flags
Line: 204 to 205
 
    • updates to the quality report, now including the text for public flags (impact and action)
  • Flags are now applied to OFF spectra as well
  • BrowseProduct: generation of a browse image for the reference spectra, when the option useReferenceSpectra is set to true
Added:
>
>
  • The FITS header keywords have been revamped in order to provide proper nicknames to the parameters featured in those headers
 

HIFI Spectrum Toolbox

Spectrum Explorer (HIFI specifics)

Line: 230 to 230
 
    • Parameters change: toleranceEmission and toleranceAbsorption has become toleranceFactorEmi and toleranceFactorAbs (may break scripts prior to this change)
    • New parameter snrType: the signal to noise ratio calculation algorithm (Intensity or Flux)
    • Implementation of the exportLines task
Deleted:
<
<
 

HiClass export tool

Changed:
<
<
>
>
  • Although this is not an HCSS development, it should be noted that GILDAS/Class now reads the HCSS-generated FITS without needing any prior conversion as it used to be the case using the HiClass task.
 

Documentation

For HCSS 14, many figures were refreshed in both manuals (HIFI Data Reduction Guide and HIFI Pipeline Specification).

Revision 352015-12-04 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 192 to 192
 

HRS Pipeline

Level 1 Pipeline

Level 2 Pipeline

Added:
>
>
  • For DBS Raster, Level 2 spectra are not averaged
 

Level 2.5 Pipeline

HIFI Products

Revision 342015-12-03 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 184 to 184
 

HIFI Pipeline

hifiPipeline task

Added:
>
>
 

Level 0 Pipeline

WBS Pipeline

HRS Pipeline

Line: 197 to 200
 
    • reorganisation of categories in the documentation (from class 1 to 3, by how severe the impact they may have on the science)
    • updates to the impact and action text following a legacy use
    • updates to the quality report, now including the text for public flags (impact and action)
Added:
>
>
  • Flags are now applied to OFF spectra as well
  • BrowseProduct: generation of a browse image for the reference spectra, when the option useReferenceSpectra is set to true
 

HIFI Spectrum Toolbox

Spectrum Explorer (HIFI specifics)

Line: 204 to 209
 

HIFI Data Processing Tools

Standing Wave Removal

Changed:
<
<
  • New parameter addMedianContinuum: allow the median continuum to be added back into the baseline fit flux
>
>
  • New parameter addMedianContinuum: allow the median continuum to be added back into the baseline fit flux [DEFAULT: False]
 

Baseline Removal

Changed:
<
<
  • New parameter addMedianContinuum: allow the median continuum to be added back into the baseline fit flux
>
>
  • New parameter addMedianContinuum: allow the median continuum to be added back into the baseline fit flux but only performs well for basemode='sub' and not basemode='div' [DEFAULT: False]
 

Deconvolution

  • The output deconvolution product ssb is now called dataset (please note that scripts prior to this change will break)

Revision 332015-12-03 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 192 to 192
 

Level 2.5 Pipeline

HIFI Products

Added:
>
>
  • New quality flags information:
    • updates of a few QFs names/descriptions (which were obsolete), deleting a few obsolete flags
    • reorganisation of categories in the documentation (from class 1 to 3, by how severe the impact they may have on the science)
    • updates to the impact and action text following a legacy use
    • updates to the quality report, now including the text for public flags (impact and action)
 

HIFI Spectrum Toolbox

Spectrum Explorer (HIFI specifics)

Revision 322015-12-02 - AlvarGarcia

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 258 to 260
 
  • Level 2/2.5 extracted spectra tables:
    • A new product is provided at Level 2 for all (line, range, chopNod, unchopped) pointed observations, and also at Level 2.5 for the subset of unchopped, rangeScan observations. This table, HPSSPEC[R|B], contains the extracted spectrum of the central spaxel, and the point source calibrated spectra "c1" and "c129" from the pipeline task extractCentralSpectrum. Since the point-source calibrated spectra are taken from the centre of the cube, it is necessary that the source is located within spaxel 2,2 (the central one). However, we make no determination that the target of the observation actually is a point source - that is up to the end-user to establish. These tables are also provided as standalone browse products.
  • Level 3 extracted spectra table:
Changed:
<
<
    • The new spectrum tables from Level 2 for the pointed, chopNod, full SED observations, are concatenated into a single table which is placed in Level 3. This table, HPSSPEC, contains the extracted spectra taken from all the observations that were taken to cover the full spectral range (the full SED) of PACS for that target (this will be usually two, sometimes three, separate observations). Moreoever, the red and blue camera, and all slices, are included this single table. This table is also provided as a standalone browse product.
>
>
    • The new spectrum tables from Level 2 for the pointed, chopNod, full SED observations, are concatenated into a single table which is placed in Level 3. This table, HPSSPEC, contains the extracted spectra for all the observations that were taken to cover the full spectral range (the full SED) of PACS for that target (this will be usually two, sometimes three, separate observations). Moreoever, the red and blue camera, and all slices, are included this single table. This table is also provided as a standalone browse product.
 
  • Interactive pipeline scripts:
    • The pipeline menu has been reworded, so that the names reflect better what the individual scripts do. The first script offered in each menu is also the preferred one, and that on which the SPG scripts are based.
    • A new pipeline script has been written to create drizzled cubes, based on (and almost hardly different to) the 'Telescope normalisation' pipeline script for chopNod line scans. With this script it is now possible to create drizzle cubes using this preferred pipeline method (in Track 13 it was not possible).

Revision 312015-12-02 - JeroenDeJong

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 27 to 27
 
  • Updates to high level products: Photometry level 2.5 and the creation of the Spectroscopy levels 2.5 and 3 (not really products but tables of spectra).
    • Photometer: Maps of solar system objects have been created for this version.
    • Spectrometer: Tables of spectra in level 2.5 and 3, as mentioned above.
Changed:
<
<
  • Unimap mapper has been upgraded to the latest version and there are new scripts making use of JScanam. There is also a script to generate drizzled cubes.
>
>
  • Unimap mapper has been upgraded to the latest version.
  • There are new scripts making use of JScanam. There is also a script to generate drizzled cubes.
 
  • New spectroscopy calibration for extended sources which also works for off-centre point sources. New beam calibration and background model.
  • New useful and analysis scripts to account for these calibration changes (beam and background related).
Added:
>
>
  • A new task runPacsSpg has been introduced, with which users can run the full PACS SPG pipeline from within HIPE.
  SPIRE
  • FTS
Line: 240 to 242
 

PACS

Pipeline

Added:
>
>
* A new task runPacsSpg has been introduced, with which a user can run a full SPG pipeline within HIPE, starting from the level-0 data in an observation context from the HSA, or even from the raw telemetry files when they are available.
 

Photometry

  • Level2.5 products:
    • Unimap and Jscanam maps are now generated for Solar System Objects, from combined scan and cross-scan observations.
Line: 262 to 267
 
  • SPG scripts:
    • The range scan observations are now flatfielded: using the spline fitting method for ranges longer than 5 microns, and using the line scan flatfielding for the shorter ranges
    • The interpolated cubes are no longer produced for oversample range scan mapping observations, and they also have a smaller spaxel size of 3" (rather than 4.7" before).
Added:
>
>
    • The interpolation algorithm in specInterpolate produces now NaN values in case one of the input spaxels has a NaN value, instead of producing skewed values due to a missing flux value. This results in more NaN values in the interpolated cubes, but avoids non-reliable values instead.
 

Calibration

Added:
>
>
 

Photometry

Added:
>
>
 

Spectroscopy

Added:
>
>
 
  • There is a new Extended Source Correction calibration file (ExtendedSourceLoss, used in the pipeline tasks specRespCal and specRespCalToTelescope) which produces a better spatial flatfield for the PACS spectrometer. The fluxes of the spaxels of all cubes will be increased by ~30% in the blue to a decrease of ~5% in the red. The calibration file can be inspected to extract the curve:

    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data

  • For point sources calibrated with extractCentralSpectrum, this extended source correction is taken out before the point source corrections are computed. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: to get this, raise a helpdesk ticket.
  • New Beam calibration files have been delivered, differing from the previous files only in the way they are normalised. These will be used in a new task being developed in HIPE 15, and do not impact on HIPE 14 or SPG 14 in any way.
Line: 271 to 281
 
  • A slightly updated telescope background model for use in the Pointing Offset Correction pipeline script for point sources has been developed. This update is to allow better consistency between the "POC" script and the default pipeline script "Telescope normaliation", for chopNod AOTs. The changes are at the percent level and largest in band B3A.

Analysis tools

Added:
>
>
 

Photometry

  • The PhotometryConvolveResolution Kernel script was added within the PACS Useful Script section. This script convolves and regrids a map at a certain wavelength to the PSF of another map at longer wavelength, by using an appropriate kernel.
Added:
>
>
 

Spectroscopy

Added:
>
>
 

Product changes

Added:
>
>
  • Many improvements to the descriptions and/or units of the header meta keywords.
  • All keywords have now a corresponding 8 character FITS keyword, so that the headers are easier to read with other software.
 

Photometry

  • The error layer within Unimap Level 2.5 products is renamed as stdDev, being the standard deviation computed at every sky pixel with respect to the samples that are falling on it. It can be properly assumed as the error map only if associated to the NaÔve map.
Added:
>
>
 

Spectroscopy

  • Interpolated cubes: Cubes produced by the task specInterpolate now have a spaxel size of 3".
  • Interpolated cubes: Are no longer provided for oversampled mapping observations.

Revision 302015-12-02 - JeroenDeJong

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 124 to 124
 

Display

  • Added shortcuts to some of the task directly in the image viewer.
Added:
>
>
  • A new viewer has been introduced to explore ParameterCube products which contain the results of the spectrum fitter on spectral cubes.
  • Several improvements to the cube tools of the spectrum explorer:
    • Tooltips added to task selection drop down menu.
    • Improved the spatial selection for the crop task.
    • Fixed some problems with displaying cubes when there is little space for the spectrum explorer, e.g. when it is embedded in the observation context viewer.
 

Analysis

Revision 292015-12-01 - AlvarGarcia

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 311 to 309
 

Documentation


Changed:
<
<
<!-- COMMENT BOX CODE - DO NOT EDIT -->
>
>
<-- COMMENT BOX CODE - DO NOT EDIT -->
 
Changed:
<
<
<script type="text/javascript">
var disqus_shortname = 'herscheltwiki'; // required: replace example with your forum shortname

(function() {
var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;
dsq.src = 'http://' + disqus_shortname + '.disqus.com/embed.js';
(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
})();
</script> <noscript>Please enable JavaScript to view the comments powered by Disqus.</noscript> blog comments powered by Disqus <!-- END OF COMMENT BOX CODE -->
>
>
blog comments powered by Disqus
<-- END OF COMMENT BOX CODE -->
 
Changed:
<
<
<!--
* Set ALLOWTOPICCHANGE = RegisteredUsersGroup, TWikiAdminGroup
-->
>
>
<--
* Set ALLOWTOPICCHANGE = RegisteredUsersGroup, TWikiAdminGroup
-->
  -- AlvarGarcia - 24 Nov 2014

Editors & Viewers

Revision 282015-12-01 - AlvarGarcia

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 10 to 10
 
Deleted:
<
<
 

Highlights

Changed:
<
<
HIPE
>
>
Core
  • Now you can pass arguments to scripts in batch mode (former jylaunch).
  • New Quick Analysis perspective allows GUI download of products and processing all within HIPE.
  • New product Auxiliary Observation Summary Product with information about the observation in a simple dataset.
  • Quality flags have been reorganised.
  HIFI
Added:
>
>
  • Changes to the output of the Deconvolution task.
  • New parameters for the *removal tasks and updated functionality for identifyLines.
  • Legacy quality documentation.
  PACS
Added:
>
>
  • Updates to high level products: Photometry level 2.5 and the creation of the Spectroscopy levels 2.5 and 3 (not really products but tables of spectra).
    • Photometer: Maps of solar system objects have been created for this version.
    • Spectrometer: Tables of spectra in level 2.5 and 3, as mentioned above.
  • Unimap mapper has been upgraded to the latest version and there are new scripts making use of JScanam. There is also a script to generate drizzled cubes.
  • New spectroscopy calibration for extended sources which also works for off-centre point sources. New beam calibration and background model.
  • New useful and analysis scripts to account for these calibration changes (beam and background related).
  SPIRE
  • FTS

Revision 272015-12-01 - AlvarGarcia

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
Changed:
<
<
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

>
>
<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->
 

What's coming up in HIPE 14

PDF Version Click here
Line: 10 to 10
 
Deleted:
<
<

Planned work

HIPE

* TBD

HIFI

* TBD

PACS

* TBD

SPIRE

* TBD

 

Highlights

Line: 281 to 266
 

Spectroscopy

  • Interpolated cubes: Cubes produced by the task specInterpolate now have a spaxel size of 3".
  • Interpolated cubes: Are no longer provided for oversampled mapping observations.
Changed:
<
<
  • Interpolated cubes: The way the task specInterpolate handles NaNs as it creates these cubes has changed. This will primarily affect cubes with saturated data. Saturated data are masked out and replaced with NaNs when rebinned cubes are built. Interpolated cubes are then created from the rebinned cubes. NaNs are now filtered out as the interpolation task does its work on the input cubes, and all affected and immediately surrounding spaxels of the output interpolated cube will also be NaN. Hence, at the affected wavelengths, the NaN area will be larger than before...but this is also more correct than before.
>
>
  • Interpolated cubes: The way the task specInterpolate handles NaNs as it creates these cubes has changed. This will primarily affect cubes with saturated data. Saturated data are masked out and replaced with NaNs when rebinned cubes are built. Interpolated cubes are then created from the rebinned cubes. NaNs are now filtered out as the interpolation task does its work on the input cubes, and all affected and immediately surrounding spaxels of the output interpolated cube will also be NaN. Hence, at the affected wavelengths, the NaN area will be larger than before...but this is also more correct than before.
 
  • Standalone browse products: we now provide the new extracted spectra tables, HPSSPEC[R|B] and HPSSPEC (see above), as SBPs.
  • Drizzle cubes: in SPG 13 the drizzle cube fluxes were incorrect. In SPG 14 they are now correct.
  • PacsCubes and PacsRebinnedCubes created by the pipeline (HPS3D [R|B] and HPS3DR [R|B]) will now display in cube viewers with a flip over the diagonal, as compared to previously. The new orientation makes it easier to compare the cubes in cube viewers (the Standard Cube Viewer or the Spectrum Explorer) to the display of the PacsSpectralFootprint viewer.

Revision 262015-11-27 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 195 to 195
 

HIFI Data Processing Tools

Standing Wave Removal

Added:
>
>
  • New parameter addMedianContinuum: allow the median continuum to be added back into the baseline fit flux
 

Baseline Removal

Added:
>
>
  • New parameter addMedianContinuum: allow the median continuum to be added back into the baseline fit flux
 

Deconvolution

Added:
>
>
  • The output deconvolution product ssb is now called dataset (please note that scripts prior to this change will break)
 

Spectral Cube creation

flagTool

identifyLines

Line: 206 to 211
 
    • Changes on the Herschel Spectral Line List columns names
    • Implementation of the image band line identification for pointed and mapping observations
    • Fix to the rms computation
Changed:
<
<
    • Implementation of the exportLines task
>
>
    • Parameters change: toleranceEmission and toleranceAbsorption has become toleranceFactorEmi and toleranceFactorAbs (may break scripts prior to this change)
    • New parameter snrType: the signal to noise ratio calculation algorithm (Intensity or Flux)
    • Implementation of the exportLines task
 

HiClass export tool

Line: 217 to 224
 
    • Updates to the chapters Standing wave Removal and HIFI Baseline Removal : adding definition of the new parameter addMedianContinuum
    • Updates to the chapter Flags in HIFI data, section Quality Flags (this chapter is being revamped, please check regularly)
    • Updates to the chapter How to make a spectral cube: added a definition for parameters ignoreOffs, outputNames and extrapolate
Changed:
<
<
    • Updates to the chapter Sideband Deconvolution: the output deconvolution product ssb is now called dataset (please note that scripts prior to this change will break)
>
>
    • Updates to the chapter Sideband Deconvolution: the output deconvolution product ssb is now called dataset (please note that scripts prior to this change will break)
 
    • Updates to the chapter Standing wave Removal, ESW correction in HEB bands section: update to the scripts in the demonstration subsection
    • Updates to the chapter How to add and remove flags, section Setting and Clearing Flags with command line tools: improved the definition of the parameter selection for Level 2.5 cubes
    • A new chapter called Understanding the uncertainty table information in your data has been added

Revision 252015-11-27 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 260 to 258
 

Calibration

Photometry

Spectroscopy

Added:
>
>
  • There is a new Extended Source Correction calibration file (ExtendedSourceLoss, used in the pipeline tasks specRespCal and specRespCalToTelescope) which produces a better spatial flatfield for the PACS spectrometer. The fluxes of the spaxels of all cubes will be increased by ~30% in the blue to a decrease of ~5% in the red. The calibration file can be inspected to extract the curve:

    w = calTree.spectrometer.extendedSourceLoss["wavelength"].data
    f = calTree.spectrometer.extendedSourceLoss["fraction"].data

  • For point sources calibrated with extractCentralSpectrum, this extended source correction is taken out before the point source corrections are computed. However, if working with off-centred point sources and using the tasks extractSpaxelSpectrum and pointSourceLossCorrection, this correction is not taken out, and hence the results will be incorrect. An extra task/function is necessary: to get this, raise a helpdesk ticket.
  • New Beam calibration files have been delivered, differing from the previous files only in the way they are normalised. These will be used in a new task being developed in HIPE 15, and do not impact on HIPE 14 or SPG 14 in any way.
  • A slightly updated telescope background model for use in the Pointing Offset Correction pipeline script for point sources has been developed. This update is to allow better consistency between the "POC" script and the default pipeline script "Telescope normaliation", for chopNod AOTs. The changes are at the percent level and largest in band B3A.
 

Analysis tools

Photometry

  • The PhotometryConvolveResolution Kernel script was added within the PACS Useful Script section. This script convolves and regrids a map at a certain wavelength to the PSF of another map at longer wavelength, by using an appropriate kernel.

Spectroscopy

Added:
>
>
 

Product changes

Photometry

  • The error layer within Unimap Level 2.5 products is renamed as stdDev, being the standard deviation computed at every sky pixel with respect to the samples that are falling on it. It can be properly assumed as the error map only if associated to the NaÔve map.

Spectroscopy

Changed:
<
<
  • Interpolated cubes: Cubes produced by the task specInterpolate now have a spaxel size of 3"
>
>
  • Interpolated cubes: Cubes produced by the task specInterpolate now have a spaxel size of 3".
  • Interpolated cubes: Are no longer provided for oversampled mapping observations.
  • Interpolated cubes: The way the task specInterpolate handles NaNs as it creates these cubes has changed. This will primarily affect cubes with saturated data. Saturated data are masked out and replaced with NaNs when rebinned cubes are built. Interpolated cubes are then created from the rebinned cubes. NaNs are now filtered out as the interpolation task does its work on the input cubes, and all affected and immediately surrounding spaxels of the output interpolated cube will also be NaN. Hence, at the affected wavelengths, the NaN area will be larger than before...but this is also more correct than before.
 
  • Standalone browse products: we now provide the new extracted spectra tables, HPSSPEC[R|B] and HPSSPEC (see above), as SBPs.
Added:
>
>
  • Drizzle cubes: in SPG 13 the drizzle cube fluxes were incorrect. In SPG 14 they are now correct.
 
  • PacsCubes and PacsRebinnedCubes created by the pipeline (HPS3D[R|B] and HPS3DR[R|B]) will now display in cube viewers with a flip over the diagonal, as compared to previously. The new orientation makes it easier to compare the cubes in cube viewers (the Standard Cube Viewer or the Spectrum Explorer) to the display of the PacsSpectralFootprint viewer.

Documentation

Revision 242015-11-26 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 201 to 201
 

Deconvolution

Spectral Cube creation

flagTool

Added:
>
>

identifyLines

  • Updates to the task include:
    • Implementation of threading to speed up the task
    • Improvements of the rejection of false detected lines
    • Changes on the Herschel Spectral Line List columns names
    • Implementation of the image band line identification for pointed and mapping observations
    • Fix to the rms computation
    • Implementation of the exportLines task
 

HiClass export tool

Revision 232015-11-25 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 79 to 79
  would open HIPE with three viewers: one for the image, a spectrum explorer for the cube, and a dataset viewer for the CSV table.
Changed:
<
<
ALERT! Note:
>
>

ALERT! Note:
  Passing a Jython script as the first argument will still behave as running HIPE in batch mode:
Line: 91 to 91
 There are interesting enhancements in the Console view with HIPE 14.0:

  • Text edit options (cut/copy/paste) are available not only by the usual key combinations, but also with the popup menu.
  • Autocompletion through the tab key has been tremendously improved in this release, making Jython coder's life much easier.
Changed:
<
<
  • The console built-in help, shown with !help, is now more practical and understandable.
>
>
  • The console built-in help, shown with help, is now more practical and understandable.
 

Plotting

TablePlotter

OverPlotter

Line: 160 to 160
  Improvement of readability of quality flags in the meta data of products in FITS files. Now one meta data item per quality flag making proper use of parameter values and descriptions.
Changed:
<
<
eg.) qflag_PARXXXXX_p <value> ⁄ <description>
>
>
eg.) qflag_PARXXXXX_p <value> ⁄ <description>
 

Systematic product generation

Line: 230 to 229
 

Pipeline

Photometry

  • Level2.5 products:
Changed:
<
<
    • Unimap and Jscanam maps are generated for Solar System Objects, by combining scan and cross-scan observations.
    • A block detection and masking method is implemented to avoid strong signal gradients generated by the calibration blocks. The quality of Level2.5 is enhanced in particular for those observations acquired at the early stage of the mission (e.g. Science Demonstration Phase) where calibration blocks were acquired in the middle of the scans.
  • ipipe script:
    • The JScanam ipipe script scanmap_JScanam_multiplePairs is designed to process multiple pairs of obsIDs and it is upgraded by introducing the scanamorphosAlignFrames Task. This Task aligns the Frames signal levels, belonging to every observation, to a common offset level, by exploiting the overlapping sky regions. This new capability is particulary useful to recover a common background level over observations that are partially or slightly overlapping.
    • the script scanmap_Unimap_memory_optimized is added within the Pipelines menu for allowing to run the Unimap release located on the userís computer by using the minimum amount of memory. It is particularly useful for processing large observations or/and if several obsIDs should be processed all together in a single run.
    • the photextended_pipeline_L3 script is available within the Pipelines menu to generate mosaics of Unimpa and Jscanam Level2.5 maps
  • Unimap : Unimap 6.2.0 is spawned by the Standard Product Generator (SPG) to create Level2.5 products. See the Photometer Data Reduction Guide and the offical Unimap webpages (http://infocom.uniroma1.it/unimap/) to know what's new about the 6.2.0 Unimap release.
>
>
    • Unimap and Jscanam maps are now generated for Solar System Objects, from combined scan and cross-scan observations.
    • A block detection and masking method has been implemented, this avoids strong signal gradients being generated by the calibration blocks. The quality of Level2.5 maps is enhanced in particular for those observations acquired at the early stage of the mission (e.g. Science Demonstration Phase), when the calibration blocks were acquired in the middle of the an observation.
  • Interactive pipeline scripts:
    • The JScanam ipipe script scanmap_JScanam_multiplePairs is designed to process multiple pairs of obsIDs and it has been upgraded by introducing the scanamorphosAlignFrames Task. This task aligns the Frames signal levels, belonging to every observation, to a common offset level, by exploiting the overlapping sky regions. This new capability is particulary useful for recovering a common background level over observations that are partially or slightly overlapping.
    • The script scanmap_Unimap_memory_optimized has been added to the Pipelines menu. This allows one to run the Unimap release located on the userís computer while using the minimum amount of memory. It is particularly useful for processing large observations or/and if several obsIDs should be processed all together in a single run.
    • The script photextended_pipeline_L3 is now available in the Pipelines menu. This can be used to generate mosaics of Unimap and Jscanam Level2.5 maps.
  • Unimap : Unimap 6.2.0 is spawned by the Standard Product Generator (SPG) to create Level2.5 products. See the Photometer Data Reduction Guide and the offical Unimap webpages (http://infocom.uniroma1.it/unimap/) to know What's New about the 6.2.0 Unimap release.

Spectroscopy

  • Level 2/2.5 extracted spectra tables:
    • A new product is provided at Level 2 for all (line, range, chopNod, unchopped) pointed observations, and also at Level 2.5 for the subset of unchopped, rangeScan observations. This table, HPSSPEC[R|B], contains the extracted spectrum of the central spaxel, and the point source calibrated spectra "c1" and "c129" from the pipeline task extractCentralSpectrum. Since the point-source calibrated spectra are taken from the centre of the cube, it is necessary that the source is located within spaxel 2,2 (the central one). However, we make no determination that the target of the observation actually is a point source - that is up to the end-user to establish. These tables are also provided as standalone browse products.
  • Level 3 extracted spectra table:
    • The new spectrum tables from Level 2 for the pointed, chopNod, full SED observations, are concatenated into a single table which is placed in Level 3. This table, HPSSPEC, contains the extracted spectra taken from all the observations that were taken to cover the full spectral range (the full SED) of PACS for that target (this will be usually two, sometimes three, separate observations). Moreoever, the red and blue camera, and all slices, are included this single table. This table is also provided as a standalone browse product.
  • Interactive pipeline scripts:
    • The pipeline menu has been reworded, so that the names reflect better what the individual scripts do. The first script offered in each menu is also the preferred one, and that on which the SPG scripts are based.
    • A new pipeline script has been written to create drizzled cubes, based on (and almost hardly different to) the 'Telescope normalisation' pipeline script for chopNod line scans. With this script it is now possible to create drizzle cubes using this preferred pipeline method (in Track 13 it was not possible).
    • There have been small, but potentially significant changes to some of the tasks in the 'Pointing offset correction' scripts (chopNod) and the '...with transients" scripts (unchopped). These are, respectively, to the tasks specDetermineChiSquare and specLongTermTransient. The improvements are "under the hood" so there is no change in the user scripts themselves.
    • The range scan flatfielding task now incorporates a spline fitting to the continuum of the spectra. This produces the same results for shorter ranges but improved results for longer ranges (especially SEDs) and where there are strong curvatures in the spectra.
  • SPG scripts:
    • The range scan observations are now flatfielded: using the spline fitting method for ranges longer than 5 microns, and using the line scan flatfielding for the shorter ranges
    • The interpolated cubes are no longer produced for oversample range scan mapping observations, and they also have a smaller spaxel size of 3" (rather than 4.7" before).
 

Calibration

Photometry

Spectroscopy

Line: 251 to 261
 

Photometry

  • The error layer within Unimap Level 2.5 products is renamed as stdDev, being the standard deviation computed at every sky pixel with respect to the samples that are falling on it. It can be properly assumed as the error map only if associated to the NaÔve map.

Spectroscopy

Changed:
<
<
>
>
  • Interpolated cubes: Cubes produced by the task specInterpolate now have a spaxel size of 3"
  • Standalone browse products: we now provide the new extracted spectra tables, HPSSPEC[R|B] and HPSSPEC (see above), as SBPs.
  • PacsCubes and PacsRebinnedCubes created by the pipeline (HPS3D[R|B] and HPS3DR[R|B]) will now display in cube viewers with a flip over the diagonal, as compared to previously. The new orientation makes it easier to compare the cubes in cube viewers (the Standard Cube Viewer or the Spectrum Explorer) to the display of the PacsSpectralFootprint viewer.
 

Documentation

SPIRE

Revision 222015-11-23 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 216 to 216
 
    • Updates to the chapter Standing wave Removal, ESW correction in HEB bands section: update to the scripts in the demonstration subsection
    • Updates to the chapter How to add and remove flags, section Setting and Clearing Flags with command line tools: improved the definition of the parameter selection for Level 2.5 cubes
    • A new chapter called Understanding the uncertainty table information in your data has been added
Added:
>
>
    • Updates to the chapter The HIFI line identification tool, adding a section called The exportLines Task
 
  • Updates to the HIFI Pipeline Specification manual
    • This manual has been restructured
    • Removal of the task mkSpur
    • Corrections to broken links pointing to the HIFI Data Reduction Guide
    • Correction due to a missing figure Level 0 Pipeline flow diagram
    • Added text to tasks which did not have text, or were poorly documented, including mkRms
Added:
>
>
    • Updates to several tasks on flags information
 

PACS

Revision 212015-11-12 - RosalindHopwood

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 159 to 161
 Improvement of readability of quality flags in the meta data of products in FITS files. Now one meta data item per quality flag making proper use of parameter values and descriptions.

eg.)

Changed:
<
<
qflag_PARXXXXX_p <value> ⁄ <description>
>
>
qflag_PARXXXXX_p <value> ⁄ <description>
 

Systematic product generation

Line: 272 to 274
 
  • A new task has been added to all pipelines that applies the extended calibration correction in order to get better agreement between FTS and Phototmeter photometry.

Interactive Analysis and Tools

Added:
>
>
  • There is a new useful script, Spectrometer Pointing Offset Corrector, for estimating and correcting pointing offset in sparse point observations. The scripts assesses the difference between the SLW and SSW spectra of the centre detectors, over the overlapping frequency range, uses this to estimate any pointing offset and corrects the data using SECT.
 

Documentation


Revision 202015-11-04 - JonathanCook

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 152 to 152
 

Quality control

Added:
>
>
Reorganization of quality flags as private or public for HCSS, HIFI, PACS and SPIRE Quality Flags. This ensures that only the flags useful for the assessment of the science by the user (the public ones) make its way to the quality summary.All the information from the quality flags remain at the quality product, for internal assessment or history.

In the quality report, a new "Ōmpact" column in the "Quality flags" table, after "Concept" and "Value" columns has been added. This new column is an automatically filled column with a brief text describing the possible consequences for science data from the situations that caused that quality flag to be raised.

Improvement of readability of quality flags in the meta data of products in FITS files. Now one meta data item per quality flag making proper use of parameter values and descriptions.

eg.) qflag_PARXXXXX_p <value> ⁄ <description>

 

Systematic product generation

Data input-output

Line: 160 to 169
 

Installer

Added:
>
>
The user installer has been reduced by 400-800MB by removing the pointing distortion maps previously included with the data a part of user installation.
 

Documentation

HIFI

Revision 192015-10-28 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 125 to 123
 

Analysis

Data cubes

Changed:
<
<
* When requesting the unit, the FITS compatible unit is returned. * The naxis keywords are not longer duplicated in the fits header. * The Wcs ia added to all datasets of the images (like the stddev, ...).
>
>
  • When requesting the unit, the FITS compatible unit is returned.
  • The naxis keywords are not longer duplicated in the fits header.
  • The Wcs ia added to all datasets of the images (like the stddev, ...).
 

Display

Deleted:
<
<
 * Added shortcuts to some of the task directly in the image viewer.

Analysis

Revision 182015-10-28 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 109 to 109
 

Other functions

Source extraction

Changed:
<
<
>
>
  • Improved error reporting for bad inputs.
 

Images

Changed:
<
<
* When requesting the unit, the FITS compatible unit is returned. * The naxis keywords are not longer duplicated in the fits header. * The Wcs ia added to all datasets of the images (like the stddev, ...).
>
>
  • When requesting the unit, the FITS compatible unit is returned.
  • The naxis keywords are not longer duplicated in the fits header.
  • The Wcs ia added to all datasets of the images (like the stddev, ...).
 

Display

Deleted:
<
<
 * Added shortcuts to some of the task directly in the image viewer.

Analysis

Revision 172015-10-26 - JavierDiaz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 148 to 150
 

Tasks

Added:
>
>
Improved history recording so that it can be added to the sub-products of a context.
 

Quality control

Systematic product generation

Revision 162015-10-23 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 54 to 54
 Recent open files

The size of the recently opened files list, which can be accessed through the File menu, is now configurable through preferences.

Added:
>
>
Metadata tables

A new option in the popup menu of metadata tables allows copying cell values in other way than with the usual key combinations (Ctrl+C or Cmd+C).

Array data viewer

The array data viewer shows tooltips over the spinners (above the data table) for informing about the array dimensions.

 

Quick Analysis

There is a new perspective in HIPE, which is accessible by clicking on the magnifying glass at the top right toolbar, for quickly inspecting scientific data from the Herschel Science Archive.

Line: 65 to 73
 Opening the observation context instead is also possible, either with the popup menu or by configuring so as the default behaviour, in the Quick Analysis preferences category.

Command line

Changed:
<
<
Starting with version 14.0, it is possible to pass arguments to HIPE from the command line, which will try to interpret them as files to be opened with the proper viewers. For example:
>
>
It is now possible to pass arguments to HIPE from the command line, which will be tried to be interpreted as files to be opened with the proper viewers. For example:
 
bash> hipe myImage.jpg cube.fits resultTable.csv

would open HIPE with three viewers: one for the image, a spectrum explorer for the cube, and a dataset viewer for the CSV table.

Revision 152015-10-08 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 76 to 76
 
bash> hipe myScript.py arg1 arg2

would not open the HIPE window, but execute the script with the given arguments passed to it.

Added:
>
>

Console

There are interesting enhancements in the Console view with HIPE 14.0:

  • Text edit options (cut/copy/paste) are available not only by the usual key combinations, but also with the popup menu.
  • Autocompletion through the tab key has been tremendously improved in this release, making Jython coder's life much easier.
  • The console built-in help, shown with !help, is now more practical and understandable.
 

Plotting

TablePlotter

OverPlotter

Revision 142015-10-06 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 63 to 63
 When clicking on an observation, its standalone browse products are opened in the editor area for quick analysis, by means of the image, spectra or cube viewers.

Opening the observation context instead is also possible, either with the popup menu or by configuring so as the default behaviour, in the Quick Analysis preferences category.

Added:
>
>

Command line

Starting with version 14.0, it is possible to pass arguments to HIPE from the command line, which will try to interpret them as files to be opened with the proper viewers. For example:

bash> hipe myImage.jpg cube.fits resultTable.csv

would open HIPE with three viewers: one for the image, a spectrum explorer for the cube, and a dataset viewer for the CSV table.

ALERT! Note:

Passing a Jython script as the first argument will still behave as running HIPE in batch mode:

bash> hipe myScript.py arg1 arg2

would not open the HIPE window, but execute the script with the given arguments passed to it.

 

Plotting

TablePlotter

OverPlotter

Revision 132015-10-06 - AlvarGarcia

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 8 to 8
 
Help Interested in what's new for other versions of HIPE? See this page for links to all the What's New documents from HIPE 4.0 onwards.
See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.
Changed:
<
<

>
>

 

Planned work

Revision 122015-10-05 - JaimeSaiz

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 43 to 43
 

Core system

HIPE

Added:
>
>

Editors & Viewers

 
Added:
>
>
Default editor

The default editor for every type of data, which is the one used when double clicking on a variable, is preserved regardless the editors opened through the Open With menu.

The previous behaviour, which is to use the last opened editor as the default, can still be set, by means of a new preference in the Editors & Viewers category.

Recent open files

The size of the recently opened files list, which can be accessed through the File menu, is now configurable through preferences.

Quick Analysis

There is a new perspective in HIPE, which is accessible by clicking on the magnifying glass at the top right toolbar, for quickly inspecting scientific data from the Herschel Science Archive.

Observations can be searched by target name, observation id, operational day or Ra/Dec.

When clicking on an observation, its standalone browse products are opened in the editor area for quick analysis, by means of the image, spectra or cube viewers.

Opening the observation context instead is also possible, either with the popup menu or by configuring so as the default behaviour, in the Quick Analysis preferences category.

 

Plotting

TablePlotter

OverPlotter

Line: 170 to 189
 
    • Unimap and Jscanam maps are generated for Solar System Objects, by combining scan and cross-scan observations.
    • A block detection and masking method is implemented to avoid strong signal gradients generated by the calibration blocks. The quality of Level2.5 is enhanced in particular for those observations acquired at the early stage of the mission (e.g. Science Demonstration Phase) where calibration blocks were acquired in the middle of the scans.
  • ipipe script:
Changed:
<
<
    • The JScanam ipipe script scanmap_JScanam_multiplePairs is designed to process multiple pairs of obsIDs and it is upgraded by introducing the scanamorphosAlignFrames Task. This Task aligns the Frames signal levels, belonging to every observation, to a common offset level, by exploiting the overlapping sky regions. This new capability is particulary useful to recover a common background level over observations that are partially or slightly overlapping.
>
>
    • The JScanam ipipe script scanmap_JScanam_multiplePairs is designed to process multiple pairs of obsIDs and it is upgraded by introducing the scanamorphosAlignFrames Task. This Task aligns the Frames signal levels, belonging to every observation, to a common offset level, by exploiting the overlapping sky regions. This new capability is particulary useful to recover a common background level over observations that are partially or slightly overlapping.
 
    • the script scanmap_Unimap_memory_optimized is added within the Pipelines menu for allowing to run the Unimap release located on the userís computer by using the minimum amount of memory. It is particularly useful for processing large observations or/and if several obsIDs should be processed all together in a single run.
    • the photextended_pipeline_L3 script is available within the Pipelines menu to generate mosaics of Unimpa and Jscanam Level2.5 maps
  • Unimap : Unimap 6.2.0 is spawned by the Standard Product Generator (SPG) to create Level2.5 products. See the Photometer Data Reduction Guide and the offical Unimap webpages (http://infocom.uniroma1.it/unimap/) to know what's new about the 6.2.0 Unimap release.
Line: 227 to 246
 <!--
* Set ALLOWTOPICCHANGE = RegisteredUsersGroup, TWikiAdminGroup
-->

-- AlvarGarcia - 24 Nov 2014

Added:
>
>

Editors & Viewers

 
META FILEATTACHMENT attachment="GoogleAnalytics.txt" attr="h" comment="Google Analytics configuration file" date="1416843301" name="GoogleAnalytics.txt" path="GoogleAnalytics.txt" size="549" stream="GoogleAnalytics.txt" user="Main.AlvarGarcia" version="1"

Revision 112015-09-30 - LucaCalzoletti

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

Line: 166 to 166
 

Pipeline

Photometry

Deleted:
<
<
  • Unimap: Unimap 6.2.0 is spawned by the Standard Product Generator (SPG) to create Level2.5 products. See the Photometer Data Reduction Guide and the offical Unimap webpages (http://infocom.uniroma1.it/unimap/) to know what's new about the 6.2.0 Unimap release.
 
  • Level2.5 products:
    • Unimap and Jscanam maps are generated for Solar System Objects, by combining scan and cross-scan observations.
Changed:
<
<
    • A block detection and masking method is implemented to avoid strong signal gradients generated by calibration blocks. The Quality of Level2.5 is enhanced in particular for observations acquired at the early stage of the mission (e.g. Science Demonstration Phase) where calibration blocks were acquired in the middle of the scans
>
>
    • A block detection and masking method is implemented to avoid strong signal gradients generated by the calibration blocks. The quality of Level2.5 is enhanced in particular for those observations acquired at the early stage of the mission (e.g. Science Demonstration Phase) where calibration blocks were acquired in the middle of the scans.
 
  • ipipe script:
Changed:
<
<
    • the script scanmap_Unimap_memory_optimized is added within the Pipelines menu for allowing to run the Unimap release located on the userís computer by using the minimum amount of memory. It is particularly useful for processing large observations or/and if several obsIDs want to be processed all together in a single run.
>
>
    • The JScanam ipipe script scanmap_JScanam_multiplePairs is designed to process multiple pairs of obsIDs and it is upgraded by introducing the scanamorphosAlignFrames Task. This Task aligns the Frames signal levels, belonging to every observation, to a common offset level, by exploiting the overlapping sky regions. This new capability is particulary useful to recover a common background level over observations that are partially or slightly overlapping.
    • the script scanmap_Unimap_memory_optimized is added within the Pipelines menu for allowing to run the Unimap release located on the userís computer by using the minimum amount of memory. It is particularly useful for processing large observations or/and if several obsIDs should be processed all together in a single run.
 
    • the photextended_pipeline_L3 script is available within the Pipelines menu to generate mosaics of Unimpa and Jscanam Level2.5 maps
Added:
>
>
  • Unimap : Unimap 6.2.0 is spawned by the Standard Product Generator (SPG) to create Level2.5 products. See the Photometer Data Reduction Guide and the offical Unimap webpages (http://infocom.uniroma1.it/unimap/) to know what's new about the 6.2.0 Unimap release.
 

Calibration

Photometry

Revision 102015-09-30 - LucaCalzoletti

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
Changed:
<
<
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

>
>
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

 

What's coming up in HIPE 14

PDF Version Click here
Changed:
<
<
Help Interested in what's new for other versions of HIPE? See this page for links to all the What's New documents from HIPE 4.0 onwards.
>
>
Help Interested in what's new for other versions of HIPE? See this page for links to all the What's New documents from HIPE 4.0 onwards.
 See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.
Changed:
<
<

>
>

 

Planned work

Line: 166 to 166
 

Pipeline

Photometry

Changed:
<
<
  • Unimap: Unimap 6.2.0 is spawned by the Standard Product Generator (SPG) to create Level2.5 products. See the Photometer Data Reduction Guide and the offical Unimap webpages (http://infocom.uniroma1.it/unimap/) to see what's new about the 6.2.0 Unimap release. A method to avoid the drifts generated by calibration blocks is implemented into the Unimap pipeline within SPG13, enhancing the quality of the final maps in particular for observations (e.g. within the Science Demontsation Phase) were calibration blocks were acquired also in the middle of the scans.
>
>
  • Unimap: Unimap 6.2.0 is spawned by the Standard Product Generator (SPG) to create Level2.5 products. See the Photometer Data Reduction Guide and the offical Unimap webpages (http://infocom.uniroma1.it/unimap/) to know what's new about the 6.2.0 Unimap release.
 
  • Level2.5 products:
Changed:
<
<
    • Unimap and Jscanam maps are generated for Solar System Objects, by combining the scan and cross-scan observations.

  • ipipe script: the script scanmap_Unimap_memory_optimized is added within the interactive script for allowing to run the Unimap release located on the userís computer by using the minimum amount of memory. It is particularly useful for processing large observations or/and when several obsIDs want to be processed all together in a single run.
>
>
    • Unimap and Jscanam maps are generated for Solar System Objects, by combining scan and cross-scan observations.
    • A block detection and masking method is implemented to avoid strong signal gradients generated by calibration blocks. The Quality of Level2.5 is enhanced in particular for observations acquired at the early stage of the mission (e.g. Science Demonstration Phase) where calibration blocks were acquired in the middle of the scans
  • ipipe script:
    • the script scanmap_Unimap_memory_optimized is added within the Pipelines menu for allowing to run the Unimap release located on the userís computer by using the minimum amount of memory. It is particularly useful for processing large observations or/and if several obsIDs want to be processed all together in a single run.
    • the photextended_pipeline_L3 script is available within the Pipelines menu to generate mosaics of Unimpa and Jscanam Level2.5 maps
 

Calibration

Photometry

Line: 218 to 219
 <!-- COMMENT BOX CODE - DO NOT EDIT -->
Changed:
<
<
<script type="text/javascript">
var disqus_shortname = 'herscheltwiki'; // required: replace example with your forum shortname

(function() {
var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;
dsq.src = 'http://' + disqus_shortname + '.disqus.com/embed.js';
(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
})();
</script> <noscript>Please enable JavaScript to view the comments powered by Disqus.</noscript> blog comments powered by Disqus <!-- END OF COMMENT BOX CODE -->
>
>
<script type="text/javascript">
var disqus_shortname = 'herscheltwiki'; // required: replace example with your forum shortname

(function() {
var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;
dsq.src = 'http://' + disqus_shortname + '.disqus.com/embed.js';
(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
})();
</script> <noscript>Please enable JavaScript to view the comments powered by Disqus.</noscript> blog comments powered by Disqus <!-- END OF COMMENT BOX CODE -->
 
Changed:
<
<
<!--
* Set ALLOWTOPICCHANGE = RegisteredUsersGroup, TWikiAdminGroup
-->
>
>
<!--
* Set ALLOWTOPICCHANGE = RegisteredUsersGroup, TWikiAdminGroup
-->
  -- AlvarGarcia - 24 Nov 2014

Revision 92015-09-29 - LucaCalzoletti

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
Changed:
<
<
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->
>
>
<!-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<!-- END OF ANALYTICS CODE -->

 

What's coming up in HIPE 14

Changed:
<
<
PDF Version Click here

Help Interested in what's new for other versions of HIPE? See this page for links to all the What's New documents from HIPE 4.0 onwards.
>
>
PDF Version Click here
 
Added:
>
>
Help Interested in what's new for other versions of HIPE? See this page for links to all the What's New documents from HIPE 4.0 onwards.
 See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.
Changed:
<
<
>
>

 

Planned work

Line: 15 to 12
 

Planned work

Deleted:
<
<
 HIPE
Changed:
<
<
† †* TBD
>
>
* TBD
  HIFI
Changed:
<
<
† †* TBD
>
>
* TBD
  PACS
Changed:
<
<
†* TBD
>
>
* TBD
  SPIRE
Changed:
<
<
† †* TBD
>
>
* TBD
 

Highlights

Line: 69 to 64
 

Images

Changed:
<
<
* When requesting the unit, the FITS compatible unit is returned. * The naxis keywords are not longer duplicated in the fits header. * The Wcs ia added to all datasets of the images (like the stddev, ...).
>
>
* When requesting the unit, the FITS compatible unit is returned. * The naxis keywords are not longer duplicated in the fits header. * The Wcs ia added to all datasets of the images (like the stddev, ...).
 

Display

Line: 85 to 78
 

Data cubes

Changed:
<
<
* When requesting the unit, the FITS compatible unit is returned. * The naxis keywords are not longer duplicated in the fits header. * The Wcs ia added to all datasets of the images (like the stddev, ...).
>
>
* When requesting the unit, the FITS compatible unit is returned. * The naxis keywords are not longer duplicated in the fits header. * The Wcs ia added to all datasets of the images (like the stddev, ...).
 

Display

Line: 101 to 92
 

Products and datasets

Changed:
<
<
A new Auxiliary Observation Summary Product which can be retrieved using the tag LATEST_OBSERVATION_SUMMARY_PRODUCT. This product contains one dataset with a summary of information for each observation. Useful methods exist for querying the data as documented here: ObservationSummaryProduct
>
>
A new Auxiliary Observation Summary Product which can be retrieved using the tag LATEST_OBSERVATION_SUMMARY_PRODUCT. This product contains one dataset with a summary of information for each observation. Useful methods exist for querying the data as documented here: ObservationSummaryProduct
 

Observation context

Changed:
<
<
A new Calibration Table Product exists under the Auxiliary Context in each Observation Context. This product contains calibration tables data for the given instrument including exactly how the hardware was configured, hard limits, soft limits etc. This provides valuable information for the long term completeness of an observation. Useful methods exist for querying the table data as documented here: SimpleCalTableProduct
>
>
A new Calibration Table Product exists under the Auxiliary Context in each Observation Context. This product contains calibration tables data for the given instrument including exactly how the hardware was configured, hard limits, soft limits etc. This provides valuable information for the long term completeness of an observation. Useful methods exist for querying the table data as documented here: SimpleCalTableProduct
 

Product Access Layer

Line: 176 to 166
 

Pipeline

Photometry

Changed:
<
<

Spectroscopy

>
>
  • Unimap: Unimap 6.2.0 is spawned by the Standard Product Generator (SPG) to create Level2.5 products. See the Photometer Data Reduction Guide and the offical Unimap webpages (http://infocom.uniroma1.it/unimap/) to see what's new about the 6.2.0 Unimap release. A method to avoid the drifts generated by calibration blocks is implemented into the Unimap pipeline within SPG13, enhancing the quality of the final maps in particular for observations (e.g. within the Science Demontsation Phase) were calibration blocks were acquired also in the middle of the scans.
  • Level2.5 products:
    • Unimap and Jscanam maps are generated for Solar System Objects, by combining the scan and cross-scan observations.

  • ipipe script: the script scanmap_Unimap_memory_optimized is added within the interactive script for allowing to run the Unimap release located on the userís computer by using the minimum amount of memory. It is particularly useful for processing large observations or/and when several obsIDs want to be processed all together in a single run.
 

Calibration

Photometry

Line: 184 to 179
 

Analysis tools

Photometry

Added:
>
>
  • The PhotometryConvolveResolution Kernel script was added within the PACS Useful Script section. This script convolves and regrids a map at a certain wavelength to the PSF of another map at longer wavelength, by using an appropriate kernel.
 

Spectroscopy

Product changes

Photometry

Added:
>
>
  • The error layer within Unimap Level 2.5 products is renamed as stdDev, being the standard deviation computed at every sky pixel with respect to the samples that are falling on it. It can be properly assumed as the error map only if associated to the NaÔve map.
 

Spectroscopy

Documentation

Line: 216 to 213
 

Interactive Analysis and Tools

Documentation

Added:
>
>

 
Changed:
<
<

<-- COMMENT BOX CODE - DO NOT EDIT -->
>
>
<!-- COMMENT BOX CODE - DO NOT EDIT -->
 
Deleted:
<
<
blog comments powered by Disqus
<-- END OF COMMENT BOX CODE -->

<--  
-->
>
>
<script type="text/javascript">
var disqus_shortname = 'herscheltwiki'; // required: replace example with your forum shortname

(function() {
var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;
dsq.src = 'http://' + disqus_shortname + '.disqus.com/embed.js';
(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
})();
</script> <noscript>Please enable JavaScript to view the comments powered by Disqus.</noscript> blog comments powered by Disqus <!-- END OF COMMENT BOX CODE -->
 
Added:
>
>

 
Added:
>
>
<!--
* Set ALLOWTOPICCHANGE = RegisteredUsersGroup, TWikiAdminGroup
-->
  -- AlvarGarcia - 24 Nov 2014

Revision 82015-09-22 - TrevorFulton

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 42 to 42
 PACS

SPIRE

Added:
>
>
  • FTS
    • Two new tasks and corresponding calibration products have been introduced for SPIRE FTS data. One task corrects for artefacts in low resolution point source calibrated spectra. The other corrects the level of all extended source calibrated data in or to get better agreement between the SPIRE Spectrometer and Photometer.
 

Core system

Revision 72015-09-22 - TrevorFulton

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 193 to 193
 

SPIRE

Calibration Products

Added:
>
>
  • Spectrometer
    • A new calibration product has been added that contains correction curves for point-source calibrated LR data.
    • A new calibration product has been added that contains polynomial coefficients that are used to generate correction curves for extended source calibrated data.
 

Calibration Framework

Line: 205 to 208
 

Photometer Pipeline

Spectrometer Pipeline

Added:
>
>
  • A new task has been added to the point source pipeline that applies the LR correction.
  • A new task has been added to all pipelines that applies the extended calibration correction in order to get better agreement between FTS and Phototmeter photometry.
 

Interactive Analysis and Tools

Revision 62015-09-22 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 162 to 162
 
    • Updates to the chapter Sideband Deconvolution: the output deconvolution product ssb is now called dataset (please note that scripts prior to this change will break)
    • Updates to the chapter Standing wave Removal, ESW correction in HEB bands section: update to the scripts in the demonstration subsection
    • Updates to the chapter How to add and remove flags, section Setting and Clearing Flags with command line tools: improved the definition of the parameter selection for Level 2.5 cubes
Added:
>
>
    • A new chapter called Understanding the uncertainty table information in your data has been added
 
  • Updates to the HIFI Pipeline Specification manual
    • This manual has been restructured
    • Removal of the task mkSpur

Revision 52015-09-17 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 167 to 167
 
    • Removal of the task mkSpur
    • Corrections to broken links pointing to the HIFI Data Reduction Guide
    • Correction due to a missing figure Level 0 Pipeline flow diagram
Changed:
<
<
    • Added text to tasks which did not have text, or were poorly documented
>
>
    • Added text to tasks which did not have text, or were poorly documented, including mkRms
 

PACS

Revision 42015-09-17 - SylvieFBeaulieu

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 154 to 154
 

HiClass export tool

Documentation

Added:
>
>
For HCSS 14, many figures were refreshed in both manuals (HIFI Data Reduction Guide and HIFI Pipeline Specification).
  • Updates to the HIFI Data Reduction Guide:
    • Updates to the chapters Standing wave Removal and HIFI Baseline Removal : adding definition of the new parameter addMedianContinuum
    • Updates to the chapter Flags in HIFI data, section Quality Flags (this chapter is being revamped, please check regularly)
    • Updates to the chapter How to make a spectral cube: added a definition for parameters ignoreOffs, outputNames and extrapolate
    • Updates to the chapter Sideband Deconvolution: the output deconvolution product ssb is now called dataset (please note that scripts prior to this change will break)
    • Updates to the chapter Standing wave Removal, ESW correction in HEB bands section: update to the scripts in the demonstration subsection
    • Updates to the chapter How to add and remove flags, section Setting and Clearing Flags with command line tools: improved the definition of the parameter selection for Level 2.5 cubes
  • Updates to the HIFI Pipeline Specification manual
    • This manual has been restructured
    • Removal of the task mkSpur
    • Corrections to broken links pointing to the HIFI Data Reduction Guide
    • Correction due to a missing figure Level 0 Pipeline flow diagram
    • Added text to tasks which did not have text, or were poorly documented
 

PACS

Revision 32015-09-17 - WimDeMeester

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 66 to 66
 

Source extraction

Images

Added:
>
>
* When requesting the unit, the FITS compatible unit is returned. * The naxis keywords are not longer duplicated in the fits header. * The Wcs ia added to all datasets of the images (like the stddev, ...).
 

Display

Added:
>
>
* Added shortcuts to some of the task directly in the image viewer.
 

Analysis

Spectra

Line: 74 to 82
 

Analysis

Data cubes

Added:
>
>
* When requesting the unit, the FITS compatible unit is returned. * The naxis keywords are not longer duplicated in the fits header. * The Wcs ia added to all datasets of the images (like the stddev, ...).
 

Display

Added:
>
>
* Added shortcuts to some of the task directly in the image viewer.
 

Analysis

Herschel Science Archive

Revision 22015-05-21 - JonathanCook

Line: 1 to 1
 
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->
Line: 82 to 82
 

Calibration sources

Products and datasets

Added:
>
>
A new Auxiliary Observation Summary Product which can be retrieved using the tag LATEST_OBSERVATION_SUMMARY_PRODUCT. This product contains one dataset with a summary of information for each observation. Useful methods exist for querying the data as documented here: ObservationSummaryProduct
 

Observation context

Added:
>
>
A new Calibration Table Product exists under the Auxiliary Context in each Observation Context. This product contains calibration tables data for the given instrument including exactly how the hardware was configured, hard limits, soft limits etc. This provides valuable information for the long term completeness of an observation. Useful methods exist for querying the table data as documented here: SimpleCalTableProduct
 

Product Access Layer

Tasks

Revision 12014-11-24 - AlvarGarcia

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="HipeWhatsNew"
<-- ANALYTICS CODE - DO NOT EDIT -->
<-- Google Analytics script BEGIN -->
<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

What's coming up in HIPE 14

PDF Version Click here

Help Interested in what's new for other versions of HIPE? See this page for links to all the What's New documents from HIPE 4.0 onwards.

See also the HIPE known issues and the Data products known issues pages for last-minute information on known problems.

Planned work

HIPE

† †* TBD

HIFI

† †* TBD

PACS

†* TBD

SPIRE

† †* TBD

Highlights

HIPE

HIFI

PACS

SPIRE

Core system

HIPE

Plotting

TablePlotter

OverPlotter

Dataset viewer

Astronomical utilities

Numeric routines

Data fitting

Data interpolation

Random numbers

Statistics

Trend analysis

Mapping

Other functions

Source extraction

Images

Display

Analysis

Spectra

Display

Analysis

Data cubes

Display

Analysis

Herschel Science Archive

Calibration sources

Products and datasets

Observation context

Product Access Layer

Tasks

Quality control

Systematic product generation

Data input-output

Virtual Observatory

Installer

Documentation

HIFI

HIFI Software Configuration and Properties

HIFI Calibration

HIFI Pipeline

hifiPipeline task

Level 0 Pipeline

WBS Pipeline

HRS Pipeline

Level 1 Pipeline

Level 2 Pipeline

Level 2.5 Pipeline

HIFI Products

HIFI Spectrum Toolbox

Spectrum Explorer (HIFI specifics)

Spectrum tasks (HIFI specifics)

HIFI Data Processing Tools

Standing Wave Removal

Baseline Removal

Deconvolution

Spectral Cube creation

flagTool

HiClass export tool

Documentation

PACS

Pipeline

Photometry

Spectroscopy

Calibration

Photometry

Spectroscopy

Analysis tools

Photometry

Spectroscopy

Product changes

Photometry

Spectroscopy

Documentation

SPIRE

Calibration Products

Calibration Framework

Preprocessing

Engineering Conversion

Common Pipeline

Photometer Pipeline

Spectrometer Pipeline

Interactive Analysis and Tools

Documentation


<-- COMMENT BOX CODE - DO NOT EDIT -->
blog comments powered by Disqus
<-- END OF COMMENT BOX CODE -->

<--  
-->

-- AlvarGarcia - 24 Nov 2014

META FILEATTACHMENT attachment="GoogleAnalytics.txt" attr="h" comment="Google Analytics configuration file" date="1416843301" name="GoogleAnalytics.txt" path="GoogleAnalytics.txt" size="549" stream="GoogleAnalytics.txt" user="Main.AlvarGarcia" version="1"
 
This site is powered by the TWiki collaboration platform Powered by Perl