Difference: DpKnownIssues (1 vs. 202)

Revision 2022017-11-27 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 2012017-09-26 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 2002017-09-07 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1992017-07-20 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1982017-04-21 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1972017-04-20 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1962017-04-17 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1952017-03-14 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1942017-03-14 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1932017-02-20 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1922017-02-09 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Changed:
<
<
<!--
* Set CURRENT_DOC_BUILD = hcss-doc-14.0
-->
>
>
<--  
  • Set CURRENT_DOC_BUILD = hcss-doc-15.0
-->
 

Data Products Known Issues

Line: 242 to 244
 
  • Observations during the steep rise of the sub-K temperature
    • This issue is fixed since HIPE v13, except for a handful of observations made before OD400: contact the HSC Helpdesk if you think data processed with HIPE v13 or later still needs correcting for this issue.
    • During the first few hours after the cooler was recycled the bolometers' temperature (the sub-K temperature) undergoes a steep rise before it reaches a stable plateau. Observations during this period suffer overcorrection of the instrument/telescope emission. This is more significant for faint targets and can be identified as an unphysical slope of the SLW spectrum, with an important negative gap in the region that overlaps with SSW, i.e. a droop at the high-frequency end of SLW (see the figure below). One way to check if your observation is within this problematic category is to get the median sub-K temperature for the first building block: print MEDIAN(obs.level0_5.get(0xA1060001).nhkt['signal']['SUBKTEMP'].data), where obs is the pre-loaded observational context. If the result is less than 0.2869 K then the observation is affected by this.
Added:
>
>
    • Tip, idea A possible fix to this problem is to subtract the smoothed off-axis detectors, because they are also subject to the same overcorrection. The Background Subtraction script from the HIPE Useful Scripts can be used for this investigation, the output of this script is shown in the Figure below.
 
Changed:
<
<
Steep-subK-case.png
>
>
Steep-subK-case
 
Line: 343 to 346
 
META FILEATTACHMENT attachment="cpCubesPartialSpecIssue.png" attr="" comment="" date="1458766760" name="cpCubesPartialSpecIssue.png" path="cpCubesPartialSpecIssue.png" size="131932" user="RosalindHopwood" version="1"
META FILEATTACHMENT attachment="unchoppedwrong.txt" attr="" comment="temporary mssg about pacs spec unchopped foul" date="1462284207" name="unchoppedwrong.txt" path="unchoppedwrong.txt" size="803" user="KatrinaExter" version="1"
META FILEATTACHMENT attachment="unimap_special_cases.txt" attr="" comment="" date="1476185755" name="unimap_special_cases.txt" path="unimap_special_cases.txt" size="851" user="LucaCalzoletti" version="1"
Added:
>
>
META FILEATTACHMENT attachment="Off-axis-subtracted.png" attr="" comment="Off-axis subtracted spectrum fixing the SLW dip" date="1486635646" name="Off-axis-subtracted.png" path="Off-axis-subtracted.png" size="21667" user="IvanV" version="1"

Revision 1912017-01-11 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
<!--
* Set CURRENT_DOC_BUILD = hcss-doc-14.0
-->
Line: 79 to 79
 
    • The release of the interactive pipeline script in HIPE 14.2 (and possibly 14.0.1) for unchopped range scans (called "...with transient correction" in the Pipelines menu) contains an overactive smoothing that can clip away spectral lines. We strongly recommend not using this script for your reductions. The script from HIPE track 15 (which is currently the development track) can be used instead. This does not affect the SPG reductions of these observations.

  • Warning for a subset of unchopped range scan PACS spectroscopy observations in HIPE/SPG 14.0.1.
Changed:
<
<
    • PACS spectroscopy observations reported in this text file are affected by a problem that occurred at end of the pipeline processing for SPG 14.0.1. For this observing mode, the off-source observation is a separate obsid to the on-source observation. The on-source and off-source obsids are processed separately and the results placed in the Level 2 of the observation data. Normally the off-source data are then subtracted from the on-source data and these results placed in the Level 2.5 of the on-source observation. Unfortunately, for a subset of unchopped range scans this did not happen, but instead the on-source data were subtracted from the off-source data and placed in the Level 2.5 of the off-source observation. However, this has now been fixed and all SPG 14.2.2 products will be correct.
>
>
    • PACS spectroscopy observations reported in this text file are affected by a problem that occurred at end of the pipeline processing for SPG 14.0.1. For this observing mode, the off-source observation is a separate obsid to the on-source observation. The on-source and off-source obsids are processed separately and the results placed in the Level 2 of the observation data. Normally the off-source data are then subtracted from the on-source data and these results placed in the Level 2.5 of the on-source observation. Unfortunately, for a subset of unchopped range scans this did not happen, but instead the on-source data were subtracted from the off-source data and placed in the Level 2.5 of the off-source observation. However, this will be fixed in the final bulk processing (early 2017) and the SPG 14.2.2 products will be correct.
 
  • Flat fluxes for certain spectral ranges
    • The pipeline task extractCentralSpectrum is used to created point-source calibrated spectra for pointed observations, working on the rebinned cubes to do this. This task does a spectral interpolation over NaNs (this is to avoid having jumps in summed spectra where one spaxel may have a NaN). The output spectra are improved as a result of this, however the central 9 spaxels of the input rebinned cubes are modified by this interpolation. The result of this modification is that for the rebinned cubes, the fluxes in the spectra in the ranges that were not flatfielded are nearly flat. In SPG 14.0.1-reduced observations, this affects those of the the pointed range scan mode, and specifically in the spectral ranges affected by leakage in the different bands. As an additional side effect, the interpolated cubes (which are created from the rebinned cubes) also show these extended flat-valued spectral ranges at some central locations. However, this has been fixed in SPG 14.2.

Revision 1902016-12-22 - PilarEsquej

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Changed:
<
<
<--    
  • Set CURRENT_DOC_BUILD = hcss-doc-14.0
-->
>
>
<!--
* Set CURRENT_DOC_BUILD = hcss-doc-14.0
-->
 

Data Products Known Issues

Line: 90 to 88
 
    • Processing an observation in HIPE 15 and HIPE 14.2 will produce the same results. However, marginal differences may appear in these processing results compared to the SPG 14.2(.0,1,2) gotten from the HSA. There is no fix for this, but the impact on the science data is insignificant.

  • SPG 14.2.0 red leak range
Changed:
<
<
    • With products processed by SPG 14.2.0, for observations with a small range falling mostly in the red leak region (>190 microns) but with a short stretch shortward of 190 microns, the entire Level 2 red spectral range was lost. This has been fixed in 14.2.2 and only the spectral ranges that fall within the red leak are suppressed.
>
>
    • With products processed by SPG 14.2.0, for observations with a small range falling mostly in the red leak region (>190 microns) but with a short stretch shortward of 190 microns, the entire Level 2 red spectral range was lost. This has been fixed in 14.2.2 and only the spectral ranges that fall within the red leak are suppressed.
 

General notes

Line: 190 to 188
 print cal.phot.beamProfList[5].meta['FWHM_majorPsw'] This works in build 15. Note that the parameter (and a few others) are actually present in the product, but in 14, they are renamed to META_1, META_2...

  • Incorrect WCS in the primary header of Level 2.5 and Level 3 maps
Changed:
<
<
    • The correct WCS for Level 2.5 and Level 3 maps is in the header of the image, error and coverage extensions. The WCS in the primary header corresponds to one of the individual maps and it is not updated for the combined or mosaicked map.
>
>
    • The correct WCS for Level 2.5 and Level 3 maps is in the header of the image, error and coverage extensions. The WCS in the primary header corresponds to one of the individual maps and it is not updated for the combined or mosaicked map.
 
  • Reprocessing with the level-2.5 pipeline
Changed:
<
<
    • Processing a list of observations with the level25_pipeline.py SPG script may lead to tiny differences (1.0e-4, 1.e-5 Jy/beam or MJy/sr) with respect to the same products in the Herschel Science Archive. This is due to the way the current pipeline incorporates the last turnaround building block of concatenated observations in order to proceed with the the deglitching. There is currently no fix, although exactly the same results, as in the archive, can be obtain by individually processing each observation with the two-pass pipeline and then running the level25_pipeline.py script.
>
>
    • Processing a list of observations with the level25_pipeline.py SPG script may lead to tiny differences (1.0e-4, 1.e-5 Jy/beam or MJy/sr) with respect to the same products in the Herschel Science Archive. This is due to the way the current pipeline incorporates the last turnaround building block of concatenated observations in order to proceed with the the deglitching. There is currently no fix, although exactly the same results, as in the archive, can be obtain by individually processing each observation with the two-pass pipeline and then running the level25_pipeline.py script.
 
  • Interactive analysis
    • SourceTimelineFitter: the errors on the fitted RA, Dec positions from the task are wrong near the poles.
Line: 322 to 320
 
Changed:
<
<

Useful Scripts

>
>

Pointing

 
Changed:
<
<
Thermoelastic
>
>
Observations performed at the the so-called 'warm' attitude range (Star Tracker 2 temperature > -15 C), could present a degradation of the pointing performance due to thermoelastic effects. These would include a larger APE and a pointing drift.
 
Added:
>
>
In order to estimate such astrometry shifts, we have created a General Useful Scripts named Astrometry Thermoelastic Drift Correction, available in HIPE 15. The script calculates the average temperature of STR2 sensors during the mid-point of the observation and, if that is > -15 C, it uses an established STR2-Z-axis offset correlation function to get the Z-axis offset and derive RA and DEC offsets. Two possible outputs are provided: (1) A new pointing product that takes into account the updated rotation matrix/quaternion and corrects the filtered attitude. This product can be used by the user to reprocess the observation. (2) New Level 2 products (only applicable to Photometry) where the offsets in RA y DEC (derived from the Z-axis offset) have been used to adjust the WCS.
 

Additional Information

HIFI documents: see the HIFI Instrument and Calibration webpage

Revision 1892016-12-22 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1882016-12-21 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1872016-12-20 - PilarEsquej

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1862016-12-19 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1852016-12-05 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1842016-12-02 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1832016-11-11 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1822016-11-11 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Changed:
<
<
<!--
* Set CURRENT_DOC_BUILD = hcss-doc-14.0
-->
>
>
<--    
  • Set CURRENT_DOC_BUILD = hcss-doc-14.0
-->
 

Data Products Known Issues

Revision 1812016-10-17 - LucaCalzoletti

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
<!--
* Set CURRENT_DOC_BUILD = hcss-doc-14.0
-->
Line: 36 to 36
 
  • JScanam
    • JScanam (an HCSS implementation of the Scanamorphos IDL map-maker) maps are available in level 2.5 and level 3 as of SPG 12 onwards. The JScanam ipipe script to process the maps is available in Hipe 11 onwards. Significant improvements have been achieved in the latest HIPE release, in terms of memory requirement, processing speed, and final map quality. Within the HSA, the JScanam Level2.5 maps are used as the Stand-alone Browse Products for the PACS Photometer scan-maps.
Added:
>
>
    • In the JScanam pipeline for the the generation of the Level 2.5 maps, a random number generator is used for filling the masked signal pixels (within the scanamorphosNoiseSpectrum task, called internally by the scanamorphosIndividualDrifts task, just before the final projection). The random number generator is inizialized at each run and this means that calling twice the task on the same input frames (i.e. by running the same SPG version twice), the maps finally generated are slightly different. This difference can be considered negligible (remember that none PACS maps is absolute calibrated), being the percentage differences pixel-to-pixel of the order of few 0.1%, and well below the level of the standerd deviation map.

 
  • Unimap
    • Unimap (a GLS mapmaker) maps are available in level 2.5 and level 3 as of SPG 13 onwards. The Unimap interactive pipeline script to reprocess the maps is available in HIPE 13 onwards. Unimap is Matlab software and it is invoked by a jython task. When running the interactive pipeline script, the Unimap release must be locally installed (instructions are given at the beginning of ipipe script).

Revision 1802016-10-11 - LucaCalzoletti

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Changed:
<
<
<-- 
  • Set CURRENT_DOC_BUILD = hcss-doc-14.0
-->
>
>
<!--
* Set CURRENT_DOC_BUILD = hcss-doc-14.0
-->
 

Data Products Known Issues

Line: 26 to 24
 

METADATA AND FITS KEYWORDS

Changed:
<
<
  • Message about DATE-OBS: Some internal HCSS metadata are renamed by the HCSS software when translating to FITS. One special case is startDate which gets written to FITS as both DATE-OBS and DATE_OBS. This is done for compatibility with legacy Spitzer software (namely MOPEX).
>
>
  • Message about DATE-OBS: Some internal HCSS metadata are renamed by the HCSS software when translating to FITS. One special case is startDate which gets written to FITS as both DATE-OBS and DATE_OBS. This is done for compatibility with legacy Spitzer software (namely MOPEX).
 

PACS Photometer (scan mapping)

General notes

Changed:
<
<
  • Important note: PACS maps from any map-maker are in essence differential maps: the absolute level is undefined due to the dominating telescope background removed by map-makers. Hence it is not unusual if your background level is negative.
>
>
  • Important note: PACS maps from any map-maker are in essence differential maps: the absolute level is undefined due to the dominating telescope background removed by map-makers. Hence it is not unusual if your background level is negative.
 
  • To produce the highest quality maps, you should consider re-processing or fine-tuning the observations with the latest HIPE User Release. Maps available from the HSA are created within a bulk processing framework, and a reprocessing while fine-tuning the mapper parameters, according to the characteristics of the observed sky region, could enhance the quality of the final maps.
Line: 46 to 42
 
    • Unimap maps into HSA were generated with the release 6.4.4, while interactive script is compatible with Unimaps versions 6.5.3 and below.
    • Distortions introduced by the Unimap GLS algorithm are generally removed by the Unimap post-processing or by the Pixel Noise compensation (see the PACS Data Reduction Guide for photometry). If you are not happy of the SPG archival results, you can run the interactive script by fine tuning the parameters.
    • In some cases, drifts due to the calibration blocks persistence and strips due to strong saturated pixels can be observed, because not properly corrected by the Unimap pre-processing.
Added:
>
>
    • For the pairs of observations (obsIDs) reported into this list unimap_special_cases.txt, the SPG 14.2.0 fails because of a bug in the Unimap release 6.4.4. They are processed with SPG 14.2.1 by using the same Unimap release (6.4.4), but by modifying the values of some Unimap parameters (wrt the default ones) according the fourth (name of parameter) and fifth (adopted value) columns of the list unimap_special_cases.txt
      .
 
  • Highpass-filtered maps (and their intrinsic limitations)
    • High-pass filtered maps in SPG14 are available at level 2 and level 2.5. The extended emission is filtered out in the maps since a rather small filter width is used to remove 1/f noise stripping: this is done to allows us to obtain the best sensitivity for point-sources. Bright sources are masked out during the highpass filtering, hence their flux is not too much affected by filtering. But faint sources are not or are inadequately masked out by the SPG processing, hence the flux loss for these point-source can reach up to 20--30%. Re-processing your data, using customised masks, is recommended here.
Line: 151 to 148
 
    • Level 2.5 maps combine parallel mode as well as SPIRE-only maps into larger maps using the standard two-pass pipeline, see The SPIRE Data Reduction Guide, sections 4.2.3.
    • Level 3 maps are mosaics of overlapping Level 2 and/or Level 2.5 maps, see The SPIRE Data Reduction Guide, sections 4.2.4.
    • Warning, important No astrometry correction is applied on the individual maps used in Level 2.5 or Level 3 processing. This may result in broadened PSFs (or double sources in the extreme case) if some individual maps are affected by astrometry offsets.
Changed:
<
<
    • Before HIPE 14.2.1, the mosaic() task calculated incorrectly the SPIRE Photometer Level-3 maps: the image and error extensions.
>
>
    • Before HIPE 14.2.1, the mosaic() task calculated incorrectly the SPIRE Photometer Level-3 maps: the image and error extensions.
 
  • Stripes in PSW, PMW and/or PLW maps
Changed:
<
<
    • All SPIRE photometry pipelines use the iterative destriper method with a constant baseline (polynomial of zeroth order). This is the best method for SPIRE maps as reported in The SPIRE Map-Making Test Report, Xu et al, 2013 (arXiv:1401.2109). In some cases there could be a residual striping due to a combination of thermal drifts, bolometer jumps and/or improper baseline subtraction. Suggested solutions is to reprocess the map using a first order polynomial for the baseline estimation in the destriper.
>
>
    • All SPIRE photometry pipelines use the iterative destriper method with a constant baseline (polynomial of zeroth order). This is the best method for SPIRE maps as reported in The SPIRE Map-Making Test Report, Xu et al, 2013 (arXiv:1401.2109). In some cases there could be a residual striping due to a combination of thermal drifts, bolometer jumps and/or improper baseline subtraction. Suggested solutions is to reprocess the map using a first order polynomial for the baseline estimation in the destriper.
 
  • De-glitchter masks faint sources
    • Removing glitches from the data is a very delicate process. In particular, for data taken in Parallel Mode (sampling at 10Hz) and at high speed (60"/s) the de-glitcher with standard parameters may flag very faint sources as glitches. Bright sources are different from glitches in that they have a gaussian (i.e. beam/PSF) shape. For faint sources, the sampling rate could be not high enough and hence they have a "delta" shape, which is similar to a small glitch. The user might try to modify the correlation parameter to 0.95: this will decrease the number of detected glitches.
Line: 175 to 172
 
    • The extended calibrated maps (extdPxW in Level 2, 2.5 or 3) incorporate zero level offsets derived from Planck-HFI. For small size SPIRE maps, smaller than ~30 arcmin, the zero-offset can be rather uncertain, due to the large Planck beam (8 arcmin). In such cases the interpretation of the zero offset as the absolute zero level must to be treated with extreme caution.

  • Missing keywords in calTree
Changed:
<
<
    • In HIPE 14.x: trying to access beamProf meta parameter 'FWHM_majorPsw' from the spireCal tree crashes with "parameter not found" exception
>
>
    • In HIPE 14.x: trying to access beamProf meta parameter 'FWHM_majorPsw' from the spireCal tree crashes with "parameter not found" exception
 cal = spireCal(pool="spire_cal_14_3")
Changed:
<
<
print cal.phot.beamProfList[5].meta['FWHM_majorPsw'] This works in build 15. Note that the parameter (and a few others) are actually present in the product, but in 14, they are renamed to META_1, META_2...
>
>
print cal.phot.beamProfList[5].meta['FWHM_majorPsw'] This works in build 15. Note that the parameter (and a few others) are actually present in the product, but in 14, they are renamed to META_1, META_2...
 
  • Interactive analysis
Changed:
<
<
    • SourceTimelineFitter: the errors on the fitted RA, Dec positions from the task are wrong near the poles.
>
>
    • SourceTimelineFitter: the errors on the fitted RA, Dec positions from the task are wrong near the poles.
 
Line: 321 to 317
 
META FILEATTACHMENT attachment="lrBeforeAfter.png" attr="" comment="" date="1456522197" name="lrBeforeAfter.png" path="lrBeforeAfter.png" size="225688" user="RosalindHopwood" version="2"
META FILEATTACHMENT attachment="cpCubesPartialSpecIssue.png" attr="" comment="" date="1458766760" name="cpCubesPartialSpecIssue.png" path="cpCubesPartialSpecIssue.png" size="131932" user="RosalindHopwood" version="1"
META FILEATTACHMENT attachment="unchoppedwrong.txt" attr="" comment="temporary mssg about pacs spec unchopped foul" date="1462284207" name="unchoppedwrong.txt" path="unchoppedwrong.txt" size="803" user="KatrinaExter" version="1"
Added:
>
>
META FILEATTACHMENT attachment="unimap_special_cases.txt" attr="" comment="" date="1476185755" name="unimap_special_cases.txt" path="unimap_special_cases.txt" size="851" user="LucaCalzoletti" version="1"

Revision 1792016-10-11 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1782016-09-29 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1772016-09-28 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1762016-09-27 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1752016-09-15 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1742016-07-16 - LucaCalzoletti

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1732016-07-15 - LucaCalzoletti

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1722016-07-14 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1712016-07-07 - LucaCalzoletti

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1702016-07-07 - LucaCalzoletti

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1692016-07-07 - LucaCalzoletti

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1682016-07-06 - PedroGarciaLario

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1672016-07-06 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1662016-07-06 - LucaCalzoletti

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1652016-07-06 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1642016-07-06 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1632016-07-06 - LucaCalzoletti

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1622016-07-06 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1612016-07-06 - LucaCalzoletti

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1602016-07-06 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1592016-07-06 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1582016-07-06 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1572016-07-06 - LucaCalzoletti

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1562016-07-06 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1552016-07-05 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1542016-07-01 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Changed:
<
<
<!--
* Set CURRENT_DOC_BUILD = hcss-doc-14.0
-->
>
>
<-- 
  • Set CURRENT_DOC_BUILD = hcss-doc-14.0
-->
 

Data Products Known Issues

Line: 187 to 189
 

General notes

Changed:
<
<
  • In order to obtain the best possible Level 2 SPIRE photometry data, the observations might have to be reprocessed with the latest HIPE User Release.
>
>
  • The best science quality Level 2 and Level 2.5 SPIRE photometry products are created with version 14.1 of the Herschel Systematic Product Generation pipelines. These data are available in the Herschel Science Archive. Reprocessing SPIRE observations with HIPE v14.2 or above will result in almost identical products.
 
Changed:
<
<
  • SPIRE-P level 2.5 and level 3 maps
    • For the definitions of the new product levels, introduced with HIPE v11, see The SPIRE Data reduction Guide, sections 3.2.3 and 3.2.4
    • These new levels will be available in the Herschel Science Archive when the observations will be bullk-reprocessed with HIPE v11. The useful user script Photometer_MapMerge.py can be used to make level 2.5 (parallel mode) or level 3 (mosaic) maps, see Section 5.8.3 in the SPIRE Data Reduction Guide.

  • Stripes in PSW, PMW and/or PLW (Level 2) maps
    • All SPIRE photometry pipelines now use by default the destriper, which improves the issue of striping in level 2 maps. Hence observers should expect potential improvements in that respect with version 9.
    • Warning, important Please note that there was a bug in the destriper task included in HIPE 9.0 that may affect your final map, especially if there are bright objects in the observed field. This has been corrected since HIPE 9.1. If your observation falls in the mentioned category, you are strongly advised to update your HIPE installation.
    • Warning, important In HIPE 10.0 the flagging of thermistor jumps in the level 0.5 to 1 data reduction is not set properly. This induces the destriper to work improperly and to leave stripes in the final map. It has been solved starting with HIPE 10.1.
<!--<br /> * Most of the stripes that are present in the final maps are due to a combination of thermal drifts (which in few cases are not efficiently removed) and median baseline subtraction. A similar effect is caused by very bright sources: in this case, the problem resides in the median baseline subtraction only. Suggested solutions:<br /> * switch to a baseline subtraction using a polynomial fitting using the optional task baselineRemovalPolynomial. If there are no jumps in the timelines, you may also try to run the baseline removal on the entire timeline;<br /> * in the case of bright sources, you may try to mask them before running the baseline removal (either median or polynomial): you can use this script as a template<br />-->
>
>
  • SPIRE-P Level 2.5 and Level 3 maps
    • Level 2.5 maps combine parallel mode as well as SPIRE-only maps into larger maps using the standard two-pass pipeline, see The SPIRE Data reduction Guide, sections 4.2.3.
    • Level 3 maps are mosaics of overlapping Level 2 and/or Level 2.5 maps, see The SPIRE Data reduction Guide, sections 4.2.4.
    • Warning, important The individual maps used in Level 2.5 or Level 3 processing are not registered onto the same astrometry reference frame and may result in blurred PSFs if some individual maps are subject to astrometry offsets.

  • Stripes in PSW, PMW and/or PLW maps
    • All SPIRE photometry pipelines use the iterative destriper method with a constant baseline (polynomial of zeroth order). This is the best method for SIRE maps as reported in The SPIRE Map-Making Test Report, Xu et al, 2013 (arXiv:1401.2109). In some cases there could be a residual striping due to a combination of thermal drifts, bolometer jumps and/or improper baseline subtraction. Suggested solutions is to reprocess the map using a first order polynomial for the baseline estimation in the destriper.
 
  • De-glitchter masks faint sources
Changed:
<
<
    • The de-glitcher is a very delicate process. In particular, for data taken in Parallel Mode (sampling at 10Hz) and at high speed (60"/s) the de-glitcher with standard parameters may flag very faint sources as glitches. Bright sources are different from glitches in that they have a gaussian (i.e. beam/PSF) shape. For faint sources, the sampling rate could be not high enough and hence they have a "delta" shape, which is similar to a small glitch. The user might try to modify the correlation parameter to 0.95: this will decrease the number of detected glitches.
>
>
    • Removing glitches from the data is a very delicate process. In particular, for data taken in Parallel Mode (sampling at 10Hz) and at high speed (60"/s) the de-glitcher with standard parameters may flag very faint sources as glitches. Bright sources are different from glitches in that they have a gaussian (i.e. beam/PSF) shape. For faint sources, the sampling rate could be not high enough and hence they have a "delta" shape, which is similar to a small glitch. The user might try to modify the correlation parameter to 0.95: this will decrease the number of detected glitches.
 
  • Some sources have saturated the ADC and the corresponding data have been masked
    • There is nothing a user can do: the source was simply too bright. If the user has other sources still not observed and of the same intensity, it is suggested to change the AORs to use the bright source mode.
Deleted:
<
<
<!-- * <b>Thermistor jumps*<br /> </b> As of HIPE 6.0.3, a new module together called signalJumpDetector in place to identify the jump and to exclude the affected thermistor(s). <br />-->
 
  • Cooler temperature variations
Changed:
<
<
    • The cooler temperature variations, as explained in greater details in the SPIRE Data Reduction Guide, section 6.4, can affect observations performed soon after the cooler recycle. The steep rise of the sub-K detector temperature is also known as the cooler burp and there is a quality flag coolerBurpDetected in HIPE v11 or later that indicates if the observation was performed during this period.
      InfoThe current list of observations known to have cooler temperature effects is here. Note that not all observations in this list raised the coolerBurpDetected flag.
>
>
    • The cooler temperature variations, as explained in greater details in the SPIRE Data Reduction Guide, section 6.4, can affect observations performed soon after the cooler recycle. The steep rise of the sub-K detector temperature is also known as the cooler burp and there is a quality flag coolerBurpDetected (as of HIPE v11 or later) that indicates if the observation was performed during this period.
      InfoThe current list of observations known to have cooler temperature effects is here. Note that not all observations in this list raised the coolerBurpDetected flag.
 
Changed:
<
<
  • NaNs pixels present in the PSW, PMW and/or PLW (Level 2) maps
>
>
  • NaNs pixels present in the PSW, PMW and/or PLW maps
 
    • This effect, related to data masked for various reasons and poor coverage (not enough redundancy), is more evident in single fast-scan Parallel Mode maps. To avoid NaNs, increase the pixel's dimension (i.e., decrease the map's resolution)
Deleted:
<
<
<!-- * <b>WCS in 3-colour images*<br /> </b> In all observation reduced with HIPE 8, the task createRgbImage puts wrong WCS in the output. Instead of using the WCS provided by the WCS input parameter, this task uses the WCS of one of the input images. This has been fixed in HIPE 9<br />-->
 
  • Quality flags
    • Currently, the quality flags at the quality context inside the observation context are just meant for HSC/ICC internal evaluation of the quality of the products and not for the users. In case the data had some serious quality problem, the PI of the program has been contacted about it. Otherwise, only information in the quality summary, when available, should concern the observers.

  • Planck derived zero offsets
Changed:
<
<
    • The extended calibrated maps (extdPxW in level-2, 2.5 or 3) incorporate zero level offsets derived from Planck-HFI. For small size SPIRE maps, smaller than ~30 arcmin, the zero-offset can be rather uncertain, due to the large Planck beam (8 arcmin). In such cases the interpretation of the zero offset as the absolute zero level must to be treated with extreme caution.
>
>
    • The extended calibrated maps (extdPxW in Level 2, 2.5 or 3) incorporate zero level offsets derived from Planck-HFI. For small size SPIRE maps, smaller than ~30 arcmin, the zero-offset can be rather uncertain, due to the large Planck beam (8 arcmin). In such cases the interpretation of the zero offset as the absolute zero level must to be treated with extreme caution.
 
Line: 229 to 225
  SPIRE Small Scan Map AOT release note: 17 Mar 2010
Deleted:
<
<
<!--<br /><a class="red" href="http://herschel.esac.esa.int/Docs/AOTsReleaseStatus/SPIRE_PointSource_AOT_ReleaseNote_30Apr2010.pdf">SPIRE Point Source Mode release note</a>: 30 Apr 2010 <br />-->
 

SPIRE Spectroscopy

Changed:
<
<
  • In order to obtain the best possible Level 2 SPIRE FTS data, the observations should be reprocessed with the latest HIPE User Release.
>
>
  • The best science quality Level 2 and Level 2.5 SPIRE photometry products are created with version 14.1 of the Herschel Systematic Product Generation pipelines. These data are available in the Herschel Science Archive. Reprocessing SPIRE observations with HIPE v14.2 or above will result in almost identical products.
 
  • Warning, important Extended calibrated spectra and spectral cubes: prior to SPG 14.0 all extended calibrated spectra, including the spectral cubes (as these are built by extended calibrated spectra) were affected by a missing correction for the far-field feed horn efficiency (ηff ). This correction is significant - a factor of 1.3-1.5 in SSW and 1.3-2.2 in SLW, as shown in the figure, where the red dashed line shows ηff , the blue squares are the average ratios of uncorrected FTS synthetic photometry to the corresponding photometry extracted from extended-calibrated photometer maps and the green circles are ground based measurements, taken pre-launch. More details on this critical problem will be available in a dedicated paper (Valtchanov et al, in preparation).
Line: 272 to 266
  Steep-subK-case.png
Deleted:
<
<
<!-- * <b>FTS Array footprint user script*<br /> </b> For very large area maps the script produces wrong overlay (offset from the real position) if the default map projections (tangential) is used. The workaround is to use smaller map where the tangential projection centre is near the FTS target position.<br /> -->
 
  • Calibration/pipeline problems/needs
Changed:
<
<
>
>
 

SPIRE Spectroscopy Release Notes

Revision 1532016-05-03 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Changed:
<
<
<-- 
  • Set CURRENT_DOC_BUILD = hcss-doc-14.0
-->
>
>
<!--
* Set CURRENT_DOC_BUILD = hcss-doc-14.0
-->
 

Data Products Known Issues

Line: 18 to 16
  In what follows, we provide a summary of the known issues that you may encounter when inspecting data processed with the automatic pipelines SPG versions 6.1 to 13.0. Most can be resolved by running the pipelines within HIPE and optimizing their parameters as explained below.
Changed:
<
<
<--This is a comment: it will be ignored by the browser -->
>
>
<!--This is a comment: it will be ignored by the browser -->
  Note that some of this information can also be found in the quality report of the observation (QC Report) and as metadata with the FITS keyword "PCAVEATS".
Line: 52 to 50
 
  • Intrinsic limitation of highpass filter maps
    • High-pass filtered maps in SPG13 are available at level 2 and level 2.5. The extended emission is filtered out in the maps since a rather small filter width is used to remove 1/f noise stripping: this is done to allows us to obtain the best sensitivity for point-sources. Bright sources are masked out during the highpass filtering, hence their flux is not too much affected by filtering. But faint sources are not or are inadequately masked out by the SPG processing, hence the flux loss for these point-source can reach up to 20--30%. Re-processing your data is recommended here.
Changed:
<
<
<--
* These are due to a non-optimal estimate of background by the high-pass filter close to a bright source. The current masked high-pass filter pipeline is a trade-off, designed to get a good sensitivity on point-sources and preserve some extended emission up to a few arcmin scale. However due to the relatively large width in the second high-pass filtering, significant stripping from the 1/f noise is still present in the level 2 maps. On the other hand a large fraction of the extended emission is filtered out. The observer is advised to play with the threshold to define the mask and with the width of the high-pass filter to reduce these effects or move to MADmap scanamorphos to preserve extended emission at all scales. Level 2.5 MADmap maps, combining scans and crossed-scans on the other hands do preserve extended emission at all spatial scales.
-->
>
>
<!--<br /> * These are due to a non-optimal estimate of background by the high-pass filter close to a bright source. The current masked high-pass filter pipeline is a trade-off, designed to get a good sensitivity on point-sources and preserve some extended emission up to a few arcmin scale. However due to the relatively large width in the second high-pass filtering, significant stripping from the 1/f noise is still present in the level 2 maps. On the other hand a large fraction of the extended emission is filtered out. The observer is advised to play with the threshold to define the mask and with the width of the high-pass filter to reduce these effects or move to MADmap scanamorphos to preserve extended emission at all scales. Level 2.5 MADmap maps, combining scans and crossed-scans on the other hands do preserve extended emission at all spatial scales.<br />-->
 
  • MADmap issues
    • MADmap -- a GLS (Generalized Least Square) map-maker -- maps are no longer created by SPG13 and beyond. However, the pipeline script to do this yourself is provided in HIPE 13.
Line: 81 to 79
 
    • One matrix (fully half of) of the red channel array was lost at the end of the mission, so from OD1375 onwards this matrix is masked out automatically in the SPG processing.
    • The quality flags in the quality context ("quality" or "qualitySummary") inside the observation context are meant for HSC/ICC internal evaluation of the quality of the products: when an observation had a serious quality problem, the PI of the program would have been contacted about it. For archive users, only the information in the "qualitySummary", when available, is useful.
Changed:
<
<
<---
* Calibration block transient in L2 maps*
Several red (160um) scan maps are affected by a calibration block downwards transient when scheduled immediately before the science observation. This can be mitigated by running a high-pass filter with a smaller width, fitting the transient or just masking the affected frames at the start of the observation.
-->
>
>
<!---<br /> * <b>Calibration block transient in L2 maps*<br /> </b> Several red (160um) scan maps are affected by a calibration block downwards transient when scheduled immediately before the science observation. This can be mitigated by running a high-pass filter with a smaller width, fitting the transient or just masking the affected frames at the start of the observation.<br />-->
 
Changed:
<
<
<--      * Scan maps processed with SPG v4.1.0 were deglitched with "MMT deglitching", a temporal deglitching of the pixel timelines. This deglitching technique works very well for deep fields (e.g. cosmological fields) but fails at high scan speed (60"/s) or even medium scan speeds on bright (>~1Jy) sources, wrongly identified as glitches.
-->
>
>
<!-- * Scan maps processed with SPG v4.1.0 were deglitched with "MMT deglitching", a temporal deglitching of the pixel timelines. This deglitching technique works very well for deep fields (e.g. cosmological fields) but fails at high scan speed (60"/s) or even medium scan speeds on bright (>~1Jy) sources, wrongly identified as glitches.<br />-->
 
Changed:
<
<
<--   * Deglitching*
The deglitching was changed to a so-called "2nd level deglitching" in SPG v6.1.0 that makes use of the spatial redundancy (a sky pixel being seen by several detector pixel readouts). Large scan maps (e.g. galactic fields) processed with SPG v6.1 are therefore of much better quality than earlier ones (SPG v4.1.0), however some low level glitches are left as a high threshold was set in the pipeline. As the deglitching is now run from level 1 to level 2 after highpass filtering, the level 1 cubes in HSA processed with SPG 6.1 do not contain anymore glitch masks. For interactive processing, several hints and methods are given in the ipipe scripts.
-->
>
>
<!-- * <b>Deglitching*<br /> </b> The deglitching was changed to a so-called "2nd level deglitching" in SPG v6.1.0 that makes use of the spatial redundancy (a sky pixel being seen by several detector pixel readouts). Large scan maps (e.g. galactic fields) processed with SPG v6.1 are therefore of much better quality than earlier ones (SPG v4.1.0), however some low level glitches are left as a high threshold was set in the pipeline. As the deglitching is now run from level 1 to level 2 after highpass filtering, the level 1 cubes in HSA processed with SPG 6.1 do not contain anymore glitch masks. For interactive processing, several hints and methods are given in the ipipe scripts.<br />-->
 
Changed:
<
<
<--
* Brightest sources core deglitched in the Level 2 map processed with old pipeline version (SPG v4.1.0)*
This is due to the MMT deglitching at high speed (60"/s), which can wrongly identify bright sources as glitches. A possible solution in these cases is to use 2nd order deglitching in the interactive pipeline, now the default in the HCSS pipeline.

* Glitches from Cosmic ray hits*
Level 2 maps in the archive (HSA) processed with SPG 4.1.0 are affected by glitches as a side effect of disabling the deglitching on bright sources. This has been corrected in SPG version 6.1.0 and above. Interactive HIPE sessions shall get rid off of all these glitches by playing with the deglitching thresholds or using the 2nd order deglitching (memory consuming).
* At 20"/s scan speed, the MMT deglitching does fine if there are no bright sources (>~1Jy), for instance for cosmological survey observations. For brighter sources (nearby galaxies or galactic fields) it is better to switch to second order deglitching. For more information see the PACS Data Reduction Guide (PDRG).
-->
>
>
<!--<br /> * <b>Brightest sources core deglitched in the Level 2 map processed with old pipeline version (SPG v4.1.0)*<br /> </b> This is due to the MMT deglitching at high speed (60"/s), which can wrongly identify bright sources as glitches. A possible solution in these cases is to use 2nd order deglitching in the interactive pipeline, now the default in the HCSS pipeline.<br /><br /> * <b>Glitches from Cosmic ray hits*<br /> </b> Level 2 maps in the archive (HSA) processed with SPG 4.1.0 are affected by glitches as a side effect of disabling the deglitching on bright sources. This has been corrected in SPG version 6.1.0 and above. Interactive HIPE sessions shall get rid off of all these glitches by playing with the deglitching thresholds or using the 2nd order deglitching (memory consuming).<br /> * At 20"/s scan speed, the MMT deglitching does fine if there are no bright sources (>~1Jy), for instance for cosmological survey observations. For brighter sources (nearby galaxies or galactic fields) it is better to switch to second order deglitching. For more information see the PACS Data Reduction Guide (PDRG).<br />-->
 
Changed:
<
<
<--
The astrometry of several PACS scan maps acquired between ODs 320 and 761 has been reported to be off by 4 arcsec (solid offset of the whole map) or even above for a few fields where the tracking stars are not homogeneously distributed in the star-tracker field-of-view. It is intended to improve the a posteriori reconstructed pointing for all observations in the future.
-->
>
>
<!--<br />The astrometry of several PACS scan maps acquired between ODs 320 and 761 has been reported to be off by 4 arcsec (solid offset of the whole map) or even above for a few fields where the tracking stars are not homogeneously distributed in the star-tracker field-of-view. It is intended to improve the a posteriori reconstructed pointing for all observations in the future.<br />-->
 
Changed:
<
<
<--
* The absolute astrometry of Level 2 maps on Solar System Objects (SSO), projected in the SSO reference frame is not reliable and can be off up to 20 arcsec. This a data processing issue only (aberration & light travel time correction), while the observations themselves were correctly performed (uplink). This issue is under investigations at HSC and PACS ICC.
-->
>
>
<!--<br /> * The absolute astrometry of Level 2 maps on Solar System Objects (SSO), projected in the SSO reference frame is not reliable and can be off up to 20 arcsec. This a data processing issue only (aberration & light travel time correction), while the observations themselves were correctly performed (uplink). This issue is under investigations at HSC and PACS ICC.<br />-->
 
Changed:
<
<
<--
* Full noise propagation in the HSC pipelines will be available at later HCSS versions, so the current noise map is relatively but not absolutely correct. Furthermore, the default pipeline results in correlated noise in individual pixels. In order to estimate a proper background standard deviation, the observers are advised to make 10+ aperture photometry measurements in different sky patches around the source and to estimate the absolute standard deviation of the image as the sigma of those photometric points.
-->
>
>
<!--<br /> * Full noise propagation in the HSC pipelines will be available at later HCSS versions, so the current noise map is relatively but not absolutely correct. Furthermore, the default pipeline results in correlated noise in individual pixels. In order to estimate a proper background standard deviation, the observers are advised to make 10+ aperture photometry measurements in different sky patches around the source and to estimate the absolute standard deviation of the image as the sigma of those photometric points. <br />-->
 
Changed:
<
<
<--
* Quality flags in the quality*

Currently, the quality flags at the quality context inside the observation context are just meant for HSC/ICC internal evaluation of the quality of the products and not for the users. In case the data had some serious quality problem, the PI of the program has been contacted about it. Otherwise, only information in the quality summary, when available, should concern the observers.
-->
>
>
<!--<br /> * <b>Quality flags in the quality*<br /><br /> </b> Currently, the quality flags at the quality context inside the observation context are just meant for HSC/ICC internal evaluation of the quality of the products and not for the users. In case the data had some serious quality problem, the PI of the program has been contacted about it. Otherwise, only information in the quality summary, when available, should concern the observers.<br />-->
 
Line: 104 to 102
  PACS photometer AOT release notes see PACS Instrument and Calibration webpage
Changed:
<
<
<---
PACS Scan Map AOT release note: 23 Feb 2010

PACS Photometer - Point/Compact Source Observations: Mini Scan-Maps & Chop-Nod AOT release note: 12 Nov 2010
-->
>
>
<!---<br /><a class="red" href="http://herschel.esac.esa.int/Docs/AOTsReleaseStatus/PACS_ScanMap_ReleaseNote_23Feb2010.pdf">PACS Scan Map AOT release note</a>: 23 Feb 2010<br /><br /><a class="red" href="http://herschel.esac.esa.int/Docs/AOTsReleaseStatus/PACS_PhotMiniScan_ReleaseNote_12Nov2010.pdf">PACS Photometer - Point/Compact Source Observations: Mini Scan-Maps & Chop-Nod AOT release note</a>: 12 Nov 2010<br />-->
 

PACS Spectroscopy

Line: 122 to 120
  For more information about the standard and the standalone browse cubes provided for PACS spectroscopy, see the PACS Products Explained HIPE help document, which is also available from the PACS documentation webpage and the HIPE download webpage, both off the Herschel Science Centre webpage.
Changed:
<
<
General notes
>
>
Warning for a subset of unchopped range scan PACS spectroscopy observations in HIPE/SPG 14
 
Changed:
<
<
  • To produce the highest quality cubes possible, you should consider re-processing or fine-tuning the observations with the latest HIPE User Release. Cubes available from the HSA are created within a bulk processing framework, and a reprocessing while fine-tuning the important pipeline task parameters, according to the characteristics of the observation and source, could enhance the quality of the final results. The first two chapters of the PACS Data Reduction Guide for spectroscopy (HIPE 13) give information about the need to reprocess, and about what to do with HSA-obtained cubes before using them for science.
>
>
PACS spectroscopy observations reported in this text file are affected by a problem that occurred at end of the pipeline processing for SPG 14. For this observing mode, the off-source observation is a separate obsid to the on-source observation. The on-source and off-source obsids are processed separately and the results placed in the Level 2 of the observation data. Normally the off-source data are then subtracted from the on-source data and these results placed in the Level 2.5 of the on-source observation. Unfortunately, for a subset of unchopped range scans this did not happen, but instead the on-source data were subtracted from the off-source data and placed in the Level 2.5 of the off-source observation.

If you wish to work with the on-source observations that have been background subtracted, you can run a useful script in HIPE that starts from the Level 2 products gotten from the HSA (i.e. you do not need to re-run the entire pipeline). Contact the Helpdesk for assitance.

 
Changed:
<
<
<--
* Off-subtraction for unchopped long-range scan observations*
Range Spectroscopy unchopped observation require off-position scans observed in a separate AOR. The off-source observation need to be subtracted from the on-scan after producing Level 2 rebinned spectra, this results a Level 2.5 product. The pipeline only generates Level 2 data products for unchopped spectroscopy in SPG 8.0. You need to combine interactively the on and off positions using the dedicated multi-observation unchopped pipeline script under the PACS pipeline menu.
-->
>
>
This will be fixed in SPG 14.2.
 
Changed:
<
<
<--
* Unchopped grating scan flux calibration*
The absolute flux calibration of the PACS spectrometer is based on observations of flux calibration standards using chopped spectroscopy modes. There are hints of systematic differences in the response scaling between chopped and unchopped mode due to response transients within the chopping pattern. In SPG 8.0 the flux calibration of unchopped data relies on the system response derived chopped scheme, therefore absolute flux values need to be carefully interpreted. Please contact Helpdesk for guidelines on the specific observation you have to deal with.
-->
>
>

General notes

 
Added:
>
>
  • To produce the highest quality cubes possible, you should consider re-processing or fine-tuning the observations with the latest HIPE User Release. Cubes available from the HSA are created within a bulk processing framework, and a reprocessing while fine-tuning the important pipeline task parameters, according to the characteristics of the observation and source, could enhance the quality of the final results. The first two chapters of the PACS Data Reduction Guide for spectroscopy (HIPE 13) give information about the need to reprocess, and about what to do with HSA-obtained cubes before using them for science.
  • Off-subtraction for unchopped long-range scan observations. Range Spectroscopy unchopped observation require off-position scans observed in a separate AOR. The off-source observation need to be subtracted from the on-scan after producing Level 2 rebinned spectra, this results a Level 2.5 product. The pipeline only generates Level 2 data products for unchopped spectroscopy in SPG 8.0. You need to combine interactively the on and off positions using the dedicated multi-observation unchopped pipeline script under the PACS pipeline menu.
  • Unchopped grating scan flux calibration The absolute flux calibration of the PACS spectrometer is based on observations of flux calibration standards using chopped spectroscopy modes. There are hints of systematic differences in the response scaling between chopped and unchopped mode due to response transients within the chopping pattern. In SPG 8.0 the flux calibration of unchopped data relies on the system response derived chopped scheme, therefore absolute flux values need to be carefully interpreted. Please contact Helpdesk for guidelines on the specific observation you have to deal with.
 
  • Spectral leakage
    • The order selection filters of the PACS spectrometer have a steep but not perfectly vertical transmission profile at the cut­off wavelengths of the spectral bands. PACS spectra near the band borders of bands R1, B3A and B2B are therefore affected by higher- or lower-order wavelengths leaking into the spectra -- both continuum and spectral lines! Consult the PACS Calibration Document for more information on the leakage regions.
Changed:
<
<
    • For the leak in band R1, it is possible to reduce these data with a specific relative spectral response function calibration file in HIPE to obtain correct line fluxes (but an incorrect continuum level). How to do this is explained in the PACS Data Reduction Guide for spectroscopy (chps 5 and 6 in the HIPE 13 version). This RSRF is not applied by default since it increases the noise in the resulting spectrum, but can be used interactively within HIPE.
>
>
    • For the leak in band R1, it is possible to reduce these data with a specific relative spectral response function calibration file in HIPE to obtain correct line fluxes (but an incorrect continuum level). How to do this is explained in the PACS Data Reduction Guide for spectroscopy (chps 5 and 6 in the HIPE 13 version). This RSRF is not applied by default since it increases the noise in the resulting spectrum, but can be used interactively within HIPE. Note that for range scan observations, this wavelength range is cut out by the SPG pipeline (i.e. they will not be present in these observations that you download from the HSA), but for line scan observation is it not (and they will be present): do not use these data for your science!
 
  • Second-pass ghosts
    • A second pass in the optics of the PACS spectrometer can cause a ghost image to appear on most spaxels (but never in the central spaxel). If a source located in one of these "originating" spaxels shows a strong spectral line (typically an atomic fine-structure line), then a weak, broadened line can be seen at an offset wavelength in its corresponding "destination" spaxel affected by the 2nd-pass ghost. The peak flux of this line is typically ~5% of the peak of the originating line. The most prominent ghost is the 122 micron feature, which originates from the usually strong CII+ 157.7 micron line. A list of strong ghosts and an image showing the directions of the projected passes on the 5x5 IFU footprint can be found in the PACS Calibration Document.
Line: 146 to 148
 
  • Check for contaminating flux in chop-off positions
    • To check for the presence of contamination from unwanted astronomical sources in the off positions of chopNod mode observations, you can use a Split On-Off pipeline script to produce an off-source and on-source cube. These cubes can then be compared to each other to check for contamination in spectral lines or by strong continuum emission, e.g. by over-plotting the respective spectra. Note that the on-source and off-source cubes produced by this task will not allow you to detect faint levels of contamination because wriggles from the RSRF are not removed by this process. For faint targets (line peak-to-continuum emission ~<5-10 Jy) you should also check the differential signal between the nodA and B on-source cubes. This is documented in the PACS Data Reduction Guide for spectroscopy. In this guide you can also find advice on checking for contamination in the unchopped mode observations, which is done either by comparing companion observations (unchopped range) or with a small script provided in the PDRG (unchopped line).
Changed:
<
<
<--
* RSRF at wavelengths below 53 microns*
The relative spectral response function (used by the pipeline task rsrfCal) is an extrapolation at wavelengths below 53 microns. This will cause problems in the spectra from module 3 (=spaxel 3,0, i.e. in the cube image you see when you look at a PacsCube or PacsRebinnedCube with the Standard Cube Viewer or the Spectrum Explorer, it is the 4th up and on the very left): the extrapolation is too steep and makes the pixel-spectra similarly follow a very steep curve. This is being corrected.
-->
>
>
  • RSRF at wavelengths below 53 microns The relative spectral response function (used by the pipeline task rsrfCal) is an extrapolation at wavelengths below 53 microns. This will cause problems in the spectra from module 3 (=spaxel 3,0, i.e. in the cube image you see when you look at a PacsCube or PacsRebinnedCube with the Standard Cube Viewer or the Spectrum Explorer, it is the 4th up and on the very left): the extrapolation is too steep and makes the pixel-spectra similarly follow a very steep curve. This is being corrected.
 
  • Unstable/incorrect broad-band (dust) features
    • Broad spectral features (of a few micrometer width) and continuum shape variations can be introduced by transient effects (for chopNod mode and more so for unchopped mode observations) and by pointing offsets distorting the Relative Spectral Response Function. The "background normalisation" pipeline script for chopNod observations is recommended for observations looking for such features, as it minimises the effect -- this is the SPG pipeline for HIPE 13 and onwards. For unchopped mode observations you could reprocess the observation with the "transients correction" pipeline script that is new to HIPE 13. However, neither of these pipelines will completely negate the effect of transients and pointing jitter-induced RSRF distortions.
Line: 160 to 160
 
  • Limitations on absolute spectrophotometric accuracy
    • The PACS spectrometer flux calibration accuracy is limited by detector response drifts and slight pointing offsets arising from the standard 1.2" (1-sigma) pointing error occurring within each and every observation. These limit both the absolute flux accuracy and relative accuracy within a band. Various pipelines deal better with these than others (see the advice in the PACS Data Reduction Guide for spectroscopy) but they can never be entirely negated. Hence the calibration uncertainty for any particular observation is a combination of the general calibration uncertainties (given in the PACS Observers Manual), the noise on the spectrum (explained in more detail in the PDRG chp 7.6), and the "activity" during any single observation.
Changed:
<
<
<--
* Line flux correction due to strong wings in the instrumental profile (IP)*
Pre-flight ground-based monochromatic measurements indicated the PACS spectrometer instrumental profile distributes measurable power in spectral line wings. The effect is below ~10% and only noticeable for wavelengths longer than ~150 micrometers. Future HIPE releases will provide a correction factor to apply on a Gaussian fit in order to compensate for line power lost in the IP wings.
-->
>
>
  • Line flux correction due to strong wings in the instrumental profile (IP) Pre-flight ground-based monochromatic measurements indicated the PACS spectrometer instrumental profile distributes measurable power in spectral line wings. The effect is below ~10% and only noticeable for wavelengths longer than ~150 micrometers. Future HIPE releases will provide a correction factor to apply on a Gaussian fit in order to compensate for line power lost in the IP wings.
 
  • NaN's in the final cubes
    • It is normal to have NaNs at the very edges of the spectral ranges of SED mode observations: this is due to gaps in the spectral sampling.
Line: 177 to 175
 

PACS Spectrometer Release Notes

Changed:
<
<
PACS spectrometer AOT release notes see PACS Instrument and Calibration webpage
<---
PACS chopped line scan and high sampling range scan AOT release note: 19 Jan 2010

PACS Wavelength Switching AOT release note: 20 Jan 2009

PACS SED and large range scan AOT release note: 10 Mar 2010

PACS Unchopped Mode AOT Release Note: 20 Sep 2010
-->
>
>
PACS spectrometer AOT release notes see PACS Instrument and Calibration webpage <!---<br /><a class="red" href="http://herschel.esac.esa.int/Docs/AOTsReleaseStatus/PACS_ChoppedLineRange_ReleaseNote_19Jan2010.pdf">PACS chopped line scan and high sampling range scan AOT release note</a>: 19 Jan 2010 <br /><br /><a class="red" href="http://herschel.esac.esa.int/Docs/AOTsReleaseStatus/PACS_WaveSwitching_ReleaseNote_20Jan2010.pdf">PACS Wavelength Switching AOT release note</a>: 20 Jan 2009<br /><br /><a class="red" href="http://herschel.esac.esa.int/Docs/AOTsReleaseStatus/PACS_SEDRange_ReleaseNote_10Mar2010.pdf">PACS SED and large range scan AOT release note</a>: 10 Mar 2010<br /><br /><a class="red" href="http://herschel.esac.esa.int/Docs/AOTsReleaseStatus/PACS_Unchopped_ReleaseNote_20Sep2010.pdf">PACS Unchopped Mode AOT Release Note</a>: 20 Sep 2010<br />-->
 

SPIRE Photometry

Line: 199 to 197
 
    • All SPIRE photometry pipelines now use by default the destriper, which improves the issue of striping in level 2 maps. Hence observers should expect potential improvements in that respect with version 9.
    • Warning, important Please note that there was a bug in the destriper task included in HIPE 9.0 that may affect your final map, especially if there are bright objects in the observed field. This has been corrected since HIPE 9.1. If your observation falls in the mentioned category, you are strongly advised to update your HIPE installation.
    • Warning, important In HIPE 10.0 the flagging of thermistor jumps in the level 0.5 to 1 data reduction is not set properly. This induces the destriper to work improperly and to leave stripes in the final map. It has been solved starting with HIPE 10.1.
Changed:
<
<
<--
* Most of the stripes that are present in the final maps are due to a combination of thermal drifts (which in few cases are not efficiently removed) and median baseline subtraction. A similar effect is caused by very bright sources: in this case, the problem resides in the median baseline subtraction only. Suggested solutions:
* switch to a baseline subtraction using a polynomial fitting using the optional task baselineRemovalPolynomial. If there are no jumps in the timelines, you may also try to run the baseline removal on the entire timeline;
* in the case of bright sources, you may try to mask them before running the baseline removal (either median or polynomial): you can use this script as a template
-->
>
>
<!--<br /> * Most of the stripes that are present in the final maps are due to a combination of thermal drifts (which in few cases are not efficiently removed) and median baseline subtraction. A similar effect is caused by very bright sources: in this case, the problem resides in the median baseline subtraction only. Suggested solutions:<br /> * switch to a baseline subtraction using a polynomial fitting using the optional task baselineRemovalPolynomial. If there are no jumps in the timelines, you may also try to run the baseline removal on the entire timeline;<br /> * in the case of bright sources, you may try to mask them before running the baseline removal (either median or polynomial): you can use this script as a template<br />-->
 
  • De-glitchter masks faint sources
    • The de-glitcher is a very delicate process. In particular, for data taken in Parallel Mode (sampling at 10Hz) and at high speed (60"/s) the de-glitcher with standard parameters may flag very faint sources as glitches. Bright sources are different from glitches in that they have a gaussian (i.e. beam/PSF) shape. For faint sources, the sampling rate could be not high enough and hence they have a "delta" shape, which is similar to a small glitch. The user might try to modify the correlation parameter to 0.95: this will decrease the number of detected glitches.
Line: 207 to 205
 
  • Some sources have saturated the ADC and the corresponding data have been masked
    • There is nothing a user can do: the source was simply too bright. If the user has other sources still not observed and of the same intensity, it is suggested to change the AORs to use the bright source mode.
Changed:
<
<
<--   * Thermistor jumps*
As of HIPE 6.0.3, a new module together called signalJumpDetector in place to identify the jump and to exclude the affected thermistor(s).
-->
>
>
<!-- * <b>Thermistor jumps*<br /> </b> As of HIPE 6.0.3, a new module together called signalJumpDetector in place to identify the jump and to exclude the affected thermistor(s). <br />-->
 
  • Cooler temperature variations
    • The cooler temperature variations, as explained in greater details in the SPIRE Data Reduction Guide, section 6.4, can affect observations performed soon after the cooler recycle. The steep rise of the sub-K detector temperature is also known as the cooler burp and there is a quality flag coolerBurpDetected in HIPE v11 or later that indicates if the observation was performed during this period.
      InfoThe current list of observations known to have cooler temperature effects is here. Note that not all observations in this list raised the coolerBurpDetected flag.
Line: 215 to 213
 
  • NaNs pixels present in the PSW, PMW and/or PLW (Level 2) maps
    • This effect, related to data masked for various reasons and poor coverage (not enough redundancy), is more evident in single fast-scan Parallel Mode maps. To avoid NaNs, increase the pixel's dimension (i.e., decrease the map's resolution)
Changed:
<
<
<--   * WCS in 3-colour images*
In all observation reduced with HIPE 8, the task createRgbImage puts wrong WCS in the output. Instead of using the WCS provided by the WCS input parameter, this task uses the WCS of one of the input images. This has been fixed in HIPE 9
-->
>
>
<!-- * <b>WCS in 3-colour images*<br /> </b> In all observation reduced with HIPE 8, the task createRgbImage puts wrong WCS in the output. Instead of using the WCS provided by the WCS input parameter, this task uses the WCS of one of the input images. This has been fixed in HIPE 9<br />-->
 
  • Quality flags
    • Currently, the quality flags at the quality context inside the observation context are just meant for HSC/ICC internal evaluation of the quality of the products and not for the users. In case the data had some serious quality problem, the PI of the program has been contacted about it. Otherwise, only information in the quality summary, when available, should concern the observers.
Line: 231 to 229
  SPIRE Small Scan Map AOT release note: 17 Mar 2010
Changed:
<
<
<--
SPIRE Point Source Mode release note: 30 Apr 2010
-->
>
>
<!--<br /><a class="red" href="http://herschel.esac.esa.int/Docs/AOTsReleaseStatus/SPIRE_PointSource_AOT_ReleaseNote_30Apr2010.pdf">SPIRE Point Source Mode release note</a>: 30 Apr 2010 <br />-->
 

SPIRE Spectroscopy

Line: 259 to 257
 
    • Very small repetition numbers (e.g. 2 or 4) make 2nd level deglitching, which is based on a statistical outlier criterion, more challenging. The deglitching module may either not identify a glitch at all or it may not remove it completely. In cases where the glitch is located within the double-sided portion of the interferogram, the additional energy from the glitch will translate into artefacts of the continuum level. This kind of problem can be identified by inspecting all detectors from all scans in the level-1 spectral products. For some detectors, one or several scans may appear to be outliers. As a work-around, it is recommended to reprocess the data with a lower thresholdFactor when calling deglitchIfgm(). If the problem persists, the identified detector should be removed from the applicable scan in the SDI product.
      (NB: This affects HIPE 6 and higher)

  • Line fitting
Changed:
<
<
    • Unresolved lines should be fitted using a sinc model with the width fixed. For the sinc model implemented in the SpectrumFitterGUI, the sinc width can be set equal to resolution/π, which, for HR is 1.2/π = 0.382 GHz. For partially resolved lines, with widths greater than 200 km/s, the sincGauss model can be used, keeping the sinc width fixed.
>
>
    • Unresolved lines should be fitted using a sinc model with the width fixed. For the sinc model implemented in the SpectrumFitterGUI, the sinc width can be set equal to resolution/π, which, for HR is 1.2/π = 0.382 GHz. For partially resolved lines, with widths greater than 200 km/s, the sincGauss model can be used, keeping the sinc width fixed.
 
  • Point-source and extended-source calibrated spectra
    • If your level-2 spectra show characteristic jumps at ~1250 GHz and ~750 GHz, so the spectra from the two bands SSW and SLW do not match, then your target may be extended or semi-extended in the SPIRE beam. You may need to use the semi-extended correction tool (SECT), available since HIPE v10. Check the SPIRE Data Reduction Guide (SDRG), section 7.6.2 "Does my spectrum need correcting?".
Line: 274 to 272
  Steep-subK-case.png
Changed:
<
<
<--  * FTS Array footprint user script*
For very large area maps the script produces wrong overlay (offset from the real position) if the default map projections (tangential) is used. The workaround is to use smaller map where the tangential projection centre is near the FTS target position.
-->
>
>
<!-- * <b>FTS Array footprint user script*<br /> </b> For very large area maps the script produces wrong overlay (offset from the real position) if the default map projections (tangential) is used. The workaround is to use smaller map where the tangential projection centre is near the FTS target position.<br /> -->
 
Line: 368 to 366
 

Other Technical Notes

Changed:
<
<
PACS documents: see PACS Instrument and Calibration webpage
<-- 
PACS Spectroscopy AOR Update Guide for Routine Phase Observations: 10 Mar 2010
-->
>
>
PACS documents: see PACS Instrument and Calibration webpage <!-- <br /><a class="red" href="http://herschel.esac.esa.int/Docs/TechnicalNotes/PACS_Spectroscopy_AOR_Update_Guide_10Mar2010.pdf">PACS Spectroscopy AOR Update Guide for Routine Phase Observations</a>: 10 Mar 2010<br />-->
  HIFI Information note on Mapping Modes: 30 Jun 2010
Line: 380 to 378
  PACS photometer and spectrometer calibration documents: see PACS Instrument and Calibration webpage
Changed:
<
<
<-- PACS Spectroscopy Performance and Calibration: 11 Mar 2010 

PACS Photometer Point Spread Function (PSF): 03 Nov 2010
-->
>
>
<!-- <a class="red" href="http://herschel.esac.esa.int/Docs/Calibration/PACS_SpectroscopyPerformanceCalibration_11Mar2010.pdf">PACS Spectroscopy Performance and Calibration</a>: 11 Mar 2010 <br /><br /><a class="red" href="http://herschel.esac.esa.int/Docs/Calibration/PACS_Point_Spread_Function_03Nov2010.pdf">PACS Photometer Point Spread Function (PSF)</a>: 03 Nov 2010 <br />-->
  SPIRE Photometer Beams (FTP repository)

HIFI System noise temperature IF spectra (FTP repository)

Changed:
<
<
<--
* Set ALLOWTOPICCHANGE = DpMgGroup, HscCommunitySupportGroup
-->
>
>
<!--<br />* Set ALLOWTOPICCHANGE = DpMgGroup, HscCommunitySupportGroup<br />-->
 
META FILEATTACHMENT attr="h" autoattached="1" comment="" date="1437490181" name="obsSVVanomaly.csv" path="obsSVVanomaly.csv" size="2980" user="Main.LucaCalzoletti" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1437489475" name="ngc253_1342221743_blue.jpeg" path="ngc253_1342221743_blue.jpeg" size="957526" user="Main.LucaCalzoletti" version="1"
Line: 396 to 394
 
META FILEATTACHMENT attachment="spec_extCalCorr_curve.png" attr="" comment="" date="1456503057" name="spec_extCalCorr_curve.png" path="spec_extCalCorr_curve.png" size="120905" user="RosalindHopwood" version="1"
META FILEATTACHMENT attachment="lrBeforeAfter.png" attr="" comment="" date="1456522197" name="lrBeforeAfter.png" path="lrBeforeAfter.png" size="225688" user="RosalindHopwood" version="2"
META FILEATTACHMENT attachment="cpCubesPartialSpecIssue.png" attr="" comment="" date="1458766760" name="cpCubesPartialSpecIssue.png" path="cpCubesPartialSpecIssue.png" size="131932" user="RosalindHopwood" version="1"
Added:
>
>
META FILEATTACHMENT attachment="unchoppedwrong.txt" attr="" comment="temporary mssg about pacs spec unchopped foul" date="1462284207" name="unchoppedwrong.txt" path="unchoppedwrong.txt" size="803" user="KatrinaExter" version="1"

Revision 1522016-04-18 - RosalindHopwood

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1512016-04-08 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1502016-04-01 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1492016-03-23 - RosalindHopwood

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1482016-03-23 - RosalindHopwood

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1472016-03-09 - RosalindHopwood

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1462016-03-03 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1452016-03-03 - RosalindHopwood

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1442016-02-26 - RosalindHopwood

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1432016-02-26 - RosalindHopwood

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1422016-02-01 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1412015-12-14 - RosalindHopwood

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1402015-12-14 - RosalindHopwood

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Revision 1392015-12-14 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="WebHome"
Added:
>
>
<-- 
  • Set CURRENT_DOC_BUILD = hcss-doc-14.0
-->
 

Data Products Known Issues

HCSS, SPG, and HIPE

Line: 20 to 24
 

TABLE OF CONTENTS

Changed:
<
<
>
>

 

METADATA AND FITS KEYWORDS

Changed:
<
<
Message about DATE-OBS Some internal HCSS metadata are renamed by the HCSS software when translating to FITS. One special case is startDate which gets written to FITS as both DATE-OBS and DATE_OBS. This is done for compatibility with legacy Spitzer software (namely MOPEX).
>
>
Message about DATE-OBS Some internal HCSS metadata are renamed by the HCSS software when translating to FITS. One special case is startDate which gets written to FITS as both DATE-OBS and DATE_OBS. This is done for compatibility with legacy Spitzer software (namely MOPEX).
 

PACS Photometer (scan mapping)

Line: 33 to 36
  Warning for Unimap maps in HIPE/SPG 13: no longer true for HIPE/SPG 14
Changed:
<
<
Some Unimap maps show overshooting effects around very bright sources that are surrounded by a diffuse and relatively faint background. See, as an example, the blue image of NGC253 (obsID: 1342221743). This effect is due to a non-optimised convergence of the GLS algorithm and it has been corrected in HIPE/SPG14. The user can request for a dedicated reprocessing to the Herschel Science Center Helpdesk or they can use the corresponding Level 2.5 JScanam maps (HPPJSMAP[B][R] blue and red products), which are not affected by this issue.
>
>
Some Unimap maps show overshooting effects around very bright sources that are surrounded by a diffuse and relatively faint background. See, as an example, the blue image of NGC253 (obsID: 1342221743). This effect is due to a non-optimised convergence of the GLS algorithm and it has been corrected in HIPE/SPG14. The user can request for a dedicated reprocessing to the Herschel Science Center Helpdesk or they can use the corresponding Level 2.5 JScanam maps (HPPJSMAP[B][R] blue and red products), which are not affected by this issue.
  Warning for pointing anomaly in HIPE/SPG 13/14
Line: 44 to 44
 

General notes

Changed:
<
<
  • To produce the highest quality maps, you should consider re-processing or fine-tuning the observations with the latest HIPE User Release. Maps available from the HSA are created within a bulk processing framework, and a reprocessing while fine-tuning the mapper parameters, according to the characteristics of the observed sky region, could enhance the quality of the final maps.
>
>
  • To produce the highest quality maps, you should consider re-processing or fine-tuning the observations with the latest HIPE User Release. Maps available from the HSA are created within a bulk processing framework, and a reprocessing while fine-tuning the mapper parameters, according to the characteristics of the observed sky region, could enhance the quality of the final maps.
 
  • Important note:
    • PACS maps from any map-maker are in essence differential maps: the absolute level is undefined due to the dominating telescope background removed by map-makers. Hence it is not unusual if your background level is negative.
Line: 52 to 52
 
  • Intrinsic limitation of highpass filter maps
    • High-pass filtered maps in SPG13 are available at level 2 and level 2.5. The extended emission is filtered out in the maps since a rather small filter width is used to remove 1/f noise stripping: this is done to allows us to obtain the best sensitivity for point-sources. Bright sources are masked out during the highpass filtering, hence their flux is not too much affected by filtering. But faint sources are not or are inadequately masked out by the SPG processing, hence the flux loss for these point-source can reach up to 20--30%. Re-processing your data is recommended here.
Changed:
<
<
<-- 
    • These are due to a non-optimal estimate of background by the high-pass filter close to a bright source. The current masked high-pass filter pipeline is a trade-off, designed to get a good sensitivity on point-sources and preserve some extended emission up to a few arcmin scale. However due to the relatively large width in the second high-pass filtering, significant stripping from the 1/f noise is still present in the level 2 maps. On the other hand a large fraction of the extended emission is filtered out. The observer is advised to play with the threshold to define the mask and with the width of the high-pass filter to reduce these effects or move to MADmap scanamorphos to preserve extended emission at all scales. Level 2.5 MADmap maps, combining scans and crossed-scans on the other hands do preserve extended emission at all spatial scales.
-->
>
>
<--
* These are due to a non-optimal estimate of background by the high-pass filter close to a bright source. The current masked high-pass filter pipeline is a trade-off, designed to get a good sensitivity on point-sources and preserve some extended emission up to a few arcmin scale. However due to the relatively large width in the second high-pass filtering, significant stripping from the 1/f noise is still present in the level 2 maps. On the other hand a large fraction of the extended emission is filtered out. The observer is advised to play with the threshold to define the mask and with the width of the high-pass filter to reduce these effects or move to MADmap scanamorphos to preserve extended emission at all scales. Level 2.5 MADmap maps, combining scans and crossed-scans on the other hands do preserve extended emission at all spatial scales.
-->
 
  • MADmap issues
    • MADmap -- a GLS (Generalized Least Square) map-maker -- maps are no longer created by SPG13 and beyond. However, the pipeline script to do this yourself is provided in HIPE 13.
Line: 83 to 81
 
    • One matrix (fully half of) of the red channel array was lost at the end of the mission, so from OD1375 onwards this matrix is masked out automatically in the SPG processing.
    • The quality flags in the quality context ("quality" or "qualitySummary") inside the observation context are meant for HSC/ICC internal evaluation of the quality of the products: when an observation had a serious quality problem, the PI of the program would have been contacted about it. For archive users, only the information in the "qualitySummary", when available, is useful.
Changed:
<
<
<--- 
  • Calibration block transient in L2 maps
    • Several red (160um) scan maps are affected by a calibration block downwards transient when scheduled immediately before the science observation. This can be mitigated by running a high-pass filter with a smaller width, fitting the transient or just masking the affected frames at the start of the observation.
-->

<--      * Scan maps processed with SPG v4.1.0 were deglitched with "MMT deglitching", a temporal deglitching of the pixel timelines. This deglitching technique works very well for deep fields (e.g. cosmological fields) but fails at high scan speed (60"/s) or even medium scan speeds on bright (>~1Jy) sources, wrongly identified as glitches.
-->
>
>
<---
* Calibration block transient in L2 maps*
Several red (160um) scan maps are affected by a calibration block downwards transient when scheduled immediately before the science observation. This can be mitigated by running a high-pass filter with a smaller width, fitting the transient or just masking the affected frames at the start of the observation.
-->
 
Changed:
<
<
<--   * Deglitching 
    • The deglitching was changed to a so-called "2nd level deglitching" in SPG v6.1.0 that makes use of the spatial redundancy (a sky pixel being seen by several detector pixel readouts). Large scan maps (e.g. galactic fields) processed with SPG v6.1 are therefore of much better quality than earlier ones (SPG v4.1.0), however some low level glitches are left as a high threshold was set in the pipeline. As the deglitching is now run from level 1 to level 2 after highpass filtering, the level 1 cubes in HSA processed with SPG 6.1 do not contain anymore glitch masks. For interactive processing, several hints and methods are given in the ipipe scripts.
-->
>
>
<--      * Scan maps processed with SPG v4.1.0 were deglitched with "MMT deglitching", a temporal deglitching of the pixel timelines. This deglitching technique works very well for deep fields (e.g. cosmological fields) but fails at high scan speed (60"/s) or even medium scan speeds on bright (>~1Jy) sources, wrongly identified as glitches.
-->
 
Changed:
<
<

<--
The astrometry of several PACS scan maps acquired between ODs 320 and 761 has been reported to be off by 4 arcsec (solid offset of the whole map) or even above for a few fields where the tracking stars are not homogeneously distributed in the star-tracker field-of-view. It is intended to improve the a posteriori reconstructed pointing for all observations in the future.
-->
>
>
<--
* Brightest sources core deglitched in the Level 2 map processed with old pipeline version (SPG v4.1.0)*
This is due to the MMT deglitching at high speed (60"/s), which can wrongly identify bright sources as glitches. A possible solution in these cases is to use 2nd order deglitching in the interactive pipeline, now the default in the HCSS pipeline.

* Glitches from Cosmic ray hits*
Level 2 maps in the archive (HSA) processed with SPG 4.1.0 are affected by glitches as a side effect of disabling the deglitching on bright sources. This has been corrected in SPG version 6.1.0 and above. Interactive HIPE sessions shall get rid off of all these glitches by playing with the deglitching thresholds or using the 2nd order deglitching (memory consuming).
* At 20"/s scan speed, the MMT deglitching does fine if there are no bright sources (>~1Jy), for instance for cosmological survey observations. For brighter sources (nearby galaxies or galactic fields) it is better to switch to second order deglitching. For more information see the PACS Data Reduction Guide (PDRG).
-->
 
Changed:
<
<
<-- 
    • The absolute astrometry of Level 2 maps on Solar System Objects (SSO), projected in the SSO reference frame is not reliable and can be off up to 20 arcsec. This a data processing issue only (aberration & light travel time correction), while the observations themselves were correctly performed (uplink). This issue is under investigations at HSC and PACS ICC.
-->
>
>
<--
The astrometry of several PACS scan maps acquired between ODs 320 and 761 has been reported to be off by 4 arcsec (solid offset of the whole map) or even above for a few fields where the tracking stars are not homogeneously distributed in the star-tracker field-of-view. It is intended to improve the a posteriori reconstructed pointing for all observations in the future.
-->
 
Changed:
<
<
<-- 
    • Full noise propagation in the HSC pipelines will be available at later HCSS versions, so the current noise map is relatively but not absolutely correct. Furthermore, the default pipeline results in correlated noise in individual pixels. In order to estimate a proper background standard deviation, the observers are advised to make 10+ aperture photometry measurements in different sky patches around the source and to estimate the absolute standard deviation of the image as the sigma of those photometric points.
-->
>
>
<--
* The absolute astrometry of Level 2 maps on Solar System Objects (SSO), projected in the SSO reference frame is not reliable and can be off up to 20 arcsec. This a data processing issue only (aberration & light travel time correction), while the observations themselves were correctly performed (uplink). This issue is under investigations at HSC and PACS ICC.
-->
 
Changed:
<
<
>
>
<--
* Quality flags in the quality*

Currently, the quality flags at the quality context inside the observation context are just meant for HSC/ICC internal evaluation of the quality of the products and not for the users. In case the data had some serious quality problem, the PI of the program has been contacted about it. Otherwise, only information in the quality summary, when available, should concern the observers.
-->
 
Line: 129 to 104
  PACS photometer AOT release notes see PACS Instrument and Calibration webpage
Changed:
<
<
<---
PACS Scan Map AOT release note: 23 Feb 2010

PACS Photometer - Point/Compact Source Observations: Mini Scan-Maps & Chop-Nod AOT release note: 12 Nov 2010 -->

>
>
<---
PACS Scan Map AOT release note: 23 Feb 2010

PACS Photometer - Point/Compact Source Observations: Mini Scan-Maps & Chop-Nod AOT release note: 12 Nov 2010
-->
 

PACS Spectroscopy

Line: 151 to 122
  For more information about the standard and the standalone browse cubes provided for PACS spectroscopy, see the PACS Products Explained HIPE help document, which is also available from the PACS documentation webpage and the HIPE download webpage, both off the Herschel Science Centre webpage.
Changed:
<
<
General notes
>
>
General notes
 
Changed:
<
<
  • To produce the highest quality cubes possible, you should consider re-processing or fine-tuning the observations with the latest HIPE User Release. Cubes available from the HSA are created within a bulk processing framework, and a reprocessing while fine-tuning the important pipeline task parameters, according to the characteristics of the observation and source, could enhance the quality of the final results. The first two chapters of the PACS Data Reduction Guide for spectroscopy (HIPE 13) give information about the need to reprocess, and about what to do with HSA-obtained cubes before using them for science.
>
>
  • To produce the highest quality cubes possible, you should consider re-processing or fine-tuning the observations with the latest HIPE User Release. Cubes available from the HSA are created within a bulk processing framework, and a reprocessing while fine-tuning the important pipeline task parameters, according to the characteristics of the observation and source, could enhance the quality of the final results. The first two chapters of the PACS Data Reduction Guide for spectroscopy (HIPE 13) give information about the need to reprocess, and about what to do with HSA-obtained cubes before using them for science.
 
Changed:
<
<
<-- 
  • Off-subtraction for unchopped long-range scan observations
    • Range Spectroscopy unchopped observation require off-position scans observed in a separate AOR. The off-source observation need to be subtracted from the on-scan after producing Level 2 rebinned spectra, this results a Level 2.5 product. The pipeline only generates Level 2 data products for unchopped spectroscopy in SPG 8.0. You need to combine interactively the on and off positions using the dedicated multi-observation unchopped pipeline script under the PACS pipeline menu.
-->
>
>
<--
* Off-subtraction for unchopped long-range scan observations*
Range Spectroscopy unchopped observation require off-position scans observed in a separate AOR. The off-source observation need to be subtracted from the on-scan after producing Level 2 rebinned spectra, this results a Level 2.5 product. The pipeline only generates Level 2 data products for unchopped spectroscopy in SPG 8.0. You need to combine interactively the on and off positions using the dedicated multi-observation unchopped pipeline script under the PACS pipeline menu.
-->
 
Changed:
<
<
<-- 
  • Unchopped grating scan flux calibration
    • The absolute flux calibration of the PACS spectrometer is based on observations of flux calibration standards using chopped spectroscopy modes. There are hints of systematic differences in the response scaling between chopped and unchopped mode due to response transients within the chopping pattern. In SPG 8.0 the flux calibration of unchopped data relies on the system response derived chopped scheme, therefore absolute flux values need to be carefully interpreted. Please contact Helpdesk for guidelines on the specific observation you have to deal with.
-->
>
>
<--
* Unchopped grating scan flux calibration*
The absolute flux calibration of the PACS spectrometer is based on observations of flux calibration standards using chopped spectroscopy modes. There are hints of systematic differences in the response scaling between chopped and unchopped mode due to response transients within the chopping pattern. In SPG 8.0 the flux calibration of unchopped data relies on the system response derived chopped scheme, therefore absolute flux values need to be carefully interpreted. Please contact Helpdesk for guidelines on the specific observation you have to deal with.
-->
 
  • Spectral leakage
Changed:
<
<
    • The order selection filters of the PACS spectrometer have a steep but not perfectly vertical transmission profile at the cut­off wavelengths of the spectral bands. PACS spectra near the band borders of bands R1, B3A and B2B are therefore affected by higher- or lower-order wavelengths leaking into the spectra -- both continuum and spectral lines! Consult the PACS Calibration Document for more information on the leakage regions.
>
>
    • The order selection filters of the PACS spectrometer have a steep but not perfectly vertical transmission profile at the cut­off wavelengths of the spectral bands. PACS spectra near the band borders of bands R1, B3A and B2B are therefore affected by higher- or lower-order wavelengths leaking into the spectra -- both continuum and spectral lines! Consult the PACS Calibration Document for more information on the leakage regions.
 
    • For the leak in band R1, it is possible to reduce these data with a specific relative spectral response function calibration file in HIPE to obtain correct line fluxes (but an incorrect continuum level). How to do this is explained in the PACS Data Reduction Guide for spectroscopy (chps 5 and 6 in the HIPE 13 version). This RSRF is not applied by default since it increases the noise in the resulting spectrum, but can be used interactively within HIPE.

  • Second-pass ghosts
Changed:
<
<
    • A second pass in the optics of the PACS spectrometer can cause a ghost image to appear on most spaxels (but never in the central spaxel). If a source located in one of these "originating" spaxels shows a strong spectral line (typically an atomic fine-structure line), then a weak, broadened line can be seen at an offset wavelength in its corresponding "destination" spaxel affected by the 2nd-pass ghost. The peak flux of this line is typically ~5% of the peak of the originating line. The most prominent ghost is the 122 micron feature, which originates from the usually strong CII+ 157.7 micron line. A list of strong ghosts and an image showing the directions of the projected passes on the 5x5 IFU footprint can be found in the PACS Calibration Document.
>
>
    • A second pass in the optics of the PACS spectrometer can cause a ghost image to appear on most spaxels (but never in the central spaxel). If a source located in one of these "originating" spaxels shows a strong spectral line (typically an atomic fine-structure line), then a weak, broadened line can be seen at an offset wavelength in its corresponding "destination" spaxel affected by the 2nd-pass ghost. The peak flux of this line is typically ~5% of the peak of the originating line. The most prominent ghost is the 122 micron feature, which originates from the usually strong CII+ 157.7 micron line. A list of strong ghosts and an image showing the directions of the projected passes on the 5x5 IFU footprint can be found in the PACS Calibration Document.
 
  • 62 micron dip
    • A dip can sometimes be seen at spectra between 62 and 63 microns. This is a filter feature. Its appearance depends on the angle at which the light from the sources goes through the filter, and so it depends on the source position and its spatial structure.
Line: 180 to 145
 
    • You may notice a fringing-like pattern in the spectra from (wide) Range scan and full SED-mode observations produced by SPG versions 13 and earlier. This is because the flatfielding is not done by the SPG pipeline in these versions of HIPE. To remove it you need to reprocess the data in HIPE with one of the interactive pipeline scripts, or download the SPG 14 products, for which the flatfielding is part of the SPG pipeline. For Line Spectroscopy observations, the spectral flatfielding is done in the SPG pipeline.

  • Check for contaminating flux in chop-off positions
Changed:
<
<
    • To check for the presence of contamination from unwanted astronomical sources in the off positions of chopNod mode observations, you can use a Split On-Off pipeline script to produce an off-source and on-source cube. These cubes can then be compared to each other to check for contamination in spectral lines or by strong continuum emission, e.g. by over-plotting the respective spectra. Note that the on-source and off-source cubes produced by this task will not allow you to detect faint levels of contamination because wriggles from the RSRF are not removed by this process. For faint targets (line peak-to-continuum emission ~<5-10 Jy) you should also check the differential signal between the nodA and B on-source cubes. This is documented in the PACS Data Reduction Guide for spectroscopy. In this guide you can also find advice on checking for contamination in the unchopped mode observations, which is done either by comparing companion observations (unchopped range) or with a small script provided in the PDRG (unchopped line).
>
>
    • To check for the presence of contamination from unwanted astronomical sources in the off positions of chopNod mode observations, you can use a Split On-Off pipeline script to produce an off-source and on-source cube. These cubes can then be compared to each other to check for contamination in spectral lines or by strong continuum emission, e.g. by over-plotting the respective spectra. Note that the on-source and off-source cubes produced by this task will not allow you to detect faint levels of contamination because wriggles from the RSRF are not removed by this process. For faint targets (line peak-to-continuum emission ~<5-10 Jy) you should also check the differential signal between the nodA and B on-source cubes. This is documented in the PACS Data Reduction Guide for spectroscopy. In this guide you can also find advice on checking for contamination in the unchopped mode observations, which is done either by comparing companion observations (unchopped range) or with a small script provided in the PDRG (unchopped line).
 
Changed:
<
<
<-- 
  • RSRF at wavelengths below 53 microns
    • The relative spectral response function (used by the pipeline task rsrfCal) is an extrapolation at wavelengths below 53 microns. This will cause problems in the spectra from module 3 (=spaxel 3,0, i.e. in the cube image you see when you look at a PacsCube or PacsRebinnedCube with the Standard Cube Viewer or the Spectrum Explorer, it is the 4th up and on the very left): the extrapolation is too steep and makes the pixel-spectra similarly follow a very steep curve. This is being corrected.
-->
>
>
<--
* RSRF at wavelengths below 53 microns*
The relative spectral response function (used by the pipeline task rsrfCal) is an extrapolation at wavelengths below 53 microns. This will cause problems in the spectra from module 3 (=spaxel 3,0, i.e. in the cube image you see when you look at a PacsCube or PacsRebinnedCube with the Standard Cube Viewer or the Spectrum Explorer, it is the 4th up and on the very left): the extrapolation is too steep and makes the pixel-spectra similarly follow a very steep curve. This is being corrected.
-->
 
  • Unstable/incorrect broad-band (dust) features
    • Broad spectral features (of a few micrometer width) and continuum shape variations can be introduced by transient effects (for chopNod mode and more so for unchopped mode observations) and by pointing offsets distorting the Relative Spectral Response Function. The "background normalisation" pipeline script for chopNod observations is recommended for observations looking for such features, as it minimises the effect -- this is the SPG pipeline for HIPE 13 and onwards. For unchopped mode observations you could reprocess the observation with the "transients correction" pipeline script that is new to HIPE 13. However, neither of these pipelines will completely negate the effect of transients and pointing jitter-induced RSRF distortions.

  • Spectral line profiles: the skew
Changed:
<
<
    • As for any slit-spectrograph, if the incoming light beam is neither homogeneously nor symmetrically illuminating a spaxel, then line profiles may be distorted from the ideal Gaussian shape. In case of a point-source observed with PACS, the spectral line profile develops a skew, increasing as the offset of the photocentre along the instrument Z-axis (perpendicular the slit direction) does. Examples of skewed line shapes are given in the PACS Observers Manual.
>
>
    • As for any slit-spectrograph, if the incoming light beam is neither homogeneously nor symmetrically illuminating a spaxel, then line profiles may be distorted from the ideal Gaussian shape. In case of a point-source observed with PACS, the spectral line profile develops a skew, increasing as the offset of the photocentre along the instrument Z-axis (perpendicular the slit direction) does. Examples of skewed line shapes are given in the PACS Observers Manual.
 
  • Point sources
    • How to extract a correctly-calibrated spectrum of a point source is documented in the pipeline scripts and in HIPE. The necessary point source corrections must be applied for a correct spectrum to be produced.

  • Limitations on absolute spectrophotometric accuracy
Changed:
<
<
    • The PACS spectrometer flux calibration accuracy is limited by detector response drifts and slight pointing offsets arising from the standard 1.2" (1-sigma) pointing error occurring within each and every observation. These limit both the absolute flux accuracy and relative accuracy within a band. Various pipelines deal better with these than others (see the advice in the PACS Data Reduction Guide for spectroscopy) but they can never be entirely negated. Hence the calibration uncertainty for any particular observation is a combination of the general calibration uncertainties (given in the PACS Observers Manual), the noise on the spectrum (explained in more detail in the PDRG chp 7.6), and the "activity" during any single observation.
>
>
    • The PACS spectrometer flux calibration accuracy is limited by detector response drifts and slight pointing offsets arising from the standard 1.2" (1-sigma) pointing error occurring within each and every observation. These limit both the absolute flux accuracy and relative accuracy within a band. Various pipelines deal better with these than others (see the advice in the PACS Data Reduction Guide for spectroscopy) but they can never be entirely negated. Hence the calibration uncertainty for any particular observation is a combination of the general calibration uncertainties (given in the PACS Observers Manual), the noise on the spectrum (explained in more detail in the PDRG chp 7.6), and the "activity" during any single observation.
 
Changed:
<
<
<-- 
  • Line flux correction due to strong wings in the instrumental profile (IP)
    • Pre-flight ground-based monochromatic measurements indicated the PACS spectrometer instrumental profile distributes measurable power in spectral line wings. The effect is below ~10% and only noticeable for wavelengths longer than ~150 micrometers. Future HIPE releases will provide a correction factor to apply on a Gaussian fit in order to compensate for line power lost in the IP wings.
-->
>
>
<--
* Line flux correction due to strong wings in the instrumental profile (IP)*
Pre-flight ground-based monochromatic measurements indicated the PACS spectrometer instrumental profile distributes measurable power in spectral line wings. The effect is below ~10% and only noticeable for wavelengths longer than ~150 micrometers. Future HIPE releases will provide a correction factor to apply on a Gaussian fit in order to compensate for line power lost in the IP wings.
-->
 
  • NaN's in the final cubes
    • It is normal to have NaNs at the very edges of the spectral ranges of SED mode observations: this is due to gaps in the spectral sampling.
Line: 218 to 177
 

PACS Spectrometer Release Notes

Changed:
<
<
PACS spectrometer AOT release notes see PACS Instrument and Calibration webpage
<---
PACS chopped line scan and high sampling range scan AOT release note: 19 Jan 2010 

PACS Wavelength Switching AOT release note: 20 Jan 2009

PACS SED and large range scan AOT release note: 10 Mar 2010

PACS Unchopped Mode AOT Release Note: 20 Sep 2010 -->

>
>
PACS spectrometer AOT release notes see PACS Instrument and Calibration webpage
<---
PACS chopped line scan and high sampling range scan AOT release note: 19 Jan 2010

PACS Wavelength Switching AOT release note: 20 Jan 2009

PACS SED and large range scan AOT release note: 10 Mar 2010

PACS Unchopped Mode AOT Release Note: 20 Sep 2010
-->
 

SPIRE Photometry

Line: 235 to 185
  Warning for pointing anomaly in HIPE/SPG 13
Changed:
<
<
PACS and SPIRE photometry observations reported in this CSV file are affected by a known problem related to the reset of the Spacecraft Velocity Vector (SVV) during the upload of the star-tracker's defective pixel table. For the affected observations, the pointing of the telescope can be off along the scan direction, and shifted up to 20 arcsec. This effect will be corrected in HIPE/SPG14.
>
>
PACS and SPIRE photometry observations reported in this CSV file are affected by a known problem related to the reset of the Spacecraft Velocity Vector (SVV) during the upload of the star-tracker's defective pixel table. For the affected observations, the pointing of the telescope can be off along the scan direction, and shifted up to 20 arcsec. This effect will be corrected in HIPE/SPG14.
 

General notes

Changed:
<
<
  • In order to obtain the best possible Level 2 SPIRE photometry data, the observations might have to be reprocessed with the latest HIPE User Release.
    • Warning, important Please note that there was a bug in the destriper task included in HIPE 9.0 that may affect your final map, especially if there are bright objects in the observed field. This has been corrected since HIPE 9.1. If your observation falls in the mentioned category, you are strongly advised to update your HIPE installation.
    • Warning, important In HIPE 10.0 the flagging of thermistor jumps in the level 0.5 to 1 data reduction is not set properly. This induces the destriper to work improperly and to leave stripes in the final map. It has been solved starting with HIPE 10.1.
>
>
  • In order to obtain the best possible Level 2 SPIRE photometry data, the observations might have to be reprocessed with the latest HIPE User Release.
 
  • SPIRE-P level 2.5 and level 3 maps
Changed:
<
<
    • For the definitions of the new product levels, introduced with HIPE v11, see The SPIRE Data reduction Guide, sections 3.2.3 and 3.2.4
    • These new levels will be available in the Herschel Science Archive when the observations will be bullk-reprocessed with HIPE v11. The useful user script Photometer_MapMerge.py can be used to make level 2.5 (parallel mode) or level 3 (mosaic) maps, see Section 5.8.3 in the SPIRE Data Reduction Guide.
>
>
    • For the definitions of the new product levels, introduced with HIPE v11, see The SPIRE Data reduction Guide, sections 3.2.3 and 3.2.4
    • These new levels will be available in the Herschel Science Archive when the observations will be bullk-reprocessed with HIPE v11. The useful user script Photometer_MapMerge.py can be used to make level 2.5 (parallel mode) or level 3 (mosaic) maps, see Section 5.8.3 in the SPIRE Data Reduction Guide.
 
  • Stripes in PSW, PMW and/or PLW (Level 2) maps
    • All SPIRE photometry pipelines now use by default the destriper, which improves the issue of striping in level 2 maps. Hence observers should expect potential improvements in that respect with version 9.
Changed:
<
<
<-- 
    • Most of the stripes that are present in the final maps are due to a combination of thermal drifts (which in few cases are not efficiently removed) and median baseline subtraction. A similar effect is caused by very bright sources: in this case, the problem resides in the median baseline subtraction only. Suggested solutions:
      • switch to a baseline subtraction using a polynomial fitting using the optional task baselineRemovalPolynomial. If there are no jumps in the timelines, you may also try to run the baseline removal on the entire timeline;
      • in the case of bright sources, you may try to mask them before running the baseline removal (either median or polynomial): you can use this script as a template
-->
>
>
    • Warning, important Please note that there was a bug in the destriper task included in HIPE 9.0 that may affect your final map, especially if there are bright objects in the observed field. This has been corrected since HIPE 9.1. If your observation falls in the mentioned category, you are strongly advised to update your HIPE installation.
    • Warning, important In HIPE 10.0 the flagging of thermistor jumps in the level 0.5 to 1 data reduction is not set properly. This induces the destriper to work improperly and to leave stripes in the final map. It has been solved starting with HIPE 10.1.
<--
* Most of the stripes that are present in the final maps are due to a combination of thermal drifts (which in few cases are not efficiently removed) and median baseline subtraction. A similar effect is caused by very bright sources: in this case, the problem resides in the median baseline subtraction only. Suggested solutions:
* switch to a baseline subtraction using a polynomial fitting using the optional task baselineRemovalPolynomial. If there are no jumps in the timelines, you may also try to run the baseline removal on the entire timeline;
* in the case of bright sources, you may try to mask them before running the baseline removal (either median or polynomial): you can use this script as a template
-->
 
  • De-glitchter masks faint sources
    • The de-glitcher is a very delicate process. In particular, for data taken in Parallel Mode (sampling at 10Hz) and at high speed (60"/s) the de-glitcher with standard parameters may flag very faint sources as glitches. Bright sources are different from glitches in that they have a gaussian (i.e. beam/PSF) shape. For faint sources, the sampling rate could be not high enough and hence they have a "delta" shape, which is similar to a small glitch. The user might try to modify the correlation parameter to 0.95: this will decrease the number of detected glitches.
Line: 262 to 207
 
  • Some sources have saturated the ADC and the corresponding data have been masked
    • There is nothing a user can do: the source was simply too bright. If the user has other sources still not observed and of the same intensity, it is suggested to change the AORs to use the bright source mode.
Changed:
<
<
<--   * Thermistor jumps 
    • As of HIPE 6.0.3, a new module together called signalJumpDetector in place to identify the jump and to exclude the affected thermistor(s).
-->
>
>
<--   * Thermistor jumps*
As of HIPE 6.0.3, a new module together called signalJumpDetector in place to identify the jump and to exclude the affected thermistor(s).
-->
 
  • Cooler temperature variations
Changed:
<
<
    • The cooler temperature variations, as explained in greater details in the SPIRE Data Reduction Guide, section 6.4, can affect observations performed soon after the cooler recycle. The steep rise of the sub-K detector temperature is also known as the cooler burp and there is a quality flag coolerBurpDetected in HIPE v11 or later that indicates if the observation was performed during this period.
      InfoThe current list of observations known to have cooler temperature effects is here. Note that not all observations in this list raised the coolerBurpDetected flag.
>
>
    • The cooler temperature variations, as explained in greater details in the SPIRE Data Reduction Guide, section 6.4, can affect observations performed soon after the cooler recycle. The steep rise of the sub-K detector temperature is also known as the cooler burp and there is a quality flag coolerBurpDetected in HIPE v11 or later that indicates if the observation was performed during this period.
      InfoThe current list of observations known to have cooler temperature effects is here. Note that not all observations in this list raised the coolerBurpDetected flag.
 
  • NaNs pixels present in the PSW, PMW and/or PLW (Level 2) maps
    • This effect, related to data masked for various reasons and poor coverage (not enough redundancy), is more evident in single fast-scan Parallel Mode maps. To avoid NaNs, increase the pixel's dimension (i.e., decrease the map's resolution)
Changed:
<
<
<--   * WCS in 3-colour images 
    • In all observation reduced with HIPE 8, the task createRgbImage puts wrong WCS in the output. Instead of using the WCS provided by the WCS input parameter, this task uses the WCS of one of the input images. This has been fixed in HIPE 9
-->
>
>
<--   * WCS in 3-colour images*
In all observation reduced with HIPE 8, the task createRgbImage puts wrong WCS in the output. Instead of using the WCS provided by the WCS input parameter, this task uses the WCS of one of the input images. This has been fixed in HIPE 9
-->
 
  • Quality flags
    • Currently, the quality flags at the quality context inside the observation context are just meant for HSC/ICC internal evaluation of the quality of the products and not for the users. In case the data had some serious quality problem, the PI of the program has been contacted about it. Otherwise, only information in the quality summary, when available, should concern the observers.

  • Planck derived zero offsets
Changed:
<
<
    • The extended calibrated maps (extdPxW in level-2, 2.5 or 3) incorporate zero level offsets derived from Planck-HFI. For small size SPIRE maps, smaller than ~30 arcmin, the zero-offset can be rather uncertain, due to the large Planck beam (8 arcmin). In such cases the interpretation of the zero offset as the absolute zero level must to be treated with extreme caution.
>
>
    • The extended calibrated maps (extdPxW in level-2, 2.5 or 3) incorporate zero level offsets derived from Planck-HFI. For small size SPIRE maps, smaller than ~30 arcmin, the zero-offset can be rather uncertain, due to the large Planck beam (8 arcmin). In such cases the interpretation of the zero offset as the absolute zero level must to be treated with extreme caution.
 
Changed:
<
<
>
>
 

SPIRE Photometer Release Notes

Line: 290 to 231
  SPIRE Small Scan Map AOT release note: 17 Mar 2010
Changed:
<
<
<--
SPIRE Point Source Mode release note: 30 Apr 2010 
-->
>
>
<--
SPIRE Point Source Mode release note: 30 Apr 2010
-->
 

SPIRE Spectroscopy

Changed:
<
<
  • In order to obtain the best possible Level 2 SPIRE FTS data, the observations should be reprocessed with the latest HIPE User Release.
>
>
  • In order to obtain the best possible Level 2 SPIRE FTS data, the observations should be reprocessed with the latest HIPE User Release.

  • Warning, important Extended calibrated spectra and spectral cubes: all extended calibrated spectra, including the spectral cubes as these are built by extended calibrated spectra, are affected by a missing correction for the far-field feedhorn efficiency. This correction is significant - a factor of 1.3-1.5 in SSW and 1.3-2.2 in SLW, as shown in the figure. More details on this critical problem will be available in a dedicated paper (Valtchanov et al, in preparation).

extCorr_v4.png

 
  • Bright source mode: observations in bright source mode processed with HIPE v7 or earlier result in spectra that have no scientific value. Bright source observations processed with HIPE v8 or above are fine.
Line: 304 to 247
 
  • High + Low resolution (H+LR) observations: The LR part of the H+LR observations is not properly calibrated and should not be used. The FTS team is working on improving the calibration and as soon as we have a validated proper calibration we will implement it and reprocess the H+LR data.
Changed:
<
<
  • Faint point source observations: if the SSW and SLW bands do not match up for a source that is known to be a point source, observers are recommended to run the background subtraction script in HIPE - more details are in the SPIRE Data Reduction Guide. If this doesn't solve the problem, the observer is encouraged to contact the HSC helpdesk or the FTS User Support Group.
>
>
  • Faint point source observations: if the SSW and SLW bands do not match up for a source that is known to be a point source, observers are recommended to run the background subtraction script in HIPE - more details are in the SPIRE Data Reduction Guide. If this doesn't solve the problem, the observer is encouraged to contact the HSC helpdesk or the FTS User Support Group.
 
  • Artefacts in the continuum for few repetitions
Changed:
<
<
    • Very small repetition numbers (e.g. 2 or 4) make 2nd level deglitching, which is based on a statistical outlier criterion, more challenging. The deglitching module may either not identify a glitch at all or it may not remove it completely. In cases where the glitch is located within the double-sided portion of the interferogram, the additional energy from the glitch will translate into artefacts of the continuum level. This kind of problem can be identified by inspecting all detectors from all scans in the level-1 spectral products. For some detectors, one or several scans may appear to be outliers. As a work-around, it is recommended to reprocess the data with a lower thresholdFactor when calling deglitchIfgm(). If the problem persists, the identified detector should be removed from the applicable scan in the SDI product.
      (NB: This affects HIPE 6 and higher)
>
>
    • Very small repetition numbers (e.g. 2 or 4) make 2nd level deglitching, which is based on a statistical outlier criterion, more challenging. The deglitching module may either not identify a glitch at all or it may not remove it completely. In cases where the glitch is located within the double-sided portion of the interferogram, the additional energy from the glitch will translate into artefacts of the continuum level. This kind of problem can be identified by inspecting all detectors from all scans in the level-1 spectral products. For some detectors, one or several scans may appear to be outliers. As a work-around, it is recommended to reprocess the data with a lower thresholdFactor when calling deglitchIfgm(). If the problem persists, the identified detector should be removed from the applicable scan in the SDI product.
      (NB: This affects HIPE 6 and higher)
 
  • Line fitting
Changed:
<
<
    • If you know that the line you wish to measure is unresolved then you may want to fix the line width to the instrumental line width. For the Sinc model, implemented in the SpectrumFitterGUI you should put the Sinc width equal to resolution/π, which, for HR is 1.2/π = 0.382 GHz.
>
>
    • If you know that the line you wish to measure is unresolved then you may want to fix the line width to the instrumental line width. For the Sinc model, implemented in the SpectrumFitterGUI you should put the Sinc width equal to resolution/π, which, for HR is 1.2/π = 0.382 GHz.
 
  • Point source and extended source spectra
Changed:
<
<
    • If your level-2 spectra show characteristic jumps at ~1250 GHz and ~750 GHz, and the spectra from the two bands SSW and SLW do not match, then your target is extended or semi-extended in the SPIRE beam. You need to use the semi-extended correction tool (SECT) available since HIPE v10. Check the the SPIRE Data Reduction Guide (SDRG).
>
>
    • If your level-2 spectra show characteristic jumps at ~1250 GHz and ~750 GHz, and the spectra from the two bands SSW and SLW do not match, then your target is extended or semi-extended in the SPIRE beam. You need to use the semi-extended correction tool (SECT) available since HIPE v10. Check the the SPIRE Data Reduction Guide (SDRG).
 
  • Quality flags in the quality
    • The quality flags at the quality context inside the observation context are just meant for HSC/ICC internal evaluation of the quality of the products and not for the users. In case the data had some serious quality problem, the PI of the program has been contacted about it. Otherwise, only information in the Quality Summary, when available, should concern the observers.
Changed:
<
<
    • For observations before OD1000 there could be an erroneously raised or a missing flag RADECACC. This flag is calculated as the difference between the observer requested target coordinates (kept in a metadata raNominal, decNominal) and the average pointing of the telescope during the observation (excluding the slew). This is an indicator of the pointing accuracy of the observation. If greater than 2.2" then RADECACC is raised in the QC context. The calculation of the average (ra, dec) during the observation does not take into account the known offset of 1.7" of the BSM home position for OD < 1000 and consequently observations with better pointing than 2.2" may be flagged and vice versa, observations at more than 2.2" may not be flagged. If you want to check the pointing offset with respect to the requested target coordinates then you should use the pointing information for the central detector SSWD4 in the level-2 context.
      A new quality parameter raDecOffset is introduced in HIPE v13, in level-2 products, which contains the correct offset between the raNominal, decNominal and ra,dec for the FTS central detector.
>
>
    • For observations before OD1000 there could be an erroneously raised or a missing flag RADECACC. This flag is calculated as the difference between the observer requested target coordinates (kept in a metadata raNominal, decNominal) and the average pointing of the telescope during the observation (excluding the slew). This is an indicator of the pointing accuracy of the observation. If greater than 2.2" then RADECACC is raised in the QC context. The calculation of the average (ra, dec) during the observation does not take into account the known offset of 1.7" of the BSM home position for OD < 1000 and consequently observations with better pointing than 2.2" may be flagged and vice versa, observations at more than 2.2" may not be flagged. If you want to check the pointing offset with respect to the requested target coordinates then you should use the pointing information for the central detector SSWD4 in the level-2 context.
      A new quality parameter raDecOffset is introduced in HIPE v13, in level-2 products, which contains the correct offset between the raNominal, decNominal and ra,dec for the FTS central detector.
 
  • Observations during the steep rise of the sub-K temperature
Changed:
<
<
    • During the first few hours after the cooler was recycled the bolometers' temperature (the sub-K temperature) undergoes a steep rise before it reaches a stable plateau. Observations during this period suffer overcorrection of the instrument/telescope emission. This is more significant for faint targets and can be identified as an unphysical slope of the SLW spectrum, with an important negative gap in the region that overlaps with SSW (see the Figure). One way to check if your observation is within this problematic category is to get the median sub-K temperature for the first building block: print MEDIAN(obs.level0_5.get(0xA1060001).nhkt['signal']['SUBKTEMP'].data), where obs is the pre-loaded observational context. If the result is less than 0.2869 K then the observation is affected by this.
      This effect is fixed in HIPE v13.
>
>
    • During the first few hours after the cooler was recycled the bolometers' temperature (the sub-K temperature) undergoes a steep rise before it reaches a stable plateau. Observations during this period suffer overcorrection of the instrument/telescope emission. This is more significant for faint targets and can be identified as an unphysical slope of the SLW spectrum, with an important negative gap in the region that overlaps with SSW (see the Figure). One way to check if your observation is within this problematic category is to get the median sub-K temperature for the first building block: print MEDIAN(obs.level0_5.get(0xA1060001).nhkt['signal']['SUBKTEMP'].data), where obs is the pre-loaded observational context. If the result is less than 0.2869 K then the observation is affected by this.
      This effect is fixed in HIPE v13.
  Steep-subK-case.png
Changed:
<
<
<--   * FTS Array footprint user script 
    • For very large area maps the script produces wrong overlay (offset from the real position) if the default map projections (tangential) is used. The workaround is to use smaller map where the tangential projection centre is near the FTS target position.
-->
>
>
<--  * FTS Array footprint user script*
For very large area maps the script produces wrong overlay (offset from the real position) if the default map projections (tangential) is used. The workaround is to use smaller map where the tangential projection centre is near the FTS target position.
-->
 
  • Calibration/pipeline problems/needs
Changed:
<
<
>
>
 

SPIRE Spectroscopy Release Notes

Line: 341 to 282
 

HIFI observations (point mode, spectral survey and mapping observations)

Changed:
<
<
In order to obtain the best possible Level 2 HIFI data the observations should be reprocessed with the latest HIPE User Release.
>
>
In order to obtain the best possible Level 2 HIFI data the observations should be reprocessed with the latest HIPE User Release.
 
  • Limitations in the mkRms task for automatic noise estimates
Changed:
<
<
    • From HIPE 13 onwards, automatic estimates of the spectra noise rms are provided to the users in the trendAnalysis > Statistisc context - these are based on an upgraded version of the task mkRms (see also the HIFI pipeline specficication document). Those estimates are in particular used to raise, or not, flags informing about whether the data are under-performing in terms of radiometric noise with respect to the theoretical predictions used at the time of the proposal. Because the noise estimate relies on the efficiency of automatically masking lines that may be present in the data, there can exist circumstances in which this masking in sub-optimally performed, therefore leading to inaccurate noise estimate, and possible false positives in the raised flags. Users should be aware of this limitation and, if necessary, cross-check the reliability of certain flags related to those noise figures through their own estimates.
>
>
    • From HIPE 13 onwards, automatic estimates of the spectra noise rms are provided to the users in the trendAnalysis > Statistisc context - these are based on an upgraded version of the task mkRms (see also the HIFI pipeline specficication document). Those estimates are in particular used to raise, or not, flags informing about whether the data are under-performing in terms of radiometric noise with respect to the theoretical predictions used at the time of the proposal. Because the noise estimate relies on the efficiency of automatically masking lines that may be present in the data, there can exist circumstances in which this masking in sub-optimally performed, therefore leading to inaccurate noise estimate, and possible false positives in the raised flags. Users should be aware of this limitation and, if necessary, cross-check the reliability of certain flags related to those noise figures through their own estimates.
 
  • Corrupted data-frames
    • There is a limited number of obsids where one or more data-frames from the spectrometers is corrupted in some fashion and leads to level 2 products that cannot be properly calibrated. A scheme was put in place some versions ago to flag or simply remove those frames at level 0 so the final products won't be affected by them. This scheme however suffered from a regression bug in 11.1 so that the corresponding products did not benefit from this correction properly. We note however that this should only concern about 50 obsids in the whole HIFI archive. This was fixed in HIPE 12.
Line: 399 to 340
 
    • Some expert users may want to reprocess their data starting from raw telemetry (so-called level-1). From HIPE 9 onwards additional telemetry (notably from the ACMS system) was added to the products. In order to avoid problems with missing these products from earlier downloads, the following can be temporarily added to your local configuration property: hcss.ia.spg.ops.AuxPlugin.products={..., herschel.ia.obs.auxiliary.acms.AcmsTelemetryProduct, ...}

  • Flux Conservation in Spectral Cubes from Mapping Observations
Changed:
<
<
    • The doGridding task in the HIFI pipeline is responsible for convolving the spectral datasets acquired in an OTF or DBS Raster mapping observation into a spectral cube with a specified pixel scale, which by default should match how the scan lines and readout points within each line were spaced during the observation. The scheme of signal filtering and interpolation to put the data on the specified grid may affect the overall flux conservation, at a level which is low but you should be aware of. For example, the total signal summed from a spectral cube produced using a Gaussian filter over the datasets of a Nyquist-sampled OTF map is generally < 1% lower than the sum of the signal taken directly from the input datasets (the Level 2 HTP datasets) before convolution. A part or all of this slight mismatch may be on the assumed versus actual beam shape at the observed frequency. If you wish to put the map on a coarser grid, effectively reducing the spatial resolution to a wider beam in order to match another observation, then the flux losses become more noticeable. Doubling the pixels sizes from their default (native map point spacing) can reduce the total flux by as much as 10%, accompanied by an increase in baseline RMS noise. The effect is more prevalent in OTF maps than DBS Raster, and in addition to deviations from ideal beam shape characteristics becoming more important, the filtering and interpolation method, and parameter values can be influential. No changes to the doGridding algorithm are planned, and this issue applies to all HIPE versions.
>
>
    • The doGridding task in the HIFI pipeline is responsible for convolving the spectral datasets acquired in an OTF or DBS Raster mapping observation into a spectral cube with a specified pixel scale, which by default should match how the scan lines and readout points within each line were spaced during the observation. The scheme of signal filtering and interpolation to put the data on the specified grid may affect the overall flux conservation, at a level which is low but you should be aware of. For example, the total signal summed from a spectral cube produced using a Gaussian filter over the datasets of a Nyquist-sampled OTF map is generally < 1% lower than the sum of the signal taken directly from the input datasets (the Level 2 HTP datasets) before convolution. A part or all of this slight mismatch may be on the assumed versus actual beam shape at the observed frequency. If you wish to put the map on a coarser grid, effectively reducing the spatial resolution to a wider beam in order to match another observation, then the flux losses become more noticeable. Doubling the pixels sizes from their default (native map point spacing) can reduce the total flux by as much as 10%, accompanied by an increase in baseline RMS noise. The effect is more prevalent in OTF maps than DBS Raster, and in addition to deviations from ideal beam shape characteristics becoming more important, the filtering and interpolation method, and parameter values can be influential. No changes to the doGridding algorithm are planned, and this issue applies to all HIPE versions.
 
  • Retrieval of calibration files from the HSA
    • In HIPE versions from the track 9, there is small bug in the getHifiCal task. The task may erroneously report that the latest calibration tree available is HIFI_CAL_9_0, while more recent versions are now available. This issue is solved in HIPE 10. This is not a problem with the HIPE 11 build. One possible work-around is to clear the calibration pool in your .hcss/lstore, and call the task again or simply use the updated HIPE version. Then the latest calibration tree should be offered for download.
Line: 420 to 361
 

Other Technical Notes

Changed:
<
<
PACS documents: see PACS Instrument and Calibration webpage
<-- 
PACS Spectroscopy AOR Update Guide for Routine Phase Observations: 10 Mar 2010
-->
>
>
PACS documents: see PACS Instrument and Calibration webpage
<-- 
PACS Spectroscopy AOR Update Guide for Routine Phase Observations: 10 Mar 2010
-->
  HIFI Information note on Mapping Modes: 30 Jun 2010
Line: 435 to 373
  PACS photometer and spectrometer calibration documents: see PACS Instrument and Calibration webpage
Changed:
<
<
<-- PACS Spectroscopy Performance and Calibration: 11 Mar 2010 

PACS Photometer Point Spread Function (PSF): 03 Nov 2010 -->

>
>
<-- PACS Spectroscopy Performance and Calibration: 11 Mar 2010 

PACS Photometer Point Spread Function (PSF): 03 Nov 2010
-->
  SPIRE Photometer Beams (FTP repository)

HIFI System noise temperature IF spectra (FTP repository)

Changed:
<
<
<--
*  Set ALLOWTOPICCHANGE = DpMgGroup, HscCommunitySupportGroup
-->
>
>
<--
* Set ALLOWTOPICCHANGE = DpMgGroup, HscCommunitySupportGroup
-->
 
META FILEATTACHMENT attr="h" autoattached="1" comment="" date="1437490181" name="obsSVVanomaly.csv" path="obsSVVanomaly.csv" size="2980" user="Main.LucaCalzoletti" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1437489475" name="ngc253_1342221743_blue.jpeg" path="ngc253_1342221743_blue.jpeg" size="957526" user="Main.LucaCalzoletti" version="1"
META FILEATTACHMENT attr="h" autoattached="1" comment="" date="1437982718" name="SVV_resets_aberration_shift.csv" path="SVV_resets_aberration_shift.csv" size="7875" user="Main.LucaCalzoletti" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1404218642" name="Steep-subK-case.png" path="Steep-subK-case.png" size="91418" user="Main.IvanV" version="1"
Added:
>
>
META FILEATTACHMENT attachment="extCorr_v4.png" attr="" comment="SPIRE FTS extended calibration correction curve" date="1450104021" name="extCorr_v4.png" path="extCorr_v4.png" size="16780" user="IvanV" version="1"

Revision 1382015-12-14 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Data Products Known Issues

Line: 29 to 29
 

PACS Photometer (scan mapping)

Changed:
<
<

Special messages

>
>

Temporary messages

  Warning for Unimap maps in HIPE/SPG 13: no longer true for HIPE/SPG 14
Line: 137 to 137
 

PACS Spectroscopy

Changed:
<
<

Special messages

>
>

Temporary messages

  Warning for drizzled cubes from HIPE/SPG 13: no longer relevant in HIPE/SPG 14
Line: 197 to 197
 
    • How to extract a correctly-calibrated spectrum of a point source is documented in the pipeline scripts and in HIPE. The necessary point source corrections must be applied for a correct spectrum to be produced.

  • Limitations on absolute spectrophotometric accuracy
Changed:
<
<
    • The PACS spectrometer flux calibration accuracy is limited by detector response drifts and slight pointing offsets arising from the standard 1.2" (1-sigma) pointing error occurring within each and every observation. These limit both the absolute flux accuracy and relative accuracy within a band. Various pipelines deal better with these than others (see the advice in the PACS Data Reduction Guide for spectroscopy) but they can never be entirely negated. Hence the calibration uncertainty for any particular observation is a combination of the general calibration uncertainties (given in the PACS Observers Manual), the noise on the spectrum (explained in more detail in the PDRG chp 7.6/HIPE 13), and the "activity" during any single observation.
>
>
    • The PACS spectrometer flux calibration accuracy is limited by detector response drifts and slight pointing offsets arising from the standard 1.2" (1-sigma) pointing error occurring within each and every observation. These limit both the absolute flux accuracy and relative accuracy within a band. Various pipelines deal better with these than others (see the advice in the PACS Data Reduction Guide for spectroscopy) but they can never be entirely negated. Hence the calibration uncertainty for any particular observation is a combination of the general calibration uncertainties (given in the PACS Observers Manual), the noise on the spectrum (explained in more detail in the PDRG chp 7.6), and the "activity" during any single observation.
 

Revision 1372015-12-13 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Data Products Known Issues

Line: 343 to 343
  In order to obtain the best possible Level 2 HIFI data the observations should be reprocessed with the latest HIPE User Release.
Added:
>
>
  • Limitations in the mkRms task for automatic noise estimates
    • From HIPE 13 onwards, automatic estimates of the spectra noise rms are provided to the users in the trendAnalysis > Statistisc context - these are based on an upgraded version of the task mkRms (see also the HIFI pipeline specficication document). Those estimates are in particular used to raise, or not, flags informing about whether the data are under-performing in terms of radiometric noise with respect to the theoretical predictions used at the time of the proposal. Because the noise estimate relies on the efficiency of automatically masking lines that may be present in the data, there can exist circumstances in which this masking in sub-optimally performed, therefore leading to inaccurate noise estimate, and possible false positives in the raised flags. Users should be aware of this limitation and, if necessary, cross-check the reliability of certain flags related to those noise figures through their own estimates.
 
  • Corrupted data-frames
    • There is a limited number of obsids where one or more data-frames from the spectrometers is corrupted in some fashion and leads to level 2 products that cannot be properly calibrated. A scheme was put in place some versions ago to flag or simply remove those frames at level 0 so the final products won't be affected by them. This scheme however suffered from a regression bug in 11.1 so that the corresponding products did not benefit from this correction properly. We note however that this should only concern about 50 obsids in the whole HIFI archive. This was fixed in HIPE 12.
Line: 353 to 356
 
    • Due to gaps in the LO output power over the HIFI frequency range, some spectral scans will contain spectra that cannot be used due their very high noise temperature levels. These data should be discarded from further processing (e.g. deconvolution). For single frequency observations, there exist some places where the theoretical noise temperature (i.e. the one advertised by HSpot) should be nominal, but the sensitivity effectively achieved at the time of observations is noticeably worse. These are cases where the mixer tuning algorithm does not fully converge. Such cases have e.g. been reported around LO frequencies of ~ 1441.5 GHz. An overview of the expected unpumped frequency areas is given in the AOT release notes. Users observing such effects should contact the Helpdesk.

  • Standing waves
Changed:
<
<
    • There are a variety of standing waves known to affect the data at Level2 - see the AOT release notes for more details. At the present time, the !FitHifiFringe tool usually does a decent job for most standing waves in SIS bands. In HEB bands, where the standing wave is not of optical nature and does have a strictly sinusoidal shape, FitHifiFringe can help when restricted to the frequency range where the line is present. Another alternative in the form of a dedicated algorithm has been developed (so-called "matching technique") and is offered in the hebCorrection task from HIPE 12.0 onwards (details can be found in Chapter 10 of the HIFI Data Reduction Guide). You can contact Ian Avruch (i.avruch@sron.nl) at the HIFI ICC for further details. In any case, it is advised not to use more than 3 components in FitHifiFringe. Since HIPE 5, a number of new features have been made available in FitHifiFringe. One of them ("sub_base" option) allows you to fit a baseline at the same time. Since FitHifiFringe can identify automatically the lines in order to set masks, users should be careful when treating lines with large wings. In this case, the user should set a window by hand. The automatic line masking should then be switched off.
>
>
    • There are a variety of standing waves known to affect the data at Level2 - see the AOT release notes for more details. At the present time, the FitHifiFringe tool usually does a decent job for most standing waves in SIS bands. In HEB bands, where the standing wave is not of optical nature and does have a strictly sinusoidal shape, FitHifiFringe can help when restricted to the frequency range where the line is present. Another alternative in the form of a dedicated algorithm has been developed (so-called "matching technique") and is offered in the hebCorrection task from HIPE 12.0 onwards (details can be found in Chapter 10 of the HIFI Data Reduction Guide). You can contact Ian Avruch (i.avruch@sron.nl) at the HIFI ICC for further details. In any case, it is advised not to use more than 3 components in FitHifiFringe. Since HIPE 5, a number of new features have been made available in FitHifiFringe. One of them ("sub_base" option) allows you to fit a baseline at the same time. Since FitHifiFringe can identify automatically the lines in order to set masks, users should be careful when treating lines with large wings. In this case, the user should set a window by hand. The automatic line masking should then be switched off.
 
    • In case of strong source continuum some standing waves can be enhanced. Such standing waves can be significantly reduced in amplitude using an alternative pipeline algorithm known as the Modified Passband Technique , which is described in the Standing Wave Removal chapter of the HIFI Data Reduction Guide. Note that in 9.0, a bug affected the doFilterLoad task in such a way that it ignored the user-input parameters, resulting in the task always applying the "cubic_spline" method with default parameters - i.e. the "fft" method will not be applied despite being set in the task inputs. This was fixed in 9.1 before the SPG 9.1.0 bulk reprocessing of the archive.

  • Baseline distortion
Changed:
<
<
    • Residual drift from the detector response can translate into imperfect baseline structures. This usually manifests as wavy structures, slopes, or overall level offsets of the spectral baselines. Such artefacts are expected to be enhanced e.g. at the borders of spectra obtained in diplexer bands (sub-bands 1 and 4 for bands 3 and 4, sub-bands 2 and 4 for bands 6 and 7). Similar artefacts are also expected when using modes without reference, such as the FreqSwitchNoRef, or the LoadChopNoRef, and are enhanced with strong continuum sources. Please refer to the Data Reduction Guide for details about the most recent improvements to the FitBaseline task.
>
>
    • Residual drift from the detector response can translate into imperfect baseline structures. This usually manifests as wavy structures, slopes, or overall level offsets of the spectral baselines. Such artefacts are expected to be enhanced e.g. at the borders of spectra obtained in diplexer bands (sub-bands 1 and 4 for bands 3 and 4, sub-bands 2 and 4 for bands 6 and 7). Similar artefacts are also expected when using modes without reference, such as the FreqSwitchNoRef, or the LoadChopNoRef, and are enhanced with strong continuum sources. Please refer to the HIFI Data Reduction Guide for details about the most recent improvements to the FitBaseline task.
 
  • Saturation
    • Saturation of the WBS CCDs may be observed at spot frequencies. These are usually due to either strong and broad spurs, or to cases where only marginal pump level could be achieved due to shortage of LO output power. Users can go back to HSpot to visualize in the frequency editor where their AORs were potentially affected by such effects. HIPE will normally flag the affected spectrometer channels.
Line: 381 to 384
 
    • The astrometry information provided in the meta-data of the various products has not always been accurate for HIFI data. This has now been fixed in the products generated by the bulk reprocessing with 8.2.1. Note that there is still a problem wit the moving targets, for which the reported ra/decNominal (i.e. the intended position at the time of observation) is not yet correct. If you observe an anomalous position reported in your (fixed target) data, please let us know via the helpdesk.

  • Quality flags in the quality product
Changed:
<
<
    • Currently, the quality flags at the quality context inside the observation context are just meant for HSC/ICC internal evaluation of the quality of the products and not for the users. In case the data had some serious quality problem, the PI of the program has been contacted about it. Otherwise, only information in the quality summary, when available, should concern the observers.
>
>
    • Up to HIPE 13, all quality flags were reported irrespective of their level of severity of applicability. As a consequence, some of them could lead to false positive that the users should in fact not worry about. A re-organisation of the flags into public and private (mostly informative to instrument scientists) and their level of criticality has been performed in HIPE 14, allowing the end user to filter more straightforwardly the relevant information.
 
  • Map visualisation and cubes
    • The visualisation software offered in HIPE 9 onwards has had significant improvements since the previous version, in particular it has been merged with the spectrum explorer. Note that there can be problems in the cubes generated in the case of single line (stripes) maps with HIPE 8.2. In those cases, the cube dimensions may not be correct. Also, cubes for raster maps may not be optimal when generated with HIPE versions earlier than 8.2. This issue is fixed from HIPE 9 onwards
Line: 400 to 403
 
  • Retrieval of calibration files from the HSA
    • In HIPE versions from the track 9, there is small bug in the getHifiCal task. The task may erroneously report that the latest calibration tree available is HIFI_CAL_9_0, while more recent versions are now available. This issue is solved in HIPE 10. This is not a problem with the HIPE 11 build. One possible work-around is to clear the calibration pool in your .hcss/lstore, and call the task again or simply use the updated HIPE version. Then the latest calibration tree should be offered for download.
Added:
>
>
 
  • Reprocessing of old products with the new HIPE 14.0 calibration tree from level 1
Changed:
<
<
    • Whenever you try to reprocess old products starting from levels as high as the level1 and want to apply a new calibration tree, the whole calibration products will be erased - this includes for example the System Temperature spectra stored in the pipeline-out products. As a consequence, for some modes like OTF-PSW or PSW, the OFF spectra, which relies on those Tsys spectra, will not be possible to be computed and an error message will be issues. You should either reprocess from level 0, or simply use the HIPE 14.0 products whenever they will become available.
>
>
    • Whenever you try to reprocess old products starting from levels as high as the level1 and want to apply a new calibration tree, the whole calibration products will be erased - this includes for example the System Temperature spectra stored in the pipeline-out products. As a consequence, for some modes like OTF-PSW or PSW, the OFF spectra, which relies on those Tsys spectra, will not be possible to be computed and an error message will be issued. You should either reprocess from level 0, or simply use the HIPE 14.0 products whenever they will become available.
 

Revision 1362015-12-11 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Data Products Known Issues

Line: 177 to 177
 
    • A dip can sometimes be seen at spectra between 62 and 63 microns. This is a filter feature. Its appearance depends on the angle at which the light from the sources goes through the filter, and so it depends on the source position and its spatial structure.

  • Spectral "fringing"
Changed:
<
<
    • You may notice a fringing-like pattern in the spectra from (wide) Range scan and full SED-mode observations. This is because the flatfielding is not done by the SPG pipeline. To remove it you need to reprocess the data in HIPE with one of the interactive pipeline scripts. For Line Spectroscopy observations, the spectral flatfielding is done in the SPG pipeline.
>
>
    • You may notice a fringing-like pattern in the spectra from (wide) Range scan and full SED-mode observations produced by SPG versions 13 and earlier. This is because the flatfielding is not done by the SPG pipeline in these versions of HIPE. To remove it you need to reprocess the data in HIPE with one of the interactive pipeline scripts, or download the SPG 14 products, for which the flatfielding is part of the SPG pipeline. For Line Spectroscopy observations, the spectral flatfielding is done in the SPG pipeline.
 
  • Check for contaminating flux in chop-off positions
    • To check for the presence of contamination from unwanted astronomical sources in the off positions of chopNod mode observations, you can use a Split On-Off pipeline script to produce an off-source and on-source cube. These cubes can then be compared to each other to check for contamination in spectral lines or by strong continuum emission, e.g. by over-plotting the respective spectra. Note that the on-source and off-source cubes produced by this task will not allow you to detect faint levels of contamination because wriggles from the RSRF are not removed by this process. For faint targets (line peak-to-continuum emission ~<5-10 Jy) you should also check the differential signal between the nodA and B on-source cubes. This is documented in the PACS Data Reduction Guide for spectroscopy. In this guide you can also find advice on checking for contamination in the unchopped mode observations, which is done either by comparing companion observations (unchopped range) or with a small script provided in the PDRG (unchopped line).
Line: 206 to 206
 
  • NaN's in the final cubes
    • It is normal to have NaNs at the very edges of the spectral ranges of SED mode observations: this is due to gaps in the spectral sampling.
Changed:
<
<
  • Flat-valued spectral ranges
    • The PACS rebinned cubes for range scan observations show interpolated nearly flat spectral values in the non-flatfielded spectral ranges for the central 3x3 spaxels. In SPG, these are the spectral ranges affected by leakage in the different bands. This is produced by the extractCentralSpectrum task. This task does a spectral interpolation over NaNs to avoid having jumps in summed spectra where one spaxel may have a NaN, and modifies accordingly the input rebinned cube. As an additional side effect, the interpolated cubes also show these extended flat-valued spectral ranges at some central locations.
>
>
  • Flat fluxes for certain spectral ranges
    • The pipeline task extractCentralSpectrum is used to created point-source calibrated spectra for pointed observations, working on the rebinned cubes to do this. This task does a spectral interpolation over NaNs (this is to avoid having jumps in summed spectra where one spaxel may have a NaN). The output spectra are improved as a result of this, however the central 9 spaxels of the input rebinned cubes are modified by this interpolation. The result of this modification is that for the rebinned cubes, the fluxes in the spectra in the ranges that were not flatfielded are nearly flat. In SPG 14-reduced observations, this affects those of the the pointed range scan mode, and specifically in the spectral ranges affected by leakage in the different bands. As an additional side effect, the interpolated cubes (which are created from the rebinned cubes) also show these extended flat-valued spectral ranges at some central locations.
 
  • Quality flags in the quality product
    • The quality flags in the quality context ("quality" or "qualitySummary") inside the observation context are meant for HSC/ICC internal evaluation of the quality of the products: when an observation had a serious quality problem, the PI of the program would have been contacted about it. For archive users, only the information in the "qualitySummary", when available, is useful.

Revision 1352015-12-10 - RengelMiriam

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Data Products Known Issues

Line: 343 to 343
  In order to obtain the best possible Level 2 HIFI data the observations should be reprocessed with the latest HIPE User Release.

  • Corrupted data-frames
Changed:
<
<
    • There is a limited number of obsids where one or more data-frames from the spectrometers is corrupted in some fashion and leads to level 2 products that cannot be properly calibrated. A scheme was put in place some versions ago to flag or simply remove those frames at level 0 so the final products won't be affected by them. This scheme however suffered from a regression bug in 11.1 so that the corresponding products did not benefit from this correction properly. We note however that this should only concern about 50 obsids in the whole HIFI archive. This will be fixed in HIPE 12.
>
>
    • There is a limited number of obsids where one or more data-frames from the spectrometers is corrupted in some fashion and leads to level 2 products that cannot be properly calibrated. A scheme was put in place some versions ago to flag or simply remove those frames at level 0 so the final products won't be affected by them. This scheme however suffered from a regression bug in 11.1 so that the corresponding products did not benefit from this correction properly. We note however that this should only concern about 50 obsids in the whole HIFI archive. This was fixed in HIPE 12.
 
  • Strong/Weak Spurs
Changed:
<
<
    • Spurs are still affecting the HIFI data at spot frequency points. They will be flagged automatically in the data using a SpurFinder task when present above a certain threshold in the internal load spectra. There are still some spur categories that are not yet properly detected but this is improving in each new versions of HIPE. Some spurs are so intense and broad that they can lead to the loss of complete WBS sub-bands. Note that the strong spur affecting the upper end of band 1a has now been completely cleaned - see the corresponding technical note: HIFI Information note on the removal of spurs in band 1a and the sampling in the mapping modes. The same is true for a weaker spur that did affect data in band 7b close to LO = 1834 GHz.
>
>
    • Spurs are still affecting the HIFI data at spot frequency points. They will be flagged automatically in the data using a SpurFinder task when present above a certain threshold in the internal load spectra. There are still some spur categories that are not yet properly detected but this is improving in each new versions of HIPE. Some spurs are so intense and broad that they can lead to the loss of complete WBS sub-bands. Note that the strong spur affecting the upper end of band 1a has now been completely cleaned - see the corresponding technical note: HIFI Information note on the removal of spurs in band 1a and the sampling in the mapping modes. The same is true for a weaker spur that did affect data in band 7b close to LO = 1834 GHz.
 
  • Unpumped data
    • Due to gaps in the LO output power over the HIFI frequency range, some spectral scans will contain spectra that cannot be used due their very high noise temperature levels. These data should be discarded from further processing (e.g. deconvolution). For single frequency observations, there exist some places where the theoretical noise temperature (i.e. the one advertised by HSpot) should be nominal, but the sensitivity effectively achieved at the time of observations is noticeably worse. These are cases where the mixer tuning algorithm does not fully converge. Such cases have e.g. been reported around LO frequencies of ~ 1441.5 GHz. An overview of the expected unpumped frequency areas is given in the AOT release notes. Users observing such effects should contact the Helpdesk.

  • Standing waves
Changed:
<
<
    • There are a variety of standing waves known to affect the data at Level2 - see the AOT release notes for more details. At the present time, the !FitHifiFringe tool usually does a decent job for most standing waves in SIS bands. In HEB bands, where the standing wave is not of optical nature and does have a strictly sinusoidal shape, FitHifiFringe can help when restricted to the frequency range where the line is present. Another alternative in the form of a dedicated algorithm has been developed (so-called "matching technique") and is offered in the hebCorrection task from HIPE 12.0 onwards (details can be found in Chapter 10 of the HIFI Data Reduction Guide). You can contact Ian Avruch (i.avruch@sron.nl) at the HIFI ICC for further details. In any case, it is advised not to use more than 3 components in FitHifiFringe. Since HIPE 5, a number of new features have been made available in FitHifiFringe. One of them ("sub_base" option) allows you to fit a baseline at the same time. Since FitHifiFringe can identify automatically the lines in order to set masks, users should be careful when treating lines with large wings. In this case, the user should set a window by hand. The automatic line masking should then be switched off.
>
>
    • There are a variety of standing waves known to affect the data at Level2 - see the AOT release notes for more details. At the present time, the !FitHifiFringe tool usually does a decent job for most standing waves in SIS bands. In HEB bands, where the standing wave is not of optical nature and does have a strictly sinusoidal shape, FitHifiFringe can help when restricted to the frequency range where the line is present. Another alternative in the form of a dedicated algorithm has been developed (so-called "matching technique") and is offered in the hebCorrection task from HIPE 12.0 onwards (details can be found in Chapter 10 of the HIFI Data Reduction Guide). You can contact Ian Avruch (i.avruch@sron.nl) at the HIFI ICC for further details. In any case, it is advised not to use more than 3 components in FitHifiFringe. Since HIPE 5, a number of new features have been made available in FitHifiFringe. One of them ("sub_base" option) allows you to fit a baseline at the same time. Since FitHifiFringe can identify automatically the lines in order to set masks, users should be careful when treating lines with large wings. In this case, the user should set a window by hand. The automatic line masking should then be switched off.
 
    • In case of strong source continuum some standing waves can be enhanced. Such standing waves can be significantly reduced in amplitude using an alternative pipeline algorithm known as the Modified Passband Technique , which is described in the Standing Wave Removal chapter of the HIFI Data Reduction Guide. Note that in 9.0, a bug affected the doFilterLoad task in such a way that it ignored the user-input parameters, resulting in the task always applying the "cubic_spline" method with default parameters - i.e. the "fft" method will not be applied despite being set in the task inputs. This was fixed in 9.1 before the SPG 9.1.0 bulk reprocessing of the archive.

  • Baseline distortion
Line: 403 to 403
 
    • Whenever you try to reprocess old products starting from levels as high as the level1 and want to apply a new calibration tree, the whole calibration products will be erased - this includes for example the System Temperature spectra stored in the pipeline-out products. As a consequence, for some modes like OTF-PSW or PSW, the OFF spectra, which relies on those Tsys spectra, will not be possible to be computed and an error message will be issues. You should either reprocess from level 0, or simply use the HIPE 14.0 products whenever they will become available.

  • Other calibration/pipeline problems/needs
Changed:
<
<
>
>
 

HIFI Release Notes

Revision 1342015-12-10 - ElenaPuga

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Data Products Known Issues

Line: 197 to 197
 
    • How to extract a correctly-calibrated spectrum of a point source is documented in the pipeline scripts and in HIPE. The necessary point source corrections must be applied for a correct spectrum to be produced.

  • Limitations on absolute spectrophotometric accuracy
Changed:
<
<
    • The PACS spectrometer flux calibration accuracy is limited by detector response drifts and slight pointing offsets arising from the standard 2" (1-sigma) pointing error occurring within each and every observation. These limit both the absolute flux accuracy and relative accuracy within a band. Various pipelines deal better with these than others (see the advice in the PACS Data Reduction Guide for spectroscopy) but they can never be entirely negated. Hence the calibration uncertainty for any particular observation is a combination of the general calibration uncertainties (given in the PACS Observers Manual), the noise on the spectrum (explained in more detail in the PDRG chp 7.6/HIPE 13), and the "activity" during any single observation.
>
>
    • The PACS spectrometer flux calibration accuracy is limited by detector response drifts and slight pointing offsets arising from the standard 1.2" (1-sigma) pointing error occurring within each and every observation. These limit both the absolute flux accuracy and relative accuracy within a band. Various pipelines deal better with these than others (see the advice in the PACS Data Reduction Guide for spectroscopy) but they can never be entirely negated. Hence the calibration uncertainty for any particular observation is a combination of the general calibration uncertainties (given in the PACS Observers Manual), the noise on the spectrum (explained in more detail in the PDRG chp 7.6/HIPE 13), and the "activity" during any single observation.
 

Revision 1332015-12-09 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Data Products Known Issues

Line: 397 to 397
 
  • Retrieval of calibration files from the HSA
    • In HIPE versions from the track 9, there is small bug in the getHifiCal task. The task may erroneously report that the latest calibration tree available is HIFI_CAL_9_0, while more recent versions are now available. This issue is solved in HIPE 10. This is not a problem with the HIPE 11 build. One possible work-around is to clear the calibration pool in your .hcss/lstore, and call the task again or simply use the updated HIPE version. Then the latest calibration tree should be offered for download.
Added:
>
>
  • Reprocessing of old products with the new HIPE 14.0 calibration tree from level 1
    • Whenever you try to reprocess old products starting from levels as high as the level1 and want to apply a new calibration tree, the whole calibration products will be erased - this includes for example the System Temperature spectra stored in the pipeline-out products. As a consequence, for some modes like OTF-PSW or PSW, the OFF spectra, which relies on those Tsys spectra, will not be possible to be computed and an error message will be issues. You should either reprocess from level 0, or simply use the HIPE 14.0 products whenever they will become available.
 

Revision 1322015-12-09 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Data Products Known Issues

HCSS, SPG, and HIPE

Changed:
<
<
The whole suite of Herschel software is known as the Herschel Common Science System (HCSS). This encompasses all the software aspects related to the mission: automatic pipelines, spacecraft calibrations, etc. As data are retrieved from the spacecraft they are ingested in the Herschel Science Archive (HSA) and processed with the current official version of the pipeline. That means that, at any given time, different data in the HSA may be reduced with different pipeline versions. The pipeline version is listed in the HSA GUI as SPG vX.X.X, where SPG stands for Standard Product Generator. The SPG version is available as the header keyword 'creator', in the data .fits files. Every HCSS version, the whole HSA is bulk re-processed with the same up-to-date version of the pipelines.
>
>
The whole suite of Herschel software is known as the Herschel Common Science System (HCSS). This encompasses all the software aspects related to the mission: automatic pipelines, spacecraft calibrations, etc. As data are retrieved from the spacecraft they are ingested in the Herschel Science Archive (HSA) and processed with the current official version of the pipeline. That means that, at any given time, different data in the HSA may be reduced with different pipeline versions. The pipeline version is listed in the HSA GUI as SPG vX.X.X, where SPG stands for Standard Product Generator. The SPG version is available as the header keyword 'creator', in the data .fits files. Every HCSS version, the whole HSA is bulk re-processed with the same up-to-date version of the pipelines.
  From the user point of view, the most important piece of the HCSS system is the Herschel Interactive Processing Environment (HIPE). HIPE allows the astronomer the possibility of inspecting the data and re-process them, if the results from the automatic pipeline are not good enough for his/her purposes. Because HIPE is part of the HCSS, the latest version of HIPE will have the most up-to-date pipeline, calibrations and documentation available.
Line: 15 to 14
  In what follows, we provide a summary of the known issues that you may encounter when inspecting data processed with the automatic pipelines SPG versions 6.1 to 13.0. Most can be resolved by running the pipelines within HIPE and optimizing their parameters as explained below.
Changed:
<
<
<!--This is a comment: it will be ignored by the browser-->
>
>
<--This is a comment: it will be ignored by the browser-->
  Note that some of this information can also be found in the quality report of the observation (QC Report) and as metadata with the FITS keyword "PCAVEATS".

TABLE OF CONTENTS

Changed:
<
<

>
>
 

METADATA AND FITS KEYWORDS

Message about DATE-OBS

Changed:
<
<
Some internal HCSS metadata are renamed by the HCSS software when translating to FITS. One special case is startDate which gets written to FITS as both DATE-OBS and DATE_OBS. This is done for compatibility with legacy Spitzer software (namely MOPEX).
>
>
Some internal HCSS metadata are renamed by the HCSS software when translating to FITS. One special case is startDate which gets written to FITS as both DATE-OBS and DATE_OBS. This is done for compatibility with legacy Spitzer software (namely MOPEX).
 
Changed:
<
<

PACSnbsp;Photometernbsp;(scannbsp;mapping)
>
>

PACS Photometer (scan mapping)

 

Special messages

Changed:
<
<
Warning for Unimap maps in HIPE/SPG 13: no longer relevant for SPG/HIPE 14
>
>
Warning for Unimap maps in HIPE/SPG 13: no longer true for HIPE/SPG 14
  Some Unimap maps show overshooting effects around very bright sources that are surrounded by a diffuse and relatively faint background. See, as an example, the blue image of NGC253 (obsID: 1342221743).
Changed:
<
<
This effect is due to a non-optimised convergence of the GLS algorithm and has been corrected in HIPE/SPG14.
>
>
This effect is due to a non-optimised convergence of the GLS algorithm and it has been corrected in HIPE/SPG14.
 The user can request for a dedicated reprocessing to the Herschel Science Center Helpdesk or they can use the corresponding Level 2.5 JScanam maps (HPPJSMAP[B][R] blue and red products), which are not affected by this issue.
Changed:
<
<
Warning for pointing anomaly in HIPE/SPG 13 and 14
>
>
Warning for pointing anomaly in HIPE/SPG 13/14
  PACS and SPIRE photometry observations reported in this CSV file are affected by a known problem related to the reset of the Spacecraft Velocity Vector (SVV) during the upload of the star-tracker's defective pixel table. For the affected observations, the pointing of the telescope can be off along the scan direction, and shifted up to 20 arcsec.

General notes

Changed:
<
<
  • To produce the highest quality maps, you should consider re-processing or fine-tuning the observations with the [[http://herschel.esac.esa.int/HIPE_download.shtml][
latest HIPE User Release]]. Maps available from the HSA are created within a bulk processing framework, and a reprocessing while fine-tuning the mapper parameters, according to the characteristics of the observed sky region, could enhance the quality of the final maps.
>
>
  • To produce the highest quality maps, you should consider re-processing or fine-tuning the observations with the latest HIPE User Release. Maps available from the HSA are created within a bulk processing framework, and a reprocessing while fine-tuning the mapper parameters, according to the characteristics of the observed sky region, could enhance the quality of the final maps.
 
  • Important note:
    • PACS maps from any map-maker are in essence differential maps: the absolute level is undefined due to the dominating telescope background removed by map-makers. Hence it is not unusual if your background level is negative.
Line: 58 to 52
 
  • Intrinsic limitation of highpass filter maps
    • High-pass filtered maps in SPG13 are available at level 2 and level 2.5. The extended emission is filtered out in the maps since a rather small filter width is used to remove 1/f noise stripping: this is done to allows us to obtain the best sensitivity for point-sources. Bright sources are masked out during the highpass filtering, hence their flux is not too much affected by filtering. But faint sources are not or are inadequately masked out by the SPG processing, hence the flux loss for these point-source can reach up to 20--30%. Re-processing your data is recommended here.
Changed:
<
<
<!--
      * These are due to a non-optimal estimate of background by the high-pass filter close to a bright source. The current masked high-pass filter pipeline is a trade-off, designed to get a good sensitivity on point-sources and preserve some extended emission up to a few arcmin scale. However due to the relatively large width in the second high-pass filtering, significant stripping from the 1/f noise is still present in the level 2 maps. On the other hand a large fraction of the extended emission is filtered out. The observer is advised to play with the threshold to define the mask and with the width of the high-pass filter to reduce these effects or move to MADmap scanamorphos to preserve extended emission at all scales. Level 2.5 MADmap maps, combining scans and crossed-scans on the other hands do preserve extended emission at all spatial scales.
-->
>
>
<-- 
    • These are due to a non-optimal estimate of background by the high-pass filter close to a bright source. The current masked high-pass filter pipeline is a trade-off, designed to get a good sensitivity on point-sources and preserve some extended emission up to a few arcmin scale. However due to the relatively large width in the second high-pass filtering, significant stripping from the 1/f noise is still present in the level 2 maps. On the other hand a large fraction of the extended emission is filtered out. The observer is advised to play with the threshold to define the mask and with the width of the high-pass filter to reduce these effects or move to MADmap scanamorphos to preserve extended emission at all scales. Level 2.5 MADmap maps, combining scans and crossed-scans on the other hands do preserve extended emission at all spatial scales.
-->
 
  • MADmap issues
    • MADmap -- a GLS (Generalized Least Square) map-maker -- maps are no longer created by SPG13 and beyond. However, the pipeline script to do this yourself is provided in HIPE 13.
Line: 87 to 83
 
    • One matrix (fully half of) of the red channel array was lost at the end of the mission, so from OD1375 onwards this matrix is masked out automatically in the SPG processing.
    • The quality flags in the quality context ("quality" or "qualitySummary") inside the observation context are meant for HSC/ICC internal evaluation of the quality of the products: when an observation had a serious quality problem, the PI of the program would have been contacted about it. For archive users, only the information in the "qualitySummary", when available, is useful.
Changed:
<
<
<!---
   * *Calibration block transient in L2 maps*
      * Several red (160um) scan maps are affected by a calibration block downwards transient when scheduled immediately before the science observation. This can be mitigated by running a high-pass filter with a smaller width, fitting the transient or just masking the affected frames at the start of the observation.
-->
>
>
<--- 
  • Calibration block transient in L2 maps
    • Several red (160um) scan maps are affected by a calibration block downwards transient when scheduled immediately before the science observation. This can be mitigated by running a high-pass filter with a smaller width, fitting the transient or just masking the affected frames at the start of the observation.
-->

<--      * Scan maps processed with SPG v4.1.0 were deglitched with "MMT deglitching", a temporal deglitching of the pixel timelines. This deglitching technique works very well for deep fields (e.g. cosmological fields) but fails at high scan speed (60"/s) or even medium scan speeds on bright (>~1Jy) sources, wrongly identified as glitches.
-->

<--   * Deglitching 
    • The deglitching was changed to a so-called "2nd level deglitching" in SPG v6.1.0 that makes use of the spatial redundancy (a sky pixel being seen by several detector pixel readouts). Large scan maps (e.g. galactic fields) processed with SPG v6.1 are therefore of much better quality than earlier ones (SPG v4.1.0), however some low level glitches are left as a high threshold was set in the pipeline. As the deglitching is now run from level 1 to level 2 after highpass filtering, the level 1 cubes in HSA processed with SPG 6.1 do not contain anymore glitch masks. For interactive processing, several hints and methods are given in the ipipe scripts.
-->

<-- 
  • Brightest sources core deglitched in the Level 2 map processed with old pipeline version (SPG v4.1.0)
    • This is due to the MMT deglitching at high speed (60"/s), which can wrongly identify bright sources as glitches. A possible solution in these cases is to use 2nd order deglitching in the interactive pipeline, now the default in the HCSS pipeline.

  • Glitches from Cosmic ray hits
    • Level 2 maps in the archive (HSA) processed with SPG 4.1.0 are affected by glitches as a side effect of disabling the deglitching on bright sources. This has been corrected in SPG version 6.1.0 and above. Interactive HIPE sessions shall get rid off of all these glitches by playing with the deglitching thresholds or using the 2nd order deglitching (memory consuming).
    • At 20"/s scan speed, the MMT deglitching does fine if there are no bright sources (>~1Jy), for instance for cosmological survey observations. For brighter sources (nearby galaxies or galactic fields) it is better to switch to second order deglitching. For more information see the PACS Data Reduction Guide (PDRG).
-->

<--
The astrometry of several PACS scan maps acquired between ODs 320 and 761 has been reported to be off by 4 arcsec (solid offset of the whole map) or even above for a few fields where the tracking stars are not homogeneously distributed in the star-tracker field-of-view. It is intended to improve the a posteriori reconstructed pointing for all observations in the future.
-->

<-- 
    • The absolute astrometry of Level 2 maps on Solar System Objects (SSO), projected in the SSO reference frame is not reliable and can be off up to 20 arcsec. This a data processing issue only (aberration & light travel time correction), while the observations themselves were correctly performed (uplink). This issue is under investigations at HSC and PACS ICC.
-->

<-- 
    • Full noise propagation in the HSC pipelines will be available at later HCSS versions, so the current noise map is relatively but not absolutely correct. Furthermore, the default pipeline results in correlated noise in individual pixels. In order to estimate a proper background standard deviation, the observers are advised to make 10+ aperture photometry measurements in different sky patches around the source and to estimate the absolute standard deviation of the image as the sigma of those photometric points.
-->
 
Changed:
<
<
<!--      * Scan maps processed with SPG v4.1.0 were deglitched with "MMT deglitching", a temporal deglitching of the pixel timelines. This deglitching technique works very well for deep fields (e.g. cosmological fields) but fails at high scan speed (60"/s) or even medium scan speeds on bright (>~1Jy) sources, wrongly identified as glitches.
-->

<!--   * *Deglitching*
      * The deglitching was changed to a so-called "2nd level deglitching" in SPG v6.1.0 that makes use of the spatial redundancy (a sky pixel being seen by several detector pixel readouts). Large scan maps (e.g. galactic fields) processed with SPG v6.1 are therefore of much better quality than earlier ones (SPG v4.1.0), however some low level glitches are left as a high threshold was set in the pipeline. As the deglitching is now run from level 1 to level 2 after highpass filtering, the level 1 cubes in HSA processed with SPG 6.1 do not contain anymore glitch masks. For interactive processing, several hints and methods are given in the ipipe scripts.
-->

<!--
   * *Brightest sources core deglitched in the Level 2 map processed with old pipeline version (SPG v4.1.0)*
      * This is due to the MMT deglitching at high speed (60"/s), which can wrongly identify bright sources as glitches. A possible solution in these cases is to use 2nd order deglitching in the interactive pipeline, now the default in the HCSS pipeline.

   * *Glitches from Cosmic ray hits*
      * Level 2 maps in the archive (HSA) processed with SPG 4.1.0 are affected by glitches as a side effect of disabling the deglitching on bright sources. This has been corrected in SPG version 6.1.0 and above. Interactive HIPE sessions shall get rid off of all these glitches by playing with the deglitching thresholds or using the 2nd order deglitching (memory consuming).
      * At 20"/s scan speed, the MMT deglitching does fine if there are no bright sources (>~1Jy), for instance for cosmological survey observations. For brighter sources (nearby galaxies or galactic fields) it is better to switch to second order deglitching. For more information see the PACS Data Reduction Guide (PDRG).
-->

<!--
The astrometry of several PACS scan maps acquired between ODs 320 and 761 has been reported to be off by 4 arcsec (solid offset of the whole map) or even above for a few fields where the tracking stars are not homogeneously distributed in the star-tracker field-of-view. It is intended to improve the a posteriori reconstructed pointing for all observations in the future.
-->

<!--
      * The absolute astrometry of Level 2 maps on Solar System Objects (SSO), projected in the SSO reference frame is not reliable and can be off up to 20 arcsec. This a data processing issue only (aberration & light travel time correction), while the observations themselves were correctly performed (uplink). This issue is under investigations at HSC and PACS ICC.
-->

<!--
      * Full noise propagation in the HSC pipelines will be available at later HCSS versions, so the current noise map is relatively but not absolutely correct. Furthermore, the default pipeline results in correlated noise in individual pixels. In order to estimate a proper background standard deviation, the observers are advised to make 10+ aperture photometry measurements in different sky patches around the source and to estimate the absolute standard deviation of the image as the sigma of those photometric points. 
-->

>
>
 
Line: 110 to 129
  PACS photometer AOT release notes see PACS Instrument and Calibration webpage
Changed:
<
<
<!---
<a class="red" href="http://herschel.esac.esa.int/Docs/AOTsReleaseStatus/PACS_ScanMap_ReleaseNote_23Feb2010.pdf">PACS Scan Map AOT release note</a>: 23 Feb 2010

<a class="red" href="http://herschel.esac.esa.int/Docs/AOTsReleaseStatus/PACS_PhotMiniScan_ReleaseNote_12Nov2010.pdf">PACS Photometer - Point/Compact Source Observations: Mini Scan-Maps &amp; Chop-Nod AOT release note</a>: 12 Nov 2010
-->
>
>
<---
PACS Scan Map AOT release note: 23 Feb 2010

PACS Photometer - Point/Compact Source Observations: Mini Scan-Maps & Chop-Nod AOT release note: 12 Nov 2010 -->

 
Changed:
<
<

PACSnbsp;Spectroscopy
>
>

PACS Spectroscopy

 

Special messages

Added:
>
>
Warning for drizzled cubes from HIPE/SPG 13: no longer relevant in HIPE/SPG 14

Due to an oversight in the pipeline scripts of HIPE 13, drizzled cubes for chop-nod line scan observations created by "SPG 13" have incorrect fluxes. Therefore you should not use the drizzled cubes downloaded from the HSA if the Meta datum "creator" is "SPG 13.0", whether they are within the ObservationContext at Level 2, or part of a Standalone Browse Product download.

Any drizzled cubes created for unchopped mode observations will be unaffected. For chop-nod range scan observations, drizzled cubes are not created by the SPG 13 pipeline anyway.

This oversight has been corrected in HIPE/SPG 14. The drizzled cubes you get in an observation downloaded from the HSA with "creator" of "SPG 14.0" have correct fluxes. In HIPE 14 there is a dedicated script to produce these drizzled cubes. Therefore we recommend you use the SPG 14.0 products, or use the dedicated pipeline in HIPE 14 if you need your drizzled cubes before the SPG 14 products are available.

FYI: Drizzled cubes can be found in an observation in the context called HPS3DD [R|B] (red and blue). They are also provided as standalone browse products in a context called HPS3DEQ [R|B] (red and blue, equidistant wavelength grid version of the drizzled cubes). On disk the FITS files for these cubes have the same set of letters in their name (but in lower case) and are at the Level 2 part of an observation.

For more information about the standard and the standalone browse cubes provided for PACS spectroscopy, see the PACS Products Explained HIPE help document, which is also available from the PACS documentation webpage and the HIPE download webpage, both off the Herschel Science Centre webpage.

 General notes
Changed:
<
<
  • To produce the highest quality cubes possible, you should consider re-processing or fine-tuning the observations with the [[http://herschel.esac.esa.int/HIPE_download.shtml][
latest HIPE User Release]]. Cubes available from the HSA are created within a bulk processing framework, and a reprocessing while fine-tuning the important pipeline task parameters, according to the characteristics of the observation and source, could enhance the quality of the final results. The first two chapters of the PACS Data Reduction Guide for spectroscopy (HIPE 13) give information about the need to reprocess, and about what to do with HSA-obtained cubes before using them for science.
>
>

 
Changed:
<
<
<!--
   * *Off-subtraction for unchopped long-range scan observations*
      * Range Spectroscopy unchopped observation require off-position scans observed in a separate AOR. The off-source observation need to be subtracted from the on-scan after producing Level 2 rebinned spectra, this results a Level 2.5 product. The pipeline only generates Level 2 data products for unchopped spectroscopy in SPG 8.0. You need to combine interactively the on and off positions using the dedicated multi-observation unchopped pipeline script under the PACS pipeline menu.
-->
>
>
  • To produce the highest quality cubes possible, you should consider re-processing or fine-tuning the observations with the latest HIPE User Release. Cubes available from the HSA are created within a bulk processing framework, and a reprocessing while fine-tuning the important pipeline task parameters, according to the characteristics of the observation and source, could enhance the quality of the final results. The first two chapters of the PACS Data Reduction Guide for spectroscopy (HIPE 13) give information about the need to reprocess, and about what to do with HSA-obtained cubes before using them for science.
 
Changed:
<
<
<!--
   * *Unchopped grating scan flux calibration*
      * The absolute flux calibration of the PACS spectrometer is based on observations of flux calibration standards using chopped spectroscopy modes. There are hints of systematic differences in the response scaling between chopped and unchopped mode due to response transients within the chopping pattern. In SPG 8.0 the flux calibration of unchopped data relies on the system response derived chopped scheme, therefore absolute flux values need to be carefully interpreted. Please contact Helpdesk for guidelines on the specific observation you have to deal with.
-->
>
>
<-- 
  • Off-subtraction for unchopped long-range scan observations
    • Range Spectroscopy unchopped observation require off-position scans observed in a separate AOR. The off-source observation need to be subtracted from the on-scan after producing Level 2 rebinned spectra, this results a Level 2.5 product. The pipeline only generates Level 2 data products for unchopped spectroscopy in SPG 8.0. You need to combine interactively the on and off positions using the dedicated multi-observation unchopped pipeline script under the PACS pipeline menu.
-->

<-- 
  • Unchopped grating scan flux calibration
    • The absolute flux calibration of the PACS spectrometer is based on observations of flux calibration standards using chopped spectroscopy modes. There are hints of systematic differences in the response scaling between chopped and unchopped mode due to response transients within the chopping pattern. In SPG 8.0 the flux calibration of unchopped data relies on the system response derived chopped scheme, therefore absolute flux values need to be carefully interpreted. Please contact Helpdesk for guidelines on the specific observation you have to deal with.
-->
 
  • Spectral leakage
Changed:
<
<
    • The order selection filters of the PACS spectrometer have a steep but not perfectly vertical transmission profile at the cut­off wavelengths of the spectral bands. PACS spectra near the band borders of bands R1, B3A and B2B are therefore affected by higher- or lower-order wavelengths leaking into the spectra -- both continuum and spectral lines! Consult the [[http://herschel.esac.esa.int/twiki/pub/Public/PacsCalibrationWeb/PacsSpectroscopyPerformanceAndCalibration_v2_4.pdf][
PACS Calibration Document]] for more information on the leakage regions.
>
>
    • The order selection filters of the PACS spectrometer have a steep but not perfectly vertical transmission profile at the cut­off wavelengths of the spectral bands. PACS spectra near the band borders of bands R1, B3A and B2B are therefore affected by higher- or lower-order wavelengths leaking into the spectra -- both continuum and spectral lines! Consult the PACS Calibration Document for more information on the leakage regions.
 
    • For the leak in band R1, it is possible to reduce these data with a specific relative spectral response function calibration file in HIPE to obtain correct line fluxes (but an incorrect continuum level). How to do this is explained in the PACS Data Reduction Guide for spectroscopy (chps 5 and 6 in the HIPE 13 version). This RSRF is not applied by default since it increases the noise in the resulting spectrum, but can be used interactively within HIPE.

  • Second-pass ghosts
Changed:
<
<
    • A second pass in the optics of the PACS spectrometer can cause a ghost image to appear on most spaxels (but never in the central spaxel). If a source located in one of these "originating" spaxels shows a strong spectral line (typically an atomic fine-structure line), then a weak, broadened line can be seen at an offset wavelength in its corresponding "destination" spaxel affected by the 2nd-pass ghost. The peak flux of this line is typically ~5% of the peak of the originating line. The most prominent ghost is the 122 micron feature, which originates from the usually strong CII+ 157.7 micron line. A list of strong ghosts and an image showing the directions of the projected passes on the 5x5 IFU footprint can be found in the [[http://herschel.esac.esa.int/twiki/pub/Public/PacsCalibrationWeb/PacsSpectroscopyPerformanceAndCalibration_v2_4.pdf][
PACS Calibration Document]].
>
>
    • A second pass in the optics of the PACS spectrometer can cause a ghost image to appear on most spaxels (but never in the central spaxel). If a source located in one of these "originating" spaxels shows a strong spectral line (typically an atomic fine-structure line), then a weak, broadened line can be seen at an offset wavelength in its corresponding "destination" spaxel affected by the 2nd-pass ghost. The peak flux of this line is typically ~5% of the peak of the originating line. The most prominent ghost is the 122 micron feature, which originates from the usually strong CII+ 157.7 micron line. A list of strong ghosts and an image showing the directions of the projected passes on the 5x5 IFU footprint can be found in the PACS Calibration Document.
 
  • 62 micron dip
    • A dip can sometimes be seen at spectra between 62 and 63 microns. This is a filter feature. Its appearance depends on the angle at which the light from the sources goes through the filter, and so it depends on the source position and its spatial structure.
Line: 143 to 182
 
  • Check for contaminating flux in chop-off positions
    • To check for the presence of contamination from unwanted astronomical sources in the off positions of chopNod mode observations, you can use a Split On-Off pipeline script to produce an off-source and on-source cube. These cubes can then be compared to each other to check for contamination in spectral lines or by strong continuum emission, e.g. by over-plotting the respective spectra. Note that the on-source and off-source cubes produced by this task will not allow you to detect faint levels of contamination because wriggles from the RSRF are not removed by this process. For faint targets (line peak-to-continuum emission ~<5-10 Jy) you should also check the differential signal between the nodA and B on-source cubes. This is documented in the PACS Data Reduction Guide for spectroscopy. In this guide you can also find advice on checking for contamination in the unchopped mode observations, which is done either by comparing companion observations (unchopped range) or with a small script provided in the PDRG (unchopped line).
Changed:
<
<
<!--
   * *RSRF at wavelengths below 53 microns*
      * The relative spectral response function (used by the pipeline task rsrfCal) is an extrapolation at wavelengths below 53 microns. This will cause problems in the spectra from module 3 (=spaxel 3,0, i.e. in the cube image you see when you look at a PacsCube or PacsRebinnedCube with the Standard Cube Viewer or the Spectrum Explorer, it is the 4th up and on the very left): the extrapolation is too steep and makes the pixel-spectra similarly follow a very steep curve. This is being corrected.
-->
>
>
<-- 
  • RSRF at wavelengths below 53 microns
    • The relative spectral response function (used by the pipeline task rsrfCal) is an extrapolation at wavelengths below 53 microns. This will cause problems in the spectra from module 3 (=spaxel 3,0, i.e. in the cube image you see when you look at a PacsCube or PacsRebinnedCube with the Standard Cube Viewer or the Spectrum Explorer, it is the 4th up and on the very left): the extrapolation is too steep and makes the pixel-spectra similarly follow a very steep curve. This is being corrected.
-->
 
  • Unstable/incorrect broad-band (dust) features
    • Broad spectral features (of a few micrometer width) and continuum shape variations can be introduced by transient effects (for chopNod mode and more so for unchopped mode observations) and by pointing offsets distorting the Relative Spectral Response Function. The "background normalisation" pipeline script for chopNod observations is recommended for observations looking for such features, as it minimises the effect -- this is the SPG pipeline for HIPE 13 and onwards. For unchopped mode observations you could reprocess the observation with the "transients correction" pipeline script that is new to HIPE 13. However, neither of these pipelines will completely negate the effect of transients and pointing jitter-induced RSRF distortions.

  • Spectral line profiles: the skew
Changed:
<
<
    • As for any slit-spectrograph, if the incoming light beam is neither homogeneously nor symmetrically illuminating a spaxel, then line profiles may be distorted from the ideal Gaussian shape. In case of a point-source observed with PACS, the spectral line profile develops a skew, increasing as the offset of the photocentre along the instrument Z-axis (perpendicular the slit direction) does. Examples of skewed line shapes are given in the [[http://herschel.esac.esa.int/Docs/PACS/html/pacs_om.html][
PACS Observers Manual]].
>
>
    • As for any slit-spectrograph, if the incoming light beam is neither homogeneously nor symmetrically illuminating a spaxel, then line profiles may be distorted from the ideal Gaussian shape. In case of a point-source observed with PACS, the spectral line profile develops a skew, increasing as the offset of the photocentre along the instrument Z-axis (perpendicular the slit direction) does. Examples of skewed line shapes are given in the PACS Observers Manual.
 
  • Point sources
    • How to extract a correctly-calibrated spectrum of a point source is documented in the pipeline scripts and in HIPE. The necessary point source corrections must be applied for a correct spectrum to be produced.

  • Limitations on absolute spectrophotometric accuracy
Changed:
<
<
    • The PACS spectrometer flux calibration accuracy is limited by detector response drifts and slight pointing offsets arising from the standard 2" (1-sigma) pointing error occurring within each and every observation. These limit both the absolute flux accuracy and relative accuracy within a band. Various pipelines deal better with these than others (see the advice in the PACS Data Reduction Guide for spectroscopy) but they can never be entirely negated. Hence the calibration uncertainty for any particular observation is a combination of the general calibration uncertainties (given in the [[http://herschel.esac.esa.int/Docs/PACS/html/pacs_om.html][
PACS Observers Manual]]), the noise on the spectrum (explained in more detail in the PDRG chp 7.6/HIPE 13), and the "activity" during any single observation.
>
>
    • The PACS spectrometer flux calibration accuracy is limited by detector response drifts and slight pointing offsets arising from the standard 2" (1-sigma) pointing error occurring within each and every observation. These limit both the absolute flux accuracy and relative accuracy within a band. Various pipelines deal better with these than others (see the advice in the PACS Data Reduction Guide for spectroscopy) but they can never be entirely negated. Hence the calibration uncertainty for any particular observation is a combination of the general calibration uncertainties (given in the PACS Observers Manual), the noise on the spectrum (explained in more detail in the PDRG chp 7.6/HIPE 13), and the "activity" during any single observation.
 
Changed:
<
<
<!--
   * *Line flux correction due to strong wings in the instrumental profile (IP)*
      * Pre-flight ground-based monochromatic measurements indicated the PACS spectrometer instrumental profile distributes measurable power in spectral line wings. The effect is below ~10% and only noticeable for wavelengths longer than ~150 micrometers. Future HIPE releases will provide a correction factor to apply on a Gaussian fit in order to compensate for line power lost in the IP wings.
-->
>
>
<-- 
  • Line flux correction due to strong wings in the instrumental profile (IP)
    • Pre-flight ground-based monochromatic measurements indicated the PACS spectrometer instrumental profile distributes measurable power in spectral line wings. The effect is below ~10% and only noticeable for wavelengths longer than ~150 micrometers. Future HIPE releases will provide a correction factor to apply on a Gaussian fit in order to compensate for line power lost in the IP wings.
-->
 
  • NaN's in the final cubes
    • It is normal to have NaNs at the very edges of the spectral ranges of SED mode observations: this is due to gaps in the spectral sampling.
Line: 173 to 216
 

PACS Spectrometer Release Notes

PACS spectrometer AOT release notes see PACS Instrument and Calibration webpage

Added:
>
>

SPIRE Photometry

 

Special messages

Line: 188 to 237
 

General notes

Changed:
<
<
  • *In order to obtain the best possible Level 2 SPIRE photometry data, the observations might have to be reprocessed with the [[http://herschel.esac.esa.int/HIPE_download.shtml][
latest HIPE User Release]].*
Warning, important Please note that there was a bug in the destriper task included in HIPE 9.0 that may affect your final map, especially if there are bright objects in the observed field. This has been corrected since HIPE 9.1. If your observation falls in the mentioned category, you are strongly advised to update your HIPE installation.
Warning, important In HIPE 10.0 the flagging of thermistor jumps in the level 0.5 to 1 data reduction is not set properly. This induces the destriper to work improperly and to leave stripes in the final map. It has been solved starting with HIPE 10.1.
>
>
  • In order to obtain the best possible Level 2 SPIRE photometry data, the observations might have to be reprocessed with the latest HIPE User Release.
    • Warning, important Please note that there was a bug in the destriper task included in HIPE 9.0 that may affect your final map, especially if there are bright objects in the observed field. This has been corrected since HIPE 9.1. If your observation falls in the mentioned category, you are strongly advised to update your HIPE installation.
    • Warning, important In HIPE 10.0 the flagging of thermistor jumps in the level 0.5 to 1 data reduction is not set properly. This induces the destriper to work improperly and to leave stripes in the final map. It has been solved starting with HIPE 10.1.
 
  • SPIRE-P level 2.5 and level 3 maps
Changed:
<
<
    • These new levels will be available in the Herschel Science Archive when the observations will be bullk-reprocessed with HIPE v11. The useful user script
Photometer_MapMerge.py can be used to make level 2.5 (parallel mode) or level 3 (mosaic) maps, see Section 5.8.3 in the SPIRE Data Reduction Guide.
>
>
    • These new levels will be available in the Herschel Science Archive when the observations will be bullk-reprocessed with HIPE v11. The useful user script Photometer_MapMerge.py can be used to make level 2.5 (parallel mode) or level 3 (mosaic) maps, see Section 5.8.3 in the SPIRE Data Reduction Guide.
 
  • Stripes in PSW, PMW and/or PLW (Level 2) maps
    • All SPIRE photometry pipelines now use by default the destriper, which improves the issue of striping in level 2 maps. Hence observers should expect potential improvements in that respect with version 9.
Changed:
<
<
<!--
      * Most of the stripes that are present in the final maps are due to a combination of thermal drifts (which in few cases are not efficiently removed) and median baseline subtraction. A similar effect is caused by very bright sources: in this case, the problem resides in the median baseline subtraction only. Suggested solutions:
         * switch to a baseline subtraction using a polynomial fitting using the optional task =baselineRemovalPolynomial=. If there are no jumps in the timelines, you may also try to run the baseline removal on the entire timeline;
         * in the case of bright sources, you may try to mask them before running the baseline removal (either median or polynomial): you can use this script as a template
-->
>
>
<-- 
    • Most of the stripes that are present in the final maps are due to a combination of thermal drifts (which in few cases are not efficiently removed) and median baseline subtraction. A similar effect is caused by very bright sources: in this case, the problem resides in the median baseline subtraction only. Suggested solutions:
      • switch to a baseline subtraction using a polynomial fitting using the optional task baselineRemovalPolynomial. If there are no jumps in the timelines, you may also try to run the baseline removal on the entire timeline;
      • in the case of bright sources, you may try to mask them before running the baseline removal (either median or polynomial): you can use this script as a template
-->
 
  • De-glitchter masks faint sources
    • The de-glitcher is a very delicate process. In particular, for data taken in Parallel Mode (sampling at 10Hz) and at high speed (60"/s) the de-glitcher with standard parameters may flag very faint sources as glitches. Bright sources are different from glitches in that they have a gaussian (i.e. beam/PSF) shape. For faint sources, the sampling rate could be not high enough and hence they have a "delta" shape, which is similar to a small glitch. The user might try to modify the correlation parameter to 0.95: this will decrease the number of detected glitches.
Line: 211 to 259
 
  • Some sources have saturated the ADC and the corresponding data have been masked
    • There is nothing a user can do: the source was simply too bright. If the user has other sources still not observed and of the same intensity, it is suggested to change the AORs to use the bright source mode.
Changed:
<
<
<!--   * *Thermistor jumps*
      * As of HIPE 6.0.3, a new module together called =signalJumpDetector= in place to identify the jump and to exclude the affected thermistor(s). 
-->
>
>
<--   * Thermistor jumps 
    • As of HIPE 6.0.3, a new module together called signalJumpDetector in place to identify the jump and to exclude the affected thermistor(s).
-->
 
  • Cooler temperature variations
Changed:
<
<
    • The cooler temperature variations, as explained in greater details in the SPIRE Data Reduction Guide, section 6.4, can affect observations performed soon after the cooler recycle. The steep rise of the sub-K detector temperature is also known as the cooler burp and there is a quality flag
coolerBurpDetected in HIPE v11 or later that indicates if the observation was performed during this period.
InfoThe current list of observations known to have cooler temperature effects is here. Note that not all observations in this list raised the coolerBurpDetected flag.
>
>
    • The cooler temperature variations, as explained in greater details in the SPIRE Data Reduction Guide, section 6.4, can affect observations performed soon after the cooler recycle. The steep rise of the sub-K detector temperature is also known as the cooler burp and there is a quality flag coolerBurpDetected in HIPE v11 or later that indicates if the observation was performed during this period.
      InfoThe current list of observations known to have cooler temperature effects is here. Note that not all observations in this list raised the coolerBurpDetected flag.
 
  • NaNs pixels present in the PSW, PMW and/or PLW (Level 2) maps
    • This effect, related to data masked for various reasons and poor coverage (not enough redundancy), is more evident in single fast-scan Parallel Mode maps. To avoid NaNs, increase the pixel's dimension (i.e., decrease the map's resolution)
Changed:
<
<
<!--   * *WCS in 3-colour images*
      * In all observation reduced with HIPE 8, the task =createRgbImage= puts wrong WCS in the output. Instead of using the WCS provided by the WCS input parameter, this task uses the WCS of one of the input images. This has been fixed in HIPE 9
-->
>
>
<--   * WCS in 3-colour images 
    • In all observation reduced with HIPE 8, the task createRgbImage puts wrong WCS in the output. Instead of using the WCS provided by the WCS input parameter, this task uses the WCS of one of the input images. This has been fixed in HIPE 9
-->
 
  • Quality flags
    • Currently, the quality flags at the quality context inside the observation context are just meant for HSC/ICC internal evaluation of the quality of the products and not for the users. In case the data had some serious quality problem, the PI of the program has been contacted about it. Otherwise, only information in the quality summary, when available, should concern the observers.

  • Planck derived zero offsets
Changed:
<
<
    • The extended calibrated maps (
extdPxW in level-2, 2.5 or 3) incorporate zero level offsets derived from Planck-HFI. For small size SPIRE maps, smaller than ~30 arcmin, the zero-offset can be rather uncertain, due to the large Planck beam (8 arcmin). In such cases the interpretation of the zero offset as the absolute zero level must to be treated with extreme caution.
>
>
    • The extended calibrated maps (extdPxW in level-2, 2.5 or 3) incorporate zero level offsets derived from Planck-HFI. For small size SPIRE maps, smaller than ~30 arcmin, the zero-offset can be rather uncertain, due to the large Planck beam (8 arcmin). In such cases the interpretation of the zero offset as the absolute zero level must to be treated with extreme caution.
 
Line: 239 to 287
  SPIRE Small Scan Map AOT release note: 17 Mar 2010
Changed:
<
<
<!--
<a class="red" href="http://herschel.esac.esa.int/Docs/AOTsReleaseStatus/SPIRE_PointSource_AOT_ReleaseNote_30Apr2010.pdf">SPIRE Point Source Mode release note</a>: 30 Apr 2010 
-->
>
>
<--
SPIRE Point Source Mode release note: 30 Apr 2010 
-->
 
Changed:
<
<

SPIREnbsp;Spectroscopy
>
>

SPIRE Spectroscopy

 
Changed:
<
<
  • *In order to obtain the best possible Level 2 SPIRE FTS data, the observations should be reprocessed with the [[http://herschel.esac.esa.int/HIPE_download.shtml][
latest HIPE User Release]]*.
>
>
  • In order to obtain the best possible Level 2 SPIRE FTS data, the observations should be reprocessed with the latest HIPE User Release.
 
  • Bright source mode: observations in bright source mode processed with HIPE v7 or earlier result in spectra that have no scientific value. Bright source observations processed with HIPE v8 or above are fine.
Line: 256 to 304
 
  • Faint point source observations: if the SSW and SLW bands do not match up for a source that is known to be a point source, observers are recommended to run the background subtraction script in HIPE - more details are in the SPIRE Data Reduction Guide. If this doesn't solve the problem, the observer is encouraged to contact the HSC helpdesk or the FTS User Support Group.

  • Artefacts in the continuum for few repetitions
Changed:
<
<
    • Very small repetition numbers (e.g. 2 or 4) make 2nd level deglitching, which is based on a statistical outlier criterion, more challenging. The deglitching module may either not identify a glitch at all or it may not remove it completely. In cases where the glitch is located within the double-sided portion of the interferogram, the additional energy from the glitch will translate into artefacts of the continuum level. This kind of problem can be identified by inspecting all detectors from all scans in the level-1 spectral products. For some detectors, one or several scans may appear to be outliers. As a work-around, it is recommended to reprocess the data with a lower
thresholdFactor when calling deglitchIfgm(). If the problem persists, the identified detector should be removed from the applicable scan in the SDI product.
(NB: This affects HIPE 6 and higher)
>
>
    • Very small repetition numbers (e.g. 2 or 4) make 2nd level deglitching, which is based on a statistical outlier criterion, more challenging. The deglitching module may either not identify a glitch at all or it may not remove it completely. In cases where the glitch is located within the double-sided portion of the interferogram, the additional energy from the glitch will translate into artefacts of the continuum level. This kind of problem can be identified by inspecting all detectors from all scans in the level-1 spectral products. For some detectors, one or several scans may appear to be outliers. As a work-around, it is recommended to reprocess the data with a lower thresholdFactor when calling deglitchIfgm(). If the problem persists, the identified detector should be removed from the applicable scan in the SDI product.
      (NB: This affects HIPE 6 and higher)
 
  • Line fitting
Changed:
<
<
    • If you know that the line you wish to measure is unresolved then you may want to fix the line width to the instrumental line width. For the Sinc model, implemented in the
SpectrumFitterGUI you should put the Sinc width equal to resolution/π, which, for HR is 1.2/π = 0.382 GHz.
>
>
    • If you know that the line you wish to measure is unresolved then you may want to fix the line width to the instrumental line width. For the Sinc model, implemented in the SpectrumFitterGUI you should put the Sinc width equal to resolution/π, which, for HR is 1.2/π = 0.382 GHz.
 
  • Point source and extended source spectra
    • If your level-2 spectra show characteristic jumps at ~1250 GHz and ~750 GHz, and the spectra from the two bands SSW and SLW do not match, then your target is extended or semi-extended in the SPIRE beam. You need to use the semi-extended correction tool (SECT) available since HIPE v10. Check the the SPIRE Data Reduction Guide (SDRG).

  • Quality flags in the quality
    • The quality flags at the quality context inside the observation context are just meant for HSC/ICC internal evaluation of the quality of the products and not for the users. In case the data had some serious quality problem, the PI of the program has been contacted about it. Otherwise, only information in the Quality Summary, when available, should concern the observers.
Changed:
<
<
    • For observations before OD1000 there could be an erroneously raised or a missing flag
RADECACC. This flag is calculated as the difference between the observer requested target coordinates (kept in a metadata raNominal, decNominal) and the average pointing of the telescope during the observation (excluding the slew). This is an indicator of the pointing accuracy of the observation. If greater than 2.2" then RADECACC is raised in the QC context. The calculation of the average (ra, dec) during the observation does not take into account the known offset of 1.7" of the BSM home position for OD < 1000 and consequently observations with better pointing than 2.2" may be flagged and vice versa, observations at more than 2.2" may not be flagged. If you want to check the pointing offset with respect to the requested target coordinates then you should use the pointing information for the central detector SSWD4 in the level-2 context.
A new quality parameter raDecOffset is introduced in HIPE v13, in level-2 products, which contains the correct offset between the raNominal, decNominal and ra,dec for the FTS central detector.
>
>
    • For observations before OD1000 there could be an erroneously raised or a missing flag RADECACC. This flag is calculated as the difference between the observer requested target coordinates (kept in a metadata raNominal, decNominal) and the average pointing of the telescope during the observation (excluding the slew). This is an indicator of the pointing accuracy of the observation. If greater than 2.2" then RADECACC is raised in the QC context. The calculation of the average (ra, dec) during the observation does not take into account the known offset of 1.7" of the BSM home position for OD < 1000 and consequently observations with better pointing than 2.2" may be flagged and vice versa, observations at more than 2.2" may not be flagged. If you want to check the pointing offset with respect to the requested target coordinates then you should use the pointing information for the central detector SSWD4 in the level-2 context.
      A new quality parameter raDecOffset is introduced in HIPE v13, in level-2 products, which contains the correct offset between the raNominal, decNominal and ra,dec for the FTS central detector.
 
  • Observations during the steep rise of the sub-K temperature
Changed:
<
<
    • During the first few hours after the cooler was recycled the bolometers' temperature (the sub-K temperature) undergoes a steep rise before it reaches a stable plateau. Observations during this period suffer overcorrection of the instrument/telescope emission. This is more significant for faint targets and can be identified as an unphysical slope of the SLW spectrum, with an important negative gap in the region that overlaps with SSW (see the Figure). One way to check if your observation is within this problematic category is to get the median sub-K temperature for the first building block:
print MEDIAN(obs.level0_5.get(0xA1060001).nhkt['signal']['SUBKTEMP'].data), where obs is the pre-loaded observational context. If the result is less than 0.2869 K then the observation is affected by this.
This effect is fixed in HIPE v13.
>
>
    • During the first few hours after the cooler was recycled the bolometers' temperature (the sub-K temperature) undergoes a steep rise before it reaches a stable plateau. Observations during this period suffer overcorrection of the instrument/telescope emission. This is more significant for faint targets and can be identified as an unphysical slope of the SLW spectrum, with an important negative gap in the region that overlaps with SSW (see the Figure). One way to check if your observation is within this problematic category is to get the median sub-K temperature for the first building block: print MEDIAN(obs.level0_5.get(0xA1060001).nhkt['signal']['SUBKTEMP'].data), where obs is the pre-loaded observational context. If the result is less than 0.2869 K then the observation is affected by this.
      This effect is fixed in HIPE v13.
  Steep-subK-case.png
Changed:
<
<
<!--   * *FTS Array footprint user script*
      * For very large area maps the script produces wrong overlay (offset from the real position) if the default map projections (tangential) is used. The workaround is to use smaller map where the tangential projection centre is near the FTS target position.
-->
>
>
<--   * FTS Array footprint user script 
    • For very large area maps the script produces wrong overlay (offset from the real position) if the default map projections (tangential) is used. The workaround is to use smaller map where the tangential projection centre is near the FTS target position.
-->
 
Line: 298 to 336
  SPIRE Spectrometer Mapping AOT release note: 30 Apr 2010
Changed:
<
<

HIFInbsp;observationsnbsp;(pointnbsp;mode,nbsp;spectralnbsp;surveynbsp;andnbsp;mappingnbsp;observations)
>
>

HIFI observations (point mode, spectral survey and mapping observations)

 
Changed:
<
<
*In order to obtain the best possible Level 2 HIFI data the observations should be reprocessed with the [[http://herschel.esac.esa.int/HIPE_download.shtml][ latest HIPE User Release]].*
>
>
In order to obtain the best possible Level 2 HIFI data the observations should be reprocessed with the latest HIPE User Release.
 
  • Corrupted data-frames
    • There is a limited number of obsids where one or more data-frames from the spectrometers is corrupted in some fashion and leads to level 2 products that cannot be properly calibrated. A scheme was put in place some versions ago to flag or simply remove those frames at level 0 so the final products won't be affected by them. This scheme however suffered from a regression bug in 11.1 so that the corresponding products did not benefit from this correction properly. We note however that this should only concern about 50 obsids in the whole HIFI archive. This will be fixed in HIPE 12.
Line: 376 to 412
 

Other Technical Notes

PACS documents: see PACS Instrument and Calibration webpage

Changed:
<
<
<!-- 
<a class="red" href="http://herschel.esac.esa.int/Docs/TechnicalNotes/PACS_Spectroscopy_AOR_Update_Guide_10Mar2010.pdf">PACS Spectroscopy AOR Update Guide for Routine Phase Observations</a>: 10 Mar 2010
-->
>
>
<-- 
PACS Spectroscopy AOR Update Guide for Routine Phase Observations: 10 Mar 2010
-->
  HIFI Information note on Mapping Modes: 30 Jun 2010
Line: 389 to 426
  PACS photometer and spectrometer calibration documents: see PACS Instrument and Calibration webpage
Changed:
<
<
<!-- <a class="red" href="http://herschel.esac.esa.int/Docs/Calibration/PACS_SpectroscopyPerformanceCalibration_11Mar2010.pdf">PACS Spectroscopy Performance and Calibration</a>: 11 Mar 2010 

<a class="red" href="http://herschel.esac.esa.int/Docs/Calibration/PACS_Point_Spread_Function_03Nov2010.pdf">PACS Photometer Point Spread Function (PSF)</a>: 03 Nov 2010 
-->
>
>
<-- PACS Spectroscopy Performance and Calibration: 11 Mar 2010 

PACS Photometer Point Spread Function (PSF): 03 Nov 2010 -->

  SPIRE Photometer Beams (FTP repository)

HIFI System noise temperature IF spectra (FTP repository)

Changed:
<
<
<!--
*  Set ALLOWTOPICCHANGE = Main.DpMgGroup, Main.HscCommunitySupportGroup
-->
>
>
<--
*  Set ALLOWTOPICCHANGE = DpMgGroup, HscCommunitySupportGroup
-->
 
META FILEATTACHMENT attr="h" autoattached="1" comment="" date="1437490181" name="obsSVVanomaly.csv" path="obsSVVanomaly.csv" size="2980" user="Main.LucaCalzoletti" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1437489475" name="ngc253_1342221743_blue.jpeg" path="ngc253_1342221743_blue.jpeg" size="957526" user="Main.LucaCalzoletti" version="1"

Revision 1312015-12-09 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Data Products Known Issues

Line: 117 to 117
 

Special messages

Changed:
<
<
Warning for drizzled cubes in HIPE/SPG 13: no longer relevant for SPG/HIPE 14

Due to an oversight in the pipeline scripts of HIPE 13, the drizzled cubes for chop-nod line scan observations created by "SPG 13" will have incorrect fluxes. Therefore you should not use the drizzled cubes downloaded from the HSA if the Meta datum "creator" is "SPG 13.0", whether they are within the ObservationContext at Level 2, or part of a Standalone Browse Product download.

Any drizzled cubes created for unchopped mode observations will be unaffected. For chop-nod range scan observations, drizzled cubes are not created by the SPG 13 pipeline anyway.

This oversight has been corrected in HIPE/SPG 14. The drizzled cubes you get in an observation downloaded from the HSA with "creator" of "SPG 14.0" have correct fluxes. In HIPE 14 there is a dedicated script to produce these drizzled cubes. Therefore we recommend you use the SPG 14.0 products, or use the dedicated pipeline in HIPE 14 if you need your drizzled cubes before the SPG 14 products are available.

FYI: Drizzled cubes can be found in an observation in the context called HPS3DD[R|B] (red and blue). They are also provided as standalone browse products in a context called HPS3DEQ[R|B] (red and blue, equidistant wavelength grid version of the drizzled cubes). On disk the FITS files for these cubes have the same set of letters in their name (but in lower case) and are at the Level 2 part of an observation.

For more information about the standard and the standalone browse cubes provided for PACS spectroscopy, see the PACS Products Explained HIPE help document, which is also available from the PACS documentation webpage and the HIPE download webpage, both off the Herschel Science Centre webpage.

General notes

>
>
General notes
 
  • To produce the highest quality cubes possible, you should consider re-processing or fine-tuning the observations with the [[http://herschel.esac.esa.int/HIPE_download.shtml][
latest HIPE User Release]]. Cubes available from the HSA are created within a bulk processing framework, and a reprocessing while fine-tuning the important pipeline task parameters, according to the characteristics of the observation and source, could enhance the quality of the final results. The first two chapters of the PACS Data Reduction Guide for spectroscopy (HIPE 13) give information about the need to reprocess, and about what to do with HSA-obtained cubes before using them for science.

Revision 1302015-12-09 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Data Products Known Issues

Line: 2 to 2
 
META TOPICPARENT name="WebHome"

Data Products Known Issues

Deleted:
<
<
 HCSS, SPG, and HIPE
Changed:
<
<
The whole suite of Herschel software is known as the Herschel Common Science System (HCSS). This encompasses all the software aspects related to the mission: automatic pipelines, spacecraft calibrations, etc. As data are retrieved from the spacecraft they are ingested in the Herschel Science Archive (HSA) and processed with the current official version of the pipeline. That means that, at any given time, different data in the HSA may be reduced with different pipeline versions. The pipeline version is listed in the HSA GUI as SPG vX.X.X, where SPG stands for Standard Product Generator. The SPG version is available as the header keyword 'creator', in the data .fits files. Every HCSS version, the whole HSA is bulk re-processed with the same up-to-date version of the pipelines.
>
>
The whole suite of Herschel software is known as the Herschel Common Science System (HCSS). This encompasses all the software aspects related to the mission: automatic pipelines, spacecraft calibrations, etc. As data are retrieved from the spacecraft they are ingested in the Herschel Science Archive (HSA) and processed with the current official version of the pipeline. That means that, at any given time, different data in the HSA may be reduced with different pipeline versions. The pipeline version is listed in the HSA GUI as SPG vX.X.X, where SPG stands for Standard Product Generator. The SPG version is available as the header keyword 'creator', in the data .fits files. Every HCSS version, the whole HSA is bulk re-processed with the same up-to-date version of the pipelines.
  From the user point of view, the most important piece of the HCSS system is the Herschel Interactive Processing Environment (HIPE). HIPE allows the astronomer the possibility of inspecting the data and re-process them, if the results from the automatic pipeline are not good enough for his/her purposes. Because HIPE is part of the HCSS, the latest version of HIPE will have the most up-to-date pipeline, calibrations and documentation available.
Line: 15 to 15
  In what follows, we provide a summary of the known issues that you may encounter when inspecting data processed with the automatic pipelines SPG versions 6.1 to 13.0. Most can be resolved by running the pipelines within HIPE and optimizing their parameters as explained below.
Changed:
<
<
<--This is a comment: it will be ignored by the browser-->
>
>
<!--This is a comment: it will be ignored by the browser-->
  Note that some of this information can also be found in the quality report of the observation (QC Report) and as metadata with the FITS keyword "PCAVEATS".

TABLE OF CONTENTS

Changed:
<
<
>
>

 

METADATA AND FITS KEYWORDS

Added:
>
>
 Message about DATE-OBS
Changed:
<
<
Some internal HCSS metadata are renamed by the HCSS software when translating to FITS. One special case is startDate which gets written to FITS as both DATE-OBS and DATE_OBS. This is done for compatibility with legacy Spitzer software (namely MOPEX).
>
>
Some internal HCSS metadata are renamed by the HCSS software when translating to FITS. One special case is startDate which gets written to FITS as both DATE-OBS and DATE_OBS. This is done for compatibility with legacy Spitzer software (namely MOPEX).
 
Changed:
<
<

PACS Photometer (scan mapping)

>
>

PACSnbsp;Photometernbsp;(scannbsp;mapping)
 

Special messages

Changed:
<
<
Warning for Unimap maps in HIPE/SPG 13
>
>
Warning for Unimap maps in HIPE/SPG 13: no longer relevant for SPG/HIPE 14
  Some Unimap maps show overshooting effects around very bright sources that are surrounded by a diffuse and relatively faint background. See, as an example, the blue image of NGC253 (obsID: 1342221743).
Changed:
<
<
This effect is due to a not optimised convergence of the GLS algorithm and it will be corrected in HIPE/SPG14. The user can request for a dedicated reprocessing to the Herschel Science Center Helpdesk or he can use the corresponding Level 2.5 JScanam maps (HPPJSMAP[B][R] blue and red products), which are not affected by this issue.
>
>
This effect is due to a non-optimised convergence of the GLS algorithm and has been corrected in HIPE/SPG14. The user can request for a dedicated reprocessing to the Herschel Science Center Helpdesk or they can use the corresponding Level 2.5 JScanam maps (HPPJSMAP[B][R] blue and red products), which are not affected by this issue.
 
Changed:
<
<
Warning for pointing anomaly in HIPE/SPG 13
>
>
Warning for pointing anomaly in HIPE/SPG 13 and 14
  PACS and SPIRE photometry observations reported in this CSV file are affected by a known problem related to the reset of the Spacecraft Velocity Vector (SVV) during the upload of the star-tracker's defective pixel table. For the affected observations, the pointing of the telescope can be off along the scan direction, and shifted up to 20 arcsec.
Deleted:
<
<
This effect will be corrected in HIPE/SPG14.
 

General notes

Changed:
<
<
  • To produce the highest quality maps, you should consider re-processing or fine-tuning the observations with the latest HIPE User Release. Maps available from the HSA are created within a bulk processing framework, and a reprocessing while fine-tuning the mapper parameters, according to the characteristics of the observed sky region, could enhance the quality of the final maps.
>
>
  • To produce the highest quality maps, you should consider re-processing or fine-tuning the observations with the [[http://herschel.esac.esa.int/HIPE_download.shtml][
latest HIPE User Release]]. Maps available from the HSA are created within a bulk processing framework, and a reprocessing while fine-tuning the mapper parameters, according to the characteristics of the observed sky region, could enhance the quality of the final maps.
 
  • Important note:
    • PACS maps from any map-maker are in essence differential maps: the absolute level is undefined due to the dominating telescope background removed by map-makers. Hence it is not unusual if your background level is negative.
Line: 51 to 58
 
  • Intrinsic limitation of highpass filter maps
    • High-pass filtered maps in SPG13 are available at level 2 and level 2.5. The extended emission is filtered out in the maps since a rather small filter width is used to remove 1/f noise stripping: this is done to allows us to obtain the best sensitivity for point-sources. Bright sources are masked out during the highpass filtering, hence their flux is not too much affected by filtering. But faint sources are not or are inadequately masked out by the SPG processing, hence the flux loss for these point-source can reach up to 20--30%. Re-processing your data is recommended here.
Changed:
<
<
<-- 
    • These are due to a non-optimal estimate of background by the high-pass filter close to a bright source. The current masked high-pass filter pipeline is a trade-off, designed to get a good sensitivity on point-sources and preserve some extended emission up to a few arcmin scale. However due to the relatively large width in the second high-pass filtering, significant stripping from the 1/f noise is still present in the level 2 maps. On the other hand a large fraction of the extended emission is filtered out. The observer is advised to play with the threshold to define the mask and with the width of the high-pass filter to reduce these effects or move to MADmap scanamorphos to preserve extended emission at all scales. Level 2.5 MADmap maps, combining scans and crossed-scans on the other hands do preserve extended emission at all spatial scales.
-->
>
>
<!--
      * These are due to a non-optimal estimate of background by the high-pass filter close to a bright source. The current masked high-pass filter pipeline is a trade-off, designed to get a good sensitivity on point-sources and preserve some extended emission up to a few arcmin scale. However due to the relatively large width in the second high-pass filtering, significant stripping from the 1/f noise is still present in the level 2 maps. On the other hand a large fraction of the extended emission is filtered out. The observer is advised to play with the threshold to define the mask and with the width of the high-pass filter to reduce these effects or move to MADmap scanamorphos to preserve extended emission at all scales. Level 2.5 MADmap maps, combining scans and crossed-scans on the other hands do preserve extended emission at all spatial scales.
-->
 
  • MADmap issues
    • MADmap -- a GLS (Generalized Least Square) map-maker -- maps are no longer created by SPG13 and beyond. However, the pipeline script to do this yourself is provided in HIPE 13.
Line: 82 to 87
 
    • One matrix (fully half of) of the red channel array was lost at the end of the mission, so from OD1375 onwards this matrix is masked out automatically in the SPG processing.
    • The quality flags in the quality context ("quality" or "qualitySummary") inside the observation context are meant for HSC/ICC internal evaluation of the quality of the products: when an observation had a serious quality problem, the PI of the program would have been contacted about it. For archive users, only the information in the "qualitySummary", when available, is useful.
Added:
>
>
<!---
   * *Calibration block transient in L2 maps*
      * Several red (160um) scan maps are affected by a calibration block downwards transient when scheduled immediately before the science observation. This can be mitigated by running a high-pass filter with a smaller width, fitting the transient or just masking the affected frames at the start of the observation.
-->
 
Changed:
<
<
<--- 
  • Calibration block transient in L2 maps
    • Several red (160um) scan maps are affected by a calibration block downwards transient when scheduled immediately before the science observation. This can be mitigated by running a high-pass filter with a smaller width, fitting the transient or just masking the affected frames at the start of the observation.
-->

<--      * Scan maps processed with SPG v4.1.0 were deglitched with "MMT deglitching", a temporal deglitching of the pixel timelines. This deglitching technique works very well for deep fields (e.g. cosmological fields) but fails at high scan speed (60"/s) or even medium scan speeds on bright (>~1Jy) sources, wrongly identified as glitches.
-->
>
>
<!--      * Scan maps processed with SPG v4.1.0 were deglitched with "MMT deglitching", a temporal deglitching of the pixel timelines. This deglitching technique works very well for deep fields (e.g. cosmological fields) but fails at high scan speed (60"/s) or even medium scan speeds on bright (>~1Jy) sources, wrongly identified as glitches.
-->
 
Changed:
<
<
<--   * Deglitching 
    • The deglitching was changed to a so-called "2nd level deglitching" in SPG v6.1.0 that makes use of the spatial redundancy (a sky pixel being seen by several detector pixel readouts). Large scan maps (e.g. galactic fields) processed with SPG v6.1 are therefore of much better quality than earlier ones (SPG v4.1.0), however some low level glitches are left as a high threshold was set in the pipeline. As the deglitching is now run from level 1 to level 2 after highpass filtering, the level 1 cubes in HSA processed with SPG 6.1 do not contain anymore glitch masks. For interactive processing, several hints and methods are given in the ipipe scripts.
-->
>
>
<!--   * *Deglitching*
      * The deglitching was changed to a so-called "2nd level deglitching" in SPG v6.1.0 that makes use of the spatial redundancy (a sky pixel being seen by several detector pixel readouts). Large scan maps (e.g. galactic fields) processed with SPG v6.1 are therefore of much better quality than earlier ones (SPG v4.1.0), however some low level glitches are left as a high threshold was set in the pipeline. As the deglitching is now run from level 1 to level 2 after highpass filtering, the level 1 cubes in HSA processed with SPG 6.1 do not contain anymore glitch masks. For interactive processing, several hints and methods are given in the ipipe scripts.
-->
 
Changed:
<
<
>
>
<!--
The astrometry of several PACS scan maps acquired between ODs 320 and 761 has been reported to be off by 4 arcsec (solid offset of the whole map) or even above for a few fields where the tracking stars are not homogeneously distributed in the star-tracker field-of-view. It is intended to improve the a posteriori reconstructed pointing for all observations in the future.
-->
 
Added:
>
>
<!--
      * The absolute astrometry of Level 2 maps on Solar System Objects (SSO), projected in the SSO reference frame is not reliable and can be off up to 20 arcsec. This a data processing issue only (aberration & light travel time correction), while the observations themselves were correctly performed (uplink). This issue is under investigations at HSC and PACS ICC.
-->
 
Changed:
<
<
<--
The astrometry of several PACS scan maps acquired between ODs 320 and 761 has been reported to be off by 4 arcsec (solid offset of the whole map) or even above for a few fields where the tracking stars are not homogeneously distributed in the star-tracker field-of-view. It is intended to improve the a posteriori reconstructed pointing for all observations in the future.
-->
>
>
<!--
      * Full noise propagation in the HSC pipelines will be available at later HCSS versions, so the current noise map is relatively but not absolutely correct. Furthermore, the default pipeline results in correlated noise in individual pixels. In order to estimate a proper background standard deviation, the observers are advised to make 10+ aperture photometry measurements in different sky patches around the source and to estimate the absolute standard deviation of the image as the sigma of those photometric points. 
-->
 
Changed:
<
<
<-- 
    • The absolute astrometry of Level 2 maps on Solar System Objects (SSO), projected in the SSO reference frame is not reliable and can be off up to 20 arcsec. This a data processing issue only (aberration & light travel time correction), while the observations themselves were correctly performed (uplink). This issue is under investigations at HSC and PACS ICC.
-->

<-- 
    • Full noise propagation in the HSC pipelines will be available at later HCSS versions, so the current noise map is relatively but not absolutely correct. Furthermore, the default pipeline results in correlated noise in individual pixels. In order to estimate a proper background standard deviation, the observers are advised to make 10+ aperture photometry measurements in different sky patches around the source and to estimate the absolute standard deviation of the image as the sigma of those photometric points.
-->

<-- 
  • Quality flags in the quality

    • Currently, the quality flags at the quality context inside the observation context are just meant for HSC/ICC internal evaluation of the quality of the products and not for the users. In case the data had some serious quality problem, the PI of the program has been contacted about it. Otherwise, only information in the quality summary, when available, should concern the observers.
-->
>
>
<!--
   * *Quality flags in the quality*

      * Currently, the quality flags at the quality context inside the observation context are just meant for HSC/ICC internal evaluation of the quality of the products and not for the users. In case the data had some serious quality problem, the PI of the program has been contacted about it. Otherwise, only information in the quality summary, when available, should concern the observers.
-->
 
Line: 130 to 110
  PACS photometer AOT release notes see PACS Instrument and Calibration webpage
Changed:
<
<
<---
PACS Scan Map AOT release note: 23 Feb 2010

PACS Photometer - Point/Compact Source Observations: Mini Scan-Maps & Chop-Nod AOT release note: 12 Nov 2010 -->

>
>
<!---
<a class="red" href="http://herschel.esac.esa.int/Docs/AOTsReleaseStatus/PACS_ScanMap_ReleaseNote_23Feb2010.pdf">PACS Scan Map AOT release note</a>: 23 Feb 2010

<a class="red" href="http://herschel.esac.esa.int/Docs/AOTsReleaseStatus/PACS_PhotMiniScan_ReleaseNote_12Nov2010.pdf">PACS Photometer - Point/Compact Source Observations: Mini Scan-Maps &amp; Chop-Nod AOT release note</a>: 12 Nov 2010
-->
 
Changed:
<
<

PACS Spectroscopy

>
>

PACSnbsp;Spectroscopy
 

Special messages

Deleted:
<
<
Warning for drizzled cubes in HIPE/SPG 13
 
Changed:
<
<
Due to an oversight in the pipeline scripts of HIPE 13, the drizzled cubes you get for chop-nod line scan observations created by "SPG 13" will have incorrect fluxes. Therefore you should not use the drizzled cubes downloaded from the HSA if the Meta datum "creator" is "SPG 13.0", whether they are within the ObservationContext at Level 2, or part of a Standalone Browse Product download.
>
>
Warning for drizzled cubes in HIPE/SPG 13: no longer relevant for SPG/HIPE 14

Due to an oversight in the pipeline scripts of HIPE 13, the drizzled cubes for chop-nod line scan observations created by "SPG 13" will have incorrect fluxes. Therefore you should not use the drizzled cubes downloaded from the HSA if the Meta datum "creator" is "SPG 13.0", whether they are within the ObservationContext at Level 2, or part of a Standalone Browse Product download.

 Any drizzled cubes created for unchopped mode observations will be unaffected. For chop-nod range scan observations, drizzled cubes are not created by the SPG 13 pipeline anyway.
Deleted:
<
<
This oversight will be corrected in HIPE/SPG 14.
 
Changed:
<
<
For the affected observations you have the following alternatives: 1) Use the projected cubes instead, which you will also find within the ObservationContext at Level 2, but which are not part of the Standalone Browse Product download for the affected observations. 2) Run one of the interactive pipeline scripts to create the drizzled cubes yourself. Contact the Herschel Helpdesk first to ask for an updated pipeline script to do this, since the interactive pipeline scripts in HIPE 13 (and 12) also contain the mistake.

For Your Information: Drizzled cubes FITS files, and their name as listed in an ObservationContext, are: HPS3DD[R|B] (red and blue) Standalone browse product drizzled cube FITS files, and their name as listed in an ObservationContext in the browse product section are: HPS3DEQ[R|B] (red and blue, equidistant wavelength grid version of the drizzled cubes). Projected cubes have the name: HPS3DP[R|B].

>
>
This oversight has been corrected in HIPE/SPG 14. The drizzled cubes you get in an observation downloaded from the HSA with "creator" of "SPG 14.0" have correct fluxes. In HIPE 14 there is a dedicated script to produce these drizzled cubes. Therefore we recommend you use the SPG 14.0 products, or use the dedicated pipeline in HIPE 14 if you need your drizzled cubes before the SPG 14 products are available.

FYI: Drizzled cubes can be found in an observation in the context called HPS3DD[R|B] (red and blue). They are also provided as standalone browse products in a context called HPS3DEQ[R|B] (red and blue, equidistant wavelength grid version of the drizzled cubes). On disk the FITS files for these cubes have the same set of letters in their name (but in lower case) and are at the Level 2 part of an observation.

  For more information about the standard and the standalone browse cubes provided for PACS spectroscopy, see the PACS Products Explained HIPE help document, which is also available from the PACS documentation webpage and the HIPE download webpage, both off the Herschel Science Centre webpage.

General notes

Changed:
<
<
  • To produce the highest quality cubes possible, you should consider re-processing or fine-tuning the observations with the latest HIPE User Release. Cubes available from the HSA are created within a bulk processing framework, and a reprocessing while fine-tuning the important pipeline task parameters, according to the characteristics of the observation and source, could enhance the quality of the final results. The first two chapters of the PACS Data Reduction Guide for spectroscopy (HIPE 13) give information about the need to reprocess, and about what to do with HSA-obtained cubes before using them for science.
>
>
  • To produce the highest quality cubes possible, you should consider re-processing or fine-tuning the observations with the [[http://herschel.esac.esa.int/HIPE_download.shtml][
latest HIPE User Release]]. Cubes available from the HSA are created within a bulk processing framework, and a reprocessing while fine-tuning the important pipeline task parameters, according to the characteristics of the observation and source, could enhance the quality of the final results. The first two chapters of the PACS Data Reduction Guide for spectroscopy (HIPE 13) give information about the need to reprocess, and about what to do with HSA-obtained cubes before using them for science.
 
Changed:
<
<
<-- 
  • Off-subtraction for unchopped long-range scan observations
    • Range Spectroscopy unchopped observation require off-position scans observed in a separate AOR. The off-source observation need to be subtracted from the on-scan after producing Level 2 rebinned spectra, this results a Level 2.5 product. The pipeline only generates Level 2 data products for unchopped spectroscopy in SPG 8.0. You need to combine interactively the on and off positions using the dedicated multi-observation unchopped pipeline script under the PACS pipeline menu.
-->

<-- 
  • Unchopped grating scan flux calibration
    • The absolute flux calibration of the PACS spectrometer is based on observations of flux calibration standards using chopped spectroscopy modes. There are hints of systematic differences in the response scaling between chopped and unchopped mode due to response transients within the chopping pattern. In SPG 8.0 the flux calibration of unchopped data relies on the system response derived chopped scheme, therefore absolute flux values need to be carefully interpreted. Please contact Helpdesk for guidelines on the specific observation you have to deal with.
-->
>
>
<!--
   * *Off-subtraction for unchopped long-range scan observations*
      * Range Spectroscopy unchopped observation require off-position scans observed in a separate AOR. The off-source observation need to be subtracted from the on-scan after producing Level 2 rebinned spectra, this results a Level 2.5 product. The pipeline only generates Level 2 data products for unchopped spectroscopy in SPG 8.0. You need to combine interactively the on and off positions using the dedicated multi-observation unchopped pipeline script under the PACS pipeline menu.
-->
 
Added:
>
>
<!--
   * *Unchopped grating scan flux calibration*
      * The absolute flux calibration of the PACS spectrometer is based on observations of flux calibration standards using chopped spectroscopy modes. There are hints of systematic differences in the response scaling between chopped and unchopped mode due to response transients within the chopping pattern. In SPG 8.0 the flux calibration of unchopped data relies on the system response derived chopped scheme, therefore absolute flux values need to be carefully interpreted. Please contact Helpdesk for guidelines on the specific observation you have to deal with.
-->
 
  • Spectral leakage
Changed:
<
<
    • The order selection filters of the PACS spectrometer have a steep but not perfectly vertical transmission profile at the cut­off wavelengths of the spectral bands. PACS spectra near the band borders of bands R1, B3A and B2B are therefore affected by higher- or lower-order wavelengths leaking into the spectra -- both continuum and spectral lines! Consult the PACS Calibration Document for more information on the leakage regions.
>
>
    • The order selection filters of the PACS spectrometer have a steep but not perfectly vertical transmission profile at the cut­off wavelengths of the spectral bands. PACS spectra near the band borders of bands R1, B3A and B2B are therefore affected by higher- or lower-order wavelengths leaking into the spectra -- both continuum and spectral lines! Consult the [[http://herschel.esac.esa.int/twiki/pub/Public/PacsCalibrationWeb/PacsSpectroscopyPerformanceAndCalibration_v2_4.pdf][
PACS Calibration Document]] for more information on the leakage regions.
 
    • For the leak in band R1, it is possible to reduce these data with a specific relative spectral response function calibration file in HIPE to obtain correct line fluxes (but an incorrect continuum level). How to do this is explained in the PACS Data Reduction Guide for spectroscopy (chps 5 and 6 in the HIPE 13 version). This RSRF is not applied by default since it increases the noise in the resulting spectrum, but can be used interactively within HIPE.

  • Second-pass ghosts
Changed:
<
<
    • A second pass in the optics of the PACS spectrometer can cause a ghost image to appear on most spaxels (but never in the central spaxel). If a source located in one of these "originating" spaxels shows a strong spectral line (typically an atomic fine-structure line), then a weak, broadened line can be seen at an offset wavelength in its corresponding "destination" spaxel affected by the 2nd-pass ghost. The peak flux of this line is typically ~5% of the peak of the originating line. The most prominent ghost is the 122 micron feature, which originates from the usually strong CII+ 157.7 micron line. A list of strong ghosts and an image showing the directions of the projected passes on the 5x5 IFU footprint can be found in the PACS Calibration Document.
>
>
    • A second pass in the optics of the PACS spectrometer can cause a ghost image to appear on most spaxels (but never in the central spaxel). If a source located in one of these "originating" spaxels shows a strong spectral line (typically an atomic fine-structure line), then a weak, broadened line can be seen at an offset wavelength in its corresponding "destination" spaxel affected by the 2nd-pass ghost. The peak flux of this line is typically ~5% of the peak of the originating line. The most prominent ghost is the 122 micron feature, which originates from the usually strong CII+ 157.7 micron line. A list of strong ghosts and an image showing the directions of the projected passes on the 5x5 IFU footprint can be found in the [[http://herschel.esac.esa.int/twiki/pub/Public/PacsCalibrationWeb/PacsSpectroscopyPerformanceAndCalibration_v2_4.pdf][
PACS Calibration Document]].
 
  • 62 micron dip
    • A dip can sometimes be seen at spectra between 62 and 63 microns. This is a filter feature. Its appearance depends on the angle at which the light from the sources goes through the filter, and so it depends on the source position and its spatial structure.
Line: 184 to 153
 
    • You may notice a fringing-like pattern in the spectra from (wide) Range scan and full SED-mode observations. This is because the flatfielding is not done by the SPG pipeline. To remove it you need to reprocess the data in HIPE with one of the interactive pipeline scripts. For Line Spectroscopy observations, the spectral flatfielding is done in the SPG pipeline.

  • Check for contaminating flux in chop-off positions
Changed:
<
<
    • To check for the presence of contamination from unwanted astronomical sources in the off positions of chopNod mode observations, you can use a Split On-Off pipeline script to produce an off-source and on-source cube. These cubes can then be compared to each other to check for contamination in spectral lines or by strong continuum emission, e.g. by over-plotting the respective spectra. Note that the on-source and off-source cubes produced by this task will not allow you to detect faint levels of contamination because wriggles from the RSRF are not removed by this process. For faint targets (line peak-to-continuum emission ~<5-10 Jy) you should also check the differential signal between the nodA and B on-source cubes. This is documented in the PACS Data Reduction Guide for spectroscopy. In this guide you can also find advice on checking for contamination in the unchopped mode observations, which is done either by comparing companion observations (unchopped range) or with a small script provided in the PDRG (unchopped line).
>
>
    • To check for the presence of contamination from unwanted astronomical sources in the off positions of chopNod mode observations, you can use a Split On-Off pipeline script to produce an off-source and on-source cube. These cubes can then be compared to each other to check for contamination in spectral lines or by strong continuum emission, e.g. by over-plotting the respective spectra. Note that the on-source and off-source cubes produced by this task will not allow you to detect faint levels of contamination because wriggles from the RSRF are not removed by this process. For faint targets (line peak-to-continuum emission ~<5-10 Jy) you should also check the differential signal between the nodA and B on-source cubes. This is documented in the PACS Data Reduction Guide for spectroscopy. In this guide you can also find advice on checking for contamination in the unchopped mode observations, which is done either by comparing companion observations (unchopped range) or with a small script provided in the PDRG (unchopped line).
 
Changed:
<
<
<-- 
  • RSRF at wavelengths below 53 microns
    • The relative spectral response function (used by the pipeline task rsrfCal) is an extrapolation at wavelengths below 53 microns. This will cause problems in the spectra from module 3 (=spaxel 3,0, i.e. in the cube image you see when you look at a PacsCube or PacsRebinnedCube with the Standard Cube Viewer or the Spectrum Explorer, it is the 4th up and on the very left): the extrapolation is too steep and makes the pixel-spectra similarly follow a very steep curve. This is being corrected.
-->
>
>
<!--
   * *RSRF at wavelengths below 53 microns*
      * The relative spectral response function (used by the pipeline task rsrfCal) is an extrapolation at wavelengths below 53 microns. This will cause problems in the spectra from module 3 (=spaxel 3,0, i.e. in the cube image you see when you look at a PacsCube or PacsRebinnedCube with the Standard Cube Viewer or the Spectrum Explorer, it is the 4th up and on the very left): the extrapolation is too steep and makes the pixel-spectra similarly follow a very steep curve. This is being corrected.
-->
 
  • Unstable/incorrect broad-band (dust) features
    • Broad spectral features (of a few micrometer width) and continuum shape variations can be introduced by transient effects (for chopNod mode and more so for unchopped mode observations) and by pointing offsets distorting the Relative Spectral Response Function. The "background normalisation" pipeline script for chopNod observations is recommended for observations looking for such features, as it minimises the effect -- this is the SPG pipeline for HIPE 13 and onwards. For unchopped mode observations you could reprocess the observation with the "transients correction" pipeline script that is new to HIPE 13. However, neither of these pipelines will completely negate the effect of transients and pointing jitter-induced RSRF distortions.

  • Spectral line profiles: the skew
Changed:
<
<
    • As for any slit-spectrograph, if the incoming light beam is neither homogeneously nor symmetrically illuminating a spaxel, then line profiles may be distorted from the ideal Gaussian shape. In case of a point-source observed with PACS, the spectral line profile develops a skew, increasing as the offset of the photocentre along the instrument Z-axis (perpendicular the slit direction) does. Examples of skewed line shapes are given in the PACS Observers Manual.
>
>
    • As for any slit-spectrograph, if the incoming light beam is neither homogeneously nor symmetrically illuminating a spaxel, then line profiles may be distorted from the ideal Gaussian shape. In case of a point-source observed with PACS, the spectral line profile develops a skew, increasing as the offset of the photocentre along the instrument Z-axis (perpendicular the slit direction) does. Examples of skewed line shapes are given in the [[http://herschel.esac.esa.int/Docs/PACS/html/pacs_om.html][
PACS Observers Manual]].
 
  • Point sources
    • How to extract a correctly-calibrated spectrum of a point source is documented in the pipeline scripts and in HIPE. The necessary point source corrections must be applied for a correct spectrum to be produced.

  • Limitations on absolute spectrophotometric accuracy
Changed:
<
<
    • The PACS spectrometer flux calibration accuracy is limited by detector response drifts and slight pointing offsets arising from the standard 2" (1-sigma) pointing error occurring within each and every observation. These limit both the absolute flux accuracy and relative accuracy within a band. Various pipelines deal better with these than others (see the advice in the PACS Data Reduction Guide for spectroscopy) but they can never be entirely negated. Hence the calibration uncertainty for any particular observation is a combination of the general calibration uncertainties (given in the PACS Observers Manual), the noise on the spectrum (explained in more detail in the PDRG chp 7.6/HIPE 13), and the "activity" during any single observation.
>
>
    • The PACS spectrometer flux calibration accuracy is limited by detector response drifts and slight pointing offsets arising from the standard 2" (1-sigma) pointing error occurring within each and every observation. These limit both the absolute flux accuracy and relative accuracy within a band. Various pipelines deal better with these than others (see the advice in the PACS Data Reduction Guide for spectroscopy) but they can never be entirely negated. Hence the calibration uncertainty for any particular observation is a combination of the general calibration uncertainties (given in the [[http://herschel.esac.esa.int/Docs/PACS/html/pacs_om.html][
PACS Observers Manual]]), the noise on the spectrum (explained in more detail in the PDRG chp 7.6/HIPE 13), and the "activity" during any single observation.
 
Changed:
<
<
<-- 
  • Line flux correction due to strong wings in the instrumental profile (IP)
    • Pre-flight ground-based monochromatic measurements indicated the PACS spectrometer instrumental profile distributes measurable power in spectral line wings. The effect is below ~10% and only noticeable for wavelengths longer than ~150 micrometers. Future HIPE releases will provide a correction factor to apply on a Gaussian fit in order to compensate for line power lost in the IP wings.
-->
>
>
<!--
   * *Line flux correction due to strong wings in the instrumental profile (IP)*
      * Pre-flight ground-based monochromatic measurements indicated the PACS spectrometer instrumental profile distributes measurable power in spectral line wings. The effect is below ~10% and only noticeable for wavelengths longer than ~150 micrometers. Future HIPE releases will provide a correction factor to apply on a Gaussian fit in order to compensate for line power lost in the IP wings.
-->
 
  • NaN's in the final cubes
    • It is normal to have NaNs at the very edges of the spectral ranges of SED mode observations: this is due to gaps in the spectral sampling.
Line: 221 to 185
 

PACS Spectrometer Release Notes

PACS spectrometer AOT release notes see PACS Instrument and Calibration webpage

Deleted:
<
<

SPIRE Photometry

>
>

SPIREnbsp;Photometry
 

Special messages

Added:
>
>
 Warning for pointing anomaly in HIPE/SPG 13

PACS and SPIRE photometry observations reported in this CSV file are affected by a known problem related to the reset of the Spacecraft Velocity Vector (SVV) during the upload of the star-tracker's defective pixel table. For the affected observations, the pointing of the telescope can be off along the scan direction, and shifted up to 20 arcsec.

Line: 241 to 200
 

General notes

Changed:
<
<
  • In order to obtain the best possible Level 2 SPIRE photometry data, the observations might have to be reprocessed with the latest HIPE User Release.
    • Warning, important Please note that there was a bug in the destriper task included in HIPE 9.0 that may affect your final map, especially if there are bright objects in the observed field. This has been corrected since HIPE 9.1. If your observation falls in the mentioned category, you are strongly advised to update your HIPE installation.
    • Warning, important In HIPE 10.0 the flagging of thermistor jumps in the level 0.5 to 1 data reduction is not set properly. This induces the destriper to work improperly and to leave stripes in the final map. It has been solved starting with HIPE 10.1.
>
>
  • *In order to obtain the best possible Level 2 SPIRE photometry data, the observations might have to be reprocessed with the [[http://herschel.esac.esa.int/HIPE_download.shtml][
latest HIPE User Release]].*
Warning, important Please note that there was a bug in the destriper task included in HIPE 9.0 that may affect your final map, especially if there are bright objects in the observed field. This has been corrected since HIPE 9.1. If your observation falls in the mentioned category, you are strongly advised to update your HIPE installation.
Warning, important In HIPE 10.0 the flagging of thermistor jumps in the level 0.5 to 1 data reduction is not set properly. This induces the destriper to work improperly and to leave stripes in the final map. It has been solved starting with HIPE 10.1.
 
  • SPIRE-P level 2.5 and level 3 maps
Changed:
<
<
    • These new levels will be available in the Herschel Science Archive when the observations will be bullk-reprocessed with HIPE v11. The useful user script Photometer_MapMerge.py can be used to make level 2.5 (parallel mode) or level 3 (mosaic) maps, see Section 5.8.3 in the SPIRE Data Reduction Guide.
>
>
    • These new levels will be available in the Herschel Science Archive when the observations will be bullk-reprocessed with HIPE v11. The useful user script
Photometer_MapMerge.py can be used to make level 2.5 (parallel mode) or level 3 (mosaic) maps, see Section 5.8.3 in the SPIRE Data Reduction Guide.
 
  • Stripes in PSW, PMW and/or PLW (Level 2) maps
    • All SPIRE photometry pipelines now use by default the destriper, which improves the issue of striping in level 2 maps. Hence observers should expect potential improvements in that respect with version 9.
Changed:
<
<
<-- 
    • Most of the stripes that are present in the final maps are due to a combination of thermal drifts (which in few cases are not efficiently removed) and median baseline subtraction. A similar effect is caused by very bright sources: in this case, the problem resides in the median baseline subtraction only. Suggested solutions:
      • switch to a baseline subtraction using a polynomial fitting using the optional task baselineRemovalPolynomial. If there are no jumps in the timelines, you may also try to run the baseline removal on the entire timeline;
      • in the case of bright sources, you may try to mask them before running the baseline removal (either median or polynomial): you can use this script as a template
-->
>
>
<!--
      * Most of the stripes that are present in the final maps are due to a combination of thermal drifts (which in few cases are not efficiently removed) and median baseline subtraction. A similar effect is caused by very bright sources: in this case, the problem resides in the median baseline subtraction only. Suggested solutions:
         * switch to a baseline subtraction using a polynomial fitting using the optional task =baselineRemovalPolynomial=. If there are no jumps in the timelines, you may also try to run the baseline removal on the entire timeline;
         * in the case of bright sources, you may try to mask them before running the baseline removal (either median or polynomial): you can use this script as a template
-->
 
  • De-glitchter masks faint sources
    • The de-glitcher is a very delicate process. In particular, for data taken in Parallel Mode (sampling at 10Hz) and at high speed (60"/s) the de-glitcher with standard parameters may flag very faint sources as glitches. Bright sources are different from glitches in that they have a gaussian (i.e. beam/PSF) shape. For faint sources, the sampling rate could be not high enough and hence they have a "delta" shape, which is similar to a small glitch. The user might try to modify the correlation parameter to 0.95: this will decrease the number of detected glitches.
Line: 263 to 223
 
  • Some sources have saturated the ADC and the corresponding data have been masked
    • There is nothing a user can do: the source was simply too bright. If the user has other sources still not observed and of the same intensity, it is suggested to change the AORs to use the bright source mode.
Changed:
<
<
<--   * Thermistor jumps 
    • As of HIPE 6.0.3, a new module together called signalJumpDetector in place to identify the jump and to exclude the affected thermistor(s).
-->
>
>
<!--   * *Thermistor jumps*
      * As of HIPE 6.0.3, a new module together called =signalJumpDetector= in place to identify the jump and to exclude the affected thermistor(s). 
-->
 
  • Cooler temperature variations
Changed:
<
<
    • The cooler temperature variations, as explained in greater details in the SPIRE Data Reduction Guide, section 6.4, can affect observations performed soon after the cooler recycle. The steep rise of the sub-K detector temperature is also known as the cooler burp and there is a quality flag coolerBurpDetected in HIPE v11 or later that indicates if the observation was performed during this period.
      InfoThe current list of observations known to have cooler temperature effects is here. Note that not all observations in this list raised the coolerBurpDetected flag.
>
>
    • The cooler temperature variations, as explained in greater details in the SPIRE Data Reduction Guide, section 6.4, can affect observations performed soon after the cooler recycle. The steep rise of the sub-K detector temperature is also known as the cooler burp and there is a quality flag
coolerBurpDetected in HIPE v11 or later that indicates if the observation was performed during this period.
InfoThe current list of observations known to have cooler temperature effects is here. Note that not all observations in this list raised the coolerBurpDetected flag.
 
  • NaNs pixels present in the PSW, PMW and/or PLW (Level 2) maps
    • This effect, related to data masked for various reasons and poor coverage (not enough redundancy), is more evident in single fast-scan Parallel Mode maps. To avoid NaNs, increase the pixel's dimension (i.e., decrease the map's resolution)
Changed:
<
<
<--   * WCS in 3-colour images 
    • In all observation reduced with HIPE 8, the task createRgbImage puts wrong WCS in the output. Instead of using the WCS provided by the WCS input parameter, this task uses the WCS of one of the input images. This has been fixed in HIPE 9
-->
>
>
<!--   * *WCS in 3-colour images*
      * In all observation reduced with HIPE 8, the task =createRgbImage= puts wrong WCS in the output. Instead of using the WCS provided by the WCS input parameter, this task uses the WCS of one of the input images. This has been fixed in HIPE 9
-->
 
  • Quality flags
    • Currently, the quality flags at the quality context inside the observation context are just meant for HSC/ICC internal evaluation of the quality of the products and not for the users. In case the data had some serious quality problem, the PI of the program has been contacted about it. Otherwise, only information in the quality summary, when available, should concern the observers.

  • Planck derived zero offsets
Changed:
<
<
    • The extended calibrated maps (extdPxW in level-2, 2.5 or 3) incorporate zero level offsets derived from Planck-HFI. For small size SPIRE maps, smaller than ~30 arcmin, the zero-offset can be rather uncertain, due to the large Planck beam (8 arcmin). In such cases the interpretation of the zero offset as the absolute zero level must to be treated with extreme caution.
>
>
    • The extended calibrated maps (
extdPxW in level-2, 2.5 or 3) incorporate zero level offsets derived from Planck-HFI. For small size SPIRE maps, smaller than ~30 arcmin, the zero-offset can be rather uncertain, due to the large Planck beam (8 arcmin). In such cases the interpretation of the zero offset as the absolute zero level must to be treated with extreme caution.
 
Changed:
<
<
>
>
 

SPIRE Photometer Release Notes

Changed:
<
<
SPIRE scan map release note : 20 October 2009
>
>
SPIRE scan map release note : 20 October 2009
 
Changed:
<
<
SPIRE Small Scan Map AOT release note: 17 Mar 2010
>
>
SPIRE Small Scan Map AOT release note: 17 Mar 2010
 
Changed:
<
<
<--
SPIRE Point Source Mode release note: 30 Apr 2010 
-->
>
>
<!--
<a class="red" href="http://herschel.esac.esa.int/Docs/AOTsReleaseStatus/SPIRE_PointSource_AOT_ReleaseNote_30Apr2010.pdf">SPIRE Point Source Mode release note</a>: 30 Apr 2010 
-->
 
Changed:
<
<

SPIRE Spectroscopy

>
>

SPIREnbsp;Spectroscopy
 
Changed:
<
<
  • In order to obtain the best possible Level 2 SPIRE FTS data, the observations should be reprocessed with the latest HIPE User Release.
>
>
  • *In order to obtain the best possible Level 2 SPIRE FTS data, the observations should be reprocessed with the [[http://herschel.esac.esa.int/HIPE_download.shtml][
latest HIPE User Release]]*.
 
  • Bright source mode: observations in bright source mode processed with HIPE v7 or earlier result in spectra that have no scientific value. Bright source observations processed with HIPE v8 or above are fine.
Line: 308 to 268
 
  • Faint point source observations: if the SSW and SLW bands do not match up for a source that is known to be a point source, observers are recommended to run the background subtraction script in HIPE - more details are in the SPIRE Data Reduction Guide. If this doesn't solve the problem, the observer is encouraged to contact the HSC helpdesk or the FTS User Support Group.

  • Artefacts in the continuum for few repetitions
Changed:
<
<
    • Very small repetition numbers (e.g. 2 or 4) make 2nd level deglitching, which is based on a statistical outlier criterion, more challenging. The deglitching module may either not identify a glitch at all or it may not remove it completely. In cases where the glitch is located within the double-sided portion of the interferogram, the additional energy from the glitch will translate into artefacts of the continuum level. This kind of problem can be identified by inspecting all detectors from all scans in the level-1 spectral products. For some detectors, one or several scans may appear to be outliers. As a work-around, it is recommended to reprocess the data with a lower thresholdFactor when calling deglitchIfgm(). If the problem persists, the identified detector should be removed from the applicable scan in the SDI product.
      (NB: This affects HIPE 6 and higher)
>
>
    • Very small repetition numbers (e.g. 2 or 4) make 2nd level deglitching, which is based on a statistical outlier criterion, more challenging. The deglitching module may either not identify a glitch at all or it may not remove it completely. In cases where the glitch is located within the double-sided portion of the interferogram, the additional energy from the glitch will translate into artefacts of the continuum level. This kind of problem can be identified by inspecting all detectors from all scans in the level-1 spectral products. For some detectors, one or several scans may appear to be outliers. As a work-around, it is recommended to reprocess the data with a lower
thresholdFactor when calling deglitchIfgm(). If the problem persists, the identified detector should be removed from the applicable scan in the SDI product.
(NB: This affects HIPE 6 and higher)
 
  • Line fitting
Changed:
<
<
    • If you know that the line you wish to measure is unresolved then you may want to fix the line width to the instrumental line width. For the Sinc model, implemented in the SpectrumFitterGUI you should put the Sinc width equal to resolution/π, which, for HR is 1.2/π = 0.382 GHz.
>
>
    • If you know that the line you wish to measure is unresolved then you may want to fix the line width to the instrumental line width. For the Sinc model, implemented in the
SpectrumFitterGUI you should put the Sinc width equal to resolution/π, which, for HR is 1.2/π = 0.382 GHz.
 
  • Point source and extended source spectra
    • If your level-2 spectra show characteristic jumps at ~1250 GHz and ~750 GHz, and the spectra from the two bands SSW and SLW do not match, then your target is extended or semi-extended in the SPIRE beam. You need to use the semi-extended correction tool (SECT) available since HIPE v10. Check the the SPIRE Data Reduction Guide (SDRG).

  • Quality flags in the quality
    • The quality flags at the quality context inside the observation context are just meant for HSC/ICC internal evaluation of the quality of the products and not for the users. In case the data had some serious quality problem, the PI of the program has been contacted about it. Otherwise, only information in the Quality Summary, when available, should concern the observers.
Changed:
<
<
    • For observations before OD1000 there could be an erroneously raised or a missing flag RADECACC. This flag is calculated as the difference between the observer requested target coordinates (kept in a metadata raNominal, decNominal) and the average pointing of the telescope during the observation (excluding the slew). This is an indicator of the pointing accuracy of the observation. If greater than 2.2" then RADECACC is raised in the QC context. The calculation of the average (ra, dec) during the observation does not take into account the known offset of 1.7" of the BSM home position for OD < 1000 and consequently observations with better pointing than 2.2" may be flagged and vice versa, observations at more than 2.2" may not be flagged. If you want to check the pointing offset with respect to the requested target coordinates then you should use the pointing information for the central detector SSWD4 in the level-2 context.
      A new quality parameter raDecOffset is introduced in HIPE v13, in level-2 products, which contains the correct offset between the raNominal, decNominal and ra,dec for the FTS central detector.
>
>
    • For observations before OD1000 there could be an erroneously raised or a missing flag
RADECACC. This flag is calculated as the difference between the observer requested target coordinates (kept in a metadata raNominal, decNominal) and the average pointing of the telescope during the observation (excluding the slew). This is an indicator of the pointing accuracy of the observation. If greater than 2.2" then RADECACC is raised in the QC context. The calculation of the average (ra, dec) during the observation does not take into account the known offset of 1.7" of the BSM home position for OD < 1000 and consequently observations with better pointing than 2.2" may be flagged and vice versa, observations at more than 2.2" may not be flagged. If you want to check the pointing offset with respect to the requested target coordinates then you should use the pointing information for the central detector SSWD4 in the level-2 context.
A new quality parameter raDecOffset is introduced in HIPE v13, in level-2 products, which contains the correct offset between the raNominal, decNominal and ra,dec for the FTS central detector.
 
  • Observations during the steep rise of the sub-K temperature
Changed:
<
<
    • During the first few hours after the cooler was recycled the bolometers' temperature (the sub-K temperature) undergoes a steep rise before it reaches a stable plateau. Observations during this period suffer overcorrection of the instrument/telescope emission. This is more significant for faint targets and can be identified as an unphysical slope of the SLW spectrum, with an important negative gap in the region that overlaps with SSW (see the Figure). One way to check if your observation is within this problematic category is to get the median sub-K temperature for the first building block: print MEDIAN(obs.level0_5.get(0xA1060001).nhkt['signal']['SUBKTEMP'].data), where obs is the pre-loaded observational context. If the result is less than 0.2869 K then the observation is affected by this.
      This effect is fixed in HIPE v13.
>
>
    • During the first few hours after the cooler was recycled the bolometers' temperature (the sub-K temperature) undergoes a steep rise before it reaches a stable plateau. Observations during this period suffer overcorrection of the instrument/telescope emission. This is more significant for faint targets and can be identified as an unphysical slope of the SLW spectrum, with an important negative gap in the region that overlaps with SSW (see the Figure). One way to check if your observation is within this problematic category is to get the median sub-K temperature for the first building block:
print MEDIAN(obs.level0_5.get(0xA1060001).nhkt['signal']['SUBKTEMP'].data), where obs is the pre-loaded observational context. If the result is less than 0.2869 K then the observation is affected by this.
This effect is fixed in HIPE v13.
 
Changed:
<
<
Steep-subK-case.png
>
>
Steep-subK-case.png
 
Changed:
<
<
<--   * FTS Array footprint user script 
    • For very large area maps the script produces wrong overlay (offset from the real position) if the default map projections (tangential) is used. The workaround is to use smaller map where the tangential projection centre is near the FTS target position.
-->
>
>
<!--   * *FTS Array footprint user script*
      * For very large area maps the script produces wrong overlay (offset from the real position) if the default map projections (tangential) is used. The workaround is to use smaller map where the tangential projection centre is near the FTS target position.
-->
 
Line: 332 to 302
 
Deleted:
<
<
 

SPIRE Spectroscopy Release Notes

Changed:
<
<
SPIRE Spectrometer Point Source AOT release note: 21 May 2010
>
>
SPIRE Spectrometer Point Source AOT release note: 21 May 2010
 
Changed:
<
<
SPIRE Spectrometer Bright Source Mode AOT release note: 7 Sep 2010
>
>
SPIRE Spectrometer Bright Source Mode AOT release note: 7 Sep 2010
 
Changed:
<
<
SPIRE Spectrometer Mapping AOT release note: 30 Apr 2010
>
>
SPIRE Spectrometer Mapping AOT release note: 30 Apr 2010
 
Changed:
<
<

HIFI observations (point mode, spectral survey and mapping observations)

>
>

HIFInbsp;observationsnbsp;(pointnbsp;mode,nbsp;spectralnbsp;surveynbsp;andnbsp;mappingnbsp;observations)
 
Changed:
<
<
In order to obtain the best possible Level 2 HIFI data the observations should be reprocessed with the latest HIPE User Release.
>
>
*In order to obtain the best possible Level 2 HIFI data the observations should be reprocessed with the [[http://herschel.esac.esa.int/HIPE_download.shtml][ latest HIPE User Release]].*
 
  • Corrupted data-frames
    • There is a limited number of obsids where one or more data-frames from the spectrometers is corrupted in some fashion and leads to level 2 products that cannot be properly calibrated. A scheme was put in place some versions ago to flag or simply remove those frames at level 0 so the final products won't be affected by them. This scheme however suffered from a regression bug in 11.1 so that the corresponding products did not benefit from this correction properly. We note however that this should only concern about 50 obsids in the whole HIFI archive. This will be fixed in HIPE 12.

  • Strong/Weak Spurs
Changed:
<
<
    • Spurs are still affecting the HIFI data at spot frequency points. They will be flagged automatically in the data using a SpurFinder task when present above a certain threshold in the internal load spectra. There are still some spur categories that are not yet properly detected but this is improving in each new versions of HIPE. Some spurs are so intense and broad that they can lead to the loss of complete WBS sub-bands. Note that the strong spur affecting the upper end of band 1a has now been completely cleaned - see the corresponding technical note: HIFI Information note on the removal of spurs in band 1a and the sampling in the mapping modes. The same is true for a weaker spur that did affect data in band 7b close to LO = 1834 GHz.
>
>
    • Spurs are still affecting the HIFI data at spot frequency points. They will be flagged automatically in the data using a SpurFinder task when present above a certain threshold in the internal load spectra. There are still some spur categories that are not yet properly detected but this is improving in each new versions of HIPE. Some spurs are so intense and broad that they can lead to the loss of complete WBS sub-bands. Note that the strong spur affecting the upper end of band 1a has now been completely cleaned - see the corresponding technical note: HIFI Information note on the removal of spurs in band 1a and the sampling in the mapping modes. The same is true for a weaker spur that did affect data in band 7b close to LO = 1834 GHz.
 
  • Unpumped data
Changed:
<
<
    • Due to gaps in the LO output power over the HIFI frequency range, some spectral scans will contain spectra that cannot be used due their very high noise temperature levels. These data should be discarded from further processing (e.g. deconvolution). For single frequency observations, there exist some places where the theoretical noise temperature (i.e. the one advertised by HSpot) should be nominal, but the sensitivity effectively achieved at the time of observations is noticeably worse. These are cases where the mixer tuning algorithm does not fully converge. Such cases have e.g. been reported around LO frequencies of ~ 1441.5 GHz. An overview of the expected unpumped frequency areas is given in the AOT release notes. Users observing such effects should contact the Helpdesk.
>
>
    • Due to gaps in the LO output power over the HIFI frequency range, some spectral scans will contain spectra that cannot be used due their very high noise temperature levels. These data should be discarded from further processing (e.g. deconvolution). For single frequency observations, there exist some places where the theoretical noise temperature (i.e. the one advertised by HSpot) should be nominal, but the sensitivity effectively achieved at the time of observations is noticeably worse. These are cases where the mixer tuning algorithm does not fully converge. Such cases have e.g. been reported around LO frequencies of ~ 1441.5 GHz. An overview of the expected unpumped frequency areas is given in the AOT release notes. Users observing such effects should contact the Helpdesk.
 
  • Standing waves
Changed:
<
<
    • There are a variety of standing waves known to affect the data at Level2 - see the AOT release notes for more details. At the present time, the FitHifiFringe tool usually does a decent job for most standing waves in SIS bands. In HEB bands, where the standing wave is not of optical nature and does have a strictly sinusoidal shape, FitHifiFringe can help when restricted to the frequency range where the line is present. Another alternative in the form of a dedicated algorithm has been developed (so-called "matching technique") and is offered in the hebCorrection task from HIPE 12.0 onwards (details can be found in Chapter 10 of the HIFI Data Reduction Guide). You can contact Ian Avruch (i.avruch@sron.nl) at the HIFI ICC for further details. In any case, it is advised not to use more than 3 components in FitHifiFringe. Since HIPE 5, a number of new features have been made available in FitHifiFringe. One of them ("sub_base" option) allows you to fit a baseline at the same time. Since FitHifiFringe can identify automatically the lines in order to set masks, users should be careful when treating lines with large wings. In this case, the user should set a window by hand. The automatic line masking should then be switched off.
>
>
    • There are a variety of standing waves known to affect the data at Level2 - see the AOT release notes for more details. At the present time, the !FitHifiFringe tool usually does a decent job for most standing waves in SIS bands. In HEB bands, where the standing wave is not of optical nature and does have a strictly sinusoidal shape, FitHifiFringe can help when restricted to the frequency range where the line is present. Another alternative in the form of a dedicated algorithm has been developed (so-called "matching technique") and is offered in the hebCorrection task from HIPE 12.0 onwards (details can be found in Chapter 10 of the HIFI Data Reduction Guide). You can contact Ian Avruch (i.avruch@sron.nl) at the HIFI ICC for further details. In any case, it is advised not to use more than 3 components in FitHifiFringe. Since HIPE 5, a number of new features have been made available in FitHifiFringe. One of them ("sub_base" option) allows you to fit a baseline at the same time. Since FitHifiFringe can identify automatically the lines in order to set masks, users should be careful when treating lines with large wings. In this case, the user should set a window by hand. The automatic line masking should then be switched off.
 
    • In case of strong source continuum some standing waves can be enhanced. Such standing waves can be significantly reduced in amplitude using an alternative pipeline algorithm known as the Modified Passband Technique , which is described in the Standing Wave Removal chapter of the HIFI Data Reduction Guide. Note that in 9.0, a bug affected the doFilterLoad task in such a way that it ignored the user-input parameters, resulting in the task always applying the "cubic_spline" method with default parameters - i.e. the "fft" method will not be applied despite being set in the task inputs. This was fixed in 9.1 before the SPG 9.1.0 bulk reprocessing of the archive.

  • Baseline distortion
Changed:
<
<
    • Residual drift from the detector response can translate into imperfect baseline structures. This usually manifests as wavy structures, slopes, or overall level offsets of the spectral baselines. Such artefacts are expected to be enhanced e.g. at the borders of spectra obtained in diplexer bands (sub-bands 1 and 4 for bands 3 and 4, sub-bands 2 and 4 for bands 6 and 7). Similar artefacts are also expected when using modes without reference, such as the FreqSwitchNoRef, or the LoadChopNoRef, and are enhanced with strong continuum sources. Please refer to the Data Reduction Guide for details about the most recent improvements to the FitBaseline task.
>
>
    • Residual drift from the detector response can translate into imperfect baseline structures. This usually manifests as wavy structures, slopes, or overall level offsets of the spectral baselines. Such artefacts are expected to be enhanced e.g. at the borders of spectra obtained in diplexer bands (sub-bands 1 and 4 for bands 3 and 4, sub-bands 2 and 4 for bands 6 and 7). Similar artefacts are also expected when using modes without reference, such as the FreqSwitchNoRef, or the LoadChopNoRef, and are enhanced with strong continuum sources. Please refer to the Data Reduction Guide for details about the most recent improvements to the FitBaseline task.
 
  • Saturation
    • Saturation of the WBS CCDs may be observed at spot frequencies. These are usually due to either strong and broad spurs, or to cases where only marginal pump level could be achieved due to shortage of LO output power. Users can go back to HSpot to visualize in the frequency editor where their AORs were potentially affected by such effects. HIPE will normally flag the affected spectrometer channels.
Line: 398 to 369
 
    • Some expert users may want to reprocess their data starting from raw telemetry (so-called level-1). From HIPE 9 onwards additional telemetry (notably from the ACMS system) was added to the products. In order to avoid problems with missing these products from earlier downloads, the following can be temporarily added to your local configuration property: hcss.ia.spg.ops.AuxPlugin.products={..., herschel.ia.obs.auxiliary.acms.AcmsTelemetryProduct, ...}

  • Flux Conservation in Spectral Cubes from Mapping Observations
Changed:
<
<
    • The doGridding task in the HIFI pipeline is responsible for convolving the spectral datasets acquired in an OTF or DBS Raster mapping observation into a spectral cube with a specified pixel scale, which by default should match how the scan lines and readout points within each line were spaced during the observation. The scheme of signal filtering and interpolation to put the data on the specified grid may affect the overall flux conservation, at a level which is low but you should be aware of. For example, the total signal summed from a spectral cube produced using a Gaussian filter over the datasets of a Nyquist-sampled OTF map is generally < 1% lower than the sum of the signal taken directly from the input datasets (the Level 2 HTP datasets) before convolution. A part or all of this slight mismatch may be on the assumed versus actual beam shape at the observed frequency. If you wish to put the map on a coarser grid, effectively reducing the spatial resolution to a wider beam in order to match another observation, then the flux losses become more noticeable. Doubling the pixels sizes from their default (native map point spacing) can reduce the total flux by as much as 10%, accompanied by an increase in baseline RMS noise. The effect is more prevalent in OTF maps than DBS Raster, and in addition to deviations from ideal beam shape characteristics becoming more important, the filtering and interpolation method, and parameter values can be influential. No changes to the doGridding algorithm are planned, and this issue applies to all HIPE versions.
>
>
    • The doGridding task in the HIFI pipeline is responsible for convolving the spectral datasets acquired in an OTF or DBS Raster mapping observation into a spectral cube with a specified pixel scale, which by default should match how the scan lines and readout points within each line were spaced during the observation. The scheme of signal filtering and interpolation to put the data on the specified grid may affect the overall flux conservation, at a level which is low but you should be aware of. For example, the total signal summed from a spectral cube produced using a Gaussian filter over the datasets of a Nyquist-sampled OTF map is generally < 1% lower than the sum of the signal taken directly from the input datasets (the Level 2 HTP datasets) before convolution. A part or all of this slight mismatch may be on the assumed versus actual beam shape at the observed frequency. If you wish to put the map on a coarser grid, effectively reducing the spatial resolution to a wider beam in order to match another observation, then the flux losses become more noticeable. Doubling the pixels sizes from their default (native map point spacing) can reduce the total flux by as much as 10%, accompanied by an increase in baseline RMS noise. The effect is more prevalent in OTF maps than DBS Raster, and in addition to deviations from ideal beam shape characteristics becoming more important, the filtering and interpolation method, and parameter values can be influential. No changes to the doGridding algorithm are planned, and this issue applies to all HIPE versions.
 
  • Retrieval of calibration files from the HSA
    • In HIPE versions from the track 9, there is small bug in the getHifiCal task. The task may erroneously report that the latest calibration tree available is HIFI_CAL_9_0, while more recent versions are now available. This issue is solved in HIPE 10. This is not a problem with the HIPE 11 build. One possible work-around is to clear the calibration pool in your .hcss/lstore, and call the task again or simply use the updated HIPE version. Then the latest calibration tree should be offered for download.
Line: 408 to 379
 

HIFI Release Notes

Changed:
<
<
HIFI Dual Beam Switching Observing Modes AOT release note: 9 Mar 2010
>
>
HIFI Dual Beam Switching Observing Modes AOT release note: 9 Mar 2010
 
Changed:
<
<
HIFI Oberving Modes AOT release note: 11 Apr 2010
>
>
HIFI Oberving Modes AOT release note: 11 Apr 2010
 
Changed:
<
<
HIFI AOT Observing Mode Release and Performance Notes v3.0: 24 September 2011
>
>
HIFI AOT Observing Mode Release and Performance Notes v3.0: 24 September 2011
 

Other Technical Notes

PACS documents: see PACS Instrument and Calibration webpage

Deleted:
<
<
<-- 
PACS Spectroscopy AOR Update Guide for Routine Phase Observations: 10 Mar 2010
-->
 
Changed:
<
<
HIFI Information note on Mapping Modes: 30 Jun 2010
>
>
<!-- 
<a class="red" href="http://herschel.esac.esa.int/Docs/TechnicalNotes/PACS_Spectroscopy_AOR_Update_Guide_10Mar2010.pdf">PACS Spectroscopy AOR Update Guide for Routine Phase Observations</a>: 10 Mar 2010
-->
 
Changed:
<
<
HIFI Beam Efficiencies from Mars Observations: 17 Nov 2010
>
>
HIFI Information note on Mapping Modes: 30 Jun 2010
 
Changed:
<
<
HIFI Information note on the removal of spurs in band 1a and the sampling in the mapping modes: 19 Nov 2010
>
>
HIFI Beam Efficiencies from Mars Observations: 17 Nov 2010
 
Added:
>
>
HIFI Information note on the removal of spurs in band 1a and the sampling in the mapping modes: 19 Nov 2010
 

Calibration Documents

PACS photometer and spectrometer calibration documents: see PACS Instrument and Calibration webpage

Changed:
<
<
<-- PACS Spectroscopy Performance and Calibration: 11 Mar 2010 

PACS Photometer Point Spread Function (PSF): 03 Nov 2010 -->

SPIRE Photometer Beams (FTP repository)

HIFI System noise temperature IF spectra (FTP repository)

<--
*  Set ALLOWTOPICCHANGE = DpMgGroup, HscCommunitySupportGroup
-->
>
>
<!-- <a class="red" href="http://herschel.esac.esa.int/Docs/Calibration/PACS_SpectroscopyPerformanceCalibration_11Mar2010.pdf">PACS Spectroscopy Performance and Calibration</a>: 11 Mar 2010 

<a class="red" href="http://herschel.esac.esa.int/Docs/Calibration/PACS_Point_Spread_Function_03Nov2010.pdf">PACS Photometer Point Spread Function (PSF)</a>: 03 Nov 2010 
-->
 
Added:
>
>
SPIRE Photometer Beams (FTP repository)
 
Added:
>
>
HIFI System noise temperature IF spectra (FTP repository)
 
Added:
>
>
<!--
*  Set ALLOWTOPICCHANGE = Main.DpMgGroup, Main.HscCommunitySupportGroup
-->
 
META FILEATTACHMENT attr="h" autoattached="1" comment="" date="1437490181" name="obsSVVanomaly.csv" path="obsSVVanomaly.csv" size="2980" user="Main.LucaCalzoletti" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1437489475" name="ngc253_1342221743_blue.jpeg" path="ngc253_1342221743_blue.jpeg" size="957526" user="Main.LucaCalzoletti" version="1"

Revision 1292015-09-16 - AlvarGarcia

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Data Products Known Issues

Line: 22 to 22
 

TABLE OF CONTENTS

Added:
>
>

METADATA AND FITS KEYWORDS

Message about DATE-OBS Some internal HCSS metadata are renamed by the HCSS software when translating to FITS. One special case is startDate which gets written to FITS as both DATE-OBS and DATE_OBS. This is done for compatibility with legacy Spitzer software (namely MOPEX).
 

PACS Photometer (scan mapping)

Special messages

Revision 1282015-08-26 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Data Products Known Issues

Line: 264 to 264
 -->

  • Cooler temperature variations
Changed:
<
<
    • The cooler temperature variations, as explained in greater details in the SPIRE Data Reduction Guide, section 6.4, can affect observations performed soon after the cooler recycle. The steep rise of the sub-K detector temperature is also known as the cooler burp and there is a quality flag coolerBurpFound in HIPE v11 or later that indicates if the observation was performed during this period.
>
>
    • The cooler temperature variations, as explained in greater details in the SPIRE Data Reduction Guide, section 6.4, can affect observations performed soon after the cooler recycle. The steep rise of the sub-K detector temperature is also known as the cooler burp and there is a quality flag coolerBurpDetected in HIPE v11 or later that indicates if the observation was performed during this period.
      InfoThe current list of observations known to have cooler temperature effects is here. Note that not all observations in this list raised the coolerBurpDetected flag.
 
  • NaNs pixels present in the PSW, PMW and/or PLW (Level 2) maps
    • This effect, related to data masked for various reasons and poor coverage (not enough redundancy), is more evident in single fast-scan Parallel Mode maps. To avoid NaNs, increase the pixel's dimension (i.e., decrease the map's resolution)

Revision 1272015-07-27 - LucaCalzoletti

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Data Products Known Issues

Line: 34 to 34
  Warning for pointing anomaly in HIPE/SPG 13
Changed:
<
<
PACS and SPIRE photometry observations reported in this CSV file are affected by a known problem related to the reset of the Spacecraft Velocity Vector (SVV) during the upload of the star-tracker's defective pixel table. For the affected observations, the pointing of the telescope can be off along the scan direction, and shifted up to 20 arcsec.
>
>
PACS and SPIRE photometry observations reported in this CSV file are affected by a known problem related to the reset of the Spacecraft Velocity Vector (SVV) during the upload of the star-tracker's defective pixel table. For the affected observations, the pointing of the telescope can be off along the scan direction, and shifted up to 20 arcsec.
 This effect will be corrected in HIPE/SPG14.

General notes

Line: 442 to 442
 -->

Changed:
<
<
>
>
 
META FILEATTACHMENT attr="h" autoattached="1" comment="" date="1437490181" name="obsSVVanomaly.csv" path="obsSVVanomaly.csv" size="2980" user="Main.LucaCalzoletti" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1437489475" name="ngc253_1342221743_blue.jpeg" path="ngc253_1342221743_blue.jpeg" size="957526" user="Main.LucaCalzoletti" version="1"
Added:
>
>
META FILEATTACHMENT attr="h" autoattached="1" comment="" date="1437982718" name="SVV_resets_aberration_shift.csv" path="SVV_resets_aberration_shift.csv" size="7875" user="Main.LucaCalzoletti" version="1"
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1404218642" name="Steep-subK-case.png" path="Steep-subK-case.png" size="91418" user="Main.IvanV" version="1"

Revision 1262015-07-23 - DavidTeyssier

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Data Products Known Issues

Line: 229 to 229
 

SPIRE Photometry

Added:
>
>

Special messages

Warning for pointing anomaly in HIPE/SPG 13

PACS and SPIRE photometry observations reported in this CSV file are affected by a known problem related to the reset of the Spacecraft Velocity Vector (SVV) during the upload of the star-tracker's defective pixel table. For the affected observations, the pointing of the telescope can be off along the scan direction, and shifted up to 20 arcsec. This effect will be corrected in HIPE/SPG14.

General notes

 
  • In order to obtain the best possible Level 2 SPIRE photometry data, the observations might have to be reprocessed with the latest HIPE User Release.
    • Warning, important Please note that there was a bug in the destriper task included in HIPE 9.0 that may affect your final map, especially if there are bright objects in the observed field. This has been corrected since HIPE 9.1. If your observation falls in the mentioned category, you are strongly advised to update your HIPE installation.
    • Warning, important In HIPE 10.0 the flagging of thermistor jumps in the level 0.5 to 1 data reduction is not set properly. This induces the destriper to work improperly and to leave stripes in the final map. It has been solved starting with HIPE 10.1.

Revision 1252015-07-22 - PedroGarciaLario

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Data Products Known Issues

Line: 9 to 9
  From the user point of view, the most important piece of the HCSS system is the Herschel Interactive Processing Environment (HIPE). HIPE allows the astronomer the possibility of inspecting the data and re-process them, if the results from the automatic pipeline are not good enough for his/her purposes. Because HIPE is part of the HCSS, the latest version of HIPE will have the most up-to-date pipeline, calibrations and documentation available.
Changed:
<
<
For example, some data in the HSA may be marked SPG v9.1.0, which means that they were processed with that HCSS version (see metadata of the observation to find pipeline processing used for current product in the archive). If the current HCSS version is 12.0.0, HIPE is informally called HIPE 12.0. If you re-reduce the data with this HIPE version with the default parameters, you will get the result that would be produced with the official 12.0.0 pipelines at the Herschel Science Centre. As of the release of HCSS 12.0 nearly all archive products have been processed with SPG version 11.1.0.
>
>
For example, some data in the HSA may be marked SPG v13.0.0, which means that they were processed with that HCSS version (see metadata of the observation to find pipeline processing used for current product in the archive). If the current HCSS version is 13.0.0, HIPE is informally called HIPE 13.0. If you re-reduce the data with this HIPE version with the default parameters, you will get the result that would be produced with the official 13.0.0 pipelines at the Herschel Science Centre.
  The latest version of HIPE can be obtained here. Known HIPE issues/problems/bugs are detailed here.
Changed:
<
<
In what follows, we provide a summary of the known issues that you may encounter when inspecting data processed with the automatic pipelines SPG versions 6.1.x to 12.x.x Most can be resolved by running the pipelines within HIPE and optimizing their parameters as explained below.
>
>
In what follows, we provide a summary of the known issues that you may encounter when inspecting data processed with the automatic pipelines SPG versions 6.1 to 13.0. Most can be resolved by running the pipelines within HIPE and optimizing their parameters as explained below.
 
<--This is a comment: it will be ignored by the browser-->

Revision 1242015-07-22 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Data Products Known Issues

Line: 34 to 34
  Warning for pointing anomaly in HIPE/SPG 13
Changed:
<
<
The observations reported in that CSV file are affected by a known problem related with the reset of the Spacecraft Velocity Vector (SVV) during the upload of the star-trackers pixel defective table. Because of this reset, the pointing of the telescope can be off along the scan direction and shifted up to 20 arcsec.
>
>
PACS and SPIRE photometry observations reported in this CSV file are affected by a known problem related to the reset of the Spacecraft Velocity Vector (SVV) during the upload of the star-tracker's defective pixel table. For the affected observations, the pointing of the telescope can be off along the scan direction, and shifted up to 20 arcsec.
 This effect will be corrected in HIPE/SPG14.

General notes

Revision 1232015-07-21 - LucaCalzoletti

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Data Products Known Issues

Line: 24 to 24
 

PACS Photometer (scan mapping)

Added:
>
>

Special messages

Warning for Unimap maps in HIPE/SPG 13

Some Unimap maps show overshooting effects around very bright sources that are surrounded by a diffuse and relatively faint background. See, as an example, the blue image of NGC253 (obsID: 1342221743). This effect is due to a not optimised convergence of the GLS algorithm and it will be corrected in HIPE/SPG14. The user can request for a dedicated reprocessing to the Herschel Science Center Helpdesk or he can use the corresponding Level 2.5 JScanam maps (HPPJSMAP[B][R] blue and red products), which are not affected by this issue.

Warning for pointing anomaly in HIPE/SPG 13

The observations reported in that CSV file are affected by a known problem related with the reset of the Spacecraft Velocity Vector (SVV) during the upload of the star-trackers pixel defective table. Because of this reset, the pointing of the telescope can be off along the scan direction and shifted up to 20 arcsec. This effect will be corrected in HIPE/SPG14.

General notes

 
  • To produce the highest quality maps, you should consider re-processing or fine-tuning the observations with the latest HIPE User Release. Maps available from the HSA are created within a bulk processing framework, and a reprocessing while fine-tuning the mapper parameters, according to the characteristics of the observed sky region, could enhance the quality of the final maps.

  • Important note:
Line: 419 to 434
 -->

Changed:
<
<
META FILEATTACHMENT attachment="Steep-subK-case.png" attr="" comment="" date="1404218641" name="Steep-subK-case.png" path="Steep-subK-case.png" size="91418" stream="Steep-subK-case.png" user="Main.IvanV" version="1"
>
>

META FILEATTACHMENT attr="h" autoattached="1" comment="" date="1437490181" name="obsSVVanomaly.csv" path="obsSVVanomaly.csv" size="2980" user="Main.LucaCalzoletti" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1437489475" name="ngc253_1342221743_blue.jpeg" path="ngc253_1342221743_blue.jpeg" size="957526" user="Main.LucaCalzoletti" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1404218642" name="Steep-subK-case.png" path="Steep-subK-case.png" size="91418" user="Main.IvanV" version="1"

Revision 1222015-06-23 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Data Products Known Issues

Line: 119 to 119
 

PACS Spectroscopy

Added:
>
>

Special messages

Warning for drizzled cubes in HIPE/SPG 13

Due to an oversight in the pipeline scripts of HIPE 13, the drizzled cubes you get for chop-nod line scan observations created by "SPG 13" will have incorrect fluxes. Therefore you should not use the drizzled cubes downloaded from the HSA if the Meta datum "creator" is "SPG 13.0", whether they are within the ObservationContext at Level 2, or part of a Standalone Browse Product download. Any drizzled cubes created for unchopped mode observations will be unaffected. For chop-nod range scan observations, drizzled cubes are not created by the SPG 13 pipeline anyway. This oversight will be corrected in HIPE/SPG 14.

For the affected observations you have the following alternatives: 1) Use the projected cubes instead, which you will also find within the ObservationContext at Level 2, but which are not part of the Standalone Browse Product download for the affected observations. 2) Run one of the interactive pipeline scripts to create the drizzled cubes yourself. Contact the Herschel Helpdesk first to ask for an updated pipeline script to do this, since the interactive pipeline scripts in HIPE 13 (and 12) also contain the mistake.

For Your Information: Drizzled cubes FITS files, and their name as listed in an ObservationContext, are: HPS3DD[R|B] (red and blue) Standalone browse product drizzled cube FITS files, and their name as listed in an ObservationContext in the browse product section are: HPS3DEQ[R|B] (red and blue, equidistant wavelength grid version of the drizzled cubes). Projected cubes have the name: HPS3DP[R|B].

For more information about the standard and the standalone browse cubes provided for PACS spectroscopy, see the PACS Products Explained HIPE help document, which is also available from the PACS documentation webpage and the HIPE download webpage, both off the Herschel Science Centre webpage.

General notes

 
  • To produce the highest quality cubes possible, you should consider re-processing or fine-tuning the observations with the latest HIPE User Release. Cubes available from the HSA are created within a bulk processing framework, and a reprocessing while fine-tuning the important pipeline task parameters, according to the characteristics of the observation and source, could enhance the quality of the final results. The first two chapters of the PACS Data Reduction Guide for spectroscopy (HIPE 13) give information about the need to reprocess, and about what to do with HSA-obtained cubes before using them for science.

Revision 1212015-04-20 - ChrisPearson

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Data Products Known Issues

Line: 249 to 249
 SPIRE Point Source Mode release note: 30 Apr 2010 -->
Changed:
<
<

SPIRE/FTS Spectroscopy

>
>

SPIRE Spectroscopy

 
  • In order to obtain the best possible Level 2 SPIRE FTS data, the observations should be reprocessed with the latest HIPE User Release.

Revision 1202015-04-13 - IvanV

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Data Products Known Issues

Line: 237 to 237
 
  • Planck derived zero offsets
    • The extended calibrated maps (extdPxW in level-2, 2.5 or 3) incorporate zero level offsets derived from Planck-HFI. For small size SPIRE maps, smaller than ~30 arcmin, the zero-offset can be rather uncertain, due to the large Planck beam (8 arcmin). In such cases the interpretation of the zero offset as the absolute zero level must to be treated with extreme caution.
Changed:
<
<
>
>
 

SPIRE Photometer Release Notes

Revision 1192015-04-13 - TanyaLim

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Data Products Known Issues