create new tag
view all tags

Data Products Known Issues


The whole suite of Herschel software is known as the Herschel Common Science System (HCSS). This encompasses all the software aspects related to the mission: automatic pipelines, spacecraft calibrations, etc. As data are retrieved from the spacecraft they are ingested in the Herschel Science Archive (HSA) and processed with the current official version of the pipeline. That means that, at any given time, different data in the HSA may be reduced with different pipeline versions. The pipeline version is listed in the HSA GUI as SPG vX.X.X, where SPG stands for Standard Product Generator. The SPG version is available as the header keyword 'creator', in the data .fits files. Every HCSS version, the whole HSA is bulk re-processed with the same up-to-date version of the pipelines.

From the user point of view, the most important piece of the HCSS system is the Herschel Interactive Processing Environment (HIPE). HIPE allows the astronomer the possibility of inspecting the data and re-process them, if the results from the automatic pipeline are not good enough for his/her purposes. Because HIPE is part of the HCSS, the latest version of HIPE will have the most up-to-date pipeline, calibrations and documentation available.

For example, some data in the HSA may be marked SPG v13.0.0, which means that they were processed with that HCSS version (see metadata of the observation to find pipeline processing used for current product in the archive). If the current HCSS version is 13.0.0, HIPE is informally called HIPE 13.0. If you re-reduce the data with this HIPE version with the default parameters, you will get the result that would be produced with the official 13.0.0 pipelines at the Herschel Science Centre.

The latest version of HIPE can be obtained here. Known HIPE issues/problems/bugs are detailed here.

In what follows, we provide a summary of the known issues that you may encounter when inspecting data processed with the automatic pipelines SPG versions 6.1 to 13.0. Most can be resolved by running the pipelines within HIPE and optimizing their parameters as explained below.

Note that some of this information can also be found in the quality report of the observation (QC Report) and as metadata with the FITS keyword "PCAVEATS".



Message about DATE-OBS Some internal HCSS metadata are renamed by the HCSS software when translating to FITS. One special case is startDate which gets written to FITS as both DATE-OBS and DATE_OBS. This is done for compatibility with legacy Spitzer software (namely MOPEX).

PACS Photometer (scan mapping)

Special messages

Warning for Unimap maps in HIPE/SPG 13

Some Unimap maps show overshooting effects around very bright sources that are surrounded by a diffuse and relatively faint background. See, as an example, the blue image of NGC253 (obsID: 1342221743). This effect is due to a not optimised convergence of the GLS algorithm and it will be corrected in HIPE/SPG14. The user can request for a dedicated reprocessing to the Herschel Science Center Helpdesk or he can use the corresponding Level 2.5 JScanam maps (HPPJSMAP[B][R] blue and red products), which are not affected by this issue.

Warning for pointing anomaly in HIPE/SPG 13

PACS and SPIRE photometry observations reported in this CSV file are affected by a known problem related to the reset of the Spacecraft Velocity Vector (SVV) during the upload of the star-tracker's defective pixel table. For the affected observations, the pointing of the telescope can be off along the scan direction, and shifted up to 20 arcsec. This effect will be corrected in HIPE/SPG14.

General notes

  • To produce the highest quality maps, you should consider re-processing or fine-tuning the observations with the latest HIPE User Release. Maps available from the HSA are created within a bulk processing framework, and a reprocessing while fine-tuning the mapper parameters, according to the characteristics of the observed sky region, could enhance the quality of the final maps.

  • Important note:
    • PACS maps from any map-maker are in essence differential maps: the absolute level is undefined due to the dominating telescope background removed by map-makers. Hence it is not unusual if your background level is negative.

  • Intrinsic limitation of highpass filter maps
    • High-pass filtered maps in SPG13 are available at level 2 and level 2.5. The extended emission is filtered out in the maps since a rather small filter width is used to remove 1/f noise stripping: this is done to allows us to obtain the best sensitivity for point-sources. Bright sources are masked out during the highpass filtering, hence their flux is not too much affected by filtering. But faint sources are not or are inadequately masked out by the SPG processing, hence the flux loss for these point-source can reach up to 20--30%. Re-processing your data is recommended here.

  • MADmap issues
    • MADmap -- a GLS (Generalized Least Square) map-maker -- maps are no longer created by SPG13 and beyond. However, the pipeline script to do this yourself is provided in HIPE 13.
    • The background MADmap map is sometimes affected by wiggles, arising from the drift correction, but the overall maps have been reliable since SPG 10.
    • MADmap maps generated by SPG 12.0 and earlier are affected by a flux overestimate of about 6% in the red channel and a minor underestimate (~2%) in the blue channel. This is due to the fact that the optical distortion correction was not applied then.
    • A post-processing removes artifacts around bright sources, but slight artifacts are still seen around the very brightest sources.

  • JScanam
    • JScanam (an HCSS implementation of the Scanamorphos IDL map-maker) maps are available in level 2.5 and level 3 as of SPG 12 onwards. The JScanam ipipe script to process the maps is available in Hipe 11 onwards. Significant improvements have been achieved in the latest HIPE release (13), in terms of memory requirement, processing speed, and final map quality.

  • Unimap issues
    • Unimap (a GLS mapmaker) maps will be available in level 2.5 and level 3 as of SPG 13 onwards. The Unimap interactive pipeline script to reprocess the maps is available in HIPE 13 onwards. Unimap is Matlab software and it is invoked by a jython task. When running the interactive pipeline script, the latest Unimap release must be locally installed (instructions are given at the beginning of ipipe script).
    • A post-processing removes artifacts around bright sources, but slight artifacts are still seen round the very brightest sources. A fine tuning of post-processing parameters could mitigate these effects (see the PACS Data Reduction Guide for photometry).
    • In some cases, drifts due to the calibration blocks persistence is observed, and this is not properly corrected by the pre-processing.

  • Error maps
    • The error maps provided for the SPG13 map-makers (high-pass-filtered, JScanam, and Unimap) do not take into account the full noise propagation along the pipelines and thus they provide and approximation of the error.
    • The error map in the high-pass filter branch is scaled from the coverage map to take into account the correlated noise.
    • In JScanam maps, the standard deviation map of the flux computed in the sky pixels is provided.
    • For Unimap maps, the standard deviation map is also provided and it can be properly assumed as the error map only when associated with the Naïve map.

  • Pointing/Astrometry issues
    • The measured Herschel APE (Absolute Pointing Error) on pointed observation is better that 2 arcsec (1 sigma) and was improved throughout the mission down to 1 arcsec (1-sigma) on most of the fields/observations reprocessed with SPG 10.3 and higher. If your source is still severely mis-pointed you should contact the HSC helpdesk.

  • Quality issues, and the quality flags
    • One matrix (fully half of) of the red channel array was lost at the end of the mission, so from OD1375 onwards this matrix is masked out automatically in the SPG processing.
    • The quality flags in the quality context ("quality" or "qualitySummary") inside the observation context are meant for HSC/ICC internal evaluation of the quality of the products: when an observation had a serious quality problem, the PI of the program would have been contacted about it. For archive users, only the information in the "qualitySummary", when available, is useful.

PACS Photometer Release Notes

PACS photometer AOT release notes see PACS Instrument and Calibration webpage

PACS Spectroscopy

Special messages

Warning for drizzled cubes in HIPE/SPG 13

Due to an oversight in the pipeline scripts of HIPE 13, the drizzled cubes you get for chop-nod line scan observations created by "SPG 13" will have incorrect fluxes. Therefore you should not use the drizzled cubes downloaded from the HSA if the Meta datum "creator" is "SPG 13.0", whether they are within the ObservationContext at Level 2, or part of a Standalone Browse Product download. Any drizzled cubes created for unchopped mode observations will be unaffected. For chop-nod range scan observations, drizzled cubes are not created by the SPG 13 pipeline anyway. This oversight will be corrected in HIPE/SPG 14.

For the affected observations you have the following alternatives: 1) Use the projected cubes instead, which you will also find within the ObservationContext at Level 2, but which are not part of the Standalone Browse Product download for the affected observations. 2) Run one of the interactive pipeline scripts to create the drizzled cubes yourself. Contact the Herschel Helpdesk first to ask for an updated pipeline script to do this, since the interactive pipeline scripts in HIPE 13 (and 12) also contain the mistake.

For Your Information: Drizzled cubes FITS files, and their name as listed in an ObservationContext, are: HPS3DD[R|B] (red and blue) Standalone browse product drizzled cube FITS files, and their name as listed in an ObservationContext in the browse product section are: HPS3DEQ[R|B] (red and blue, equidistant wavelength grid version of the drizzled cubes). Projected cubes have the name: HPS3DP[R|B].

For more information about the standard and the standalone browse cubes provided for PACS spectroscopy, see the PACS Products Explained HIPE help document, which is also available from the PACS documentation webpage and the HIPE download webpage, both off the Herschel Science Centre webpage.

General notes

  • To produce the highest quality cubes possible, you should consider re-processing or fine-tuning the observations with the latest HIPE User Release. Cubes available from the HSA are created within a bulk processing framework, and a reprocessing while fine-tuning the important pipeline task parameters, according to the characteristics of the observation and source, could enhance the quality of the final results. The first two chapters of the PACS Data Reduction Guide for spectroscopy (HIPE 13) give information about the need to reprocess, and about what to do with HSA-obtained cubes before using them for science.

  • Spectral leakage
    • The order selection filters of the PACS spectrometer have a steep but not perfectly vertical transmission profile at the cut­off wavelengths of the spectral bands. PACS spectra near the band borders of bands R1, B3A and B2B are therefore affected by higher- or lower-order wavelengths leaking into the spectra -- both continuum and spectral lines! Consult the PACS Calibration Document for more information on the leakage regions.
    • For the leak in band R1, it is possible to reduce these data with a specific relative spectral response function calibration file in HIPE to obtain correct line fluxes (but an incorrect continuum level). How to do this is explained in the PACS Data Reduction Guide for spectroscopy (chps 5 and 6 in the HIPE 13 version). This RSRF is not applied by default since it increases the noise in the resulting spectrum, but can be used interactively within HIPE.

  • Second-pass ghosts
    • A second pass in the optics of the PACS spectrometer can cause a ghost image to appear on most spaxels (but never in the central spaxel). If a source located in one of these "originating" spaxels shows a strong spectral line (typically an atomic fine-structure line), then a weak, broadened line can be seen at an offset wavelength in its corresponding "destination" spaxel affected by the 2nd-pass ghost. The peak flux of this line is typically ~5% of the peak of the originating line. The most prominent ghost is the 122 micron feature, which originates from the usually strong CII+ 157.7 micron line. A list of strong ghosts and an image showing the directions of the projected passes on the 5x5 IFU footprint can be found in the PACS Calibration Document.

  • 62 micron dip
    • A dip can sometimes be seen at spectra between 62 and 63 microns. This is a filter feature. Its appearance depends on the angle at which the light from the sources goes through the filter, and so it depends on the source position and its spatial structure.

  • Spectral "fringing"
    • You may notice a fringing-like pattern in the spectra from (wide) Range scan and full SED-mode observations. This is because the flatfielding is not done by the SPG pipeline. To remove it you need to reprocess the data in HIPE with one of the interactive pipeline scripts. For Line Spectroscopy observations, the spectral flatfielding is done in the SPG pipeline.

  • Check for contaminating flux in chop-off positions
    • To check for the presence of contamination from unwanted astronomical sources in the off positions of chopNod mode observations, you can use a Split On-Off pipeline script to produce an off-source and on-source cube. These cubes can then be compared to each other to check for contamination in spectral lines or by strong continuum emission, e.g. by over-plotting the respective spectra. Note that the on-source and off-source cubes produced by this task will not allow you to detect faint levels of contamination because wriggles from the RSRF are not removed by this process. For faint targets (line peak-to-continuum emission ~<5-10 Jy) you should also check the differential signal between the nodA and B on-source cubes. This is documented in the PACS Data Reduction Guide for spectroscopy. In this guide you can also find advice on checking for contamination in the unchopped mode observations, which is done either by comparing companion observations (unchopped range) or with a small script provided in the PDRG (unchopped line).

  • Unstable/incorrect broad-band (dust) features
    • Broad spectral features (of a few micrometer width) and continuum shape variations can be introduced by transient effects (for chopNod mode and more so for unchopped mode observations) and by pointing offsets distorting the Relative Spectral Response Function. The "background normalisation" pipeline script for chopNod observations is recommended for observations looking for such features, as it minimises the effect -- this is the SPG pipeline for HIPE 13 and onwards. For unchopped mode observations you could reprocess the observation with the "transients correction" pipeline script that is new to HIPE 13. However, neither of these pipelines will completely negate the effect of transients and pointing jitter-induced RSRF distortions.

  • Spectral line profiles: the skew
    • As for any slit-spectrograph, if the incoming light beam is neither homogeneously nor symmetrically illuminating a spaxel, then line profiles may be distorted from the ideal Gaussian shape. In case of a point-source observed with PACS, the spectral line profile develops a skew, increasing as the offset of the photocentre along the instrument Z-axis (perpendicular the slit direction) does. Examples of skewed line shapes are given in the PACS Observers Manual.

  • Point sources
    • How to extract a correctly-calibrated spectrum of a point source is documented in the pipeline scripts and in HIPE. The necessary point source corrections must be applied for a correct spectrum to be produced.

  • Limitations on absolute spectrophotometric accuracy
    • The PACS spectrometer flux calibration accuracy is limited by detector response drifts and slight pointing offsets arising from the standard 2" (1-sigma) pointing error occurring within each and every observation. These limit both the absolute flux accuracy and relative accuracy within a band. Various pipelines deal better with these than others (see the advice in the PACS Data Reduction Guide for spectroscopy) but they can never be entirely negated. Hence the calibration uncertainty for any particular observation is a combination of the general calibration uncertainties (given in the PACS Observers Manual), the noise on the spectrum (explained in more detail in the PDRG chp 7.6/HIPE 13), and the "activity" during any single observation.

  • NaN's in the final cubes
    • It is normal to have NaNs at the very edges of the spectral ranges of SED mode observations: this is due to gaps in the spectral sampling.

  • Quality flags in the quality product
    • The quality flags in the quality context ("quality" or "qualitySummary") inside the observation context are meant for HSC/ICC internal evaluation of the quality of the products: when an observation had a serious quality problem, the PI of the program would have been contacted about it. For archive users, only the information in the "qualitySummary", when available, is useful.

PACS Spectrometer Release Notes

PACS spectrometer AOT release notes see PACS Instrument and Calibration webpage

SPIRE Photometry

Special messages

Warning for pointing anomaly in HIPE/SPG 13

PACS and SPIRE photometry observations reported in this CSV file are affected by a known problem related to the reset of the Spacecraft Velocity Vector (SVV) during the upload of the star-tracker's defective pixel table. For the affected observations, the pointing of the telescope can be off along the scan direction, and shifted up to 20 arcsec. This effect will be corrected in HIPE/SPG14.

General notes

  • In order to obtain the best possible Level 2 SPIRE photometry data, the observations might have to be reprocessed with the latest HIPE User Release.
    • Warning, important Please note that there was a bug in the destriper task included in HIPE 9.0 that may affect your final map, especially if there are bright objects in the observed field. This has been corrected since HIPE 9.1. If your observation falls in the mentioned category, you are strongly advised to update your HIPE installation.
    • Warning, important In HIPE 10.0 the flagging of thermistor jumps in the level 0.5 to 1 data reduction is not set properly. This induces the destriper to work improperly and to leave stripes in the final map. It has been solved starting with HIPE 10.1.

  • SPIRE-P level 2.5 and level 3 maps
    • For the definitions of the new product levels, introduced with HIPE v11, see The SPIRE Data reduction Guide, sections 3.2.3 and 3.2.4
    • These new levels will be available in the Herschel Science Archive when the observations will be bullk-reprocessed with HIPE v11. The useful user script Photometer_MapMerge.py can be used to make level 2.5 (parallel mode) or level 3 (mosaic) maps, see Section 5.8.3 in the SPIRE Data Reduction Guide.

  • Stripes in PSW, PMW and/or PLW (Level 2) maps
    • All SPIRE photometry pipelines now use by default the destriper, which improves the issue of striping in level 2 maps. Hence observers should expect potential improvements in that respect with version 9.

  • De-glitchter masks faint sources
    • The de-glitcher is a very delicate process. In particular, for data taken in Parallel Mode (sampling at 10Hz) and at high speed (60"/s) the de-glitcher with standard parameters may flag very faint sources as glitches. Bright sources are different from glitches in that they have a gaussian (i.e. beam/PSF) shape. For faint sources, the sampling rate could be not high enough and hence they have a "delta" shape, which is similar to a small glitch. The user might try to modify the correlation parameter to 0.95: this will decrease the number of detected glitches.

  • Some sources have saturated the ADC and the corresponding data have been masked
    • There is nothing a user can do: the source was simply too bright. If the user has other sources still not observed and of the same intensity, it is suggested to change the AORs to use the bright source mode.

  • Cooler temperature variations
    • The cooler temperature variations, as explained in greater details in the SPIRE Data Reduction Guide, section 6.4, can affect observations performed soon after the cooler recycle. The steep rise of the sub-K detector temperature is also known as the cooler burp and there is a quality flag coolerBurpDetected in HIPE v11 or later that indicates if the observation was performed during this period.
      InfoThe current list of observations known to have cooler temperature effects is here. Note that not all observations in this list raised the coolerBurpDetected flag.

  • NaNs pixels present in the PSW, PMW and/or PLW (Level 2) maps
    • This effect, related to data masked for various reasons and poor coverage (not enough redundancy), is more evident in single fast-scan Parallel Mode maps. To avoid NaNs, increase the pixel's dimension (i.e., decrease the map's resolution)

  • Quality flags
    • Currently, the quality flags at the quality context inside the observation context are just meant for HSC/ICC internal evaluation of the quality of the products and not for the users. In case the data had some serious quality problem, the PI of the program has been contacted about it. Otherwise, only information in the quality summary, when available, should concern the observers.

  • Planck derived zero offsets
    • The extended calibrated maps (extdPxW in level-2, 2.5 or 3) incorporate zero level offsets derived from Planck-HFI. For small size SPIRE maps, smaller than ~30 arcmin, the zero-offset can be rather uncertain, due to the large Planck beam (8 arcmin). In such cases the interpretation of the zero offset as the absolute zero level must to be treated with extreme caution.

SPIRE Photometer Release Notes

SPIRE scan map release note : 20 October 2009

SPIRE Small Scan Map AOT release note: 17 Mar 2010

SPIRE Spectroscopy

  • In order to obtain the best possible Level 2 SPIRE FTS data, the observations should be reprocessed with the latest HIPE User Release.

  • Bright source mode: observations in bright source mode processed with HIPE v7 or earlier result in spectra that have no scientific value. Bright source observations processed with HIPE v8 or above are fine.

  • Low resolution (LR) observations: the LR mode processing with HIPE v9 or above are improved with respect to previous versions, but there may still be artefacts which in most cases can be corrected using the background subtraction useful script.

  • High + Low resolution (H+LR) observations: The LR part of the H+LR observations is not properly calibrated and should not be used. The FTS team is working on improving the calibration and as soon as we have a validated proper calibration we will implement it and reprocess the H+LR data.

  • Faint point source observations: if the SSW and SLW bands do not match up for a source that is known to be a point source, observers are recommended to run the background subtraction script in HIPE - more details are in the SPIRE Data Reduction Guide. If this doesn't solve the problem, the observer is encouraged to contact the HSC helpdesk or the FTS User Support Group.

  • Artefacts in the continuum for few repetitions
    • Very small repetition numbers (e.g. 2 or 4) make 2nd level deglitching, which is based on a statistical outlier criterion, more challenging. The deglitching module may either not identify a glitch at all or it may not remove it completely. In cases where the glitch is located within the double-sided portion of the interferogram, the additional energy from the glitch will translate into artefacts of the continuum level. This kind of problem can be identified by inspecting all detectors from all scans in the level-1 spectral products. For some detectors, one or several scans may appear to be outliers. As a work-around, it is recommended to reprocess the data with a lower thresholdFactor when calling deglitchIfgm(). If the problem persists, the identified detector should be removed from the applicable scan in the SDI product.
      (NB: This affects HIPE 6 and higher)

  • Line fitting
    • If you know that the line you wish to measure is unresolved then you may want to fix the line width to the instrumental line width. For the Sinc model, implemented in the SpectrumFitterGUI you should put the Sinc width equal to resolution/π, which, for HR is 1.2/π = 0.382 GHz.

  • Point source and extended source spectra
    • If your level-2 spectra show characteristic jumps at ~1250 GHz and ~750 GHz, and the spectra from the two bands SSW and SLW do not match, then your target is extended or semi-extended in the SPIRE beam. You need to use the semi-extended correction tool (SECT) available since HIPE v10. Check the the SPIRE Data Reduction Guide (SDRG).

  • Quality flags in the quality
    • The quality flags at the quality context inside the observation context are just meant for HSC/ICC internal evaluation of the quality of the products and not for the users. In case the data had some serious quality problem, the PI of the program has been contacted about it. Otherwise, only information in the Quality Summary, when available, should concern the observers.
    • For observations before OD1000 there could be an erroneously raised or a missing flag RADECACC. This flag is calculated as the difference between the observer requested target coordinates (kept in a metadata raNominal, decNominal) and the average pointing of the telescope during the observation (excluding the slew). This is an indicator of the pointing accuracy of the observation. If greater than 2.2" then RADECACC is raised in the QC context. The calculation of the average (ra, dec) during the observation does not take into account the known offset of 1.7" of the BSM home position for OD < 1000 and consequently observations with better pointing than 2.2" may be flagged and vice versa, observations at more than 2.2" may not be flagged. If you want to check the pointing offset with respect to the requested target coordinates then you should use the pointing information for the central detector SSWD4 in the level-2 context.
      A new quality parameter raDecOffset is introduced in HIPE v13, in level-2 products, which contains the correct offset between the raNominal, decNominal and ra,dec for the FTS central detector.

  • Observations during the steep rise of the sub-K temperature
    • During the first few hours after the cooler was recycled the bolometers' temperature (the sub-K temperature) undergoes a steep rise before it reaches a stable plateau. Observations during this period suffer overcorrection of the instrument/telescope emission. This is more significant for faint targets and can be identified as an unphysical slope of the SLW spectrum, with an important negative gap in the region that overlaps with SSW (see the Figure). One way to check if your observation is within this problematic category is to get the median sub-K temperature for the first building block: print MEDIAN(obs.level0_5.get(0xA1060001).nhkt['signal']['SUBKTEMP'].data), where obs is the pre-loaded observational context. If the result is less than 0.2869 K then the observation is affected by this.
      This effect is fixed in HIPE v13.


SPIRE Spectroscopy Release Notes

SPIRE Spectrometer Point Source AOT release note: 21 May 2010

SPIRE Spectrometer Bright Source Mode AOT release note: 7 Sep 2010

SPIRE Spectrometer Mapping AOT release note: 30 Apr 2010

HIFI observations (point mode, spectral survey and mapping observations)

In order to obtain the best possible Level 2 HIFI data the observations should be reprocessed with the latest HIPE User Release.

  • Corrupted data-frames
    • There is a limited number of obsids where one or more data-frames from the spectrometers is corrupted in some fashion and leads to level 2 products that cannot be properly calibrated. A scheme was put in place some versions ago to flag or simply remove those frames at level 0 so the final products won't be affected by them. This scheme however suffered from a regression bug in 11.1 so that the corresponding products did not benefit from this correction properly. We note however that this should only concern about 50 obsids in the whole HIFI archive. This will be fixed in HIPE 12.

  • Strong/Weak Spurs
    • Spurs are still affecting the HIFI data at spot frequency points. They will be flagged automatically in the data using a SpurFinder task when present above a certain threshold in the internal load spectra. There are still some spur categories that are not yet properly detected but this is improving in each new versions of HIPE. Some spurs are so intense and broad that they can lead to the loss of complete WBS sub-bands. Note that the strong spur affecting the upper end of band 1a has now been completely cleaned - see the corresponding technical note: HIFI Information note on the removal of spurs in band 1a and the sampling in the mapping modes. The same is true for a weaker spur that did affect data in band 7b close to LO = 1834 GHz.

  • Unpumped data
    • Due to gaps in the LO output power over the HIFI frequency range, some spectral scans will contain spectra that cannot be used due their very high noise temperature levels. These data should be discarded from further processing (e.g. deconvolution). For single frequency observations, there exist some places where the theoretical noise temperature (i.e. the one advertised by HSpot) should be nominal, but the sensitivity effectively achieved at the time of observations is noticeably worse. These are cases where the mixer tuning algorithm does not fully converge. Such cases have e.g. been reported around LO frequencies of ~ 1441.5 GHz. An overview of the expected unpumped frequency areas is given in the AOT release notes. Users observing such effects should contact the Helpdesk.

  • Standing waves
    • There are a variety of standing waves known to affect the data at Level2 - see the AOT release notes for more details. At the present time, the FitHifiFringe tool usually does a decent job for most standing waves in SIS bands. In HEB bands, where the standing wave is not of optical nature and does have a strictly sinusoidal shape, FitHifiFringe can help when restricted to the frequency range where the line is present. Another alternative in the form of a dedicated algorithm has been developed (so-called "matching technique") and is offered in the hebCorrection task from HIPE 12.0 onwards (details can be found in Chapter 10 of the HIFI Data Reduction Guide). You can contact Ian Avruch (i.avruch@sron.nl) at the HIFI ICC for further details. In any case, it is advised not to use more than 3 components in FitHifiFringe. Since HIPE 5, a number of new features have been made available in FitHifiFringe. One of them ("sub_base" option) allows you to fit a baseline at the same time. Since FitHifiFringe can identify automatically the lines in order to set masks, users should be careful when treating lines with large wings. In this case, the user should set a window by hand. The automatic line masking should then be switched off.
    • In case of strong source continuum some standing waves can be enhanced. Such standing waves can be significantly reduced in amplitude using an alternative pipeline algorithm known as the Modified Passband Technique , which is described in the Standing Wave Removal chapter of the HIFI Data Reduction Guide. Note that in 9.0, a bug affected the doFilterLoad task in such a way that it ignored the user-input parameters, resulting in the task always applying the "cubic_spline" method with default parameters - i.e. the "fft" method will not be applied despite being set in the task inputs. This was fixed in 9.1 before the SPG 9.1.0 bulk reprocessing of the archive.

  • Baseline distortion
    • Residual drift from the detector response can translate into imperfect baseline structures. This usually manifests as wavy structures, slopes, or overall level offsets of the spectral baselines. Such artefacts are expected to be enhanced e.g. at the borders of spectra obtained in diplexer bands (sub-bands 1 and 4 for bands 3 and 4, sub-bands 2 and 4 for bands 6 and 7). Similar artefacts are also expected when using modes without reference, such as the FreqSwitchNoRef, or the LoadChopNoRef, and are enhanced with strong continuum sources. Please refer to the Data Reduction Guide for details about the most recent improvements to the FitBaseline task.

  • Saturation
    • Saturation of the WBS CCDs may be observed at spot frequencies. These are usually due to either strong and broad spurs, or to cases where only marginal pump level could be achieved due to shortage of LO output power. Users can go back to HSpot to visualize in the frequency editor where their AORs were potentially affected by such effects. HIPE will normally flag the affected spectrometer channels.

  • Beam efficiencies
    • The easiest way to convert the level2 data into the Tmb scale is to use the task "doMainBeamTemp".

  • Side-band ratio and continuum
    • The data processed at level2 are corrected for the fact that HIFI observes in double-side band. In practice, a side-band ratio is assumed over most of the HIFI operational range, which simply means that single-side band intensities are twice those in double-side band. In the limited ranges where this side-band ratio deviates from unity, the intensity scaling will differ for respective USB and LSB spectra. It will also imply that the continuum cannot be trusted in these frequency range. A dedicated task to recover the continuum corresponding to a balanced side-band ratio was made available in HIPE 9.0.

  • H/V polarisation pointing information
    • The observed differences in HIFI horizontal (H) and vertical (V) polarisation line profiles has made it desirable to have astrometry available for both H and V spectrometers. The possibility that small scale structural or velocity variations may be responsible can then be examined. In the past, the pointing information associated with both H- and V- spectra has been that of a synthesized aperture, which is a position midway between the H and V spectrometer beams. From HIPE 4.2 onwards, an option has been introduced in the doPointing task, allowing to compute separate attitude for the respective mixer polarizations. From HIPE 5 onwards, this option is the one used by default. In practice, note that this slight pointing mis-alignment can lead to differences between the H and V signals observed both in spectral lines and continuum, especially when the source structure features abrupt brightness temperature variation on small spatial scales.

  • CCD "scratches"
    • The WBS CCDs are known to be affected by so-called "scratches" present on their optics. These small damages can translate into small spikes showing up at particular channels of the WBS data, however they should be removed by the calibration scheme. This removal was improved in particular from HIPE 6.1 onwards. Users who think that they might be still observing such effects should contact the Helpdesk.

  • Platforming effect
    • There exists the possibility that residual platforming effect, i.e. imbalance between the average baseline level of individual WBS sub-bands, is still observed at level2. This is usually due to temperature drifts of the CCD during the observations. There is no dedicated tool in the HCSS to correct this effect and users should avoid stitching their data when such artifact is present in the data. Correction of the relative baseline level of each sub-band is left to the user's judgment.

  • RA/Dec information in headers
    • The astrometry information provided in the meta-data of the various products has not always been accurate for HIFI data. This has now been fixed in the products generated by the bulk reprocessing with 8.2.1. Note that there is still a problem wit the moving targets, for which the reported ra/decNominal (i.e. the intended position at the time of observation) is not yet correct. If you observe an anomalous position reported in your (fixed target) data, please let us know via the helpdesk.

  • Quality flags in the quality product
    • Currently, the quality flags at the quality context inside the observation context are just meant for HSC/ICC internal evaluation of the quality of the products and not for the users. In case the data had some serious quality problem, the PI of the program has been contacted about it. Otherwise, only information in the quality summary, when available, should concern the observers.

  • Map visualisation and cubes
    • The visualisation software offered in HIPE 9 onwards has had significant improvements since the previous version, in particular it has been merged with the spectrum explorer. Note that there can be problems in the cubes generated in the case of single line (stripes) maps with HIPE 8.2. In those cases, the cube dimensions may not be correct. Also, cubes for raster maps may not be optimal when generated with HIPE versions earlier than 8.2. This issue is fixed from HIPE 9 onwards

  • Browse products
    • HIPE 9 introduced browse products for mapping and spectral scan observations. For maps of moving targets, note that the browse product images are displayed in the comoving frame, so a centre of RA/Dec = (0",0") is used. These include deconvolved spectra (but see following note) for spectral scans and spectra/line maps for a standardly gridded map for mapping observations.

  • Deconvolution of Frequency Switching data
    • The deconvolution software will not work properly with frequency switching data. This is intrinsically linked to the fact that the deconvolution algorithm has troubles with constant steps in the frequency dimension, and that frequency switching observations uses uniform frequency throws. This creates some periodic ghost artifacts close to strong lines.

  • Reprocessing from raw data
    • Some expert users may want to reprocess their data starting from raw telemetry (so-called level-1). From HIPE 9 onwards additional telemetry (notably from the ACMS system) was added to the products. In order to avoid problems with missing these products from earlier downloads, the following can be temporarily added to your local configuration property: hcss.ia.spg.ops.AuxPlugin.products={..., herschel.ia.obs.auxiliary.acms.AcmsTelemetryProduct, ...}

  • Flux Conservation in Spectral Cubes from Mapping Observations
    • The doGridding task in the HIFI pipeline is responsible for convolving the spectral datasets acquired in an OTF or DBS Raster mapping observation into a spectral cube with a specified pixel scale, which by default should match how the scan lines and readout points within each line were spaced during the observation. The scheme of signal filtering and interpolation to put the data on the specified grid may affect the overall flux conservation, at a level which is low but you should be aware of. For example, the total signal summed from a spectral cube produced using a Gaussian filter over the datasets of a Nyquist-sampled OTF map is generally < 1% lower than the sum of the signal taken directly from the input datasets (the Level 2 HTP datasets) before convolution. A part or all of this slight mismatch may be on the assumed versus actual beam shape at the observed frequency. If you wish to put the map on a coarser grid, effectively reducing the spatial resolution to a wider beam in order to match another observation, then the flux losses become more noticeable. Doubling the pixels sizes from their default (native map point spacing) can reduce the total flux by as much as 10%, accompanied by an increase in baseline RMS noise. The effect is more prevalent in OTF maps than DBS Raster, and in addition to deviations from ideal beam shape characteristics becoming more important, the filtering and interpolation method, and parameter values can be influential. No changes to the doGridding algorithm are planned, and this issue applies to all HIPE versions.

  • Retrieval of calibration files from the HSA
    • In HIPE versions from the track 9, there is small bug in the getHifiCal task. The task may erroneously report that the latest calibration tree available is HIFI_CAL_9_0, while more recent versions are now available. This issue is solved in HIPE 10. This is not a problem with the HIPE 11 build. One possible work-around is to clear the calibration pool in your .hcss/lstore, and call the task again or simply use the updated HIPE version. Then the latest calibration tree should be offered for download.

HIFI Release Notes

HIFI Dual Beam Switching Observing Modes AOT release note: 9 Mar 2010

HIFI Oberving Modes AOT release note: 11 Apr 2010

HIFI AOT Observing Mode Release and Performance Notes v3.0: 24 September 2011

Other Technical Notes

PACS documents: see PACS Instrument and Calibration webpage

HIFI Information note on Mapping Modes: 30 Jun 2010

HIFI Beam Efficiencies from Mars Observations: 17 Nov 2010

HIFI Information note on the removal of spurs in band 1a and the sampling in the mapping modes: 19 Nov 2010

Calibration Documents

PACS photometer and spectrometer calibration documents: see PACS Instrument and Calibration webpage

SPIRE Photometer Beams (FTP repository)

HIFI System noise temperature IF spectra (FTP repository)

Topic attachments
I Attachment History Action Size Date Who Comment
PNGpng Steep-subK-case.png r1 manage 89.3 K 2014-07-01 - 12:44 IvanV  
JPEGjpeg ngc253_1342221743_blue.jpeg r1 manage 935.1 K 2015-07-21 - 14:37 LucaCalzoletti  
Edit | Attach | Watch | Print version | History: r129 < r128 < r127 < r126 < r125 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r129 - 2015-09-16 - AlvarGarcia
This site is powered by the TWiki collaboration platform Powered by Perl