Chapter 7. Pipeline processing and data products

Table of Contents

7.1. PACS photometer standard data processing
7.1.1. Scan map pipeline processing issues
7.1.2. Level 2 pipeline products for scan maps generated with HCSS in the HSA archive
7.2. Spectroscopy processing levels and data products

This chapter describes :

7.1. PACS photometer standard data processing

7.1.1. Scan map pipeline processing issues

The current automatic pipeline processing provides reductions up to PACS photometer level 1 (detector readouts calibrated and converted to physical units) and level 2 products (fits maps). At this point in the Herschel mission, the goal of the pipeline is to deliver useable, but not necessarily optimal, products. Instead, the automatic pipelines are optimized for stability, speed and delivering browse quality data. The pipelines are expected to mature as both the instrument effects and the understanding on how to mitigate said effects evolves. The major data processing modules of the pipeline are discussed below:

  • Cosmic rays removal: The default deglitching algorithm for PACS photometer is the multi-resolution median transform (MMT), which uses wavelet scales (Starck et al, 1998, PASP, 110, 193S) to differentiate between a cosmic ray hit and signal. The method produces reasonable results except for bright point-like sources (e.g. compact galactic nuclei, etc.) and fast (60"/sec) scan speeds. Under such conditions, MMT incorrectly masks point source cores. Users are advised to check the coverage map to look for any gaps in exposure depth at the location of point sources. The optional module, "2nd order deglitching" can be used in the interactive data processing as a replacement for MMT deglitching. "2nd order deglitching" uses spatial redundancy and sigma-clipping to reject cosmic ray hits.
  • Cross-talk correction: The red bolometer array shows cross-talk between column 0 and 16. That is, when a source is present in column 16, its flux is also observed in column 0. Investigations on proper removal of cross-talks are underway. Users are currently advised to mask all signal in columns 0 and 16 to avoid any artifacts from cross-talk in the final maps when observing very bright (point-)sources
  • Offset drift correction: Offset drift correction is performed by applying a simple high-pass filter on the data. This is adequate for point source fields, but not for structured fileds and may result in dark halo artefacts. Increasing the filter window so that it covers a scan leg length already improves the map in the case of structured objects.
  • Map reconstruction: Map reconstruction with photProject is a simple projection of the data cube on the map grid, considering the intersection areas of the native pixels with the map pixels. For this purpose the ra/dec coordinates of the four corners of each detector pixel in the cube are computed on the fly in photProject. An alternative, optimal map maker, MADmap (Cantalupo et al. 2009), is provided in the data processing environment. This is a java implementation of the original MADmap C code. MADmap uses optimal map-making techniques to produce the best-fit solution for the final reconstructed maps. The primary advantage of MADmap is that it does not require the use of high-pass filter to mitigate 1/f noise and thus preserves spatial structures up to the size of final maps. Currently, MADmap is only available as an interactive tool and requires pre-processing to remove correlated signal drifts prior to optimal map reconstruction. MADmap additionally requires apriori knowledge of the detector noise properties (as the inverse of the time-time noise correlation matrix). Investigations on the noise filters are underway from the in-flight data. Current version of the data processing environment relies of pre-launch estimates from ground-based tests. Users wishing to use MADmap for their data processing needs should contact the helpdesks at ESA or NHSC for further information.

7.1.2. Level 2 pipeline products for scan maps generated with HCSS in the HSA archive

There are 3 types of products in the level 2 produced by the pipeline for the PACS photometer in scan map mode. MADmap as been disabled from the HSC pipeline with HCSS 2.0, hence the second product (HPPMMAP) is not available in recently processed scan maps observations. These maps are produced by automatic pipeline scripts and shall only be considered as a preview, and not for science directly.

  • HPPPMAPB & HPPPMAPR stands for "Herschel Pacs Photometer PhotProject MAP Blue/Red" This refers to maps produced by the photProject task, i.e. a simple projection of each frame (10Hz), after running a temporal high-pass filter with a width of n=20 (i.e subtracting a median with a width of 2*n+1 frames). This allows to filter a signicant part of the 1/f noise at the expense of removing completely ALL spatial scales larger than this width (i.e. typically larger than 1 arcmin), and creating negative undershooting around bright sources along the scan direction. To preserve extended emission, the pipeline script shall be re-run with higher width in the high pass filtering and masking bright source when necessary or an alternative map-making algorithm/tool. This processing is mostly targeted to detect point-sources with good sensitivity. Scan maps are in Jy/pixel.
  • HPPMMAPB & HPPMMAPR stands for "Herschel Pacs Photometer MADMap Blue/Red" MADmap maps are currently disabled from the HSC piepline, until scan and crossed-scan maps can be processed together.
  • HPPNMAPB & HPPNMAPR stands for for "Herschel Pacs Photometer Naive Map Blue/Red" Averaged signal map after pixel-to-pixel offset correction. This image is used by MADmap as its first value for the sky map and is subsequently improved and optimized iteratively as described above, hence the full optimized matrix inversion has not been performed on the data. Again these products are not avaialble in recent HSC pipeline maps.