Chapter 9. Pipeline description and data product expectations

Table of Contents

9.1. PACS photometry standard data processing
9.1.1. Processing levels
9.1.2. Processing steps
9.2. PACS spectrometry standard data processing
9.2.1. Processing levels
9.2.2. Processing steps

In this chapter, the different intermediate formats of the PACS data throughout the reduction process are described, and the standard processing steps for the different photometry and spectroscopy observations of the PACS instrument are mentioned. For a more detailed description of the pipeline, refer to the PACS Data Processing User's Manual.

9.1. PACS photometry standard data processing

9.1.1. Processing levels

There is a Herschel-wide convention on processing levels of the different instruments.

  • Raw Telemetry : All telemetry packets produced by the instrument in the course of the observation. In PACS IA, we store/manipulate this level as a PacketSequence .

  • Level 0 data: Telemetry data as measured by the instrument, minimally manipulated and stored as Data Frames. For PACS photometry, this level is stored/manipulated in a DataFrameSequence : a sequence of PACS dataframes, which are decompressed SPU buffers. What is contained in every decompressed SPU buffer depends on the SPU reduction mode. Typically there are several reduced readouts for every active detector (averaged detector signals), 40Hz or 20Hz readouts for a few selected pixels and mechanism/status information sampled at 40Hz/20Hz by the DecMec, the so-called DMC Header.

  • Level 0.5 data : The information contained in the Level 0 data is not sufficient to be able to process those data. The level 0.5 data is a bundle of Level 0 data and the data needed to fully process those data: auxiliary data for the timespan covered by the Level 0 data, such as the spacecraft pointing (attitude history), the time correlation, selected spacecraft housekeeping, etc... It is also possible to include in this bundle the level 0 data of 'associated' observations - e.g. flatfields or photometric checks taken throughout the operational day.

    The Frames class for reduced data and the PhotRaw class for additional raw channel data will be the basic data data products for this processing steps

  • Level 1 data: Detector readouts calibrated and converted to physical units, in principle instrument and observatory independent. For PACS photometry this is a data cube with flux densities with associated sky coordinates.

    The Frames class will be the basic level 1 product of photometer data

    Possibly the level 1 data generation can be done automatically to a large extend after the instrument has been calibrated.

  • Level 2 data: Further processed level-1 data to such a level that scientific analysis can be performed. For optimal results many of the processing steps involved to generate level-2 data may require human interaction, based both on instrument understanding as well as understanding of the scientific aims of the observation.

  • Level 3 data: These are the publishable science products where level-2 data products are used as input. These products are not only from the specific instrument, but are usually combined with theoretical models, other observations, laboratory data, catalogues, etc. Their formats should be VO compatible and these data products should be suitable for VO access.

9.1.2. Processing steps

pipeline slide 1

Figure 9.1. pipeline slide 1

pipeline slide 2

Figure 9.2. pipeline slide 2

pipeline slide 3

Figure 9.3. pipeline slide 3

pipeline slide 4

Figure 9.4. pipeline slide 4

pipeline slide 5

Figure 9.5. pipeline slide 5

pipeline slide 6

Figure 9.6. pipeline slide 6