2.2. Navigating an ObservationContext

2.2.1. In HIPE

The ObservationContext is what you get from the HSA if you download an entire observation: see the Launch Pads, for spectroscopy or photometry, for more information on downloading PACS data. An ObservationContext is a grand container of all the individual products and datasets for an observation. To get an overview of its contents, open the ObservationContext with the Observation viewer (see the HOG chap. 15), accessed via a right-click on the observation as listed in the Variables or Outline panel; the viewer will open in the Editor panel.

A spectroscopy ObservationContext, produced by SPG 12.1 and viewed with the Observation Viewer.

Figure 2.1. A spectroscopy ObservationContext, produced by SPG 12.1 and viewed with the Observation Viewer.


In the upper part you see a Summary tab. The small image in this tab is a browse-image/postcard, see Chapter 5 for an explanation of how these postcards are created. The Meta Data tab contains information about the observation; the Meta data shown belong to whatever is highlighted in the Data tab below, i.e. either to the entire ObservationContext or to any of the other products contained therein; some of these meta data are listed in Chapter 4. The Data tab shows all the products, contexts, and data arrays that are held in the ObservationContext: think of it as a directory viewer. To its right is an area where the default viewer for the product clicked on in the Data tab will open, if the object is "viewable" (e.g. the Image viewer or the Spectrum Explorer). From the Data tab you can extract products from the ObservationContext by dragging them to the Variables panel, or you can right-click to "Create Variable".

The Data tab shows the "directory" of the products in an observation, as the figure below shows:

The Levels directory listing in an ObservationContext for a spectroscopy observation produced by SPG 13. Some spectroscopy and photometry observations will also have a Level 2.5 and a Level 3.

Figure 2.2. The Levels directory listing in an ObservationContext for a spectroscopy observation produced by SPG 13. Some spectroscopy and photometry observations will also have a Level 2.5 and a Level 3.


Listings in red have not yet been loaded into memory, those in black have been; to load a product into memory you only have to access it (e.g. click on it).

  • The science data are located in the contexts called +level0/0_5/1/2_5/3. These levels hold data at different stages of processing: Level 0 are raw, Level 0_5 are partially processed (instrumental corrections and bad-data masking), Level 1 are also partially processed (de-glitched, flatfielded, partially (spectroscopy) or fully (photometry) calibrated), Level 2 are of science quality (spectral and spatial units, fully calibrated), Level 2.5 are combinations of Level 2 cubes/maps, and Level 3 is a super-combination of observations into a single map (photometry) or a combined table of extracted spectra (spectroscopy).

  • The pacsObsSummary is the output of the pipeline task "obsSummary" and it contains a summary of the observation: AOT name, proposal details, wavelength ranges, and, where they exist, quality information. To see this in a nice viewer, click directly on the "pacsObsSummary". Note that this summary can only be found when using HIPE, it does not exist as a FITS file in the HSA-download.

  • The browseProduct contains the standalone browse products, which are maps, cubes, and tables (see Chapter 5), and the browseImageProduct contains the postcard (the image at the top-right of the Summary tab).

  • For observations taken directly from the HSA there will be a calibration entry in the Data tab, containing the calibration tree that was used to reduce these data by the SPG pipeline (see e.g. chp. 3 of the PDRG for more information). For observations which were previously saved to disc without specifically saving also the calibration tree, this will be absent.

    Note that the calibration tree for PACS can be gotten online, via HIPE.

  • The auxiliary layer contains satellite and instrument data used in the pipeline, e.g. for calibrating the pointing and timing.

  • The History contains a history of the reductions; there is a history for every layer you can look at. The History contains everything that was done to "its" product, and is appended to as processing is done to the data. For more detail on the History, see Section 2.5

  • The quality/qualitySummary contains quality information. For more detail on the Quality, see Section 2.5.

2.2.2. On disc

Observations can be gotten from the HSA as a tarball, and they can also be saved to disc from within HIPE (see e.g. PDRG to learn how). Data gotten from the HSA as a tarball will unpack into a directory with the request ID as the name, inside which is a directory with the obsid name and directories of auxiliary data and the calibration files. Within the obsid directory are the observation data, are organised much as they are presented in HIPE. The auxiliary Compare the screenshot below (taken from a spectroscopy observation but the layout is the same for photometry observations) to that above and those which follow.

Screengrabs of parts of a spectroscopy ObservationContext stored on disc, having been downloaded from the HSA as a tarball

Figure 2.3. Screengrabs of parts of a spectroscopy ObservationContext stored on disc, having been downloaded from the HSA as a tarball


If you save an observation from within HIPE using saveObservation or getObservation (the tasks that the PACS pipelines use), the organisation of the data on disc is very different: the data are organised into directories of product classes, rather than level. When you save an observation to "pool" with get/saveObservation, the screenshot below is what you can expect to see in the pool directory. If you used the default options to save, then this pool will be in HOME/.hcss/lstore/[obsid]. If you asked to save to a specific location, then that is where the pool will be found. The screenshot is from /Users/me/Downloads/1342235641, where "/Users/me/Downloads/" is the "store" and "1342235641" is the "pool".

Screen-grabs of parts of a spectroscopy ObservationContext stored as a pool on disc

Figure 2.4. Screen-grabs of parts of a spectroscopy ObservationContext stored as a pool on disc


We assume that if you saved an observation to disc from within HIPE using saveObservation or getObservation, then you will be accessing these data from HIPE using the methods used in the pipeline scripts and explained in the data reduction guides. If you downloaded and unpacked the tarball directly from the HSA, we assume that you will want to access the data directly from that disc rather than from HIPE: therefore the FITS filename for the products discussed in this chapter are given in Table 2.7, so you can locate the FITS files that correspond to any particular product mentioned here.