Herschel Data Processing
Introduction to Herschel Data Processing
All data obtained with Herschel will be systematically pipeline-processed with the Herschel Data Processing system, at the Herschel Science Centre (HSC), to generate a range of scientific and auxiliary data products.
All raw and processed data are ingested in the Herschel Science Archive immediately after the pipeline processing is completed, typically 1-2 days after execution. Notification e-mails are sent to the proposal PIs when basic quality control checks have been completed, something which may take from a few additional days to weeks, depending on the circumstances.
The Herschel Data Processing (DP) system is a joint development effort by the HSC, the Herschel Principal Investigator Instrument consortia, and the NASA Herschel Science Center. The software is coded in Java and Jython, and runs on Linux, Windows, and Macintosh platforms.
The software and the associated documentation are available to the community for download from the HSC (just like HSpot) and incorporate the state of the art in calibration and pipelines.
For any queries about the system, please contact the HSC Helpdesk.
Overview of the Herschel Data Processing system
The Data Processing system consists of a pipeline processing software to be run 'hands-off' and an Interactive Analysis environment where the system can be used 'hands-on' by individual astronomers.
Pipeline processing software
The Herschel Data Processing system is used by the HSC to systematically process Herschel data in order to generate products associated with each observation.
The systematic pipeline processing consists of:
- Standard Product Generation (SPG) software, run systematically for all data at HSC to generate a set of standard products associated with each observation. It consists of routines available in the Interactive Processing environment
- Quality Control (QC) processing. It generates quality control information on all observational data. An automated part will run systematically for all data, but the data will also be visually inspected by HSC personnel at least once per observation.
Herschel Interactive Processing Environment (HIPE)
In addition to the standard product pipeline processing the Herschel Data
Processing system also provides the tools to interactively reduce the
data using the same routines.
The Herschel Interactive Processing Environment (HIPE) enables the user to:
- access data from the Herschel Science Archive
- perform interactive data reduction from raw data to publishable products, using Herschel-provided and user-developed routines, both in GUI form or in console-batch mode
- visualise and manipulate data
- perform science analysis
- get access to context-sensitive documentation and help
How to access HIPE
Linux, Windows, or Mac installers for the latest user version of HIPE can be retrieved here.
In addition, you can access the latest developer builds. These versions are not user releases, they are untested, or under test, and are provided to you at your own risk. In particular, Helpdesk support is not provided for them.
Currently, a cycle of 4 releases per year is planned to accommodate the fast evolution of the pipelines in the early phase of the mission.
How to learn more, customize and contribute to HIPE
HIPE is open to external contributions; the Key Programme consortia and the astronomical community in general are encouraged to feed back their data products and share the tools and algorithms developed to produce them with the HSC for possible inclusion in the Data Processing system. HIPE pipelines are organised in modules (called tasks), easily interchangeable by user-customized tasks.
Analysis of Herschel Data
For information on reducing and analysing your Herschel data, see the Instrument Handbooks listed below.
For quick start guides to HIPE you should also check the "Read me First" and "Quick Start Guide" documents. We encourage astronomers wishing to analyse or propose observations with Herschel to also check the useful AOT Release Notes in the AOTs Release Status page.
Instructional videos about data import are available here
DP Interest Lists
The following mailing lists have been created and hosted at the HSC to allow
user-to-user sharing of information about specific needs and problems, as well as questions and answers on the reduction and analysis of the different Herschel AOT's. You can also find two lists for general HIPE and advanced HIPE questions and answers. The links below will direct you to the pages where you can subscribe to these lists and consult or search the discussion archives.
|
PACS Photometry (point source and small maps)
|
Subscribe
|
|
SPIRE Photometry (point source and small maps)
|
Subscribe
|
|
Large maps and point source extraction for PACS and SPIRE
|
Subscribe
|
|
PACS Spectroscopy
|
Subscribe
|
|
SPIRE Spectroscopy
|
Subscribe
|
|
Spectral maps for PACS, SPIRE and HIFI
|
Subscribe
|
|
HIFI Point sources and spectral scan
|
Subscribe
|
|
