(r62) SpireCalibrationWeb < Public

Public Web>SpireCalibrationWeb (revision 62) (raw view)~~EditAttach~~
---+ SPIRE instrument and calibration web pages 

%TOC{depth="3"}%

[[http://herschel.esac.esa.int/twiki/bin/view/Public/WebHome][Back to main Herschel Public TWiki page]]

---++ *Introduction* 
This page provides up-to-date information about using the SPIRE instrument: from preparing observations to reducing your data. This page also provides you with the latest calibration accuracies and known SPIRE calibration issues. 

---++ *Observing with SPIRE*

The most up to date information on instrument calibration and performance is given in the SPIRE Observers' Manual. This is the reference document used by all the rest of the SPIRE user guides (eg data reduction guide, cookbooks etc). Sometimes it may happen that outdated values are quoted in some of the documents. In such a case use the values given in the SPIRE Observers' Manual.

   * The *[[http://herschel.esac.esa.int/Docs/SPIRE/html/spire_om.html][SPIRE Observer's Manual (HTML)]]* and *[[http://herschel.esac.esa.int/Docs/SPIRE/pdf/spire_om.pdf][PDF (15.5 MB)]]*. 
   * A quick guide to the SPIRE instrument: *[[http://herschel.esac.esa.int/Docs/Flyers/SPIRE_flyer_20May2010.pdf][SPIRE fact sheet]]*
   *  *Summary papers from the A&A Special Issue:* 
      * %ICON{"warning"}% Some values on the performance are now outdated. Please consult the SPIRE Observers' Manual for most up to date information.
      *  [[http://herschel.esac.esa.int/twiki/pub/Public/SpireCalibrationWeb/aa14519-10.pdf][The Herschel-SPIRE instrument and its in-flight performance]], Griffin et al., 2010, A&A, 518, L3, in-flight scientific capabilities of the SPIRE instrument.
      *  [[http://herschel.esac.esa.int/twiki/pub/Public/SpireCalibrationWeb/aa14605-10.pdf][In-flight calibration of the Herschel-SPIRE instrument]]: Swinyard et al., 2010, A&A, 518, L4
   * *Detailed documents describing the pipeline algorithms (to be updated soon):*
      *  The *[[http://herschel.esac.esa.int/twiki/pub/Public/SpireCalibrationWeb/Pipe_Description_2.1.pdf][SPIRE pipeline description document]]* provides an overview of the pipeline structure, listing every module and the details of its purpose, inputs, outputs and algorithms
      *  The *[[http://herschel.esac.esa.int/twiki/pub/Public/SpireCalibrationWeb/Phot_Pipeline_Issue7.pdf][SPIRE Analogue Signal Chain and Photometer Detector Data Processing Pipeline]]*  sets out the general bolometer processing and Photometer pipeline algorithms in much more detail.
      *  The *[[http://herschel.esac.esa.int/twiki/pub/Public/SpireCalibrationWeb/SPIRE-BSS-DOC-002966_SPIRE_Spectrometer_pipeline_description_Issue_4_0_1.pdf][SPIRE Spectrometer Pipeline Description]]*  sets out the Spectrometer pipeline algorithms in detail. 

----+++ AOT release notes

|  *Photometer* | |
| Photometer point source mode | [[http://herschel.esac.esa.int/Docs/AOTsReleaseStatus/SPIRE_PointSource_AOT_ReleaseNote_30Apr2010.pdf][SPIRE photometer point source release note (30 April 2010)]]  |
| Photometer scan map mode | [[http://herschel.esac.esa.int/Docs/AOTsReleaseStatus/SPIRE_ScanMap_ReleaseNote_20Oct2009.pdf][SPIRE scan map release note (20 October 2009)]], [[http://herschel.esac.esa.int/Docs/AOTsReleaseStatus/SPIRE_ScanMap_ReleaseNote_Addendum_02Jun2010.pdf][ADDENDUM to scan map mode (2 June 2010)]]  |
| Photometer small scan map mode | [[http://herschel.esac.esa.int/Docs/AOTsReleaseStatus/SPIRE_Small_ScanMap_ReleaseNote_17Mar2010.pdf][SPIRE small scan map mode (17 March 2010)]] |

| *Spectrometer* | |
| FTS point source mode (sparse) | [[http://herschel.esac.esa.int/Docs/AOTsReleaseStatus/SPIRE_Spectrometer_PointSource_ReleaseNote_21May2010.pdf][Spectrometer point source mode release note (21 May 2010)]]  |
| FTS mapping mode (intermediate, full) | [[http://herschel.esac.esa.int/Docs/AOTsReleaseStatus/SPIRE_Spectrometer_Mapping_ReleaseNote_30Apr2010.pdf][Spectrometer mapping mode release note (30 April 2010))]] |
| FTS bright source modes | [[http://herschel.esac.esa.int/Docs/AOTsReleaseStatus/SPIRE_Spectrometer_BrightMode_ReleaseNote_07Sep2010.pdf][Spectrometer bright source release note (7 September 2010)]]  |


---++ *Reducing SPIRE data*

----+++ Software and documentation

   *  *HIPE (Herschel Interactive Processing Environment)*: The latest User Release HCSS version that you should use for reducing SPIRE data is HIPE v9.0.0. It can be downloaded from: http://herschel.esac.esa.int/HIPE_download.shtml. This corresponds to the developer's CIB (Continuous Integration Build) HIPE 9.0 build 2974.

   *  We also provide access to the latest stable developer build (latest stable CIB).
      * *Beware* _These developer builds do not undergo the same in-depth testing as the user releases do._ The latest developer build can be found [[http://herschel.esac.esa.int/CIB_disclaimer.html][here]].

   * Within HIPE you can access all the SPIRE data reduction and HIPE-user documentation. The *SPIRE Data Reduction Guide (SDRG)* follows the user pipeline scripts and also explains the details of pipeline processing and data analysis. It is also available online here:
      * [[http://herschel.esac.esa.int/hcss-doc-9.0/][HIPE-user documentation for latest released version (html)]]
      * [[http://herschel.esac.esa.int/hcss-doc-9.0/print/spire_drg/spire_drg.pdf#spire_drg][SDRG PDF format]] 

   * *SPIA*: The SPIRE Photometer Interactive Analysis (SPIA) package is available as a plug-in for HIPE. SPIA provides a structured GUI based access to the more intricate parts of the scan map photometer pipeline for SPIRE without the immediate need to resort to scripts. More information can be found in the SDRG or on the  [[https://nhscsci.ipac.caltech.edu/sc/index.php/Spire/SPIA][SPIA web page]]



----+++ The SPIRE Launch Pads

   *  The *SPIRE Launch Pads* are single sheet quick entries (like a cheat sheet) into SPIRE data reduction and providing quick references to the relevant sections in the SPIRE Data Reduction Guide. There are launch pads for Data Access, SPIRE Photometer and Spectrometer data reduction.

%TABLE{tablewidth="60%" valign="top" headerrows="1" sort="off" initsort="1" initdirection="down"}%
| *Document* |   *Version* |
| [[%ATTACHURL%/SPIRE_DataLaunchPad_v1.5.pdf][SPIRE Data Launch Pad]] | version 1.5 |
| [[%ATTACHURL%/SPIRE_PhotometerLaunchPad_v1.6.pdf][SPIRE Photometer Launch Pad]] | version 1.6 |
| [[%ATTACHURL%/SPIRE_SpectrometerLaunchPad_v1.6.pdf][SPIRE Spectrometer Launch Pad]] | version 1.6 |


----+++ Photometer data reduction

The best source of information for reducing SPIRE Photometer data is the *SPIRE Data Reduction Guide* available through the HIPE help. This runs through the User Pipeline scripts step by step, describes several other Useful Scripts, and offers advice for specific issues that might be encountered.


   * Note that SPIRE maps are in units of *Jy/beam*, and are calibrated in the assumption of a point source having a spectral index equal to -1, i.e. &#957;S<subscript>&#957;</subscript> = const. To calibrate your data for other cases or convert to e.g. Jy/sr, please refer to [[http://herschel.esac.esa.int/Docs/SPIRE/html/spire_om.html#x1-910005.2][section 5.2 of the SPIRE Observers' Manual]] .


The main issues that you might find in your data are: _undetected glitches_, _thermistor or detector jumps_, and _bad baseline removal_.

   * *Stripes in PSW, PMW and/or PLW (Level 2) maps*
      * All SPIRE photometry pipelines now use the destriper by default, which improves the issue of stripes in Level 2 maps. There should be noticeable improvements in that respect with HIPE version 9. The destriper documentation can be found on the [[https://nhscsci.ipac.caltech.edu/sc/index.php/Spire/PhotScanMapDestriper][NHSC website]]

   * *De-glitcher masks faint sources*
      *  For data taken in Parallel Mode in particular (sampling at 10Hz, at high speed 60"/s), the de-glitcher may flag very faint sources as glitches when it is run with standard parameters. Faint sources may have a "delta function" shape due to the low sampling rate, which looks similar to a small glitch. Try modifying the "correlation parameter" to 0.95: this will decrease the number of detected glitches - it may be better to have a limited detection rate in first level deglitching and defer to Level 2 deglitching.

   * *Cooler temperature variations (Cooler Burps)*
      * The SPIRE cooler is recycled after 48 hours. Between 6 to 7h after the cooler recycle ends, its temperature rises steeply and reaches the stable _plateau_. Observations taken during such times may exhibit stripes in the final maps. An option to correct for this effect is now available in the User Pipelines (See the SPIRE DRG for details).

   * *NaN pixels present in the PSW, PMW and/or PLW (Level 2) maps*
      * This effect, related to data masking or poor coverage, is more evident in single fast-scan Parallel Mode maps. To avoid !NaNs, increase the pixel size (i.e., decrease the map's resolution).
      * This effect can also occur with destriped maps. In this case check if increasing the sigma parameter or switching off the Level 2 deglitcher helps. 

   * *WCS in 3-colour images*
      * Problems with the wrong WCS in the output RGB images in all observations reduced with HIPE 8 have been fixed in HIPE 9

   * *Quality flags in the quality context*
      * Currently, the quality flags at the quality context inside the observation context are just meant for HSC/ICC internal evaluation of the quality of the products and not for the users. In case the data had some serious quality problem, the PI of the program has been contacted about it. Otherwise, only information in the quality summary, when available, should concern the observers. 

----+++ Source Extraction and Photometry

   * The current recommended method for photometry =sourceExtractorTimeline= task (formerly known as the Timeline Fitter) which works on the detector timelines. The Map based algorithm  =sourceExtractorSussex= (SUSSEXtractor) providers good results and is useful on larger maps where the  =sourceExtractorTimeline= will be significantly slower. =sourceExtractorDaophot= (DAOphot) also provides a reasonable estimate of the source flux but may require an aperture correction.


----+++ Spectrometer data reduction

The best source of information for reducing SPIRE Spectrometer data is the *SPIRE Data Reduction Guide* available through the HIPE help. This runs through the User Pipeline scripts step by step, describes several other Useful Scripts, and offers advice for specific types of sources:

   * Faint (<10 Jy) and medium (<100 Jy) strength sources
   * Bright sources (>500 Jy)
   * Extended sources
   * Observations with few repetitions
   * H+L observations

For faint sources, the subtraction of instrument, telescope and background emission is particularly important. Optimum subtraction can be performed in several ways (read the *SPIRE Data Reduction Guide* for details):

   1. Subtract the Dark Sky spectrum closest to your observation (use the "Background Subtraction" script in HIPE)
   2. Subtract the spectrum of surrounding detectors (use the "Background Subtraction" script in HIPE)

Dark Sky observations are observed on every SPIRE Spectrometer OD, and are all public in the Archive. 

[[http://herschel.esac.esa.int/twiki/bin/view/Public/SpireDailyDarkObservations][A listing of the available *Dark Sky observations* can be found here]].

----+++ Spectral Cube Analysis

*Spectral cubes* are produced by the Spectrometer pipeline for mapping observations. Some tips, suggestions and examples of spectral cube analysis for SPIRE data are provided [[http://herschel.esac.esa.int/twiki/bin/view/Sandbox/FtsCubeSection9][here]].

---++ *Cookbooks* 

Cookbooks are provided inside the SPIRE Data Reduction Guide (see above).

The standalone "Photometry Cookbook", is no longer maintained - it is being incorporated into the SPIRE DRG - please see the SDRG  for photometry cookbook information, and raise a [[http://herschel.esac.esa.int/esupport/Helpdesk][Helpdesk]] ticket if you find  something missing.

---++ *SPIRE calibration file versions*

The available calibration trees for SPIRE are listed below (with the current operational version at the top). 

%TABLE{ tablewidth="100%" valign="top" headerrows="1" sort="off" initsort="2" initdirection="up"}%
| *SPIRE Calibration Tree* | *Applicable HIPE Version* | *Comment* | 
| *SPIRE_CAL_9_1*  | *HIPE v9* | *Final v9 cal tree*, currently used in operations.  | 
| SPIRE_CAL_8_1  | HIPE v8 | Final v8 cal tree  | 
| SPIRE_CAL_7_0  | HIPE v7 | Final v7 cal tree.  | 
| SPIRE_CAL_6_1  | HIPE v6   | Final v6 cal tree  | 
|  (SPIRE_CAL_6_0)  | HIPE v6  | Spec major update  | 
| SPIRE_CAL_5_2  | HIPE v5 | Final v5 cal tree | 
|  (SPIRE_CAL_5_1)  | HIPE v5 |  | 
|  (SPIRE_CAL_5_0)   | HIPE v5  | Phot flux conv. based on Neptune. Spec major update  | 
| SPIRE_CAL_4_0   | HIPE v4  | Spec point source flux conv based on Uranus | 
| SPIRE_CAL_3_2  | HIPE v3 |  | 
|  (SPIRE_CAL_3_1)  | HIPE v3 |  | 
|  (SPIRE_CAL_3_0)  | HIPE v3  |  | 
| SPIRE_CAL_2_1  | HIPE v2  | Spec point source flux conv based on Vesta | 
|  (SPIRE_CAL_2_0)  | HIPE v2 |  | 
| SPIRE_CAL_1_2  | HIPE v1  | Phot flux conv based on Ceres  | 
|  (SPIRE_CAL_1_1)  | HIPE v1 | Pre-launch dummy values | 

More details of the changes in each version are given [[http://www.spire.rl.ac.uk/icc/CalibrationTrees.html][here]]. 

Any of the calibration trees can be retrieved in HIPE from the HSA using (e.g.)
=cal = spireCal(calTree="spire_cal_9_1")= etc. The default (applicable to the Hipe version) can be obtained with =cal = spireCal(calTree="spire_cal")=
The latest calibration context for SPIRE can be obtained as a .zip file from [[ftp://ftp.sciops.esa.int/pub/hsc-calibration/latest_cal_tree/][Latest calibration trees]]. The zipped file is a pool, so it should be unpacked in the local store folder and then loaded in HIPE with =cal = spireCal(pool="spire_cal_9_1")=.

See [[http://herschel.esac.esa.int/hcss-doc-9.0/][the SPIRE Data Reduction Guide]] for more details. 

---++ *SPIRE calibration and performance* 

----+++ Photometer calibration

   * *SPIRE Photometer Beams:* 
      * These are available in the SPIRE calibration context, at the standard map pixel size of (6,10,14) arcsec/pixel for (250,350,500) µm bands, and can be accessed in HIPE after a calibration context has been loaded (see above).
      * %ICON{tip}% The observed beams at much finer scale of 1 arcsec/pixel, as well as the theoretical ones, are available from [[ftp://ftp.sciops.esa.int/pub/hsc-calibration/SPIRE/PHOT/Beams/][here]] . Please read the [[http://herschel.esac.esa.int/twiki/pub/Public/SpireCalibrationWeb/beam_release_note_v1-1.pdf][release note]] for more details. 

   * *SPIRE Photometer filter transmission curves:* 
      * These are also available in the SPIRE calibration context (photRsrf) and can be accessed in HIPE after a calibration context has been loaded (See above).

   * *Neptune and Uranus models used for the SPIRE photometer flux calibration:* 
      * The ESA2 models currently used in the SPIRE calibration are available [[ftp://ftp.sciops.esa.int/pub/hsc-calibration/PlanetaryModels/ESA2/][here]]. 

----+++ Spectrometer calibration

Important FTS information, including details of the calibration, point source and extended source calibration etc, is available in the SPIRE Observers' Manual, Sections 4.2 and 5.3. These two sections are a must-read for anybody processing SPIRE FTS data. 

---++ *Interest groups and scripts*

   * The following interest groups relate to processing of observations taken with SPIRE. The links provided allow subscription to these interest groups.
      * [[http://www.sciops.esa.int/mailman/listinfo/herschel_dp_spirephot/][subscribe]] to the SPIRE Photometer interest group
      * [[http://www.sciops.esa.int/mailman/listinfo/herschel_dp_parallel/][subscribe]] to the SPIRE and PACS parallel mode large map and point source extraction interest group 
      * [[http://www.sciops.esa.int/mailman/listinfo/herschel_dp_spirespec/][subscribe]] to the SPIRE Spectrometer interest group
      * [[http://www.sciops.esa.int/mailman/listinfo/herschel_dp_specmap/][subscribe]] to the PACS, SPIRE and HIFI spectral maps interest group
   * User contributed scripts: Users are welcome to submit scripts and software that they believe could be of general interest to the community to the Herschel [[http://herschel.esac.esa.int/esupport/Helpdesk]].

---++ *Further information*

Contact the [[http://herschel.esac.esa.int/esupport/Helpdesk][Helpdesk]]
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
pdf	Phot_Pipeline_Issue7.pdf	r1	manage	453.9 K	2011-05-06 - 11:07	LucaConversi
pdf	Pipe_Description_2.1.pdf	r1	manage	2078.9 K	2011-05-06 - 11:06	LucaConversi
pdf	SPIRE-BSS-DOC-002966_SPIRE_Spectrometer_pipeline_description_Issue_4_0_1.pdf	r1	manage	793.5 K	2012-02-28 - 09:49	EdwardPolehampton
pdf	SPIREPhotometryCookbook_jul2011_2.pdf	r1	manage	62.0 K	2011-08-11 - 09:27	IvanV	SPIRE Photo cookbook
pdf	SPIRE_DataLaunchPad_v1.5.pdf	r1	manage	203.3 K	2012-02-04 - 20:49	ChrisPearson	SPIRE Data Launch Pad v1.5
pdf	SPIRE_PhotometerLaunchPad_v1.5.pdf	r2 r1	manage	203.1 K	2012-02-10 - 11:56	ChrisPearson	SPIRE Photometer Launch Pad v1.5
pdf	SPIRE_PhotometerLaunchPad_v1.6.pdf	r1	manage	204.4 K	2012-03-14 - 16:48	ChrisPearson	SPIRE Photometer Launch Pad v1.6
pdf	SPIRE_SpectrometerLaunchPad_v1.6.pdf	r1	manage	192.0 K	2012-02-04 - 20:50	ChrisPearson	SPIRE Spectrometer Launch Pad v1.6
pdf	aa14519-10.pdf	r1	manage	1321.5 K	2011-01-24 - 09:51	AnthonyMarston	SPIRE A&A paper, Griifin et al 2010
pdf	aa14605-10.pdf	r1	manage	264.1 K	2011-01-24 - 16:48	IvanV	SPIRE in-flight calibration
pdf	beam_release_note_v1-1.pdf	r1	manage	4044.5 K	2011-05-11 - 09:44	LucaConversi
txt	bendoSourceFit_v0_9.py.txt	r1	manage	25.7 K	2011-08-10 - 09:03	IvanV
txt	maskTimelines.py.txt	r1	manage	5.8 K	2011-05-27 - 10:34	LucaConversi
Topic revision: r62 - 2012-07-30 - IvanV
Public
Webs
Public
TWiki