Difference: SpireCalibrationWeb (82 vs. 83)

Revision 832013-06-04 - LucaConversi

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

SPIRE instrument and calibration web pages

Line: 70 to 70
 

Photometer data reduction

Changed:
<
<

Overview

>
>
Overview
 The best source of information for reducing SPIRE Photometer data is the SPIRE Data Reduction Guide available through the HIPE help. This runs through the User Pipeline scripts step by step, describes several other Useful Scripts, and offers advice for specific issues that might be encountered.
Changed:
<
<

New definition of Leve2 products

>
>
New definition of Leve2 products
 
  • For versions of the HCSS prior to HIPE 10.0, a single point source calibrated (Jy/beam) map was provided in the Level 2 product for each of the PSW, PMW, PLW bands. However, for observations processed with HIPE 10.0 or later, more than one map calibration is made available within the Level 2 product. Maps are provided for the following scenarios for post HIPE 10.0 processing:
Line: 96 to 96
 
  • The SPIRE Photometer filter transmission curves, also known as Relative Spectral Response Functions (RSRF) are available here. For more details, please read the .readme file in this ftp folder.
Changed:
<
<

Data Processing Issues

>
>
Planck-HFI & Herschel-SPIRE cross calibration: absolute offset re-processing

Herschel-SPIRE detectors are only sensitive to relative variations, as a consequence the absolute brightness of the observed region is unknown and maps are constructed such that they have zero median. Planck-HFI detectors are similar to the SPIRE ones, however its observing strategy allows it to (almost) observe a sky's great circle every minute (having a 1 rpm spinning rate). By comparing the sky brightness as measured by COBE-FIRAS at the galactic poles (where the dust emission is lower), HFI is capable of setting an absolute offset to its maps. SPIRE and HFI share two channels with overlapping wavebands: SPIRE-PMW and HFI-857 have a similar filter profile, while SPIRE-PLW and HFI-545 are shifted by $\sim 10$\%.

As of HCSS 10, a new task named zeroPointCorrection is available: this task calculates the absolute offset for a SPIRE map based on cross-calibration with HFI-545 and HFI-857 maps, colour-correcting HFI to SPIRE wavebands assuming a grey body function with fixed beta. At first, Planck data needed by the task were delivered to HSC under special agreement: as a consequence, Herschel users were not able to re-process the absolute offset calculation. However, Planck data became public in April 2013 and it is now possible to exectue the zeroPointCorrection.

Files needed:

  • Download the HFI545 and HFI-857 maps from the HSC/SPIRE FTP area. These maps are derived from the ones available in the Planck Legacy Archive, but convolved with an 8 arcmin Gaussian beam in order to circularize the effective maps' beams, plus the maps absolute offset as estimated by the Planck-HFI team via cross-calibration with FIRAS (see Planck Collaboration VIII. 2013, In preparation)
  • Download the colour correction table file SpireHfiColourCorrTab_v1.1

The offsets are computed on extdPxW maps, calibrated for extended emission, with extended gain correction applied and in units of MJy/sr (as explained in the section 5.7 of the SPIRE Data Reduction Guide). Hence, the re-processing will start from a level-1 context (which may be the result of merging multiple observations, see e.g. the Photometry Map Merging scirpt available in HIPE under the menu ScriptsSPIRE Useful script) and then executing the following code:

# The script assumes that:
# 1. a Level1Context is already defined and it is named "level1"
# 2. a Level2Context is already defined and it is named "level2"

# Define properties needed by zeroPointCorrection task
Configuration.setProperty("spire.spg.hfi.colorc", "PATH_TO_FILE/SpireHfiColourCorrTab_v2.2.fits")
Configuration.setProperty("spire.spg.hfi.545map", "PATH_TO_FILE/DX9_map_545_smooth_8arcmin.fits")
Configuration.setProperty("spire.spg.hfi.857map", "PATH_TO_FILE/DX9_map_857_smooth_8arcmin.fits")

# Check if properties are correctly set
if (zeroPointIsRunnable): 
   level2ZeroPoint = MapContext()
   for key in level2.meta.keySet():
      level2ZeroPoint.meta[key]=level2.meta[key].copy()
   #
   # Load relative gain correction file
   chanRelGains = obs.calibration.phot.chanRelGain
   #
   # Create new Level1Context
   scansZeroPoint = Level1Context()
   scansZeroPoint.meta = level1.meta
   #
   # Apply relative gain correction, loading the original Level1Context from the "level1" variable
   for i in range(level1.getCount()):
      psp = level1.getProduct(i)
      if psp.type=="PPT":  psp.setType("PSP")   #for old Level 1 contexts
      psp = applyRelativeGains(psp, chanRelGains)
      scansZeroPoint.addProduct(psp)
   #
   # Try to load the de-striper diagnostic products to speed-up re-processing
   for array in arrays:
      diagref = level2.refs['psrc'+array.upper()+'diag']
      if diagref != None:
         diag = diagref.product
      else: 
         diag = None
      #
      # (Re-)run destriper on new Level1Context
      newscans,mapZero,diagZero, p4,p5 = destriper(level1=scansZeroPoint, array=array, nThreads=2, \
            withMedianCorrected=True, useSink=True, startParameters=diag)
      #
      # Save diagnostic product, this time with prefix extd, into the "level2" variable
      level2ZeroPoint.refs.put(array,ProductRef(mapZero))
      level2.refs.put('extd'+array.upper()+'diag', ProductRef(diagZero))
   #
   # Run the zeroPointCorrection tasks on extdPxW maps
   zeroPointMaps, zeroPointParam=zeroPointCorrection(level2=level2ZeroPoint, hfiFwhm=8.0)
   #
   # Populate the "level2" variable with extdPxW maps, with the computed offset
   for array in arrays:
         level2.refs.put("extd"+array.upper(),ProductRef(zeroPointMaps.getProduct("extd"+array.upper())))

Data Processing Issues
  The main issues that you might find in your data are: undetected glitches, thermistor or detector jumps, and bad baseline removal.
Line: 119 to 188
 
  • Quality flags in the quality context
    • Currently, the quality flags at the quality context inside the observation context are just meant for HSC/ICC internal evaluation of the quality of the products and not for the users. In case the data had some serious quality problem, the PI of the program has been contacted about it. Otherwise, only information in the quality summary, when available, should concern the observers.
Changed:
<
<

Known Issues in ODs 1304 & 1305

>
>
Known Issues in ODs 1304 & 1305
 For (yet) unknown reasons, the three detectors PSW-B5, PSW-E9 and PSW-F8 - that use to behave well during the entire mission - were noisy during the two operational days 1304 and 1305. The result are stripes visible in the final PSW map which the current (HIPE 10) pipeline is not able to correct. The solution is to mask and exclude these detectors from the analysis. This could be done in 2 ways:
  1. You can use the SpireMaskEditor GUI as described in sec. 7.4.2 of the SPIRE Data Reduction Guide: write-click on your observation context variable and then select Level1_SpireMaskEditor and set to Master all samples in all scans (listed as BBID) for the detectors mentioned above.
  2. Alternatively, you can use the following line of codes:
Line: 177 to 246
  A listing of the available Dark Sky observations can be found here.
Changed:
<
<

Spectral Cube Analysis

>
>
Spectral Cube Analysis
  Spectral cubes are produced by the Spectrometer pipeline for mapping observations. Some tips, suggestions and examples of spectral cube analysis for SPIRE data are provided here.
Line: 272 to 340
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1330422558" name="SPIRE-BSS-DOC-002966_SPIRE_Spectrometer_pipeline_description_Issue_4_0_1.pdf" path="SPIRE-BSS-DOC-002966_SPIRE_Spectrometer_pipeline_description_Issue_4_0_1.pdf" size="812580" user="Main.EdwardPolehampton" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="SPIRE Photometer Launch Pad v1.7" date="1345816708" name="SPIRE_PhotometerLaunchPad_v1.7.pdf" path="SPIRE_PhotometerLaunchPad_v1.7.pdf" size="210362" user="Main.ChrisPearson" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="SPIRE A&A paper, Griifin et al 2010" date="1295862676" name="aa14519-10.pdf" path="aa14519-10.pdf" size="1353253" user="Main.AnthonyMarston" version="1"
Added:
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1370344174" name="SpireHfiColourCorrTab_v1.1.fits" path="SpireHfiColourCorrTab_v1.1.fits" size="20160" user="Main.LucaConversi" version="1"
 
This site is powered by the TWiki collaboration platform Powered by Perl