Difference: SpireCalibrationWeb (21 vs. 22)

Revision 222011-05-27 - LucaConversi

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

SPIRE instrument and calibration web pages

Line: 80 to 80
 
  • Neptune and Uranus models used for the SPIRE flux calibration: the ESA2 models currently used in the SPIRE calibration are available here.
Deleted:
<
<

Data processing

HIPE data processing documentation can all be found at:

 

Maps

Line: 91 to 87
 
  • By default, the SPIRE pipeline uses a nšive map-maker. In this case, the error map is simply the standard deviation of all the data points falling into a given pixel. As a consequence, error maps contain increased errors associated with binning data from Gaussian sources, producing a torus shape; this is an artifact of the map-making process.
Added:
>
>

Data processing known issues

  • In order to obtain the best possible Level 2 SPIRE photometry data, the observations might have to be reprocessed with the latest HIPE User Release.

  • Stripes in PSW, PMW and/or PLW (Level 2) maps
    • Most of the stripes that are present in the final maps are due to a combination of thermal drifts (which in few cases are not efficiently removed) and median baseline subtraction. A similar effect is caused by very bright sources: in this case, the problem resides in the median baseline subtraction only. Suggested solutions:
      • switch to a baseline subtraction using a polynomial fitting using the optional task baselineRemovalPolynomial. If there are no jumps in the timelines, you may also try to run the baseline removal on the entire timeline;
      • in the case of bright sources, you may try to mask them before running the baseline removal (either median or polynomial): you can use this script as a templeate

  • De-glitchter masks faint sources
    • The de-glitcher is a very delicate process. In particular, for data taken in Parallel Mode (sampling at 10Hz) and at high speed (60"/s) the de-glitcher with standard parameters may flag very faint sources as glitches. Bright sources are different from glitches in that they have a gaussian (i.e. beam/PSF) shape. For faint sources, the sampling rate could be not high enough and hence they have a "delta" shape, which is similar to a small glitch. The user might try to modify the correlation parameter to 0.95: this will decrease the number of detected glitches.

  • Some sources have saturated the ADC and the corresponding data have been masked
    • There is nothing a user can do: the source was simply too bright. If the user has other sources still not observed and of the same intensity, it is suggested to change the AORs to use the bright source mode.

  • Thermistor jumps
    • As of HIPE 6.0.3, a new module together called signalJumpDetector in place to identify the jump and to exclude the affected thermistor(s).

  • NaNs pixels present in the PSW, PMW and/or PLW (Level 2) maps
    • This effect, related to data masked for various reasons and poor coverage (not enough redundancy), is more evident in single fast-scan Parallel Mode maps. To avoid NaNs, increase the pixel's dimension (i.e., decrease the map's resolution)

  • Quality flags in the quality
    • Currently, the quality flags at the quality context inside the observation context are just meant for HSC/ICC internal evaluation of the quality of the products and not for the users. In case the data had some serious quality problem, the PI of the program has been contacted about it. Otherwise, only information in the quality summary, when available, should concern the observers.
 

Tips to re-reduce your data

  • Always remember to update to the latest calibration tree compatible with the HIPE built you are using (See the SPIRE Data Reduction Guide, Chapter 3 for a detailed explanation and examples). Assuming the observation is loaded into HIPE as a variable named obs:
Line: 103 to 124
 
    • Undetected glitches: you may try to play with the parameters of the waveletDeglitcher, in particular changing correlationThreshold parameter; other solution is to use the alternative sigmaKappaDeglitcher
    • Thermistor jumps: this should be automatically solved re-reducing your observation as of HIPE v. 6. If this is not the case, you must exclude the affected thermistor when running the temperatureDriftCorrection adding e.g. pswThermistorSelect='T1'
    • Failure of Temperature Drift Correction: Due to an update of the Temperature Drift Correction task in the pipeline, the pipline may fail with an Index argument 0 is out of range error if run with Calibration Tree spire_cal_6_0. Please update to use spire_cal_6_1 to solve the problem (See the SPIRE Data Reduction Guide, Chapter 3).
Changed:
<
<
    • Bad baseline removal as of Hipe v. 6.x, a new polynomial fit (in comparison to the standard median) for baseline removal has been added as a prototype. Assuming that your Observation Context is stored in a variable named obs, you can call it as e.g.:
>
>
    • Bad baseline removal (see also above) as of Hipe v. 6.x, a new polynomial fit (in comparison to the standard median) for baseline removal has been added as a prototype. Assuming that your Observation Context is stored in a variable named obs, you can call it as e.g.:
 
from herschel.spire.ia.pipeline.phot.baseline import BaselineRemovalPolynomialTask
baselineRemovalPolynomial = BaselineRemovalPolynomialTask()
Line: 168 to 189
 
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1304680364" name="SPIRE-BSS-DOC-002966_SPIRE_Spectrometer_pipeline_description_Draft_3_1.pdf" path="SPIRE-BSS-DOC-002966_SPIRE_Spectrometer_pipeline_description_Draft_3_1.pdf" size="774644" user="Main.LucaConversi" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1305107081" name="beam_release_note_v1-1.pdf" path="beam_release_note_v1-1.pdf" size="4141606" user="Main.LucaConversi" version="1"
Added:
>
>
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1306492467" name="maskTimelines.py.txt" path="maskTimelines.py.txt" size="5945" user="Main.LucaConversi" version="1"
 
META FILEATTACHMENT attr="" autoattached="1" comment="SPIRE in-flight calibration" date="1295887708" name="aa14605-10.pdf" path="aa14605-10.pdf" size="270415" user="Main.IvanV" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1304680043" name="Phot_Pipeline_Issue7.pdf" path="Phot_Pipeline_Issue7.pdf" size="464813" user="Main.LucaConversi" version="1"
META FILEATTACHMENT attr="" autoattached="1" comment="" date="1304679998" name="Pipe_Description_2.1.pdf" path="Pipe_Description_2.1.pdf" size="2128829" user="Main.LucaConversi" version="1"
 
This site is powered by the TWiki collaboration platform Powered by Perl