Difference: HPDPDefinitionPage (34 vs. 35)

Revision 352016-03-02 - KatrinaExter

Line: 1 to 1
 
META TOPICPARENT name="HSC.WebHome"

Herschel Highly Processed Data Product Page

Line: 46 to 46
 
HPDP nameDescription, type of productPriorityRationalHowRequirementsEffortNumber of obsidsNotes
Point-source calibrated spectra: off-centred cases Provide the point-source corrected spectrum for pointed observations with point-like sources not located in the central spaxel; SpectrumTable (type of TableDataset) High All pointed observations have a SpectrumTable with point-source calibrated spectra in it, and if the astronomer knows the source is centred and a point, then this is a science product; if the source is off-centred, however, they would have to use HIPE to produce the PS spectrum Using the "POC" interactive pipeline script This script has been validated suficiently to be used for almost any flux level, but we would need a formal validation report and JB to tell us what parameters to set in the specific tasks Once we have the validation report and parameter values from JB (few days of work), just run the script and produce the standard result. Most of the effort is on identifying the off-centred point-like sources, which will require some investigation (a few weeks?) Need to check all pointed obsids (3500 unchoped and chopnod) but it is likely only a few dozen will produce this HPDP If producing this for off-centred point sources, why not for all point sources?
Red-leak spectra Produce the Level 2/2.5 products for spectra in the red leak region; entire L2/2.5 context High (but see notes) Currently the spectra are cut off at just after 190 microns, but we can now produce a reasonably-well line flux calibrated spectrum in this region Using the "calblock" SPG script Once validation of the red leak RSRF and the flatfielding has been done, this is ready to go Automatic, just running an SPG script There are 533 SED obsids covering this region; will be some line scan and range scan also However, this will probably become part of SPG 14.2
Changed:
<
<
Unchopped obsids with continuum removed
Subtract the continuum from all unchopped obsids so as to provide a cleaner spectrum; Level 2/2.5 context Mid (low on ICC list but high on mine) The continuum is very uncertain and it had been agreed last year at an ICC meeting that removing this was a "good thing to do". It will avoid users being confused about funny continuum levels or shapes and, especially for the targets less than some-10sJy continuum level this will be the science-ready end product Range scans: use the RV methods to smooth and subtract the continuum; Line scans: use an optimised version of the line-finding task in the flatfielding task of the SPG. Fit/smooth and remove continuum and then produce all final results Need to test methods and thereafter process should be automatic Producing the scripts: a few weeks (?), checking the results: continuous (but semi-automatic) all ~1200 on-sources unchopped obsids: We also need to produce a report on the unchopped mode: based on DF paper but more examples and a proper measure of uncertainty based on e.g. comparing repeat observations, scatter in super-off, etc.
New transients script results for unchopped obsids Producing the output of the unchopped obsids reduced with the new transients script Low (because that script has not been validated sufficiently) This script is supposed to produce better results and users may want to use it (esp. if they read the DF paper). It can only be done in HIPE. Running the ipipe script; do the parameters need to be optimised to the target? (unknown) Need to know that this script produces reliable results when run in a semi-automatic way (would be too much work to do all obsids manually) The validation is most of the work: could take several weeks. If the results of the bulk that the NHSC are running can be used, then the validation is the only work to do here Are ~1200 on-source unchopped obsids; Priority on the red camera and the range scans
Unchopped observations with possible long-term transient If observed after a bright source, an unchopped observation will have a long-term transient and hence incorrect continuum level and shape; two possible outcomes (1) just a quality comment warning of this possibilty (2) a corrected Level 2/2.5 context High as QC, Low as HPDP The continuum shape and level will definitely be more wrong for these obsids than for the average unchopped obsid; warning the users of this case is necessary so they know what science they can get, providing corrected results can only be done in HIPE Identify sources taken after a bright observation. Running the new transients script and playing with the background subtraction task To find the candidate sources: look at the calibration block for a different response, look at the V/s of the different scans to look for a transient; look at the flux levels of the obsids preceeding the unchopped one; If the new transients script works well, then can use the NHSC bulk results; if need to play with the script to get the best results, then need to learn how to do this Finding the candidates could take a few weeks; correcting the results is unknown because no-one at HSC or still in the ICC has enough experience with the new transients script Are ~1200 on-source unchopped obsids;
Identifying background contamination; Removing the background contamination For unchopped and chop-nod. QC: comment on obsids likely to be affected by contamination in the off; HPDP remove the continamination before doing the off-subtraction and produce a Level 2/2.5 context High as a QC, Low as an HPDP Results in SPG will be definitely not science-ready if the off position is contaminated Need to identifty contaminated sources by comparing spectra in different off positions, spectra again a super-off, looking for line peaks in the off-source spectrum Much effort in finding the sources, as while we have ideas how to do it they have not been tested; removing line contamination is probably fairly straightforward but will require intervention in the tasks of the SPG script, for continuum contamination we we will probably have to just remove a standard background Potentially some months, depends on whether our current idea work out-of-the-box or not All obsids need to be searched, but hoping some dozen of sources are only affected
Line list catalogue Spectral Feature catalogue Nice to have but ICC have no interest here and consider it Low SPIRE and HIFI have one, so why not PACS. Catalogues always produce unexpected science and are a very good Legacy product Need scripts to find lines, fit lines and maybe identify lines Need to create the scripts: KE has a student at Leuven testing out fitting lines, and may have a student in 2016-17 to test out finding lines If the student projects go well, some time in 2017 the resulting scripts could be run on the PACS spectroscopy archive (or subsets of that)
>
>
Unchopped obsids with continuum removed
Subtract the continuum from all unchopped obsids so as to provide a cleaner spectrum; Level 2/2.5 context Mid (low on ICC list but high on mine) The continuum is very uncertain and it had been agreed last year at an ICC meeting that removing this was a "good thing to do". It will avoid users being confused about funny continuum levels or shapes and, especially for the targets less than some-10sJy continuum level this will be the science-ready end product Range scans: use the RV methods to smooth and subtract the continuum; Line scans: use an optimised version of the line-finding task in the flatfielding task of the SPG. Fit/smooth and remove continuum and then produce all final results Need to test methods and thereafter process should be automatic Producing the scripts: a few weeks (?), checking the results: continuous (but semi-automatic) all ~1200 on-sources unchopped obsids: We also need to produce a report on the unchopped mode: based on DF paper but more examples and a proper measure of uncertainty based on e.g. comparing repeat observations, scatter in super-off, etc.
New transients script results for unchopped obsids Producing the output of the unchopped obsids reduced with the new transients script Low (because that script has not been validated sufficiently) This script is supposed to produce better results and users may want to use it (esp. if they read the DF paper). It can only be done in HIPE. Running the ipipe script; do the parameters need to be optimised to the target? (unknown) Need to know that this script produces reliable results when run in a semi-automatic way (would be too much work to do all obsids manually) The validation is most of the work: could take several weeks. If the results of the bulk that the NHSC are running can be used, then the validation is the only work to do here Are ~1200 on-source unchopped obsids; Priority on the red camera and the range scans
Unchopped observations with possible long-term transient If observed after a bright source, an unchopped observation will have a long-term transient and hence incorrect continuum level and shape; two possible outcomes (1) just a quality comment warning of this possibilty (2) a corrected Level 2/2.5 context High as QC, Low as HPDP The continuum shape and level will definitely be more wrong for these obsids than for the average unchopped obsid; warning the users of this case is necessary so they know what science they can get, providing corrected results can only be done in HIPE Identify sources taken after a bright observation. Running the new transients script and playing with the background subtraction task To find the candidate sources: look at the calibration block for a different response, look at the V/s of the different scans to look for a transient; look at the flux levels of the obsids preceeding the unchopped one; If the new transients script works well, then can use the NHSC bulk results; if need to play with the script to get the best results, then need to learn how to do this Finding the candidates could take a few weeks; correcting the results is unknown because no-one at HSC or still in the ICC has enough experience with the new transients script Are ~1200 on-source unchopped obsids;
Identifying background contamination; Removing the background contamination

For unchopped and chop-nod. QC: comment on obsids likely to be affected by contamination in the off ; HPDP remove the continamination before doing the off-subtraction and produce a Level 2/2.5 context, or provide the on-source and off-source cubes separately so the user can assess the damage

High as a QC, Low as an HPDP Results in SPG will be definitely not science-ready if the off position is contaminated Need to identifty contaminated sources by comparing spectra in different off positions, spectra again a super-off, looking for line peaks in the off-source spectrum Much effort in finding the sources, as while we have ideas how to do it they have not been tested; removing line contamination is probably fairly straightforward but will require intervention in the tasks of the SPG script, for continuum contamination we we will probably have to just remove a standard background Potentially some months, depends on whether our current idea work out-of-the-box or not All obsids need to be searched, but hoping some dozen of sources are only affected

Hav eto consider dases where there is emision in the off that is also in the on, whether at the same or a different level, or wavelength shifted or not, and also that is not in the on. For both, the SPG results will be wrong, but with the first you may not know (with the second you will probably see absorption lines where the contamination has been subtracted)

Line list catalogue Spectral Feature catalogue Nice to have but ICC have no interest here and consider it Low SPIRE and HIFI have one, so why not PACS. Catalogues always produce unexpected science and are a very good Legacy product Need scripts to find lines, fit lines and maybe identify lines Need to create the scripts: KE has a student at Leuven testing out fitting lines, and may have a student in 2016-17 to test out finding lines If the student projects go well, some time in 2017 the resulting scripts could be run on the PACS spectroscopy archive (or subsets of that)
 

SPIRE-P

 
This site is powered by the TWiki collaboration platform Powered by Perl