This page compiles the plans and needs to prepare and serve Highly Processed Data Products (HPDPs) to the Scientific Communities. There are two main ways considered to fulfil this: a web page, similar to that used for User Provided Data Products (UPDP, see also
), and the HSA User Interface (HUI). The following focusses mostly on the latter.
is a template for release notes to be associated with HSC-curated HPDP deliveries.
HPDPs have been identified by each instrument team, although it is likely that not all of them will be possible to prepared in certain cases. We indicate hereafter the HPDPs currently contemplated by each instruments, with some common delivery categories that will be considered or not.
| HPDP name |
Description |
Data |
Catalogue (tables) |
Images (PNGs, etc) |
Documentation (Readme, QSummary, etc) |
Prototype delivery |
| Expert Reduced Spectral Scans |
Baseline-cleaned deconvolved spectral scans |
Yes - level2.5 equivalent FITS files |
No |
Yes - postcards |
Yes |
|
| Expert Reduced Spectral Maps |
Baseline-cleaned regridded cubes l scans |
Yes - level2.5 equivalent FITS files |
No |
Yes - postcards |
Yes |
|
| Isolated OFF spectra |
OFF spectra as produced by the SPG, potentially cleaned further |
Yes - level2.5 equivalent FITS files |
No |
Yes - postcards |
Yes |
|
| Non-averaged level2 for point mode |
Level2 data as produced by modified version of SPG |
Yes - level2.5 equivalent FITS files (multiple spectra) |
No |
Yes - postcards |
Yes |
|
| Uncertainty tables |
Uncertainty tables as produced by the SPG |
No |
Yes - TableDataset as FITS files |
No |
Yes |
|
| Line list catalogue |
Identified Spectral Line catalogue |
No |
Yes - one file per obsid, and one concatenated file |
Yes - static (PNGs) and interactive (html) images |
Yes |
|
| Anomalous observations |
Fix obsids that processed wrongly with 14.1 - see HIFI-6380 |
Yes - level2.5 equivalent FITS files |
No |
Yes - postcards |
Yes |
|
| HPDP name |
Description; product |
Priority |
Rational |
How |
Requirements |
Effort |
Number of obsids |
Notes |
| Red-leak spectra |
Produce the Level 2/2.5 products for spectra in the red leak region; Level 2/2.5 context |
High |
Currently the spectra are cut off at just above 190 microns, but we can now produce a reasonably-well line flux calibrated spectrum in this region |
Using a slightly modified version of the the "calblock" SPG script |
No extra requirements |
Automatic, just running an SPG script |
~350 |
|
| PointingOffsetCorrection spectra for pointed observations |
Provide the point-source corrected spectrum for pointed observations; possibly focussing only on those with point-like sources not located in the central spaxel in them; a SpectrumTable |
High |
The results of the POC script can be better than those provided in the SPG, and this is especially so for off-centred point sources |
Using the "POC" interactive pipeline script |
This script has been validated sufficiently to be used for almost any flux level, but we would need to run sanity checks on the results to be sure this applies to the fainter sources. If we limit the provision to point-like sources, off-centred or not, then we need to identify those |
Most of the effort is on identifying the candidates (if not done for all pointed observations, willy-nilly). Some effort on sanity checking |
|
|
| Blue spectra (<55mum) |
Provide the blue end of the spectral range, which is not part of SPG as it is uncalibratable. |
High |
A slightly modified version of the SPG script can be used to create these ranges, where we do not the the flux calibration |
Using a slightly modified ipipe/SPG script |
need to identify the obsids with this range |
not much effort, mostly on making sure that this "blue bulk" runs probably and creates correct products |
|
|
| Unchopped obsids with continuum removed |
Subtract the continuum from all unchopped obsids so as to provide a cleaner spectrum; Level 2/2.5 context |
Mid |
The continuum is very uncertain and it had been agreed last year at an ICC meeting that removing this was a "good thing to do” |
Range scans: use the RV methods to smooth and subtract the continuum; Line scans: use an optimised version of the line-finding task in the flatfielding task of the SPG. Fit/smooth and remove continuum and then produce all final results |
Need to test methods and thereafter processing should be automatic |
Producing the scripts: a few weeks (?), checking the results: continuous (but semi-automatic) |
all ~1200 on-sources unchopped obsid |
We also need to produce a report on the unchopped mode: based on DF paper but more examples and a proper measure of uncertainty based on e.g. comparing repeat observations, scatter in super-off, etc. |
| Ad-hoc HPDPs |
For obsids found to be badly handled by the SPG, an expert will make the necessary adjustments to the pipeline so it can run; Level 2/2.5 context |
Mid |
These obsids are either not provided, or the science products are not of the same standard as all other obsids |
These are obsids identified in an ad-hoc manner as needing help. Jira tickets, Kayako, unchopped with bad transients identified while the unchopped study is done, etc |
Will only be done if it is feasible in terms of time and manpower |
|
so far <10 candidates suggested |
|
| Line list catalogue |
Spectral Feature catalogue |
Nice to have but ICC have no interest here and consider it Low |
SPIRE and HIFI have one, so why not PACS? Catalogues always produce unexpected science and are a very good Legacy product |
Need scripts to find lines, fit lines and maybe identify lines |
Need to create the scripts: KE has a student at Leuven testing out fitting lines, and may have a student in 2016-17 to test out finding lines |
If the student projects go well, some time in 2017 the resulting scripts could be run on the PACS spectroscopy archive (or subsets of that) |
|
Is being done by KME as a science project |
| Identifying background contamination; Removing the background contamination DE-SCOPED as an HPDP, but identifying them for QC is still on the table |
|
Low |
Results in SPG will be definitely not science-ready if the off position is contaminated |
Need to identifty contaminated sources by comparing spectra in different off positions, spectra again a super-off, looking for line peaks in the off-source spectrum |
|
|
|
|
| New transients script results for unchopped obsids DE-SCOPED |
Producing the output of the unchopped obsids reduced with the new transients script |
Low |
This script is supposed to produce better results and users may want to use it (esp. if they read the DF paper) %RED; now realised that that this is not the case |
|
|
|
~1200 |
|
| HPDP name |
Description |
Data |
Catalogue (tables) |
Images (PNGs, etc) |
Documentation (Readme, QSummary, etc) |
Prototype delivery |
| Semi-Extended calibration correction |
SECT correction for semi-compact sources |
Yes - level2 equivalent FITS files |
No |
Yes - postcards |
Yes |
|
| Extended background correction |
Extended background correction for point sources |
Yes - level2 equivalent FITS files |
No |
Yes - postcards |
Yes |
|
| Anomalous data correction |
Isolated cases of failed processing due to some on-board anomaly (see also DATA-75) |
Yes - level2 equivalent FITS files |
No |
Yes - postcards |
Yes |
|
| Line list catalogue |
Spectral Feature catalogue (see also this page ) |
No |
Yes - one file per obsid, and one concatenated file |
Yes |
Yes - paper |
|
| FTS high absoluteResolution cubes |
Remove isolated spectra with offending resolution and regenerate final cubes |
Yes - level2 equivalent FITS files |
No |
? |
Yes |
|
| Spectral scans |
Un-averaged level-2 spectra |
Yes - level2 equivalent products |
No |
YES |
No |
We provide here the HUI requirements needed to support the storage, queries and access to the various HPDPs and Ancillary Data Products:
), and the HSA User Interface (HUI). The following focusses mostly on the latter.
Public Web
Create New Topic
Index
Search
Changes
Notifications
Statistics
Preferences
Public
