Table of Contents
Last updated 3 February, 2014.
Welcome to the HIFI Launch Pad!
The Launch Pad is intended to get you off the ground and into HIFI data reduction quickly. We summarise how you get your data, quickly inspect it, and re-pipeline it (if needed) and point you toward information in the rest of the HIFI Data Reduction Guide and the Herschel Data Analysis Guide about common aspects of HIFI data reduction and analysis. More detailed information regarding specific observing modes can be found in the cookbooks in the following chapters.
Herschel data are stored in ESA’s Herschel Science Archive (HSA):
Herschel data are identified with a unique number known as the Observation ID (ObsID) (e.g. 1342212115)
You can query the HSA from within HIPE using the Data Access perspective, from the → menu (see Figure 1.1).
You can also access the Herschel Science Archive by selecting (left-click) the HSA User Interface button (see Figure 1.2).
HIPE expects the data in the form of a pool. A pool is like a database, with observations organised in an Observation Context, containing links to all data, calibration, and supplementary files.
...from the HSA
There are several ways to get your data from the HSA, they are described in detail in the first chapter of the Herschel Data Analysis Guide (DAG) and summarised here.
To get multiple observations, download a tar file (not a pool) using the shopping basket (see details in the DAG first chapter)
To get one observation you can directly access the HSA:
via the command line
ObsID, useHsa=True) # example obs=getObservation(
Note that if you are not logged into HSA, a login window will pop-up, allowing you to log in.
If you find that you have trouble obtaining the latest version of your observation it may be that you have it cached on your disk. In that case, try:
ObsID, useHsa=True, useCache=False)
or you can use the “Send to external application” in the HSA User Interface.
If you directly access the HSA the data is not automatically saved to your disk and you should immediately save your observation with:
Once the data are saved on your hard disk, the Observation Context can be read into HIPE using:
If you try to access this pool in the same HIPE session in which you saved it, you must update your
list of local pools before HIPE will be able to find it. To do this, run the following command before
PoolManager.getInstance().propertyChange(java.beans.PropertyChangeEvent \ (PoolCreatorFactory.getCreators().get("lstore"), \ LocalStoreFactory.LOCAL_POOL_DEFINITIONS_PROP, None, None))
Alternatively, you can press the Refesh button in Storages and Pools in the HIPE Preferences.
...from a tar file
If your data came from a tar file (multiple observations download), uncompress and untar the file, then, from HIPE view Navigator (see HIPE Owners Guide for more details), locate the folder on your disk, and open the content of the folder by a left-click on the plus sign. You will see folders associated to all your observations, an auxiliary folder, a calibration folder, and all your obsIDs with a small image on the left-side of the obsID's name. To import an obsID into HIPE, just double-left-click on the file with an image (see Figure 1.3). The ObservationContext of that obsID will then load in the HIPE view Variables, under Observations.
Note that it is also possible to uncompress and untar your file directly from the HIPE view Navigator - once you locate your tar file, just double-left-click on the .tar.gz file. A task called decompress will open and allow you to unpack you tar file.
Similarly to data accessed from HSA, you can save your observation in a localPool once it is uppacked:
# obsID located in the HIPE view Variables, under Observations saveObservation(obsID, poolName="myPool")
...A quick look
The best way to quickly inspect your data is to look at the Browse Product in the Observation Summary, which you can click on to enlarge. The browse products show:
Point Mode observations: Two plots of Level 2 spectra with the H-polarisation to the left and the V-polarisation to the right. The upper and lower axes of the plots show the LSB and USB frequency scale, respectively. The AOR label and observation number are used to title the plot. The observing mode, source name, requested RA and dec are below the plot title.
Mapping observations: The map average spectra for the WBS-H (top) and WBS-V (bottom) is plotted per subband beside the integrated map of that subband. The upper and lower axes of the spectral plots show the LSB and USB frequency scale, respectively. The plots are arranged in order of increasing frequency. The integrated maps created with no correction done for any baseline issues. The x- and y- axes show the RA and declination, respectively, while the auxiliary axes show pixel coordinates. The colour scale used for the image is 'heat' and the intensity scale used is 'ramp' so the strongest emission in the map appears white.
The AOR label and observation number are used to title the browse product. The observing mode, source name, requested RA and dec are below the plot title.
Spectral Scans: The single sideband solution after deconvolution of the Level 2 WBS spectra. No baseline correction has been done prior to deconvolution. The H-polarisation is shown to the left and the V-polarisation to the right. The AOR label and observation number are used to title the plot. The observing mode, source name, requested RA and dec are below the plot title.
...A deeper look
SpectrumExplorer is the main tool to plot and inspect spectra.
It can be started by right mouse clicking on a SpectrumDataset or product (e.g. HTP ) and selecting → . When viewing the contents of an HTP you must click in the box next to the variable name to plot the spectra.
The selector panel appears at the bottom, with a row for each scan and a box (initially grey) for each subband. Clicking on each square will plot the spectrum for that subband, double-clicking will remove the plot.
Tooltips describe the actions possible from the button bar above the plot.
When should I reprocess my data?
The HIFI ICC recommends that your data should ideally be processed using the HIPE version that you are using for data analysis to ensure that you do not run into data-software compatibility problems. However, it is not anticipated that you will have problems if you keep within one HIPE version, e.g. perform data analysis in HIPE 13 on data that was pipelined with HIPE 12. In some cases we may recommend that certain observing modes are processed with a particular HIPE version, see the HIFI Instrument and Calibration page .
How do I reprocess my data?
To reprocess (re-run the pipeline) data you can find
hifiPipeline in the Applicable Tasks
menu when you click on an Observation Context in the Variables View. Select the spectrometers and the levels
between which you want to reprocess for and hit accept. Or, in the command line:
# Create a new Observation Context called newobs by reprocessing all levels of the Observation Context “obs” newobs = hifiPipeline(obs = obs, save=False) # # To just reprocess from Level 1 to Level 2, and only for the HRS newobs = hifiPipeline(obs = obs, fromLevel=1, upToLevel=2, apids = ['HRS-H', 'HRS-V'], save=False)
See Chapter 5 for more details.
How do I use newer calibration data?
To re-pipeline data using the latest calibration version available from the HSA, rather than the calibration data that was
shipped with your data, you must first configure the pipeline to go to the HSA for the calibration data. To do so, first
calconfig = configureHifiPipeline(useHsa=True)
You can tell the pipeline to use the newer calibration by checking the
cal box in the GUI and by passing
calconfig to the
palStore bullet, before running the pipeline.
In the command line this is all done as follows:
calconfig = configureHifiPipeline(useHsa=True) obs_1 = hifiPipeline(obs=obs, cal=1, palStore=calconfig)
See Section 5.3 for more details.
Standing waves can be fitted with sine waves and subtracted using
which appears as an applicable task when one clicks on the Observation Context variable.
On the command line, it is typically done as follows:
See Chapter 12 for more details.
Baselines can be easily removed using the
fitBaseline task. To create a new
observation context with baseline corrected Level 2 products, use:
obs_fit = fitBaseline(data=obs)
and follow the instructions that appear in the console. For more information, see Section 13.3
When a spectrum is selected in the Variables View, HIPE becomes the spectrum toolbox; a range of tools for performing spectral arithmetics and manipulation become automatically available.
View a spectrum in Spectrum Explorer and click on the crossed hammer and spanner icon in the toolbar above, the toolbox will appear to the right of the plot. Select the task you wish to perform from the drop-down menu and complete the GUI form.
The spectrum tasks available in the toolbox can also be used in scripts or in the command line. The simplest way to find the syntax is to copy the command from the console after using the GUI. Here, we give a couple of examples.
# # First extract a spectrum from the Observation Context, for example the Level 2 WBS-H-USB spectrum=obs.refs["level2"].product.refs["WBS-H-USB"].product.refs["box_001"].product["0001"] # #Stitch subbands together: StitchedSpectrum = stitch(ds=spectrum, variant="crossoverPoints", edgeTolerance=0.01,stepsize=0.0) # #extract a range in frequency (the ranges are given in the units in the spectrum) ExtractedSpectrum = extract(ds=spectrum,ranges=[(555.0, 558.7)])
The use of all the tasks is described generally in Spectrum Toolbox of the Herschel Data Analysis Guide where you will find more information. Some HIFI-specific examples can also be found in Chapter 17.
Spectral features can be fitted and removed using the SpectrumFitterGUI , which is available from the Spectrum Explorer. The tool can be used to interactively fit a single spectrum, or in automatic mode on multiple spectra using a previously defined model.
To interactively fit a spectrum, extract the region of interest or stitch all the subbands of the data together so that you have only one segment of data (this is one box in the Spectrum Explorer selector panel).
Interactive tasks and data issues that are particular to given observing modes are described in a series of cookbooks in Chapter 2:
Section 2.2: Single Point Mode: Dual Beam Switch
Section 2.3: Single Point Mode: Position Switch (included are some discussion on Platforming and Off-Emission)
Section 2.4: Single Point Mode: Frequency Switch
Section 2.5: Single Point Mode: Load Chop
Section 2.6: Spectral Map Mode
Section 2.7: Spectral Scan Mode
HIFI data can be exported to CLASS readable FITS files using the
task. You may wish to stitch subbands together before exporting to CLASS.
# Export one dataset to a FITS file: HiClassTask()(data = myspectra, fileName = 'myspectra.fits') # # Export one HIFI timeline product to a FITS file: HiClassTask()(data = myhtp, fileName = 'myhtp.fits') # Export the Level 2 spectra to a FITS file by supplying the ObservationContxt: HiClassTask()(data = obs, fileName = 'myl2spectra.fits')
The FITS file is written in the directory you started HIPE from unless you specify a path.
To read the FITS file into CLASS:
file out MyHIFISpectra.hifi mul fits read MyHIFISpectra.fits # # Now you have a CLASS file named MyHIFISpectra.hifi (you can use whatever you want as an # extension) that you can access like you always do in CLASS: # file in MyHIFISpectra.hifi find get first set unit f i device image white plot
Note that you need to use a recent version of CLASS, i.e. the version from April 2010 onwards.
See Chapter 23 for detailed information and examples on how to use the task.
Use the simpleFitsWriter to save spectra to FITS file.
Use the exportSpectrumToAscii task available from the Spectrum Toolbox to save spectra to text file.