Chapter 7. Spectral Fitting

Table of Contents

7.1. Spectrum fitting
7.1.1. Using the Spectrum Fitter GUI: an overview
7.1.2. Using the Spectrum Fitter (command-line fitting): an overview
7.1.3. Fitting tips
7.2. Worked Example: Fitting a polynomial to the baseline/continuum
7.2.1. Worked Example: Fitting a polynomial to the baseline/continuum in the command line
7.3. Worked Example: Fitting a polynomial to a spectral cube (or any multi-spectrum dataset)
7.3.1. Worked Example: Fitting a polynomial to a spectral cube (or any multi-spectrum dataset) in the command line
7.4. Worked Example: Fitting Gaussians and a polynomial to a spectrum
7.4.1. Worked Example: Fitting Gaussians and a polynomial to a spectrum in the command line
7.5. Worked Example: Fitting multiple lines (Gaussians) and a Polynomial baseline to a cube and making maps of the results
7.5.1. With the GUI
7.5.2. On the command line
7.5.3. Making 2d maps from the fit results
7.6. Adding and Initialising Models
7.7. Configuring the Spectrum Fitter GUI to automatically apply a fit upon opening
7.8. Setting weights
7.9. Setting limits to model parameters
7.10. Fixing model parameters
7.11. Modifying Models
7.12. Applying a fit
7.13. Inspecting fit parameter results
7.14. Deleting models and excluding models from a fit
7.15. Resetting and restarting fitting
7.16. Saving a script
7.17. Saving the residual and models
7.18. Saving a SpectralLineList
7.19. Obtaining a line integral
7.20. Using Saved models
7.21. Automatic fitting of multiple datasets
7.22. Continuing work on the residual outside of the Spectrum Fitter GUI
7.23. Using the Combo Model
7.24. Models available to the fitter
7.25. How to add your own model
7.26. Selecting the best fitter engine
7.27. NaNs and the Spectrum Fitter
7.28. Making images from fitting results to cubes: the ParameterCube
7.28.1. After fitting with the MultiFitter tab of the Spectrum Fitter GUI
7.28.2. After fitting with the MultiFitter on the command line
7.28.3. Manipulating the images taken from the ParameterCube.
7.29. Calculating uncertainty and error after fitting
7.29.1. Introduction to errors or fitting and confidence
7.29.2. Practical information for getting the fitting errors in HIPE
7.29.3. Advanced practical information
7.30. Troubleshooting and limitations of the fitter

7.1. Spectrum fitting

In this chapter we show how to fit spectra. Fitting can be done using the Spectrum Fitter GUI (SFG), which provides a user-friendly interface and allows you to immediately see the results of the fitting. For command-line fitting you can use the Spectrum Fitter (which is the core of the SFG). Any SpectrumContainer can be fit, such as Spectrum1D, Spectrum2D, the HIFI and SPIRE extensions of Spectrum1/2D, SpectralSimpleCubes and the PACS extension of SpectralSimpleCube.

In addition to fitting features in a single spectrum, which is how you will typically start fitting, the fitter tools allow you to automatically fit models to a set of data (multi-fitting), which is useful when working with large sets of data and with spectral cubes. It is also possible to make fits using several models with linked parameters (combo-fitting), which is useful, for example, for fitting hyper-fine lines; this options can only be done with command-line fitting. You can also constrain fitting parameters to within defined limits. You can save the models you define to HIPE to be re-used in another session and there are various export options available for the fitting results. It is also possible to create your own models to use with the fitting tools.

This chapter concerns fitting spectra. It is possible to fit to array datasets such as Double1d using the various fitter functions available in HIPE and this is described in the Scripting Guide. The Spectrum Fitter and Spectrum Fitter GUI are built upon these fitter functions but are designed to work specifically on spectra.

We begin with a general introduction on how the GUI and the command-line approaches work, and follow that with worked examples that are tuned to what Herschel users are likely to do. The rest of this chapter contains detailed information about the usage of the SFG and the Spectrum Fitter that will allow you to fine-tune your spectrum fitting and take best advantages of the capabilities offered by the spectrum fitting package in HIPE.

[Tip] Tip

Spaxels and pixels: mean the same thing, but HIFI uses "pixel" while PACS and SPIRE use "spaxel". These are the spatial-spectral unit of the cube, so one spaxel/pixel is one spatial element of your cube (one "square") with a full spectrum contained within it. If you change your spatial grid, e.g by regridding the cube, the spaxels/pixel is still the same thing, it is just that their sizes have changed.

[Tip] Tip

Worked examples: scripts that take the user through the processing of fitting cubes using command-line fitting have been provided in the PACS build of HIPE ("Scripts" menu). While they work from PACS example observations and are designed for those type of data, the fitting parts of these scripts will also work, with only slight changes, on HIFI and SPIRE cubes.

7.1.1. Using the Spectrum Fitter GUI: an overview Starting the GUI

Both the SFG and the Spectrum Fitter can be used with anything that you can display in the Spectrum Explorer. If you are unfamiliar with fitting then you should begin with the Spectrum Fitter GUI, particularly because—as shown in the worked examples in the following sections—you can export a script from the Spectrum Fitter GUI that can be used as a seed for future scripts you can write to do spectrum fitting.

The SFG is available in the Tasks panel (because it must be registered to work on Spectrum Containers, which makes HIPE show it as an available task) but it should only be opened from the Spectrum Explorer. To open the SFG from the Spectrum Explorer, follow these steps (and see the figure below):

  1. From your spectrum/cube in the Variables view, right click and select the Spectrum Explorer (often a double click will also do this). (The use of the Spectrum Explorer is explained in Chapter 6 for spectral cubes and Chapter 5 for single spectra.) The Spectrum Explorer opens in the Editor view of HIPE.

  2. From there, display one spectrum—the one you want to fit or, if working with multi-spectra datasets such as cubes, the one on which you wish to test the models to fit to the entire dataset. The way a spectrum is selected/deselected for display depends on whether you are fitting to a spectrum in a spectrum dataset or in a cube. For a spectrum dataset, click on a square in the row indicating the spectrum you want to fit in the Data Selection panel (see Section 5.3.1 to learn more). For a cube, click on a single spaxel/pixel from the cube image of the Data Selection panel of the Spectrum Explorer (see Section 6.6.2 to learn more). In both cases, the selected spectrum will appear in the Spectrum panel, this is illustrated in Figure 7.1 for a SPIRE cube and a HIFI WBS spectrum.

    Selecting one spectrum to fit with the Spectrum Fitter GUI. Here a pixel near the centre of the cube (highlighted with a green box) is displayed in the top left Spectrum Explorer and the second subband of a HIFI WBS spectrum is displayed in the bottom right Spectrum Explorer.

    Figure 7.1. Selecting one spectrum to fit with the Spectrum Fitter GUI. Here a pixel near the centre of the cube (highlighted with a green box) is displayed in the top left Spectrum Explorer and the second subband of a HIFI WBS spectrum is displayed in the bottom right Spectrum Explorer.

  3. Click on the SFG icon in the Spectrum Explorer button bar: . Alternatively, you can open the SFG from the Dialogue menu you get when you right-click in the Spectrum panel.

  4. A new panel will appear to the right of the spectrum plot: this is the SpectrumFitterGUI panel.

If you try to open the SFG on multiple spectra—or on no spectra—a warning message will appear informing you that have not selected any spectra to fit and that you should do so. Clicking OK will generate a message in the toolbox "Initializing failed, press Reset to retry". Ensure that you have selected a spectrum before reselecting the SFG.

If you do have multiple spectra plotted, you can select (or highlight) the spectrum of interest by clicking on the arrow icon in the Spectrum Explorer button bar () and then clicking on the spectrum, which should then be plotted with a solid line and dot marker symbols. Selecting the SFG will then open it on the selected spectrum. To learn how to select cube spectra from the Spectrum Explorer, read Section 6.7.2, and for other spectra read Section 5.4.2. However, this is not a recommended way of using the Spectrum Explorer, as the plot will later become very busy with more spectra created by the SFG. What you see when the SFG is opened

The SFG opens up in the same space as the tasks parameter form in the Spectrum Toolbox or the Cube Toolbox. There are four panels in this Spectrum Explorer, each with a different purpose.

The SFG accessed via the Spectrum Explorer, and showing a Polynomial fitted to one pixel from a SPIRE cube. The labelled Spectrum Explorer and Fitter GUI panes are described in the text below.

Figure 7.2. The SFG accessed via the Spectrum Explorer, and showing a Polynomial fitted to one pixel from a SPIRE cube. The labelled Spectrum Explorer and Fitter GUI panes are described in the text below.

  • Spectrum panel: The selected spectrum and a weight line are plotted here. Models will be added as you start fitting, and the final fit and its residual will also be displayed here.

  • Data Selection panel: In here a new tab, called FitResult, will appear. It contains selectable rows including the input spectrum and a weight line (click on the coloured button to show or hide them in the plot panel above), and once fitting is started it will also contain the models, the total (or global) model, and the residual after fitting. In each row representing a model the following are also shown: the names of the model parameters, their values and the standard deviation of the fit from the data as well as the integrated flux under the model.

    The data plotted in the Spectrum panel is colour-matched with the coloured button in the Data Selection under the column headed "0", you can plot or remove from the plot by clicking on these buttons. In Figure 7.2 the colours are: the spectrum you chose (dark blue), the total model (dark blue, again, because it is not plotted), the polynomial model (brown—it is the only model and thus the same as the total model), the residual (magenta) and the weights (green).

  • Working area: The working area of the SFG is in the Toolbox panel, on the right. In different tabs you are able to: work with models; assign weights to data; select the fitter engine to be used; export the fitting results and models; and use the multifitter to apply models to multiple sets of data. These are described in the following sections.

  • Preview panel: This panel in the lower right will show a preview of a spectrum in the Data Selection panel when clicking on its row or hovering over a spaxel of a cube. The preview panel is not significant in the use of the SFG. Procedure to fit to a single spectrum

A typical procedure to follow when fitting to a single and relatively uncomplicated spectrum may go like this:

  1. Plot one spectrum in the Spectrum Explorer.

  2. Open the Spectrum Fitter GUI.

  3. Add a model. This is done in the working area. See Figure 7.3 (you may need to resize the panel to see it all):

    The working area of the SFG.

    Figure 7.3. The working area of the SFG.

    To add new models, you will click on addModel. Then select the type of model you want from the drop-down menu. An initial estimate will be applied to the data and at the same time the model drawn on the spectrum plot. You can add as many models as you wish. For simple spectra containing one spectral line and a simple continuum the result you see upon initialisation (this happens as soon as a model is selected) is usually pretty good, and when adding new models the SFG usually makes a decent guess at the parameters.

  4. Adjust the model(s). You can change the model parameters in the area assigned to each model in the Models tab. For the polynomial model you will need to select and Update the order. For many of the parameters for the other models you can use the mouse to make selections in the Spectrum Panel, see Section 7.6 and Section 7.11 for instructions. If, upon initialisation, your chosen model could not make a first guess at the parameters, you can set the manually in this way. There are other steps you can take to optimise a model fit:

    • In the Weights tab behind the Models tab you can set weighted regions that are taken into account when fitting is done so that you can give more significance to a strong feature or less significance to very noisy regions. You can also mask out regions so they are not included in the fitting (e.g. mask out spectral lines so that a polynomial can fit a continuum well) by setting weights. See Section 7.8.

    • In that same Weights tab you can also ask that the weights in the spectrum (i.e. that are already present in the data) are included in the fitting. However, note that you cannot use data weights and also set weighted regions: the data weights are never taken into account if weighted regions are set.

    • You can add more models and you can also choose to not use them in a fit, or to delete them, see Section 7.14.

    • In the Engine tab behind the Weights tab you can change the fitter engine to use. The default usually produces a good result but see Section 7.26.

  5. Press Accept at the bottom on the Working Area to make a global fit. This fit applies all the models you have defined to the original spectrum (the one you displayed when opening the SFG). The fitter always works on the original data (not to e.g. the residual of a previous fit), see Section 7.12.

  6. Inspect the fitting results in the Spectrum Explorer. The individual models, the total model and the residual will all be found in the FitResult tab of the Data Selection Panel, you can plot/remove them by clicking on their coloured buttons. You can also find this FitResult variable in Variables (this product contains the details of the fit), see Section 7.13.

  7. Save (export) the fit results. Using the Export tab in the working area, you can save a script of your actions and save the original data, the residual and the models, see Section 7.16 and Section 7.17. (The residual can be run through the SFG again for additional fitting.)

    Upon fitting, a set of results is immediately produced and sent to the Variables pane: a single context called SFGResultsContext containing the: model spectrum(a), total spectrum, residual spectrum as individual products. The class of these spectra is the same as the class of the input, meaning that if you fit a cube then these outputs are also cubes, but with only the selected spaxel of each cube having a spectrum in it. The export tab is useful if you want to save to disk in the same or a different format. A product called "FitResult" is also produced: this contains the fitting results as shown in the FitResult tab you can see in the Data Selection panel of the Spectrum Explorer.

  8. You may wish to work on the residual with other data processing tools. When you export the fit results you have an option to save the residual or the models in the same format as the original data. Taking this option will add a product called (by default) SFGResidualDataset to the SFGResultsContext. This contains your residual in some spectral format. You can click on this variable name as listed in the SFGResultsContext product (e.g. in the Product Viewer or Outline pane) to display it in the Spectrum Explorer and you can drag it into the Variables view to make a new variable to work on. From the Variables you can also export as FITS. Procedure to fit a cube or multi-spectrum dataset

[Warning] Warning

You should make a variable of the dataset you wish to fit rather than opening a spectrum/cube directly from the Observation Context in the Observation Viewer. If you do not work from a variable (i.e. if you do not actually extract out the spectrum or cube from the Observation Context), the SpectrumFitter will not know where to look for the rest of the spectra you fit to and will hang. No warning message is given in this circumstance.

The procedure to fit multi-spectrum datasets, such as cubes, follows on from the fitting to a single spectrum.

  1. First fit a single spectrum and get the model-set that you like, as described above. Alternatively, load a model XML file from a previous fitting, see Section 7.17 and Section 7.20.

  2. Then go to the MultiFit tab in the Working area. You can choose to save the parameters of the multi-fit (upon execution) to an ASCII file on disk from here. Fit the entire dataset with the model(s) by clicking the Accept button at the bottom of the panel.

    If any spaxels/pixels could not be fit, a message will appear informing you of the number of failed fits. Any spaxel/pixels that are entirely NaNs (e.g. the edge spaxels of mosaic cubes) will result in a failed fit.

  3. Inspect the fitting results. In addition to any ASCII file you produced you will see, in the Variables View, the following new products: MultiFit_Residual, MultiFit_TotModel, MultiFit_M1[M2, M3...], which are all SpectralSimpleCubes with the indicated spectral results in them; M1, M2, M3, etc are the individual models you defined).

    Also created are: MultiFit_Parms, which is a product containing TableDatasets that hold your results, and MultiFit_ParameterCube, which contains the fitting results as a ParameterCube (see Section 7.28 to learn more about this).

    The MultiFit_Parms output.

    Figure 7.4. The MultiFit_Parms output.

If you want to tie models together (combo model) then you need to do spectrum fitting on the command line, using the Spectrum Fitter. See Section 7.23.

Bear in mind that when working with a cube, it is possible that the spectra differ substantially from spaxel/pixel to spaxel/pixel, and this could affect the accuracy of the results. The Spectrum Fitter GUI in multi-fit mode is an automatic fitter, and so cannot account for sharp and large changes to the spectra across the cube.

7.1.2. Using the Spectrum Fitter (command-line fitting): an overview

The best way to learn how to script fitting is to follow the Worked Examples in the following sections, which each contain a full script of the fitting done after a GUI-based description. This includes fitting single spectra and multi-spectral datasets, and includes creating images from the results of fitting on cubes. The remainder of this chapter also contains code snippets in each section to show how to perform the actions described in the command line, again after the GUI-based description.

Whether you use the graphical interface or a script to fit spectra is primarily down to your own preference. Scripting is, of course, very efficient and can be the best way to deal with large amounts of data. The fitting tools require initial estimates before making a fit, which can be difficult to do by eye, and—as for all fitting tools—if the initial estimate supplied is not a good one the fitting can be very bad or even fail. An advantage of using the GUI is that it makes the initial estimates for you based on the original data. If you prefer to script it can be helpful to make initial fits to a good-quality spectrum in your dataset using the GUI, before making a script of that fit and modifying it for your needs.

In the command line, the format in which data is passed to the SpectrumFitter depends on the type of data (spectrum or spectral cube) being used.

The SpectrumFitter class accepts all the main Herschel spectral types: SpectralSegments, Spectrum1d, Spectrum2d and spectral cubes. Whenever you start the Spectrum Fitter, a plot window is opened by default, unless you set an additional parameter to False:

sf = SpectrumFitter(mySpectrum)  # Plot window created
sf = SpectrumFitter(mySpectrum, False)  # Plot window not created

Example 7.1. Creating a new instance of the Spectrum Fitter with and without a plot window.

If your SpectrumDataset contains multiple spectra, you can specify the spectrum to fit to by giving the pointSpectrum and segment number:

sf = SpectrumFitter(mySpectrumContainer, spectrum, segment)

Example 7.2. Creating a Spectrum Fitter specifying the particular spectra by segment number(s).

Remember that pointSpectra, corresponding to the dataset number, are numbered starting from 0, while segments are numbered starting from 1.

If your input is a cube, you can instead indicate the spaxel row and column coordinate of the spectrum you want to select for fitting:

sf = SpectrumFitter(myCube, column, row)

Example 7.3. Creating a Spectrum Fitter specifying the spectra by cube coordinates.

[Warning] Coordinates in cubes

In the cube image: At the bottom left of the cube display you will see indicated the (row, column) of the spaxel/pixel under the mouse. If this says, e.g (5,1), then your mouse will be 6 spaxels/pixels high along the vertical axis (the counting starts at 0) and 2 spaxels/pixels along the horizontal axis. This is the reverse of the usual convention for Astronomical images.

The Spectrum Fitter follows normal Astronomer convention and of (column, row) and it calls these (X,Y). Therefore, if you see (5, 1) as you hover your mouse over your cube image, you should use the coordinates (1, 5) in the Spectrum Fitter.

You can also select one spectrum from a cube using spectrum and spectrum segment indices with the following syntax, including setting the last parameter to True:

sf = SpectrumFitter(myCube, spectrum, segment, True, True)

Example 7.4. Creating a new instance of the Spectrum Fitter specifying both segment number and the display of the plot window.

For cubes, the segment value is always 1. The first True parameter refers to the creation of a plot window.

Instead of writing a script from scratch, you can use the Save as script option in the Export tab of the SFG to save a script that will recreate the last global fit performed (Section 7.1: and remember to press Accept to actually save the script). You can use this script as a template to create your own for other data. Hence, beginning with the SFG before using the Spectrum Fitter on the command line is a good idea.

Note that the script saved by the SFG starts at the point where the data to be fit is loaded into the Fitter (depending on your previous actions, the data passed to the Fitter may be a FitResult), you must script any earlier actions yourself; use the History tab of the Console to see the commands you ran before. Note also that the script is not completely tailored to what you just did before you saved it—some parts of the script never change.

Two useful extra things that can be done with the command-line fitting are:

  • The combo model, with which you can tie models parameters to each other. For example, two Gaussians could be constrained to have the same FWHM as each other. See Section 7.23. This is unique to command-line fitting.

  • Limiting fit parameters, where you can limit a fit to a parameter to fall within a given range. See Section 7.9.

7.1.3. Fitting tips

  • When fitting a Polynomial to a baseline and also fitting a spectral line you may have more success if you use an initialisation range for the Polynomial, see Section 7.6, rather than trying to use weights. The reason for this is that weights are applied to all models, including the one you use for the line.

  • When fitting a baseline/continuum and line profiles, you could try first fitting and subtracting or dividing the baseline/continuum, and then fitting the lines only. When fitting to a cube with the Spectrum Fitter, a residual cube is created automatically; if you wish to divide by the continuum instead of subtract you can use the Spectrum Toolbox "divide" task with the original-cube and the fit-cube as inputs (see Chapter 5 and Chapter 6 for more information).

  • If a fit fails, try allowing any parameters you have fixed to float. This should help you to identify where the fit is failing. If the fitting still fails then try temporarily not using one model at at time until the data can be fit, you can then try tweaking the parameters of the problematic model until a fit is attained.

    When fitting to spectral cubes, the MultiFit_Parms output from the multi-fitter will indicate the spaxel/pixel coordinates of the fits that failed, so you can check on those (and not forgetting the swap in coordinates between the SFG and the display, mentioned in the Warning above).

  • When automatically fitting to complicated or varying spectral profile in a dataset try running the multifitter several times and constraining the fitter to a specific feature on each attempt. For example, in the case of an emission line that changes to an absorption across a field, fit to the emission first by setting the limits of the amplitude to be positive. Then fit to the absorption by limiting the amplitude to be negative, see Section 7.9. (Then you may have to merge the results.)

  • For information on the fitting routines upon which Spectrum Fitter is built, see the Scripting Guide: Section 5.8, “Fitting data”. See also the fitting reference documentation.

  • When chosing the reference spectrum to fit first before multifitting on a cube, try selecting one of the brightest ones rather than an average one; it can produce better overall results.