1.10. Saving data (products and observations) to disk

Prerequisites. You have made changes on an observation, for instance by reprocessing data with a pipeline script. You now want to save your reprocessed data. As you will see, the procedure is the same whether you want to save an entire observation or a smaller piece of data that is still a product.

GUI method

To save an observation or product to disk, follow these steps:

  1. Right click on the data variable name in the Variables view and choose Send toLocal pool .

    The Save Products Tool opens in the Editor view (see Figure 1.27). Your observation variable name appears in the Products column.

  2. Assign one or more tags to the observation you want to save. To assign a tag to a product, double click the corresponding cell in the Tags column, write the tag and press Enter .

    Tags are keywords or phrases you can associate to a product, to better describe and remember its contents. For example, you could assign to a product the tag "To be completed" to remember that you have not finished processing it. When defining tags, you are free to use the keywords and phrases that work best for you. Tags are especially useful to recognise different versions of the same observation, which by definition have the same obsid.

  3. Select a pool from the Select Pool drop-down list, or write a pool name. If a pool with that name does not exist, it is created.

  4. Make sure that the observations and products you want to save are selected and click Save to store them into the pool.

    When you press Save there is no success or failure message. You can check the Console view, where the corresponding command has been echoed, to make sure that the data have been saved correctly.

The Save Products tool.

Figure 1.27. The Save Products tool.


Command-line method

To save products (including observations) to disk, use the saveProduct command. You must specify the variable you want to save, the pool you want to save it to and the tag you want to associate to it. Adding a tag is not compulsory but strongly recommended.

Tags are keywords or phrases you can associate to a product, to better describe and remember its contents. For example, you could assign to a product the tag "To be completed" to remember that you have not finished processing it. When defining tags, you are free to use the keywords and phrases that work best for you. Tags are especially useful to recognise different versions of the same observation, which by definition have the same obsid.

bg("saveProduct(product=myObs, pool='myPool', tag='My reprocessed data')")

Example 1.25. Save an observation to disk specifying the pool name and a tag.


If a pool with the name you specify does not exist, it is created.

[Note] Note

If you use the saveObservation command, note these differences with respect to saveProduct :

  • saveObservation does not support tags.

  • saveObservation does not store the calibration tree when used with observations.

Where are my data?  Your data are now in a local pool , typically in a subdirectory of .hcss/lstore in your home directory. Do not touch the files directly, but keep working on them from HIPE. If you modify the files outside HIPE, you may corrupt the pool structure.

Where do I go from here? Now that your data are saved, you have these possibilities: