This section describes where HIPE keeps Herschel data on your disk. Note that you do not need to know where a certain data product or observation is stored, because you can use HIPE to search through all the Herschel data stored on your computer. See Section 1.7 for more information.
Data from the Herschel Science Archive. Observations you download from the Herschel Science Archive are stored in tar files. Once you unpack an observation and load it into HIPE, the contents are indexed and referenced by a special data pool called MyHSA .
If you get your data from the Herschel Science Archive via the
command, with the
options, the data are stored directly in the MyHSA pool. See
You can set the directory where the MyHSA pool is stored by choosing
left-hand side list. The repository is kept by default in a
directory inside the
Note that MyHSA contains only unmodified data downloaded from the Archive. It is like your local copy of the Herschel Science Archive with the observations of interest to you. Reprocessed data are stored elsewhere.
Do not change the contents of the
Reprocessed observations. Observations and data products you save after reprocessing are stored into pools , which are grouped into storages . A pool is a repository you can use to save, load and search observations and data products. A storage groups one or more pools. Every pool must be registered to a storage. A common situation is a storage containing a single pool. If a storage contains more than one pool, only the first registered pool is accessible for writing.
Figure 1.2. Pools and storages. All pools must be registered to a storage. A pool can be registered to more than one storage. A common situation is a storage containing a single pool.
There are many types of pools, for handling local and remote data.
, is probably the one you will use most often. As the
name suggests, this pool is held locally on your system, usually in a
directory under your
home directory. Although products are stored as FITS files, you should
the graphical tools provided by HIPE and described in this chapter (see
) rather than manipulating the files
Local pools are also called local stores for historical reasons, but they are pools , not storages .
See Section 1.3.1 for more information on how to manage pools and storages on your system.
This chapter contains all you need to know to use pools and storages in most situations. If you want to delve deeper and learn how to manage pools and storages from the command line, see the Scripting Guide : Chapter 7.
Storages and pools are the two tools with which you can store and retrieve data on your computer (see Section 1.3 for more details). With the Storages & Pools panel in the Preferences dialogue window (see Figure 1.3 ) you can create, delete and associate storages and pools.
To open this window, choose or press → Alt+Enter , then go to Data Access > Storages & Pools .
Within this window you can accomplish the following tasks:
Creating and deleting pools. In the Pools area, click , enter the pool name and click . You can choose what kind of pool to create from the drop-down list. If you are unsure, or if you just want to store data on your local disk, leave the Local Pool default.
For pools fetching remote data (Versant Database Pool and HTTP Pool) you can cache data locally by ticking the Use cache checkbox.
You can set other properties of the new pool (for instance, the directory of a Local Pool) in the parameters table.
Click when you are done.
To delete a pool, select it from the list and click .
To refresh the list of pools, click . This is useful if, for instance, you copy a pool into your local pool directory while HIPE is running.
Creating and deleting storages. Use the and buttons in the Storages area, in the same way as with pools.
Registering pools to storages. Select a pool in the Pools area and drag it to a storage in the Storages area.
By ticking the Add pools/storages from variables automatically checkboxes, any new pools or storages you create elsewhere in HIPE (for instance, from the command line) will appear automatically in this dialogue window.
Note that this does not work the other way around: pools and storages created in this dialogue window will not appear as variables in the Variables view.
Do not remove pools from disk while you have variables in HIPE referring to data in those pools. Since HIPE does not always keep all the contents of a variable in memory, you may not be able to save the variable contents to disk again.