Known issues in HIPE 12

SmoothSpectrumTask and ResampleSpectrumTask do not propagate weights correctly

Ticket number: HCSS-15583 (Won't fix)

Versions affected: All versions of HIPE since 8.0

Description: If any of the tasks is applied to a spectrum that contains a column of errors instead of weights, the task converts the errors to weights, propagates the weights and repopulates the error column by converting the propagated weights back to errors. This gives an error column that has not been propagated correctly for the operation applied.

Workaround: Take into account that the output error column should be disregarded in the case of source data that lack a proper error column.

HIPE FITS Writer format could be incorrectly read by other software

Ticket number: HCSS-19248

Versions affected: HIPE 12.x

Description: HIPE makes use of OGIP, a non-standard convention to include long values in FITS fields. The OGIP convention writes an & at the end of a line (for example, describing a column name) that would be considered valid for all FITS compliant readers (it contains less than 68 characters) followed by the special keyword CONTINUE '' / & and completing the long value with a COMMENT Rest of value line. For example:

TTYPE1  = 'designation&'
CONTINUE  '' / &
COMMENT Original element type: char, Empty values appear as null

Many FITS readers ignore this convention, so the special character & is read and included as the final character of the value. Others (IDL) do the same but, additionally, consider & an invalid character and it is replaced by an underscore _.

Workaround: None at this time.

Old syntax using HIFI class PolarPair gives wrong results

Ticket number: none (API change)

Versions affected: HIPE 12.0

Description: PolarPair (the class) use is now discouraged for users and polarPair (the task, note capitalisation) has been modified in order to allow it to be used as a part of the interactive HIFI pipeline and to bring its syntax in-line with other tasks in the Spectral Toolbox. If the old syntax is used, e.g.,

pp = PolarPair(wbs_h, wbs_v)
av = pp.avg()

incorrect results are obtained.

This problem did not affect previous HIPE versions. A warning will be written to the console in HIPE 13 if the old syntax is used.

Workaround: Use the new syntax running the task:

av=polarPair(ds1=wbs_h, ds2=wbs_v)

Limitations of the task for updating scripts that use the old syntax to access return parameters

Ticket number: HCSS-19080 (Won't fix)

Versions affected: HIPE 12.

• Task calls spanning multiple lines (with \ separation) are not detected. Some SPIRE scripts use multiple line task call scripts.
• The dialog box should show all lines which are going to be changed, but only the first line is shown there.
• Some non-standard task calls, like using two = signs to put the result into two variables at once, are not detected.

Old Java 6 entries in the dynamic library environment variable cause HIPE 12 to crash on OS X

Ticket number: HCSS-18891 (Third party; Won't fix)

Versions affected: HIPE 12.

Error message:

$ ./hcss.dp.pacs-12.0.1553/bin/hipe
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x000000010794f647, pid=49119, tid=6403
#
# JRE version: Java(TM) SE Runtime Environment (7.0_45-b18) (build 1.7.0_45-b18)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (24.45-b08 mixed mode bsd-amd64 compressed oops)
# Problematic frame:
# C  [libjvmlinkage.dylib+0x3647]  JVM_GetClassCPEntriesCount+0x17
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /Users/USER/hcss/hcss.dp.pacs-12.0.2340/bin/hs_err_pid49119.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.sun.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

Root cause: The cause is that the shared library path has references to the old Java 6 installation.

$ env | grep DYLD_LIBRARY_PATH
DYLD_LIBRARY_PATH=:/Applications/MATLAB/MATLAB_Compiler_Runtime/v716/runtime/maci64:
/Applications/MATLAB/MATLAB_Compiler_Runtime/v716/sys/os/maci64:
/Applications/MATLAB/MATLAB_Compiler_Runtime/v716/bin/maci64:
/System/Library/Frameworks/JavaVM.framework/JavaVM:
/System/Library/Frameworks/JavaVM.framework/Libraries

Workaround: The current workaround is to remove the entries that contain JavaVM from this environment variable. To do that, modify or create the .bash_profile configuration file of the terminal and add the environment variable in an export statement without the corresponding entries. That should persist the changes.

LocalStoreFactory.getStore(pool_name).rebuildIndex()

store.writablePool.rebuildIndex()

herschel.ia.task.TaskException: Error processing getObservation task: Unknown format version: -11

1. Open HIPE 9 and import the exported variables with File --> Session --> Import.
2. Set Save variables on exit in the Preferences dialogue window, under General --> Startup & Shutdown.
3. Exit HIPE 9.
4. Open HIPE 10. The variables shall be restored in the Variables view.
5. Export the variables with the new format of HIPE 10, with File --> Session --> Export.

1. Go to the .hcss directory in your open directory. Open the hipe.props file, or create it if it does not exist.
2. Add the following line to the file:
   java.vm.options = -XX:CompileCommand=exclude,Jama/LUDecomposition,<init>
3. Restart HIPE.

• Solved known issue with step by step execution. The known issue found in HIPE 7.0 with step by step execution has been solved. You can revert to the old behaviour by choosing Edit --> Preferences, going to Editors & Viewers --> Jython Editor and unticking the Use improved step by step execution checkbox.

• Choose Edit --> Preferences and go to Data Access --> Storages & Pools. Select a pool in the Pools pane and click the Clear cache button. Note that this button will not be available if there are no pools listed in the Pools pane.

1. In the Product Browser view, select a query source and search for an obsid that you know not to exist.
2. In the Variables view, variables QUERY_RESULT and p appear.
3. The QUERY_RESULT variable is correctly empty, as you can verify by issuing the command print QUERY_RESULT in the Console view.
4. The p variable should not have been returned. You can safely delete it.

• An error like the following can sometimes appear in the command line window from which you are starting HIPE. This error is harmless and can be ignored. May also affect older HIPE versions. Fixed in HIPE 6.0.

• The following scripts in the Pipeline --> PACS menu in HIPE are known to crash: extended_pipeline.py, L1_smallSource.py, L2_scanMap.py. Other scripts may crash or not be applicable because they depend on the results of other non-working scripts. Many scripts are not adequately commented.

Known issues in HIPE 11

Reading observations stored as tarball archives that contain new data types does not work in HIPE 11

Versions affected: Potentially all versions older than HIPE 12.

Error message: While loading an observation from a tarball archive (.TAR file), HIPE returns the following error:

WARNING: Could not open /home/user/1342219630-herschel.ia.obs.ObservationContext-580157.xml: 
herschel.ia.task.TaskException: Error processing getObservation task: 
Class definition not found for urn urn:hsa:herschel.spire.ia.dataset.PhotApertureEfficiency:0: herschel.spire.ia.dataset.PhotApertureEfficiency

Root cause: The root cause is that the class herschel.spire.ia.dataset.PhotApertureEfficiency (among others) is new for data processed with HCSS 12 and doesn't exist in previous versions of HIPE. Previous versions cannot open tarballs containing these types of data.

Workaround: The recommended workaround is updating HIPE to version 12.1.

If you cannot upgrade HIPE, please contact the Helpdesk.

Migrating pools across incompatible versions of HIPE

HIPE 10 developer versions (builds between 2069 and 2674) and HIPE 11 (all builds) pools are incompatible with any other version of HIPE (including all public releases of HIPE 10).

In order to use these existing pools with an incompatible version, you should rebuild the index of the pool. Given that it is not guaranteed that newer data should work with older versions of the software, this guide will focus only on making the pools workable in the following cases:

• Case 1: Using pools created/modified in a HIPE 10 version prior to 2069 with a HIPE 10 version between 2069 and 2674.
• Case 2: Using pools created/modified in a HIPE 10 version between 2069 and 2674 with a HIPE 10 version between after 2674.
• Case 3: Using pools created/modified in a HIPE 10 version other than the 2069-2674 range with HIPE 11.
• Case 4: Using pools with an index rebuilt for a HIPE 10 version other than the 2069-2674 range with HIPE 11.

Prerequisite:

A pool created with a HIPE 10 version prior to 2069.

Case 1:

Pool you want to open: A pool like the one detailed in the prerequisite.

Error message: There is no error displayed and the pool is completely usable in HIPE 10 versions 2069-2674.

Case 2:

Pool you want to open: A pool that is like the prerequisite or has been processed in a scenario like CASE 1.

Error message: Querying the pool using the Product Browser returns nothing (no error message). Opening the observation from the Console using getObservation returns the following error message:

herschel.ia.task.TaskException: Error processing getObservation task: Index Version not compatible. Expected : 4 Existing: 6. 
Pool <pool_name> requires upgrading before you can use it with this software. In order to do so, you need to run pool_name.rebuildIndex() to upgrade. 
Depending on the size of the pool this process can take a long time, please be patient! 
More information can be found in the Data Analysis Guide, section 1.2.2.1 (Update of index format for local stores).

Notes about the error message:

• pool_name is the name of the pool you are trying to load.
• pool_name.rebuildIndex() is not the proper command to rebuild the pool. See below for the appropriate commands.
• The section of the Data Analysis Guide referenced is not correct. When ready, a copy of this procedure will be available in the section 1.11 (Migrating pools across incompatible versions of HIPE)

Workaround: Any of the following commands will rebuild the index of the pool:

• This ensures we are rebuilding the specified pool using a static method from the class LocalStoreFactory:

LocalStoreFactory.getStore(pool_name).rebuildIndex()

• This rebuilds the pool using a ProductStorage instance named store:

store.writablePool.rebuildIndex()

Case 3:

Pool you want to open: A pool that is like the prerequisite.

Error message: There is no error displayed and the pool is completely usable in HIPE 11.

Case 4:

Pool you want to open: A pool which index was rebuilt following CASE 2 instructions.

Error message: There is no error displayed and the pool is completely usable in HIPE 11.

That's it! You should now be able to use your pool in any of the above cases. If you find you cannot, contact the Helpdesk.

Note that the index files will not be backed up while you run rebuildIndex(), unless you invoke Configuration.setProperty("hcss.ia.pal.pool.lstore.index.backup", "true") first. For more information execute the following command: help('dag:sec-lstore-index-compatibility'). See the HIPE Owner's Guide: Section 4.

Known issues in HIPE 10.3

Problems seeing PACS error maps

This problem, reported in 10.3, but previously present too, is caused by the history dataset which is "hiding" the error. To resolve it, after running photCoverage2Noise task, give these commands:

history = image["History"]

image.remove("History")

image["History"] = history

Where the "image" is the product that you have been working on. This will append the history dataset at the end of the product and "solve" the problem. There will be a fix in place for this issue in HIPE 11.

Known issues in HIPE 10.2

None known in addition to those listed for HIPE 10.1.

Known issues in HIPE 10.1

Problems with the aberration correction for Solar System Objects in HIFI

There is a previously existing issue that has become important in some cases when processing Solar System Object observations in HIPE that affects all Level 2.5 HIFI map products. Ephemeris positions are calculated (and stored in the ra_centre/dec_centre columns) with a spurious correction for stellar aberration. This is being fixed but, until patched, the recommended doOffset approach to translate Level 2 coordinates to a co-moving frame (see the corresponding known issue below on this issue in HIPE 10.0) will lead to coordinate offsets that are off by ~10" on average (depending on the relative velocity between target and Herschel at observation time).

This is being fixed in HIPE 12 and it is expected that a patch will be made for HIPE 11 and, possibly, also patched in later releases of HIPE 10.

Known issues in HIPE 10.0

Can't open plug-ins panel under KDE or Gnome 3

The Plug-ins panel cannot be opened (from the Tools menu) if you're working under Linux and you don't have the Gnome 2 library installed. Gnome 2 is a desktop environment, and typically if you use a different desktop environment (Gnome 3, KDE, Unity, etc), then you will not have this library.

In both cases, the problem can be solved by installing the Gnome 2 library. For example on Ubuntu (or Kubuntu), the command is:

sudo apt-get install libgnome2-0

A patch is planned for this issue in HIPE 10.2.

Clicking on the 'plug-in' button to add the cassis plugin, nothing happens and you see multiple java exceptions

The "Desktop API", which is used to access the browser when you click "Find more.." button in the HIPE plugin dialogue, is not supported on KDE. It "was designed to work on Windows and Gnome". However, you can also open it directly by pointing your browser directly to http://herschel.esac.esa.int/twiki/bin/view/Public/DpHipePlugins, and copying the URL to the respective .jar files for the particular plugin you are interested in and pasting it in the HIPE plugins dialog to install it.

If you would like to have the "Find more.." button working, all you need is to have the Gnome libraries installed.

Some instructions: - To install Gnome 3 libraries on Ubuntu (http://www.filiwiese.com/installing-gnome-on-ubuntu-12-04-precise-pangolin/). Note that if you're using version 11.10 or later, you probably don't need the first command (sudo apt-add-repository ...). I would just skip it and if the last command does not complain about "package not found" or something similar, then all is well. - To install Gnome 2 libraries on Ubuntu: http://askubuntu.com/questions/123690/how-to-install-gnome2-on-ubuntu-11-10

A patch is planned for this issue in HIPE 10.2.

Pools unreadable with Unknown format version: -11 error

If you installed a developer build of HIPE 10 between 10.0.2069 and 10.0.2674 and modified any of your pools you will find those pools are no longer readable in any other HIPE version; when trying to load a pool with getObservation you will get the following error message:

herschel.ia.task.TaskException: Error processing getObservation task: Unknown format version: -11

See the Data Analysis Guide for a recovery procedure. You can download from this page the first and second script mentioned in the procedure. Change the extension from .py.txt to .py after downloading.

The WCS explorer for cubes does not work with PACS and SPIRE cubes

The WCS explorer does not work on cubes for which ImageIndex is used to specify the wavescale axis. This is true for SPIRE and PACS cubes. If crpix3, crval3 and cdelt3 are used instead, the WCS explorer works. To be fixed in HIPE 11.

Issues with reading text files from SExtractor or Topcat

While reading files from SExtractor or Topcat you may encounter the following issues:

• Some SExtractor headers are not read successfully, due to an issue with the column position of units in the file header. Workaround: read the file with the tableType parameter set to SPACES. You can then fill any missing information by hand.
• Some Topcat files are not read successfully, due to an issue with quoted cells with spaces. Workaround: export the data from Topcat in FITS format, then read the FITS file back into HIPE. For information about reading FITS files into HIPE, see the Data Analysis Guide.

Format change for exported variables

The file format of variables exported with File --> Session --> Export has changed in HIPE 10. This means that HIPE 10 will not be able to read session variables exported with HIPE 9, and vice-versa.
If you exported variables with HIPE 9 or a previous version, follow this workaround to read them in HIPE 10:

1. Open HIPE 9 and import the exported variables with File --> Session --> Import.
2. Set Save variables on exit in the Preferences dialogue window, under General --> Startup & Shutdown.
3. Exit HIPE 9.
4. Open HIPE 10. The variables shall be restored in the Variables view.
5. Export the variables with the new format of HIPE 10, with File --> Session --> Export.

HIPE cannot send data to an external application via SAMP

When you try to send data to an external application via the SAMP Virtual Observatory protocol, the following window may appear:

The reason is that HIPE tries to send data before the external application has initialised properly. This issue seems to occur more frequently with VOSpec.

The solution is to send the data again without closing the external application.

Separate, undocked instances Cube Toolbox talk to each other

If you have two or more separate Spectrum Explorer GUIs open and at least one is undocked, when using its Cube Toolbox the separate SE instances talk to each other--spectra or new cubes you create in one instance will also appear in the other one.

New PACS calibration version 45 should not be used with HIPE 10

In HIPE 10 a new spectroscopy telescope background calibration was introduced. This update is not compatible with the code in HIPE 9.

You can alway check the calibration version that you are using with the following command:

print getCalTree().version

If you use HIPE 10, you should update to version 45 of the calibration tree to make use of several new calibration files.

Generating a co-moving frame for HIFI mapping observations of solar system objects

There is a task in HIPE that allows you to transform level-2 coordinates into a coordinate system co-moving with the target ephemeris. That task is called doOffset,

http://herschel.esac.esa.int/hcss-doc-10.0/index.jsp#hifi_um:hdrg_offsetposn_use

A typical code snippet would be:

comet = getObservation(1342227173, useHsa=True)

htp=comet.refs["level2"].product.refs["WBS-H-USB"].product # extract USB, pol H

htp2=doOffset(htp=htp, overwrite=0, relative=1) # make coordinates relative to ephemeris

HiClassTask()(product=htp2, fileName='cometHUSB_relativeCoords.fits') # export in CLASS-readable format

However, the target ephemeris inside HIPE doesn't get updated. I.e., doOffset uses the ephemeris current as of observation uplink to the spacecraft. Work is in progress to automatically update target ephemerids. If your target ephemeris was very uncertain at observation time (and if it has improved since), then you may be better off using a manual approach of downloading the Horizons ephemeris for each target.

Printing a HIPE script to a file in Snowleopard sends the job to a printer instead

When a user tries to print a script to a file on disk, HIPE sends it to the printer instead. A patch will be issued for this problem in HIPE 11 and a definitive fix included in HIPE 12.

Known issues in HIPE 9.2

New PACS calibration version 45 should not be used with HIPE 9

In HIPE 10 a new spectroscopy telescope background calibration was introduced. This update is not compatible with the code in HIPE 9.

If you use the background normalisation ipipe script for chopped spectroscopy in HIPE 9, you should not update the PACS calibration tree to version 45. You should keep using the calibration set 41 and cancel the update when the automatic calibration updater in HIPE asks to install set 45.

If you have updated to set 45 and want to use the background normalisation in HIPE 9, you can switch back to version 41 in two ways:

• With the command getCalTree(version=41).
• By removing the file PCalBase_TimeDependency_FM_v45.fits from the directory ~/.hcss/data/pcal-community.

You can alway check the calibration version that you are using with the following command:

print getCalTree().version

If you use HIPE 10, you should update to version 45 of the calibration tree to make use of several new calibration files.

Issues with computePVMap task

The computePVMap task has the following issues:

• There are remaining aesthetic issues regarding the pvMap aspect ratio modification in the task.
• There is a known error when extracting a pvMap with row < col (velocity dimension < spatial dimension). Workaround: set the correctAspectRatio parameter to False (in the graphical interface, untick the corresponding checkbox).

Known issues in HIPE 9.1

None known in addition to those listed for HIPE 9.0.

Known issues in HIPE 9.0

Performance issues of convolution task with NaN values

The performance of the imageConvolution task is severely degraded when NaN values are found in input data. HIPE 8.3 is not affected by this problem. Fixed in HIPE 9.1.

Performance issues with very large images

Images larger than about 100 MB in FITS format can still take over a minute to load. During such time HIPE is unresponsive. Image analysis operations, such as cropping and adding contours, also take a long time on large images.

Problem with PACS background normalization script

The ChopNodBackgroundNormalizationRange interactive pipeline script (Pipelines --> PACS --> Spectrometer --> Chopper large range scan SED --> Background Normalization) is missing a definition of the variable target. After loading the observation, assuming it is stored in a variable called obs, you can define the missing target variable as follows:

target = obs.meta["object"].value.replace(" ","_")

PACS task specFlatFieldLine problem with NaN values

The specFlatFieldLine task cannot handle NaNs in the fluxes of PacsCube products. This affected one observation as far as we tested. If you encounter this problem then you can mask the NaNs by running the function shown below. Copy and paste the function code in a new Jython script window in the Editor view of HIPE, then click the green double arrow icon in the HIPE toolbar to run the script. After that you can call the function from the Console view, passing the variable contaning your cube as argument: addInvalidMask(myCube).

def addInvalidMask(slicedCube):
    for i in range(0,slicedCube.numberOfSlices):
       cube = slicedCube.get(i)
       invalid = IS_NAN(cube.flux)
       if not cube.containsMask("INVALID"):
          cube.addMask("INVALID","Invalid signal values")
       cube.setMask("INVALID",invalid)
       slicedCube.replace(i,cube)

HIPE hangs when pressing the activate/deactivate button in the mask panel of the Spectrum Explorer PACS extensions

This button is enabled only when you change the state of one or more masks. Fixed in HIPE 9.1.

<-- PACS-4626 -->

Image analysis tasks start only from Applicable section

Image analysis tasks will open only from the Applicable folder of the Tasks view. These tasks will not open when selected from the All or By Category folders. Note that you must select an image variable in the Variables view for the Applicable folder to be filled with the available tasks. To be fixed in HIPE 10. The problem affects the following tasks:

• annularSkyAperturePhotometry
• circleHistogram
• crop
• ellipseHistogram
• fixedSkyAperturePhotometry
• polygonHistogram
• profile
• rectangleHistogram
• rectangularSkyAperturePhotometry
• sourceFitting

PV diagrams created with computePVMap on HIFI cubes cannot be easily viewed

HIFI cubes have in general a small number of rows and columns, while the spectral dimension is rather large (thousands of channels). The PV map is very narrow in the x direction and really wide in the y-direction, so that it does not fit in the Spectrum panel. When displaying the y-axis values, at its smaller zoom, only a small number of channels can be seen. Workaround: open the variable created from the variables pane and use the full Display/Image Viewer.

<-- use the old Cube Spectrum Analysis Toolbox, still available in HIPE 9. Legacy documentation is available in Appendix C of the Data Analysis Guide. For position-velocity diagrams, see subsection C.6.8.1.-->

HIPE crash linked to JAMA (Java Matrix) library

HIPE may crash without warning when executing parts of the bundled JAMA library. If you experience a sudden crash when running a script, this may be the cause, even if there are no obvious references to matrices in the script. Take the following steps to prevent the crash from happening:

1. Go to the .hcss directory in your open directory. Open the hipe.props file, or create it if it does not exist.
2. Add the following line to the file:
   java.vm.options = -XX:CompileCommand=exclude,Jama/LUDecomposition,<init>
3. Restart HIPE.

Problem with transpose task

In some cases the transpose task does not convert correctly the WCS of the input image. This problem occurs only when the WCS of the image defines the rotation angle (CROTA2). When the transformation matrix (PCx_y or CDx_y) is used, the problem does not occur. To be fixed in HIPE 10.

Velocity map creation in Legacy Cube Spectrum Analysis Toobox (CSAT) fails

The creation of velocity maps with the Legacy CSAT fails; the task will hang without an error message. CSAT is no longer supported and the new cube toolbox, which uses an improved algorithm for velocity map making, should be used instead. See 6.8.3 in the Herschel Data Analysis Guide for information about using the new cube toolbox.

Flag set in HIFI's flagTool task do not appear to be used by the fitBaseline task

This is a display issue only, the flags set by flagTool are, in fact, used.

PACS specific: convertL1ToScanam fails

convertL1ToScanam fails because of a missing import for the calibration tree, when assignRaDec is set to True. The work around solution is :

calTree = getCalTree(time=frames.startDate)
frames=photAssignRaDec(frames,calTree=calTree)
convertL1ToScanam(frames,assignRaDec=False)

Fixed in HIPE 9.2.

<-- ANALYTICS CODE - DO NOT EDIT -->

<-- Google Analytics script BEGIN -->

<-- Google Analytics script END -->

<-- END OF ANALYTICS CODE -->

Known issues in HIPE 8.3

None known in addition to those listed for HIPE 8.2.

Known issues in HIPE 8.2

• Task createRgbImage puts wrong WCS in the output. Instead of using the WCS provided by the Wcs input parameter, this task uses the WCS of one of the input images.

• PACS photometer scan map processing : Due to the new scheme of filtering turnaround manoeuvres between scan legs with the task filterOnScanSpeed task (masking frames in the 'ScanSpeedMask' mask, instead of removing the frames), photProject will project maps including the slew to target, hence exploding the memory in Hipe and leading to huge map sizes, because the photProject WCS doesn't take into account this new mask. A workaround solution is to run the filterSlew task previously, at the expense of possibly loosing a few frames on the first scan leg or alternatively run photProject in the following way:

ScanSpeedMask = frames.getMask('ScanSpeedMask')[0,0,:] 
indexSpeed    = ScanSpeedMask.where(ScanSpeedMask==False)
map          = photProject(frames.select(indexSpeed), calTree=calTree,calibration=True)

Known issues in HIPE 8.1

None known in addition to those listed for HIPE 8.0.

Known issues in HIPE 8.0

• Problem with transpose task. In some cases the transpose task does not convert correctly the WCS of the input image. This problem occurs only when the WCS of the image defines the rotation angle (CROTA2). When the transformation matrix (PCx_y or CDx_y) is used, the problem does not occur.

• Problem with NaN values with imageConvolution task. Starting from HIPE 7, the imageConvolution task works with images containing NaNs too. However, the results when NaNs are present (either in the map or around the coverage area) are not reliable. A fixed version of the task is scheduled for HIPE 9.

• Problem with correction of MADmap point-source artifacts. The PACS task photCorrMadmapArtifacts, used to correct point-source artifacts in MADmap produced Level 2.5 products, only works when the initial time-ordered data product (TOD) is created with scale factor of 1. All other values of scale factor may produce incorrect results. A fix, possibly distributed as a plug-in, will be available later.

• Problem with units in the Cube Spectrum Analysis Toolbox. This tool returns the wrong units of measurement with extraction plots and integrated maps. This issue has been detected with PACS data and is under investigation. Previous HIPE versions are not affected.

• User pool names (pool ID) restrictions: Old pools that do not follow the restrictions on the allowed names cannot be read in HIPE (eg a dot "." in the name is such a case). The user can safely rename the folder (found in the local store) following the rules: allowed characters in name: alphanumeric, dash ("-") and underscore ("_"), the name cannot start with a dash ("-") and cannot have "urn" as name. The pool ID is case-sensitive and must be with less than 255 characters.

• High memory usage with Mac OS 10.6 Snow Leopard. There are reports of HIPE 8.0.1 using all available memory and freezing while reprocessing observations under Mac OS 10.6 Snow Leopard. This problem does not seem to appear on other platforms, including other Mac OS versions. The problem is being investigated.

• Velocity Maps in the Cube Spectrum Analysis Toolbox. Fitting of faint lines (less than 2 sigma) or non-gaussian lines can result in incorrect velocities in the velocity map. It is advised to fit the velocity in a few positions manually in the spectrum explorer and verify the consistency with the calculated velocities at these positions in the velocity map in the cube explorer.

• Upgrade to Jython 2.5. Starting from version 8.0, HIPE will include a new version of Jython. You may have to adapt your scripts for them to work with the new version. See this page for the details.

• Links in the User's Reference Manual. Some hyperlinks in entries of the User's Reference Manual do not work, pointing instead to the top of the same page.

• Opening the HSA User Interface under Windows XP: If you open the HSA User Interface from HIPE while being logged in, the following error appears: JNLParseException[ Could not parse launch file. Error at line 0.]. The workaround is to log out in HIPE and start the HSA User Interface. If necessary, log in in the HSA User Interface and log back in in HIPE.

• Sending data from HIPE to DS9 under Mac OS: Using DS9 version 6.2, due to a problem with the DS9 application, HIPE fails to start DS9 automatically. Either use DS9 version 6.1 or start DS9 manually and re-send your image from HIPE. The DS9 team states that they plan on fixing this in the next release.

• Sending data from DS9 to HIPE under Windows. The following issues were reported when trying to send images via SAMP from DS9 to HIPE under Windows:
  • Windows 7: The option to send an image is sometimes disabled in DS9.
  • Windows XP: A SAMP message appears to be sent, but the image is not received by HIPE.
  • All Windows versions: At DS9 shutdown, the following Cygwin warning appears, even when data transfer works correctly, like in Windows Vista.

• Solved known issue with step by step execution. The known issue found in HIPE 7.0 with step by step execution has been solved. You can revert to the old behaviour by choosing Edit --> Preferences, going to Editors & Viewers --> Jython Editor and unticking the Use improved step by step execution checkbox.

• HIPE drops Versant database connection under Windows XP: A problem can occur if HIPE is run under Windows XP connected via WiFi to a Versant database server. If the connection is interrupted briefly, the HIPE session may fail with a "Network layer read error". Linux does not have this problem because it keeps retrying for several minutes.

• Garbled text after horizontal scrolling in Ubuntu 11.10. A problem on systems with NVidia cards and running Ubuntu 11.10 can cause text to appear garbled when scrolling horizontally, as shown by the figure below. To solve the problem, add the following property to the user.props file, located in the .hcss directory within your home directory. If there is no user.props file, create it.

java.vm.options = -Dsun.java2d.opengl=true

<-- COMMENT BOX CODE - DO NOT EDIT -->

<-- END OF COMMENT BOX CODE -->

Known issues in HIPE 7.3

None known in addition to those listed for HIPE 7.0.

Known issues in HIPE 7.2

None known in addition to those listed for HIPE 7.0.

Known issues in HIPE 7.1

None known in addition to those listed for HIPE 7.0.

Known issues in HIPE 7.0

• Cannot download data from the Herschel Archive due to corrupted cache.

Error messages about product cache corruption may vary. One example is the following:

herschel.share.util.ConfigurationException: Failed to locate product in cache

To solve the problem, try one or more of the following:

• Choose Edit --> Preferences and go to Data Access --> Storages & Pools. Select a pool in the Pools pane and click the Clear cache button. Note that this button will not be available if there are no pools listed in the Pools pane.
• If the above does not work, clear the HSA cache manually by deleting the home/.hcss/pal_cache/hsa directory, where home is your home directory.
• If the above does not work, clear all caches manually by deleting the home/.hcss/pal_cache directory.

• High disk activity with many local pools

If you have a high number of local pools on your disk (of the order of hundreds or more) you may experience noticeably high disk activity at HIPE startup.

• Step by step execution sometimes runs two statements at once

This happens when executing a compound statement, such as a for loop. The first command after the compound statement will be executed in the same step as the compound statement. Consider for example the following script:

for i in range(3):
  print i
print "Finished"

The line by line execution will run the for loop and the print "Finished' statement in the same step. A workaround for this behaviour is to add a pass command after the end of the compound statement. The pass command will be executed in the same step as the for loop, but without any effect:

for i in range(3):
  print i
pass
print "Finished"

HIPE known issues for old versions

See this page for known issues on later HIPE versions.

Known issues in HIPE 6.1

None known in addition to those listed for HIPE 6.0.

Known issues in HIPE 6.0

Core

• Inverted sign in Observation Context radialVelocity metadata

Usually the velocity of a source with respect to the observer is positive if it results in an increase of the observed wavelength with respect to the rest value, and negative otherwise, according to the formula v = (lambda - lambda_rest) / lambda_rest. In the current implementation of the radialVelocity metadata into the ObservationContext product, this convention is inverted: the sign of the spacecraft velocity is positive when it is moving towards the target.

• Failure in mosaic task

The mosaic task fails if your default language uses a different delimiter for decimal numbers than English (for example, a comma rather than a dot).

A workaround is to add the following line to your hipe.props file:

java.vm.options = -Duser.language=en -Duser.region=en

The file should be in the .hcss subdirectory of your home directory. If the file does not exist, create it. If the file already contains a line defining the java.vm.options property, you can add the two values -Duser.language=en and -Duser.region=en at the end of the line, separated by a space from the other values.

• Wrong units and sign in Cube Spectrum Analysis Toolbox

The cursor readout from a velocity map created in the Cube Spectrum Analysis Toolbox indicates km/s, but the values are in m/s. Also, velocity maps created from a spectral cube with an upper sideband (USB) frequency scale have the wrong sign.

<-- HCSS-11540 -->

• Query with no results wrongly returns an empty product.

When using the Product Browser perspective, a query with no results creates an empty product called p in the Variables view.

Steps to reproduce:

1. In the Product Browser view, select a query source and search for an obsid that you know not to exist.
2. In the Variables view, variables QUERY_RESULT and p appear.
3. The QUERY_RESULT variable is correctly empty, as you can verify by issuing the command print QUERY_RESULT in the Console view.
4. The p variable should not have been returned. You can safely delete it.

See the Data Analysis Guide for more information on the Product Browser perspective.

• Slow startup due to error messages.

You may see many errors like the following in the command line window from which you are starting HIPE. These errors cause a considerable delay in HIPE startup:

31-Jan-11 10:00:17.797 WARNING NavigatorView: Error checking file /bin: java.util.concurrent.TimeoutException

hcss.hipe.refreshPeriod = 3600000

Known issues in HIPE 5.3

None known in addition to those listed for previous 5.x versions.

Known issues in HIPE 5.2

None known in addition to those listed for previous 5.x versions.

Known issues in HIPE 5.1

Issues in previous 5.x versions may still apply to this version.

Core

• You may see many errors like the following in the command line window from which you are starting HIPE. These errors cause a considerable delay in HIPE startup:

31-Jan-11 10:00:17.797 WARNING NavigatorView: Error checking file /bin: java.util.concurrent.TimeoutException

hcss.hipe.refreshPeriod = 3600000

• An error like the following can sometimes appear in the command line window from which you are starting HIPE. This error is harmless and can be ignored. May also affect older HIPE versions. Fixed in HIPE 6.0.
  

Exception in thread "AWT-EventQueue-0" java.util.ConcurrentModificationException
        at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
        at java.util.AbstractList$Itr.next(AbstractList.java:343)
        at herschel.ia.gui.kernel.parts.impl.AbstractSite.opened(AbstractSite.java:468)
        ...

• Caches of PAL pools (most often used for accessing the HSA) can be corrupted when switching back and forth between HIPE v5.1 and previous versions. The result of this corruption is that the cache can silently give you other data than the data you were asking for. If you are switching back and forth between 5.1 and previous versions, clear your caches in the Data Access Preferences panel: From the Edit menu, select Preferences. Under Data Access, locate the Pools and Storages section. In the bottom right section of the panel, select the pool that has a cache enabled and click "Clear cache". Fixed in HIPE 6.0 and 5.3.

Known issues in HIPE 5.0

Core

• The cursor readout from a velocity map created in the Cube Spectrum Analysis Toolbox indicates km/s, but the values are in m/s. Also, velocity maps created from a spectral cube with an upper sideband (USB) frequency scale have the wrong sign.

  <-- HCSS-11540 -->

• HIPE does not start if there are too many local pools. This happens when the list of local pool names, separated by commas, exceeds 8096 characters.
• Zooming on an image may cause HIPE to hang. The zoom level at which this happens depends on the original size of the image.
• The saveObservation task with default settings fails to save an Observation Context to a pool.
  Workaround: delete the hsa_cache local pool directory and the $HOME/.hcss/pal_cache/hsa directory. Alternatively, move the hsa_cache local pool to $HOME/.hcss/pal_cache/hsa/pool/hsa_cache/. Note that you have to move the hsa_cache directory from your local pool directory into the above directory, so that you will have paths that have hsa_cache twice, for example: $HOME/.hcss/pal_cache/hsa/pool/hsa_cache/hsa_cache/herschel.ia.obs.ObservationContext.
• The ra and dec values in the metadata of an Observation Context (and hence the RA and DEC entries in the summary) can be significantly different from the target coordinates (raNominal and decNominal) entered in HSpot. This is caused from averaging the target and chop positions. Coordinates in the actual data are correct.
• The NearestNeighbourProjectionTask task, which creates a spectral cube from three-dimensional data, fails to create a target grid out of a set of target grids. A command such as targetGrid = nearestNeighbourProjection.targetGrid([tg1,tg2,tg3]) fails with an IllegalArgumentException.
  For more information on NearestNeighbourProjectionTask, see the HCSS User's Reference Manual in the HIPE Help System.
• In the LocalStoreWriter task, the default location of new local pools is $HOME/.hcss instead of the default $HOME/.hcss/lstore. This results in new pools being created in non-standard directories. Information about the locations of these pools is written in the $HOME/.hcss/hipe.props file.
  To remove a local pool in a non-standard location, delete the pool directory and the corresponding configuration line in hipe.props.
  To create local pools in the standard location, change the value in the Location field of the LocalStoreWriter task to $HOME/.hcss/lstore, where $HOME is your home directory (for example /home/mrbean/.hcss/lstore on Linux systems).
  Fixed in HIPE 5.1.
• A bug in Firefox may prevent the Help System from starting correctly (a browser window opens but nothing appears). This can be solved by deleting all the browser cookies, or just the cookies coming from 127.0.0.1 (the local system). See also General Help System issues on this page.
• Most entries in the User's Reference Manual have wrong or missing categories.
• The documentation shipped with HIPE still mentions JIDE in several places. JIDE was the data reduction application that came before HIPE and is no longer supported. With the exception of Appendix B of the Scripting and Data Mining guide, all references to JIDE can be applied to HIPE.
• The message INTERRUPTED while loading image can sometimes appear in the command line window from which you launched HIPE. This message is harmless and can be ignored.

HIFI

• Ra/Dec coordinates in data cube meta-data are incorrect.
• In CLASS files generated via the HiClass task, the source velocity is not included or applied (i.e. the frequency scale is wrong).
• When the frequency calibration of WBS is computed from the values of the HRS spectra, the coefficient of the polynomial to be used are function of the temperature. These are not available in the calibration tree.

PACS

• The following scripts in the Pipeline --> PACS menu in HIPE are known to crash: extended_pipeline.py, L1_smallSource.py, L2_scanMap.py. Other scripts may crash or not be applicable because they depend on the results of other non-working scripts. Many scripts are not adequately commented.
• The data/pcal/PCalSpectrometer_ArrayInstrument_FM_v5.fits calibration file, dealing with spectrometer spatial calibration, contains incorrect values. This results in offsets of the order of one arcsecond between reported coordinates and commanded positions.
• Some tasks may be holding data products in memory unnecessarily, thus slowing down HIPE and possibly causing out of memory errors.
• In the photProjectPointSource task, the calibration = false mode (no longer recommended) is not flux conserving.

SPIRE

• The signalJumpDetector task is not registered at HIPE startup.
  Workaround: import it manually with the following command:
  from herschel.spire.ia.pipeline.common.sigjump import SignalJumpDetectorTask
• The AbstractMapperTask may treat incorrectly a Wcs provided as input. When the input Wcs is defined using the CDELTi keywords, the result is correct, but when the pixels are defined with the CDi_j keywords, these are translated into CDELTi and CROTA2 keywords. This is not equivalent, if not rectangular pixels are required.
• The AbstractMapperTask has a bug in the generation of tod files which can result in an ArrayIndexOutOfBoundsException at the map generation.
• The spireCal task outputs an empty calibration context if the input pool does not exist, instead of reporting the problem. This results in errors when user scripts attempt to access the calibration files.

<--
  Known issues in HIPE 4.0 

  PAL Storage Manager: Can't add pools to storages 

After creating a new ProductStorage using the PAL Storage Manager, you have to specify the pools that should be added to the storage. After selecting them and clicking "OK" in the GUI, the application appears to hang.



The work-around is to click Cancel, after clicking OK. Clicking Cancel closes the panel, releases the deadlock, and allows the pools to be added to the storage, as was requested by the click on OK.

Fixed in HIPE 4.1.
-->