HIPE known issues for old versions

See this page for known issues on later HIPE versions.

Known issues in HIPE 9.2

New PACS calibration version 45 should not be used with HIPE 9

In HIPE 10 a new spectroscopy telescope background calibration was introduced. This update is not compatible with the code in HIPE 9.

If you use the background normalisation ipipe script for chopped spectroscopy in HIPE 9, you should not update the PACS calibration tree to version 45. You should keep using the calibration set 41 and cancel the update when the automatic calibration updater in HIPE asks to install set 45.

If you have updated to set 45 and want to use the background normalisation in HIPE 9, you can switch back to version 41 in two ways:

  • With the command getCalTree(version=41).
  • By removing the file PCalBase_TimeDependency_FM_v45.fits from the directory ~/.hcss/data/pcal-community.

You can alway check the calibration version that you are using with the following command:

print getCalTree().version

If you use HIPE 10, you should update to version 45 of the calibration tree to make use of several new calibration files.

Issues with computePVMap task

The computePVMap task has the following issues:

  • There are remaining aesthetic issues regarding the pvMap aspect ratio modification in the task.
  • There is a known error when extracting a pvMap with row < col (velocity dimension < spatial dimension). Workaround: set the correctAspectRatio parameter to False (in the graphical interface, untick the corresponding checkbox).

Known issues in HIPE 9.1

None known in addition to those listed for HIPE 9.0.

Known issues in HIPE 9.0

Performance issues of convolution task with NaN values

The performance of the imageConvolution task is severely degraded when NaN values are found in input data. HIPE 8.3 is not affected by this problem. Fixed in HIPE 9.1.

Performance issues with very large images

Images larger than about 100 MB in FITS format can still take over a minute to load. During such time HIPE is unresponsive. Image analysis operations, such as cropping and adding contours, also take a long time on large images.

Problem with PACS background normalization script

The ChopNodBackgroundNormalizationRange interactive pipeline script (Pipelines --> PACS --> Spectrometer --> Chopper large range scan SED --> Background Normalization) is missing a definition of the variable target. After loading the observation, assuming it is stored in a variable called obs, you can define the missing target variable as follows:

target = obs.meta["object"].value.replace(" ","_")

PACS task specFlatFieldLine problem with NaN values

The specFlatFieldLine task cannot handle NaNs in the fluxes of PacsCube products. This affected one observation as far as we tested. If you encounter this problem then you can mask the NaNs by running the function shown below. Copy and paste the function code in a new Jython script window in the Editor view of HIPE, then click the green double arrow icon in the HIPE toolbar to run the script. After that you can call the function from the Console view, passing the variable contaning your cube as argument: addInvalidMask(myCube).

def addInvalidMask(slicedCube):
    for i in range(0,slicedCube.numberOfSlices):
       cube = slicedCube.get(i)
       invalid = IS_NAN(cube.flux)
       if not cube.containsMask("INVALID"):
          cube.addMask("INVALID","Invalid signal values")

HIPE hangs when pressing the activate/deactivate button in the mask panel of the Spectrum Explorer PACS extensions

This button is enabled only when you change the state of one or more masks. Fixed in HIPE 9.1.

Image analysis tasks start only from Applicable section

Image analysis tasks will open only from the Applicable folder of the Tasks view. These tasks will not open when selected from the All or By Category folders. Note that you must select an image variable in the Variables view for the Applicable folder to be filled with the available tasks. To be fixed in HIPE 10. The problem affects the following tasks:

  • annularSkyAperturePhotometry
  • circleHistogram
  • crop
  • ellipseHistogram
  • fixedSkyAperturePhotometry
  • polygonHistogram
  • profile
  • rectangleHistogram
  • rectangularSkyAperturePhotometry
  • sourceFitting

PV diagrams created with computePVMap on HIFI cubes cannot be easily viewed

HIFI cubes have in general a small number of rows and columns, while the spectral dimension is rather large (thousands of channels). The PV map is very narrow in the x direction and really wide in the y-direction, so that it does not fit in the Spectrum panel. When displaying the y-axis values, at its smaller zoom, only a small number of channels can be seen. Workaround: open the variable created from the variables pane and use the full Display/Image Viewer.

HIPE crash linked to JAMA (Java Matrix) library

HIPE may crash without warning when executing parts of the bundled JAMA library. If you experience a sudden crash when running a script, this may be the cause, even if there are no obvious references to matrices in the script. Take the following steps to prevent the crash from happening:

  1. Go to the .hcss directory in your open directory. Open the hipe.props file, or create it if it does not exist.
  2. Add the following line to the file:
    java.vm.options = -XX:CompileCommand=exclude,Jama/LUDecomposition,<init>
  3. Restart HIPE.

Problem with transpose task

In some cases the transpose task does not convert correctly the WCS of the input image. This problem occurs only when the WCS of the image defines the rotation angle (CROTA2). When the transformation matrix (PCx_y or CDx_y) is used, the problem does not occur. To be fixed in HIPE 10.

Velocity map creation in Legacy Cube Spectrum Analysis Toobox (CSAT) fails

The creation of velocity maps with the Legacy CSAT fails; the task will hang without an error message. CSAT is no longer supported and the new cube toolbox, which uses an improved algorithm for velocity map making, should be used instead. See 6.8.3 in the Herschel Data Analysis Guide for information about using the new cube toolbox.

Flag set in HIFI's flagTool task do not appear to be used by the fitBaseline task

This is a display issue only, the flags set by flagTool are, in fact, used.

PACS specific: convertL1ToScanam fails

convertL1ToScanam fails because of a missing import for the calibration tree, when assignRaDec is set to True. The work around solution is :

calTree = getCalTree(time=frames.startDate)

Fixed in HIPE 9.2.

Known issues in HIPE 8.3

None known in addition to those listed for HIPE 8.2.

Known issues in HIPE 8.2

  • Task createRgbImage puts wrong WCS in the output. Instead of using the WCS provided by the Wcs input parameter, this task uses the WCS of one of the input images.

  • PACS photometer scan map processing : Due to the new scheme of filtering turnaround manoeuvres between scan legs with the task filterOnScanSpeed task (masking frames in the 'ScanSpeedMask' mask, instead of removing the frames), photProject will project maps including the slew to target, hence exploding the memory in Hipe and leading to huge map sizes, because the photProject WCS doesn't take into account this new mask. A workaround solution is to run the filterSlew task previously, at the expense of possibly loosing a few frames on the first scan leg or alternatively run photProject in the following way:
ScanSpeedMask = frames.getMask('ScanSpeedMask')[0,0,:] 
indexSpeed    = ScanSpeedMask.where(ScanSpeedMask==False)
map          = photProject(frames.select(indexSpeed), calTree=calTree,calibration=True)
This bug shall be corrected in the next Hipe versions.

Known issues in HIPE 8.1

None known in addition to those listed for HIPE 8.0.

Known issues in HIPE 8.0

  • Problem with transpose task. In some cases the transpose task does not convert correctly the WCS of the input image. This problem occurs only when the WCS of the image defines the rotation angle (CROTA2). When the transformation matrix (PCx_y or CDx_y) is used, the problem does not occur.

  • Problem with NaN values with imageConvolution task. Starting from HIPE 7, the imageConvolution task works with images containing NaNs too. However, the results when NaNs are present (either in the map or around the coverage area) are not reliable. A fixed version of the task is scheduled for HIPE 9.

  • Problem with correction of MADmap point-source artifacts. The PACS task photCorrMadmapArtifacts, used to correct point-source artifacts in MADmap produced Level 2.5 products, only works when the initial time-ordered data product (TOD) is created with scale factor of 1. All other values of scale factor may produce incorrect results. A fix, possibly distributed as a plug-in, will be available later.

  • Problem with units in the Cube Spectrum Analysis Toolbox. This tool returns the wrong units of measurement with extraction plots and integrated maps. This issue has been detected with PACS data and is under investigation. Previous HIPE versions are not affected.

  • User pool names (pool ID) restrictions: Old pools that do not follow the restrictions on the allowed names cannot be read in HIPE (eg a dot "." in the name is such a case). The user can safely rename the folder (found in the local store) following the rules: allowed characters in name: alphanumeric, dash ("-") and underscore ("_"), the name cannot start with a dash ("-") and cannot have "urn" as name. The pool ID is case-sensitive and must be with less than 255 characters.

  • High memory usage with Mac OS 10.6 Snow Leopard. There are reports of HIPE 8.0.1 using all available memory and freezing while reprocessing observations under Mac OS 10.6 Snow Leopard. This problem does not seem to appear on other platforms, including other Mac OS versions. The problem is being investigated.

  • Velocity Maps in the Cube Spectrum Analysis Toolbox. Fitting of faint lines (less than 2 sigma) or non-gaussian lines can result in incorrect velocities in the velocity map. It is advised to fit the velocity in a few positions manually in the spectrum explorer and verify the consistency with the calculated velocities at these positions in the velocity map in the cube explorer.

  • Upgrade to Jython 2.5. Starting from version 8.0, HIPE will include a new version of Jython. You may have to adapt your scripts for them to work with the new version. See this page for the details.

  • Links in the User's Reference Manual. Some hyperlinks in entries of the User's Reference Manual do not work, pointing instead to the top of the same page.

  • Opening the HSA User Interface under Windows XP: If you open the HSA User Interface from HIPE while being logged in, the following error appears: JNLParseException[ Could not parse launch file. Error at line 0.]. The workaround is to log out in HIPE and start the HSA User Interface. If necessary, log in in the HSA User Interface and log back in in HIPE.

  • Sending data from HIPE to DS9 under Mac OS: Using DS9 version 6.2, due to a problem with the DS9 application, HIPE fails to start DS9 automatically. Either use DS9 version 6.1 or start DS9 manually and re-send your image from HIPE. The DS9 team states that they plan on fixing this in the next release.

  • Sending data from DS9 to HIPE under Windows. The following issues were reported when trying to send images via SAMP from DS9 to HIPE under Windows:
    • Windows 7: The option to send an image is sometimes disabled in DS9.
    • Windows XP: A SAMP message appears to be sent, but the image is not received by HIPE.
    • All Windows versions: At DS9 shutdown, the following Cygwin warning appears, even when data transfer works correctly, like in Windows Vista.


  • Solved known issue with step by step execution. The known issue found in HIPE 7.0 with step by step execution has been solved. You can revert to the old behaviour by choosing Edit --> Preferences, going to Editors & Viewers --> Jython Editor and unticking the Use improved step by step execution checkbox.

  • HIPE drops Versant database connection under Windows XP: A problem can occur if HIPE is run under Windows XP connected via WiFi to a Versant database server. If the connection is interrupted briefly, the HIPE session may fail with a "Network layer read error". Linux does not have this problem because it keeps retrying for several minutes.

  • Garbled text after horizontal scrolling in Ubuntu 11.10. A problem on systems with NVidia cards and running Ubuntu 11.10 can cause text to appear garbled when scrolling horizontally, as shown by the figure below. To solve the problem, add the following property to the user.props file, located in the .hcss directory within your home directory. If there is no user.props file, create it.

java.vm.options = -Dsun.java2d.opengl=true


Known issues in HIPE 7.3

None known in addition to those listed for HIPE 7.0.

Known issues in HIPE 7.2

None known in addition to those listed for HIPE 7.0.

Known issues in HIPE 7.1

None known in addition to those listed for HIPE 7.0.

Known issues in HIPE 7.0

  • Cannot download data from the Herschel Archive due to corrupted cache.

Error messages about product cache corruption may vary. One example is the following:

herschel.share.util.ConfigurationException: Failed to locate product in cache

To solve the problem, try one or more of the following:

  • Choose Edit --> Preferences and go to Data Access --> Storages & Pools. Select a pool in the Pools pane and click the Clear cache button. Note that this button will not be available if there are no pools listed in the Pools pane.
  • If the above does not work, clear the HSA cache manually by deleting the home/.hcss/pal_cache/hsa directory, where home is your home directory.
  • If the above does not work, clear all caches manually by deleting the home/.hcss/pal_cache directory.

  • High disk activity with many local pools

If you have a high number of local pools on your disk (of the order of hundreds or more) you may experience noticeably high disk activity at HIPE startup.

  • Step by step execution sometimes runs two statements at once

This happens when executing a compound statement, such as a for loop. The first command after the compound statement will be executed in the same step as the compound statement. Consider for example the following script:

for i in range(3):
  print i
print "Finished"

The line by line execution will run the for loop and the print "Finished' statement in the same step. A workaround for this behaviour is to add a pass command after the end of the compound statement. The pass command will be executed in the same step as the for loop, but without any effect:

for i in range(3):
  print i
print "Finished"

Known issues in HIPE 6.1

None known in addition to those listed for HIPE 6.0.

Known issues in HIPE 6.0


  • Inverted sign in Observation Context radialVelocity metadata

Usually the velocity of a source with respect to the observer is positive if it results in an increase of the observed wavelength with respect to the rest value, and negative otherwise, according to the formula v = (lambda - lambda_rest) / lambda_rest. In the current implementation of the radialVelocity metadata into the ObservationContext product, this convention is inverted: the sign of the spacecraft velocity is positive when it is moving towards the target.

  • Failure in mosaic task

The mosaic task fails if your default language uses a different delimiter for decimal numbers than English (for example, a comma rather than a dot).

A workaround is to add the following line to your hipe.props file:

java.vm.options = -Duser.language=en -Duser.region=en

The file should be in the .hcss subdirectory of your home directory. If the file does not exist, create it. If the file already contains a line defining the java.vm.options property, you can add the two values -Duser.language=en and -Duser.region=en at the end of the line, separated by a space from the other values.

  • Wrong units and sign in Cube Spectrum Analysis Toolbox

The cursor readout from a velocity map created in the Cube Spectrum Analysis Toolbox indicates km/s, but the values are in m/s. Also, velocity maps created from a spectral cube with an upper sideband (USB) frequency scale have the wrong sign.

  • Query with no results wrongly returns an empty product.

When using the Product Browser perspective, a query with no results creates an empty product called p in the Variables view.

Steps to reproduce:

  1. In the Product Browser view, select a query source and search for an obsid that you know not to exist.
  2. In the Variables view, variables QUERY_RESULT and p appear.
  3. The QUERY_RESULT variable is correctly empty, as you can verify by issuing the command print QUERY_RESULT in the Console view.
  4. The p variable should not have been returned. You can safely delete it.

See the Data Analysis Guide for more information on the Product Browser perspective.

  • Slow startup due to error messages.

You may see many errors like the following in the command line window from which you are starting HIPE. These errors cause a considerable delay in HIPE startup:

31-Jan-11 10:00:17.797 WARNING NavigatorView: Error checking file /bin: java.util.concurrent.TimeoutException
Workaround: set the property hcss.hipe.refreshPeriod as follows:
hcss.hipe.refreshPeriod = 3600000
See the HIPE Owner's Guide for information on how to set properties.

Known issues in HIPE 5.3

None known in addition to those listed for previous 5.x versions.

Known issues in HIPE 5.2

None known in addition to those listed for previous 5.x versions.

Known issues in HIPE 5.1

Issues in previous 5.x versions may still apply to this version.


  • You may see many errors like the following in the command line window from which you are starting HIPE. These errors cause a considerable delay in HIPE startup:
31-Jan-11 10:00:17.797 WARNING NavigatorView: Error checking file /bin: java.util.concurrent.TimeoutException
Workaround: set the property hcss.hipe.refreshPeriod as follows:
hcss.hipe.refreshPeriod = 3600000
See the HIPE Owner's Guide for information on how to set properties.

  • An error like the following can sometimes appear in the command line window from which you are starting HIPE. This error is harmless and can be ignored. May also affect older HIPE versions. Fixed in HIPE 6.0.
Exception in thread "AWT-EventQueue-0" java.util.ConcurrentModificationException
        at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
        at java.util.AbstractList$Itr.next(AbstractList.java:343)
        at herschel.ia.gui.kernel.parts.impl.AbstractSite.opened(AbstractSite.java:468)

  • Caches of PAL pools (most often used for accessing the HSA) can be corrupted when switching back and forth between HIPE v5.1 and previous versions. The result of this corruption is that the cache can silently give you other data than the data you were asking for. If you are switching back and forth between 5.1 and previous versions, clear your caches in the Data Access Preferences panel: From the Edit menu, select Preferences. Under Data Access, locate the Pools and Storages section. In the bottom right section of the panel, select the pool that has a cache enabled and click "Clear cache". Fixed in HIPE 6.0 and 5.3.

Known issues in HIPE 5.0


  • The cursor readout from a velocity map created in the Cube Spectrum Analysis Toolbox indicates km/s, but the values are in m/s. Also, velocity maps created from a spectral cube with an upper sideband (USB) frequency scale have the wrong sign.
  • HIPE does not start if there are too many local pools. This happens when the list of local pool names, separated by commas, exceeds 8096 characters.
  • Zooming on an image may cause HIPE to hang. The zoom level at which this happens depends on the original size of the image.
  • The saveObservation task with default settings fails to save an Observation Context to a pool.
    Workaround: delete the hsa_cache local pool directory and the $HOME/.hcss/pal_cache/hsa directory. Alternatively, move the hsa_cache local pool to $HOME/.hcss/pal_cache/hsa/pool/hsa_cache/. Note that you have to move the hsa_cache directory from your local pool directory into the above directory, so that you will have paths that have hsa_cache twice, for example: $HOME/.hcss/pal_cache/hsa/pool/hsa_cache/hsa_cache/herschel.ia.obs.ObservationContext.
  • The ra and dec values in the metadata of an Observation Context (and hence the RA and DEC entries in the summary) can be significantly different from the target coordinates (raNominal and decNominal) entered in HSpot. This is caused from averaging the target and chop positions. Coordinates in the actual data are correct.
  • The NearestNeighbourProjectionTask task, which creates a spectral cube from three-dimensional data, fails to create a target grid out of a set of target grids. A command such as targetGrid = nearestNeighbourProjection.targetGrid([tg1,tg2,tg3]) fails with an IllegalArgumentException.
    For more information on NearestNeighbourProjectionTask, see the HCSS User's Reference Manual in the HIPE Help System.
  • In the LocalStoreWriter task, the default location of new local pools is $HOME/.hcss instead of the default $HOME/.hcss/lstore. This results in new pools being created in non-standard directories. Information about the locations of these pools is written in the $HOME/.hcss/hipe.props file.
    To remove a local pool in a non-standard location, delete the pool directory and the corresponding configuration line in hipe.props.
    To create local pools in the standard location, change the value in the Location field of the LocalStoreWriter task to $HOME/.hcss/lstore, where $HOME is your home directory (for example /home/mrbean/.hcss/lstore on Linux systems).
    Fixed in HIPE 5.1.
  • A bug in Firefox may prevent the Help System from starting correctly (a browser window opens but nothing appears). This can be solved by deleting all the browser cookies, or just the cookies coming from (the local system). See also General Help System issues on this page.
  • Most entries in the User's Reference Manual have wrong or missing categories.
  • The documentation shipped with HIPE still mentions JIDE in several places. JIDE was the data reduction application that came before HIPE and is no longer supported. With the exception of Appendix B of the Scripting and Data Mining guide, all references to JIDE can be applied to HIPE.
  • The message INTERRUPTED while loading image can sometimes appear in the command line window from which you launched HIPE. This message is harmless and can be ignored.


  • Ra/Dec coordinates in data cube meta-data are incorrect.
  • In CLASS files generated via the HiClass task, the source velocity is not included or applied (i.e. the frequency scale is wrong).
  • When the frequency calibration of WBS is computed from the values of the HRS spectra, the coefficient of the polynomial to be used are function of the temperature. These are not available in the calibration tree.


  • The following scripts in the Pipeline --> PACS menu in HIPE are known to crash: extended_pipeline.py, L1_smallSource.py, L2_scanMap.py. Other scripts may crash or not be applicable because they depend on the results of other non-working scripts. Many scripts are not adequately commented.
  • The data/pcal/PCalSpectrometer_ArrayInstrument_FM_v5.fits calibration file, dealing with spectrometer spatial calibration, contains incorrect values. This results in offsets of the order of one arcsecond between reported coordinates and commanded positions.
  • Some tasks may be holding data products in memory unnecessarily, thus slowing down HIPE and possibly causing out of memory errors.
  • In the photProjectPointSource task, the calibration = false mode (no longer recommended) is not flux conserving.


  • The signalJumpDetector task is not registered at HIPE startup.
    Workaround: import it manually with the following command:
    from herschel.spire.ia.pipeline.common.sigjump import SignalJumpDetectorTask
  • The AbstractMapperTask may treat incorrectly a Wcs provided as input. When the input Wcs is defined using the CDELTi keywords, the result is correct, but when the pixels are defined with the CDi_j keywords, these are translated into CDELTi and CROTA2 keywords. This is not equivalent, if not rectangular pixels are required.
  • The AbstractMapperTask has a bug in the generation of tod files which can result in an ArrayIndexOutOfBoundsException at the map generation.
  • The spireCal task outputs an empty calibration context if the input pool does not exist, instead of reporting the problem. This results in errors when user scripts attempt to access the calibration files.

blog comments powered by Disqus
Edit | Attach | Watch | Print version | History: r8 < r7 < r6 < r5 < r4 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r5 - 2013-08-08 - AlvarGarcia
This site is powered by the TWiki collaboration platform Powered by Perl