Plug-in Framework Design Document

This document describes the design choices underpinning the Plug-in Framework in HIPE. The SCR that originally requested the Plug-in Framework was HcssScr:9471.

What a Plug-in is and can do

Many people write software either in Java or in Jython and then want to share this software with other people. Sometimes this software is a script that has to be run manually, every time HIPE is started (for example if it defines some Tasks). Or people generate pools of data that they want to share.

This software and this data can be zipped and put on a web-server. If the zip-file has a version in it, this version is lost when the zip-file is unpacked: After importing the data pool in HIPE, HIPE doesn't "remember" which version this was or where it came from. It will be like any other data pool.

The Plug-in Framework makes this kind of sharing easier. The bundles that are shared are called plug-ins. Plug-ins can contain user scripts (such as pipeline scripts), initialization scripts (scripts that have to be run on every start-up of HIPE), pools and even software written in Java.

A plug-in, then, should be able to handle three kinds of "objects":

  • one or more LocalStores
  • JARs and Jython initialization scripts
  • user scripts

The LocalStores are "made available" in HIPE by adding to the pools in the Data Access Preferences panel (see Edit --> Preferences).

The user scripts are scripts that a user should execute, such as a pipeline scripts. This contrasts with scripts that should be executed automatically when HIPE starts, such as scripts defining tasks. The latter is called an "initialization script".

User scripts that are contributed by plug-ins are added to a sub-menu of the Tools menu in HIPE, similar to the instrument menus under the Pipeline menu in HIPE.

The Jython initialization scripts should be executed on HIPE start-up. From Jython (the HIPE console) it is possible to import Java classes from the JARs from the plug-in. For this, the Java classes have to be on the Jython system path somehow. To be able to add the jars to the Jython path (such that they will be picked up), the jars have to be unpacked. Jython 2.1 does not appear to be capable of dealing with JARs that are added to the system path dynamically (i.e. using Jython commands and not by setting the CLASSPATH before start-up). The jars are unpacked in a directory called "work" in the plug-in directory. Therefore, the plug-in is not allowed to have a directory called "work" already from itself, for this will interfere.

Any JARs can be contributed as a plug-in as an easier way of providing Java code to the system for people external to the HSC. Until now, there were three ways to add a JAR to the system: Either it has to be either accepted as a third-party library and added to the reference platform. Or the software could become part of the HCSS in the form of a module, in which case the software will be allowed to make use of the HCSS itself as well (libraries on the reference platform can only be referenced by the HCSS and cannot reference the HCSS themselves). Or the user of the library has to add the JAR to his/her classpath manually. This means that, in case of using the reference platform or an HCSS module, one will have to follow HSC procedures and most importantly, updates are only possible when the entire HCSS is update, or manual actions from the user are required. In the case of plug-ins, a new version can be released and installed in HIPE as often as one pleases and installation is fully automatic.

A plug-in will also be able to provide configuration information about itself, all of it optional. This information goes into a configuration file plugin.xml, in the root directory of the plug-in. (The configuration file itself is completely optional, it is ok if it doesn't exist or if it's empty.)

Packaging and publishing plug-ins

A plug-in installable bundle, as a plug-in is published, is a JAR or a ZIP file. This file has the following contents:

  • XML file plugin.xml. This is the "deployment descriptor" containing all sorts of (optional) information about the plug-in, from simple things such as the plug-in description and homepage, to more advanced items like the location of a custom installer.
  • A class file for a class called Installer to execute the configuration phase of the plug-in. See the section on plug-in installation for details. This is thought to be most useful in combination with a custom installer.
  • Jython script: This script is executed to "start" the plug-in (directly after installation and at HIPE start-up). It can be compared to the for Java packages in the HCSS.
  • Directory called jars. This directory should contain all jar files that should be added to the classpath.
  • Directory called scripts. This directory should contain all Jython scripts that the plug-in contributes. These should be scripts that are intended for the user to execute, such as pipeline scripts. This is contrary to scripts like which are executed by HIPE automatically.
  • Directory called pools. This directory should contain all LocalStore pools that the plug-in contributes.
  • License file LICENSE.txt

All the above elements are optional: It is perfectly ok to omit any one.

When the author has the plug-in ready, it can be sent to the HSC and we will add it to the public plug-ins Wiki. Also, from HIPE there will be a button "Find plug-ins", which will open a browser window pointing to this wiki page. This page/web-site can be extended in the future, but for now we will use a manually maintained Wiki: DpHipePlugins. If the maintenance gets out of hand, we can migrate to a more automated system, that allows authors to upload directly (for example).

Installing plug-ins

After the user specifies a URL where a plug-in is located, the installation will be performed completely automatically. It will be possible as well to browse for the plug-in bundle on the local file system. The installation has three phases: bootstrap, custom, and configuration. The bootstrap installation is implemented by HIPE and is executed always when a plug-in is installed, the other two phases are optional.

The plug-in directory is by default under $HOME/.hcss/apps/hipe/plugins. This means that the installed plug-ins are preserved when, for example, a new version of HIPE is installed. Plug-ins are shared between all installed HIPE versions. Plug-ins are not shared between users: The framework currently provides no mechanism for a directory with plug-ins that is shared by multiple users.

Installation phase 1: Bootstrap

The first step during the bootstrap phase to perform the first basic installation (basically, unjarring or unzipping in the plug-in directory). After this, the various contributions from the plug-in are added to HIPE: For contributed JARs, nothing special needs to be done, since they will be added to the classpath dynamically on every HIPE start-up (from the next restart on). LocalStores are added in-place to the HIPE Preferences under Data Access. This means that an entry for the LocalStore is added to the pool in the Preferences, which "points" to the LocalStore inside the plug-in. The LocalStore is not copied. The same for contributed scripts.

Installation phase 2: Custom installation

The custom install phase allows for the plug-in author to specify a completely custom installation program. The system offers two ways of doing this: By specifying the URL of an installer that can be started using Java WebStart, or by including the installer as a Java program in the plug-in.

The WebStart URL is specified in the plug-in deployment descriptor, plugin.xml. For example, CASSIS is a program that has its own installer. This installer will be executed during this phase.

The other possibility is to specify the class name of the main class in the plugin.xml configuration file. If this main class implements the interface herschel.ia.gui.apps.plugin.Installer, then the install method of this class will be called. The advantage of this is that the install method is passed a PluginEnv object. This PluginEnv object can be used to add JAR files to the classpath. Normally this will not be necessary: Any JARs that exist in the jars directory in the plug-in are added to the classpath automatically. But if the custom installer installs JARs in a different location, you will want to access these JARs from HIPE. To do so, they must be added to the classpath. This work can also be done during the configuration phase (phase 3 -- see next paragraph).

Another advantage of having access to the PluginEnv object can be that it gives access to several plug-in attributes (custom properties, license text), though most of this will already be known and not particularly useful at this stage.

If the custom installer main class does not implement herschel.ia.gui.apps.plugin.Installer, then its main method will be called. If it doesn't have a main method either, then an error is generated.

Installation phase 3: Configuration

Finally, during the configuration phase, code is executed that is contained in the plug-in. This should be a class called Installer in the default package (i.e. not in any package) implementing an interface called herschel.ia.gui.apps.plugin.Installer. This Installer must have a public zero-argument constructor. This step can be used to inform HIPE about any external JAR archives that may have been installed during the custom install phase.

If JARs are added to the classpath this way, they are recorded in a file called bundle.xml and used when the plug-in is loaded or activated (I use loading and activation as synonyms).

Loading/activating plug-ins

Once installed in the plug-ins directory, the plug-in will have a sub-directory jars (optionally). The plug-in directory will also, optionally, contain a bundle.xml listing jar files.

HIPE will construct a custom classloader that includes the jars in the jars directory in the plug-in directory, as well as the jars listed in the bundle.xml. All these jars will then also be dynamically added to the Python system path, for which they have to be unpacked (in Jython 2.1 – probably no need to unpack with Jython 2.5). The JARs will be unpacked into the work directory in the plug-in directory. Note that this means you have two copies of the class-files: Once in the JAR and once unpacked in the work-directory. The JARs are used from Java and the unpacked class files are used from Jython. This is not optimal, but it's a current limitation of Jython (2.1).

If any scripts are contained in the plug-in, a menu item with the name of the plug-in will be added to the Tools menu. Similarly to the instrument items under the Pipeline menu, pointing to that item will open the list of contributed scripts.

Finally, HIPE will run the Jython initialization script in the plug-in directory (hard-coded to be

Plug-in status panel

Selecting "Plug-ins" from the Tools menu, opens something similar to the Add-ons panel in Firefox, where a user can see the plug-ins he has installed, including:

  • plug-in name
  • description
  • version
  • plug-in homepage
  • status (enabled/disabled)
  • license
  • custom properties

From this GUI one can enable/disable the plug-ins (requires HIPE restart). It is also possible to uninstall a plug-in (see next section).

Updating and uninstalling plug-ins

We will provide a mechanism to uninstall a plug-in. We provide a default uninstaller. No mechanism to provide a custom uninstaller is currently implemented. The default uninstaller will simply delete the plug-in directory.

Currently, no mechanism for automatic updates is available. The user will simply have to install a new version of the plug-in to update it to a new version. Both versions (the existing one and the new one) will be installed, and when HIPE is restarted, only the highest version will be picked up and activated.

Working with Plug-ins in HIPE

See the relevant section of the Developer Manual.

Open issues

Plug-in dependencies: Can a plug-in require for example that the PACS classes are installed? If we are talking about a PACS calibration data plug-in, should this disable itself if it finds we are running HIPE with no PACS classes?

If plug-ins disable themselves if they find that they are not compatible with the HIPE version or project (hcss, hcss.dp.core or hcss.dp.pacs), should they go through the list of the plug-ins again when we run a different HIPE version? I believe Firefox does this: It checks your plug-ins when you start a new Firefox that is a different version from the one you ran yesterday. This requires that you somehow store the version of HIPE that you are running in the plugins directory, or it requires that you check all on every start-up. But the latter defeats the purpose of disabling plug-ins and writing this to disk.

Do we need the possibility to specify a custom uninstaller?

Read-only access to plug-in files: Currently there is no protection against the user modifying the scripts and pools that are delivered with a plug-in. Should this be protected?

Auto-update: When a new version of a plug-in comes on-line, does HIPE discover this automatically?

Installer failures: If the custom installer (for example WebStart installer) fails somehow, or is cancelled, currently there is no way to pick this up.

-- PaulBalm - 16 Aug 2010

Edit | Attach | Watch | Print version | History: r8 | r5 < r4 < r3 < r2 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r3 - 2010-09-21 - PaulBalm
This site is powered by the TWiki collaboration platform Powered by Perl