TWiki> Public Web>PalConcepts>HttpPool (revision 8)EditAttach

Http Pool

This page serves as the main documentation for running an HTTP server pool. The corresponding client pool is contained in the build and not covered here other than in brief. Most of this page is about configuring Tomcat to run servlets using the HCSS in general, rather than specifically HttpPool.

Installation

The server pool requires a web server that supports Java servlets. This page assumes that Tomcat is used for this purpose. It should work with the latest version (6.0.18 at the time of writing). The first step then is to install Tomcat, although adding the pool servlet to an existing installation is also fine. Note that this does not detail all possible configurations that experts may wish to set up, but a single simple one.

Note that only one HTTP pool server is needed, regardless of the number of pools it accesses.

There is no need to install Tomcat in a privileged account, and probably good security reasons for not doing so. Set the environment variable CATALINA_HOME to point to the directory where it is installed. You can also use the CATALINA_OPTS variable to set JVM options. This should be used to increase the available memory. My settings look like this:

CATALINA_HOME=/home/spire/hcssbld/tomcat
CATALINA_OPTS=-server -Xmx4096m

The global configuration files are in Tomcat's conf directory. You should only need to make minor changes to server.xml. The default port is 8080 and defined in the server.xml file. You can change this if you like, but running on a privileged port makes things a bit more complicated, see below.

User Access Control

I recommend enabling this for two reasons:

  1. If you do not your pools will be open to the world...
  2. It makes troubleshooting easier as you can see who is doing what in the logs.

There are several ways to set this up with Tomcat. I use the simplest, which is what is described here. Its main disadvantage is that it does not scale well to large numbers of users and major enterprise systems. See the Tomcat documentation for other possibilities.

Edit the server.xml file in the conf directory. Look for the Realm entry and specify digest as SHA. It should now look like this:

      <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
             resourceName="UserDatabase" digest="SHA"/>

Edit the tomcat-users.xml file in the conf directory. Add some roles, like this:

  <role rolename="spire_admin"/>
  <role rolename="spire_user"/>
You might want to leave the tomcat role as a placeholder for future general Tomcat admin.

This is how to add a user.

  1. Get a user name and password. Passwords should not be the same as login passwords, as while they are encrypted, the means used is not particularly secure. There are no restrictions on passwords. The main purpose is to offer some protection and log who is doing what and when.
  2. Encrypt the password. The command is java -cp $CATALINA_HOME/lib/catalina.jar:$CATALINA_HOME/bin/tomcat-juli.jar org.apache.catalina.realm.RealmBase -a SHA password. You may wish to define an alias for this...
  3. Edit the file tomcat-users.xml and add a new user line. password should be set to the encrypted password and roles should normally be set to the user role above (give yourself admin as well).

Adding a servlet

Servlets live inside the webapps directory. Applications have their own area in this directory and run in a sandbox isolated from the others. Note that a single application area can contain any number of servlets.

Now we go through how to set up an application called hcss. In the webapps directory, create a directory structure like this:

hcss - WEB-INF - lib
               - classes

The general idea is to put jar files in the lib directory and class files in classes. Everything you need should be in there. Don't assume that it will pick up your CLASSPATH environment variable - it won't.

Things are currently a little confused due to the transition period between the "new style" and the "continuous integration builds", and I am unsure about the best way to do some things right now. The following section mostly refers to the "new-style" builds.

Copy these libraries into the lib directory:

  • hcss-core.jar
  • dp-core.jar
  • dp-{inst}.jar
  • hcssExtLibs.jar (needs workaround first, see below)
  • jvi7.0.1-jdk1.4.jar (only required if you need database access)

These libraries should be updated whenever a new version of the HCSS is deployed, although in practice it usually continues working anyway (except for schema evolutions which kill database accessing servlets).

For the hcssExtLibs library, it is necessary to work around HcssSpr:5410. You can do it like this:

  1. Put the library in an empty temporary directory.
  2. Extract the contents with jar xf hcssExtLibs.jar.
  3. Delete the directory javax/servlet.
  4. Delete the jar file itself.
  5. Recreate it with jar cf hcssExtLibs.jar *

It is also necessary to create the application configuration file web.xml in WEB-INF. A version with a servlet defined is in the next section. You can also look at the versions of this file in the other applications that come preinstalled with Tomcat.

Adding the Http pool servlet

The servlet classes are contained in the hcss.services project of the CIB. It contains these modules:

  • ia_server_util (utility classes)
  • ia_pal_pool_http_server (the PAL HTTP server)
  • access_server (telemetry and data frame server)

These can be installed by either checking them out from CVS and compiling them, or using the CIB installer and extracting the jar files (see reference to current uncertainty above).

CVS method

This method is a little messy, but does have the advantage that it leaves you with an already set up developer environment, which is useful for troubleshooting.

From the classes directory, get the source code and put it in the right place, e.g.

cvs export -r D_IA_SERVER_UTIL_0_2 develop/main/herschel/ia/server/util
cp -r develop/main/herschel/ia/server herschel/ia

Now do the same for the ia_pal_pool_http_server module, e.g.

cvs export -r D_IA_PAL_POOL_HTTP_SERVER_0_2 develop/main/herschel/ia/pal/pool/http/server
mv develop/main/herschel/ia/pal herschel/ia

Then compile the source code (javac will do). The source files can be left there and as an aid to debugging, although Tomcat won't look at them.

CIB method

TBW

Application configuration

Now create the web.xml file in the WEB-APPS directory that will tell Tomcat what to do with all this lot. It should look something like this (you can paste this directly, but don't forget to change the role names to your instrument):

<!DOCTYPE web-app
    PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.2//EN"
    "http://java.sun.com/j2ee/dtds/web-app_2_2.dtd">

<web-app>

    <display-name>HCSS applications</display-name>
    <description>
    Servlets that use the HCSS.
    </description>

    <filter>
        <filter-name>GzipFilter</filter-name>
        <filter-class>herschel.ia.server.util.GzipFilter</filter-class>
    </filter>
    <filter-mapping>
        <filter-name>GzipFilter</filter-name>
        <url-pattern>/*</url-pattern>
    </filter-mapping>

    <servlet>
        <servlet-name>pal</servlet-name>
        <servlet-class>herschel.ia.pal.pool.http.server.HttpPoolServlet</servlet-class>
        <init-param>
            <param-name>allowLoadAccess</param-name>
            <param-value>true</param-value>
        </init-param>
        <init-param>
            <param-name>allowSaveAccess</param-name>
            <param-value>false</param-value>
        </init-param>
    </servlet>

    <servlet-mapping>
        <servlet-name>pal</servlet-name>
        <url-pattern>/pal</url-pattern>
    </servlet-mapping>

  <security-constraint>
    <web-resource-collection>
       <web-resource-name>Protected Area</web-resource-name>
       <url-pattern>/</url-pattern>
    </web-resource-collection>
    <auth-constraint>
       <!-- Anyone with one of the listed roles may access this area -->
       <role-name>spire_user</role-name>
       <role-name>spire_admin</role-name>
    </auth-constraint>
  </security-constraint>

  <!-- Default login configuration -->
  <login-config>
    <auth-method>BASIC</auth-method>
  </login-config>

  <!-- Security roles referenced by this web application -->
  <security-role>
    <role-name>spire_user</role-name>
    <role-name>spire_admin</role-name>
  </security-role>

</web-app>

This example does two things in addition to defining the servlet:

  1. It sets up user authentication for this area (note that different authentication can be applied to different application areas).
  2. It defines a compression filter. This is for performance reasons; the effect is TBC.

HCSS configuration

These properties should be set for the PAL server to work correctly. You can put them in user.props.

hcss.interpreter.python.path = {${var.hcss.dir}/lib}
hcss.ia.io.fits.handle_missing_classes = true

I have also found that I have needed to unpack the jar files in the HCSS installation (not in the Tomcat directory), in order to avoid an error like this from the server:

SEVERE: Failed to create pool Traceback (innermost last):
  File "<string>", line 1, in ?
ImportError: no module named herschel
This means that the above settings cause Jython to initialise from a different location to the Java classes. There may be a better way of doing this...

Additionally you may wish to configure pool locations etc, for example:

hcss.ia.pal.pool.lstore.dir=/misc/winchester1/data/products

Starting and stopping

The scripts can be found in Tomcat's bin directory. Use startup to start it, and shutdown to stop it. You may wish to check the catalina.out log to see if it started correctly.

Tomcat should automatically notice if a new servlet is defined and load it. You can force a servlet to reload by starting and stopping Tomcat. Wait a few seconds between stopping and starting or it might get confused (see next section). There is also a Tomcat "Manager" application that can do this from a web interface without stopping Tomcat - see the Tomcat documentation for details.

Occasionally it can get in a mess, which can be caused by things such as running out of memory, and refuse to restart. In this case you should see a message in the log along the lines of:

LifecycleException:  Protocol handler initialization failed: java.net.BindException: Address already in use:8080
In this case you need to kill the process (look for catalina) before restarting it.

Trying it out

Following the instructions above and starting Tomcat, we should now have a running PAL server with url http://whatever:8080/hcss/pal. Check the log for any initialisation errors. Here is an example Jython script for trying it out on the client side:

from herschel.ia.pal.pool.http import HttpClientPool
pool = HttpClientPool ("http://wakefield.bnsc.rl.ac.uk/hcss/pal", "sims3")
st = ProductStorage (pool)
st.authenticate()
r = st.select (Query (ObservationContext, "1"))

Troubleshooting

It is important to understand that the Tomcat scripts do not use the Java CLASSPATH environment variable. All required resources must be present in the classes or lib directories. For full details read the section on "Classloading" in the Tomcat documentation.

The first rule of troubleshooting is to check the log file. The principle log file is $CATALINA_HOME/logs/catalina.out.

Running on a privileged port

This is normally the standard HTTP port 80. The port is set by editing the server.xml file in the conf directory. I was not able to get the default Tomcat startup script to work when installed on this port. Here is my script:

#!/bin/sh
# Adapt the following lines to your configuration
JAVA_HOME=/home/spire/hcssbld/java/jdk1.6.0_10/
CATALINA_HOME=/home/spire/hcssbld/tomcat
TOMCAT_USER=hcssbld
TMP_DIR=/var/tmp
CATALINA_OPTS=-Xmx4096m
CLASSPATH=\
$JAVA_HOME/lib/tools.jar:\
$CATALINA_HOME/bin/commons-daemon.jar:\
$CATALINA_HOME/bin/bootstrap.jar

case "$1" in
  start)
    #
    # Start Tomcat
    #
    $CATALINA_HOME/bin/jsvc \
    -user $TOMCAT_USER \
    -home $JAVA_HOME \
    -Dcatalina.home=$CATALINA_HOME \
    -Djava.io.tmpdir=$TMP_DIR \
    -Djava.library.path=$VERSANT_ROOT/lib:$VERSANT_ROOT/lib/jvi/1p \
    -outfile $CATALINA_HOME/logs/catalina.out \
    -errfile '&1' \
    $CATALINA_OPTS \
    -cp $CLASSPATH \
    org.apache.catalina.startup.Bootstrap
    #
    # To get a verbose JVM
    #-verbose \
    # To get a debug of jsvc.
    #-debug \
    ;;

  stop)
    #
    # Stop Tomcat
    #
    PID=`cat /var/run/jsvc.pid`
    kill $PID
    ;;

  *)
    echo "Usage tomcat.sh start/stop"
    exit 1;;
esac

This script must be started with root privilege. Note that the daemon still runs as a less privileged user.

Further complications arise if there is a need to access the Versant library, as this accesses native shared libraries, e.g. for DbPool or the TM/data frame server. Whilst the startup script uses the java.library.path argument to pass the location of these libraries to the daemon process, this does not entirely work since the first called library then calls other libraries, which does not work since the process does not inherit the value of the LD_LIBRARY_PATH environment variable (it defines its own). A workaround solution is to link to the required libraries from the Java installation, which is accessible, e.g.

cd  $JAVA_HOME/jre/lib/amd64
ln -s $VERSANT_ROOT/lib/liboscfe.so liboscfe.so
ln -s $VERSANT_ROOT/lib/libxa.so libxa.so

Note that this is only necessary since the server is running as a daemon process in order to access the privileged port 80. There may well be a better solution than this.

Using the telemetry and data frame server

Put the classes for the access_server module in the hcss applications area, as for the PAL pool.

Then add this servlet definition to the web.xml file in WEB-APPS.

    <servlet>
        <servlet-name>tm</servlet-name>
        <servlet-class>herschel.access.server.HcssTmHttpServer</servlet-class>
    </servlet>
    <servlet-mapping>
        <servlet-name>tm</servlet-name>
        <url-pattern>/tm</url-pattern>
    </servlet-mapping>

This corresponds to the url http://whatever:8080/hcss/tm, assuming default settings.

This servlet has a feature to allow redirection of a database request to a specific server. This is done by means of HCSS properties. (It might be cleaner to redefine them as parameters specified in the web.xml file). The easiest way to set them is to put them in a HcssTmHttpServer.defaults file in the same directory as the corresponding class file. These are:

hcss.access.server.default
If a server is not specified, use this one. It itself defaults to localhost.
hcss.access.server.redirect
Redirect all database server requests to this one. The purpose of this is to ensure that operationally critical servers cannot be accessed by this process; the data of course has to be present on the server redirected to.

This property should also be set on the server side. This will be made the default for this application in the next version of the module. Note that this value is specific to the server and inappropriate for interactive use.

hcss.access.store = herschel.access.db.SimpleStoreHandler

This servlet is accessed by using the normal access API. These client-side properties need to be set:

hcss.access.connection=herschel.access.net.NetworkConnection
hcss.access.authentication=true
hcss.access.url=http://whatever:8080/hcss/tm

Convenience methods to set these properties dynamically are provided in the herschel.access.Access class.

-- SteveGuest - 28 Apr 2009

Edit | Attach | Watch | Print version | History: r34 | r10 < r9 < r8 < r7 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r8 - 2009-04-28 - SteveGuest
 
This site is powered by the TWiki collaboration platform Powered by Perl