$Id: SERVLETS,v 1.8 2002/04/09 17:51:49 ndp Exp $
----------------------------------------------------------------------------
Release 1.0.6 04/08/2002 (ndp)
Requirements:
You will need a servlet engine. I developed This release of Java-DODS using "Tomcat".
Tomcat:
First you will need to install a web server which supports servlets (aka a
servlet engine).
The current "reference standard" for this release is jakarta-tomcat-3.3a from
Apache. Go get it from: http://jakarta.apache.org/tomcat/
Follow the instructions and get it installed.
javax.servlet:
In order to compile the servlet code you will need the javax.servlet package.
You can get this from Sun (http://java.sun.com/products/servlet/2.1/index.html)
Or you can get it from inside the Tomcat distribution (TOMCATHOME/lib/common)
Either way you will need to get the javax.servlet jar file on your CLASSPATH
before you can compile the dods.servlet and dods.servers packages.
To build the servlets:
Once you've got the javax.servlet stuff and your CLASSPATH straightened
out a call to "make" in the top level directory will build everything.
To specifically build the server stuff do a "make servers" in the top
level directory.
Core Servlets:
The servlets distributed with the core Java-DODS software are:
DODS Test Server (DTS):
Class name: dods.servers.test.dts
This servlet is primarily used to test the
core code (in particular the DAP). This servlet will take any
DDS in it's DDScache (see below) and populate it with invented
data per client request. This allows the testing of unusual
DDS structures.
DODS Relational Database Server (DRDS):
Class name: dods.servers.sql.drds
This DODS server serves data from a relational database to DODS
clients. The DRDS uses the JDBC interface to connect to the
database. This means that to use this server you will need
to locate and install JDBC drivers for your database system.
This requirement is for the server only. The DODS clients that
use the server need know nothing about JDBC or SQL.
In this release the DRDS does not support SQL JOIN operations.
Each database table must appear as a DODS Sequence data type
in its own DDS file. If your data crosses multiple tables then you
will need to make a "view" or a "virtual table" in the database
in order to serve the data with the DRDS. This situation will improve
in the next major revision. (I have an as yet un-implemented plan
to allow the DRDS to support SQL JOIN operations.)
Installation:
The easiest way to install the distibuted servlets is to get
the "dods.war" file from:
ftp://ftp.unidata.ucar.edu/pub/dods/DODS-Java-1.1/
Or, you can build it by doing a "make war" in the Java-DODS
directory. This will make the "dods.war" file from the source
code and place it in the Java-DODS/webapps directory.
Put the "dods.war" file in appropriate directory of your
servlet engine. For tomacat3.3a you would place it in
$TOMCATHOME/webapps.
Restart the servlet engine (tomcat), this will unpack the
"dods.war" file into the directory $TOMCATHOME/webapps/dods
Once it's unpacked you will need to configure the servlets.
Go to the directory $TOMCATHOME/webapps/dods/WEB-INF and edit
the file "web.xml" Instructions for it's modifcation are in the
"Servlet Configuration" section below.
Once you have the "web.xml" file whipped into shape, restart the
servlet engine (tomcat) again. You should be up and running!
If things aren't working for you, try turning on the Debug flags
showRequest and showResponse. (See bleow)
If you get stuck contact support@unidata.ucar.edu
If they can't help you, they will find me (Nathan Potter), and
I will help you get it working.
Servlet Configuration:
The DODSServlet relies on the javax.servlet.ServletConfig
interface (in particular the getInitParameter() method)
to retrieve configuration information used by the servlet.
InitParameters common to all DODSServlets are:
DebugOn - This controls ouput to the terminal from which
the servlet engine was launched. The value is a list of
flags that turn on debugging instrumentation in different
parts of the code. Common values are : showRequest, showResponse
Other debugging values that are specific to each servlet should
be documented in each servlets javadoc documentation. Look in
Java-DODS/doc.
INFOcache - This is should be set to the directory containing the
files used by the ".info" service for the servlet. This directory
should contain any dataset specific "over-ride" files (see below),
any dataset specific additional information files (see below), and any
servlet specific information files(see below).
DDScache - This is should be set to the directory containing the DDS
files for the datasets used by the servlet. Some servlets have been
developed that do not use DDS's that are cached on the disk, however
the default behaviour is for the servlet to load DDS images from disk.
DAScache - This is should be set to the directory containing the DAS
files for the datasets used by the servlet. Some servlets have been
developed that do not use DAS's that are cached on the disk, however
the default behaviour is for the servlet to load DAS images from disk.
Here is an example entry from the web.xml file (for tomcat3.3a) for
the DODS Test Server (DTS):
dts
dods.servers.test.dts
DebugOn
showRequest showResponse
INFOcache
/usr/Java-DODS/sdds-testsuite/info/
DDScache
/usr/Java-DODS/sdds-testsuite/dds/
DAScache
/usr/Java-DODS/sdds-testsuite/das/
Other servlets have other (very important) InitParameters. The DODS Relational
Database Server (DRDS) relies on JDBC to make it's connections to DBMS's.
Specific InitParameters for the DRDS are:
JDBCdriver - This should be set to the java CLASSPATH name of the JDBC
drivers that the servlet is to use to make the JDBC connection to the
DBMS. For example, in my server that is using the Merant Data Direct
drivers the value is set to: "com.merant.sequelink.jdbc.SequeLinkDriver"
JDBCconnectionURL - This is the connection URL (aka the connection string)
that the DRDS is to use to connecto to the DBMS. This is usually defined
by the devlopers of the JDBC driver. It is not always easy to ascertain for
a particular installation. For example, in my server that is using the Merant Data Direct
drivers the value is set to: "jdbc:sequelink://sugar.oce.orst.edu:19996"
JDBCusername - This is the user name for the DBMS that the JDBC connection
will be made under. This is often set to "guest".
JDBCpassword - The password associated with the above username. This is stored
as simple text, so make sure that the JDBC user doesn't have any significant
privileges!
JDBCMaxResponseLength - This limits the number of lines that the DRDS will
for a given client request. For debugging I use 300. For production I use 300000.
UseDatasetName - This is a (probably temporary) hack. Some databases (MS-SQL
Server 7.0) require that the database name and the owner of the database be
specified in every variable and table name. This is akward for the current
implmentation of the DRDS. The work around is to name the dataset (in the DDS file)
with the database name and owner name of the table being served. For example in
one dataset that I server the database name is "EOSDB" and the owner of the
database is "DBO". For this database I set the value of UseDatasetName to true
and then I define the DDS as follows:
Dataset {
Sequence {
Float64 battery;
Float64 checksum;
Float64 data_age;
} Drifters;
} EOSDB.DBO;
Thus the hack is invoked. If you don't want to use this hack then DO NOT
even included the InitParameter "UseDatasetName" in the web.xml entry for
the DRDS.
Here is an example entry from the web.xml file (for tomcat3.3a) associated
with a DRDS servlet:
drds
dods.servers.sql.drds
JDBCdriver
com.merant.sequelink.jdbc.SequeLinkDriver
JDBCconnectionURL
jdbc:sequelink://sugar:19996
JDBCusername
guest
JDBCpassword
JDBCMaxResponseLength
300
UseDatasetName
INFOcache
/usr/Java-DODS/sdds-testsuite/info/
DDScache
/usr/Java-DODS/sdds-testsuite/dds/
DAScache
/usr/Java-DODS/sdds-testsuite/das/
DebugOn
showRequest showResponse JDBC
Additional Information Files:
The current implementaion of the DODSServlet provides a ".info" service in
addition to the ".dds", ".das", and ".dods" services. This ".info" service
provides an html document with human readable content. This document is a
synthesis of the contents of the DDS and the DAS. There are provisions for
adding additional information, and for overriding the default document. There
are three types of files used by the ".info" service to achieve this. They are:
Dataset Over-Ride files:
These files live in the INFOcache directory. To override the .info
service output for a particular dataset, place a file of html text
containing the overide document in the INFOcache directory. The file
should be a complete HTML document that can be rendered by most browsers.
The file should have the same name as the dataset with an additonal ".ovr"
suffix. For example, is my dataset is called "test.22" then the
over-ride file is called "test.22.ovr"
Dataset Additional Information Files:
These files live in the INFOcache directory. If you want to supplement
the ".info" service output provided by the servlet, place a file of html
text containing the additional information in the INFOcache directory.
This html text should be a snippet of html, not a complete document. It
will be inserted into the ".info" service response. It should not
contain
, , or other document definition tags.
The file should have the same name as the dataset with an additonal
".html" suffix. For example, is my dataset is called "test.22" then
the additional information file is called "test.22.html"
Servlet Information Files:
These files live in the INFOcache directory. They are used to provide
servlet specific information in the ".info" service responce. If you
write a servlet and want to add information to the ".info" service
response about your servlets idiosyncrasies, place a file of html
text containing the additional information in the INFOcache directory.
This html text should be a snippet of html, not a complete document. It
will be inserted into the ".info" service response. It should not
contain , , or other document definition tags.
The file should have the same name as the java CLASSPATH of the servlet
followed by a ".html" suffix. For example the DODS Test Servlet (DTS)
that I developed lives in "Java-DODS/dods/servers/dts". Its class name
is "dods.servers.test.dts" Thus, its servlet information file is called
"dods.servers.test.dts.html"
Examples of all of these files can be found in the directory
"Java-DODS/sdds-testsuite/info" in the full distribution of the Java-DODS core.
Legacy code:
Jake,s original servlet code still exists in dods.servlet.jake but we
have basically scrapped Jake's original design in favor of the current
one in dods.servlet.
4/8/02 ndp
----------------------------------------------------------------------------
10/19/98 jhrg
This is the text of an e-mail from Jake Hamby which describes how to install
the DODS Java servlets. Note that he mentions that the `Server:' header we're
using is also used by the Java Servlets. We will change from `Server:' to
`XDODS-Server:' around release 2.19-2.20. 10/19/98 jhrg
First, you will need to install a web server which supports servlets.
Previously, I had used JRun with somewhat mixed success (it worked okay
on Solaris, but for some reason, I was unable to install it on IRIX the
last time I tried). This time, I thought I'd install JServ, the
official servlet module from http://java.apache.org/. Instead of
getting the 0.9.11 version, which is many months old, I skipped ahead
and downloaded the latest daily build. This requires you to get a copy
of Sun's servlet developer kit (JSDK 2.0), and is not the simplest thing
in the world to get configured and running (I found a few setup bugs
which may be fixed by the time you download it). However, since these
servlets are very generic, you should be able to use JRun, Sun's Java
Webserver, or anything else you choose, although the installation
instructions will be slightly different, of course.
Anyway, I haven't added a Makefile target to compile the servlets,
because you'll need to make sure the servlet classes are in your
CLASSPATH first. In the case of Sun's JSDK, you'll need to add jsdk.jar
to your CLASSPATH. Once you've done this, typing "javac
dods/server/*.java" should do the trick. I did add the package to "make
doc", however.
Now, you'll need to install the servlets. Make sure you can run some of
the example servlets, like SimpleServlet, before you try this. I just
made a symlink from the servlets directory to the dods directory
containing the Java classes, so that the classes could be found in
/usr/local/apache/servlets/dods/server/*. Now, you'll need to tell the
server which servlets are installed and what parameters to use. In the
case of JServ (at least the beta version I used), there's a
zone.properties file, which you can use to create aliases (which is
necessary since the servlets are in a package, not the main directory)
and startup parameters (only necessary for the proxy server). Here's
what I used:
servlet.proxy.code=dods.server.ProxyServlet
servlet.proxy.initArgs=manta=http://manta.jpl.nasa.gov/cgi-bin/nph-hdf
servlet.dap.code=dods.server.DAPServlet
Now, assuming you can run the servlets, what do they do? DAPServlet
fakes a real DODS server by reading the DAS, DDS, and data from separate
files in the document root. If you access:
http://localhost/servlets/dap/foo.hdf
it will try to find foo.hdf.das, foo.hdf.dds, and foo.hdf.dods for DAS,
DDS, and Data requests, respectively. The .dods file is a binary
dataset file (compressed or uncompressed) as created by
dods.client.Geturl -p, or by calling an existing DODS server CGI by
hand.
ProxyServlet looks at the first component of its path, and searches for
it in its list of arguments. If it finds a match, it constructs a new
URL to retrieve the information from, then sends it back out to the
client. For example,
http://localhost/servlets/proxy/manta/foo.hdf
turns into:
http://manta.jpl.nasa.gov/cgi-bin/nph-hdf/foo.hdf
When testing this, I discovered one big problem, which only affects
sequences: Apache won't normally allow you to override the "Server:"
field (I'm guessing that the DODS CGI's can do it only because they
construct their own headers). Thus, even though I set the Server header
in DispatchServlet (the parent of both servlets), it wasn't working, and
so clients assumed the old sequence deserialize protocol instead of the
new. I've attached a patch to Apache's http_protocol.c which fixes
this. I don't know if you'll have the same problem with JRun (although
I'd expect you would, and I'd expect the attached patch to fix it there
too), and I don't know if other servlet engines, such as Sun's Java
Webserver allow you to override Server or not.
I guess that's about it. If you have any problems, let me know.
Naturally, these servlets are pretty naive and don't perform any caching
(which is one thing you would probably expect a proxy server to do). If
you plan to build on this framework, it might be a good idea to build
some sort of generic caching mechanism that all DODS servers could use.
Of course you have to be careful about threadsafety because doGet() can
be called from several threads at once.
As for adding the constraint evaluator, I suggest you don't add any
state to the regular BaseType classes. My fear is that, if you do start
caching DDS's, two simultaneous client requests might want to access
different parts of the same dataset, and then you couldn't share a
single copy of that dataset in the cache. Since a big benefit of
servlets are that they're multithreaded, it would be a good idea to
handle this case properly. One approach would be to have some sort of
shadow BaseType classes which point to the real data, and contain an
extra boolean for whether to send that variable. Then you could call
serialize on the shadow class and expect it to do the right thing. The
downside of this approach is that you'd need to write a shadow class for
each DODS type, and override the serialize methods to work properly.
You'd also need a method to set up an entire shadow class hierarchy for
each simultaneous client request (of course, when you're finished, you
could put them into a pool for the next client to reuse, instead of
throwing it away and making a new one each time). The only other
possibility I could think of would be to keep all the constraint
information in a single object (let's call it CE), which lists all
variables to send, then pass that object as an extra argument to the
serialize method. The only problem with that is if you stored the
variable list by name, how could you easily find the full name of a
given variable from inside its serialize method? Since there's no
getParent method, you probably couldn't. Perhaps you could just store a
reference to the BaseType inside the CE object, instead of the name:
then you'd get the added benefit that comparison would be much faster
than any String compare. Does this make sense?
-Jake
--------------474CBB33E22F57292E88B46B
Content-Type: text/plain; charset=us-ascii;
name="patch1"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="patch1"
*** http_protocol.c.orig Tue Oct 13 17:19:43 1998
--- http_protocol.c Tue Oct 13 17:24:31 1998
***************
*** 1160,1165 ****
--- 1160,1166 ----
API_EXPORT(void) ap_basic_http_header(request_rec *r)
{
char *protocol;
+ const char *server;
#ifdef CHARSET_EBCDIC
int convert = ap_bgetflag(r->connection->client, B_EBCDIC2ASCII);
#endif /*CHARSET_EBCDIC*/
***************
*** 1193,1199 ****
protocol, " ", r->status_line, "\015\012", NULL);
ap_send_header_field(r, "Date", ap_gm_timestr_822(r->pool, r->request_time));
! ap_send_header_field(r, "Server", ap_get_server_version());
ap_table_unset(r->headers_out, "Date"); /* Avoid bogosity */
ap_table_unset(r->headers_out, "Server");
--- 1194,1203 ----
protocol, " ", r->status_line, "\015\012", NULL);
ap_send_header_field(r, "Date", ap_gm_timestr_822(r->pool, r->request_time));
! /* allow servlets to override Server field if desired
! */
! server = ap_table_get(r->headers_out, "Server");
! ap_send_header_field(r, "Server", server ? server : ap_get_server_version());
ap_table_unset(r->headers_out, "Date"); /* Avoid bogosity */
ap_table_unset(r->headers_out, "Server");
--------------474CBB33E22F57292E88B46B--