Back to top

OPeNDAP is pleased to announce Hyrax 1.13.1

Submitted by jimg on Wed, 06/08/2016 - 08:52

Hyrax by OPeNDAP logo Hyrax 1.13.1 release includes performance improvements, data handling improvements, improved return as netCDF functionality, bug fixes, and new developer features.

Server Documentation

1.13.1 release (6 June 2016)

Welcome to the Hyrax 1.13.1 release. This release contains new features, and bug patches.

New Features & Fixes
General improvements.
Added memory cache for BES Catalog responses. This improves the performance of the server in certain situations, particularly those where a large number of small requests are being made. The size and refresh interval are specified by the "<CatalogCache>" element in the olfs.xml file. If the element is missing then the memory cache will be disabled.
  • Increased the default number of simultaneous requests from 10 to 200 (the latter is the default limit for Tomcat).
  • Fixed large file issues in w10n.
  • Fixed build for WCS-2.0
netCDF file responses.
In previous versions of Hyrax, netCDF files that can be used to return data did not include information about their provenance. Now we include a 'history' attribute as described by the CF-1.6 specification.
  • The server now has an option (on by default) where a dataset's existing extension is replaced by 'nc (or nc4) instead of always appending 'nc'. The server can be switched back to the old behavior using an option set in the olfs.xml configuration file. This feature actually works with all of the different 'file format responses' of Hyrax (netcdf3, netcdf4, geotiff, jpeg2000).
HDF5 Support.
Add support for HDF5 scalar datasets and unlimited dimensions to the CF option.
HDF4 Support.
Improved the calculation of XDim and YDim for the sinusoidal projection and a bug fix for the case when the _FillValue is NaN.
WMS Support
The ncWMS project's latest product, ncWMS2 can be configured to work with Hyrax and provide WMS services for conformant datasets. (More about the configuration steps here.)
For developers:
Added an in-memory cache that handlers can use to cache DDS, DMR, ..., objects. The handlers have to be modified to use this, but the performance benefits are potentially huge. See bes/dap/ObjMemCache and the netcdf_handler for the code and an example of its use.
  • Server functions can now return multiple variables and have then appear as discrete things without being 'wrapped' in a parent Structure variable. They do this by using a structure name that ends with '_unwrap'. This means that server functions can return values in a way that is usable by clients built using the netCDF library (i.e., Matlab, IDL, ArcGIS, Ferret).
  • Added support for the version response to the to 'functions' module (in 'bes/functions') so that the server will report on its presence. This could be simple code can be used as a template and propagated to other modules that previously lacked a version response, simplifying many debugging tasks.
  • Patches for gcc-6 in both libdap and the BES (submitted by Orion Poplawski - thanks).

Required External Dependencies

In order to get Hyrax 1.13.1 running, you will need:

  • Java 1.7 or greater
  • Tomcat 7.x or 8.x
  • Linux (We provide RPMs for CentOS 6.6; install them with yum); OSX source build using the optional hyrax-dependencies.tar sources or your own copies of HDF4, etc.

Software downloads for Hyrax 1.13.1

To run the Hyrax server, you will need to download and install the following (from source or binary):

  • OLFS (Java 1.7)
  • libdap
  • BES
  • ncWMS2 (optional)

Binaries for Hyrax 1.13.1

Java icon OLFS (Java-1.7)
  • OLFS 1.16.0 Web Archive File (gpg signature) Unpack using 'tar -xvf filename' and follow the instructions in the README file. (Requires Java 1.7 Built using Java 8 Tested against Tomcat 8.0.32).
  • CentOS 6.3 comes with Java 6; Oracle has stopped supporting that version of Java; update your machine if you haven't already. Use yum search java-1.7 to find the correct package. For our servlet you will only need the JRE - the Java Runtime Environment. You will also need to employ the "alternatives" tool to make java 7 JVM the preferred alternative.
Java icon ncWMS2 (Java-1.7)
Use the EDAL web page to locate the latest ncWMS2 "Servlet Container" software bundle as a WAR file. Install it into the same Tomcat instance as the OLFS. The configuration instructions may be found here.
Linux Tux Logo BES
Linux (CentOS 6.6) x86_64 RPMs - All of the RPMs we build, including the devel and debuginfo packages
  • libdap-3.18.0 (gpg signature)
  • bes-3.17.2-static (gpg signature) This RPM includes statically linked copies of all of the modules/handlers we support, including HDF4 & 5 with HDFEOS support. There is no need to install packages from EPEL with this RPM. Other sources of RPM packages will likely provide a bes RPM that uses handlers linked (dynamically) to dependencies from their distributions (CentOS, Fedora, etc.). RPMFind is a good tool to locate those RPMs.
Linux (CentOS 7.1) x86_64 RPMs - The RPM packages for CentOS 7/EL 7, otherwise, the same code as above.

Installing the binary distribution

  • Download the RPM packages above
  • Use yum to install the libdap and bes RPMs (sudo yum install libdap-3.18*.rpm bes-3.17.2*.rpm). Unless you're going to be building software from source for Hyrax, skip the *-devel and *-debuginfo RPMs.
  • At this point you can test the BES by typing the following into a terminal:
    • start it: sudo service besd start or use the script in /etc/init.d with sudo /etc/init.d/besd start
    • connect using a simple client: bescmdln
    • and get version information: show version;
    • exit from bescmdln
  • BES NOTE - If you are upgrading to Hyrax 1.13.1 from an existing installation older than 1.13.0
    In the bes.conf file the keys BES.CacheDir, BES.CacheSize, and BES.CachePrefix have been replaced with BES.UncompressCache.dir, BES.UncompressCache.size, and BES.UncompressCache.prefix respectively. Other changes include the gateway cache configuration (gateway.conf) which now uses the keys Gateway.Cache.dir, Gateway.Cache.size, and Gateway.Cache.prefix to configure its cache. Changing the names enabled the BES to use separate parameters for each of its several caches, which fixes the problem of 'cache collisions.'
  • Check the java version on your system: "java -version"
  • If you have earlier than 1.7.x then you'll need to install java 7:
    • yum install java-1.7.0
    • If you are on a modern-ish Linux system you may need to use the alternatives tool to make java 7 the current JVM. Run the command:
      "alternatives --config java"
      You'll get an interactive terminal application that will allow you to select your newly installed java 8 as the preferred alternative.
  • Get Apache Tomcat-7.x if you don't already have it.
  • Download the OLFS Web ARchive (war) file (see above, by the Java logo) and put opendap.war in the tomcat webapps directory (cp opendap.war apache-tomcat-7.0.59/webapps). Alternatively, use yum to install tomcat.noarch and put the opendap.war file in /usr/share/tomcat/webaps. Make the directory /etc/olfs, change its group to tomcat, and set it 'group writable' (mkdir /etc/olfs; chgrp tomcat /etc/olfs; chmod g+w /etc/olfs).
  • At this point you should be able to start tomcat (./apache-tomcat-7.0.59/bin/startup.sh or sudo service tomcat start) and test the server:
    • In a web browser, use http://localhost:8080/opendap/
    • Look at sample data files shipped with the server
  • If you are installing the OLFS in conjunction with ncWMS2 version 2.0 or higher: Copy both the opendap.war and the ncWMS2.war files into the Tomcat webapps directory. (Re)Start Tomcat. Go read about, and then configure ncWMS2 and the OLFS to work together.
  • From here, or if you are having problems, see our Hyrax documentation page
  • OLFS Note - If you are upgrading to Hyrax 1.13.1 from any previous installation
    The OLFS now has much better support for sites that use rpm to install Tomcat. To complement this, it has a new procedure for locating its configuration directory. During start up, the OLFS looks in two places for its configuration information. It checks the directory given by the environment variable OLFS_CONFIG_DIR first and then the directory /etc/olfs/, using the configuration information in the first place it is found. If either of those directories exist but are empty, the OLFS will unpack its default configuration information there. You can customize the configuration information using a text editor; the OLFS will not overwrite your changes. If neither OLFS_CONFIG_DIR nor /etc/olfs/ exist, the OLFS will fall back to its default configuration. Note that the OLFS_CONFIF_DIR or /etc/olfs must be writable by the tomcat user for this to work.
  • In order for the CatalogCache feature to be enabled you must update your olfs.xml file. Use the one located in $CATALINA_HOME/webapps/opendap/WEB-INF/conf as a template for your new one.

Source code for Hyrax 1.13.1

Source from GitHub

  • All of our source code is on GitHub. There you will find the hyrax repo, which is a meta-project that contains scripts to clone and build all of Hyrax. You will also see all of the repos that contain the Hyrax source code (libdap4, bes, and all of the handlers).
  • Directions on building Hyrax from GitHub are available at our documentation site.