Hyrax 1.11 release

Submitted by jimg on Thu, 05/07/2015 - 15:23

Hyrax by OPeNDAP logo OpenDAP is pleased to announce Hyrax 1.11.2!

1.11.2 release (07 May 2015)

The 1.11.2 release of Hyrax is a bug fix for the new THREDDS datasetScan implementation.

This incremental release contains changes to the OLFS only, so you do not need to update existing BES installations to take advantage of it.

1.11.1 release (27 April 2015)

The 1.11.1 release of Hyrax includes a new implementation of the THREDDS datasetScan feature. All of the features of datasetScan described in the TDS documentation are now supported. Additionally, non UTF-8 characters have been removed from Hyrax web output, and many of the pages have been streamlined.

This incremental release contains changes to the OLFS only, so you do not need to update existing BES installations to take advantage of it.

1.11.0 release (07 Apr. 2015)

Welcome to the Hyrax 1.11.0 release. This release contains numerous new features, bug patches, and security modifications.

New Features

  • Swath Aggregation Service - We have added a new Web service for aggregation to Hyrax. This service can be used to build aggregations of satellite swath data using lists of granules passed to the service from a client. While usable by anyone, this was specifically designed for use by search systems that need to return information spanning many granules/files when those either do not, or cannot, have aggregations defined by data providers using NcML. The service can be used with any multi-file data; it is not limited to swath data. As part of this service, several server functions are also being made available. For more information, see the Aggregation Enhancements documentation.
    • Added server functions: tabular(), roi(), bbox(), and bbox_union() as part of the new Swath Aggregation features.
  • w10n - This release contains an implementation of the Webification (w10n) navigation and data access service. This service provides seamless navigation and data access using JSON responses from the server. The w10n specification can be found here. More about how w10n can be useful can be found here.
  • WMS Service - With this release we are making available a version of the ncWMS WMS service with the Godiva web client. Based almost entirely on code from the good people at University of Reading, the instance of ncWMS that we have produced works with Hyrax to provide WMS services.
  • Ugrid Subsetting - With this release we are including a new server side function that can be used to subset an unstructured grid (aka irregular mesh) as long as that ugrid matches the specification being developed by ASA, inc. See the online documentation for the ugrid subsetting function. More information about server side functions can be found here.

  • DAP4 Implementation - This version of the BES includes support for DAP4. In addition, the following data readers (handlers) are now DAP4 capable:

    • csv_handler
    • freeform_handler
    • gdal_handler (geotiff, jpeg2000, other formats)
    • netcdf_handler
    • hdf5_handler
    • fits_handler
  • Simplified binary installation - We have changed the packaging of the source and binary distributions for Hyrax. Instead of using a separate package for each component of the server, we've condensed the BES daemon and the handlers we consider part of the 'standard' Hyrax release into one RPM and one corresponding source tar.gz distribution. The libdap RPM and source tar.gz are still separate as are the OLFS binary and source. In practice, this means that only three packages must be downloaded and installed instead of 18.
    • We have built a special RPM that includes many handlers statically linked with third-party code that either has no RPM or is very hard to find. This RPM, for the bes, has the name 'NASA' as a suffix to the RPM version information. Thus our handlers for HDF4/HDFEOS, Ugrid subsetting, GeoTiff, and JPEG200 work without any extra installation steps.
    • We've also included an experimental set of Debian packages for use on Ubuntu 14 and related distributions.

Patches for specific formats or modules

  • The asciival handler was updated so that server functions will now correctly return ASCII responses.
  • The HTML data request form was patched using a contribution from LARC.
  • The fileout_json now handles arrays of strings properly.
  • The fileout_netcdf handler has a contributed fix from NSIDC:
    • Convert OPeNDAP Byte array data to NetCDF NC_SHORT. The handler's output was interpreting unsigned Bytes from OPeNDAP as signed bytes (NC_BYTE). To avoid misrepresenting data and retain compatibility with NetCDF3/classic we changed the handler to output this data as a NC_SHORT.
  • The besd script has been fixed and is now installed correctly by the RPM package so that the bes can easily be configured to start at boot time.
  • The code now builds using autoconf 2.63; automake 1.11.1 and libtool 2.2.6. These are the versions found on a RHEL 6 machine.
  • The built-in 'dapreader' handler can now be used to reduce interdependencies between handlers when writing regression and integration tests.

Required External Dependencies

In order to get Hyrax 1.11 running, you will need:

  • Java 1.7 or greater
  • Tomcat 7.x (Not tested with Tomcat 8)
  • Individual handlers may have their own external dependencies that you can satisfy with YUM/RPM on Linux (Use yum install *.rpm and Yum will take care of getting the required packages if you've set up the EPEL repository. To set that up see the EPEL home page or just install its rpm if you're using CentOS 6.)

Software downloads for Hyrax 1.11

To run the Hyrax server, you will need to download and install the following (from source or binary):

  • OLFS (Java 1.7)
  • libdap
  • BES
  • ncWMS (optional)

Binaries for Hyrax 1.11

Java icon OLFS (Java-1.7)
  • OLFS 1.13.2 Web Archive File (gpg signature) Unpack using 'tar -xvf filename' and follow the instructions in the README file. (Requires Java 1.7. Built using Java 7 Tested against Tomcat 7.0.33). Note: If you are upgrading an existing installation you will need to update your olfs.xml file to reflect the many changes in the OLFS. Additionally you will need to copy any file in $CATALINA_HOME/webapps/opendap/initialContent not present in $CATALINA_HOME/content/opendap to said direcotry. The new olfs.xml file can be found here $CATALINA_HOME/webapps/opendap/initialContent/olfs.xml. Localize it to reflect your BES configuration and other options and copy it to $CATALINA_HOME/content/opendap/olfs.xml.
  • CentOS 6.3 comes with Java 6; Oracle has stopped supporting that version of Java; update your machine if you haven't already. Use yum search java-1.7 to find the correct package. For our servlet you will only need the JRE - the Java Runtime Environment. You will also need to the "alternatives" tool to make java 7 JVM the preferred alternative.
Java icon OLFS+ncWMS (Java-1.7)
OLFS 1.13.2 + ncWMS 1.3 Web Archive Files (gpg signature) Unpack using 'tar -xvf filename' and follow the instructions in the README file. (Requires Java 1.7. Built using Java 7 Tested against Tomcat 7.0.33).
Linux Tux Logo BES
Linux (CentOS 6.3) x86_64 RPMs
libdap-3.14 (gpg signature)
bes-3.14 (gpg signature) or bes-3.14-NASA (gpg signature) Which includes handlers that support HDF4/HDFEOS2, etc., as noted above.
Experimental debian packages for Ubuntu 14
libdap-3.14 (gpg signature)
bes-3.14-NASA (gpg signature)

Installing the binary distribution

  • Download the RPM packages above

  • If your CentOS/Fedora/RHEL 6 machine does not have the EPEL repository configured (rpm -q epel-release will tell if that has been configured) then get the Extra Packages For Linux (EPEL) rpm and install it using sudo rpm -ivh epel-*.rpm Be careful to use the -i option option. Also note that some CentOS 6 images (e.g., Amazon AMIs) ship with EPEL installed but disabled. If that's the case, run yum using --enablerepo=epel​ or ​Modify /etc/yum.repos.d/epel.repo. Under the section marked [epel], change enabled=0 to enabled=1. (see the Amazon Linux AMI FAQ for more information).

  • Use yum to install the libdap and bes RPMs (sudo yum install libdap-3.14*.rpm bes-3.14*.rpm). Unless you're going to be developing software for Hyrax, skip the libdap-devel and bes-devel since those are for development only.
  • At this point you can test the BES by typing the following into a terminal:
    • start it: sudo besctl start
    • connect using a simple client: bescmdln
    • and get version information: show version;
    • exit from bescmdln
  • Check the java version on your system: "java -version"
  • If you have earlier than 1.7.x then you'll need to install java 7:
    • yum install java-1.7.0
    • If you are on a modernish Linux system you may need to use the alternatives tool to make java 7 the current JVM. Run the command:
      "alternatives --config java"
      You'll get an interactive terminal application that will allow you to select your newly installed java 8 as the preferred alternative.
  • Get Apache Tomcat-7.x if you don't already have it.
  • Download the OLFS Web ARchive (war) file (see above, by the Java logo) and put opendap.war in the tomcat webapps directory (cp opendap.war apache-tomcat-7.0.59/webapps).
  • At this point you should be able to start tomcat (./apache-tomcat-7.0.59/bin/startup.sh) and test the server:
    • In a web browser, use http://localhost:8080/opendap/
    • Look at sample data files shipped with the server
  • If you are installing the OLFS/ncWMS bundle, copy both the opendap.war and the ncWMS.war files in the Tomcat webapps directory. (Re)Start Tomcat. Go read about, and then configure ncWMS and the OLFS to work together.
  • From here, or if you are having problems, see our Hyrax documentation page

Source code for Hyrax 1.11

Source from GitHub

  • All of our source code is on GitHub. There you will find the hyrax repo, which is a meta-project that contains scripts to clone and build all of Hyrax. You will also see all of the repos that contain the Hyrax source code (libdap4, bes, and all of the handlers).
  • Directions on building Hyrax from GitHub are available at our documentation site.