OPeNDAP is pleased to announce Hyrax 1.8!

Submitted by jimg on Mon, 02/13/2012 - 13:36

Hyrax by OPeNDAP logo Updated 14 June 2013.

FITS Handler bug fix

A problem in the FITS handler that prevented it from reading attributes from many FITS files has been fixed. A second problem that resulted in a memory error deep in the cfitsio library has also been fixed. The new version is labeled 1.0.9 and is available in source form. If you need binaries, please let us know.

New for the 1.8.8 release:

This release includes improved support for both hdf4 and hdf5, particularly when requesting netCDF files as responses.

Also included in this release is a beta release of software to support the GDS/F-TDS server expression syntax. It is easy to use this syntax to run server functions defined in/by hyrax. Also included in this beta release is a new expression caching system. To turn this cache on, simply make a directory for the cache. Email us for details if you want to participate in the beta test!

All new RPMs for CentOS 5 and CentOS 6 are included in this release, as well as the source distributions.

New for the 1.8.7 release:

This release includes an update for the HDF5 handler to version 2.1.0. This version improves the performance of the handler when dealing with big files with thousands of internal datasets. Version 2.0.0 of the handler introduced a re-engineered version of the CF option. Now the CF option is on by default and instead of being a compile-time option, this can be toggled at run-time using a server parameter. Also, the handler includes a new parameter that can be used to control the parsing of large 'Structure Metadata' records that are often not needed by end users of the data but which take up significant space in responses.

New for the 1.8.6 release:

This release is a both a fix for the new GeoTiff/GRiB/JP2 (GDAL) handler and an update for the HDF4 handler.

The fix for the GDAL handler (0.9.2) addresses a build issue and requires the updated BES (version 3.11.0). The updated libdap (version 3.11.5) is suggested as well since it fixes a memory leak triggered when the server returns XML responses. The GDAL handler is still beta software. We'd really like feedback on it, especially WRT it's representation of GRiB2 data.

The updates to the HDF4 handler (3.10.0) include much better support for CF conventions (the handler translates existing metadata into CF!) configured using a Hyrax run-time configuration parameter (no more compile-time switches) and works both with and without the hdf-eos2 library. Thanks to THG for this update!

New for the 1.8.5 release:

This release adds beta support for GeoTiff, GRiB2 and JPEG2000 files, based on the GDAL 1.9.1 library. The handler is available as source only for now. Please let us know what additional features you would like for this new handler. We are trying a new way of bundling the handler's dependencies with the source, so that you do not have to get a separate copy of GDAL to build the handler. If you already have a copy of the GDAL library, the handler's configure script will automatically find and use that.

New for the 1.8.4 release:

This release of Hyrax addresses a security problem. Only the OLFS needs to be changed. Simply download the OLFS, Unpack using 'tar -xvf filename' and follow the instructions in the README file. Here's a direct link to the upgraded OLFS (sig).

New for the 1.8.3 release:

A major rewrite for the HDF5 handler. See the README for more details; here's a summary:

  • No more compile-time options - now use run-time options settings to control the handler
  • Better support for the CF conventions (EnableCF run-time option, etc.)
  • Better integration with the DAP2 data model
  • The RPM (CentOS 6 only) works with your existing Hyrax 1.8 binaries
New for the 1.8.2 release:

An updated OLFS (1.9.3) package provides the following bug fixes:

  • Repairs problems with HTTP headers returned for simple files. (Ticket 1890)
  • Corrects a problem that was generating errors during shutdown. (Ticket 1887)
  • Corrects problems with updating the in memory representation of THREDDS catalogs when the on disk (static) catalogs files are changed. (Ticket 1891)

Note: If you already have the Hyrax-1.8.1 release you can upgrade to Hyrax-1.8.2 simply by upgrading the OLFS component to OLFS-1.9.3

Features and fixes inherited from the 1.8.1 release
  • CentOS 6 RPMs (for 64-bit only, unless we get requests for 32-bit).
  • We've dropped support for the 32-bit CentOS 5.x platform, although we can make those RPMs available if they are needed.
  • The BES did not install from RPMs correctly on CentOS; it could not create the bes.pid file or the bes.log file.
  • On CentOS 6, the yum package manager installs netCDF 4.1.1 which the handler did not detect.
  • Hyrax used port 10002 for inter communications but this port is a well-known port for some linux distributions; we've changed the default to 10022.
Features and fixes inherited from the 1.8.0 release:
  • RDF output for metadata
  • XML response for data: it's now possible to request data encoded in XML (as ASCII). This is primarily intended for use in semantic applications although it likely has many other uses. ...Not intended for large data volumes!
  • FITS: We have added FITS format support.
  • New Administrator's interface! Start, stop, control logging and more from a browser.
  • Beta: ISO 19115 output for metadata plus a Rubric that indicates the degree of conformance with that particular standard. We have an update pending for this that will be out in a few weeks. At that time, all you'll need to do is drop an new opendap.war file in the tomcat webapps directory - easy! Until then, the iso and rubric links won't show up in a browser, but you can get these responses by adding '.iso' and '.rubric' at the end of the URL in a browser.
  • Updated online documentation for the server and for specific handlers.
Features and fixes for specific formats or modules
HDF4
  • Fixed the bug in uint16/uint32 type attribute handling.
  • Correced the handling the scale/offset for MODIS products. The MODIS scale/offset equation is quite different from the CF standard.
  • Added the support for LAMAZ projection data.
  • Fixed memory release error that occurs on iMac (OS X Lion) with the STL string:string Map.
HDF5
  • Updated for version 2.0 of the handler, released on 6/14/12. Works with the existing Hyrax 1.8 binaries.
  • The H5.IgnoreUnknownTypes run-time configuration parameter has been replaced by EnableCF. See the README and INSTALL files for more information.
  • The int8 to int16 conversion is now handled correctly.
NetCDF 3 and 4
  • The handler now reads all netcdf 4 files and externalizes all but a very small number of its native data types (64-bit ints, vlens and groups are yet to be supported).
  • Also added to the handler are several configuration parameters. Briefly, they are:
    • IgnoreUnknownTypes
    • ShowSharedDimensions
    • PromoteByteToShort
NCML
  • Fixes for Grids so that their Map vectors now appear as top-level variables to mimic the 'shared dimensions' hack used to make older netcdf-based clients work.
  • Added a new configuration parameter to address the question about just what to do with top-level (global) attributes. The parameter is named NCML.GlobalAttributesContainerName and if present and set to a value, a container of that name will be used to hold all of the top-level attributes.
Gateway
  • Added NoProxy switch to the handler's conf file so that it can be configured to not use the proxy server for certain hosts (e.g., localhost).

Required External Dependencies

In order to get the basic Hyrax 1.8.0 server running, you will need:

  • Java 1.6
  • Tomcat 6.x
  • Individual handlers may have their own external dependencies that you can satisfy with YUM/RPM on linux. For OS/X, our packages will contain references to all of the required dependencies. Roughly speaking, for a complete server hears what you'll need:
    • libxml2, libcurl, libreadline
    • hdf4, hdf5, netcdf4, icu, cfits - these you needs only if you want to read data from the corresponding handler. That said, on any of RedHat family of Linux distributions, all you need to do is type 'yum install hdf hdf5 netcdf4 icu'

What To Get Here

To run the Hyrax server, you will need to download and install the following (from source or binary):

  • OLFS (which is a Java binary and runs on any computer with Java 1.6)
  • libdap
  • BES
  • General Purpose Handlers (aka dap-server)
  • One or more data handlers depending on which data you wish to serve
  • Our software distributions are signed using a public/private key pair. See Public Key for more information.

Binaries

Java icon Java
  • Updated OLFS 1.9.5 Web Archive File (gpg signature) Unpack using 'tar -xvf filename' and follow the instructions in the README file. (Requires Java 1.6. Built using Java 6. Tested against Tomcat 6.0.x) - Note: If you are upgrading an exisiting installation you will need to update your olfs.xml file to reflect the many changes in the OLFS. The new file can be found here $CATALINA_HOME/webapps/opendap/initialContent/olfs.xml. Localize it to reflect your BES configuration and other options and copy it to $CATALINA_HOME/content/opendap/olfs.xml.
Linux Tux Logo Linux (CentOS 6) x86_64 RPMs <(all RPMs new for Hyrax 1.8.8)
Linux Tux Logo Linux (CentOS 5.7) x86_64 RPMs (All RPMS new for 1.8.8)

Source code

*All updated for 1.8.8*