Download Hyrax-1.17.0

James Gallagher

| Jan 25, 2024

Hyrax is open source, giving you the flexibility to choose how you install and run it. We use continuous integration and continuous delivery (CI/CD) to build our software for every change of the source code.

Citing Hyrax

To cite Hyrax 1.17.0 use

Broadly, Hyrax 1.17.0 is comprised of

Download Options

We offer several options to suit your needs:

1) Docker Images

2) Linux Binaries

3) Source Code

1) Docker Images

Simplest way to deploy Hyrax as it eliminates compatibility issues with your operating system and other software by creating a self-contained environment. You only need the Docker software itself, available for Mac, Windows, and Linux. There are various images options to chose from:

Hyrax – (Most stable). The complete Hyrax service in a single Docker image.
Hyrax with ncWMS2 – (optional) The Hyrax service bundled with ncWMS2 in a single Docker image.
Full list of Snapshot docker images (Less stable) – These builds are pushed to our Docker Hub site and as part of the CI/CD process. These are the absolute-latest version of the server and will likely include some features that are partially implemented.

The Docker image installation and configuration instructions can be found on the Hyrax guide. It also explains how to configure the docker images for various common data distribution cases.

See the interactive demo

2) Linux Binaries

This requires manual installation of pre-compiled libraries for Linux (RHEL-8) available below.

Required external dependencies:

Java >= 11
Tomcat >= 9

OLFS

OLFS 1.18.14 Web Archive File (gpg signature) Tested against Tomcat-9.0.64.
OLFS Automatic robots.txt generation for 1.18.14 (gpg_signature) This archive contains a web archive file that runs in the Tomcat server’s root context that returns a response for ‘/robots.txt’ so that your site can be crawled using the automatically-built site maps added in 1.15.2. This is beta software; we’d appreciate feedback on it.

ncWMS2 (optional)

Use the EDAL web page to locate the latest ncWMS2 “Servlet Container” software bundle as a WAR file. Install it into the same Tomcat instance as the OLFS. The configuration instructions may be found here.

BES and libdap

Linux (RHEL-8) x86_64 RPMs Our Enterprise Linux builds are released on el8 (using rocky-8)

libdap-3.21.0-27 (gpg signature).
bes-3.21.0-46.static (gpg signature) – This RPM includes statically linked copies of all of the modules/handlers we support, including HDF4 & 5 with HDFEOS support. There is no need to install packages from EPEL with this RPM. Other sources of RPM packages will likely provide a bes RPM that uses handlers linked (dynamically) to dependencies from their distributions (CentOS, Fedora, etc.).

The instructions for installing the pre-compiled binaries and testing the server can be found on the Hyrax Guide.

3) Source Code

The complete source code is available for developers who want to customize Hyrax for their specific needs. There are two ways to get the source code:

3a) Source binaries for Hyrax 1.17.0

libdap 3.21.0-27, gpg signature.
BES 3.21.0-46, gpg signature.
olfs 1.18.14 source (gpg signature).
Collected dependencies for Hyrax 1.17.0 (gpg signature) – This bundles the NetCDF, HDF4, HDF5, and other libraries that the Hyrax handlers require. Note that for Hyrax 1.17 with Direct I/O for some netCDF4 file responses, you need this when building from source because we use a special version of netCDF not publically available.

3b) Source from GitHub

All of our source code is available at our GitHub site. There you will find the hyrax project repository, which is a meta-project that contains scripts to clone and build all of Hyrax. You will also see all of the repos that contain the Hyrax source code (libdap4, the bes and all of its handlers, and the olfs).

Directions on building Hyrax from GitHub are described on the Hyrax Guide..

Added in Hyrax-1.17.0

DMR++ support (direct access to data in S3) now supports much more of the HDF5 data model. We still have two gaps: support for certain Compound variables and support for some kinds of string arrays.
Support for direct I/O transfers from HDF5 to NetCDF4 when using DMR++.
Two-level caching of remote objects.
Much improved support for response size limits and better error messages for users when those (configurable) limits are exceeded.
The bundled grid() and geogrid() server side functions now support DAP4.
Support for very large data objects.
Many bug fixes.

OLFS:

The OLFS now uses Java17 (jdk17)
Changed maxResponseSize type from int to long.
Units scalar set to 2^10 (kilobytes) when the units value is junk.
Preformatted text for 400 errors
Reduced beslistener pool from 200 to 25 for NGAP deployments
Fixes NASA/ESDIS Earthdata Login (EDL) token handling so that redirected are not issued and session (aka cookie) management is not required of the client.

BES:

Reworked the way we implement the BES’s singletons (or more of them at least) so that they use the Meyers Singleton pattern. This includes the EffectiveUrlCache and TheBESKeys classes.
We have made significant advances to supporting HDF5 files in the DMR++ builder and interpreter. Our support is close to complete with only a very few features not supported. This support now includes HDF5’s Varying-length string arrays. The software now also supports scalar strings in all their forms. Note that we also added support for fixed length string arrays in the DMR++ builder and interpreter.
This version of the BES includes a new feature, dependent on the newest version of libdap, that reduces the time needed to build NetCDF4 responses by a factor of 10 or more when the variables in the response are not spatially subset (that is, when all variables in the response are returned in their entirety). It is still possible to subset at the variable level and get this performance boost. This is the ‘Direct I/O’ feature added ti libdap. NOTE: To use this feature, you must rebuild DMR++ files using the version of the DMR++ builder associated with this version of the BES (or get_dmrpp tool from the hyrax-1.16.8-356 build).
We improved the performance of finding the effective URL for a data item when it is accessed via a series of redirect operations, the last of which is a signed AWS URL. This is a common case for data stored in S3.
We have added generic Memory and File caching, tailored specifically toward the cases that arise when serving data from S3 using the DMR++ system.
The BES now has much better support for response size and time limits. Users are warned about responses that are too large before they are built and the BES now exits gracefully when a response takes too long to build. The limits are configurable.
We replaced a home-grown HTTP connection pool with a scheme provided by libcurl. The two have equivalent performance, but the libcurl version is much easier to maintain and might offer room for improvement in the future.
C99 compatibility improvements for the GCTP code in/used-by the HDF5 handler. Thanks ti Florian Weimer for those fixes.
We have added a BES module that can work with S3 using the DMR++ system. This provides a data flow that is similar to the one we provide for Hyrax in the Cloud as developer for NASA, but this new module does not make use of the NASA/ESDIS CMR system to resolve ‘NASA Granules’ to URLs. This will enable other groups to use the DMR++ system to serve data from S3.
We have adopted C++-14 as aggressively as we can, resulting in fewer lines of code, better memory management and more efficient code.
We have moved more of the handlers to build/use DAP4 as the default response format and build DAP2 responses from those. This reduces the amount of code we have to maintain.
The BES can sign S3 URLs using the AWS V4 signing scheme. This uses the Credentials Manager system.
The bundled grid() and geogrid() server side functions now support DAP4.

libdap4:

Added new Direct I/O support so that modules written using libdap can pass compressed data buffers read directly from disk to output files without expensive decompression and recompression operations.
Merged contributed fixes from Bo Anderson Fix handling of libtirpc pkg-config files with -L flags (#228)
Merged contributed fixes from Dan Horák <da at @danny.cz> add missing include (#227) With GCC 13 the header isn’t included thru other headers any more, thus include it explicitly. Otherwise uint8_t or uint32_t type remain undefined in Vector.cc. Fixes: #226 add missing big endian baselines (#196)
Fixed a bug where the copy ctor for D4Maps failed to correctly set the parent Array.
Merged contributed fix from Orion Poplawski Add missing cstdint include for uint32_t (#219)
Removed support for RHEL 7 (CentOS 7)
Now require C++-14 to build the code.
Moved the functionality of is_dap4_projected() into libdap4 (#213). This had a number of consequences and there are new methods to support the feature.
Added support for 64-bit sized arrays. This was done by adding a set of ‘size methods’ that have the suffix ‘_ll’ (for long long). These should be used in place of the old methods, which are still in the code.