OPeNDAP at Earth Science Information Partners (ESIP) 2024

OPeNDAP had a strong presence at this year’s ESIP summer 2024 meeting, held in Asheville, North Carolina. James Gallagher, President of OPeNDAP, and Miguel Jimenez-Urias, Scientific Community Director, led an interactive workshop for data access to OPeNDAP-server data. Broadly 30 people attended, and had a lively Q&A with many interested users and collaborators. In addition, James Gallagher gave an oral presentation, on behalf of NASA JPL/PO.DAAC intern and University of Washington student Ayush Nag, about some of the latest developments for cloud-optimizing access to OPeNDAP served data, exploiting the DMR++ sidecar files, and its integration with the package Virtualizarr.

Here are more details about our activities: 

Oral Presentation

• Title: “VirtualiZarr and DMR++“
• Presented by James Gallagher on behalf of NASA JPL/PO.DAAC intern and UW student Mr. Ayush Nag.

Description

James Gallagher presented work done at PO.DAAC about ongoing efforts to enable the (still under development) python package called VirtualiZarr to parse OPeNDAP-generated DMR++ documents. A DMR++ is a sidecar file adjacent to a dataset that contains relevant metadata information about the dataset, including Chunk References. Initial benchmark results show that enabling VirtualiZarr to read DMR++ documents results in very fast virtual aggregations of multiple granules, essentially as fast as Zarr. There was significant interest in this and in access to DMR++ documents in general.

The presentation can be found on Zenodo (10.5281/zenodo.13176038)

Workshop

• Title: “FAIRness with OPeNDAP: Interactive workshop for accessing and subsetting remote geospatial data“.
• Facilitators: Miguel A. Jimenez-Urias and James Gallagher.

Session Description

OPeNDAP is a long-recognized, language-agnostic, and domain-neutral standard protocol for remote access and transmission of complex hierarchical data structures beyond traditional data cubes. Because OPeNDAP’s response model inherently separates data from its self-describing metadata, OPeNDAP allows for lazy and interactive data inspection and subsetting of massive volumes of remote data that are too big to fit into memory. This capability remains widely utilized by various client API software, evidence of the broad adoption of the OPeNDAP protocol across the geosciences. As a long-standing, discipline-neutral, and open-source technology, OPeNDAP supports the principles of Findable, Accessible, Interoperable, and Reusable (FAIR) datasets.

During this session, the OPeNDAP team will begin with a brief introductory presentation, followed by interactive examples demonstrating remote and “lazy” access, visualization, and subsetting of chunked geospatial data. These examples will emulate workflow components commonly used within the geophysical scientific community. The team will showcase several use cases, including access to data hosted on the Earth Data Cloud via tokens, interactive inspection, visualization, and subsetting of remote data from various sources and data layouts. Additionally, they will describe best practices for efficiently utilizing the DAP protocol.

Tutorials can be found in here: https://github.com/OPENDAP/ESIP2024

To learn more about ESIP go to https://www.esipfed.org/. To learn more about this ESIP community meeting, you can browser the sched here.