Prev Up Next Index
Go backward to A.1.2 ASCII Data (Text)
Go up to A.1 Outputs
Go forward to A.2 Inputs

A.1.3 Binary Data

The DODS data document consists of two parts, a DDS describing the data returned, and the data itself. It may also consist of ASCII data describing an error condition.

DDS

The DDS returned with the data differs slightly from the DDS returned by a simple request using the .dds URL suffix. Whereas the DDS returned in response to that URL is a description of the dataset available, the DDS returned in a data document describes the data being returned. If you ask for an entire dataset, there will be no difference between these two DDS's. However, if you request only a part of a dataset, the DDS included in the data document will only reflect the part of the dataset returned to you.

As an example, consider a dataset containing an array with one hundred elements. The DDS for this dataset will contain an array with one hundred elements. However, if you send a server a request for just the first fifty elements of the array, the DODS data document returned will contain a DDS with only fifty elements. This is illustrated in figure A.1.3.

Data

The data requested from a server is (unsurprisingly) also contained in the DODS server's response to a data request. The data is encoded using the XDR standard (eXternal Data Representation), and packed into the second part of the response document. For more information on XDR, see Internet RFC 1014. For further details, see Section A.1.3, below.

Error Messages

A DODS data document may contain an error message. This is a DODS Error object (containing an ASCII error message and some other data) stuck into the HTTP document where the data would usually go. This is where error conditions on the server are noted, as well as badly formed constraint expressions and file names.

 

The DODS Data Document and the DDS. For the dataset containing the vector temp, with 100 elements, the top "Simple DDS Request" shows what the DDS might look like for that dataset. The bottom "DODS Data Document" shows what might be returned by a request for all the even-numbered elements of the temp array. Note that the DDS has been altered to allow for the reduced number of elements in the returned data array.

The HTTP response containing the DODS data document is formatted like this:
HTTP/1.0 200 OK
XDODS-Server: <server/dap version string>
Content-type: application/octet-stream
Content-Description: dods-data
Content-Encoding: deflate
<blank line>
<DDS>
Data:
<binary data, each variable given in the order it is listed in the DDS>
<EOF>

Note the following:

Encoding the DAP Data Types

The DODS transmission protocol spearates variables into three classes: scalars, arrays, and sequences. A scalar is included in the DODS data document simply by writing its XDR representation. An array is sent by writing the number of elements (as an XDR int) followed by the elements themselves, also in the XDR format. However, arrays of Byte, Int16, Int32, UInt16, UInt32, Float32, and Float64 actually have their size sent twice, once by the DAP software and once by the XDR software. Here's a hex dump of some Int32 array data that shows this behavior:

00000000: 4461 7461 7365 7420 7b0a 2020 2020 496e  Dataset {.    In
00000010: 7433 3220 755b 7469 6d65 5f61 203d 2031  t32 u[time_a = 1
00000020: 5d5b 6c61 7420 3d20 315d 5b6c 6f6e 203d  ][lat = 1][lon =
00000030: 2032 315d 3b0a 7d20 666e 6f63 313b 0a44   21];.} fnoc1;.D
00000040: 6174 613a 0a00 0000 1500 0000 15ff fff9  ata:............
00000050: 40ff fff6 6fff fff3 e5ff fff1 ffff fff3  @...o...........
00000060: 4aff fff6 9aff fffb 1c00 0002 9600 0009  J...............
00000070: b300 000b 5e00 000b 0300 000b 8200 000a  ....^...........
00000080: b900 000a ae00 000b 7300 000a 2900 0008  ........s...)...
00000090: 5b00 0007 3500 0006 da00 0007 6900 0007  [...5.......i...
000000a0: 3e                                       >

The length bytes for the <data> section start at address 0x45 (The length is 0x00000015.) and is repeated at 0x49. Here's how you can see this:

geturl "http://dods.gso.uri.edu/cgi-bin/nph-nc/data/fnoc1.nc?u[0:0][0:0][0:20]"

The DDS for this dataset is (use geturl -d):

Dataset {
    Int32 u[time_a = 16][lat = 17][lon = 21];
    Int32 v[time_a = 16][lat = 17][lon = 21];
    Float64 lat[lat = 17];
    Float64 lon[lon = 21];
    Float64 time[time = 16];
} fnoc1;

Structures and Grids are sent by serializing their components, one by one, in the order of their declaration in the DDS.

Sequences have a much more complex encoding scheme because one sequence may contain another sequence and because the sequence type provides no information about the length of a given instance. To send sequences the DAP uses two different algorithms, one up to DODS version 2.14 servers and clients and a better (more compact one) after that. DODS clients at version 2.15 and later can all read from both servers but they assume that a server that does not announce its version is very old and will attempt to read Sequences using the old algorithm. (So if you're writing a new DODS server that serves Sequences, you must remember to address the version requirement.)

Here's how the new algorithm encodes a sequence:

  1. Serialize row of the sequence by:
    1. Writing the start of instance marker (0x5A), and
    2. Serializing (writing the XDR encoding of) each element in the row in the order of appearance, then
  2. Write the end of sequence marker (0x5A).

Tom Sgouros, July 2, 2004

Prev Up Next