1.2.3.3. Physical File format

This section describes how NeXus structures are mapped to features of the underlying physical file format. This is a guide for people who wish to create NeXus files without using the NeXus-API.

Choice of HDF as Underlying File Format

At its beginnings, the founders of NeXus identified the Hierarchical Data Format (HDF) as a capable and efficient multi-platform data storage format. HDF was designed for large data sets and already had a substantial user community. HDF was developed and maintained initially by the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign (UIUC) and later spun off into its own group called The HDF Group (THG: http://www.hdfgroup.org/). Rather then developing its own unique physical file format, the NeXus developers chose to build NeXus on top of HDF.

HDF (now HDF5) is provided with software to read and write data (this is the application-programmer interface, or API) using a large number of computing systems in common use for neutron and X-ray science. HDF is a binary data file format that supports compression and structured data.

Mapping NeXus into HDF

NeXus data structures map directly to HDF structures. Content groups are stored as HDF5 groups, content fields (or data sets) as HDF5 datasets and content attributes as HDF5 attributes of HDF5 groups or datasets. The content attribute NX_class, the type of the NeXus group that describes the content group, is stored as an HDF5 attribute of the HDF5 group. (For legacy NeXus data files using HDF4, content groups are stored as HDF4 vgroups and content fields as HDF4 SDS (scientific data sets). The class of a content group is the class of its vgroup, set with Vsetclass() and read with Vgetclass(), not an HDF4 attribute, although HDF4 supports attributes on vgroups since HDF 4.1r1 (1997).)

In HDF5, a content link is usually stored as an HDF5 hard link (see Example: content links in HDF5).

Note

Examples are provided in the Examples of writing and reading NeXus data files chapter. These examples include software to write and read NeXus data files using the NAPI, as well as other software examples that use native (non-NAPI) libraries. In some cases the examples show the content of the NeXus data files that are produced. Here are links to some of the examples:

Perhaps the easiest way to view the implementation of NeXus in HDF5 is to look at the data structure. For this, we use the h5dump command-line utility provided with the HDF5 support libraries. Short examples are provided for the basic NeXus data components:

  • content group: created in C NAPI by:

    NXmakegroup (fileID, "entry", "NXentry");

  • content field: created in C NAPI by:

    NXmakedata (fileID, "two_theta", NX_FLOAT32, 1, &n);
  NXopendata (fileID, "two_theta");
NXputdata (fileID, tth);

  • content attribute: created in C NAPI by:

    NXputattr (fileID, "units", "degrees", 7, NX_CHAR);

  • content link, stored as an HDF5 hard link, created in C NAPI by:

    NXmakelink (fileid, &itemid);
# -or-
NXmakenamedlink (fileid, "linked_name", &itemid);

h5dump of an NXentry content group

 1GROUP "entry" {
 2  ATTRIBUTE "NX_class" {
 3     DATATYPE  H5T_STRING {
 4           STRSIZE 7;
 5           STRPAD H5T_STR_NULLPAD;
 6           CSET H5T_CSET_ASCII;
 7           CTYPE H5T_C_S1;
 8        }
 9     DATASPACE  SCALAR
10     DATA {
11     (0): "NXentry"
12     }
13  }
14  # ... group contents
15}

h5dump of a content field (HDF5 dataset)

 1 DATASET "two_theta" {
 2    DATATYPE  H5T_IEEE_F64LE
 3    DATASPACE  SIMPLE { ( 31 ) / ( 31 ) }
 4    DATA {
 5    (0): 17.9261, 17.9259, 17.9258, 17.9256, 17.9254, 17.9252,
 6    (6): 17.9251, 17.9249, 17.9247, 17.9246, 17.9244, 17.9243,
 7    (12): 17.9241, 17.9239, 17.9237, 17.9236, 17.9234, 17.9232,
 8    (18): 17.9231, 17.9229, 17.9228, 17.9226, 17.9224, 17.9222,
 9    (24): 17.9221, 17.9219, 17.9217, 17.9216, 17.9214, 17.9213,
10    (30): 17.9211
11    }
12    ATTRIBUTE "units" {
13       DATATYPE  H5T_STRING {
14             STRSIZE 7;
15             STRPAD H5T_STR_NULLPAD;
16             CSET H5T_CSET_ASCII;
17             CTYPE H5T_C_S1;
18          }
19       DATASPACE  SCALAR
20       DATA {
21       (0): "degrees"
22       }
23    }
24    # ... other attributes
25 }

h5dump of a content attribute

 1ATTRIBUTE "axes" {
 2   DATATYPE  H5T_STRING {
 3         STRSIZE 9;
 4         STRPAD H5T_STR_NULLPAD;
 5         CSET H5T_CSET_ASCII;
 6         CTYPE H5T_C_S1;
 7      }
 8   DATASPACE  SCALAR
 9   DATA {
10   (0): "two_theta"
11   }
12}

h5dump of a content link

 1# A content link stored as an HDF5 hard link has two parts.
 2
 3# The dataset is created in some group.
 4# A "target" attribute is added to indicate the HDF5 path to this dataset.
 5
 6ATTRIBUTE "target" {
 7   DATATYPE  H5T_STRING {
 8         STRSIZE 21;
 9         STRPAD H5T_STR_NULLPAD;
10         CSET H5T_CSET_ASCII;
11         CTYPE H5T_C_S1;
12      }
13   DATASPACE  SCALAR
14   DATA {
15   (0): "/entry/data/two_theta"
16   }
17}
18
19# then, the HDF5 hard link is created that refers to the dataset above
20# (Since the name is "two_theta" in this example, it is understood that
21# this HDF5 hard link is created in a different HDF5 group than "/entry/data".)
22
23DATASET "two_theta" {
24   HARDLINK "/entry/data/two_theta"
25}