2.3.2. EPICS Area Detector Examples

Two examples in this section show how to write NeXus HDF5 data files with EPICS Area Detector images. The first shows how to configure the HDF5 File Writing Plugin of the EPICS Area Detector software. The second example shows how to write an EPICS Area Detector image using Python. HDF5 File Writing Plugin

This example describes how to write a NeXus HDF5 data file using the EPICS [1] Area Detector [2] HDF5 file writing plugin [3]. We will use the EPICS SimDetector [4] as an example. (PV prefix: 13SIM1:) Remember to replace that with the prefix for your detector’s IOC.

One data file will be produced for each image generated by EPICS.

You’ll need AreaDetector version 2.5 or higher to use this as the procedures for using the HDF5 file writing plugin changed with this release.

configuration files

There are two configuration files we must edit to configure an EPICS AreaDetector to write NeXus files using the HDF5 File Writer plugin:




what information to know about from EPICS and other sources


where to write that information in the HDF5 file

Put these files into a known directory where your EPICS IOC can find them. attributes.xml

The attributes file is easy to edit. Any text editor will do. A wide screen will be helpful.

Each <Attribute /> element declares a single ndattribute which is associated with an area detector image. These ndattribute items can be written to specific locations in the HDF5 file or placed by default in a default location.


The attributes file shown here has been reformatted for display in this manual. The downloads section below provides an attributes file with the same content using its wide formatting (one complete Attribute per line). Either version of this file is acceptable.

 1 <?xml version="1.0" standalone="no" ?>
 2 <!-- Attributes -->
 3 <Attributes
 4     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 5     xsi:noNamespaceSchemaLocation=
 6     "https://github.com/areaDetector/ADCore/blob/master/iocBoot/NDAttributes.xsd"
 7     >
 8     <Attribute name="AcquireTime"         
 9                type="EPICS_PV" 
10                source="13SIM1:cam1:AcquireTime"      
11                dbrtype="DBR_NATIVE"  
12                description="Camera acquire time"/>
13     <Attribute name="ImageCounter"        
14                type="PARAM"    
15                source="ARRAY_COUNTER"                
16                datatype="INT"        
17                description="Image counter"/>
18     <Attribute name="calc1_val"           
19                type="EPICS_PV" 
20                source="prj:userCalc1.VAL"            
21                datatype="DBR_NATIVE" 
22                description="some calculation result"/>
23     <Attribute name="calc2_val"           
24                type="EPICS_PV" 
25                source="prj:userCalc2.VAL"            
26                datatype="DBR_NATIVE" 
27                description="another calculation result"/>
28     <Attribute name="MaxSizeX"            
29                type="PARAM"    
30                source="MAX_SIZE_X"                   
31                datatype="INT"        
32                description="Detector X size"/>
33     <Attribute name="MaxSizeY"            
34                type="PARAM"    
35                source="MAX_SIZE_Y"                   
36                datatype="INT"        
37                description="Detector Y size"/>
38     <Attribute name="CameraModel"         
39                type="PARAM"    
40                source="MODEL"                        
41                datatype="STRING"     
42                description="Camera model"/>
43     <Attribute name="CameraManufacturer"  
44                type="PARAM"    
45                source="MANUFACTURER"                 
46                datatype="STRING"     
47                description="Camera manufacturer"/>
48 </Attributes>

If you want to add additional EPICS process variables (PVs) to be written in the HDF5 file, create additional <Attribute /> elements (such as the calc1_val) and modify the name, source, and description values. Be sure to use a unique name for each ndattribute in the attributes file.


ndattribute : item specified by an <Attribute /> element in the attributes file. layout.xml

You might not need to edit the layout file. It will be fine (at least a good starting point) as it is, even if you add PVs (a.k.a. ndattribute) to the attributes.xml file.

 1 <?xml version="1.0" standalone="no" ?>
 2 <hdf5_layout>
 3   <group name="entry"> 
 4     <attribute name="NX_class" source="constant" value="NXentry" type="string"/>
 5     <group name="instrument"> 
 6       <attribute name="NX_class" source="constant" value="NXinstrument" type="string"/> 
 7       <group name="detector"> 
 8         <attribute name="NX_class" source="constant" value="NXdetector" type="string"/> 
 9         <dataset name="data" source="detector" det_default="true"> 
10           <attribute name="NX_class" source="constant" value="SDS" type="string"/> 
11           <attribute name="signal" source="constant" value="1" type="int"/> 
12           <attribute name="target" source="constant" value="/entry/instrument/detector/data" type="string"/> 
13         </dataset> 
14         <group name="NDAttributes"> 
15           <attribute name="NX_class" source="constant" value="NXcollection" type="string"/> 
16           <dataset name="ColorMode" source="ndattribute" ndattribute="ColorMode"/> 
17         </group>          <!-- end group NDAttribute --> 
18       </group>            <!-- end group detector --> 
19       <group name="NDAttributes" ndattr_default="true"> 
20         <attribute name="NX_class" source="constant" value="NXcollection" type="string"/> 
21       </group>            <!-- end group NDAttribute (default) --> 
22       <group name="performance"> 
23         <dataset name="timestamp" source="ndattribute"/> 
24       </group>            <!-- end group performance --> 
25     </group>              <!-- end group instrument --> 
26     <group name="data"> 
27       <attribute name="NX_class" source="constant" value="NXdata" type="string"/> 
28       <hardlink name="data" target="/entry/instrument/detector/data"/>
29       <!-- The "target" attribute in /entry/instrument/detector/data is used to 
30            tell Nexus utilities that this is a hardlink -->
31     </group>              <!-- end group data --> 
32   </group>                <!-- end group entry -->
33 </hdf5_layout>

If you do not specify where in the file to write an ndattribute from the attributes file, it will be written within the group that has ndattr_default="true". This identifies the group to the HDF5 file writing plugin as the default location to store content from the attributes file. In the example layout file, that default location is the /entry/instrument/NDAttributes group:


To specify where PVs are written in the HDF5 file, you must create <dataset /> (or <attribute />) elements at the appropriate place in the NeXus HDF5 file layout. See the NeXus manual [5] for placement advice if you are unsure.

You reference each ndattribute by its name value from the attributes file and use it as the value of the ndattribute in the layout file. In this example, ndattribute="calc1_val" in the layout file references name="calc1_val" in the attributes file and will be identified in the HDF5 file by the name userCalc1:



A value from the attributes file is only written either in the default location or in the location named by a <dataset/> or <attribute/> entry in the layout file. Expect problems if you define the same ndattribute in more than one place in the layout file.

You can control when a value is written to the file, using when="" in the layout file. This can be set to one of these values: OnFileOpen, OnFileClose

Such as:




additional configuration

Additional configurations of the EPICS Area Detector and the HDF5 File Plugin are done using the EPICS screens (shown here using caQtDM [6]):

configuration screens

ADBase and NDFileHDF5 configuration screens

Additional configuration on the ADBase screen:

  • Set Image mode to “Single”

  • Set Exposure time as you wish

  • Set # Images to 1

  • for testing, it is common to bin the data to reduce the image size

  • The full path to the attributes.xml file goes in the bottom/left File box

Additional configuration on the NDFileHDF5 screen:

  • Set the File path and “File name” to your choice.

  • Set Auto save to “Yes”.

  • Set Compression to “zlib” if you wish (optional)

  • Set Enable to “Enable” or the HDF5 plugin won’t get images to write!

  • Set Callbacks block to “Yes” if you want to wait for HDF5 files to finish writing before collecting the next image

  • The full path to the layout.xml file goes into the bottom/right XML File name box

  • Leave the Attributes file box empty in this screen.

When you enter the names of these files in the configuration screen boxes, AreaDetector will check the files for errors and let you know.

Example view

We collected data for one image, /tmp/mrinal_001.h5, in the HDF5 file provided in the downloads section. You may notice that the values for calc1_val and calc2_val were arrays rather than single values. That was due to an error in the original attributes.xml file, which had type="PARAM" instead of type="EPICS_PV". This has been fixed in the attributes.xml file presented here. Python code to store an image in a NeXus file

Suppose you want to write area detector images into NeXus HDF5 files python code. Let’s assume you have the image already in memory in a numpy array, perhaps from reading a TIFF file or from an EPICS PV using PyEpics. The file write_nexus_file.py (provided below) reads an image from the sim detector and writes it to a NeXus HDF5 data file, along with some additional metadata.

using the h5py package

This example uses the h5py [7] package to write the HDF5 file.

 1 import numpy as np
 2 import h5py
 3 import datetime
 5 def write_nexus_file(fname, image, md={}):
 6     """
 7     write the image to a NeXus HDF5 data file
 9     Parameters
10     ----------
11     fname : str
12         name of the file (relative or absolute) to be written
13     image : numpy array
14         the image data
15     md : dictionary
16         key: value where value is something that can be written by h5py
17              (such as str, int, float, numpy array, ...)
18     """
19     nexus = h5py.File(fname, "w")
20     nexus.attrs["filename"] = fname
21     nexus.attrs["file_time"] = datetime.datetime.now().astimezone().isoformat()
22     nexus.attrs["creator"] = "write_nexus_file()"
23     nexus.attrs["H5PY_VERSION"] = h5py.__version__
25     # /entry
26     nxentry = nexus.create_group("entry")
27     nxentry.attrs["NX_class"] = "NXentry"
28     nexus.attrs["default"] = nxentry.name
30     # /entry/instrument
31     nxinstrument = nxentry.create_group("instrument")
32     nxinstrument.attrs["NX_class"] = "NXinstrument"
34     # /entry/instrument/detector
35     nxdetector = nxinstrument.create_group("detector")
36     nxdetector.attrs["NX_class"] = "NXdetector"
38     # /entry/instrument/detector/image
39     ds = nxdetector.create_dataset("image", data=image, compression="gzip")
40     ds.attrs["units"] = "counts"
41     ds.attrs["target"] = "/entry/instrument/detector/image"
43     # /entry/data
44     nxdata = nxentry.create_group("data")
45     nxdata.attrs["NX_class"] = "NXdata"
46     nxentry.attrs["default"] = nxdata.name
48     # /entry/data/data --> /entry/instrument/detector/image
49     nxdata["data"] = nexus["/entry/instrument/detector/image"]
50     nxdata.attrs["signal"] = "data"
52     if len(md) > 0:
53         # /entry/instrument/metadata (optional, for metadata)
54         metadata = nxinstrument.create_group("metadata")
55         metadata.attrs["NX_class"] = "NXcollection"
56         for k, v in md.items():
57             try:
58                 metadata.create_dataset(k, data=v)
59             except Exception:
60                 metadata.create_dataset(k, data=str(v))
62     nexus.close()
65 if __name__ == "__main__":
66     """demonstrate how to use this code"""
67     import epics
68     prefix = "13SIM1:"
69     img = epics.caget(prefix+"image1:ArrayData")
70     size_x = epics.caget(prefix+"cam1:ArraySizeX_RBV")
71     size_y = epics.caget(prefix+"cam1:ArraySizeY_RBV")
72     # edit the full image for just the binned data
73     img = img[:size_x*size_y].reshape((size_x, size_y))
75     extra_information = dict(
76         unique_id = epics.caget(prefix+"image1:UniqueId_RBV"),
77         size_x = size_x,
78         size_y = size_y,
79         detector_state = epics.caget(prefix+"cam1:DetectorState_RBV"),
80         bitcoin_value="15000",
81     )
82     write_nexus_file("example.h5", img, md=extra_information)

The output from that code is given in the example.h5 file. It has this tree structure:

 1 example.h5 : NeXus data file
 2   @H5PY_VERSION = "3.6.0"
 3   @creator = "write_nexus_file()"
 4   @default = "entry"
 5   @file_time = "2022-03-07 14:34:04.418733"
 6   @filename = "example.h5"
 7   entry:NXentry
 8     @NX_class = "NXentry"
 9     @default = "data"
10     data:NXdata
11       @NX_class = "NXdata"
12       @signal = "data"
13       data --> /entry/instrument/detector/image
14     instrument:NXinstrument
15       @NX_class = "NXinstrument"
16       detector:NXdetector
17         @NX_class = "NXdetector"
18         image:NX_UINT8[1024,1024] = __array
19           __array = [
20               [76, 77, 78, '...', 75]
21               [77, 78, 79, '...', 76]
22               [78, 79, 80, '...', 77]
23               ...
24               [75, 76, 77, '...', 74]
25             ]
26           @target = "/entry/instrument/detector/image"
27           @units = "counts"
28       metadata:NXcollection
29         @NX_class = "NXcollection"
30         bitcoin_value:NX_CHAR = b'15000'
31         detector_state:NX_INT64[] = 
32         size_x:NX_INT64[] = 
33         size_y:NX_INT64[] = 
34         unique_id:NX_INT64[] = 


Alternatively, the metadata shown in this example might be placed in the /entry/instrument/detector (NXdetector) group along with the image data since it provides image-related information such as size.

In the interest of keeping this example simpler and similar to the one above using the HDF5 File Writing Plugin, the metadata has been written into a NXcollection group at /entry/instrument/metadata location. (Compare with the NXcollection group /entry/instrument/NDAttributes above.)

using the nexusformat package

The nexusformat [8] package for python simplifies the work to create a NeXus file. Rewriting the above code using nexusformat:

 1 import numpy as np
 2 from nexusformat.nexus import *
 5 def write_nexus_file(fname, image, md={}):
 6     """
 7     write the image to a NeXus HDF5 data file
 9     Parameters
10     ----------
11     fname : str
12         name of the file (relative or absolute) to be written
13     image : numpy array
14         the image data
15     md : dictionary
16         key: value where value is something that can be written by h5py
17              (such as str, int, float, numpy array, ...)
18     """
19     nx = NXroot()
20     nx['/entry'] = NXentry(NXinstrument(NXdetector()))
21     nx['entry/instrument/detector/image'] = NXfield(image, units='counts',
22                                                     compression='gzip')
23     nx['entry/data'] = NXdata()
24     nx['entry/data'].makelink(nx['entry/instrument/detector/image'])
25     nx['entry/data'].nxsignal = nx['entry/data/image']
27     if len(md) > 0:
28         # /entry/instrument/metadata (optional, for metadata)
29         metadata = nx['/entry/instrument/metadata'] = NXcollection()
30         for k, v in md.items():
31             metadata[k] = v
33     nx.save(fname, 'w')
36 if __name__ == "__main__":
37     """demonstrate how to use this code"""
38     import epics
39     prefix = "13SIM1:"
40     img = epics.caget(prefix+"image1:ArrayData")
41     size_x = epics.caget(prefix+"cam1:ArraySizeX_RBV")
42     size_y = epics.caget(prefix+"cam1:ArraySizeY_RBV")
43     # edit the full image for just the binned data
44     img = img[:size_x*size_y].reshape((size_x, size_y))
46     extra_information = dict(
47         unique_id = epics.caget(prefix+"image1:UniqueId_RBV"),
48         size_x = size_x,
49         size_y = size_y,
50         detector_state = epics.caget(prefix+"cam1:DetectorState_RBV"),
51         bitcoin_value="15000",
52     )
53     write_nexus_file("example.h5", img, md=extra_information) Visualization

You can visualize the HDF5 files with several programs, such as: hdfview [9], nexpy [10], or pymca [11]. Views of the test image shown using NeXPy (from the HDF5 file) and caQtDM (the image from EPICS) are shown.

visualization screens

Views of the image in NeXPy (left) and in caQtDM (right)

Get the installation instructions for any of these programs from a web search. Other data analysis programs such as MatLab, IgorPro, and IDL can also read HDF5 files but you might have to work a bit more to get the data to a plot. Downloads




The attributes file


The layout file


example NeXus HDF5 file written from EPICS


Python code to get images from EPICS and write a NeXus file


write_nexus_file.py rewritten with nexusformat package


example NeXus HDF5 file written from Python Footnotes