h52nx tutorial#
10 min read
the h52nx application can be used to convert acquisition from hdf5/bliss to a nexus - NXtomo compliant format. This format will also be stored in .h5 / .hdf5 / .nx file. For comprehension we will use the nexus format (.nx) in this tutorial.
To call this application you can call directly
$ nxtomomill h52nx [options]
Tip
You can also call it direcly from python executable using:
$ python -m nxtomomill h52nx [options]
Then you can convert bliss h5 file without configuration file or with a configuration file
without configuration file#
The first two parameters should be:
input-file-name: bliss.hdf5 master file, containing the details of the acquisition
output-file: destination file where result will be store.
Sor for example to convert a file name ‘bliss.hdf5’ to a ‘my_nxtomo_file.nx’ you should call:
$ python -m nxtomomill h52nx bliss.hdf5 my_nxtomo_file.nx
Tip
you can provide an output directory instead of a file path. Then the file basename will remane the same, only the extension will be changed to .nx
$ python -m nxtomomill h52nx bliss.hdf5 .
Changed in version v0.13: if the output is not provided the default behavior is to create the NXtomo to the PROCESSED_DATA directory. To overstep this you can provide a directory.
You can also access the help of h52nx by calling:
$ python -m nxtomomill h52nx --help
By default if some information are missing the converter will ask you for missing input (it can be the case of the incoming energy for example). If you want to avoid converter for information you can add the –no-input option.
An acquisition file can contain several sequence (so several acquisition). h52nx will convert them all create one file per acquisition and one file referencing all acquisitions. You can ask the converter to keep all the acquisition into a single file using the ‘–single-file’ option.
Input type#
z-series#
z-series are handled by nxtomomill since 0.4. It will create one entry per ‘z’ found.
h52nx-settings#
The h5tonx command is using the ‘h5_to_nx’ function from ‘converter’ module. This will used by default h52nx-settings defined in nxtomomill.settings.py file If you want you can overwrite them from the command line.
camera name#
- In order to know if an NXDetector should be converted or not it relies on ‘H5_VALID_CAMERA_NAMES’ defined in nxtomomill.settings.py.
If the value is None (default) then we will first try to retrieve group (under instrument) that are defined as an ‘NXdetector’ NX_class. If none are found then we try to retrieve group that ‘looks’ like a detector (containing a dataset name ‘data’ and which is of dimension 3).
It can also be set as a tuple of string. Each string is a detector name to be handle. Those can handle Linux wildcard (like ‘frelon*’)
the values defined in settings can be overwrite by –valid_camera_names option (see help).
other (rotation angle, translation…)#
Most of the key used to retrieve other operation can be overwrite from an option from command line
$ nxtomomill h52nx --help
convert data acquired as hdf5 from bliss to nexus `NXtomo` classes. For
`zseries` it will create one entry per `z`
positional arguments:
input_file master file of the acquisition
output_file output .nx or .h5 file
optional arguments:
-h, --help show this help message and exit
--file_extension FILE_EXTENSION
extension of the output file. Valid values are
.h5/.hdf5/.nx
--single-file merge all scan sequence to the same output file. By
default create one file per sequence and group all
sequence in the output file
--overwrite Do not ask for user permission to overwrite output
files
--debug Set logs to debug mode
--entries ENTRIES Specify (root) entries to be converted. By default it
will try to convert all existing entries.
--ignore-sub-entries IGNORE_SUB_ENTRIES
Specify (none-root) sub entries to ignore.
--raises-error Raise errors if some data are not met instead of
providing some default values
--no-input, --no-input-for-missing-information
The user won't be ask for any inputs
--standard-format Format of the input file is the 'standard' tomography
format
--sample_x_keys SAMPLE_X_KEYS, --sample-u-keys SAMPLE_X_KEYS
sample x translation key in bliss HDF5 file (direction is independent of the rotation angle)
--sample_y_keys SAMPLE_Y_KEYS, --sample-y-keys SAMPLE_Y_KEYS
sample y translation key in bliss HDF5 file (direction is independent of the rotation angle)
--translation_z_keys TRANSLATION_Z_KEYS, --translation-z-keys TRANSLATION_Z_KEYS
z translation key in bliss HDF5 file
--valid_camera_names VALID_CAMERA_NAMES, --valid-camera-names VALID_CAMERA_NAMES
Valid NXDetector dataset name to be considered.
Otherwise will try to deduce them from NX_class
attribute (value should beNXdetector) or from
instrument group child structure.
--rot_angle_keys ROT_ANGLE_KEYS, --rot-angle-keys ROT_ANGLE_KEYS
Valid dataset name for rotation angle
--acq_expo_time_keys ACQ_EXPO_TIME_KEYS, --acq-expo-time-keys ACQ_EXPO_TIME_KEYS
Valid dataset name for acquisition exposure time
--x_pixel_size_key X_PIXEL_SIZE_KEY, --x-pixel-size-key X_PIXEL_SIZE_KEY
X pixel size key to read
--y_pixel_size_key Y_PIXEL_SIZE_KEY, --y-pixel-size-key Y_PIXEL_SIZE_KEY
Y pixel size key to read
--init_titles INIT_TITLES, --init-titles INIT_TITLES
Titles corresponding to init scans
--init_zserie_titles INIT_ZSERIE_TITLES, --init-zserie-titles INIT_ZSERIE_TITLES
Titles corresponding to zserie init scans
--dark_titles DARK_TITLES, --dark-titles DARK_TITLES
Titles corresponding to dark scans
--flat_titles --flat_titles --ref_titles FLAT_TITLES, --ref-titles FLAT_TITLES
Titles corresponding to ref scans
--proj_titles PROJ_TITLES, --proj-titles PROJ_TITLES
Titles corresponding to projection scans
--align_titles ALIGN_TITLES, --align-titles ALIGN_TITLES
Titles corresponding to alignment scans
--set-params [SET_PARAMS [SET_PARAMS ...]]
Allow manual definition of some parameters. Valid
parameters (and expected input unit) are: energy
(kev), x_pixel_size (m), y_pixel_size (m). Should be
added at the end of the command line because will try
to cover all text set after this option.
--config CONFIG, --config-file CONFIG, --configuration CONFIG, --configuration-file CONFIG
file containing the full configuration to convert from
h5 bliss to nexus
with a configuration file#
Generate a default configuration file#
You can create a default configuration file by calling
$ nxtomomill h5-config [output_conf_file.cfg]
This will generate a configuration file based on the existing Settings that you can edit.
Once the configuration file fit your needs you can execute it by calling h52nx.
$ nxtomomill h52nx input_bliss_file.h5 [output_nexus_file.nx] --config [output_conf_file.cfg]
Warning
when you provide a configuration file to h52nx then no other input can be provided (excepted for the output file which is optional).
Note
If you want to provide configuration from scan titles you can ignore the FRAME_TYPE_SECTION and generate a default configuration file with the option –from-title-names. On the contrary if you want to ignore the ENTRIES_AND_TITLES_SECTION you can use the –from-scan-urls option.
Example of a configuration file#
[GENERAL_SECTION]
# general information.
# input file if not provided must be provided from the command line
input_file =
# output file name. if not provided will use the input file basename and the file extension
output_file =
# overwrite output files if exists without asking
overwrite = False
# file extension. ignored if the output file is provided and contains an extension
file_extension = .nx
# log level. valid levels are "debug", "info", "warning" and "error"
log_level = warning
# raise an error when met one. otherwise continue and display an error log
raises_error = False
# ask or not the user for any inputs (if missing information)
no_input = False
# if true then will create a single file for all found sequences. if false create one nexus file per sequence and one master file with links to each sequence
single_file = False
# acquisition type. if not provided will try to guess it. valid values are "standard", "xrd-ct" and "" if undetermined
format = standard
# force output to be a `full` or a `half` acquisition. if not provided we parse raw data to try to find this information.
field_of_view =
[KEYS_SECTION]
# identify specific path and datasets names to retrieve information from the bliss file.
# nxtomomill will try to deduce cameras from dataset metadata and shape if none provided (default).if provided take the one requested. unix shell-style wildcards are managed
valid_camera_names =
# list of key to look for in order to find rotation angle
rotation_angle_keys = ('rotm', 'mhsrot', 'hsrot', 'mrsrot', 'hrsrot', 'srot', 'srot_eh2', 'diffrz', 'hrrz_trig', 'rot')
# list of keys / paths to look for in order to find translation in x
x_translation_keys = ('sx', 'd3tx', 'tfx', 'px')
# list of keys / paths to look for in order to find translation in y
y_translation_keys = ('sy', 'd3ty', 'hry', 'py')
# list of /paths keys to look for in order to find translation in z
z_translation_keys = ('sz', 'd3tz', 'hrz', 'pz', 'mrsz')
# key used to deduce the estimated center of rotation for half acquisition
y_rot_keys = instrument/positioners/yrot
# list of keys to look for diode (if any)
diode_keys = ('fpico3',)
# list of keys to look for the exposure time
exposure_time_keys = ('acq_expo_time',)
# list of keys / paths to look for the x pixel size
x_pixel_keys = ('technique/optic/sample_pixel_size ', 'technique/optic/sample_pixel_size', 'technique/detector/pixel_size', 'hry_step_size')
# list of keys / paths to look for the y pixel size
y_pixel_keys = ('technique/optic/sample_pixel_size ', 'technique/optic/sample_pixel_size', 'technique/detector/pixel_size', 'hry_step_size')
# list of keys / paths to look for sample to detector distance
sample_detector_distance = ('technique/scan/sample_detector_distance',)
[ENTRIES_AND_TITLES_SECTION]
# optional section
# define titles meaning. titles allows frame type deduction for each group.
# list of root entries (sequence initialization) to convert. if not provided will convert all root entries
entries =
# list of sub entries (non-root) to ignore
sub_entries_to_ignore =
# list of title to consider the group/entry as a initialization (sequence start). ignored if dark_groups, flat_groups, projection_groups ... are provided.
init_titles = ('tomo:basic', 'tomo:fullturn', 'sequence_of_scans', 'tomo:halfturn', 'tomo:multiturn')
# list of title to consider the group/entry as a zserie initialization (sequence start). ignored if dark_groups, flat_groups, projection_groups ... are provided.
zserie_init_titles = ('tomo:zseries',)
# list of title to consider the group/entry as a dark. ignored if dark_groups, flat_groups, projection_groups ... are provided.
dark_titles = ('dark images', 'dark')
# list of title to consider the group/entry as a reference / flat. ignored if dark_groups, flat_groups, projection_groups ... are provided.
flat_titles = ('flat', 'reference images', 'ref', 'refend')
# list of title to consider the group/entry as a projection. ignored if dark_groups, flat_groups, projection_groups ... are provided.
proj_titles = ('projections', 'ascan rot 0 ', 'ascan diffrz 0 180 1600 0.1')
# list of title to consider the group/entry as an alignment. ignored if dark_groups, flat_groups, projection_groups ... are provided.
alignment_titles = ('static images', 'ascan diffrz 180 0 4 0.1')
[FRAME_TYPE_SECTION]
# optional section
# allows to define scan to be used for nxtomo conversion
# the sequence order will follow the order provided.
# list of scans to be converted. frame type should be provided for each scan.
# expected format is:
# data_scans = (
# (frame_type=projections, entry=silx:///path/to/file?/path/to/scan/node, copy=true),
# (frame_type=projections, entry=/path_relative_to_file),
# )
# * `frame_type` (mandatory): values can be `projection`, `flat`, `dark`, `alignment` or `init`.
# * `entry` (mandatory): dataurl with path to the scan to integrate. if the scan is contained in the input_file then you can only provide path/name of the scan.
# * copy (optional): you can provide a different behavior for the this scan (should we duplicate data or not)
# more details are available here: todo: provide link
data_scans =
# you can duplicate data inside the input file or create a link to the original frames. in this case you should keep the relative position of the files
default_data_copy = False
[PCOTOMO_SECTION]
# pcotomo specific section
# if provided then acquisition parameters `nb_loop` and `nb_tomo` will be ignored. instead `tomo_n` nxtomo will be created from pcotomo. all angles before `start_angle_offset_in_degree` will be ignored
start_angle_offset_in_degree = None
# if 'start_angle_offset_in_degree' provided then specify the number of nxtomo to create. if -1 provided then will create as much nxtomo as possible
tomo_n = -1
# angle interval - range to create if 'start_angle_offset_in_degree' is provided. 180 or 360 is expected
angle_interval_in_degree = 360
# shift all angle nxtomo angle to `angle_interval_in_degree` interval by shifting them of start_angle_offset_in_degree + angle_interval_in_degree
shift_angles = False
[EXTRA_PARAMS_SECTION]
# optional section
# you can predefined values which are missing in the input .h5 file
# handled parameters are ('energy_kev', 'x_pixel_size_m', 'y_pixel_size_m', 'detector_sample_distance_m')
Warning
the “data_scans” values from the FRAME_TYPE_SECTION section should be provided with indentation (at least one empty space. Otherwise they will be ignored by configparser.
New in version v0.9: There is a dedicated section regarding PCO tomo. If the bliss acquisition is a detected as a PCOtomo by default it will generate nb_tomo * nb_loop NXtomo. And those will start from the first projections found.
It might append you want to ‘refine’ NXtomo to generate and this is the goal of this section.
Here is the details of the parameters:
angle offset (the projections before will be ignored. This will not affect the dark and the flat that will always be copied for pco tomo)
tomo_n: how many NXtomo you want to generate
angle_interval_in_degree: do you want to create NXtomo which will cover 180 or 360 degrees.
shift_angles: do you want to reset angles of the final NXtomo to 0
Constrain to the conversion#
nxtomomill h52nx will create an HDF5 Virtual dataset to create the /instrument/detector/data dataset. This HDF5 Virtual dataset require the creation of a VirtualLayout which force all the frame to the of the same data type (uint16, int32…)
So if you have bliss scans (entries) with different data type the conversion will fail. If this happen then there is a bug wuith the bliss acquisition.
Nevertheless there is a workaround for it. You can add to your instrument/{detector_name} group a dataset named data_cast near the data dataset.
If it exist then it will be picked by nxtomomill h52nx instead of the data dataset. Of course this second dataset must have a coherent type with the rest of the acquisition.
Warning
Be caution with the casting.
Hint
You can find here a script casting a dataset to another type