Multi-entry (NXtomo) File

A file can contain several NXtomos/entries. These NXtomos can be:

• Groups fully containing data (no external links).

• Groups containing external links (common when data comes from Bliss).

Tomwer can handle these multi-entry files, but there are some known limitations due to HDF5. If all data is fully contained in a single file, conflicts are more likely.

If you process your dataset locally, Tomwer uses a ‘stack’ mechanism designed to avoid conflicts on the GPU, significantly reducing the risk of dataset conflicts. Additionally, all files are opened with file locking disabled (see HDF5File), leaving conflict handling to HDF5 itself.

However, if you trigger multiple reconstructions across Slurm and multiple computers with the same file, there is a risk of conflicts due to HDF5 mechanisms. (Note for developers: Consider replacing SharedLockPool with flufl.lock to improve robustness.)

If you encounter conflicts (e.g., if Nabu fails to read the full dataset or if a third-party library encounters errors while reading datasets), a safer approach is to:

• Split NXtomos into separate files. There is an nxtomomill application that performs this task:

  nxtomomill split-nxfile [input_file]
  

• Reconstruct the data locally using the default Tomwer stack. Be cautious if you are reconstructing slices and volumes in parallel, as conflicts may still occur.