Multi entry (NXtomo) file

A file can contains several NXtomos / entry. Those NXtomos can be: * groups fully containing data (no external links) * groups containing external links (majority of the case if the data comes from bliss)

tomwer can treat those multi-entry files but there is some know limitations coming from HDF5. If all data is (fully) contained into a single file, the most chance you will have to met conflict.

If you process your dataset locally then tomwer has some ‘stack’ mecanism which are made to avoid conflict on GPU. As a consequence this will also highly reduce risk of conflict on datasets. Furthermore all the file are open within disabling the file locking (see HDF5File). Then this is up to HDF5 to handle it.

If you trigger several reconstruction over slurm and over several computers with the same file there is a risk to met conflict with the file (due to HDF5 mecanism). (note for developer: maybe replacing the SharedLockPool by a flufl.lock could improve robustness).

So if you met those kind of conflict (like if nabu fails to read the full dataset or if a third part library mets some errors reading datasets) a safer way is to:

• split NXtomos into several file. There is an nxtomomill application which does this:

  nxtomomill split-nxfile [input_file]
  

• reconstruct it locally using default tomwer stack. In this case be carreful if you reconstruct slice and volume in parrallel you might still met some conflicts.