Using slurm cluster

slurm (Simple Linux Utility for Resource Management) is a free and open-source job scheduler for Linux.

tomwer can benefit from slurm cluster and trigger remote processing from the cluster. for now a limited set of bricks can be run on a remote slurm node:

• nabu slice reconstruction (cat A)

• nabu volume reconstruction (cat A)

• multi-cor (sa-axis) (cat B)

• multi-pag (sa-delta/beta calculation) (cat B)

This allow you to request dedicated and appropriate resources from the cluster (and avoid waste of ressources - block ressources for an interactive job). There is two categories of widgets able to do remote processing:

• categoty A (cat A): widgets part of this category can request remote processing to the cluster and produce a future_data object that can be supervise and convert bact to a data object if the processing went well.

• category B (cat B): widgets part of this category have ‘embeded’ display of the processing and expect the users to provide a feedbac to continue the processing.

  Widgets from this categoru will request some remote processing but they will never generate a future_data object. They will wait for the future to be done in order to update the GUI and ask feedback to the user.

to see how to benefit from the slurm cluster you can have a look at:

• reconstructions from local resoures to slurm cluster

Requirements:

• have (local) access to slurm

• target slurm partition with gpus (p9gpu, gpu, p9gpu-long)

• get tomwer version >= 0.9

• have dask-jobqueue installed

Note

you can create one dedicated slurm cluster per processes. You are not enforce to reuse the same one each time.