how to submit job to slurm

Some jobs can be time consuming and we want to avoid reserving a powerful node for only interaction. So what we can do is have the ‘interactive part’ (tomwer node) with fewer ressources (no gpu for example) and submit jobs to a node that will be released once the processing is done.

in tomwer resource-consuming tasks can be executed to slurm:

• slice / volume reconstruction

  • nabu slice

  • nabu volume

  • sa-axis

  • sa-delta-beta

• volume casting

warning: to launch jobs on slurm you need to be on a slurm-client of the esrf-cluster. Simple way to check is to make sure you can access the sbatch command.

Then you will need to handle the job submission and future object by the * slurm cluster widget * future supervisor widget

note: both are part of the ‘cluster’ group

slurm cluster widget

This widget will be used to know the partition and the configuration to create nodes.

You need to add it to the canvas and configure it like

Then you can connect it to one of the widget handling it (presented upper) using the ‘cluster_config’ input. Here we connect it with the nabu volume widget.

but this time the widget launching job to slurm will return a future tomo obj object. Basically this is used to wait for the processing to be done.

And to wait for the processing to be done and convert it back to a ‘standard’ data or volume object we need to use the future supervisor widget

future supervisor widget

This widget will give a feedback of the different jobs currently executed remotly.

Then by default once a job is finished you can used it back to your workflow.

bash script

Each processes compatible with slurm will create bash script (.sh) under the ‘slurm_scripts’ folder.

Log file related to those script have the same name but a different extension (.out).

If needed you can relaunch them from the command line using sbatch command.

---

demo video of a submission to slurm

Here is a video of a video demonstration of this feature.

note: the interface of the slurm cluster have been updated (simplified) since the video but the behavior / logic behind is the same.

from IPython.display import YouTubeVideo

YouTubeVideo("HYufV7Ya9v8", height=500, width=800)

---

hand on - exercise A

reconstruct a slice the crayon.nx NXtomo (available from /scisoft/tomo_training/part6_edit_nxtomo/) or another dataset if you like.

Then reconstruct a few slices of the volume (~20) by sybmitting the volume reconstruction to the cluster.