Ewoks tutorial 2 - launching a workflow using ewoks submit#

Sometime it can be convenient to trigger a workflow over slurm to run some processing.

Requirements#

This tutorial consider you already created your workflow file (.ows) file. As we will submit the entire workflow to slurm you should have no ‘direct’ slurm job submission in this workflow ( <=>no ‘slurm cluster’ widget)
We also consider you alreay have a valid slurm access token. If not you can create one with the ‘scontrol token lifespan=xxxx’ command like:

For information Here is the official confluence page about slurm REST

Submit a (tomo) job over slurm#

For this example we will use the simple slice reconstruction workflow. For now no dataset is defined within the workflow. Only widget settings is defined (like the algorithm to be used for the cor, Paganin value to be used…)

create a client on a node of the cluster (on a node with gpu)

module load ewoks slurm-token-env

then trigger the worker

need message broker

create config file for ewoksjob and provide uri:

export EWOKS_CONFIG_URI=config

ewoksjob worker –pool=slurm -Q slurm -n slurm@payno: –slurm-url=http://slurm-api.esrf.fr:6820 –slurm-user=payno –slurm-token=eyJhbGciOiJIUzI1NiIsInR5cCI6[…..]cQRf1VSdNsO1WyR19jlzoN1dLtGzD7SdwTQNcrZHPmM –slurm-pre-script=’module load tomotools/stable’ –slurm-log-directory=’/tmp_14_days/payno/slurm_logs’ -sp time_limit=240 -sp current_working_directory=/tmp_14_days/payno/test_workflow_submission