Ctrl+K
1.3
  • youtube
  • pypi
  • gitlab
  • youtube
  • pypi
  • gitlab

Ewoks tutorial 1 - launching a workflow using ewoks execute#

Since version 1.1 you will be able to execute and convert an orange workflow files (.ows) with ewoks.

Requirements#

  • Have a tomography data processing workflow already defined and saved as .ows. (processing_data.ows in this example)

  • tomwer >= 1.1 installed

Execute the worflow from the command line Interface#

For this we will use the ewoks execute and provide some input(s) to it.

Today we expect users to provide at most one scan or one volume.

For this tutorial we will provide a scan (NXtomo) to the processing_data.ows file. workflow. It will define the data input on tomwer workflow.

Let’s consider providing an NXtomo saved in ‘my_dataset.nx’ file and located under entry hdf5 group. The identifier for this example is:

hdf5:scan:my_dataset.nx?path=entry0000

See identifiers definition from tomoscan for more information

Then to execute the workflow with ewoks go for:

ewoks execute processing_data.json --parameter data=hdf5:scan:bambou_hercules_0001/bambou_hercules_0001.nx?path=entry0000

In the case of providing a volume (workflow with volume casting for example) the url would have look like:

  • for a volume stored in HDF5:

    hdf5:volume:./bambou_hercules_0001slice_1080.hdf5?path=entry0000/reconstruction

  • for a volume stored in tiff (one frame per file)

    tiff:volume:/path/to/my/my_folder

  • for a volume stored in edf:

    edf:volume:/path/to/my/my_folder

  • for a volume stored in jpeg 2000:

    jp2k:volume:/path/to/my/my_folder

and the execution of the workflow would look like:

ewoks execute processing_volume.ows --parameter volume={volume_url}

Ewoks python API#

You can also use the ewoks python API in order to execute workflows directly from ewoks. For details please have a look at ewoks python API

Editing a workflow from its file#

Limitations#

Python script#

For now the python script from orange canvas is only handling the ‘data’ and ‘volume’ inputs / outputs when executed with ewoks

Convert from orange workflow (.ows) to ewoks workflow#

For this example we will consider the following workflow saved under the processing_data.ows file:

../_images/workflow_under_orange.png

The .ows looks like:

<?xml version='1.0' encoding='utf-8'?>
<scheme version="2.0" title="" description="">
    <nodes>
        <node id="0" name="scan selector" qualified_name="orangecontrib.tomwer.widgets.control.DataSelectorOW.DataSelectorOW" project_name="tomwer" version="" title="scan selector" position="(205.0, 97.0)" />
        <node id="1" name="dark and flat field construction" qualified_name="orangecontrib.tomwer.widgets.reconstruction.DarkRefAndCopyOW.DarkRefAndCopyOW" project_name="tomwer" version="" title="dark and flat field construction" position="(361.0, 156.0)" />
        <node id="2" name="default center of rotation" qualified_name="orangecontrib.tomwer.widgets.reconstruction.AxisOW.AxisOW" project_name="tomwer" version="" title="default center of rotation" position="(520.0, 207.0)" />
        <node id="3" name="nabu slice reconstruction" qualified_name="orangecontrib.tomwer.widgets.reconstruction.NabuOW.NabuOW" project_name="tomwer" version="" title="nabu slice reconstruction" position="(683.0, 246.0)" />
        <node id="4" name="nabu volume reconstruction" qualified_name="orangecontrib.tomwer.widgets.reconstruction.NabuVolumeOW.NabuVolumeOW" project_name="tomwer" version="" title="nabu volume reconstruction" position="(840.0, 370.0)" />
        <node id="5" name="volume viewer" qualified_name="orangecontrib.tomwer.widgets.visualization.VolumeViewerOW.VolumeViewerOW" project_name="tomwer" version="" title="volume viewer" position="(987.0, 440.0)" />
        <node id="6" name="data viewer" qualified_name="orangecontrib.tomwer.widgets.visualization.DataViewerOW.DataViewerOW" project_name="tomwer" version="" title="data viewer" position="(848.0, 242.0)" />
    </nodes>
    <links>
        <link id="0" source_node_id="0" sink_node_id="1" source_channel="data" sink_channel="data" enabled="true" />
        <link id="1" source_node_id="1" sink_node_id="2" source_channel="data" sink_channel="data" enabled="true" />
        <link id="2" source_node_id="2" sink_node_id="3" source_channel="data" sink_channel="data" enabled="true" />
        <link id="3" source_node_id="3" sink_node_id="4" source_channel="data" sink_channel="data" enabled="true" />
        <link id="4" source_node_id="4" sink_node_id="5" source_channel="data" sink_channel="data" enabled="true" />
        <link id="5" source_node_id="3" sink_node_id="6" source_channel="data" sink_channel="data" enabled="true" />
    </links>
    <annotations />
    <thumbnail />
    <node_properties>
        <properties node_id="0" format="literal">{'_scanIDs': [], 'controlAreaVisible': True, 'savedWidgetGeometry': None, '__version__': 1}</properties>
        <properties node_id="1" format="literal">{'_ewoks_default_inputs': {'data': None, 'dark_ref_params': None}, '_rpSetting': {}, 'controlAreaVisible': True, 'savedWidgetGeometry': None, '__version__': 1}</properties>
        <properties node_id="2" format="literal">{'_ewoks_default_inputs': {'data': None, 'axis_params': {'MODE': 'sino-coarse-to-fine', 'POSITION_VALUE': None, 'CALC_INPUT_TYPE': 'transmission_nopag', 'ANGLE_MODE': '0-180', 'USE_SINOGRAM': True, 'SINOGRAM_LINE': 'middle', 'SINOGRAM_SUBSAMPLING': 10, 'AXIS_URL_1': '', 'AXIS_URL_2': '', 'LOOK_AT_STDMAX': False, 'NEAR_WX': 5, 'FINE_STEP_X': 0.1, 'SCALE_IMG2_TO_IMG1': False, 'NEAR_POSITION': 0.0, 'PADDING_MODE': 'edge', 'FLIP_LR': True, 'COMPOSITE_OPTS': {'theta': 10, 'oversampling': 4, 'n_subsampling_y': 10, 'take_log': True, 'near_pos': 0.0, 'near_width': 20}, 'SIDE': 'left', 'COR_OPTIONS': ''}, 'gui': {'mode_is_lock': True, 'value_is_lock': False, 'auto_update_estimated_cor': True}}, '_rpSetting': {}, 'controlAreaVisible': True, 'savedWidgetGeometry': b'\x01\xd9\xd0\xcb\x00\x03\x00\x00\x00\x00\x01}\x00\x00\x00\xe2\x00\x00\x06\x03\x00\x00\x03\xa9\x00\x00\x01}\x00\x00\x01\x07\x00\x00\x06\x03\x00\x00\x03\xa9\x00\x00\x00\x00\x00\x00\x00\x00\x07\x80\x00\x00\x01}\x00\x00\x01\x07\x00\x00\x06\x03\x00\x00\x03\xa9', '__version__': 1}</properties>
        <properties node_id="3" format="literal">{'_ewoks_default_inputs': {'data': None, 'nabu_params': {'preproc': {'flatfield': 1, 'double_flatfield_enabled': 0, 'dff_sigma': 0.0, 'ccd_filter_enabled': 0, 'ccd_filter_threshold': 0.04, 'take_logarithm': True, 'log_min_clip': 1e-06, 'log_max_clip': 10.0, 'sino_rings_correction': 'None', 'sino_rings_options': 'sigma=1.0 ; levels=10', 'tilt_correction': '', 'autotilt_options': ''}, 'reconstruction': {'method': 'FBP', 'angles_file': '', 'axis_correction_file': '', 'angle_offset': 0.0, 'fbp_filter_type': 'ramlak', 'padding_type': 'edges', 'iterations': 200, 'optim_algorithm': 'chambolle-pock', 'weight_tv': 0.01, 'preconditioning_filter': 1, 'positivity_constraint': 1, 'rotation_axis_position': '', 'translation_movements_file': '', 'clip_outer_circle': 1, 'centered_axis': 1, 'start_x': 0, 'end_x': -1, 'start_y': 0, 'end_y': -1, 'start_z': 0, 'end_z': -1, 'enable_halftomo': 0}, 'dataset': {'binning': 1, 'binning_z': 1, 'projections_subsampling': 1}, 'tomwer_slices': 'middle', 'output': {'file_format': 'hdf5', 'location': ''}, 'phase': {'method': 'Paganin', 'delta_beta': '100.0', 'padding_type': 'edge', 'unsharp_coeff': 0, 'unsharp_sigma': 0, 'ctf_geometry': ' z1_v=None; z1_h=None; detec_pixel_size=None; magnification=True', 'beam_shape': 'parallel', 'ctf_advanced_params': ' length_scale=1e-05; lim1=1e-05; lim2=0.2; normalize_by_mean=True', 'ctf_translations_file': ''}, 'configuration_level': 'optional', 'mode_locked': False, 'cluster_config': None}}, '_rpSetting': {'preproc': {'flatfield': 1, 'double_flatfield_enabled': 0, 'dff_sigma': 0.0, 'ccd_filter_enabled': 0, 'ccd_filter_threshold': 0.04, 'take_logarithm': True, 'log_min_clip': 1e-06, 'log_max_clip': 10.0, 'sino_rings_correction': 'None', 'sino_rings_options': 'sigma=1.0 ; levels=10', 'tilt_correction': '', 'autotilt_options': ''}, 'reconstruction': {'method': 'FBP', 'angles_file': '', 'axis_correction_file': '', 'angle_offset': 0.0, 'fbp_filter_type': 'ramlak', 'padding_type': 'edges', 'iterations': 200, 'optim_algorithm': 'chambolle-pock', 'weight_tv': 0.01, 'preconditioning_filter': 1, 'positivity_constraint': 1, 'rotation_axis_position': '', 'translation_movements_file': '', 'clip_outer_circle': 1, 'centered_axis': 1, 'start_x': 0, 'end_x': -1, 'start_y': 0, 'end_y': -1, 'start_z': 0, 'end_z': -1, 'enable_halftomo': 0}, 'dataset': {'binning': 1, 'binning_z': 1, 'projections_subsampling': 1}, 'tomwer_slices': 'middle', 'output': {'file_format': 'hdf5', 'location': ''}, 'phase': {'method': 'Paganin', 'delta_beta': '100.0', 'padding_type': 'edge', 'unsharp_coeff': 0, 'unsharp_sigma': 0, 'ctf_geometry': ' z1_v=None; z1_h=None; detec_pixel_size=None; magnification=True', 'beam_shape': 'parallel', 'ctf_advanced_params': ' length_scale=1e-05; lim1=1e-05; lim2=0.2; normalize_by_mean=True', 'ctf_translations_file': ''}, 'configuration_level': 'optional', 'mode_locked': False, 'cluster_config': None}, 'controlAreaVisible': True, 'savedWidgetGeometry': b'\x01\xd9\xd0\xcb\x00\x03\x00\x00\x00\x00\x02\x03\x00\x00\x014\x00\x00\x05|\x00\x00\x03{\x00\x00\x02\x03\x00\x00\x014\x00\x00\x05|\x00\x00\x03{\x00\x00\x00\x00\x00\x00\x00\x00\x07\x80\x00\x00\x02\x03\x00\x00\x014\x00\x00\x05|\x00\x00\x03{', '__version__': 1}</properties>
        <properties node_id="4" format="literal">{'_ewoks_default_inputs': {'data': None, 'nabu_volume_params': None, 'nabu_params': None}, '_rpSetting': {'start_z': 30, 'end_z': 35, 'gpu_mem_fraction': 0.9, 'cpu_mem_fraction': 0.9, 'use_phase_margin': True, 'postproc': {'output_histogram': 1}, 'new_output_file_format': '', 'new_output_location': '', 'cluster_config': None}, 'controlAreaVisible': True, 'savedWidgetGeometry': b'\x01\xd9\xd0\xcb\x00\x03\x00\x00\x00\x00\x02\xd5\x00\x00\x01\xbc\x00\x00\x04\xaa\x00\x00\x02\xf3\x00\x00\x02\xd5\x00\x00\x01\xbc\x00\x00\x04\xaa\x00\x00\x02\xf3\x00\x00\x00\x00\x00\x00\x00\x00\x07\x80\x00\x00\x02\xd5\x00\x00\x01\xbc\x00\x00\x04\xaa\x00\x00\x02\xf3', '__version__': 1}</properties>
        <properties node_id="5" format="literal">{'controlAreaVisible': True, 'savedWidgetGeometry': None, '__version__': 1}</properties>
        <properties node_id="6" format="literal">{'_viewer_config': {}, 'controlAreaVisible': True, 'savedWidgetGeometry': None, '__version__': 1}</properties>
    </node_properties>
    <session_state>
        <window_groups />
    </session_state>
</scheme>

Now we can convert this file to an ewoks workflow file. For this we will use the ewoks convert command. In our case we will choose JSON as the format

ewoks convert processing_data.ows processing_data.json

Then you can open the .json file. Settings of the different task are defined by the ‘default_inputs’ keys like:

{
    "directed": true,
    "multigraph": false,
    "graph": {
        "id": "processing_data",
        "label": "Ewoks workflow 'processing_data'",
        "schema_version": "1.0"
    },
    "nodes": [
        ...
        {
            ...
            "default_inputs": [
                {
                    "name": "axis_params",
                    "value": {
                        "MODE": "sino-coarse-to-fine",
                        "POSITION_VALUE": null,
                        "CALC_INPUT_TYPE": "transmission_nopag",
                        "ANGLE_MODE": "0-180",
                        "USE_SINOGRAM": true,
                        "SINOGRAM_LINE": "middle",
                        "SINOGRAM_SUBSAMPLING": 10,
                        "NEAR_WX": 5,
                        "FINE_STEP_X": 0.1,
                        "SCALE_IMG2_TO_IMG1": false,
                        "NEAR_POSITION": 0.0,
                        "PADDING_MODE": "edge",
                        "COMPOSITE_OPTS": {
                            "theta": 10,
                            "oversampling": 4,
                            "n_subsampling_y": 10,
                            "take_log": true,
                            "near_pos": 0.0,
                            "near_width": 20
                        },
                        "SIDE": "left",
                        "COR_OPTIONS": ""
                    }
                },
            ],
            "id": "2"
        },
        ...
    ],
    "links": [
        ...
    ]
}

And you can execute the .json file the same way you execute the .ows file. See Execute the worflow from the command line Interface

Note

Widgets not doing any processing (like pure visualization widget) will be converted to an ‘ewoks neutral task’. Simply ‘passing’ the dataset or the volume to the next task when necessary. Otherwise it will execute the run method of the ewoks Task class. Each tomwer task has implemented this function. Task classes are stored in tomwer.core.process.

On this page