Setting up the workflow¶

The starting point for operating the workflow is the Start page. To open an existing workflow, double-click its name in the list on the Start page. To create a new workflow, click the New workflow button on that page.

Understanding the setup controls¶

When you create or open a workflow, the workflow is displayed in the workflow designer. The main elements of the workflow designer are as follows:

The workflow design surface is a diagram showing the blocks and the links between them. Here you can add, modify, or delete blocks and links, as well as connect blocks to workflow inputs and outputs. At the top of the diagram there is a command menu and a toolbar for a number of tasks related to the workflow setup.
The Workflow contents panel is displayed by clicking the button on the toolbar. Here you can review the contents of all composite blocks and select any block or the entire workflow to view or edit their properties.
The Block library panel is displayed by clicking the button on the toolbar. From the Common library area on this panel, you can add new blocks to the workflow. The Workflow library area represents a library of blocks that were added to the workflow.
The block configuration panel is used to view or change basic properties and settings of the block, including its inputs and outputs. This panel appears if any block is selected in the workflow diagram or in the Workflow contents panel. The Composite properties panel for configuring composite blocks differs from the Block properties panel for configuring other blocks. The panel for configuring the workflow is completely similar to the composite block configuration panel and appears if the root element is selected in the Workflow contents panel, or the [In] or [Out] area is selected in the workflow diagram.
The Run setup panel is used to view or change the workflow input parameters and their values. Here you can also specify which data should be stored in the workflow run results. The Run setup panel appears if no block is selected in the workflow diagram. To deselect all blocks, click in an empty area of the workflow diagram.

Adding and configuring blocks and links¶

To add blocks to the workflow, open the Block library panel by clicking the button. Blocks are added by dragging them from this panel onto the workflow design surface. You can also add a block by double-clicking its name in the Block library panel.

Select (click) a block in the workflow diagram to view or change its basic settings in the block configuration panel. If the panel does not show up, click the button on the toolbar. Double-click the block to open a dialog where you can view or change all of the block's configuration settings. For their description, refer to the documentation for this block.

Hover over the block in the workflow diagram to display the link handle . To create a link between two blocks, drag the link handle from one block to another. A dialog will appear prompting you to specify which ports you want to link.

The links between blocks in the workflow diagram are indicated by arrows. Hover over the arrow to view the links it represents. Double-click the arrow to open a dialog where you can view and modify those links.

Managing composite blocks¶

There are two ways to add a composite block to the workflow:

Group existing blocks and their links by placing them in a new composite block, and then proceed to editing the workflow within that block. To create a composite block in this way, select one or more blocks in the workflow diagram and click the button on the toolbar.
Create an empty composite block and then proceed to composing a workflow within that block. To create an empty composite block, click the button on the toolbar.

To go inside a composite block, hover over it and click the icon that appears. The workflow inside a composite block is built in the same way as the entire workflow: you can add blocks and links to it, configure inputs and outputs, etc. To exit a composite block, hover over the [In] or [Out] area in its workflow diagram and click the icon that appears.

Configuring inputs and outputs of a composite block¶

To configure the inputs and outputs of a composite block, use the Composite properties panel. In this panel, you can add inputs and outputs, as well as manage their settings, including:

Enable batch mode

If batch mode is enabled, the block expects to receive a certain array of values for processing on this port. The computations will be performed for each element of the array; concurrent processing of multiple elements is possible (see Parallelization settings). The block output will contain an array of computation results. If batch mode is enabled on multiple ports, the workflow must ensure that the input array size is the same on all those ports.
Uplink

Connects this port to the input or output of the parent block. For details, see Uplinks.
Make this port a workflow parameter

If the port is a parameter, its value can be set when configuring workflow runs. For details, see Configuring parameter input and results output.
Add to the workflow results

If a port is added to the workflow results, the results pane displays the value of that port after the workflow run is completed. For details, see Running workflow and viewing results.

You can add input and output ports for a composite block from its Composite properties configuration panel. However, the ports added in this way by default have no connection with the workflow enclosed within the block. In this case, additional configuration is required by creating uplinks.

Uplinks¶

Like any block, a composite block has inputs and outputs. The difference is that within a composite block, its inputs and outputs can be linked to inputs and outputs of its child blocks. Such links are called uplinks as they propagate child inputs and outputs up the block hierarchy.

Uplinks enable you to connect blocks that are outside of a composite block with its child blocks. For example, suppose you want to connect the output of block A, which is outside of the composite block, to the input of block B, which is inside the composite block. This requires the following steps:

In the composite block, add an input port and create an uplink from that input port to the input port of block B.
Connect the output port of block A to the input port of the composite block that you prepared in the previous step.

There are two ways to uplink a certain port within a composite block:

Depending upon the port direction, add an input or output in the Composite properties panel, and then create a link by dragging the link handle to the [In] area (in case of input port) or to the [Out] area (in case output port).

or
Select the block whose port you want to uplink, and then, in the block configuration panel, click the uplink button next to the name of that port. Depending upon the port direction, this will add a new input or output to the composite block and link it with that port.

From a configuration perspective, the entire workflow is similar to a composite block. As with configuring a composite block, you can link certain ports of the child blocks of the workflow to its inputs and outputs. However, unlike the inputs and outputs of a composite block, the outputs of a workflow serve to issue its results, and the inputs are used to pass parameters to the workflow.

Configuring parameter input and results output¶

At the beginning of the workflow execution, it is usually necessary to assign initial values to some ports. To do this, you can use the default values stored in the block settings. However, changing the initial values in this case requires changing the settings for each port separately. It is more efficient to use workflow parameters to change the initial values for multiple ports at once.

The values of all parameters can be preset collectively when setting up the run of the workflow. If a parameter value is specified, then it persists on the respective port until the port receives a different value during the workflow execution. The parameter value overrides the default port value, if any was set on the port when configuring block properties.

Upon completion of the workflow execution, it is normally required to save the values of certain ports for further analysis. These ports need to be added to the workflow results. The port values will be saved and, after the workflow execution is completed, they will remain available in the run directory.

Managing parameters and their presets¶

The workflow parameters are listed under Input parameters in the Run setup panel. For a port to be listed, it must have the setting enabled in the block configuration panel for the respective block. As the Input parameters area also lists all blocks of the workflow, you can add ports to the parameter list as follows:

Hover over the name of the block and click the icon that appears at the end of the respective entry in the list.
Select the desired ports in the pop-up box that appears.

In the parameter list, the parameter value is displayed next to the port name. Click a parameter value to open the edit box. The parameter value can be changed either directly in the edit box or using the value editor. To open the editor, click the icon at the right end of the edit box.

Each set of parameter values specified in the Run setup panel is called a preset. Multiple presets can be created. Before starting a workflow run, it is possible to select the desired preset, and the workflow run starts with the parameter values that are specified in the selected preset.

The name of the currently selected preset is displayed in the Preset field at the top of the Input parameters area. Initially, there is only one preset named default. To manage presets, click the icon at the right end of the Preset field and use the commands from the menu that appears. For example, to create a new preset, click Add preset.

A new preset is created by copying the current one, and has the same parameter values. In the new preset, you can change parameter values to meet your requirements. If there are several presets, the desired one can be selected from the drop-down list in the Preset field. All parameters will be assigned values from the selected preset.

Managing results output¶

The ports included in the workflow results are listed under Results in the Run setup panel. By default, the list includes all outputs of the workflow. For other ports to be included in the workflow results, they must have the setting enabled in the block configuration panel for the respective block. As the Results area also lists all composite blocks of the workflow, you can add ports to the results list as follows:

Hover over the name of the block and click the icon that appears at the end of the respective entry in the list.
Select the desired ports in the pop-up box that appears.

Only ports of composite blocks can be included directly into workflow results. As for ports of other blocks, uplinks must be leveraged to store port values in the workflow results. For example, you could uplink the port to an output of a composite block, and then add that output to the results list.

Managing block memory limits¶

pSeven Enterprise can monitor block memory usage and forcefully stop blocks with high memory consumption, if there is not enough memory to run other blocks. This behavior depends on the resource management settings, which only administrators can change.

When pSeven Enterprise forces a block to stop, the workflow run fails with a message like:

Block Your block name error: not enough memory, or the block exceeded its memory limit.

To avoid that error, specify memory limits to your blocks:

Open the workflow and select the block that was stopped by pSeven Enterprise.
In the Block properties pane, expand Settings at the bottom and set a memory limit.

To find out how much memory your block needs, you can review its memory usage statistics in the previous runs:

Open a completed run and open the Workflow contents pane on the run tab. The pane will display the peak memory usage for each block.
Find the block that was forced to stop and note its memory usage.
If you have more runs of the same workflow, check them too. Find the maximum value of the block's memory usage across all runs.
Set the block's memory limit according to the value you have found. For greater stability, add 10-20% on top of that maximum.

When you set a memory limit for a block, pSeven Enterprise will start the block only if that amount of free memory is available, and will reserve this memory for the block while it runs. The block, in turn, must respect the limit: if its memory consumption exceeds the limit, pSeven Enterprise will force it to stop after a short grace period.

Any block with a memory limit set has priority over all blocks without limits. If there is a shortage of memory, pSeven Enterprise will begin to stop the blocks without limits, while the memory-limited blocks will continue running, as long as they respect their memory limits.

How to check if resource management is enabled

Resource management, with its memory usage monitoring, is a system-wide feature, which may be enabled or disabled by pSeven Enterprise administrators. To find out if it is enabled in your deployment, run an empty workflow and see the first message in its run log.

If you see the following message:

Resource management is disabled. pSeven Enterprise disregards block resource requirements and does not limit block resource usage.

then resource management is disabled. In this case, you can change the block resource settings and save them, but they will apply only if your workflow is moved to another pSeven Enterprise deployment, where resource management is enabled.
If you see the following message:

Total run requirements: memory: 512 MB, cpu: 0.01 cores, jobs: 1

then resource management is enabled. In this case, the block memory limit settings apply as described in this section.

Updating blocks to a newer version¶

After the release of a new version of a block, the prototype of that block is first updated in the common library. The pSeven Enterprise administrator performs this update during the deployment of the new release. For reference, you can view the version number of the block in the Block library panel: when you hover over a block in that panel, the version number is displayed in a pop-up box.

Even after the deployment of a new release is complete and the blocks are updated in the common library, the blocks are not updated automatically in the existing workflows. Each workflow has its own block library, which you can view under Workflow library in the Block library panel. Updating blocks in the workflow is performed from this library.

After updating blocks in the common library, some of the blocks in the workflow may have a lower version number than in the common library. In the Workflow library area, an update notification appears next to the name of each block that has a lower version number. Click this notification to update all occurrences of the block in the workflow. You can roll back the update using the Undo command if, for some reason, the newer version of the block is not suitable for this workflow.