Program¶

The Program block lets you embed any program into your workflow, provided that it supports command-line execution and file-based input and output (batch mode). The block automates program launches and handles file transfers between your storage in pSeven Enterprise, the program, and other blocks in the same workflow. It is often used with Text blocks set up to generate input files for the program and parse its output files.

Getting started¶

Commonly the Program block is used to run heavyweight, resource-intensive software installed on an extension node - a high-performance Windows workstation or server connected to pSeven Enterprise over the network. To set up the block, you'll need:

The network name of the computer where your software is installed. This computer will be the run host for both the Program block and the software.
The command that launches your software from the command line on the run host. You have to be familiar with this command and its options.
In particular, the command options used to specify inputs, such as the model or project to run, and the location for outputs, such as a file or folder where the software will save its results.
Access to the run host and a user account there, so you can sign in to test commands locally and gather troubleshooting information in case of errors.

Linux software

The block can also run Linux software, but this software must be installed on every Kubernetes cluster node accessible to pSeven Enterprise. Ask your admin whether this is possible in your deployment.

By default, there is no software on the cluster nodes except system packages and Python provided by pSeven Enterprise. Technically you can run Python scripts with the Program block - however, in most cases the Python script block is more practical for this.

A basic example of a command used for a normal local software launch is toysolver -i "D:\Project X\input.dat" -o "D:\Project X\result.dat" where toysolver is the name of the program executable, -i and -o are options specifying the input and output paths. Depending on the installation type of your software, running it might require a full path to the executable, such as C:\Programs\ToySolver\toysolver.exe instead of just toysolver.

Given this example command, general steps to set up the block are:

Prepare example input files. Sign in to the computer where the software is installed (the run host) and test running the software in the normal command line there, using the example files. Make sure that your commands complete without errors and produce expected results.
Add a Program block to your workflow and select it to view its settings in the Block properties pane. Program blocks are entirely configured in this pane; opening the block just displays its configuration overview.
At the top of the Block properties pane, under "Run on:", select the run host.
Add an input file to block configuration:
1. Click next to the Inputs header in Block properties and choose Input file from the drop-down menu.
2. In the Add an input file dialog that opens, enter the label you want to use for this file - for instance: settings file. Note this is not the file name - it's only a label that you'll use in the command.
3. Click OK in the dialog, and a new input port Input files.settings file appears in Inputs. This is the port that receives the input file.
Similarly, configure an output file: click in Inputs, select Output file, enter label results file. Two new ports will appear:
- The Output files.results file port in Outputs. This port outputs the result file.
- The Output files.results file input port in Inputs. This is an optional setting used to specify the save location for the output file. Output files are temporary by default - they exist just long enough to pass them to downstream blocks, then are automatically deleted.
On the Command line input port, enter the same command as you've used when running the software normally on the run host, but replace file paths with the ${file label} placeholders.
```
toysolver -i "${settings file}" -o "${results file}"
```
Pass the input file to the Input files.settings file port:
- select an existing file anywhere in your storage or
- link it to an output port of another block that generates the input file - for instance, the Output file port of a Text block set up to generate program input files.
Link the Output files.results file output port to a downstream block that processes the program output - for instance, to the Input file port of a Text block set up to parse program output files.
If you want to save a copy of the program output file, edit the Output files.results file port value: open the Edit value dialog and replace the default ~ (means a temporary file) with the name or relative path.

Allowed save locations for output files

Blocks cannot write files outside the current workflow run directory. It is recommended that you keep the default file location selected in the dialog ("Block working directory") and only change the path. Another allowed location is "Workflow or run" with a relative path - with such settings, the block will save the file to the run directory root.

During a workflow run, pSeven Enterprise sends the block to the run host you've selected and starts it there. Here's the breakdown of what happens next, assuming that all settings not mentioned in the above example are left default.

A temporary working copy of the input file is created locally on the run host. The block gets the absolute local path to that copy.
The block gets an absolute local path for the output file. This is the path where the program output file must be found after the program finishes; otherwise the block cannot output that file nor save it with the workflow results.
A temporary working directory for the block is created on the run host.

Blocks use safe working copies of files to avoid conflicts

Every block instance gets unique paths and directory names to avoid file conflicts.

For example, if you run the same workflow twice without any changes, there will be two instances of the block running on the same host at the same time. Moreover, both of them will run with the same settings for the input and output files. However, each instance will run from its own working directory, use an own working copy of the input file, and save the output file in a specific directory - so conflicts won't occur.
The block replaces ${settings file} and ${results file} in the command with the absolute local paths to the input and output file.
The block changes to its local working directory and executes the command. This sets the block working directory as the current working directory to the program. Note that input and output files aren't copied to the working directory by default. (1)
1. If your software requires input files in its working directory, or an input folder with a certain structure, see the example in Input files.
You can test this kind of program launch in Windows

The program launch performed by the block is essentially similar to running a batch script that creates temporary directories and working copies of files for the program.
```
set input_file=D:\Project X\input.dat
set output_file=D:\Project X\workflow runs\run #0004\result.dat

set workflow_run_tmp=%temp%\workflow-TestWorkflow-run0004-%random%

set input_tmp=%workflow_run_tmp%\inputs-%random%
mkdir %input_tmp%
set input_working_copy=%input_tmp%\input.dat
copy %input_file% %input_working_copy%

set output_tmp=%workflow_run_tmp%\outputs-%random%
mkdir %output_tmp%
set output_working_copy=%output_tmp%\result.dat

set block_working_directory=%workflow_run_tmp%\block-Program-%random%
mkdir %block_working_directory%
cd %block_working_directory%

toysolver.exe -i %input_working_copy% -o %output_working_copy%

copy %output_working_copy% %output_file%
```
This example script is not an exact representation of the block behavior; it is greatly simplified for illustration purposes. Key points:
- Working copies of files are created in temporary directories.
- Directory names (hence, working paths) are generated during the run.
- Input and output files are not in the working directory and are not copied into there by default.
- In general, input files are located in different temporary directories. Each file might be placed in a directory of its own.
While the program runs, its console output - all messages it normally prints to screen - is captured by the block, displayed in the Run log pane, and saved into a temporary log file on the run host.
Once the program exits:
- If the program has run without errors, and its output file is found at the path required by the block (substituted for ${results file}):
  - The output file is copied from the run host to your storage as you've specified using the Output files.results file input port. If you've accepted the default temporary file setting, this file will appear in the block's directory in the current run, and will be automatically deleted later.
  - That file is also output to the Output files.results file port.
  - The block outputs the exit code produced by the program (the Exit code output port) and a completion signal to the @go output port.
- If the program exits with an error, or the output file is not found as required:
  - The block raises an error, which by default stops the current workflow run.
  - The block does not send anything to its output ports.
  - The block saves a full log file in its directory in the run results for troubleshooting. That file is named command.log by default.

This section provides a basic example, assuming that most settings - in particular, logging and error handling - are left default. In most cases, your tasks will require certain adjustments to that basic configuration, which depend on the features of the software you're integrating, desired workflow behavior and so on. Further sections describe the block settings in more detail and provide guidelines for their usage in several common setups.

Input files¶

The general rule for input files is that to pass a file as a program input, you have to add that file to the block configuration, thus creating an input file port. After this, the actual location of the file no longer matters: the port allows you to select an input file from any folder in your personal storage, or any workspace, or send that file from another block in the same workflow, or select files from the computer where your software is installed.

The trade-off is that the block will create temporary working copies of all input files and folders, generally in different locations on the local filesystem. This means that you shouldn't make assumptions about the location of those copies or about the file and folder structure in the program's working directory. If the program supports command line options for input and output files, such assumptions are usually irrelevant, as you can simply pass their paths by adding the ${file label} placeholders in the command line.

If your software requires a specific name for the input file, or a certain location of input files, or a certain input folder structure, you'll have to run additional commands to prepare the input before launch.

Move an input file to a required folder

Assume that toysolver.exe doesn't support the -i option for the input file, and instead only reads INPUTS\input.dat from a subfolder in the directory where it runs.

Before the run command, add the commands to create the INPUTS subfolder in the current working directory and move the input file there. Join the commands with &&.

mkdir INPUTS && move "${settings file}" INPUTS && toysolver -o "${results file}"

See also Assembling program input from multiple sources for guidelines about working with an input project folder where you need to replace or add certain files. Those guidelines rely on batch scripting; check the Running scripts section first.

Output files¶

The block doesn't track what output files the program creates and where do they appear, and doesn't save those files by default. It will only process the files that exist at the paths designated as outputs by the block - such as the path substituted to the command line in place of the ${results file} placeholder in the getting started example. These paths are not preset: every time the block runs, it generates a new path for every output file you've configured. That is, you have to add output files to the block configuration to obtain those paths and pass them to the program.

If, after the program exits, any output file isn't found at its required path, the block stops with an error by default. If your software doesn't provide a command line option to specify an output file (writes it to a preset location), you'll have to add commands that move the program output file to the required path.

Move an output file to the to a path required by the block

Assume that toysolver.exe doesn't support the -o option for the output file, and instead always writes it to OUTPUTS\result.dat in the directory where it was run (the working directory).

After the run command, add the command to move the output file to the path required by the block. Join the commands with &&.

toysolver -i "${settings file}" && move OUTPUTS\result.dat "${results file}"

All output files are temporary by default: the block uploads them from the run host to the current run directory - this is required to pass them to other blocks - however those uploaded copies are deleted automatically afterwards. To save an output file, set the save location using that file's input port, such as the Output files.results file input in the getting started example. It can be:

a path relative to the block working directory in the current run directory (select "Block working directory" in the Edit value dialog), or
a path relative to the root of the current run directory (select "Workflow or run" in the dialog).

Note that the block cannot write output files outside the current run directory. That is, writing to the workflow folder and other folders is prohibited for blocks.

If you want to use a temporary file but a downstream block requires a certain file extension, set the filename to ~.{extension} - for instance, ~.prt. With this setting, the block will output a temporary file with the specified extension, and that file will be deleted afterwards.

Finally, you can temporarily disable a file output without deleting it, which might be useful for troubleshooting: unset the Output files.{your file label} input port value (clear the path in the Edit value dialog), and the block will not output nor save that file. To revert this, reset the port value.

Input and output folders¶

Folders are generally set up in the same way as files. In addition, they support the optional file copy mask setting - the {your folder label} mask input port. It is useful if a folder contains files you don't need.

By default (empty mask), folders are copied to and from the run host with all their contents. If you add a mask, the block copies only the files and subfolders that match the mask.

Example masks

*.dat - all files with the .dat extension in the folder root; doesn't match such files in subfolders
**\*.dat - same but also matches .dat files in all subfolders
result-?? - files and subfolders named result-01, result-02, and so on

Program options as ports¶

Programs often provide command line options - additional run settings such as -n {number} for the number of solver threads. For instance:

toysolver.exe -n 4 -a -R -i "D:\Project X\input.dat" -o "D:\Project X\result.dat"

In this example, you can add ports that set options like this:

Click in the Inputs pane, select Substitution, label it N threads.
The Substitutions.N threads input port will appear. Set its value to 4.
Add another substitution, label it Options.
Set the Substitutions.Options value to -a -R.

On the Command line port:

toysolver -n ${N threads} ${Options} -i "${settings file}" -o "${results file}"

This way you can change program settings by editing port values or linking the substitution ports to other blocks.

Environment variables¶

Your program might read settings from environment variables, or require a certain variable to be set - for example, a variable containing a license file path. To ensure that all required variables exist when launching the program, add and set them in the block configuration.

For example, assume that toysolver.exe requires its license path in the TS_LICENSE_FILE variable.

Click in the Inputs pane, select Environment variable.
In the Add environment variable dialog, enter the exact name of the variable: TS_LICENSE_FILE.
The Environment variables.TS_LICENSE_FILE input port will appear. Set the license file path as its value.

With those settings, the block will set TS_LICENSE_FILE to the port value before launching the program. This will not change the value of the system variable TS_LICENSE_FILE, if it already exists on the run host. All environment variables defined in the block are set temporarily and only for the program (commands, script) it launches.

You can test this kind of program launch in Windows

Setting environment variables via the block is similar to running your program from a new command prompt where you set variables for this prompt only.

Open a new command prompt window.
Input set TS_LICENSE_FILE=C:\Toy Solver\TS 0123 4567.lic. This sets TS_LICENSE_FILE in the current window only, doesn't require the TS_LICENSE_FILE existing in the system, and doesn't change the value of that system variable if it exists.
In the same command prompt, run the program or a batch script you use to test the program launch. They will inherit the TS_LICENSE_FILE setting from the current window, not the system.

Alternative method, which in some cases is more convenient, is when you set variables right in the command line, before the program run command.

set TS_LICENSE_FILE=C:\Toy Solver\TS 0123 4567.lic && toysolver.exe ${options} -i "${input}" -o "${output}"

Or, if the program reads settings, input and output paths from environment variables:

set TS_OPTIONS=${options} && set TS_INPUT=${input} && set TS_OUTPUT=${output} && toysolver.exe

That second example is also the way to pass substitutions and file paths when running scripts.

Logging¶

By default, the block captures all messages, which your commands normally print to screen (the stdout and stderr streams), and redirects them to the run log. You can view that log in the Run log pane on a workflow run tab. This includes messages from the program you launch as well as other commands such as echo or dir, which are generally useful in troubleshooting.

If you don't want any command logs in the workflow run log, set the Logging.Show command log input port to No. With this setting, the block will only log its own errors and warnings - for instance, if it can't find a program output file you've specified.

If the program logs to a file, set the Logging.Command log file input port to the location of that file. While running, the block will continuously read that file and copy new messages from there to the run log and to the full log file.

To save the full log inside the current run directory in your storage:

Set the save location on the Logging.Log file input port. Default is the command.log file in the block working directory in the current run.
By default, that file is only saved if the command exits with an error. You can set the Logging.Save log file input port to save the log always or never.

The saved file contains the full block logs and command logs, as well as messages read from the additional log file, if you've set Logging.Command log file.

Handling command errors¶

If an error occurs while running the program or any of your additional commands, the block also stops with an error by default. In this case, it does not send anything to its output ports and does not save any files except the full log, if this is allowed by your Logging.Log file and Logging.Save log file settings.

If you switch the Command error input port to "Best effort", the block suppresses command and program errors. In this mode, after a command error, the block finishes normally and sends outputs for troubleshooting.

The command exit code is output to the Exit code port.
If your Logging.Log file and Logging.Save log file settings enable saving the full log file, it is saved and also output to the Log file port.
File and folder output ports, such as Output files.{your file label} and Output folders.{your folder label}, send program output files if they exist. If they don't, such ports output the paths where those files and folders were supposed to exist.

This behavior enables designing workflows robust to program errors.

If the downstream blocks that receive files from Program can handle missing input files: simply select "Best effort" on the Command error port and link file outputs directly to those blocks.
Otherwise, use a Condition block to check the exit code sent by Program to the Exit code output. Set up links between Condition and the downstream blocks so they get the input files only if that code is 0, which is normally the only success code a program can send. If you have defined a custom list of success codes (the Success codes input port), set up Condition to check that the exit code is in that list.

Using exit codes¶

To detect program errors, the block uses program exit code. It is an integer value that every program sends to the operating system when it finishes, errors, crashes or otherwise terminates its process. In Windows, this is also known as error level or the built-in ERRORLEVEL variable in batch scripts.

If your workflow stops due to a command error in a Program block, usually there is a line in the run log that tells the command exit code. Commonly encountered error codes are:

1 - general error
2 - user error: wrong option name, incorrect command syntax
127 - system command not found; typically used in Linux
9009 - system command not found; this code is used by Windows

A program can define its own exit codes. General convention is that 0 is the success code and is the only such code, and errors are indicated by exit code 1 or greater. Unfortunately, this convention is not enforced by OS, and some programs abuse that by always returning 0 - even on error - or returning a non-zero code to indicate a normal exit with a warning.

To find out the exit codes used by your program, first check its documentation and command line help. If the codes are not documented, try and run the program from a command line but add incorrect options, invalid or missing inputs and so on. After each run, type echo %ERRORLEVEL% - it outputs the exit code of the preceding command.

Checking command exit codes

rem Normal launch assuming the input file exists and is valid, the project folder is writable and so on.
rem Exit code should be 0.
toysolver -i "D:\Project X\input.dat" -o "D:\Project X\result.dat"
echo Exit code: %ERRORLEVEL%

rem Typo in a system command (`dir /o`).
rem Exit code should be 9009 since the command after && is not executed after an error.
die /o && toysolver.exe
echo Exit code: %ERRORLEVEL%

rem Invalid option -BAD.
rem Exit code should be 1 or greater, typically 2.
toysolver -BAD -i "D:\Project X\input.dat" -o "D:\Project X\result.dat"
echo Exit code: %ERRORLEVEL%

If the program returns non-zero exit codes on success, or you want to ignore certain errors, add those codes to the list on the Success codes input port.

Exit code 124 is set by the block

If you have set a timeout, the block sets the exit code to 124 if it stops the program by timeout.

If the program returns 0 (the success code) even on errors, check the program logs for error messages. Since the block captures program logs, it can analyze them and detect failure patterns in log messages. Use the Logging.Fail pattern input port to add such rules. Its value is a pattern or list of patterns; any log line matching any of those patterns is considered an error.

The block detects failure patterns in the logs only if the Command error port is set to "Block error" (default). If it is "Best effort", all patterns are ignored.

Error message patterns

Let error messages from the program begin with the line: "An error occurred!", and an actual error message follows.

Set Logging.Fail pattern to error occurred or any other substring usually found in program error messages.
Let error messages start with various prefixes like "[error]", "[failure]", "[FATAL!!!]", or contain typical phrases like "unhandled exception" and so on:
- Open the Edit value dialog for Logging.Fail pattern.
- At the top left, select type: List[String].
- At the top right, add rows as you need, and enter a pattern per row.

If you're familiar with regular expressions, you can also use them to define failure patterns. Open the Edit value dialog for Logging.Fail pattern, select type: RegEx or List[RegEx] at the top left, and input your expressions.

Command timeout¶

On certain errors, the program might stop responding but will not exit (hang). In such cases, the block will wait indefinitely for the program to finish, and consequently the workflow will hang waiting for the block.

To avoid such deadlocks, you can set a hard time limit using the Timeout input port. Once that limit is reached, the block interrupts the program, shuts down all its child processes, and raises an error, which is handled according to your setting on the Command error port. The block also sets the exit code to 124.

The Timeout port accepts time in these formats:

String: 1d 2h 34m 56s - days, hours, minutes, seconds. Supports variations like 1d 12h, 5h, 30m and so on. Spaces are optional.
String: 26:34:56 - common format, hours:minutes:seconds. Uses hours greater than 24 to denote days.
Integer: 95696 - time in seconds.

Running scripts¶

The Program block actually sends commands to the system interpreter, so you can use all system command shell features: run multiple commands from a single line, launch scripts, call other interpreters such as PowerShell or even Python, which is provided by pSeven Enterprise.

General setup steps to run a script:

Prepare the script and upload it to the workflow folder. It can be a batch script (.bat), PowerShell (.ps), Python (.py), or anything else that the OS on the block's run host supports.
Workflow files are copied to every new run by default. You can keep the default behavior to save a snapshot of your script in run results. If you don't want this, select the script file in the Explorer sidebar and click the near its name or select Workflow files setup Exclude from runs from the menu in the title bar.
Add a new input file to the block configuration. Label it main script, for instance.
Set the Input files.main script port: select the script file you've uploaded.

Set the Command line port:

BatchPowerShellPython

"${main script}" [arguments]

powershell.exe -noprofile -executionpolicy bypass -file "${main script}" [arguments]

python "${main script}" [arguments]

The ${label} placeholders for files, folders, and substitutions don't work inside scripts. To pass file paths and substitutions to your script, set environment variables before the script run command.

In the command line:
```
set PRG_SETTINGS_FILE=${settings file} && set PRG_RESULTS_FILE=${results file} && set PRG_OPTS=${options} && "${main script}"
```
In the script:
```
@echo off

toysolver %PRG_OPTS% -i %PRG_SETTINGS_FILE% -o %PRG_RESULTS_FILE%
```

Other settings such as logging and error handling work in the same way as for commands. You can also log additional details and define your own error exit codes in the script.

Check that a required output file exists, exit with code 22 if it doesn't

if exist "%PRG_RESULTS_FILE%" (
    echo Found result file: %PRG_RESULTS_FILE%
) else (
    echo [error] Result file not found at %PRG_RESULTS_FILE%
    exit /b 22
)

Programs without options for inputs and outputs¶

Some programs don't provide any command options that would allow you to pass the input and output paths, and can only read or write files in the current working directory (the folder they were run from).

The Input files and Output files provide basic examples for such cases. They can be combined into a single command line via the && operator, or you can run a script, which is more convenient when working with multiple input and output files.

For instance, assume the program requires that the input files are named "MAIN.IN" and "OPT.IN", always writes outputs to a subfolder named "RESULTS", and also creates a report file in the current directory with the current date in its filename, like "REPORT-2026-01-12.TXT".

In the command line, set environment variables containing the file paths to pass them to the script.

set PRG_IN1=${file1} && set PRG_IN2=${infile2} && set PRG_OUT=${outfolder} && set PRG_REP=${report} && "${script}"

In the script, move input files to the current working directory, run the program, then move program outputs to their required paths.

@echo off

:: %CD% stands for the current directory.
:: Note the quotes: the paths stored in variables can contain spaces.
move "%PRG_IN1%" "%CD%\MAIN.IN"
move "%PRG_IN2%" "%CD%\OPT.IN"

:: List the current working directory contents.
:: The output is displayed in the workflow run log by default.
tree /f

:: Run the program.
toysolver.exe

:: Pass program results to the block.
if exist "RESULTS" (
    move "RESULTS" "%PRG_OUT%"
) else (
    echo [error] Result folder not found in "%CD%"
    exit /b 22
)
if exist "REPORT-*.TXT" (
    move "REPORT-*.TXT" "%PRG_REP%"
) else (
    echo [error] Report file not found in "%CD%"
    exit /b 22
)

Moving input files and folders to the current working directory (%CD%) is safe because every running block instance gets its own working directory on the run host, and that directory is initially empty. Even if you launch several runs of the same workflow with identical Program block settings, the block instances in those runs will use different working directories on the run host.

Assembling program input from multiple sources¶

Programs that take a project folder as an input typically require a certain structure in that folder. If your workflow uses two or more blocks to prepare input files, or you select those files from different folders in your storage, you'll have to run a script that assembles the input project folder with the required structure.

For instance, assume the block receives an input folder with the base project (a project template) and an input file that should replace a certain file in that project.

In the command line, set environment variables containing the input and output paths to pass them to the script.

set PRJ_BASE=${base project} && set NEW_GEOM=${geometry file} && set PRJ_OUT=${outfolder} && "${script}"

If the program provides command line options to specify inputs and outputs, move the input file to the input folder and run.

@echo off

:: Note the quotes: the paths stored in variables can contain spaces.
move /y "%NEW_GEOM%" "%PRJ_BASE%"
toysolver -i "%PRJ_BASE%" -o "%PRJ_OUT%"

If the program requires a project in the current working directory, move the input project and input file there, run the program, then move the updated project to the required output path.

@echo off

:: %CD% stands for the current directory.
:: Note the quotes: the paths stored in variables can contain spaces.
move "%PRJ_BASE%" "%CD%\working_project"
move /y "%NEW_GEOM%" "%CD%\working_project"

:: List the current working directory contents.
:: The output is displayed in the workflow run log by default.
tree /f

:: Run the program, point it to the copy of the project in the current directory.
toysolver "%CD%\working_project"

:: Pass the updated project back to the block.
move "%CD%\working_project" "%PRJ_OUT%"