Changelog

v2023.11

Updates

• Updated keyboard navigation in the Edit value dialog:

  • Down and Up while editing move to edit next/previous cell.
  • Tab and Shift+Tab while editing move right/left.
  • Enter finishes editing but no longer moves down (replaced by Down).
  Keyboard navigation in the Edit value dialog

  

• A REST API client working with an AppsHub app can now check that app for changes. The new "fingerprint" key in app details provides a unique identifier, which changes with any change in the app.

  GET apps/{app ID}/ and find "fingerprint"

  {
      "name": "Disk optimization",
      "path": "/Apps/Disk optimization.p7wf",
      ...
      "state": "READY",
      "fingerprint": "0e8e4146f23e18686db40211c502f94c"
  }
  

• Upgraded the embedded pSeven Core package to version 6.50. For details on recent improvements and changes, see the pSeven Core changelog.

• Updated pSeven Enterprise Help:
  • Added the Design space exploration block guide.
  • Added notes on managing block memory consumption to Setting up the workflow.
  • Improved search capabilities and fixed a few issues with search.

Avoid using environment variables prefixed DA__P7__ in Python script blocks

Due to a bug in the Python script block, it is currently possible to read values of block runtime environment variables in the block's script. Those variables (with names starting in DA__P7__) are actually intended for block developers only. This bug will be fixed in future, thus the DA__P7__ variables will no longer be accessible from the user script in Python script. If you are currently using such variables in your scripts, update them beforehand to avoid future issues.

• DA__P7__BLOCK__WORKFLOW_DIR: use api.wfdir (see Reading files containing input data).
• DA__P7__BLOCK__WORKFLOW_RUN_DIR: use api.rundir (see Reading files containing input data) or the block working directory (see Working directories).
• DA__P7__BLOCK__TEMP_DIR: use api.tempdir (see Temporary file storage).

Deployment and administration

• Added a number of advanced parameters to values.yaml, using which you can set annotations and labels to all Kubernetes objects created by a pSeven Enterprise deployment.
• Updated the experimental mode where blocks run natively on Kubernetes (sizing.runOnKubernetes) to improve its stability, as a part of work to prepare it for use in production.

• Updated the pSeven Enterprise deployment guide:

  • Added the Hardware resources summary.
  • Added the Requirements summary.
  • Added Deployment overview.
  • Added details on deployment options - trial and production.
  The hardware summary for trial and for production

  

• Added the missing v2023.07 upgrade notice about the end of support for PostgreSQL 11 and the new PostgreSQL 12 requirement. If you already use PostgreSQL 12 or greater, you are safe to ignore this notice.

• Fixed a regression issue from v2023.09 where, if you set entrypoint.allowedHosts in values.yaml, no one could sign in to pSeven Enterprise because opening the sign-in page resulted in a "Bad host" error.
• Fixed an issue where a web application firewall (WAF) made Python script blocks unusable - users were getting a "HttpErrorResponse" message when trying to save the block or test the script. Note that users will have to update the Python script blocks in their workflows to resolve this issue.
• Fixed an issue where a workflow run interrupted by user entered the Stopping state but never actually stopped, and you could not manually stop it in the admin interface.
• Fixed a stability issue where users could not open any block for editing, if the htcondormanager service is malfunctioning.

Warning

The DATADVANCE Registry address changes to registry.pseven.io. The former address docker.pseven.datadvance.net keeps working but will be disabled in 3 months.

User block development

• Fixed a regression issue from v2022.06, which affected the compatibility of newer pSeven Enterprise versions (v2022.06 and above) with the user blocks developed for the older versions (v2022.05 and below) that work with custom data types. Due to this issue, for example, when editing a port value which is a custom table type, incomprehensible error messages appeared when adding or removing table rows.

Bugfixes

• Fixed an issue where, if you stop a workflow run, it entered the Stopping state but never actually stopped.
• Fixed an issue with workflow references where, if both the main and the referenced workflows are being edited at the same time, numerous error messages appeared on the main workflow tab, and the runs of the main workflow stopped with an error.
• Fixed a "value too long for type character varying(1024)" error when opening a workflow run that failed.
• Fixed the Ctrl+V hotkey behavior in lists and dictionaries in the Edit value dialog.
• Fixed incorrect behavior of the Edit value dialog when editing nested dictionaries.
• Fixed an issue with the Text block where configuring it to run on a Windows node caused a "No module named 'astor'" error when running the workflow.
• Fixed a vulnerability issue in the table, list, and dictionary editors in the Edit value dialog.
• Fixed an issue where pSeven Enterprise blocked unpacking an archive with high compression rate, because it was incorrectly recognized as a ZIP bomb.
• Fixed several UI inconsistencies in the AppsHub Runs pane.
• Fixed UI issues in Explorer related to moving files in the Files folder:
  • Fixed drag-and-drop not working inside Files.
  • Fixed using cut and paste, which seemed to copy the file instead of moving it.

pSeven Enterprise upgrade notice

When upgrading, make sure that you use the new DATADVANCE Registry address, registry.pseven.io. The former address (docker.pseven.datadvance.net) is working but will be disabled in 3 months - update early to avoid registry access errors in future.

v2023.09

Updates

• The new Uncertainty quantification (beta) block enables uncertainty simulation and reliability analysis studies in pSeven Enterprise.

  Reliability analysis in the Uncertainty quantification (beta) block

  

• The new Text (beta) block provides for integrating the software, which uses input and output text files, and exchanging data between that software and other blocks in a workflow.

  The Text (beta) block being configured to parse a program output

  

• Python script block:

  • Added the signal @go ports, which you can use to order the execution of Python script blocks in a workflow - see Using signal ports).
• The storage usage tooltip in Explorer now lists the space used by your files and AppsHub app run results.
• Upgraded the embedded pSeven Core package to version 6.49. For details on recent improvements and changes, see the pSeven Core changelog.

Avoid using environment variables prefixed DA__P7__ in Python script blocks

Due to a bug in the Python script block, it is currently possible to read values of block runtime environment variables in the block's script. Those variables (with names starting in DA__P7__) are actually intended for block developers only. This bug will be fixed in future, thus the DA__P7__ variables will no longer be accessible from the user script in Python script. If you are currently using such variables in your scripts, update them beforehand to avoid future issues.

• DA__P7__BLOCK__WORKFLOW_DIR: use api.wfdir (see Reading files containing input data).
• DA__P7__BLOCK__WORKFLOW_RUN_DIR: use api.rundir (see Reading files containing input data) or the block working directory (see Working directories).
• DA__P7__BLOCK__TEMP_DIR: use api.tempdir (see Temporary file storage).

Deployment and administration

• Fixed a regression issue from v2023.08 where the deployment stopped with an error, if you use an NFSv3 file storage server.
• Fixed a vulnerability issue in the pSeven Enterprise authentication service (Keycloak).
• Help updates:
  • Updated and corrected the information about pSeven Enterprise network connections and ports - see the new Network parameters section in the pSeven Enterprise deployment guide.
  • Added a page explaining the pSeven Enterprise release lifecycle policy - see Release lifecycle.
  • Added a section to the administration guide that covers information security incidents and related operations - see Information security incidents and operations.

User block development

• Fixed an issue with blocks that run on Windows extension nodes and launch the CAD/CAE packages installed on that node, where the launch failed if the package requires reading the user registry hive (HKEY_CURRENT_USER).

Bugfixes

• Design space exploration block (beta):
  • Fixed issues with variables set up to switch to constants during a workflow run (using the {variable name}.Constant ports).
  • Fixed an issue where you had to open the block to reset some parameter mapped to a port, because resetting the value of that port in the Block properties pane did not work as intended.
  • Fixed an issue with N/A values in results where they could be displayed as empty strings, if the block configuration contains a string categorical variable.
  • Fixed an issue where the block stopped with an error if any of the result arrays is empty - for example, no feasible points were found.
• Fixed a version compatibility issue with workflows created in pSeven Enterprise v2022.05 or earlier where you could not open the workflow runs saved earlier, or the list of workflow runs appeared empty even though saved runs exist - for example, when you get the list of workflow runs using REST API.
• Fixed an issue where copying a folder to the same location a few times caused an error.
• Fixed an issue where incorrect values of block's peak memory usage were output to the workflow run log - the logged peak value could be less than the average usage, or could decrease during a workflow run.
• Fixed an issue with using macOS Safari where file downloads from pSeven Enterprise stopped with an error.
• Fixed a minor UI issue where the button sometimes appeared in the list of Composite block inputs or outputs even if you did not select any ports to delete.
• Fixed a UI issue in AppsHub where, if you publish the same workflow a few times, the AppsHub main page displayed several app duplicates until you reload that page.
• Fixed minor UI issues with long names of app runs in AppsHub.

v2023.08

Updates

• You can automate publishing workflows to AppsHub using the pSeven Enterprise REST API. See Workflow publishing in the REST API guide.
• FMI support via FMPy. The fmpy module is available in Python script blocks and user blocks.
• Design space exploration block (beta):
  • The Adaptive design for GP technique now supports mixed designs with all types of variables.
• While block:
  • No longer in beta. No new issues have been discovered during the extended stability testing period of the block, so its current version is considered stable.
• An update for pSeven Enterprise deployments with run resource management enabled (optional feature, disabled by default): you are no longer required to set the memory and CPU limits for blocks, so it is possible to grant a block unlimited resources. However, blocks with unlimited resources have lower priority and may be forced to stop by pSeven Enterprise, if there are not enough free resources to run other blocks that do declare their memory and CPU requirements.
• Upgraded the embedded pSeven Core package to version 6.48. For details on recent improvements and changes, see the pSeven Core changelog.

Avoid using environment variables prefixed DA__P7__ in Python script blocks

Due to a bug in the Python script block, it is currently possible to read values of block runtime environment variables in the block's script. Those variables (with names starting in DA__P7__) are actually intended for block developers only. This bug will be fixed in future, thus the DA__P7__ variables will no longer be accessible from the user script in Python script. If you are currently using such variables in your scripts, update them beforehand to avoid future issues.

• DA__P7__BLOCK__WORKFLOW_DIR: use api.wfdir (see Reading files containing input data).
• DA__P7__BLOCK__WORKFLOW_RUN_DIR: use api.rundir (see Reading files containing input data) or the block working directory (see Working directories).
• DA__P7__BLOCK__TEMP_DIR: use api.tempdir (see Temporary file storage).

Deployment and administration

• The "on-demand" block execution strategy is now default for new deployments, as it appears to be the preferred configuration choice in most of the known use-cases. The "all-at-once" strategy is now an option, supported for backwards compatibility.
• Added an experimental feature to run workflow blocks natively on Kubernetes. When enabled, pSeven Enterprise creates a Pod for each running block instance, instead of placing blocks into slots provided by task executors. To enable, see the sizing.runOnKubernetes parameter in values.yaml.
• Fixed an extension node deployment issue where the deployment stopped with a "Failed to install C++ redistributable" error, if a Microsoft Visual C++ Redistributable package is already installed on the node.
• Fixed a security issue where a block created by a user could gain access to files of other users.
• Fixed an issue where pSeven Enterprise could not delete some of the temporary files while cleaning up the user temporary files during a version upgrade.
• Updated the pSeven Enterprise deployment guide to explain how to get the full version number from the Helm chart.

User block development

• The FMPy module is now available in the block runtime environment by default, so blocks that work with FMUs can simply import fmpy.
• Fixed a regression issue from v2022.06, which affected the compatibility of newer pSeven Enterprise versions (v2022.06 and above) with the user blocks developed for the older versions (v2022.05 and below). Due to this issue, for example, certain valid port values in those blocks - such as empty tables - were marked as invalid by pSeven Enterprise and caused errors. This issue appeared because pSeven Enterprise v2022.06 and above use the new value format, which was developed to be compatible with the old value format used in v2022.05 and below, but in fact did not provide the compatibility in full due to a few defects in implementation. Since the previously developed blocks use the old value format, those incompatibilities between the old and new formats were causing various issues with port values in those blocks.

Bugfixes

• Design space exploration block (beta):
  • Fixed a regression issue from v2023.07 where the block could stop with an error while processing the results of Surrogate-based optimization or Adaptive design.
• Fixed an issue which caused minute-long delays in response to certain workflow editing actions - for example, when setting up links, moving blocks, or viewing their properties.
• Fixed an issue where the Workflow contents pane displayed incorrect values of peak memory usage, because the displayed peak value could decrease during a workflow run.
• [2836] Fixed an issue where pSeven Enterprise displayed errors when you try to edit a workflow, if that workflow is referenced from another.
• Fixed an issue where pSeven Enterprise could not remove some of the temporary files created while running workflows, so part of your storage space was occupied by those leftover files.

pSeven Enterprise upgrade notice

Since v2023.08, the "on-demand" block execution strategy is preferred over the "all-at-once" strategy (previous default). In new deployments, the default strategy is now "on-demand". If you are currently using the "all-at-once" strategy, consider switching to "on-demand" during the upgrade: see the sizing.blockRunStrategy parameter in the deployment configuration file values.yaml.

v2023.07

Updates

• If you select a workflow block that can be updated to a newer block version, the Block properties pane displays an update notification.

  The Block properties pane with an outdated block selected

  

• Design space exploration block (beta):

  • Updated the block UI to replace its old built-in version of the Edit value dialog with the current version used in pSeven Enterprise Studio.
• Upgraded the embedded pSeven Core package to version 6.47. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

pSeven Enterprise v2023.07 requires PostgreSQL 12

pSeven Enterprise is no longer compatible with PostgreSQL 11 as this version of PostgreSQL is at the end of life. If you use PostgreSQL 11, update to version 12 or 13 before you upgrade pSeven Enterprise to v2023.07. Updating from PostgreSQL 11 to 14 or greater is not recommended because of the changes in the authentication schema introduced by PostgreSQL 14.

• If your deployment does not include Windows extension nodes (see Extension nodes), you can slightly reduce its resource consumption by disabling the pSeven Enterprise services that handle connections to the extension nodes: see the new sizing.enableExtensionNodes parameter in the deployment configuration file values.yaml.
• Fixed an issue with Windows extension nodes where a node could not connect to other computers in its network - for example, license servers required by the software installed on that node - because all node's network traffic was redirected to pSeven Enterprise.
• Updated the Administration guide with a note describing the logging subsystem with changes introduced in v2023.05, explaining the log cache and providing some guidelines about PostgreSQL database maintenance - see Understanding run logs storage.
• The Open-source components page provides a list of third-party open-source software components used in pSeven Enterprise, their names, links to source, licenses, and copyright notices.

User block development

• Ports with custom types can now specify the placeholder displayed to users when there is no value assigned to the port. To add it, set the @nullStr property in the port schema. For example, {allOf: [{$ref: "r"}, {"@nullStr": "Auto"}]} sets up a port that expects a floating point value and displays Auto when no value is set.
• Fixed an issue with input ports of enumeration type, where users could not change the value assigned to such a port in the Block properties pane.

Bugfixes

• Fixed an issue with workflow run launch where, if the number of ongoing runs has reached the license limit, pSeven Enterprise canceled every new run with an error. Now such runs are placed in a queue, and pSeven Enterprise starts them automatically when a license is available.
• Fixed an issue with the Run log pane where leading whitespaces were removed from every log line, breaking indentation in log messages.
• Fixed an issue where a workflow run stopped with an error if the workflow contains a driver block that is set up to run on a Windows extension node.
• Fixed an issue with blocks running on a Windows extension node where string values of block runtime environment variables were encoded incorrectly if they contain Unicode characters.
• Fixed an issue with workflows created in previous versions of pSeven Enterprise (v2023.05 and below) where, if you have used the Undo command while editing such a workflow in newer pSeven Enterprise version, that workflow could be saved in an incorrect state and become unusable, and reopening it resulted in a "Document initialization error" message.
• Fixed an issue where a block description in the Block properties pane could be reset when you save a block.
• Fixed minor UI issues with the Mail notifications checkbox in the user menu.

pSeven Enterprise upgrade notice

Previous versions of pSeven Enterprise are compatible with PostgreSQL 11 or greater. pSeven Enterprise v2023.07 is no longer compatible with PostgreSQL 11 and requires at least PostgreSQL 12 (version 11 is at the end of life).

If you are using PostgreSQL 12 or greater, no action is required.

If you are using PostgreSQL 11, update it to version 12 or 13 before you upgrade pSeven Enterprise to v2023.07. Note that updating to PostgreSQL 14 or greater is not recommended in this case because of the changes in the authentication schema introduced by PostgreSQL 14.

v2023.06

Updates

• Added a shortcut to preview a workflow app UI - click in the workflow edit toolbar or select the Open app UI command from the toolbar menu. As before, you can also preview the app UI by opening the index.html file from the workflow UI folder in Explorer.

  The app UI preview command in the edit toolbar menu

  

• You can edit multiple workflow references at once: select the reference blocks you want to edit and use the Reference another workflow... command from the menu in the workflow edit toolbar. After you confirm the new workflow choice in the workflow reference dialog, all blocks you had selected will reference that new workflow. This might be useful, for example, when you need to switch to an updated version of a child workflow, which is referenced by several blocks in the main workflow.

  The workflow reference command in the edit toolbar menu

  

• Upgraded the embedded pSeven Core package to version 6.46. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

• Fixed a vulnerability issue where a disabled user account could gain API access to pSeven Enterprise.
• Fixed a number of CVE vulnerabilities in the pSeven Enterprise images.
• Fixed an issue with Windows extension nodes where a node could not establish connection with pSeven Enterprise and hence was not displayed to users in pSeven Enterprise Studio, if that node was allocated to the same subnet as the Kubernetes cluster nodes.
• Fixed an issue where the pSeven Enterprise images contained test files used in vulnerability and stability testing routines, which were recognized as security threats by protection utilities.
• Cleaned up the administrative interface, removing various unused and disabled buttons and commands.

• Added initial WebDAV support: users can mount their pSeven Enterprise home folders as WebDAV shares, provided that they have WebDAV access enabled.

  WebDAV support is a beta feature

  WebDAV access is a beta feature, which is currently disabled by default for all users. To enable it, set the app_auth | user | Can use WebDAV protocol user permission. Users with that permission can get the access URL using the Copy HTTP/WebDAV URL command from the Explorer pane menu.

• Added a note about the WebSocket protocol requirement to the Reverse proxy configuration notes in the pSeven Enterprise deployment guide.

User block development

• Fixed an issue with the example block implementation (Example user block.p7protoblock) which potentially caused instability when running workflows that include blocks based on that example: there was no timeout and no retries after sending a request from the block to pSeven Enterprise, so if the block did not receive a response (for example, due to network connectivity issues), it could wait indefinitely or raise an error, thus disrupting the workflow run.

  Review your block code as it may require an update

  If you develop and support your own blocks, it is advisable to review their implementation to avoid issues similar to the described above. For guidance, see the usage of the backoff module in the updated version of the user block example.

• Fixed an issue where the HOME variable in the block environment was unset, which caused an error if the block had to access the directory pointed by HOME.

Bugfixes

• Fixed a regression issue with REST API from v2023.03 where a correct authorization request returned a response with status code 500 Internal Server Error.
• Fixed a stability issue with the blocks packaged with pSeven Enterprise, which could reveal itself if the network connectivity within the deployment Kubernetes cluster is disrupted, causing unexpected behavior and various errors when running workflows - for example, a block that never finishes execution, or an unhandled error on a block startup.
• Design space exploration block (beta):
  • Fixed an issue with the Adaptive design for GP technique where the block stopped with an irrelevant error, if it received a NaN or ±Inf response value.
  • Fixed a rare issue where the block could unexpectedly pause at some intermediate solving iteration, causing the workflow run to wait indefinitely.
• Python script block:
  • Fixed an issue where the HOME environment variable was unset in the script, which caused an error when using certain modules - for example, skimage.
• Fixed an issue where pSeven Enterprise could redirect you to the sign-in page when you open a block or go to AppsHub.
• Fixed an issue where you received a 500 Internal Server Error message when opening a workflow app UI preview using a direct link to the index.html file in the workflow UI folder, if that workflow does not exist or belongs to another user.
• Fixed misleading error messages appearing in the workflow run log when a block was shut down due to exceeding its memory usage limit.

v2023.05

Updates

• You can set up workflow run logging to keep the full log file, regardless of its size (default), or to keep only a certain amount of recent logs, truncating the log file from the beginning once its size exceeds the limit you set. Log truncation might be useful in workflows that write extensive logs or run for an extended time, resulting in a large log. This is a per-workflow, per-run setting: you can enable or disable it independently for each workflow in the Run setup pane, prior to starting the run.

  The log truncation setting in the Run setup pane

  

Deployment and administration

• pSeven Enterprise v2023.05 increases the PostgreSQL database load, comparing to the previous versions. The increase is due to the update in the logging subsystem, which addresses several issues with workflow run logs. See the pSeven Enterprise upgrade notice below for details.
• Fixed a denial of service issue where the appworker* services, which handle user interaction, did not close their connections to the redis service properly, so after a certain time period (typically months) the Redis database connection pool was exhausted and it refused all new connections, leading to malfunction of the appworker* services. This caused the "ERR max number of clients reached" errors to users and finally could render pSeven Enterprise unusable due to inability to edit or run workflows.

Bugfixes

• Fixed a number of issues with the Run log pane where the log did not load properly, the "loading..." messages were displayed indefinitely, and common operations with the log (like scrolling or filtering) caused incorrect behavior. Run logs are now pre-processed on access, which improves the Run log pane stability and responsibility while viewing logs but introduces a pre-processing delay, noticeable in certain cases - for example, when you open a workflow run for the first time, or return to a workflow run that you have not opened lately.
• Fixed an issue where the "ERR max number of clients reached" errors appeared when editing or running workflows.
• Fixed an issue with the Composite block where it entered an extended wait, adding a noticeable delay in workflow execution, if one of its nested blocks had failed to stop properly or did not shut down within timeout.
• Fixed an issue where the run log message informing that a block in the workflow did not shut down properly was incorrectly issued as an error (now warning).
• Fixed an issue where a misleading error message "None has no attribute 'set_result'" appeared in the run log, if you stop the run manually.
• Fixed a rare issue with folder sharing where the file list in Explorer displayed empty after you have shared a folder and then revoked access to it.
• Fixed an error when opening a folder with a name containing certain characters - for example, backticks (`).
• Fixed an issue with the Python script block that sometimes caused a delay when finishing a workflow run - that is, all blocks in the workflow you run have finished working, but you had to wait a few minutes until the run finally stops.

pSeven Enterprise upgrade notice

In pSeven Enterprise v2023.05, the logging subsystem is updated with a new log pre-processing mechanism, addressing various issues experienced by users when viewing workflow run logs. With this update, each run log opened by a user is pre-processed and loaded to the PostgreSQL database for faster access. The pre-processed log is kept in the database and gets removed if the user does not access that run log for a certain time.

Consequently, v2023.05 increases the PostgreSQL database load and resource consumption, primarily the storage space usage due to temporarily keeping the pre-processed logs. Before you upgrade to v2023.05, it is advisable to review your database server configuration and increase the amount of resources available to the database, mainly the disk space. After you upgrade, monitor the resource usage on the database server to verify that it handle the increased load.

v2023.04

Updates

• While block (beta):
  • You can configure the block so it checks the input values against the condition and does not start the loop if the condition is not met. That mode is now default for all new While blocks created in pSeven Enterprise v2023.04 and above. Blocks in existing workflows created in previous versions keep their old behavior, which you can change using the Unconditionally run the first iteration toggle in the block configuration dialog. See the updated While block help for more details.
  • You can also use an input port value as a parameter in the condition - for example, configure the block to check a certain loop response value received at every iteration against a threshold value received to an input port once on start.
  • Updated the block help to more accurately and clearly describe its operation and setup, including its two operation modes and the new mode switch.
• For viewing convenience, logs in the Run log pane now use a monospaced font.
• Upgraded the embedded pSeven Core package to version 6.44. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

• pSeven Enterprise deployment and version upgrade no longer require you to manually copy secrets (the keycloak.clientSecret, jobcluster.*, and extensionNode.* parameters) to the deployment configuration file values.yaml. The secrets are now saved to Kubernetes and loaded during the upgrade.

  This update changes the set of secrets used by pSeven Enterprise. When upgrading an existing deployment, you will have to generate new secrets and apply them once during installation to save the secrets. Future updates will load those saved secrets automatically. See the pSeven Enterprise upgrade notice below for instructions.

• To avoid wasting storage space, pSeven Enterprise now cleans up user temporary files during a version upgrade.

• Fixed an issue where a failed workflow run entered the Stopping state but never actually stopped, and you could not manually stop it in the admin interface.
• Fixed an issue with the Python script block where its implementation contained files that are not required by the block but consume a certain amount of storage space for every instance of the block added to any workflow by users.
• Fixed an issue with the SMTP server settings in the deployment configuration file values.yaml, where pSeven Enterprise did not stop the deployment if both the SSL and TLS mail protocols are enabled (those are mutually exclusive) - so you could complete the deployment with such settings, after which pSeven Enterprise did not send email notifications to users.
• Fixed an issue where pSeven Enterprise deployment stopped with an error if there is a corrupt workflow run in some user folder.
• Fixed an issue where the user-specified CPU limit for a block had no effect if the "on-demand" block execution strategy (experimental feature) is enabled in your deployment.
• Updated the Kubernetes node disk space requirements in the pSeven Enterprise deployment guide.
• Updated the pSeven Enterprise deployment and version upgrade guides with regard to changes in secrets management.

Bugfixes

• Fixed several issues, which caused noticeable UI slowdowns when editing workflows - for example, when adding, renaming or moving blocks. If you still encounter such issues in pSeven Enterprise v2023.04, clear the workflow undo and redo history.
• Fixed a regression issue from v2023.02 where renaming a table column in the Edit value dialog caused all data in that column to be lost.
• Fixed a regression issue from v2023.02 where you could not get the workflow from an app you have published to AppsHub, if you have disabled that permission for other users when publishing.
• Fixed an issue with batch mode ports in Composite blocks where you could not assign a List value to the port (a batch of values to process).
• Fixed an issue where various file operations in the Explorer pane were taking unexpectedly long time - for example, copying files or calculating the size of a folder.
• Fixed an issue where a failed run entered the Stopping state but never actually stopped.
• Fixed issues with the Edit value dialog where it was unresponsive when editing big lists or dictionaries.
• Fixed an issue where you could create files (folders, workflows) with the same name in different case (for example, result and Result), which led to confusion and possible workflow run errors - for example, in workflows that contain blocks configured to run on Windows extension nodes, where file and folder names are case-insensitive.
• Design space exploration block (beta):
  • Fixed an issue with results in optimization and other tasks with minimization or maximization responses where, if several Pareto-equal solutions were found, only one of those solutions was included into the optimal designs set.
  • Fixed an Adaptive design technique issue with initial samples containing missing response values, where the block evaluated responses for the initial sample points that violate the variable bounds or have invalid variable levels.

pSeven Enterprise upgrade notice

The new secrets management engine introduced in pSeven Enterprise v2023.04 uses a new set of secrets, which you must create and apply during the upgrade. This is required only once, when upgrading from a pre-v2023.04 version to v2023.04 or above.

1. Before you upgrade, login to your Docker Registry and use the following commands to get the full version number and create the secrets file pseven-secrets.yaml for the new version:

   helm show chart pseven-{YYYY.MM.DD}.tgz | grep appVersion | cut -d" " -f2
   docker run --rm {registry}/pseven-secretgen:{version} > pseven-secrets.yaml
   

   For more information about these commands, see Secrets in the pSeven Enterprise deployment guide.

2. When setting parameters in the deployment configuration file values.yaml, do not copy the secrets related parameters keycloak.clientSecret, jobcluster.secret, jobcluster.token, and extensionNode.*. Those parameters are no longer used and must be removed from values.yaml.

3. When installing, ensure that the secrets file pseven-secrets.yaml is in the same directory as values.yaml. To apply the new secrets, add the -f pseven-secrets.yaml option to the helm install command, for example:

   helm install pseven-rl pseven-{YYYY.MM.DD}.tgz -f values.yaml -f pseven-secrets.yaml -n pseven-ns --timeout 60m --wait --debug | tee pseven-{YYYY.MM.DD}.log
   

   For more information about using this command, see upgrade steps in the Upgrading pSeven Enterprise section.

After completing with the upgrade, keep a backup copy of the secrets file pseven-secrets.yaml you have created.

v2023.03

Updates

• Several usability improvements related to workflow references:
  • You can create a workflow reference by dragging a workflow from Explorer and dropping it into the workflow you are editing.
  • A reference block in a workflow is outlined red, if it references a workflow that no longer exists.
  • Better handling of paths in the workflow reference dialog: you can use paths relative to your home folder and omit the .p7wf extension when referencing a workflow by path.
• You can copy and paste list and dictionary data between the Edit value dialog and Excel or another spreadsheet editor that uses a similar clipboard format (tab-delimited tabular data).
• All blocks packaged with pSeven Enterprise now show a warning, if you close a block configuration dialog with unsaved changes.
• Cleaned up the error messages printed to the workflow run log when some block stops with an error: moved error details to the debug log, removed extra messages.

Deployment and administration

• Fixed a version upgrade issue where the deployment of a new pSeven Enterprise version stopped with an error, if you have changed the keycloak.clientSecret parameter in the deployment configuration file values.yaml during the upgrade.
• Fixed an issue where a user who is not authorized to run blocks on some extension node could circumvent that restriction by configuring a Windows-only block to run on any available host.
• Updated information about minimum Kubernetes cluster and file storage resource requirements in the Prerequisites section of the pSeven Enterprise deployment guide (moved: see Requirements).
• Since pSeven Enterprise is commonly deployed in a dedicated Kubernetes namespace, corresponding instructions in the deployment and admin guides are updated to reflect that.

User block development

• Added clarifications to the user block guide about using the DEV tag to make the block run in development mode - in particular, this mode is required to disable the block code cache while testing and debugging a block on a Windows extension node. For details, see Tagging the block under development.
• In the block prototype example (Example user block.p7protoblock) packaged with pSeven Enterprise, development mode is now enabled by default (the DEV tag is set).

• Fixed a regression issue from v2022.06 where the Edit value dialog did not display the names of custom data types properly.

  The users of your block who have encountered this issue will have to refresh the block configuration saved with their workflows by opening the block window and saving its configuration once again. Note there is no need to make any changes in the current block configuration.

• Fixed an issue with blocks that run third-party engineering software (CAD/CAE packages) on Windows extension nodes, where the block could not launch the software package it uses, if that package needs to read the APPDATA or USERPROFILE Windows environment variable in order to start.

Bugfixes

• Design space exploration block (beta):
  • Fixed an issue with categorical variables in Adaptive Design, which caused an error, if you do not specify the Study target setting.
  • Fixed an issue with the Orthogonal Array technique, which caused an error, if all design variables are frozen (constant) except one continuous variable.
• Python script block:
  • Fixed an issue where the block did not show the source of a runtime error, if that error occurs in an additional Python module, which you import in your script.
  • Fixed an issue with editing default values of variables having the Any type where string values were not recognized unless you enclose them in double quotes. Also fixed a similar issue where True and False were not recognized as Boolean values.
  • Fixed an issue an issue with editing default values of variables where the block did not accept a default value you input, if you had changed the type of a variable in its settings.
• Fixed an issue with the Edit value dialog where you could only assign scalar values inside tables, lists, and dictionaries - for example, it did not support editing nested lists.
• Fixed an issue where pSeven Enterprise Studio UI was becoming generally unresponsive, if you have a few workflows opened in background tabs, and those workflows contain many links or blocks with many ports.
• Fixed issues with the Block properties pane, which caused unexpected behavior when editing port values - for example, the port value you input could be reset back to the value that was previously assigned to that port.
• Fixed a UI issue with value input fields where, if you input an invalid value and then reset it, the input field got a persistent red highlight.
• Fixed an issue where the AppsHub main page was taking too much time to load and looked broken, if you opened AppsHub from a device that has no Internet access, because rendering that page required downloading fonts from Google.
• Fixed an AppsHub issue with the stock app UI where it could reset a string parameter value when you open the Edit value dialog for that parameter.
• Fixed an AppsHub issue where an app could take a long time to load when you open the app page, if pSeven Enterprise was updated after that app had been published.
• Fixed several issues with handling missing (N/A) values and displaying them in the UI.
• Fixed an issue with the Block properties and Run setup panes where you could not change the port value if a Boolean value is currently assigned to that port.
• Fixed a UI issue with dropdown menus in the Runs pane in AppsHub.
• Fixed various minor UI issues.

v2023.02

Updates

• You can edit text and Markdown files in pSeven Enterprise - for example, if you want to make a quick edit in a Python script used by your workflow, you no longer have to download the script file to proceed with your edits. To open a file for editing, double-click it in Explorer, then click the Edit button in the file viewer.

  Workflow description opened for editing

  

• Added a more convenient way to change a workflow reference - select the reference block and click the referenced workflow name in the Workflow reference properties pane on the right to open the workflow reference dialog.

  Click the referenced workflow name to change it

  

• When you try to delete a workflow with ongoing runs or a folder containing such runs, you now get an additional warning before deletion to avoid accidentally interrupting and losing a run in progress.

• Design space exploration block (beta):

  • For the Surrogate-based optimization and Adaptive design techniques, the block now provides estimates of the total exploration budget and initial sample size. Note that the estimates require a complete problem definition - for example, they may be unavailable, if you map certain properties of variables or responses to input ports and do not set those properties in block configuration window.

    Exploration budget estimates in the Design space exploration block

    

• Python script block:

  • Minor usability improvements in the script testing UI (the Log and Results tabs).

• Upgraded the embedded pSeven Core package to version 6.42. For details on recent improvements and changes, see the pSeven Core changelog.

• To avoid page loading issues, pSeven Enterprise Help now uses your local system fonts instead of loading typefaces from Google Fonts, which required Internet access to connect to the Google font servers.

Deployment and administration

• Added one more branding option - you can set a custom page title for pSeven Enterprise by editing title.txt in the branding customizations folder (see Branding). Your custom title will be displayed to users in the title of the browser tab where they open pSeven Enterprise, and also on top of the AppsHub main page.

  A custom title in branding

  

• Fixed an issue where pSeven Enterprise incorrectly set owner of new folders and ZIP archives created by users, due to which users could exceed their storage quotas.

• Fixed an internal issue with Windows extension node configuration, which could cause errors in user workflows that run blocks on Windows nodes due to incorrect initialization of the block runtime environment on the node.
• Fixed mistakes (wrong and missing commands) in the Python installation verification instructions in section Node check of the extension node deployment guide.
• Minor updates and clarifications in the pSeven Enterprise deployment, administration, and version upgrade guides, as well as in the Technical support section.

User block development

• Fixed a regression issue from v2023.01 where the Block properties pane displayed a red highlight instead of a port value in certain cases: now such values are displayed as invalid and have a tooltip, which explains the reason.
• Added the vector data type to the pSeven Enterprise typesystem. A vector is a homogeneous flat array of numeric or string values.

Bugfixes

• Fixed an issue with workflow references where pSeven Enterprise did not restore the reference as expected, after you have deleted the referenced workflow and then restored it or created a new one with the same name and location as the deleted one.
• Fixed an issue with composite blocks that have a batch input port, which is also added to workflow results (both the and port settings enabled), where a workflow run with such a block stopped with an error once the block received a matrix value to its batch input port.
• Fixed inconsistencies of the AppsHub REST API, which caused issues when you need to develop a REST client that uses both the workflow and app REST APIs.
• Fixed an issue with the table editor in the Edit value dialog where sometimes you could not save a valid table - for example, after you have entered an invalid cell value and then fixed it.
• Fixed issues with behavior of the Run log pane when scrolling logs.
• Fixed a UI issue where button and menu command tooltips never disappeared if you work with pSeven Enterprise in Firefox, unless you refresh the pSeven Enterprise tab in the browser.
• Fixed a usability issue with the location bar in Explorer where, if you input a full path to a subfolder inside Runs, Files, or another special folder displayed in tree view, Explorer opened that folder but did not expand the tree to the subfolder you have specified in the path.
• Fixed an issue where pSeven Enterprise Help pages were taking a few minutes to load, if you opened Help from a device that has no Internet access.
• Fixed a minor UI issue with rendering block update notifications in the Block library pane, which were taking too much space if the block name in the pane is long enough.
• Fixed various minor UI issues in the Edit value dialog, Block library pane, workflow editor.

v2023.01

Updates

• Updated the Runs pane in AppsHub:

  • You can rename completed runs.

  • Runs in the list can be grouped by adding a common prefix to their names - the prefix is used as the run group name. The prefix should be separated from the run name with a dot (.).

    An app runs list with the new and old (archived) groups

    

  • Updated the run list sorting and filtering options.

• Clicking an app thumbnail on the main AppsHub page opens the app info page with a list of app runs.

  The app info page

  

• Pack any of your files into a ZIP archive using the new Zip command from the menu in Explorer to free up some storage space or to spend less time downloading a single large file.

  The Zip command in Explorer

  

• Redesigned the workflow reference dialog for more convenience (see Referencing workflows). You can select a workflow to be referenced from the list, which displays all workflows you have previously opened, and filter that list by typing a part of the workflow name. To select a workflow you have never opened yet, use the button or copy the workflow path into the dialog.

  A list of workflows in the add reference dialog, filtered by test in the name

  

• To reduce inconvenience caused by long port names while setting up links, the Links and Autolink dialogs are now resizeable.

• Updated and fixed the dictionary editor in the Edit value dialog.

• Design space exploration block (beta):

  • Reworked the handling of discrete variables in the Surrogate-based optimization and Adaptive design technique algorithms, improving result quality and algorithm stability in tasks with discrete variables.

• Python script block:

  • If you configure the block to run on a Windows extension node, the comtypes module is now available in the block's script.

• Upgraded the embedded pSeven Core package to version 6.41. For details on recent improvements and changes, see the pSeven Core changelog.

• Updated section Basic concepts in workflow guides to describe composite block information ports and parallelization settings (see Parallelization settings, Information ports).

Deployment and administration

• Fixed a deployment issue where specifying a non-default value to the entrypoint.allowedHosts parameter in the deployment configuration file values.yaml caused pSeven Enterprise malfunction after the deployment.
• Fixed a licensing issue where a license expired one day earlier than it should - at the beginning of its expiration date.
• Fixed a vulnerability issue where a user could start uploading a large file that exceeds the user's storage quota, consuming most of the free disk space in the user data storage and causing various issues for all pSeven Enterprise users.
• Fixed an issue where user data could exceed the user's storage quota - for example, certain files in workflow run results were quota-free.
• Fixed an issue with run resource management where a block running on a Windows extension node could exceed its memory usage limit but was not stopped properly, potentially consuming all memory available on the node.
• Updated the pSeven Enterprise deployment guide:
  • Added the requirement to use connection pooling on the PostgreSQL server - see Connection pooling.
  • Added notes about preventing known issues caused by inappropriate configuration of a reverse proxy (load balancer, ingress controller) - see sections Reverse proxy and Reverse proxy configuration notes.

User block development

• Implemented the code cache for blocks intended to run on Windows extension nodes. When a user first opens the configuration dialog of a Windows-only block, pSeven Enterprise copies the protoblock folder to the Windows node that hosts the block, where the protoblock is added to a user-specific local cache. This resolves certain issues previously caused by copying the protoblock over the network every time a user opens the block configuration dialog or runs a workflow containing Windows-only blocks. The protoblock that is cached locally on the node is updated only when required - for example, when pSeven Enterprise detects a newer block version in the user workflow library.

  Disable the code cache during development

  While you develop a block that runs on a Windows extension node, code cache for that block should be disabled because it interferes with testing and debugging the block. Switch your block to development mode by adding the DEV tag in the block's manifest - see section Tagging the block under development for instructions.

• Blocks intended to run on Windows extension nodes may now use the comtypes Python module.

• Enabled Python print() in the block code for debugging purposes.

Bugfixes

• Fixed an issue where you could not start a new workflow run after renaming or deleting a run in progress.
• Fixed an issue where you could upload a file larger than the available free storage space, which caused further errors. The file size is now checked before the upload.
• Fixed an issue where sometimes you could not set up a link in the Links dialog.
• Fixed an issue where you could not remove a workflow from favorites (the Starred list in Explorer) if that workflow is shared to you by another user.
• Fixed a UI issue where scroll did not appear in long menus that cannot fit into the browser window.
• Fixed an issue with the table editor in the Edit value dialog where pasting clipboard data to the dialog added an empty bottom row to the table.
• Fixed various minor UI issues in the block properties pane, matrix value editor, Condition block.

• Design space exploration block (beta):

  • Fixed a regression issue with the Adaptive design and Adaptive design for GP techniques from v2022.11 where, if the blackbox block that evaluates responses did not support batch evaluations, the workflow stopped with an error once the Design space exploration block sent a request to the blackbox block - even if the Batch mode option is disabled.
  • Fixed an issue with the Gradient-free optimization technique where the number of linear response evaluations was much higher than expected.
  • Fixed an issue with linear responses of the Evaluation type where results could contain values of such responses calculated using an internal response model (not real evaluations).
  • Fixed issues with checkboxes in the Ports and parameters dialog.

• While block (beta):

  • Fixed an issue where the Links dialog displayed only driver ports of the While block but not its input and output ports.

v2022.12

Updates

• Implement complex projects with interconnected workflows, where a main workflow can be composed of smaller child workflows created and maintained by different teams. The special Workflow reference block enables using any workflow as a block in one or more main workflows, without the need to copy and paste workflow parts or manually update them when the referenced workflow changes. For details, see section Referencing workflows.

  Referencing workflows

  

• Get an overview of all your app runs in AppsHub using the new Runs pane added to the main AppsHub page.

  • The Runs pane is placed on the left and is collapsed by default.
  • Click a run in the pane to open it in a new tab.
  • To delete a run you no longer need, hover it with your mouse cursor and click the button that appears.
  • If you are an app owner, you will see all runs of your app in the Runs pane, including the runs started by other users.
  The Runs overview pane in AppsHub

  

  Runs pane functions

  

• Use drag-n-drop in the tab bar to place workflow and run tabs in any order convenient for you.

• The new While block (beta) enables the creation of processing loops in a workflow without any programming - for example, you can use it to implement a loop with condition (a while loop) or with a given number of iterations (a for loop).

  The While block

  

• Improved startup times of blocks that run on a Windows extension node in the case when that node and pSeven Enterprise are located in different networks.

• Design space exploration block (beta):

  • Enabled the batch evaluation mode support for the Gradient-free optimization technique.

  • In optimization and adaptive design, if you enable batch mode, the Exploration budget setting regards the Maximum batch size option value and limits the total number of batches output for evaluation - that is, the number of the blackbox block startups. Previously, Exploration budget in batch mode limited the total number of response evaluations requested from the blackbox block, disregarding their distribution in batches - which is usually inconvenient in tasks that use batch evaluations.

    This change applies to the following techniques: Gradient-based optimization, Gradient-free optimization, Surrogate-based optimization, and Adaptive design.

• Upgraded the embedded pSeven Core package to version 6.40. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

• Fixed a regression issue from v2022.10 where pSeven Enterprise was unresponsive for a certain time following the deployment, caused by excessive NFS storage checks.

  As a part of this fix, the NFS protocol version to use (NFSv3 or NFSv4) is now specified in the deployment configuration file values.yaml. For details, see the pSeven Enterprise upgrade notice below.

• Fixed an issue where the extension node deployment stopped with an error if a certain user password length is required by Windows security policy on that node.

• Blocks running on Windows extension nodes may now create Python cache files (.pyc) on the node, which in some cases makes the block startup noticeably faster. The cache files are created in the %USERPROFILE%\.pseven\pycache directory of the user who runs the block.

User block development

• You can get a unique identifier of the current workflow run containing your block by reading the DA__P7__BLOCK__WORKFLOW_RUN_ID environment variable. This identifier may be useful, for example, if you need to track data generated by your block in different workflow runs.

• Blocks are now allowed to cache the Python bytecode (create .pyc files upon startup) when running on Windows extension nodes, unless bytecode caching is disabled in the block startup script for Windows start.bat - for example, the -B option is given to the Python interpreter. It is advisable to enable bytecode caching, as this behavior is default for Python, and it also improves the block startup time on an extension node when that node and the pSeven Enterprise deployment are located in different networks.

  Existing user blocks require an upgrade to enable bytecode caching

  The block prototype example (Example user block.p7protoblock) packaged with previous versions of pSeven Enterprise disables the bytecode caching by default - in the Windows startup script start.bat, the -B option is included in the run command:

  %DA__P7__RUNENV__PYTHON3__EXECUTABLE% -B -m userblock %*
  

  To enable bytecode caching in your blocks when running under Windows, remove the -B option from the start.bat startup script - for example:

  :: %DA__P7__RUNENV__PYTHON3__EXECUTABLE% -B -m userblock %*
  %DA__P7__RUNENV__PYTHON3__EXECUTABLE% -m userblock %*
  

  After that, publish the block update following the instructions in section Publishing block updates and guide the users of your block to upgrade it in their workflows as described in section Upgrading the block in the workflow of the user blocks guide.

Bugfixes

• Fixed a regression issue with REST API from v2022.10 where a result port description was an array instead of a string.
• Fixed an issue with the file open dialog where you could not select a file located inside a shared folder, because it was impossible to open a shared folder in the dialog.
• Fixed an issue with the file list filter in the Explorer pane where typing a partial name into the filter had no effect unless you add the * wildcards.
• Fixed an issue with the Explorer pane where the file list filter was not reset when you navigate to a different folder, causing confusion.
• Fixed an issue with user search in the file sharing dialog where usernames were case-sensitive.
• Fixed an AppsHub issue with the stock app UI where it used an old version of the Edit value dialog, which had different design and functionality compared to the current version used in pSeven Enterprise Studio.
• Fixed an AppsHub issue where you could not open an app with the stock app UI that has parameters mapped to certain special ports - for example, the ports that set Exploration budget and Study target values in the Design space exploration block.
• Design space exploration block (beta):
  • Fixed an issue with editing the Initial sample port value where the Edit value dialog opened in the JSON editing mode instead of a table editor.
  • Fixed an issue where the block did not write the intermediate result information to the run log while solving a task with categorical variables.
  • Fixed an issue where you could not set up a vector constraint that has only one bound, unless you explicitly specify that the other bound is infinite (add a +Inf or -Inf value).
  • Fixed various minor issues in the block UI.

pSeven Enterprise upgrade notice

pSeven Enterprise v2022.12 requires you to specify the NFS protocol version to use when connecting to the file storage NFS servers in the deployment configuration file values.yaml. The default is set to NFSv3, which is the correct setting if your NFS servers are configured as described in section Disabling NFSv4. If you are using NFSv4, or you are going to switch from NFSv3 to NFSv4 during the upgrade to pSeven Enterprise v2022.12, you must specify NFSv4 when preparing a deployment configuration file for the upgrade.

Previously, pSeven Enterprise determined the NFS protocol version automatically, which caused various responsiveness issues as it required testing the NFS connection when communicating with the file storage servers.

If you want to start using NFSv4, enable the NFSv4 protocol on your file storage servers before deploying the v2022.12 upgrade, and run deployment with the -⁠-⁠set fixStorages=true option. For instructions, see Enabling NFSv4 in the pSeven Enterprise upgrade notice for v2022.10.

1. Identify the NFS protocol version enabled on each of the file storage NFS servers. You can use the test described in section Disabling NFSv4:

   1. Create a test directory on the server and try mounting it using NFSv4.

   2. If a command like the following runs without errors, the server has NFSv4 enabled:

      $ sudo mount -t nfs -o vers=4 localhost:<test directory path> <mountpoint>
      

   3. If the above command outputs a "Protocol not supported" error, NFSv4 is disabled.

2. Prepare the updated deployment configuration file values.yaml as described in Upgrading pSeven Enterprise.

3. In the prepared deployment configuration file, set the following parameters specifying the NFS protocol version enabled on a corresponding file storage server:

   • storage.userdata.nfs.version
   • storage.appdata.nfs.version
   • storage.shareddata.nfs.version

4. If you have changed the file storage server configuration while preparing for the upgrade (enabled or disabled NFSv4), run the deployment with the storage check enabled, as noted in the upgrade guide: add the -⁠-⁠set fixStorages=true option to the helm install command - see the example in section Upgrading pSeven Enterprise. Note that the storage check may take some time depending on the amount of storage data.

v2022.11

Updates

• Design space exploration block (beta):

  • New optimization technique - Gradient-free optimization, which can provide better quality solutions in tasks with many discrete variables. The technique uses the Nevergrad solver by Meta AI Research.

    The Gradient-free optimization technique is not enabled in SmartSelection yet, so you have to select it manually to try it.

  • If you close a block configuration dialog with unsaved changes, you get a related warning.

• Updated the Python runtime environment version to 3.9 and updated versions of pre-installed Python packages. This version of Python is used in Python script blocks, user blocks, and on Windows extension nodes.

• Updated the top navigation bar in pSeven Enterprise Studio:
  • You can go from Studio to AppsHub using the dropdown menu at the left side of the bar.
  • Added a separate Help menu with links to the Getting started page and changelogs.
• Updated and fixed the list editor in the Edit value dialog.
• Upgraded the embedded pSeven Core package to version 6.39. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

• If you are logged into pSeven Enterprise Studio, you can use the new dropdown menu in the top navigation bar at its left to go to the admin interface.
• Updated the pSeven Enterprise deployment guide with details on using a reverse proxy to secure the deployment - see sections Reverse proxy, Reverse proxy configuration notes, and updated deployment diagrams.

User block development

• Fixed a regression issue from v2022.10 where a user block that does not handle the shutdown request from pSeven Enterprise caused incorrect workflow run behavior (the run never finished). Now, if a block did not stop within 1 minute after receiving the shutdown request, it is unconditionally forced to stop.

  Note that pSeven Enterprise will not allow any block to process the shutdown request longer than 1 minute. When implementing the shutdown request handler, you will have to make sure it can finish its job before that timeout expires.

• Fixed an issue where pSeven Enterprise did not set the idle timeout value specified in the block manifest, when you add the block to the workflow.

Bugfixes

• Fixed a regression issue from v2022.10 where you could not set workflow parameter values after you have created a run in the "Configuration" state.
• Fixed an issue where you were not redirected to the sign-in page after you have logged out.
• Fixed an issue with incorrect workflow run behavior after you rename the folder that contains your workflow or the run directory. Now you get a warning dialog when you try to rename a folder that contains a workflow run in progress.
• Fixed an issue where adding a .p7history file to a report caused an error.
• Fixed issues with the Runs sidebar in AppsHub:
  • If you delete an ongoing app run, it was stopped but not deleted, so you had to click again.
  • The sidebar did not show all existing app runs (created by any app user) to the app owner.
• Fixed an issue with editing Enumeration type values where the Edit value dialog required you to input the value manually instead of selecting it from a predefined list as intended.
• Fixed various minor UI issues in the Run setup pane and the Edit value dialog.

v2022.10

Updates

• Design space exploration block (beta):

  • Added optional per-variable and per-response result ports, which you can use to get the result values for specific input and output components of your task.
  • Added optional per-component initial sample ports.
  • Enable those ports in the Ports and parameters dialog in the block configuration.
  Optional per-component ports in the Ports and parameters dialog

  

• Python script block:

  • You can now rename request (argument) and response (function) ports - for example, to display human-readable names in the Block properties pane and the Links dialog, instead of Python variable identifiers.

    Port name settings in Python script

    

  • Added the code autoformatting toggle in the Configuration pane, which you can use to disable the default formatting if you find that more convenient.

    Code autoformatting toggle in Python script

    

  • Added the support for missing (N/A) values: it is now possible to pass arrays with missing values through ports, and pass N/A or None as a value of a float (Real), int (Integer), or str (String) variable.

• You can copy and paste table and matrix data between the Edit value dialog and Excel or another spreadsheet editor that uses a similar clipboard format (tab-delimited tabular data).

• Added the support for tables and matrices with missing (N/A) values.
• Added the support for string matrices.
• Various small improvements in the table and matrix editors in the Edit value dialog.
• Minor UI improvements in the Condition block.
• pSeven Enterprise is now case insensitive when checking names for uniqueness (workflows, reports, blocks, ports, and so on) - for example, you cannot add two inputs named Force and force to the same block, because those names are considered identical. This helps in avoiding various usability issues related to case-sensitive name validation.
• Upgraded the embedded pSeven Core package to version 6.38. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

• Added experimental support for NFSv4 file storages. Disabling NFSv4 on the file storage NFS server is now optional.

  If you are going to enable NFSv4 on the file storage NFS server during an upgrade of an existing deployment (which currently uses NFSv3), you will have to deploy with the storage check enabled (the -⁠-⁠set fixStorages=true option). See Enabling NFSv4 in the pSeven Enterprise upgrade notice below for details.

• Updated pSeven Enterprise for compatibility with Kubernetes versions 1.15.5 through 1.25.2, removing the "batch/v1beta1 CronJob is deprecated" warning that appeared during deployment on Kubernetes 1.21 and newer versions.

• To reduce some of the security risks related to allowing users to connect to a pSeven Enterprise deployment directly, all X-Forwarded-* headers in HTTP requests are now ignored by default.

  If your deployment is protected by a trusted reverse proxy, it is advisable to enable parsing the X⁠-⁠Forwarded⁠-⁠* headers in the deployment configuration file values.yaml. For details, see Deployment configuration in the pSeven Enterprise upgrade notice below.

• Improved stability by resolving several memory cache issues that could lead to restarts and unavailability of various pSeven Enterprise services.

• The SSH key for extension nodes is now stored in the deployment configuration file values.yaml (the new extensionNode.blockSSHGateClientKey parameter) and is saved on each extension node during its deployment, which slightly improves startup times of workflow blocks running on extension nodes.

  When preparing the values.yaml file for pSeven Enterprise v2022.10, you will have to generate the extension node SSH key and add it to values.yaml. For instructions, see Deployment configuration in the pSeven Enterprise upgrade notice below.

• In addition to the custom logos and Getting started page, you can set a custom favicon for pSeven Enterprise - see section Branding for details.

  There are a few changes in the branding file structure in pSeven Enterprise v2022.10. For differences, see Branding in the pSeven Enterprise upgrade notice below.

• In user workflow and app runs that complete without errors, pSeven Enterprise performs a log files cleanup, so successful runs take slightly less disk space than in previous versions.

• Fixed a compatibility issue with LDAP over SSL (LDAPS), which caused the sun.security.provider.certpath.SunCertPathBuilderException error during setup of user account federation with a user identity storage provider using LDAPS.

  Using LDAPS requires importing the public key of the LDAP server into pSeven Enterprise. The key must be prepared before starting the version upgrade to v2022.10. For instructions, see Using LDAPS in the pSeven Enterprise upgrade notice below.

• Fixed a compatibility issue with the PgBouncer database connection pooler, which led to malfunction if your PostgreSQL database provider uses PgBouncer.

• Fixed an issue where a user who runs a workflow that contains a block with a high CPU limit could leave pSeven Enterprise unusable for other users because their workflows could not start until that workflow containing a CPU-intensive block is stopped or a 10 minute timeout is exceeded.
• Fixed a regression issue with pSeven Enterprise branding from v2022.07, where your custom logo did not apply to the pSeven Enterprise sign-in page.
• Various updates and fixes in the pSeven Enterprise deployment and administration guides.

User block development

• Fixed an issue with user blocks and the REST API where you could not start a workflow run using the REST API commands if that workflow contains a user block with an empty description of some input port, and that port is a workflow parameter.

Bugfixes

• Fixed an issue with the Design space exploration block (beta) where it did not stop as intended when receiving an invalid response from a blackbox block - for example, a matrix with incorrect number of columns.
• Fixed an issue with the table and matrix editors in the Edit value dialog where the button was usable only to delete the bottom row.
• Fixed an AppsHub issue where you could not get the workflow from an app due to incorrect workflow name validation.
• Fixed a usability issue with the Explorer pane when viewing the Runs or Files special folders that contain files or subfolders with long names.
• Fixed an issue where you could not use slashes (/ or \) in the name of a composite block.
• Fixed a usability issue where you could not delete a link from a workflow using the Remove command from the menu in the workflow edit toolbar.
• Fixed a minor UI issue with the Explorer pane navigation bar, which displayed pSeven Enterprise reports as common folders.

pSeven Enterprise upgrade notice

pSeven Enterprise v2022.10 features a number of updates that may require additional preparation steps for the v2022.10 upgrade, depending on your current configuration. The summary of those steps is as follows:

• In values.yaml, set the new entrypoint.usingTrustedReverseProxy parameter.
• Generate a new SSH key for Windows extension nodes and add it to values.yaml as the new extensionNode.blockSSHGateClientKey parameter.
• If you are going to connect pSeven Enterprise with a LDAP server using LDAPS, place the server public key certificate into the pSeven Enterprise shared data storage so it is imported during the upgrade.
• If your deployment is branded, update the branding file structure in the shared data storage.
• If you are going to use NFSv4 (experimental), run deployment with the -⁠-⁠set fixStorages=true option.

Each of the additional steps is described in detail in paragraphs below. Note that they are mostly done while you prepare for the upgrade, prior to running the installation. The descriptions below supplement the general updating instructions found in section Version upgrade.

Deployment configuration

When preparing the deployment configuration file values.yaml for pSeven Enterprise v2022.10, note the new entrypoint.usingTrustedReverseProxy and extensionNode.blockSSHGateClientKey parameters:

1. Prepare the updated deployment configuration file values.yaml as described in Upgrading pSeven Enterprise.

2. If your deployment is protected by a trusted reverse proxy that sets the X⁠-⁠Forwarded⁠-⁠* headers for user HTTP requests, and users cannot establish HTTP connections directly to pSeven Enterprise, set the entrypoint.usingTrustedReverseProxy parameter in the prepared values.yaml file to true.

   With that setting, pSeven Enterprise will consider the X⁠-⁠Forwarded⁠-⁠* HTTP headers to be trustworthy, and will use the header information for identification of clients (users). Parsing the the X⁠-⁠Forwarded⁠-⁠* headers is disabled by default as a safe setting, and should be enabled only if your pSeven Enterprise deployment is accessible to users through a trusted reverse proxy, and not also accessible directly.

3. The prepared values.yaml file also contains a new extensionNode.blockSSHGateClientKey parameter. This key is required to establish connections with Windows extension nodes and must be specified in values.yaml.

   1. Get the full version number and generate a new file with pSeven Enterprise v2022.10 secrets using the following commands:

      helm show chart pseven-{YYYY.MM.DD}.tgz | grep appVersion | cut -d" " -f2
      docker run --rm -it {registry}/pseven-secretgen:{version} > pseven-secrets-2022-10-extnodessh.txt
      

   2. From the generated pseven-secrets-2022-10-extnodessh.txt, copy only the extensionNode.blockSSHGateClientKey parameter to the values.yaml file you prepare for the upgrade deployment. Do not change other secrets in values.yaml (the jobcluster.* and extensionNode.* parameters, except extensionNode.blockSSHGateClientKey) - those must be copied from the existing configuration, as described in Upgrading pSeven Enterprise.

   3. Also copy the extensionNode.blockSSHGateClientKey parameter to your backup copy of pSeven Enterprise secrets.
   4. Remove the generated pseven-secrets-2022-10-extnodessh.txt file.

Using LDAPS

pSeven Enterprise v2022.10 supports connecting to user identity storage providers using LDAP over SSL (LDAPS), which requires importing the public key of the LDAP server. The easiest way to import that key is to place the public key certificate of the LDAP server into the pSeven Enterprise shared data storage before you run the deployment:

1. Obtain the public key certificate from the LDAP server. The certificate must be exported to a DER or Base-64 encoded X.509 file. Request the certificate file from your LDAP server administrator.
2. Locate the pSeven Enterprise shared data storage (see the storage.shareddata.nfs.* parameters in values.yaml). Create a directory named certs in the shared data storage directory specified by the storage.shareddata.nfs.path parameter.
3. Rename the certificate file to ldaps.cert and copy it to the certs directory. The expected path to the ldaps.cert file is <shared data>/certs/ldaps.cert, where <shared data> is the path specified by the storage.shareddata.nfs.path parameter in values.yaml.

When you run the deployment, pSeven Enterprise will import the key from <shared data>/certs/ldaps.cert. Now when you set up the user identity storage provider (see section Provider setup in the administration guide), you can specify the ldaps:// prefix in the Connection URL field to use LDAPS.

Branding

There are a few changes in the branding file structure in pSeven Enterprise v2022.10. The Branding section describes the current structure; compared to previous versions, the differences are:

• The logo-100x266.png file, as well as the <shared data>/branding/auth/ folder, are no longer used.
• All logo customizations should now be placed into the <shared data>/branding/pseven/ folder.
• The logo for the sign-in screen is now named block-logo-200x530.png. For best results, this logo image should be 200×530 pixels.
• There is an additional logo used in navigation bars, named header-logo-64x64.png. This logo image should be 64×64 pixels.
• If you want to use a custom favicon, place your favicon.ico file into the <shared data>/branding/pseven/ folder alongside the custom logos.

Enabling NFSv4

pSeven Enterprise versions prior to v2022.10 are compatible only with NFSv3 file storage servers. The experimental NFSv4 support is added only in v2022.10, so all existing deployments of prior versions are configured to use NFSv3 only, and NFSv4 is disabled on the NFS server.

If you want to start using NFSv4, enable the NFSv4 protocol on your file storage server before deploying the v2022.10 upgrade, and run deployment with the -⁠-⁠set fixStorages=true option:

1. Before you run the deployment, revert the changes described in section Disabling NFSv4 of the pSeven Enterprise deployment guide. Use the test described there to verify that NFSv4 is now enabled:

   1. Create a test directory and try mounting it using NFSv4.

   2. Verify that a command like the following runs without errors:

      $ sudo mount -t nfs -o vers=4 localhost:<test directory path> <mountpoint>
      

   3. Additionally you can run the mount | grep <test directory name> command to check that the test directory is mounted using NFSv4: look for the mountvers option in the command output, it should read mountvers=4.

2. When running the v2022.10 deployment, enable the storage check by adding the -⁠-⁠set fixStorages=true option to the helm install command, as noted in the upgrade guide - see the command example in section Upgrading pSeven Enterprise. Note that the storage check may take some time depending on the amount of storage data.

3. Once the deployment finalization message appears, verify that the storage check has run. The message should contain the following note:

   Fixing storages enabled: deployment was run with the "--set fixStorages=true" option, fixing user permissions in data storages.

v2022.09

Updates

• pSeven Enterprise can email you when your workflow run completes or interrupts. Enable mail notifications in the pSeven Enterprise user menu.

  Notifications toggle in the user menu

  

• The Share dialog supports username autocompletion so you no longer have to input full names or email addresses of users with whom you want to share your files.

  Autocompletion suggestions in the Share dialog

  

  Autocompletion settings are managed by your administrators, who may decide to disable username suggestions for security reasons.

• You can specify the following block run settings in the Block properties pane:

  Block run settings in the Block properties pane

  

  • Memory limit (MB) - the amount of memory reserved for a block. By specifying a higher memory limit than default, you can request more memory for a block that has high memory consumption.
  • Idle timeout (sec.) - the timeout (in seconds) after which a ready block, which captures runtime resources and waits for input data, is stopped by pSeven Enterprise to free up the resources. In particular, you can set the timeout to NEVER to keep a block running until the workflow run stops - this is useful in workflows that run as a service, where certain blocks have to run continuously in order to improve the service response times.

  Block run settings are always editable, though they have effect only if the workflow runs on a pSeven Enterprise deployment where administrators have enabled the resource management features. This may be useful if you edit and configure your workflow on a test deployment with resource management disabled and then move the ready workflow to another, working deployment where resource management is enabled.

• The Workflow contents pane on a workflow tab provides an overview of memory limits you have set for workflow blocks.

  Block memory limits overview in the Workflow contents pane

  

• Updated and fixed the matrix editor in the Edit value dialog: that editor now provides all matrix editing functions that were available in previous versions (fill, transpose, and so on).

• When you create a run of a read-only shared workflow and select a folder where to create the run directory, the default location is the folder you currently have open in Explorer. In previous versions the default location was the root of your home folder, which is often inconvenient.
• Upgraded the embedded pSeven Core package to version 6.37. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

• Since v2022.09, pSeven Enterprise supports integration with Active Directory and LDAP services - see section Users from Active Directory or LDAP server in the administration guide for instructions.

  If you want to enable Active Directory or LDAP integration during an upgrade of an existing deployment, where users already have accounts created by means of the pSeven Enterprise admin interface, you will have to perform account matching as a part of that setup in order to prevent users from losing access to their data. For instructions, see the pSeven Enterprise upgrade notice below.

• You can set up single sign-on (SSO) for pSeven Enterprise users, integrating with an external OpenID provider, such as a Keycloak-based identity provider. See section Setting up single sign-on (SSO) in the administration guide for instructions.

• You can enable username and email autocompletion in pSeven Enterprise UI - for example, in the Share dialog, so the user who shares a folder to others will not have to input their full usernames or email addresses. Autocompletion is disabled by default; to enable, set the exposeUsersInfo parameter in the deployment configuration file values.yaml to true when upgrading your deployment to pSeven Enterprise v2022.09.

  Enabling autocompletion is a security risk

  A malicious user can exploit autocompletion to gather information about names and email addresses of other users. It is strongly recommended to disable this feature in public deployments, and enable it only if you are deploying pSeven Enterprise in a private environment that allows user contact sharing.

• Fixed an issue with blocks configured to run on a Windows extension node - such a block was unable to launch a software package installed on the node, if that package requires certain environment variables (a typical requirement of various engineering software). Note that in order to avoid this issue, an additional node configuration step is needed - you must add all required variables to the node's block runtime environment as described in section Engineering software on extension nodes of the administration guide.

• To avoid stability issues when many users actively edit blocks (open block configuration windows) and run workflows, the block edit sessions are now handled by the new appworkerblockjobs service while workflow run tasks are handled by the appworkerworkflowjobs service. Previously tasks of both kinds were handled by appworkerworkflowjobs, which caused issues for users - in particular, when a user cannot start any workflow run because other users have many block configuration windows open, thus overloading the appworkerworkflowjobs service so it cannot start new workflow run tasks.
• Improved run resource management for Windows extension nodes to avoid issues caused by a high number of blocks running at the same time on a given Windows node.
• Fixed a regression issue from v2022.07, which caused incompatibility with certain PostgreSQL providers that use PgBouncer as the database connection pooler: pSeven Enterprise deployment stopped with an error when initializing the Keycloak database (executing the initkeycloakdb Helm hook).
• Fixed an issue where a < or > character in the database password (the postgres.password parameter value in values.yaml) caused a deployment error.
• Fixed an issue with run resource management where, if there are insufficient free resources to start a workflow run, that run started and immediately failed with an "Insufficient resources to run the workflow" error, instead of being put into the run queue as intended.
• Updated the pSeven Enterprise administration guide:
  • Added instructions for using event logging to audit activities of pSeven Enterprise users and administrators - see section Auditing user and admin activity.
  • Added instructions on how to enable users to change their username - see section Changing the username.
• Updated the service interaction diagram in the pSeven Enterprise deployment guide: added the new appworkerblockjobs service and changed the layout to make the relationships between diagram elements more clear.

Bugfixes

• Fixed a regression issue from v2022.08 where workflow runs failed with "The following IDs are unknown" or "Locked document must be a directory" errors.
• Fixed an issue where a prolonged workflow run created too many temporary files and auxiliary logs, due to which it occupied much disk space and caused performance and responsiveness issues when you navigate that run's results in Explorer or view run logs.
• Fixed a regression issue from v2022.06 where a composite block with batch mode ports did not actually enable batch processing for arrays and lists sent to those ports, if the @Number of parallel workers setting is 1.
• Fixed a regression issue from v2022.08 where uploading a file using drag and drop caused incorrect behavior of the Explorer pane.
• Fixed a regression issue from v2022.08 where the AppsHub Runs sidebar was displayed when you open any HTML file from the Explorer pane.
• Fixed an issue with the pSeven Enterprise load balancing system that caused confusing behavior: if other users have many block configuration windows open at the same time, you could not start any workflow run with no apparent reason. As a part of that fix, pSeven Enterprise now limits the number of block configuration windows you can open at once, and keeping them open does not affect workflow runs.
• Fixed a regression issue from v2022.08 where pSeven Enterprise could not open a workflow run in a tab and displayed a "variable referenced before assignment" error.
• Fixed a REST API regression issue from v2022.06: with some workflows, a correct PATCH request to set workflow run parameters returned a response with status code 500 Internal Server Error.
• Fixed a number of issues with the Design space exploration block (beta):
  • Inconvenient data types used by request ports, which send values of variables to blackbox blocks for response evaluation.
  • Incorrect behavior when you configure a space-filling DoE technique and disable batch mode in run options: when sending generated DoE points to blackbox blocks evaluating responses, the Design space exploration block output all points as a single batch instead of the expected sequential output mode.
  • Incorrect behavior of the problem definition ports (for example, variable bounds ports): if you assign a value to such a port and connect a link to it, the block always used the assigned value while it should wait for and use a value received from the link.
  • An error when saving a block with a valid initial sample, which contains NaN or missing (N/A) response values.
  • Minor UI issue when you set the Maximum batch size option and the Gradient-based optimization technique is selected - that option does not have any effect in this case as the technique does not provide the support for real concurrency in response evaluations, but the block UI did not make it clear.
• Fixed an issue where a workflow run continued indefinitely if the workflow contains a composite block with a batch mode port, and a value is assigned to that port.
• Fixed an issue with the Python script block that sometimes caused a "block init was called more than once" error when starting a workflow run.
• Fixed UI responsiveness issues in the Block properties and Composite properties panes.
• Fixed a convenience issue with editing input values in block properties and Run setup panes, where a data editor window always popped up even if the value is a plain scalar number or a string.
• Fixed an incomprehensible text in the error message that appears when you undo changes in a workflow but pSeven Enterprise cannot apply the undo immediately.
• Fixed several minor UI issues in the Explorer, Block properties, and Composite properties panes.

pSeven Enterprise upgrade notice

Since version v2022.09, users can sign in to pSeven Enterprise with their accounts from an external storage such as Active Directory Domain Services or an LDAP server. To enable this feature after upgrading pSeven Enterprise, user account federation must be set up and run as described in Users from Active Directory or LDAP server. As a result, for each user account from the external storage that matches the federation settings, the associated account of the pSeven Enterprise user will be created. If any pSeven Enterprise users have previously created non-federated accounts, you must link their new federated accounts to existing authorization accounts as described below. Otherwise, upon the first sign-in with the new federated account, those users will get new authorization accounts that consume the pSeven Enterprise license and do not provide access to the existing user data associated with the old authorization accounts.

Account matching

Running federation results in creating new accounts in the pSeven Enterprise authentication service. Upon the first sign-in to pSeven Enterprise, an authorization account is created for the user account, which provides access to the user data such as their home directory, files, workflows, and apps. If the user already has an account created by means of the pSeven Enterprise admin interface (see Adding accounts), you must link the respective authorization account to the new user account, created by running federation. This ensures that having signed in with the new account, the user can access their data in pSeven Enterprise.

The authorization account of the user in question is initially linked to their old account, which was created by means of the pSeven Enterprise admin interface. Before allowing the user to sign in to pSeven Enterprise with their new account, you must complete the following steps:

1. Disable the old account of that user in the pSeven Enterprise authentication service.
2. Link the authorization account of that user to the new user account, which federation has created for that user.

Follow the instructions below to complete each of these steps.

To disable the old account in the pSeven Enterprise authentication service:

1. Sign in to the pSeven Enterprise admin interface and click Manage users in the page header.
2. Look up the old account: on the Users page of the authentication service interface that appears, enter the username or a part of the username in the Search box, and then press Enter.
3. In the lookup results list, click the ID of the old account to open its edit page.
4. On the Details tab of the old account edit page, turn the User Enabled switch OFF and click Save.

Copy and save the username provided in the Username field on the Details tab of the old account edit page. This username will be required to locate the authorization account.

The old account might prevent the creation of the new federated account. This happens, for example, if the account in the external storage has the same email or username as the old account in pSeven Enterprise. In this case, having copied and saved the username, you should delete the old account, and then run federation again.

To link the authorization account to the new user account:

1. On the Users page of the authentication service interface, look up the new user account and open its edit page.
2. Copy and save the contents of the ID field displayed on the Details tab of the edit page - this is the unique identifier of the new user account.
3. Click the icon in the upper right corner of the authentication service interface page, and then select the Manage user permissions menu item to open the list of authorization accounts.
4. In the list that appears, click the username of the old user account.
5. In the Keycloak user ID field on the Change user page that appears, enter the identifier of the new user account and save the changes.

The user can now sign in to pSeven Enterprise with the new account. The account matching you performed ensures that the user keeps access to their data.

v2022.08

Updates

• You can view and manage app runs in AppsHub using the new Runs sidebar available in all apps, including those published with a custom app UI.

  • Click at the upper left of the app page to open the Runs sidebar.
  • Click a run in the sidebar to view its results.
  • To delete runs you no longer need, select them in the sidebar and click the button that appears.
  • By default, you can view and delete only your runs. The app owner can view and delete all runs.
  The app Runs sidebar

  

• Updated the Design space exploration block (beta), resolving a number of usability and compatibility issues. Since this version, future pSeven Enterprise updates will keep compatibility with the current Design space exploration block version.

  An optimization problem set up in a Design space exploration block

  

• You can use the table editor in the Edit value dialog to create any custom table, as it now supports adding, removing, renaming and reordering table columns.

  Column settings in the table editor

  

• Improved Edit value dialog usability for simple values (numbers and strings).

Deployment and administration

• Windows extension node setup no longer requires Internet access.

• In a deployment with the "on-demand" block execution strategy enabled, a deployment administrator can control the block shutdown timeout with the new Inactive block timeout setting.

  The "on-demand" block execution strategy is an experimental feature.

• Updated the pSeven Enterprise administration guide:

  • Added instructions on how to install additional Python modules into pSeven Enterprise deployment and on Windows extension nodes - see Installing additional Python modules.
  • Added instructions on how to configure user session expiration - see User session expiration settings.
  • Updated section Configuring user authorization on extension nodes with notes on username changes.
• Fixed a user management issue where pSeven Enterprise deployment could stop with an error if some users have changed their usernames since the previous version upgrade or restart.
• Fixed a stability issue in workflow run management where a run that failed to start was blocking subsequent workflow runs for all users, and all those runs got stuck in the "Configuration" state until you restart pSeven Enterprise.
• Fixed a stability issue with the user interface service (appserver) where it could run out of memory if there are users who monitor run logs during execution, which caused a service restart, creating UI issues for other Studio users.

User block development

• You can implement a user block that runs an existing pSeven Enterprise workflow as the current user running a workflow with your block. Such user blocks work with the pSeven Enterprise REST API described in the REST API guide. pSeven Enterprise provides access to the REST API through the following block environment variables:
  • DA__P7__BLOCK__REST_API_URL - contains the REST API URL.
  • DA__P7__BLOCK__REST_API_TOKEN - contains the REST API authorization token of the current user.
• Fixed an issue with development of Windows-only blocks, where a user who opens such a block was receiving a "Can not submit job to the cluster!" error.

Bugfixes

• Fixed an issue where you could not sign in to pSeven Enterprise after waiting at the sign-in page for a certain time.
• Fixed an issue where file upload, copy, and other file operations in Explorer caused a "connection already closed" error.
• Fixed an issue with logging in long-running workflows, which caused slowdowns and unresponsiveness when working with such workflow runs:
  • Disabled extended block logging by default to reduce the amount of logs.
  • Logs displayed in the Run log pane are truncated for faster loading.
  • To get a full log, you can download the log file with the Download command from the Run log pane menu.
• Fixed an issue with Windows-only blocks where a "Can not submit job to the cluster!" error appeared when you open the block configuration dialog.
• Fixed an issue where sending a Matrix value to a batch mode port of some composite block caused an error, which stopped the workflow run.
• Fixed an issue with delays in workflow loop execution, that became noticeable after a certain high number of loop iterations.
• Fixed an issue with workflow loops where, if the loop completed but did not send some of the required responses to the loop driver block, that block waited indefinitely instead of raising an error and interrupting the workflow.
• Fixed an AppsHub issue with the stock app UI that caused incorrect app behavior if your network connection with AppsHub got interrupted while the app runs.
• Fixed an issue where a Python script block configured to output a NumPy array of objects (for example, an array of lists) caused an error during a workflow run.
• Fixed a UI issue with the Workflow library pane where, if you add a block to your workflow and close its tab or hide the pane before it finishes loading the block, the added block was hidden from the pane until you add one more block.
• Fixed an issue where an error occurred when you copy and paste a block into the same workflow, if the name of that block is a number.
• Fixed a UI issue with the Block properties pane where it could display incorrect variable names in port tooltips for Condition blocks.
• Fixed an issue with the Explorer pane where pSeven Enterprise did now display an appropriate error message if you input a non-existing path in the Explorer address bar.

v2022.07

Updates

• You can manage your account settings - change email, password, and other - in your user profile. To open it, use the new Edit profile command from the pSeven Enterprise user menu.

  The Edit profile command in user menu

  

  This feature is enabled by the new user authentication system introduced in pSeven Enterprise v2022.07. Due to changes in authentication, you will be required to verify your account once after the upgrade to v2022.07. Until you verify your account, sign-in attempts to pSeven Enterprise v2022.07 will result in an "Invalid username or password" message. To verify, click the "Forgot Password?" link in the pSeven Enterprise v2022.07 sign-in form. Upon clicking that link, you will receive an email with further instructions.

• To make it easier to find run tabs related to a particular workflow, pSeven Enterprise opens new run tabs next to the related workflow tab, instead of opening them after the last existing tab.

  Run tabs open next to their related workflow tab

  

• You can quickly show or hide all block presets, expand and collapse all block groups in the block library using the new and buttons in the Workflow library and Common library panes.

• Upgraded the embedded pSeven Core package to version 6.36. For details on recent improvements and changes, see the pSeven Core changelog.
• Updated the REST API guide in the pSeven Enterprise Help to describe REST API v2.

Deployment and administration

Important

When upgrading to pSeven Enterprise v2022.07, follow the instructions found in the pSeven Enterprise upgrade notice below. Otherwise, users may be unable to sign in after the upgrade due to deployment configuration errors, and fixing those errors will require you to re-deploy v2022.07.

• pSeven Enterprise v2022.07 works with a new authentication service based on Keycloak, which improves user account security and provides additional account management options.
• Added the number of remaining Studio and AppsHub user licenses to the Licenses page in the pSeven Enterprise admin interface.
• Fixed a regression issue from v2022.06 where pSeven Enterprise deployment stopped with an error, if you set the entrypoint.allowedHosts parameter in the deployment configuration file values.yaml.
• Fixed a stability issue with the user interface service (appserver) where it could become unresponsive if many users actively switch tabs in Studio, causing UI slowdowns and errors for all Studio users.
• Updated and extended the pSeven Enterprise deployment and Administration guides following the introduction of a new authentication service based on Keycloak:
  • Updated the initial admin account setup instructions in section Admin access.
  • Updated the service interaction diagram.
  • Updated the instructions for adding new users in section Adding accounts.
  • Added the instructions for administering existing user accounts, see Managing user account data.
• Added section Backing up and restoring user data to the administration guide.

User block development

• Fixed an issue where loading a block prototype to the workflow library by dragging its folder from the Explorer pane onto the Workflow library pane caused a "Device or resource busy" or "Directory not empty" error.
• Fixed an issue where a value in scientific notation (for example, 1e-08) in block manifest caused an error when you add that block to the workflow library.

Bugfixes

• Fixed a regression issue from v2022.06 where a workflow run stopped with an error, if the workflow contains a block that outputs a large data array.

  If you encounter that issue, update blocks in the workflow library by clicking the update notification next to the block name in the Workflow library section of the Block library pane. See Updating blocks to a newer version for more details.

• Fixed a regression issue from v2022.06 where you could not run an app in AppsHub due to a workflow run error, if app parameters have no default values.

pSeven Enterprise upgrade notice

Due to the introduction of a new authentication service based on Keycloak, an upgrade to pSeven Enterprise v2022.07 requires a few additional configuration steps, which are summarized below.

Before you upgrade to pSeven Enterprise v2022.07:

1. It is recommended to create a backup of the pSeven Enterprise database. You may need that backup if you decide to rollback to a previous pSeven Enterprise version: the v2022.07 upgrade adds new database objects, which may cause issues if the database once upgraded to v2022.07 is used with an older version. The pSeven Enterprise database is specified by the postgres.dbname parameter in your deployment configuration file values.yaml. For database backup instructions, see Backup and Restore in the PostgreSQL server documentation.

Additional upgrade steps required for pSeven Enterprise v2022.07:

1. Prepare the updated deployment configuration file values.yaml as described in Upgrading pSeven Enterprise.
2. Check the entrypoint.allowedHosts and entrypoint.allowedAuthPorts parameter settings in the prepared values.yaml file (see the comments to those parameters for details). Note that entrypoint.allowedAuthPorts is a new parameter, which must be set correctly, considering the domain name and port numbers used to access your pSeven Enterprise deployment. Otherwise, you users will be unable to sign in to pSeven Enterprise after the upgrade.

3. The prepared values.yaml file also contains a new keycloak.clientSecret parameter. This secret is required to establish connection to the new authentication service (keycloak) and must be specified in values.yaml, otherwise user authentication will be disabled, and users will be unable to sign in.

   1. Get the full version number and generate a new file with pSeven Enterprise v2022.07 secrets using the following commands:

      helm show chart pseven-{YYYY.MM.DD}.tgz | grep appVersion | cut -d" " -f2
      docker run --rm -it {registry}/pseven-secretgen:{version} > pseven-secrets-2022-07-keycloak.txt
      

   2. From the generated pseven-secrets-2022-07-keycloak.txt, copy only the keycloak.clientSecret parameter to the values.yaml file you prepare for the upgrade deployment. Do not change other secrets in values.yaml (the jobcluster.* and extensionNode.* parameters) - those must be copied from the existing configuration, as described in Upgrading pSeven Enterprise.

   3. Also copy the keycloak.clientSecret parameter to your backup copy of pSeven Enterprise secrets.
   4. Remove the generated pseven-secrets-2022-07-keycloak.txt file.

4. Verify your entrypoint.allowedHosts, entrypoint.allowedAuthPorts, and keycloak.clientSecret parameter settings in the updated values.yaml file you prepare. Then continue with the upgrade deployment, following the Version upgrade guide.

After the upgrade to pSeven Enterprise v2022.07:

1. All admin accounts will have their password reset to admin. Once the upgrade deployment completes, sign in as administrator immediately and change the default password for all admin accounts. To sign in, specify an existing admin username (email address) and the admin password. For password reset instructions, see Managing user account data.

2. The default administrator account is assigned the username admin instead of admin@admin.admin.

   If user authorization is configured for that account on an extension node (see Configuring user authorization on extension nodes), you will have to change the regular expression in the DA__P7__ALLOWED_USERS environment variable to match the new username admin. Otherwise the user admin will lose existing permission to run blocks on that specific extension node.

3. All user accounts will have their password unset, and users will have to set a password. Notify your users that they need to pass a one-time account verification by resetting their passwords using the "Forgot Password?" link at the sign-in page. Users who have not set a password will be unable to sign in to pSeven Enterprise v2022.07, and their sign-in attempts will result in an "Invalid username or password" message. Otherwise, you can also set user passwords in the pSeven Enterprise admin interface (see Managing user account data).

Upon the v2022.07 upgrade, existing user accounts keep their usernames, so existing users should sign-in with an email address. The exception is the default administrator account, which is assigned the username admin after the upgrade. New users added after the upgrade are no longer required to use an email address as a username.

If there are no active admin accounts in your deployment, the v2022.07 upgrade will also re-enable the default admin account (username: admin, password: admin).

v2022.06

Updates

• In the Share dialog, you can press Enter to add another user instead of clicking every time.
• Enabled renaming block working directories in run result folders of completed workflow runs.
• Added the support for N-dimensional NumPy array outputs in the Python script block:
  • 2-dimensional arrays are output as matrices by default, as earlier
  • Other arrays are output as nested lists by default
  • A flat (1-dimensional) array is also compatible with the matrix data type and can be converted to a single-column matrix
• The pSeven Enterprise REST API protocol is updated to v2 with minor changes in the run parameters and run results structures (see sections Run parameters and Run results in the REST API guide for details). The new protocol version is used only if you access the REST API methods by URL /pseven/.rest/v2/. pSeven Enterprise keeps compatibility with the REST API protocol v1 (URL /pseven/.rest/v1/), so existing REST clients, which use REST API v1, will continue to work with no issues. However it is advisable to update existing clients to REST API v2, as well as use only REST API v2 in new client implementations.
• You can write data tables for reports from a Python script block using the api.write_report_table() function from the block's embedded api module. Function parameters are write_report_table(path, data), where path is the table file full path with an optional .p7table extension, and data is the table data as pandas.DataFrame.
• You can view shell scripts (.cmd, .bat, .sh) in pSeven Enterprise, in addition to previously supported file formats. To view a file, double-click it in Explorer, or select it and press Space.
• Added a few commonly used shortcuts for keyboard navigation in Explorer.
• Removed the Show active document folder command from the menu in Explorer as it is more conveniently available from the tab bar: while viewing a workflow, run, or report click in the tab bar or double-click the tab title to open the related folder in Explorer.
• Upgraded the embedded pSeven Core package to version 6.34. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

• Increased the number of the block configuration UI router service (blockrouter) replicas to 2 to avoid interrupting user work in case of possible blockrouter failure.
• Fixed a deployment issue where possible errors in configuration of the database server or NFS storage server were reported only at the final deployment stage, leading to additional downtime during pSeven Enterprise version upgrades or redeployment. pSeven Enterprise now tests connection to the database and file storages at the start of the deployment.
• Fixed an issue where Windows extension nodes could not connect to pSeven Enterprise if it is deployed on a Kubernetes cluster that uses the Calico CNI plugin.
• Revised and updated admin guides in the pSeven Enterprise Help:
  • Added a separate Version upgrade section to help locate the appropriate instructions, replacing the upgrade sections previously included in the pSeven Enterprise administration and extension node deployment guides.
  • Expanded the administration guide with section Understanding accounts to provide a better understanding of account setup in pSeven Enterprise.
  • Moved the section describing user access settings on extension nodes from the extension node deployment guide to Administration - see Configuring user authorization on extension nodes.
  • Moved all uninstallation instructions to a separate Uninstallation guide.
  • Several other minor updates and additional explanations.

User block development

• Redesigned the Edit value and View value dialogs to better support custom tables with predefined columns and other custom data types.
• Fixed a regression issue from v2022.03 where pSeven Enterprise did not properly remove files from the block data directory (DA__P7__BLOCK_DATA_DIR) after they had been deleted by the block.
• Updated section Block assets in the user blocks guide with more information about publishing and updating assets, and also using assets to add Python modules to blocks.

Bugfixes

• Fixed a regression issue from v2022.05 where you could not download large files from pSeven Enterprise.
• Fixed a regression issue from v2022.04 where moving a block to another workflow using cut and paste would not remove the block from the source workflow, causing errors when you edit the source workflow.
• Fixed an issue where passing complex List or Dict values between blocks caused delays in workflow execution.
• Fixed an issue with the Python script block where a 1-dimensional NumPy array sent to a request driver port was converted to a single-column matrix by default. 1-dimensional arrays are compatible with the matrix data type, but such an array should be converted to a single-column matrix only if you pass it through a port that has its type set to matrix.
• Fixed an issue with editing workflow notes where they could reset position or disappear if you are actively editing notes.
• Fixed an issue with renaming subfolders in the Files special folder in workflows where you could add the reserved .p7wd extension to any folder, which caused various filesystem error messages.
• Fixed a UI issue with the workflow editor where, if you remove some port that has a link connected to it, that link was automatically removed as intended but was still displayed in the workflow until you refresh a browser tab, which caused confusion.
• Fixed several issues with the Run setup pane where it incorrectly displayed the current parameter preset.
• Fixed incorrect behavior of the info pane in Explorer when selecting a shared folder (workflow, report) that is no longer available - for example, was moved or deleted by its owner.
• Fixed an issue with the View as text command in Explorer where viewing a binary file caused UI unresponsiveness.
• Design space exploration block (beta):
  • Fixed slowdowns when adding or removing the block from a workflow.
  • Fixed incorrect order of ports in the Block properties pane.
  • Fixed several issues that caused block execution to stop with an error.

Compatibility issues

• Due to changes in data type conversion rules that were required to add the support for N-dimensional NumPy arrays in the Python script block, in certain cases the arrays that were previously converted to a single-column matrix may now instead be converted to flat lists if you pass them through untyped ports (the "Any" port type). Typically this causes a "'float' object is not subscriptable" error in the block that receives such an array. In affected workflows, that error appears only after you update the Python script block in the workflow library to the most recent version (block version 20). To revert to the old behavior, you can change port type in affected blocks to the matrix type - to force conversion of a 1-dimensional array to a single-column matrix.
• The typesystem.File() constructor previously used in the Disk optimization example to create data tables for reports is no longer supported in Python script blocks, as it was replaced with the more convenient api.write_report_table() function from the block's embedded api module.

v2022.06.07

v2022.06.07 is a hotfix release for several issues related to the "Failed to send interrupt request to block" error appearing in v2022.05.

Bugfixes

• Fixed a regression issue from v2022.05 where numerous "Failed to send interrupt request to block" error messages appeared in the Run log pane when you stop the workflow.
• Fixed a regression issue from v2022.05 where the stop button on a running block did not work.
• Python script block: fixed a regression issue from v2022.05 where api.interrupted() returned False even if the workflow run is stopping when the block calls this function, instead of returning True in that case, as intended.

User block development

• Fixed a regression issue from v2022.05 where, when a user stops a workflow run, pSeven Enterprise did not send an interrupt request (the interrupt control endpoint) to running blocks, so a block could not handle the run stop properly.

v2022.05

Updates

• To update the stock app UI in a workflow created in previous pSeven Enterprise versions or to revert UI changes in a workflow, you can use the new app UI reset command from the menu in Explorer. To update an app published with the stock UI, reset the app UI in the workflow you published as that app, then re-publish the workflow.

  The app UI reset command in the Explorer menu

  

• Timestamps in the Run log pane use the time format from your system (browser).

• Added a couple of convenience features for the tree view in the Explorer pane:
  • If you navigate between folders that are displayed in tree view (for example, run results), the pane now remembers the tree state in a few folders you have recently visited.
  • If you input a path pointing to a file or folder in a tree, the pane expands the corresponding tree node upon navigating to that path.

• The Explorer pane uses icon and background colors to indicate status of workflow runs. The color scheme is consistent with the one used on run tabs.

  Colored run icons in Explorer indicating run status

  

• Added an early preview version of the Design space exploration block, which provides methods for design of experiments and optimization.

  The Design space exploration block in the library

  

  This early preview version of the Design space exploration block may appear to be incompatible with future pSeven Enterprise updates and should be used for testing only.

• Upgraded the embedded pSeven Core package to version 6.33. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

• To prevent situation when some user opens too many blocks or starts too many workflow runs at the same time, which interferes with other users' work, you can now set limits for the number of open blocks and the number of workflow (or app) runs - common ones on the Settings page, which apply to every user by default, and individual limits for specific users in their user settings.
• Added information about storage space used and storage quota to the Users page in the pSeven Enterprise admin interface.
• Fixed an issue with pSeven Enterprise version upgrade where the deployment of a new version could take several hours, depending on the amount of stored user data, because it always ran a filesystem check procedure, which is often not required. That procedure is now optional, does not run by default, and you can enable it by adding -⁠-⁠set fixStorages=true to the helm install deployment command.
• Fixed an issue where pSeven Enterprise could stop responding to user actions and sign-ins, which was caused by appserver service errors and required you to manually restart appserver. Now if any of the appserver replicas stops responding for longer than a minute, that replica is restarted automatically, which may interfere with work of some users but avoids a complete functional failure.
• Fixed an issue where user workflow runs were blocked by the license limit if a number of runs had failed earlier - sometimes the run failure status was not set correctly, so failed runs could count as still active.
• Updates for pSeven Enterprise deployments with run-time resource management enabled (optional feature, disabled by default):
  • Added a block execution scheduler to avoid overallocation of resources when running user workflows.
  • Fixed an issue where pSeven Enterprise could cease to run user workflows until you restart the task execution service.

Bugfixes

• Fixed an issue where you could not sign in to pSeven Enterprise or it stopped responding after certain errors - for example, when someone opens a corrupt workflow or report - which could be resolved only manually by a deployment administrator.
• Fixed a security issue in the REST API where a malicious user could set permissions on a folder owned by other user.
• Fixed a UI issue where the Block properties and Composite properties panes incorrectly displayed unconnected optional input ports (gray dot) as required (red dot).
• Fixed a convenience issue with the list of recent runs for shared workflows in the Start tab where the list included runs of all users who have access to that workflow instead of including only your runs.
• Fixed an AppsHub issue with the stock app UI where an attempt to run an app that was deleted after you opened the app page produced an incomprehensible error message.
• Fixed a UI issue where the Run setup pane did not display long values of parameters in full even if you extend pane width.
• Fixed a UI issue where refreshing the Explorer pane in the tree view collapsed all tree nodes.
• Fixed a UI issue with editing workflow notes where the browser context menu did not appear when you right-click a note.
• Fixed a UI issue where special folders like Files or Runs were not always sorted to the top in the Explorer pane as intended.

v2022.04

Updates

• Updated the stock app UI for AppsHub: if you open an app page directly by its URL, the app recognizes query string parameters as its arguments and automatically sets parameter values in the UI. This is useful, for example, if you integrate pSeven Enterprise apps to some other environment where users select parameter values before they launch an app: now you can pass user values to app in the query string, so users do not have to repeat parameter input in the app. See section App URL parameters for details. Note that previously published apps do not get this update automatically. To update an existing app, update the stock app UI files in the UI folder of that app's source workflow, then re-publish the workflow.

• If a workflow run has stopped with an error, you can see error details in the run status tooltip on the run toolbar.

  Run status tooltip with error details

  

• Several improvements in the Block properties and Composite properties panes:

  • If you are viewing contents of a composite block, its Composite properties pane displays port connection status regarding uplinks inside that composite block. Previously the pane was always showing status of external connections to the composite block, which was inconvenient when checking uplinks from nested blocks to ports of their parent composite block.
  • The sorting buttons work independently - each button switches the sort order in a specific list, not everywhere in the pane. For example, you can use different sort order in the input and output port lists.

  • Better alphabetical sorting: it is now case insensitive, and numbered ports are sorted in natural order - for example, port 9 is sorted before port 10.

    Port sorting

    

  • Sorting now applies to the order of port groups. Previously, sorting applied only to ports inside each group but never changed the group order, and port groups were always displayed before non-grouped ports in the list.

  • When you edit a port name in the Composite properties pane, leading and trailing space characters are trimmed to avoid accidentally creating ports with names that differ in the number of surrounding spaces only and look confusingly the same in the pane.
  • Port name validation in composite blocks is now case insensitive. In the Composite properties pane, it is no longer possible to create ports with names that differ by case only, like value and Value.
  • Improved pane UI responsiveness when working with large lists of ports.

• Upgraded the embedded pSeven Core package to version 6.32. For details on recent improvements and changes, see the pSeven Core changelog.

• Updated pSeven Enterprise Help:
  • Updated the Python script block guide with a new section explaining how to get a complete list of Python modules available in the block - see Listing modules.
  • Added a few notes on troubleshooting module import errors to section Python modules in the Python script block guide.
  • Updated section Publishing the workflow as an app with information about using query string parameters in the app URL to pass parameter values to an app (see App URL parameters).

Deployment and administration

• pSeven Enterprise v2022.04 contains updates in the license mechanism, which add flexibility to license settings, enabling certain optimization of license usage (see below). Due to these updates, previously issued licenses are not compatible with v2022.04. You will need to request an updated license key and have it ready before you begin the update - so you can update the license immediately after deploying v2022.04. Otherwise, your users will be unable to sign in to pSeven Enterprise until you update the license key in your deployment. For further details, see the pSeven Enterprise upgrade notice below.
• You can configure an admin account so that it does not count as a licensed user. See sections User permissions and Admin account configuration in the pSeven Enterprise administration guide.
• pSeven Enterprise license since v2022.04 counts the number of AppsHub users and Studio users separately. AppsHub users can sign in to AppsHub and run apps from there. Studio users can sign in to both Studio and AppsHub, can edit and run workflows, publish them to AppsHub and run apps. AppsHub and Studio access require different user permissions, so you can configure user accounts to optimize license usage - see sections User permissions and End user account configuration in the pSeven Enterprise administration guide.
• If a workflow or an app run has failed, you can now see the fail reason on the run page in the admin interface.
• Removed the "Job info" column from the Workflow runs page in the admin interface as it was taking too much screen space. Job info is available on each run page.
• Fixed an issue where numerous pint.util warnings appeared in logs of every Kubernetes pod.
• Fixed an internal configuration issue, which could cause zombie processes appearing in pSeven Enterprise service containers, potentially leading to various failures that would be hard to investigate.
• Updated the pSeven Enterprise deployment guide with notes about additional CPU requirements for the Kubernetes cluster worker nodes (moved: see Requirements).
• Slightly cleaned up and improved the layout of the deployment diagram in the pSeven Enterprise deployment guide.
• Updated section User management in the pSeven Enterprise administration guide.

User block development

• Fixed an issue where names of assets in the block manifest (see Block assets) were not checked properly, which allowed using assets with names that can lead to a name clash in block environment variables that store asset paths.

Bugfixes

• Fixed several internal issues with task management services that led to instability being one cause of incorrect behavior when running workflows - such as stopping with an error because a block failed to start, or inability to start another run after some run has stopped with an error.
• Fixed a bug in workflow execution, which contributed to the issue where a workflow run just started by a user could get stuck in the "Submitting" state. The issue is not fixed completely yet, although that situation is more rare now.
• Fixed an issue where a workflow run could get stuck in the "Queued to run" state if the user who started that run is near the storage quota limit.
• Fixed the primary cause of the issue where the message "Failed to send updates to platform: [Errno 32] Broken pipe!" appeared repeatedly in the run log. That warning may still appear rarely, although it does not indicate a run error.
• Fixed an issue where saving a block in its configuration dialog reset or unset its input values specified in the Block properties pane.
• Fixed an issue with the Run setup pane where you could unintentionally unset a parameter, because for parameters of certain types (matrix, list, and some others) clicking outside the input field or pressing Enter while editing the parameter reset its value instead of accepting edits as intended.
• Fixed an issue where a driver block in a running workflow started a loop that cannot complete because some of the loop body blocks do not receive all their input values, after which the run waited indefinitely for that loop. Now driver blocks check their loops before start and cease to run if required input data cannot enter the loop - this is possible, for example, if a block in the loop body receives some input from a Condition block. Also, if some block in the loop body cannot run (for example, one of its required input ports is not connected), a warning is issued to the run log pointing to the faulty block.
• Fixed an issue with the Block properties and Composite properties panes where adding many ports to workflow parameters could cause incorrect UI behavior or unresponsiveness.
• Fixed an issue with the Composite properties pane where you could create a port with a name that contains only space characters.
• Fixed an AppsHub issue with the stock app UI where you could not set a value to a matrix type parameter if that parameter has no default value.
• Fixed an issue where you could not open, move, or delete a folder copied from a run results folder.
• Fixed a UI issue with the tab bar, which sometimes did not display the left and right shift buttons and the scrollbar when you have many tabs open.
• Fixed a minor UI issue with workflow navigation in run tabs where you could not go inside a composite block with a double-click (like in workflow tabs) and had to always click the icon on the block to view its contents.

pSeven Enterprise upgrade notice

Since v2022.04, pSeven Enterprise license counts the number of AppsHub users and Studio users separately - the limits for them are set independently by the license. The type of a user depends on access permissions you set for that user. This also allows creating admin-only accounts, which do not have user access permissions - that is, cannot sign in to pSeven Enterprise as users. Admin-only accounts are not counted by the license, so you can create a number of such accounts as needed.

Licenses issued for previous pSeven Enterprise versions count only the total number of users, so they are not compatible with v2022.04. You will need to request an updated license key and have it ready before you begin the update - so you can update the license immediately after deploying v2022.04.

Important

Get an updated license key for pSeven Enterprise v2022.04 before you begin the upgrade. You will have to remove the existing key and add the new key immediately after upgrading to pSeven Enterprise v2022.04. Otherwise, user sign-in to pSeven Enterprise after the upgrade will be blocked until you replace the license key in your deployment.

The advisable upgrade sequence for pSeven Enterprise v2022.04 is:

1. pSeven Enterprise licenses are bound to a specific Installation ID. Get and save your Installation ID as described in section License of the pSeven Enterprise deployment guide.
2. Request a license update for your Installation ID. Postpone the upgrade to v2022.04 until you receive an updated license key.
3. Having received the new license key, upgrade to v2022.04 as described in the Version upgrade guide.
4. Immediately after the v2022.04 deployment finishes, sign in as admin and open the Licenses page of the administrative interface (URL: <admin URL>/app_auth/license/).
5. Remove the existing license. Licenses from previous versions are not compatible with pSeven Enterprise v2022.04 and may cause issues later.
6. Add the updated license key at the Add license page (URL: <admin URL>/app_auth/license/add/).

After replacing the license key, verify that you can sign in as a user. If you cannot sign in or encounter other issues during the upgrade, contact the DATADVANCE support team for advice (see Technical support).

After you complete the v2022.04 deployment and license update, consider also updating user permissions and configuring user and admin accounts as described in section User management of the pSeven Enterprise administration guide.

v2022.03

Updates

• More tab bar enhancements:
  • When adding a large number of tabs, their widths are proportionally reduced in order to fit more tabs into the bar, until tab width reaches a certain minimum.
  • When tabs fill the pane, left and right shift buttons appear to enable mouse-click scrolling through tabs.
  • The tab bar menu now contains a list of all open tabs, so you can simply select a tab name from the list to navigate to the desired tab.
  • In addition to scrolling tabs with your mouse wheel, you can also use swipe gestures - for example, on Mac.
  • The new tab button is no longer stuck at the far right.
• Upgraded the embedded pSeven Core package to version 6.31.1. For details on recent improvements and changes, see the pSeven Core changelog.
• Tooltips on links that may cause errors or unexpected behavior when running the workflow now display all warning and error details at once, instead of displaying only one of those messages.

Deployment and administration

• Updated guidelines. The pSeven Enterprise deployment guide was becoming overloaded, so it was split into several guides:
  • The pSeven Enterprise deployment guide now describes only important deployment steps. This guide was also updated to address a number of issues that may arise when installing pSeven Enterprise.
  • The Administration section contains an initial version of the support and maintenance guide.
  • Technical support contains support contacts and troubleshooting instructions.
  • Branding explains branding-related customizations.
• Updated the sizing.* parameter descriptions in the pSeven Enterprise deployment configuration file values.yaml to better explain their role.

User block development

• Your block can now read data from the shared data storage - see section Block assets for details.
• Fixed an issue where pSeven Enterprise sometimes deleted folders created by the block in its data storage (DA__P7__BLOCK__DATA_DIR).
• Fixed an issue where port values in the Results pane were always highlighted red (invalid) if the port data type is a custom table with predefined columns.

Bugfixes

• Resolved the most prominent of stability issues that caused the "Task scheduler service restart" and "No such file or directory" errors when running a workflow. The fixes applied do not remove those errors completely yet, though significantly reduce the probability of them appearing during a workflow run.
• Fixed an issue that caused the "Block process failed to start" error when running a workflow.
• Fixed an issue where users who exceeded their storage quota could not stop a workflow run in progress, and that run continued with many errors in the run log.
• Fixed an issue that caused damage to the workflow when in a situation where the user's storage quota is exceeded.
• Fixed an issue where the block configuration dialog failed to open, displaying an empty window along with a "Block edit session failed" error message.
• Fixed an issue with the Functions pane in the Python script block configuration dialog where pressing Enter while the cursor is in the Function name field removed one of the function arguments.
• Fixed a regression issue from v2022.02 where moving a block to another workflow using cut and paste would not remove the block from the source workflow, until you refresh the browser tab with pSeven Enterprise or reopen that workflow.
• Fixed UI issues with long names in the Links dialog. Long names now have an ellipsis to fit in the dialog width, and full names can be viewed by hovering over the name in the list.
• Fixed an issue where a workflow run tab could remain in the loading or waiting state until you refresh a web browser tab with pSeven Enterprise.
• Fixed an issue that resulted in an error message when double-clicking a workflow run tab while the run is initializing.
• Fixed an issue with the Block library pane where a block update notification in Workflow library did not appear until you select that block in the pane.
• Fixed an issue where the workflow run log sometimes did not update for about half a minute due to a log file initialization error.
• Fixed an issue due to which the command for automatic layout of blocks in the workflow could place some blocks very far from the rest.
• Fixed an issue where the search function in the Russian language version of Help ignored English words.

v2022.02

Updates

• Updated Links and Autolink dialogs:

  • The Links dialog now supports port groups. You can collapse and expand the list of ports in each group.
  • When linking a driver block, you can select which ports to display: driver only, input (output) only, or all.
  • The Autolink dialog now suggests links in one direction only, from source to target block. Its old version, which suggested links in both directions, proved to be inconvenient for linking blocks with many similarly named ports.
  Port grouping in the Links dialog

  

• Upgraded the embedded pSeven Core package to version 6.30. For details on recent improvements and changes, see the pSeven Core changelog.

Bugfixes

• Fixed an error when running a workflow with a Composite block that has a batch input port added to workflow results.
• Fixed an issue where the recent list on the Start tab was emptied if you launch many workflow runs in succession.
• Fixed an issue where editing a workflow after applying automatic block layout could unpredictably change the layout.
• Fixed an issue where UI was becoming unresponsive if you move blocks in the workflow using keyboard.
• Fixed an incorrect "You are trying to access an item that no longer exists" warning appearing when you copy blocks or notes in the workflow.
• Fixed an issue with renaming workflow runs and items in the workflow Files folder where it was impossible to select text with a mouse during rename.
• Fixed an issue where sometimes you could not rename a folder immediately after copying it.
• Fixed a UI error that could be seen in the browser console when you open a block or edit port values in the Block properties pane.

v2022.01

Updates

• Redesigned the tab bar for more convenience:

  • No more multi-row tab stack clutter. Tab titles always have the same width and are always displayed in a single line, which you can scroll with your mouse wheel, when there are too many open tabs to show all their titles at once.
  • Run tab titles use colored icons, which indicate run status.
  • Added tab bar menu with a few useful commands.
  The new tab bar

  

• The main toolbar in a run tab displays current run status.

  Run status in the run toolbar

  

• The Explorer pane displays your total and available storage space in its title bar.

  Storage information in Explorer

  

• When you select one or a few blocks to view their log messages, the Run log pane title displays names of selected blocks, so you can always tell which logs you are viewing at the moment.

• You can rename workflow runs in Explorer - for example, to tag important or debug runs.

  Note that you cannot rename a run in progress. To create a new run with a custom name, use the New run command, which prepares a run but does not immediately start it. Then while your new run is in configuration, rename the run folder in Explorer.

• You can view log files, Python scripts, CSV files, JSON and YAML data files in pSeven, in addition to previously supported Markdown and plain text files. To view a file, double-click it in Explorer.

• Upgraded the embedded pSeven Core package to version 6.29. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

• User storage quotas. A deployment administrator can set file storage quota which applies to all users (as default), and override that default with user-specific quota settings. Quota management requires disk quota support on the file storage server - see section Enabling user storage quota in the pSeven Enterprise deployment guide for details.

Bugfixes

• Fixed a workflow publishing issue where the published app could not run, if a block in the app's workflow reads files from the workflow directory.
• Fixed an issue where refreshing a browser tab with pSeven Enterprise or AppsHub sometimes entered a loop, indefinitely redirecting you to the sign-in page and back.
• Resolved a stability issue that could lead to errors appearing when you open a block, after which the block UI did not load - but could load successfully the next time you open the block.
• Fixed a UI regression issue from v2021.12 where pSeven sometimes displayed a run that has already finished as a queued run or a run in progress, and you had to reopen the run tab to fix that.

v2021.12

Updates

• Added initial support for the TensorFlow machine learning (ML) platform: in Python script blocks, you can use TensorFlow (import tensorflow) and Keras API (from tensorflow import keras) to develop and train ML models. Also added lasio (import lasio) to read and write Log ASCII Standard (LAS) files.

  Using TensorFlow, Keras API, or lasio in a Python script block that is configured to run on a Windows extension node is currently not supported.

• You can now test Python script blocks without running the workflow - see section Testing the script in the Python script block guide.

• Upgraded the embedded pSeven Core package to version 6.28. For details on recent improvements and changes, see the pSeven Core changelog.
• Added logging to the stock app UI in AppsHub, so you can see progress of an app run, its error details, and so on. To view or download app logs, click the Log button in the app page header. Previously published apps do not get this update automatically, but you can re-publish them with the new version of the stock app UI.
• You can now see folder (workflow, run) size in Explorer - select a folder, then expand the properties pane at the bottom and click calculate to get the selected folder size.
• To prevent clutter in the workflow run log, the Run logs pane displays memory usage messages from blocks only if you enable showing the debug log in the pane's menu.
• Error toast messages no longer disappear from screen by timeout - you close them manually, so there is no risk of missing an important error message. Information messages are automatically closed after a certain time, same as in previous versions.
• Links that may cause errors or unexpected behavior when running a workflow are highlighted orange (warnings) and red (errors). Warning and error details are available in link tooltips.
• Updated pSeven Enterprise Help:
  • Added section Testing the script to the Python script block guide.

Deployment and administration

• Fixed a deployment issue where the command to generate secrets failed due to a configuration error.
• Fixed an issue with pSeven Enterprise admin interface where searching for a user caused an error.

Bugfixes

• Fixed an issue where autolinking blocks could unexpectedly remove manually created links between those blocks.
• Fixed an issue with the Run logs pane where it was not showing logs of the selected block, or was showing logs of a different block than the selected one.
• Fixed an issue where error messages in a workflow run log were too wordy and always included full traceback, often making it harder to identify the error cause.
• Fixed block sorting in the Block library pane.
• Fixed various issues with keyboard navigation in the Explorer pane.
• Fixed a UI issue with the Explorer pane, which was too wide even when resized to its minimum.
• Fixed a UI issue with Run overview on the Start tab where it kept displaying the "Loading..." message instead of the usage tip text if there are no workflow runs that a user can view, causing confusion.

v2021.11

Updates

• Mass uplinking and adding ports to workflow parameters is now more convenient: in the Block properties pane, you can select multiple ports in Inputs or Outputs section, then use the and buttons that appear in the section header. adds uplinks by default, however if all selected ports are uplinked already, removes their uplinks. Similarly, excludes selected ports from parameters, if all of them are parameters already. And, since an output port cannot be a parameter, the button appears in Inputs only.
• The Block library pane loads noticeably faster in the case when the library contains blocks with many ports and a number of configuration presets.

• During a workflow run, the Workflow contents pane shows block memory usage, and blocks also regularly report their peak and average memory consumption to the run log.

  Block memory usage in the Workflow contents pane

  

• Run status indicators on the Start tab are slightly redesigned to unify their style and make color scheme consistent with the background color scheme used on run tabs.

  Run status indicators redesigned in v2021.11

  

• Completed workflow runs take less storage space because intermediate data files are automatically deleted when a run finishes.

• Changed the default parallelization ratio of batch-processing Composite blocks from 4 to 2 (the @Number of parallel workers setting). This is done to avoid situation where a workflow run that includes nested parallel Composite blocks requests too much resources by default, forcing you to manually edit settings of all Composite blocks to reduce the number of block instances created due to parallelization.
• Partially resolved workflow run performance issues caused by using List and Dict data types to pass big volumes of data through links between blocks. As a result, delays in workflow execution were reduced but not completely removed yet.
• Removed a distracting warning pop-up that appeared in certain cases when you add or remove an uplink in the Block properties pane.

Deployment and administration

• A deployment administrator can now control resource usage with the following new settings:
  • The Max blocks per run setting limits the number of block instances that may be created in a single workflow run started by any user. Use it to avoid situations where a single user consumes all cluster resources by running a workflow that creates a high number of block instances - for example, due to extensive usage of Composite blocks with enabled parallel processing. Note that each block nested into Composite with parallel processing creates multiple instances when running, and the number of those instances is controlled by user through the Composite block settings.
  • The Max open blocks setting limits the total number of blocks that can be open for editing at the same time, considering all blocks opened by every one of the users. Use it to avoid situation where the number of open blocks is high enough to consume most of cluster resources, which causes block edit and save errors as well as various other issues.
• The task executor settings in the pSeven Enterprise deployment configuration file (the sizing.htcondorexecuteResources.* parameters in values.yaml) now set resource requests for that service instead of resource limits. This improves the service stability in cases when its actual resource consumption (the amount of resources used by running blocks) exceeds the expected one (the request): the service can now use additional resources as long as they are not claimed by other services, instead of shutting down as soon as it reaches the limit. For further information about new settings, see the pSeven Enterprise upgrade notice below.
• Adjusted several deployment settings in the Helm chart to improve pSeven Enterprise stability:
  • Increased the number of user interface service (appserver) replicas from 2 to 3.
  • Increased the number of task submit service (htcondorsubmit) replicas from 1 to 2.
  • Increased amount of requested resources for some services.
• Removed the disableFileLock parameter from the pSeven Enterprise deployment configuration file (values.yaml) as it is intended only for troubleshooting in a few rare cases, while being dangerous and essentially useless in production deployments.
• Optimized UI to lower the number of requests to the pSeven Enterprise user interface service (appserver), reducing its load.
• Fixed an issue where the user interface service (appserver) repeatedly ran out of memory and restarted while there is a user who keeps open a tab with an ongoing workflow run. Frequent restarts caused UI issues noticeable to all users and, in some cases, errors during sign in.
• Fixed an issue with workflow run status notifications, due to which a few concurrently running workflows could overload the pSeven Enterprise user interface service (appserver), causing UI unresponsiveness and incorrect run statuses being displayed on the Start tab.
• Fixed an issue with published user blocks where pSeven Enterprise deployment failed if the folder name of some published block contains a space.
• Fixed extension node deployment issues:
  • After installing Poetry, node deployment could fail to install Python packages due to an issue with Poetry cache.
  • Node deployment could fail if you specify a relative path to the installation or temporary files directory when running bootstrap.bat.

User block development

• Fixed section Publishing a custom block in the block development guide: added missing instructions about setting access permissions for published blocks.

Bugfixes

• Fixed an issue where watching an ongoing run in a tab disrupted general UI functionality, so the UI became unresponsive for all users, and some users also experienced errors during sign in.
• Fixed unexpected slowdowns in workflow editing, which were noticeable when you have several workflow tabs open.
• Fixed an issue where a large workflow with many links open in a background tab caused unexpected slowdowns and UI unresponsiveness when editing workflows in other tabs.
• Resolved a stability issue that appeared when several users actively edit different blocks in their workflows at the same time. In particular, that issue caused errors when opening a block or saving block configuration changes.
• Fixed an issue with the Links dialog where it did not show ports that you have recently added to one of the blocks you are linking.
• Fixed an issue with the Links dialog where lists of ports disappeared when linking two blocks of the same type.
• Fixed issues with the Block properties pane where uplinking many ports caused slowdowns, incorrect UI behavior and errors.
• Fixed an issue with the Block properties pane where it did not properly update the list of ports when you select different blocks in the workflow and displayed ports of a previously selected block.
• Fixed an issue with the Block properties pane where it displayed uplinked ports as not uplinked.
• Fixed an issue where you could not uplink multiple ports to the same port of a Composite block using buttons in the Block properties pane.
• Fixed an issue where a block input value specified in the Block properties pane was not saved if you deselect that block or select another block immediately after input.
• Fixed an issue where input parameter values in the Run setup pane were not updated when you change parameter presets.
• Fixed an issue where uploading large files caused a 502 Bad Gateway error.
• Fixed delays in log output in the Run logs pane.
• Fixed an issue where a running workflow, which contains Composite blocks with enabled parallel processing, did not stop when a block nested into one of those Composite blocks fails to start, which caused various workflow errors later on.
• Fixed an issue with block working directory synchronization, due to which workflow runs occasionally stopped with a file access error.
• Fixed an issue where a workflow run stopped with a "Task failed" error immediately after start.
• Fixed a UI issue where a run tab stopped updating after you switch to another tab and back - for example, when you start several runs and switch between them.
• Fixed an issue with the Start tab where it displayed workflow run status incorrectly due to unexpected delays in status update.
• Fixed a UI issue where the Start tab did not show the list of runs in Run overview if you do not interact with UI after opening a new Start tab.
• Fixed some of the UI issues that appear on low-resolution displays (tablets and laptops), or when you resize the browser window with pSeven Enterprise.
• Fixed an issue with high resolutions (for example, on 4K displays), which caused delays when switching between tabs in the pSeven Enterprise UI.
• Fixed an issue with the Block library pane where the Workflow library section sometimes did not show block update notifications even if a newer version of a block is available from the Common library.
• Fixed several issues that caused incorrect behavior when editing workflow annotations - for example, the text you edit could be lost or added to another annotation instead of the one you are editing.
• Fixed a workflow copying issue where active workflow runs were "copied" with the workflow. A copy of the workflow, which was created while that workflow is running, contained runs that were incorrectly displayed as active (running) while in fact those runs in the copy are interrupted.
• Fixed an issue where dragging a workflow in a run tab reset block startup counters to 0.
• Fixed an issue where a workflow run that contains a Composite block with enabled parallel processing failed with an "unknown value id" error.
• Fixed an issue where you could accidentally clear some result value in the Run setup pane, because the value reset button was displayed when you click a result despite that button is intended for input parameters only.
• Fixed issues with the stock app UI in AppsHub:
  • The Stop button did not work after you refresh the app page.
  • The app UI did not clear the list of files in the Results pane once you start a new calculation.
• Fixed a UI issue in the Explorer pane where a file upload progress bar remained at 0 if the cursor is away from the pane.

pSeven Enterprise upgrade notice

In the pSeven Enterprise v2021.11 deployment configuration file (values.yaml), resource settings for task executors specify resource requests instead of limits: the sizing.htcondorexecuteResources.limits.* parameters are replaced with the sizing.htcondorexecuteResources.requests.* parameters. Note that requests (sizing.htcondorexecuteResources.requests.*) are now required while limits (sizing.htcondorexecuteResources.limits.*) are optional, and the v2021.11 deployment configuration file sets only requests parameters by default.

When transferring parameter settings from an existing deployment to the v2021.11 configuration file (see Upgrading pSeven Enterprise), there are two advisable ways to update resource settings for task executors:

• To improve pSeven Enterprise stability under load, specify requests only. Set requests to the same values that were previously specified as limits. This allows task executors to temporarily exceed the expected resource consumption, using additional resources as long as they are not claimed by other services.
• To reproduce behavior of previous pSeven Enterprise versions, specify requests and limits, and set both to the same values that were previously specified as limits. Note that with such settings, Kubernetes pods where task executors run will be given a QoS class of Guaranteed (see QoS classes in the Kubernetes documentation).

v2021.10

Updates

• The code editor in Python script blocks now supports search and a few other common functions - see section Keyboard shortcuts in the Python script block guide.
• Ctrl+S or Cmd+S in Python script now saves the block.
• Newly created workflows now contain the README.md file template. When publishing a workflow to AppsHub, you can use README.md to add descriptions of the app, its parameters and results. See Adding descriptions to the stock app UI for more information.
• Improved editing and display of parameters and results in the stock app UI in AppsHub:
  • Non-default parameter values are now displayed in bold, so you can easily tell which parameters you have already changed.
  • Each parameter now has a button, which unsets the parameter value.
  • If a result value is too long to display it in the result field (for example, the value is a matrix, a long list or string), a button appears. Clicking it opens a dialog where you can view the full value.
  • Increased value preview length in result tooltips.
• You can now unset input values in the Block properties and Run setup panes: click a value to edit, then click in the input field.
• Pan action in a workflow tab now works more smoothly.
• Blocks in workflows of example apps in AppsHub are now automatically updated to current versions upon a pSeven Enterprise version upgrade.
• Added a beta tag to the GraphQL API command in the user menu to indicate that pSeven Enterprise GraphQL API is unstable and is not yet ready to be used in production.
• Updated pSeven Enterprise Help:
  • Added section Keyboard shortcuts to the Python script block guide.
  • Updated license information and instructions in section License of the deployment guide as the pSeven Enterprise license is now deployment specific and bound to the Installation ID.
  • The Russian language version of pSeven Enterprise Help now contains a full changelog, the same as the English version.

Deployment and administration

• pSeven Enterprise licenses are now deployment specific. Each new deployment has a unique Installation ID (which is now generated during installation) and requires a valid license bound to this specific Installation ID. An existing Installation ID is kept upon a pSeven Enterprise version upgrade, so a license previously issued for that Installation ID remains valid after the upgrade, within its licensing period. However if you upgrade an existing deployment that does not yet have an Installation ID, you will need to request an updated license, as the upgrade will generate a new Installation ID for your deployment. For further details, see the pSeven Enterprise upgrade notice below.
• Fixed an issue where file storage requirements (see section NFS server in the pSeven Enterprise deployment guide) were not checked properly during installation, so deployment with an unsupported file storage continued but failed after a long timeout.
• Fixed an issue with the extension node setup package, due to which a node could appear working after installation but stopped functioning later - for example, after a reboot or after some of the pSeven Enterprise services on the node had been restarted.
• Fixed an issue where the OpenVPN port setting for extension nodes specified in the pSeven Enterprise deployment configuration file (the entrypoint.gateOpenvpn parameter in values.yaml) did not apply, requiring you to manually edit the OpenVPN configuration file (winnode.ovpn) on every extension node if you change the entrypoint.gateOpenvpn parameter during deployment.

User block development

• It is now possible to implement and use a Windows-only block, which starts on a Windows extension node both in edit (configuration) and run (execution) modes. A user configuring such a block should select a Windows node to run on in the Block properties pane prior to opening block configuration. Thus a block in edit mode will be able to access node-specific resources - for example, Windows-only software tools installed on the selected node.

Bugfixes

• Fixed an issue where saving a block was taking too much time.
• Fixed unexpected slowdowns when deleting files in Explorer, deleting blocks in the workflow or in the Workflow library pane.
• Fixed an issue where pSeven Enterprise was becoming unresponsive if you edit a complex workflow containing many blocks and links.
• Fixed an issue where changes in workflows and apps were not showing until you delete the browser cache, which happened after a pSeven Enterprise version upgrade.
• Fixed an issue where a workflow failed to start after you update a block in the Workflow library to a newer version.
• Fixed slow loading of the Block properties pane.
• Fixed an issue with the Run logs pane where it sometimes did not show the log until you reopen the pane.
• Fixed an issue where scroll position in the Run logs pane was reset if you switch between document (workflow, report) tabs.
• Fixed an issue with the Block properties pane, which was incorrectly highlighting an optional input port as a port required to start the block.
• Fixed an AppsHub issue where refreshing an app page reset the app UI and unexpectedly stopped the app if it was running calculation.
• Fixed issues with the stock app UI in AppsHub:
  • The UI sometimes did not show results of a completed calculation.
  • The UI blocked editing of a parameter if it has no default value.
  • A parameter value could be ignored if that parameter is an input port of some Composite block in the app's workflow.
  • The UI did not clear results of a previous calculation when you start a new one, causing confusion.
• Fixed an issue with environment settings on extension nodes where pSeven Enterprise incorrectly set MKL_NUM_THREADS, NUMEXPR_NUM_THREADS, OMP_NUM_THREADS, and several other environment variables for the process it starts on the node. Those settings limited the number of CPU cores provisioned to the process, which could cause errors when running multithreaded CAD or CAE packages.
• Fixed an issue where you could not copy a shared workflow if another user is editing a block in that workflow at the moment.
• Fixed an issue with REST API where results of a finished run could be empty.
• Fixed an issue where a workflow run could enter a deadlock, if the workflow contains a Python script block that declares a function but never calls it in the script, and the response port corresponding to that function has a link connected to it and actually receives function values, which the block does not use.
• Fixed several bugs that sometimes caused unexpected behavior of common file management commands in Explorer - for example, the sequence of Ctrl+C and Ctrl+V hotkeys, which commonly creates a copy of the selected file or directory, was not working as intended.
• Fixed minor UI issues in the Composite properties pane.
• Fixed unexpected UI behavior when quickly opening and closing block configuration dialogs.

pSeven Enterprise upgrade notice

Since v2021.10, pSeven Enterprise licenses become bound to a specific Installation ID. A unique Installation ID is automatically generated during installation, and each deployment requires a license key valid for its Installation ID. The license key contains information about the ID, and can only be used for the deployment with that ID.

Deployments of pSeven Enterprise v2021.10 and future versions will keep their once generated Installation ID upon a version upgrade, so a deployment specific license for them has to be issued only once.

Deployments of prior versions (v2021.09 and earlier) have no Installation ID. For such deployments, a new ID is generated during a version upgrade, so the previously used license becomes incompatible as it was not generated for a deployment with a specific Installation ID.

If you upgrade an existing deployment to v2021.10, you will need to request an updated license valid for your newly generated Installation ID. To get the ID, sign in as admin after v2021.10 installation finishes and copy the ID string from the admin page header (see section License in the pSeven Enterprise deployment guide for more details). This is a one-time update required to switch to the new version of license, which supports Installation ID; future upgrades will keep the same ID.

v2021.09

Updates

• Added initial ONNX support: the Python script block now provides the ONNX Runtime (import onnxruntime), which enables the usage of ONNX models from deep learning frameworks such as PyTorch and TensorFlow as well as classical machine learning libraries such as scikit-learn, XGBoost, and other.

  Using the ONNX Runtime in a Python script block that is configured to run on a Windows extension node is currently not supported.

• The stock app UI in AppsHub now supports uploading input files and downloading files from app results.

• Extended the Python script block functionality:
  • Connect to MongoDB databases with PyMongo (import pymongo).
  • Retry function calls (for example, HTTP requests) with backoff (import backoff).
  • Establish SSH connections with Paramiko (import paramiko).

• Added caching support in the Python script block. For example, if your block runs in a loop, you can cache certain data or objects to avoid initializing them every time the block runs. See Caching Python objects in the Python script block guide for details.

  Note that the cache should not be used as a persistent storage, because it may be reset between block runs.

• Added a few common convenience features to the code editor in the Python script block:

  • code folding support
  • highlight all occurrences of a word under the caret
  • Ctrl+/ to comment the current line or selected lines

• NumPy arrays sent to output ports of a Python script block are now required to be 2-dimensional in order to avoid errors caused by inability to convert an array to the matrix data type. The support for N-dimensional arrays may be added in future versions.

  This change may require you to update your scripts in Python script blocks - see section Compatibility issues for details.

• Upgraded the embedded pSeven Core package to version 6.24. For details on recent improvements and changes, see the pSeven Core changelog.

• The Run setup pane now supports tree-like grouping of parameters and results. For example, parameters named geometry.x and geometry.L are displayed as a collapsible group geometry containing inputs x and L.
• Blocks in the Workflow library pane are now sorted by name, with different versions of the same block sorted in the version decreasing order.
• Added block help shortcuts to the Block library pane: hover over a block in the library and click the :fontawesome-solid-question-circle: icon that appears.
• Improved AppsHub search to find search term occurrences in app descriptions. Also, you can search by username to find all apps published by a certain user.
• Updated design of title bars in block configuration dialogs for consistency. Block title bar now always contains the Save and Help buttons. The Save button now saves configuration but does not close the dialog (use Alt+F4 to close). The Apply button is removed.
• Added port tooltips in the Block properties and Composite properties panes, explaining the current state of each port.
• If you open a folder in the Explorer pane and it takes a long time to load and display folder contents, the pane now shows a loading message.
• To avoid errors, a driver block can now be grouped into a Composite block only if you also group to the same Composite the entire body of a loop controlled by that driver block (select the driver block and all blocks in the loop before grouping).
• Added a message explaining the workflow run error caused by incorrect usage of conditional branching inside a Composite block with enabled batch processing.
• Updated tooltips in the Explorer pane to better explain the use of commands, and added missing tooltips.
• Removed the Open button from the collapsible information pane in Explorer as redundant.
• Disabled the workflow run hotkeys F9 and F10 as they often conflict with browser hotkeys.
• Updated pSeven Enterprise Help:
  • Updated the description of Kubernetes cluster resource requirements in the Prerequisites section of the pSeven Enterprise deployment guide (moved: see Requirements).
  • Reworked the deployment diagram in the pSeven Enterprise deployment guide and added the service interaction diagram showing pSeven Enterprise services, network connections and ports in more detail.
  • Extended the pSeven Enterprise deployment guide with detailed instructions on how to configure an NFS server for file storage - see NFS server.
  • Added a note about required ports to the Extension node deployment guide (see section Before installation).
  • Added description of pSeven Enterprise branding options - see Branding.
  • Added notes on required user permissions to the User blocks guide.
  • Updated the Python script block guide with the Working directories section describing how to configure the block working directory.
  • Added section Caching Python objects to the Python script block guide.
  • Added Python examples to the REST API guide.

Deployment and administration

• The Technical support section now explains what information you should collect and provide when contacting support with deployment issues.
• Optimized resource requirements for pSeven Enterprise services in the Helm chart so the services are now easier to distribute among the Kubernetes cluster resources. Requirements for non-replicated services are now set in a specific way as required to give their pods a QoS class of Guaranteed, improving their stability (see QoS classes in the Kubernetes documentation).
• In the pSeven Enterprise deployment configuration file (values.yaml), you can now set annotations which are added only to Kubernetes jobs and pods created by jobs - see the annotations.job parameter. The annotations added by annotations.job override annotations from annotations.all.
• Removed memory limits for post-install jobs started by Helm, which run once to check and finalize the pSeven Enterprise installation. This change is done to avoid deployment failures, which sometimes occurred because a one-time post-install job exceeded its memory limit and was stopped by Kubernetes. Also, post-install jobs are now allowed to restart up to 3 times in case of errors.
• Increased the ephemeral storage size for the pSeven Enterprise edge router service to avoid its restarts, which sometimes occurred under a peak load because the service log size exceeded storage size and Kubernetes destroyed the service.
• Added branding options: you can now set a custom logo on the pSeven Enterprise sign-in page and in the navigation bar, and add a custom Getting Started page. See Branding for details.

User block development

• Getting block source code (a .p7protoblock folder) from the block library now requires the app_auth | user | Can develop blocks user permission. New and existing users will have this permission by default, and a deployment administrator can revoke it. Users without this permission will be unable to drag blocks from the block library to the Explorer pane as well as add blocks from Explorer to the Workflow library pane.
• Fixed an issue where you could not add a block from the Explorer pane to the workflow library if the block manifest did not define default values of all ports.

Bugfixes

• Fixed an issue where the block UI sometimes did not load when you open the block configuration dialog.
• Fixed the "Task scheduler service restart" error, which sometimes occurred while running a workflow.
• Fixed errors and slowdowns when viewing large run logs.
• Fixed an issue where running a workflow with a Python script block that produces many log messages resulted in a "Memory limit exceeded" error, regardless of the memory limit specified in block configuration.
• Fixed an issue in the Python script block where function arguments in the Functions pane were listed in alphabetical order instead of their declaration order, causing confusion.
• Fixed an issue in the Python script block where using the concurrent call syntax with 3 or more functions (for example, (F1&F2&F3)(x1, x2, x3)) caused an error.
• Fixed an issue with the Python script block where writing a large amount of data to the block's temporary storage directory (api.tempdir) could cause an error in all running workflows.
• Fixed an issue in the Python script block where it was possible to send an N-dimensional NumPy array to an output port, causing an error because an N-dimensional array cannot be converted to the matrix data type.
• Fixed an issue with the variable quick add syntax in the Python script block where ranges in names of variables were sometimes parsed incorrectly.
• Fixed an issue in the Python script block where names of variables added using the quick add syntax were not properly validated.
• Fixed an issue where changing the value assigned to an uplinked port did not change the value assigned to the corresponding port of a parent block as intended.
• Fixed the inability to group a block into a Composite block, if there are multiple links connected to the same port of the block you are grouping.
• Fixed an error while running a workflow containing two or more Composite blocks with similar names, which have ports added to workflow results.
• Fixed an AppsHub issue where re-publishing a workflow changed its app ID, which required you to also update all REST clients using that app.
• Fixed an AppsHub issue where you could overwrite an existing app by publishing another workflow under the same name, even if you are not the owner of that app.
• Fixed an issue with the stock app UI in AppsHub where descriptions of results appeared only after you run the app.
• Fixed an AppsHub issue where you could not open an app if its description was saved with encoding different from UTF-8.
• Fixed incorrect behavior of the Publish workflow dialog - for example, it allowed you to click the Publish button multiple times, resulting in repeated publishing of the workflow.
• Fixed an issue where you could not create a copy of a read-only shared folder.
• Fixed an issue where deleting a workflow sometimes caused an error, after which you could not delete that workflow.
• Fixed an issue where workflow run folders were occupying more storage space than intended because they contained hidden system files.
• Fixed a sign-in issue where re-opening the pSeven Enterprise sign-in page after a successful sign-in did not properly redirect you to pSeven Enterprise or AppsHub.
• Fixed an issue where double-clicking a workflow, run, or report tab title did not open the corresponding folder in the Explorer pane as intended if the pane is collapsed.
• Fixed an issue where you could not preview the app UI in a workflow (open the index.html file from the workflow UI folder in Explorer) if the workflow name or path to the workflow contain special characters.
• Fixed an issue where you could not open an HTML file located in a run directory or the workflow Files folder.
• Fixed incorrect status code and error message in a response to an invalid REST API file upload request.
• Fixed an issue with renaming files and folders in the Explorer pane where it was impossible to select text with a mouse during rename because this started a drag-and-drop action.
• Fixed a UI issue with the parameter preset selector in the Run setup pane, which sometimes displayed duplicated presets.

Compatibility issues

This update requires NumPy arrays sent to output ports of a Python script block to be 2-dimensional, which in particular disables the output of 1-dimensional arrays. In previous versions, an 1-dimensional NumPy array was converted to a single-column matrix on output. In the new Python script block version coming with pSeven Enterprise v2021.09, that conversion has to be done explicitly in your Python script code.

If your workflows contain Python script blocks that output 1-dimensional NumPy arrays, you will need to update your Python code in those blocks accordingly before upgrading the Python script block in the workflow library to the latest version. Note that the Python code that converts 1-dimensional arrays to 2-dimensional before output is going to work correctly with either block version.

v2021.07

Updates

• You can now enable or disable run-time resource management in your deployment when installing pSeven Enterprise or upgrading it to a new version. In pSeven Enterprise v2021.07, resource management is disabled by default. See the pSeven Enterprise upgrade notice below for details.
• Adjusted resource requirements for pSeven Enterprise services in the Helm chart to their actual resource consumption based on the recent resource usage statistics gathered from real pSeven Enterprise deployments.
• Improved pSeven Enterprise stability in many cases where users experienced issues when configuring blocks, editing or running workflows.
• pSeven Enterprise administrators can now create users who have AppsHub access only. Such users may run apps from AppsHub but cannot create, edit, or run workflows.
• When you download a workflow or a folder containing some workflows, you can now skip downloading the run results stored with workflows to speed up download and reduce its size.
• You can now upload workflows or folders in ZIP archives and extract them in pSeven Enterprise using the new Unzip command from the menu in Explorer.
• Upgraded the embedded pSeven Core package to version 6.23. For details on recent improvements and changes, see the pSeven Core changelog.
• The Python script block now provides optional pandas dependencies like pytables, tabulate, and other.
• Updated pSeven Enterprise Help:
  • Added section Workflow guides, which explains pSeven Enterprise concepts and provides a general user guide.
  • Added section Block guides, which provides help for the Python script and Condition blocks.
  • Added section User blocks guide, which provides an updated block development guide.
  • Minor improvements in the pSeven Enterprise deployment guide.

User block development

• Fixed an issue where pSeven Enterprise displayed an empty Common library pane after a block with an invalid manifest had been published.
• Fixed an issue due to which it was impossible to implement a block that allows changing default values of its ports.
• Added the block development guide to pSeven Enterprise Help - see User blocks guide.
• Removed the outdated User Block Example from AppsHub. Note that if you upgrade an existing deployment, this example is not removed automatically. It is recommended to delete it manually after the upgrade - see section Compatibility issues for details.

Bugfixes

• Fixed an issue where run-time resource management did not work as intended when running a workflow that contains a block configured to run on an extension node.
• Fixed a deployment issue where annotations specified in the pSeven Enterprise deployment configuration file (the annotations.all parameter in values.yaml) were not added to some of the Kubernetes pods during deployment.
• Fixed an issue where saving a script with a syntax error in the Python script block produced a generic error message instead of a message that locates the error.
• Fixed an issue where you could add a function with no arguments in the Python script block, which caused an error when running the workflow.
• Fixed an issue where you could not change access rights of a folder you have shared earlier.
• Fixed an issue where you could not preview the app UI in a workflow (open the index.html file from the workflow UI folder in Explorer) if the workflow had been shared to you by another user.
• Fixed incorrect UI behavior and various issues that occurred when you can no longer access a shared workflow because its owner had deleted the workflow or revoked your access.
• Fixed errors caused by deleting a workflow that is currently running.
• Fixed an AppsHub issue where you could not open an app if its name contains a dot.
• Fixed an issue where the pop-up message displayed after publishing a workflow to AppsHub contained an invalid link to the app.
• Fixed an issue where you sometimes could not save configuration in the Example user block.
• Fixed an issue where the Ctrl+V hotkey was not working in the workflow editor.

Compatibility issues

• pSeven Enterprise v2021.07 disables run-time resource management by default. To enable the resource management features, which were introduced in v2021.06, set the sizing.enableResourceManagement parameter in the pSeven Enterprise deployment configuration file (values.yaml) to true. For details, see the pSeven Enterprise upgrade notice below.
• The default maximum number of users (the maxUsers parameter in values.yaml) in pSeven Enterprise v2021.07 is increased to 256 (previously the default was 128). When upgrading, it is recommended to keep the new default value, unless a non-default value was specified in your existing deployment.

• pSeven Enterprise v2021.07 no longer installs the User Block Example to AppsHub, as this example is outdated and is completely replaced by the new block development guide (see User blocks guide) and the example implementation of a user block available from the block library. However if you upgrade an existing deployment to v2021.07, the old example is not removed from AppsHub automatically. To avoid using that outdated example, it is recommended to manually delete it from the AppsHub data storage after you upgrade to v2021.07.

  • The AppsHub data storage location is specified by the storage.appdata.* parameters in the pSeven Enterprise deployment configuration file (values.yaml). Connect to the server specified by the storage.appdata.nfs.server parameter and locate the directory specified by storage.appdata.nfs.path. This is the AppsHub data storage directory.

  • To delete the outdated User Block Example, remove the User Block Example.p7wf subdirectory and all its contents from the AppsHub data storage directory.

    Warning

    Do not remove any subdirectories other than User Block Example.p7wf from the AppsHub data storage. Those subdirectories contain more example apps, as well as apps published by pSeven Enterprise users.

pSeven Enterprise upgrade notice

pSeven Enterprise v2021.07 deployment configuration file (values.yaml) is updated with a new parameter, sizing.enableResourceManagement, which you can use to enable or disable the run-time resource management features introduced in v2021.06 (default is disabled). This setting applies once during installation and cannot be changed afterwards, unless you re-install.

If you choose to enable resource management, configure it as described in the pSeven Enterprise upgrade notice for v2021.06.

If you choose to keep resource management features disabled, you can limit the maximum load by adjusting the sizing.htcondorexecuteSlots parameter, which sets the number of tasks that may run on a task execution service instance.

v2021.06

Updates

• Improved run-time resource management:

  • Blocks now declare their memory and CPU requirements, and pSeven Enterprise uses this information to schedule block execution.
  • pSeven Enterprise sets run-time memory and CPU limits for blocks, allowing a block to use no more resources than it requires.
  • In Python script blocks, you can set per-block resource requirements in the new Resources section of the block configuration dialog.
  • User blocks can declare resource requirements in block manifest and can also implement per-block resource settings in their configuration dialogs.
  • If a block does not declare its resource requirements, default limits apply, set by an administrator in deployment configuration.

  Warning

  The resource management updates outlined above require you to update all blocks in workflows created in previous versions of pSeven Enterprise. To update a block, click the update notification next to the block's name in the Workflow library section of the Block library pane.

  If you do not update a block, the workflow will keep using an old block version, which does not declare resource requirements, so default limits always apply. Since the default limits are rather low, you may experience various issues if you run a workflow that contains old block versions.

• You can now reduce the size of a workflow folder by removing the history of changes - open the workflow and use the Clear history command from the menu in the workflow edit toolbar.

• Plotly is now available in Python script blocks.

• To access files in Python script blocks, you can now use the api.rundir and api.wfdir variables that contain, respectively, the path to the current run directory and the path to the workflow folder. Import the api module (import api) to use. Both variables are pathlib.Path objects.

  Join paths

  The join operator for pathlib paths is /. For example, to get the full path to the file named file_name.ext, located at the root of the run directory:

  import api
  
  file_path = api.rundir / 'file_name.ext'
  

  Files in the block's working directory

  To work with files located inside the block's working directory, you do not need any special methods - just use relative paths. This works because the block sets its working directory as the current working directory for the script it runs.

• Workflow run folders now occupy less storage space.

• Improved responsiveness and loading speed of the Block properties pane.
• Improved autolinking: the Autolink dialog suggests links between ports with partially matching names, and can match ports from different port groups.
• Recently closed workflows and reports now appear on top in the list of recent documents, so they are easier to find.
• Helm post-install hooks and jobs no longer disappear during deployment, so you can review their logs when needed.
• pSeven Enterprise deployment no longer creates persistent volumes that require additional privileges.
• Upgraded the embedded pSeven Core package to version 6.22. For details on recent improvements and changes, see the pSeven Core changelog.
• Added more details about NFS server configuration to section File storage in the pSeven Enterprise deployment guide (moved: see NFS server).
• Reworked the Version upgrade guide to avoid issues with re-using configuration files from an existing deployment. Until further notice, the recommended method to prepare a configuration file for an update is to manually transfer settings from the existing configuration.
• Updated the REST API guide:
  • Added section File upload.
  • Added section Downloading the run log in File download.
  • Added a note about getting the API token from the pSeven Enterprise user menu in API access.
• Updated the Russian language version of pSeven Enterprise Help to bring it in line with the English language version. To select your language, use the menu in the Help's top bar.

User block development

• Due to the resource management updates, blocks should also be updated so they declare resource requirements, and users should be additionally notified to update blocks in their workflows. See the User block upgrade notice for details.
• Fixed an issue where users could not update a block in their workflows if the name of the folder where developers have published the updated block version was not following a specific pattern.

Bugfixes

• Fixed an issue where pSeven Enterprise could stop responding due to a malfunctioning task scheduler service.
• Fixed an issue where a workflow with blocks configured to run on a Windows extension node often stopped with an error, if these blocks were nested into a Composite block with parallel processing enabled.
• Fixed performance issues caused by adding a large file to the workflow Files folder.
• Fixed an issue where editing a block always reset its previously assigned port values to default, as well as resetting related workflow parameters.
• Fixed an issue where pSeven did not correctly create working directories for a Composite block with working directory set to indexed and parallel processing enabled.
• Fixed an issue where pSeven sometimes did not update a run tab properly, displaying a run as active (running) while it has already finished.
• Fixed the unexpected UI behavior after opening several workflow tabs in rapid succession.
• Fixed errors when deleting, moving or copying a folder that contains many subfolders.
• Fixed an issue where it was possible to create duplicate links - for example, two links connecting the same port pair.
• Fixed an issue where pSeven displayed incorrect values of block start-up counters on a run tab.
• Fixed a few minor issues with the Links and Autolink dialogs.
• Fixed an issue where pSeven kept displaying long-running operation pop-ups at the bottom.
• Fixed an issue where a side panel with debugging information (Django admin panel) appeared in AppsHub and in pSeven Enterprise admin interface.

Compatibility issues

• pSeven Enterprise v2021.06 changes the requirements to the file storage (NFS) server: now it supports NFSv3 only and requires disabling NFSv4 - see section NFS server). in the pSeven Enterprise deployment guide for details.

  Important

  Reconfigure the NFS server before upgrading to v2021.06, otherwise the v2021.06 deployment will fail.

• Due to the resource management updates in pSeven Enterprise v2021.06, user blocks should also be updated so they declare resource requirements. Otherwise, the general memory and CPU usage limits set in the deployment configuration will apply. The general limits may be too low for some user blocks - for example, if a block processes large data arrays, it may run out of memory. Also, users should be additionally notified to update blocks in their workflows created in previous versions of pSeven Enterprise. For details, see the User block upgrade notice below.

• pSeven Enterprise v2021.06 uses an updated deployment configuration file, which contains new parameters related to resource management. These parameters should be adjusted when upgrading to v2021.06. For configuration details, see the pSeven Enterprise upgrade notice below.

pSeven Enterprise upgrade notice

pSeven Enterprise v2021.06 deployment configuration file (values.yaml) contains several new parameters related to resource management. To ensure correct resource allocation and to avoid issues when running workflows, these parameters should be set deliberately and with regard to resources available in your deployment.

Follow the guidelines in section Upgrading pSeven Enterprise to generate a new configuration file for v2021.06 and transfer parameter settings from your existing deployment to the new file. In addition to this, find the following sizing.* parameters in the new file and adjust their values:

• sizing.blockrunDefaultResources.memory - default memory usage limit for a block run task.
• sizing.blockrunDefaultResources.cpu - default CPU usage limit for a block run task.
• sizing.htcondorexecuteReplicas - the number of task execution service instances to create.
• sizing.htcondorexecuteResources.limits.memory - total memory usage limit for a task executor.
• sizing.htcondorexecuteResources.limits.cpu - total CPU usage limit for a task executor.

Limits set by the sizing.blockrunDefaultResources.* parameters apply to blocks that do not declare their specific resource requirements. Do not set these default limits too high as this leads to inefficient resource usage, which causes performance issues.

Values of the sizing.htcondorexecuteReplicas and sizing.htcondorexecuteResources.limits.* parameters should be selected carefully with regard to expected load and to resources that are in fact available in your deployment. For details, see comments to these parameters in the v2021.06 configuration file (values.yaml).

User block upgrade notice

User blocks developed for prior pSeven Enterprise versions do not declare their resource requirements as expected in pSeven Enterprise v2021.06. Due to this, default run-time memory and CPU limits will apply to these user blocks unless you prepare an updated block version and add it to the block library. The default limits may be too low for some user blocks, so users may experience issues when running workflows that contain old block versions.

Before you upgrade pSeven Enterprise to v2021.06, it is recommended to update each user block, adding run-time resource requirements to the block's manifest, and upload the updated version to the block library beforehand - so the block update is readily available to users once you upgrade to v2021.06.

• Make a copy of your .p7protoblock directory, which will be the new block version. For example, if you have a directory named TheBlock-v0.p7protoblock, copy it to TheBlock-v1.p7protoblock.

• In the copy (TheBlock-v1.p7protoblock) open the block's manifest (manifest.yaml) and edit the following parameters:

  • Under default_preset.template, add the resources parameter specifying the block's requirements as in the following example:

    default_preset:
        template:
            # Do not copy this example as is.
            resources:
                run:
                    memory: 1024
                    cpu: 250
            # Other template parameters follow...
    

    Note that the memory and CPU requirements must be set according to the real block memory and CPU usage. Memory is specified in MB, so memory: 1024 sets the memory requirement to 1 GB. CPU requirement is specified in millicores, so cpu: 250 sets it to 0.25 vCPU.

    When adding the resources parameter, do not remove any of the existing parameters under default_preset.template.

  • After specifying the resource requirements, increase the version parameter by 1 - for example:

    # version: 2
    version: 3
    

  Important

  Do not change the id parameter in the manifest - otherwise pSeven Enterprise will recognize your update as an entirely different block, and users will not be able to update blocks in their workflows automatically.

  Also do not change the manifest_version and protocol_version parameters - they are reserved for future use and should not be edited.

• Save your changes to manifest.yaml and upload the new block version (TheBlock-v1.p7protoblock) to the pSeven Enterprise block library (the /protoblocks directory in the shared data storage). This updated version will appear in the Common library section of the Block library pane in the pSeven Enterprise user interface. In workflows that contain an old version of the block, the Workflow library section of the Block library pane will show the old version along with an update notification.

• After upgrading to pSeven Enterprise v2021.06, notify your users that they need to update all blocks in their workflows created in previous versions of pSeven Enterprise - otherwise they may experience issues if they run a workflow that contains old block versions.

  To update a block in a workflow, click the update notification next to the block's name in the Workflow library section of the Block library pane.

v2021.05

Updates

• Workflow and report downloads now run faster and have less size: when archiving the document to download, pSeven no longer adds the history of document changes to the archive. Previously, downloads always kept a full history, which could greatly increase the archive size.
• Added a few convenience improvements related to configuring and linking blocks that have driver ports - for example, Python script blocks with function and argument ports:
  • Driver ports and generic ports are now sorted separately in the Links and Autolink dialogs.
  • Output driver ports are labeled "Request": a block uses them to request evaluations from other blocks connected to it.
  • Input driver ports are labeled "Response": they accept responses (evaluation results) from connected blocks.
• Upgraded the embedded pSeven Core package to version 6.21. For details on recent improvements and changes, see the pSeven Core changelog.
• Reduced the extension node setup package size. Removed a few unused versions of packaged components, which could cause issues during a node deployment.
• Updated the pSeven Enterprise deployment guide for more clarity:
  • Reworked section Deployment (moved: see Deployment configuration, Secrets, Installation, and further).
  • Added section Upgrade (moved: see Version upgrade).
  • Updated section Verify installation.
• Updated the Extension node deployment guide, added section Allowed users (moved: see Configuring user authorization on extension nodes).
• Added the Russian language version to pSeven Enterprise Help. To select your language, use the menu in the Help's top bar.

Note

pSeven Enterprise Help is now also available at the DATADVANCE website. See www.pseven.io/permalink/pseven-enterprise-help for the latest version.

User block development

• When block developers update a block in the common library, users can now update this block in their workflows in a few clicks - instead of manually replacing old versions of the block. pSeven shows block update notifications in the Workflow library pane; click the notification to start the update.
• Fixed issues in block manifest validation, due to which you could add an unusable block with an invalid name to the block library, or add a block with an invalid id, which you then could not delete.
• Fixed an issue where the Block library pane sometimes displayed wrong icons for user blocks.

Bugfixes

• Fixed an issue where any user with admin access could see workflow runs of other users on the Start tab, and an attempt to open one of these runs led to an error.
• Fixed a rare issue where a workflow run did not start but pSeven did not show any error, so the run appeared to be waiting indefinitely.
• Fixed a stability issue which could disable editing and running a workflow if there was an error while editing this workflow, so you had to create its copy to continue.
• Fixed an issue where pSeven was showing an incorrect inline preview for a matrix value - for example, port values in the Block properties pane or parameter values in the Run setup pane.
• Removed dummy test blocks from the block library.
• Fixed Enter not working as a confirmation in the file deletion dialog.

v2021.04.20

Updates

• Added the Condition block, which enables conditional branching in workflows.
• Upgraded the embedded pSeven Core package to version 6.20. For details on recent improvements and changes, see the pSeven Core changelog.
• pSeven Enterprise Help is now available.
• Updated the Extension node deployment guide, added section Upgrade (moved: see Upgrading extension nodes).
• Minor updates in the pSeven Enterprise deployment guide.

User block development

• Deployment administrators can now publish user blocks. Copy a user block to the directory named protoblocks in the shared data storage to make it available to all pSeven users.

  The storage server address and the shared data directory path are specified by the storage.shareddata.* parameters in the deployment configuration file (values.yaml).

Bugfixes

• Fixed an issue in the Python script block, due to which it sometimes did not save configuration changes.
• Fixed a bug due to which unfinished Kubernetes jobs were building up and could consume all resources, potentially leading to a general failure.
• Fixed a bug in UDP package routing, which could cause instability of the OpenVPN connection between pSeven and Windows extension nodes.

Compatibility issues

• Since this release, pSeven supports only NFS file storages. If your current deployment uses a persistent volume claim or a custom storage class, you will have to set up an NFS storage and transfer all files there before upgrading to v2021.04.20. Note the requirements in section NFS server of the pSeven Enterprise deployment guide.
• If you use extension nodes, you will have to manually remove HTCondor on each node before updating it with the new node setup package - see the Extension node upgrade notice below.

Extension node upgrade notice

The version of the HTCondor service in the v2021.04.20 extension node setup package is not compatible with previous installations, so you will have to remove HTCondor on each of the extension nodes manually before upgrading.

• Open the Windows Control Panel - Programs and Features (or run appwiz.cpl to open Programs and Features directly). Find HTCondor in the list and uninstall it. If the uninstaller prompts for a restart, reboot the node before you continue.
• Get the new pSevenExtensionNode.zip setup package from pSeven (see Setup package).
• Unpack pSevenExtensionNode.zip to a temporary directory. The path to this directory must not contain spaces, for example: C:\Temp\pSevenExtensionSetup.
• From this directory, run bootstrap.bat without parameters. The setup scripts will use settings from the previous installation, which are stored into environment variables on the node.
• When bootstrap.bat finishes, check its output for errors. If there are none, reboot. If you see errors, save the script's output from the command prompt and contact the DATADVANCE support team (see Technical support).
• After the reboot, check the node as described in the Extension node deployment guide, section Node check.

v2021.04

Updates

• It is now possible to restrict user access to a Windows extension node connected with pSeven - see section Configuring user authorization on extension nodes for details.
• Added the Stop button to the stock app UI in AppsHub. You can use that button to stop a running app. Also improved the stock app UI stability in case of connection issues.
• Workflow and report copy now works faster, and newly created copies take less space because pSeven no longer copies the history of changes with the document. The original document keeps a full history, but the history in a copy will contain only newer changes.
• Added the Getting started page, which provides a quick overview of pSeven's functions and features and links to more guides. To open it, click the Getting started button on the Start tab.
• Since reports in pSeven are currently limited, and many of their planned features are missing, the New report dialog now contains a beta notice to avoid misunderstanding.

User block development

• Fixed an issue where adding a user block to a workflow created duplicates of this block in the workflow library.
• Fixed an issue where you could not copy and paste a user block in a workflow if the block contains a large amount of code or embedded data.

Bugfixes

• Fixed slowdown issues, which could occur when running a workflow where a Composite block receives or outputs a large data array - for example, a 100×100 matrix.
• Fixed the inability to save a Python script block that contains a very long script.
• Fixed an issue with the stock app UI in AppsHub where the UI reported a timeout error if an app run took longer than 10 minutes.
• Fixed a bug due to which a block in a running workflow failed to start if you have copied this block from another workflow.
• Fixed an issue where pSeven sometimes was not showing notifications until you refresh its tab in your browser.

v2021.03

Updates

• Convenience improvements in the Links and Autolink dialogs:
  • Quicker linking for ports with matching names. Ctrl-click and drag when creating a link to instantly open the Autolink dialog. In this case, OK in the Autolink dialog links selected port pairs and closes the dialog.
  • Switch between autolinking and manual linking. In the Links dialog, click Autolink to open the simplified dialog. In this case, OK in the Autolink dialog selects the links to create and switches back to Links, where you can continue with manual link editing.
• The Python script block now automatically formats the script code to comply with PEP 8 and checks the code for syntax errors, when you click Apply or Save in the block's configuration dialog.
• OpenTURNS support. You can now use the openturns module in Python script blocks and user blocks implemented in Python.
• The Block properties pane now shows inputs and outputs sorted in the order defined by the block. You can also switch to alphabetical order using the sorting buttons in the pane. For grouped ports, alphabetical sort order applies to port names in a group but does not change the order of groups.
• When a message receive timeout is exceeded, the api.message_pop() REST API method in the Python script block now raises a TimeoutError instead of a generic exception. If you use messaging in Python script, it is recommended to update your scripts to specifically handle the TimeoutError from api.message_pop().
• Added logging for block editing operations. When you add, reconfigure, or delete a block, pSeven writes debug information to edit log.txt in the workflow folder.

User block development

• User blocks can now access the Redis database that provides the REST API messaging functionality. This makes it possible to add your custom messaging implementation to a user block, functionally similar to the api.message_push() and api.message_pop() methods available in Python script blocks. The database hostname and port are available to blocks as the DA__P7__BLOCK__REDIS_HOST and DA__P7__BLOCK__REDIS_PORT environment variables.

Bugfixes

• Fixed various issues with incorrect run order and statuses displayed on the Start tab.
• Fixed an AppsHub issue where you could not edit some parameters in the stock app UI.
• Fixed an issue with the stock app UI where an app could not start if there is a parameter that accepts any data type.
• Fixed a vulnerability issue in the messaging mechanism, which is in particular used by the Python script block REST API implementation. Before the fix, exploiting this vulnerability allowed a user to access messages of Python script blocks from workflows of other users, and to access messages used internally by pSeven. This could cause data loss in workflows and general malfunctioning. The fix properly isolates messaging for every user.
• Fixed a bug in the Python script block REST API, due to which api.message_pop() returned a bytearray instead of a string.
• Fixed an issue where pSeven could show a failed workflow run as active (running) indefinitely if the run fails to start because of a block start-up error.
• Fixed a similar issue where pSeven could not handle a block error when you open the block to edit its configuration.
• Fixed pSeven showing a workflow run as active while it has failed due to an internal error. Now pSeven automatically cancels runs that stopped responding and shows them as failed.
• Fixed a similar issue where pSeven did not properly stop a block if there was an error while editing its configuration. Now pSeven automatically stops a block running in the edit mode if it becomes unresponsive.

Compatibility issues

• Since api.message_pop() in a Python script block now returns a string as intended (was bytearray), you may be required to update your scripts to correctly handle the return value, depending on how it is processed.

v2021.02

Updates

• Enabled the intended functionality of the Files folder in a workflow. This special folder is a prototype of the workflow's working directory:
  • When you configure block's working directories, pSeven creates folders in Files - so you can preview the working directory structure while editing a workflow. When you move, rename, or reconfigure blocks, pSeven automatically updates the folder structure in Files.
  • You can upload data to Files and to folders inside it to make this data available to blocks during a workflow run. When a workflow starts, pSeven copies the files and folders from Files to the workflow run directory, creating the initial state of block working directories.
  • Running blocks work with copies of files in the run directory, so you can continue editing the workflow and Files as needed, even while a workflow run is active.
  • Note that if a block's working directory type is "indexed", the block creates a new (numbered) working directory every time it starts. These directories have a single prototype in Files - a folder with the [####] name suffix. When you run a workflow, pSeven copies this folder's contents to every new working directory created by the block.
  • To avoid needless copying, place read-only files somewhere outside of the Files folder.
• Simplified behavior of the block linking dialogs. The Autolink dialog no longer opens when you create a link, and you can switch to autolinking with the Autolink button in the Links dialog. Creating or editing a link now always opens the full Links dialog.
• The Python script block now validates names of variables, functions, and arguments to avoid syntax issues and name conflicts.
• Added a command to download the run log file, available in the Run logs pane menu.
• REST API clients can now download workflow run logs. To get a download URL, request run details and read logfile_url from the response.

User block development

• Fixed the inability to set a port value if its data type is a custom table with predefined columns.

Bugfixes

• Fixed licensing issues, due to which any user could start an unlimited number of workflow runs or calculations in AppsHub, potentially consuming all resources and disrupting other user's activity.
• Fixed a bug with messaging support in AppsHub, due to which an app did not work if its workflow contained a Python script block that uses messaging (the api.message_push() and api.message_pop() REST API methods).
• Fixed an AppsHub issue where you could not open the Edit value dialog for an app parameter that currently has an invalid value.
• Fixed an AppsHub issue where an app could not start due to an error in processing a port that is both linked and selected as a parameter.
• Fixed a GUI error that sometimes occurred when deleting a block from a workflow.

Compatibility issues

• Due to bugfixes in the license mechanism, this version requires an updated license valid for v2021.02 or above.

v2021.01

Initial release.