Skip to content

Changelog

v2024.03

Updates

  • Design space exploration block (beta):

    • 3088 The block now supports the evaluation history and intermediate results files (History.csv and Designs.csv) for the space-filling DoE techniques (LHS, for example) and Gradient-free optimization.

    • 3088 Enabled the batch size setting for the space-filling DoE techniques. If you run DoE in batch mode and set the Maximum batch size option, the block splits the generated DoE in a number of batches and outputs them one by one. If you enable batch mode but leave Maximum batch size default, the full DoE is output in a single batch.

      LHS and other DoE techniques recognize the Maximum batch size setting

  • 3084 Slightly improved the responsiveness of the Studio UI.

  • 3094 Upgraded the embedded pSeven Core package to v2024.02. For details on recent improvements and changes, see the pSeven Core changelog.
  • Updates in Help:
    • 2723 Updated the Windows extension node deployment and upgrade guides, removed the requirement to reboot the node after installation.
    • 3088 Updated the Design space exploration block guide due to changes in this version: Result files section, Maximum batch size option description.
Avoid using environment variables prefixed DA__P7__ in Python script blocks

Due to a bug in the Python script block, it is currently possible to read values of block runtime environment variables in the block's script. Those variables (with names starting in DA__P7__) are actually intended for block developers only. This bug will be fixed in future, thus the DA__P7__ variables will no longer be accessible from the user script in Python script. If you are currently using such variables in your scripts, update them beforehand to avoid future issues.

Deployment and administration

  • 722 In the deployments that run blocks natively on Kubernetes (sizing.runOnKubernetes enabled, experimental mode), you can set up network policies for blocks, using the new parameters in values.yaml: blocks.networkPolicySpecIngress for the incoming, and blocks.networkPolicySpecEgress for the outgoing connections.

    You can set restrictive network policies for blocks without worrying about their basic communication with pSeven Enterprise. Internally, your policies are always combined with the built-in system policies, which secure the normal block functions.

    Examples

    Default rules allow all connections to and from a running block:

    blocks:
    networkPolicySpecEgress:
        # Allow all outgoing connections.
        - to:
            - ipBlock:
                cidr: 0.0.0.0/0
    networkPolicySpecIngress:
        # Allow all incoming connections.
        - to:
            - ipBlock:
                cidr: 0.0.0.0/0

    Restrict the block's access to internal Kubernetes networks with a rule like the following (the actual network addresses may be different in your cluster):

    blocks:
    networkPolicySpecEgress:
        # Allow all outgoing traffic but block connections to the
        # Kubernetes internal networks.
        - to:
            - ipBlock:
                cidr: 0.0.0.0/0
                except:
                    # Pod network
                    - 10.42.0.0/16
                    # Service network
                    - 10.43.0.0/16

    Allow blocks to connect to a specific service in the cluster:

    blocks:
    networkPolicySpecEgress:
        # Allow all outgoing traffic but block connections to the
        # Kubernetes internal networks.
        - to:
            - ipBlock:
                cidr: 0.0.0.0/0
                except:
                    # Pod network
                    - 10.42.0.0/16
                    # Service network
                    - 10.43.0.0/16
        # Unblock outgoing connections to Pods labeled "app=redis" in
        # the namespace "common".
        - to:
            - podSelector:
                matchLabels:
                    app: redis
              namespaceSelector:
                matchLabels:
                    kubernetes.io/metadata.name: common

  • 3114 Fixed a vulnerability issue in the REST API where it was possible to list contents of directories located outside the (workflow, app) run directory, and to download files from those directories using the REST API.

  • 2952 Fixed a vulnerability issue where certain system-level environment variables were available in the block runtime environment, so user blocks could exploit them to gain unauthorized access to user or system data.

User block development

  • 3105 Your block can identify the user who edits or runs it: added the DA__P7__BLOCK__USERNAME and DA__P7__BLOCK__USERID runtime environment variables. The DA__P7__BLOCK__USERNAME contains the current name of the user; note that deployment admins can rename users, as well as allow users to rename themselves. The DA__P7__BLOCK__USERID variable contains a persistent string identifier of the user, which cannot change in an existing deployment.

  • 2952 Fixed an issue with the DA__P7__* environment variables where the block runtime environment provided access to a number of system-level variables, which must not be available to the block - for example, DA__P7__APP__* variables or DA__P7__CLUSTER__* variables.

    This fix might break compatibility with existing user blocks

    It is recommended to review existing user block implementations, as they might currently use some of the system-level variables that are removed by this fix.

    • DA__P7__APP__APPDATA_DIR, DA__P7__APP__USER_HOME_DIR, and DA__P7__CLUSTER__TASK_USERNAME are still available to user blocks, despite they are system-level variables. This is a temporary compatibility measure only - these variables will be removed in future versions. Existing blocks should be updated as soon as possible to stop using these variables.
    • The DA__P7__BLOCK__* and DA__P7__RUNENV__* are kept - these are block-level variables.
    • All other DA__P7__* variables are now removed from the block runtime environment.

    For maximum compatibility with pSeven Enterprise version upgrades, blocks should use only the supported block-level environment variables. You can find the full list of supported variables in the startup scripts (start.sh, start.bat) in the example block implementation (Example user block.p7protoblock).

Bugfixes

  • Design space exploration block (beta):

    • 3088 Fixed an issue with writing string categorical variables to Designs.csv where their actual values were replaced with numeric codes.
    • 3088 Fixed an issue with Designs.csv where it could be inconsistent with the result data output to block ports.
    • 3088 3094 Fixed several issues with the optimization and Adaptive design results where the result contained incorrect values of linear and quadratic responses, if you have defined any maximization type responses, or categorical variables, or constants.
    • 3088 3094 Fixed an issue where certain values of linear and quadratic responses, which were actually requested by the block, were missing from its results.

  • Uncertainty quantification block (beta):

    • 2985 Fixed the "Could not save the block" error sometimes appearing on save.
    • 2990 Fixed an issue where the block did not display a warning, if you close its window with unsaved changes.

  • 3114 Fixed an issue with the REST API where an invalid download request returned a response with status code 500 Internal Server Error instead of a proper error code (for example, 404 Not Found if the requested file does not exist, 400 Bad Request if the file was not specified).

  • 1571 Fixed an issue with renaming ports of Composite blocks where it was possible to use certain special characters in port names, which could lead to various errors further.
  • 3138 Fixed a UI issue in Studio with disappearing breadcrumbs in the workflow editor.

v2024.02

Updates

  • 1612 App versions support in AppsHub.

    Set app version when publishing your workflow from Studio

    Select the app version to use in AppsHub

    App version menu

    The stock app UI displays the app version with the app name

  • 1612 App REST API clients can specify the app version to use - add @X (X is the version number) to the app id, for example: appshub/.rest/v2/apps/524e0442e15446fb9f5863c55cb11ad0@3/ to use version 3 (the version-specific API URL). Specifying the version number is not required: the URL without the version suffix (the default API URL) connects to the default app version selected by the app developer. Note that the app developer can freely change the version designated as the default.

  • 2964 The stock app UI can display a dynamic report while the app runs. It reads the report contents from the REPORT.html file in the run directory, and your app's workflow should write and update REPORT.html to provide the report.

    A possible use of report in an optimization app

  • Design space exploration block (beta):

    • 2852 Removed the Unsolvable problem behavior option, which was available for the optimization and Adaptive design techniques, as the default behavior is preferable in the known use cases.

      Default behavior in unsolvable problems

      If the Design space exploration block determines that the current problem is infeasible or cannot be solved, it stops but the workflow run continues, that is:

      • The block stops normally without raising an error.
      • The block outputs all collected data to the the All designs port. It might not output anything to the other result ports - for example, if the problem is infeasible, there is no data to output to Feasible designs.
      • The workflow can continue, though it must handle the above situation, when the Design space exploration block outputs All designs only.

      The behavior described here is now the only supported one for unsolvable problems.

  • 3087 It is again possible to get an app thumbnail via REST API: added the "thumbnail" key in the app list and in the app details, which contains the thumbnail image URL (or null if the app has no thumbnail).

    This update resolves the issue introduced by v2023.02, which removed the "image" key from app details. Note that "thumbnail" provides the image URL only, while "image" contained the Base64 encoded image itself (which added unwanted load to the API).

    GET appshub/.rest/v2/apps/ or appshub/.rest/v2/apps/{app ID}/ and find "thumbnail" 
    [
    ...
    {
        "name": "Disk optimization",
        "path": "/Apps/Disk optimization.p7wf",
        "id": "e9710699359f4f968df887ab6b5f335d",
        ...
        "thumbnail": "https://app.pseven.io/appshub/.rest/v2/apps/e9710699359f4f968df887ab6b5f335d/thumbnail/"
    }
    ...
]
    • For testing purposes, the "thumbnail" key is also added to workflow details. There it always contains null since workflows have no actual thumbnails.
    • For consistency, the "thumbnail" key is also added to the workflow publish POST request, and the "image" key in that request is now deprecated. See Workflow publishing.

  • 2161 If your deployment admin has enabled run-time resource management, and pSeven Enterprise rejects running your workflow due to a lack of resources, the run log now contains more information about the required resources and reasons why the run was rejected.

  • 3076 Upgraded the embedded pSeven Core package to version 6.52. For details on recent improvements and changes, see the pSeven Core changelog.
  • Updates in Help:
    • 3063 Using app reports and working with app versions in Publishing the workflow as an app.
    • 3087 Setting the app version and adding the app thumbnail when you publish a workflow using the REST API.
    • 2992 Added missing notes about the intermediate result and evaluation history support in the Design space exploration (beta) block: both features are currently supported only for Gradient-based optimization, Surrogate-based optimization, and Adaptive design.
    • 3087 2172 Added the missing details about AppsHub REST API changes introduced by v2023.02 - see the v2023.02 changelog.
Avoid using environment variables prefixed DA__P7__ in Python script blocks

Due to a bug in the Python script block, it is currently possible to read values of block runtime environment variables in the block's script. Those variables (with names starting in DA__P7__) are actually intended for block developers only. This bug will be fixed in future, thus the DA__P7__ variables will no longer be accessible from the user script in Python script. If you are currently using such variables in your scripts, update them beforehand to avoid future issues.

Deployment and administration

  • 2888 pSeven Enterprise now enables admin event logging by default - switches on Save Events in Admin Events Settings of the authentication service (Keycloak).

  • 1612 If your update stops with a message indicating errors in AppsHub, you have the option to re-run helm install with --set fixApps=true for an automatic fix. This is a worst case option.

    The automatic fix can delete apps forever

    The automatic fix is your last option as it cannot repair apps - it only deletes the apps that cause errors. You cannot restore the deleted apps. Always check the error details before using that fix: the error might be caused by incorrect deployment configuration or by incorrectly restoring from a backup, and the automatic fix does not help in such cases.

Bugfixes

  • Design space exploration block (beta):

    • 2214 Fixed an issue with the Gradient-free optimization technique where the Optimal designs port output all feasible designs, including the non-optimal feasible designs.

    • 2214 Fixed an issue where the block sometimes did not display the "Auto" budget estimate for a technique that supports this feature.

      Note that the Gradient-free optimization technique does not yet support the "Auto" budget estimates.

    • 2899 Fixed an issue where, if you set a string value as the initial guess for a numeric variable, the block did not warn you about this invalid setting.

  • 2618 Fixed an issue with editing nested lists and dictionaries, where the Edit value dialog unexpectedly switched to the JSON editing mode.

  • 2313 Fixed an issue with the table, list, and dictionary editors where you could not save the value after changing the type of some element (column).
  • 2507 Fixed an issue where, if you have a workflow open in a background tab, recent changes in that workflow were not displayed when you bring that tab up - for example, you could open a workflow with references in background, then break a reference by deleting the related workflow and be unaware of it, when you return to the main workflow tab.
  • 2946 Fixed an issue with run log caching, due to which the current run log displayed in the Run log pane contained random messages from the older run logs of the same workflow.
  • 3091 Fixed a UI issue in the block setup panes where the buttons (, and other) got stuck in a transitional state (the settings they control were applied correctly though).
  • 2754 Fixed a minor UI issue in AppsHub where the recently deleted runs were listed in dialogs confirming mass operations - for example, when you select several runs and stop or delete them.
  • 2156 Fixed the order of buttons in a few UI dialogs where it was inconsistent with the rest of the UI.

v2024.01

Updates

Block updates required

pSeven Enterprise v2024.01 requires updating the following blocks in your existing workflows:

  • Design space exploration (beta)
  • Uncertainty quantification (beta)
  • Text (beta)

In v2024.01, this update is mandatory: old block versions will raise errors. To update blocks in a workflow, open the block library panel and click the update notifications in the workflow library.

  • 2992 Track optimization and adaptive design progress in the Design space exploration (beta) block: while running, the block writes the evaluation history and intermediate results to the History.csv and Designs.csv files located in its working directory. For details, see Result files in the block guide.

    These features are currently supported only for the Gradient-based optimization, Surrogate-based optimization, and Adaptive design techniques. Other techniques do not produce the History.csv and Designs.csv files.

    Designs.csv example 
    x.r2   , x.r3   , x.r4   , f.Mass  , c.Stress , feasible, optimal, initial, description
0.123  , 0.154  , 0.198  , 10.17492, 530.22611, 0       , 0      , 1      , "initial infeasible"
0.12625, 0.15625, 0.19375, 12.48462, 480.60218, 1       , 0      , 1      , "initial feasible"
0.12   , 0.15   , 0.2    , 7.07893 , 1292.3499, 0       , 0      , 1      , "initial infeasible"
0.13856, 0.15   , 0.2    , 13.6342 , 499.4409 , 1       , 0      , 0      , feasible
0.12   , 0.16777, 0.2    , 13.92028, 516.75395, 0       , 0      , 0      , infeasible
0.1305 , 0.15506, 0.19178, 12.08644, 467.51349, 1       , 1      , 0      , "feasible optimal(?)"
0.14   , 0.168  , 0.17   , 32.56638, NaN      , 0       , 0      , 0      , infeasible

  • 2992 The Design space exploration (beta) and Uncertainty quantification (beta) blocks also save their final results in the CSV format in addition to the pSeven Enterprise table (.p7table) format.

  • 3029 The Design space exploration (beta) block received an update to the batch mode implementation in the Surrogate-based optimization and Adaptive design techniques. When the said techniques request evaluations of the generic (expensive) responses, they aim to fill the batch up to the maximum size you have specified, which in the vast majority of cases provides full batches on every such request.

  • Text block (beta):

    • 2995 Added the support for input and output files located in subfolders of the block's working directory - you can specify input and output file paths relative to the working directory root. The names of ports that specify the input and output files were changed to Input file path and Output file path accordingly.
    File path ports in the Text block

    To apply this update to Text blocks in existing workflows:

    1. Update the Text block in the workflow library.
    2. Open each Text block in the workflow and click the save button (re-save the block).

  • Python script block:

    • 2978 Updated NumPy to 1.26.2 (was 1.22.3).

      You may need to update your scripts for compatibility with NumPy 1.26.2

      If you are using NumPy in your scripts, note that there are a number of expired deprecations between NumPy 1.22.3 and 1.26.2. Deprecated NumPy code that raised warnings in previous pSeven Enterprise versions will now raise exceptions, if it contains expired deprecations - most notably, the type aliases like np.int or np.float, which were deprecated in 1.20.0 and expired in 1.24.0.

      For full details on expired deprecations, see the NumPy Release notes from 1.22.4 to 1.26.2.

    • 2904 Removed the memory and CPU settings from the block UI because they are available in the Block properties pane in the workflow.

    Avoid using environment variables prefixed DA__P7__ in Python script blocks

    Due to a bug in the Python script block, it is currently possible to read values of block runtime environment variables in the block's script. Those variables (with names starting in DA__P7__) are actually intended for block developers only. This bug will be fixed in future, thus the DA__P7__ variables will no longer be accessible from the user script in Python script. If you are currently using such variables in your scripts, update them beforehand to avoid future issues.

  • 1460 The block library icon on the workflow toolbar now notifies you if block updates are available for the workflow you are editing.

    Updates available (left); all blocks up to date (right)

  • 3027 Upgraded the embedded pSeven Core package to version 6.51. For details on recent improvements and changes, see the pSeven Core changelog.

  • Updated the descriptions of study results on the Design space exploration and Uncertainty quantification block help pages.

Deployment and administration

Warning

pSeven Enterprise v2024.01 updates NumPy from 1.22.3 to 1.26.2 in the embedded Python. This change is important for the user block developers and Python script users and must be announced early - see the pSeven Enterprise upgrade notice below for details.

  • 3022 Improved stability of the experimental mode where blocks run natively on Kubernetes (sizing.runOnKubernetes) for the common case when users do not set resource limits for their blocks.
  • 3018 Fixed an issue where a user workflow run could stop responding if a block in that run crashed while reading its input data.
  • 3064 Fixed an issue with AppsHub where an app run could consume any amount of the memory and CPU resources, because apps ignored the resource limits set for the blocks in their workflows.
  • 3050 Fixed an issue where users could not unpack any archive if you enable the quota support on the NFS storage server but do not set the user storage quotas.
  • 3032 Fixed an issue where blocks in user workflows did not start, if a Pod without labels exists in the Kubernetes cluster where pSeven Enterprise is deployed.
  • Added the missing v2023.09 upgrade notice about the new Helm version 3.10 requirement, leading to the requirement of Kubernetes 1.22 or greater. If you are already using Helm 3.10 or greater, you are safe to ignore that notice.

Warning

The DATADVANCE Registry address changes to registry.pseven.io. The former address docker.pseven.datadvance.net keeps working but will be disabled in 2 months.

User block development

  • 2978 Updated NumPy in the block runtime environment to 1.26.2 (was 1.22.3).

    You may need to update your blocks for compatibility with NumPy 1.26.2

    If your blocks use NumPy, note that there are a number of expired deprecations between NumPy 1.22.3 and 1.26.2. Deprecated NumPy code that raised warnings in previous pSeven Enterprise versions will now raise exceptions, if it contains expired deprecations - most notably, the type aliases like np.int or np.float, which were deprecated in 1.20.0 and expired in 1.24.0.

    For full details on expired deprecations, see the NumPy Release notes from 1.22.4 to 1.26.2.

  • 2882 2004 Fixed a regression issue from v2023.06 where the Linux-only HOME environment variable was set also for blocks running on a Windows extension node, which caused errors in certain third-party engineering software (CAD/CAE packages) launched by the block on that node.

  • 3014 Fixed an issue where opening a block configured to run on a Windows extension node displayed an empty block window, if Windows settings on that node specify an incorrect MIME type for JavaScript files.

Bugfixes

  • Design space exploration block (beta):
    • 3067 Fixed an issue with initial samples, where the block assumed that the sample columns always follow a fixed order - variables, objectives, constraints - and thus disregarded the order of responses specified in its configuration.
    • 3057 Fixed the block's help page that described the controlled generation feature, which in fact is not supported by the block.
  • 1918 Fixed an issue where repeatedly using the Undo and Redo commands caused errors.
  • 2980 Fixed an issue with workflow references where a "Workflow reference is outdated" error message could pop up if both the main and the referenced workflows are being edited at the same time.
  • 2880 2902 Fixed an issue where rapid edits in a workflow - for example, massively adding ports to a Composite or fast switching between blocks to configure them - could damage the workflow so you could no longer edit or re-open it.
  • 2776 Fixed an issue with the Run log pane where it did not keep the scroll position when you select different blocks.
  • 2590 2965 Various UI and usability fixes in the Edit value dialog.

pSeven Enterprise upgrade notice

pSeven Enterprise v2024.01 contains a newer version of NumPy - 1.26.2 (previous versions provided NumPy 1.22.3). All Python user code that uses NumPy must be checked for compatibility with NumPy 1.26.2 before the upgrade, including the custom user block code and the user scripts in the Python script blocks. Also, users will be required to update other blocks in the existing workflows after the upgrade to v2024.01.

Before the upgrade to pSeven Enterprise v2024.01:

  • Notify the user block developers about the incoming NumPy update from 1.22.3 to 1.26.2. There is a number of expired deprecations between those NumPy versions. The developers have to check the NumPy Release notes from 1.22.4 to 1.26.2 and remove all expired deprecations from their code.
  • Notify the users about that update too. The users as well have to check their scripts in Python script blocks for expired deprecations, if they use NumPy.

When preparing values.yaml for the upgrade:

  • Make sure that you use the new DATADVANCE Registry address, registry.pseven.io. The former address (docker.pseven.datadvance.net) is still working but will be disabled in 2 months - update early to avoid registry access errors in future.

After the upgrade:

  • Notify the users that they have to update the following blocks in their existing workflows to avoid errors (these blocks use NumPy):

    • Design space exploration (beta)
    • Uncertainty quantification (beta)
    • Text (beta)
    • All custom blocks that require an update for compatibility with NumPy 1.26.2.

v2023.11

Updates

  • 2710 Updated keyboard navigation in the Edit value dialog:

    • Down and Up while editing move to edit next/previous cell.
    • Tab and Shift+Tab while editing move right/left.
    • Enter finishes editing but no longer moves down (replaced by Down).
    Keyboard navigation in the Edit value dialog

  • 2887 A REST API client working with an AppsHub app can now check that app for changes. The new "fingerprint" key in app details provides a unique identifier, which changes with any change in the app.

    GET apps/{app ID}/ and find "fingerprint" 
    {
    "name": "Disk optimization",
    "path": "/Apps/Disk optimization.p7wf",
    ...
    "state": "READY",
    "fingerprint": "0e8e4146f23e18686db40211c502f94c"
}

  • 2938 Upgraded the embedded pSeven Core package to version 6.50. For details on recent improvements and changes, see the pSeven Core changelog.

  • Updated pSeven Enterprise Help:
Avoid using environment variables prefixed DA__P7__ in Python script blocks

Due to a bug in the Python script block, it is currently possible to read values of block runtime environment variables in the block's script. Those variables (with names starting in DA__P7__) are actually intended for block developers only. This bug will be fixed in future, thus the DA__P7__ variables will no longer be accessible from the user script in Python script. If you are currently using such variables in your scripts, update them beforehand to avoid future issues.

Deployment and administration

  • 2909 Added a number of advanced parameters to values.yaml, using which you can set annotations and labels to all Kubernetes objects created by a pSeven Enterprise deployment.
  • 2905 2906 2909 Updated the experimental mode where blocks run natively on Kubernetes (sizing.runOnKubernetes) to improve its stability, as a part of work to prepare it for use in production.

  • 2815 2853 Updated the pSeven Enterprise deployment guide:

    The hardware summary for trial and for production

  • Added the missing v2023.07 upgrade notice about the end of support for PostgreSQL 11 and the new PostgreSQL 12 requirement. If you are already using PostgreSQL 12 or greater, you are safe to ignore that notice.

  • 2969 Fixed a regression issue from v2023.09 where, if you set entrypoint.allowedHosts in values.yaml, no one could sign in to pSeven Enterprise because opening the sign-in page resulted in a "Bad host" error.
  • 2945 Fixed an issue where a web application firewall (WAF) made Python script blocks unusable - users were getting a "HttpErrorResponse" message when trying to save the block or test the script. Note that users will have to update the Python script blocks in their workflows to resolve this issue.
  • 2903 Fixed an issue where a workflow run interrupted by user entered the Stopping state but never actually stopped, and you could not manually stop it in the admin interface.
  • 2761 Fixed a stability issue where users could not open any block for editing, if the htcondormanager service is malfunctioning.

Warning

The DATADVANCE Registry address changes to registry.pseven.io. The former address docker.pseven.datadvance.net keeps working but will be disabled in 3 months.

User block development

  • 2839 2832 Fixed a regression issue from v2022.06, which affected the compatibility of newer pSeven Enterprise versions (v2022.06 and above) with the user blocks developed for the older versions (v2022.05 and below) that work with custom data types. Due to this issue, for example, when editing a port value which is a custom table type, incomprehensible error messages appeared when adding or removing table rows.

Bugfixes

  • 2903 Fixed an issue where, if you stop a workflow run, it entered the Stopping state but never actually stopped.
  • 2967 Fixed an issue with workflow references where, if both the main and the referenced workflows are being edited at the same time, numerous error messages appeared on the main workflow tab, and the runs of the main workflow stopped with an error.
  • 2919 Fixed a "value too long for type character varying(1024)" error when opening a workflow run that failed.
  • 2710 Fixed the Ctrl+V hotkey behavior in lists and dictionaries in the Edit value dialog.
  • 2710 Fixed incorrect behavior of the Edit value dialog when editing nested dictionaries.
  • 2989 Fixed an issue with the Text block where configuring it to run on a Windows node caused a "No module named 'astor'" error when running the workflow.
  • 2693 Fixed a vulnerability issue in the table, list, and dictionary editors in the Edit value dialog.
  • 2429 Fixed an issue where pSeven Enterprise blocked unpacking an archive with high compression rate, because it was incorrectly recognized as a ZIP bomb.
  • 2683 2750 2752 2753 Fixed several UI inconsistencies in the AppsHub Runs pane.
  • Fixed UI issues in Explorer related to moving files in the Files folder:
    • 2028 Fixed drag-and-drop not working inside Files.
    • 2875 Fixed using cut and paste, which seemed to copy the file instead of moving it.

pSeven Enterprise upgrade notice

When upgrading, make sure that you use the new DATADVANCE Registry address, registry.pseven.io. The former address (docker.pseven.datadvance.net) is working but will be disabled in 3 months - update early to avoid registry access errors in future.

v2023.09

Updates

  • 1853 The new Uncertainty quantification (beta) block enables uncertainty simulation and reliability analysis studies in pSeven Enterprise.

    Reliability analysis in the Uncertainty quantification (beta) block

  • 1210 The new Text (beta) block provides for integrating the software, which uses input and output text files, and exchanging data between that software and other blocks in a workflow.

    The Text (beta) block being configured to parse a program output

  • Python script block:

    • 2859 Added the signal @go ports, which you can use to order the execution of Python script blocks in a workflow - see Using signal ports).
  • 2005 The storage usage tooltip in Explorer now lists the space used by your files and AppsHub app run results.
  • 2870 Upgraded the embedded pSeven Core package to version 6.49. For details on recent improvements and changes, see the pSeven Core changelog.
Avoid using environment variables prefixed DA__P7__ in Python script blocks

Due to a bug in the Python script block, it is currently possible to read values of block runtime environment variables in the block's script. Those variables (with names starting in DA__P7__) are actually intended for block developers only. This bug will be fixed in future, thus the DA__P7__ variables will no longer be accessible from the user script in Python script. If you are currently using such variables in your scripts, update them beforehand to avoid future issues.

Deployment and administration

pSeven Enterprise v2023.09 requires Helm 3.10 and Kubernetes 1.22

pSeven Enterprise is no longer compatible with outdated Helm versions and requires at least Helm 3.10, which supports only Kubernetes 1.22 and above. See the pSeven Enterprise upgrade notice below for further details.

  • 2891 Fixed a regression issue from v2023.08 where the deployment stopped with an error, if you use an NFSv3 file storage server.
  • 2849 Fixed a vulnerability issue in the pSeven Enterprise authentication service (Keycloak).
  • Help updates:
    • 2811 Updated and corrected the information about pSeven Enterprise network connections and ports - see the new Network parameters section in the pSeven Enterprise deployment guide.
    • 2810 Added a page explaining the pSeven Enterprise release lifecycle policy - see Release lifecycle.
    • 2812 Added a section to the administration guide that covers information security incidents and related operations - see Information security incidents and operations.

User block development

  • 2847 Fixed an issue with blocks that run on Windows extension nodes and launch the CAD/CAE packages installed on that node, where the launch failed if the package requires reading the user registry hive (HKEY_CURRENT_USER).

Bugfixes

  • Design space exploration block (beta):
    • 2742 Fixed issues with variables set up to switch to constants during a workflow run (using the {variable name}.Constant ports).
    • 2771 Fixed an issue where you had to open the block to reset some parameter mapped to a port, because resetting the value of that port in the Block properties pane did not work as intended.
    • 2771 Fixed an issue with N/A values in results where they could be displayed as empty strings, if the block configuration contains a string categorical variable.
    • 2798 Fixed an issue where the block stopped with an error if any of the result arrays is empty - for example, no feasible points were found.
  • 2768 Fixed a version compatibility issue with workflows created in pSeven Enterprise v2022.05 or earlier where you could not open the workflow runs saved earlier, or the list of workflow runs appeared empty even though saved runs exist - for example, when you get the list of workflow runs using REST API.
  • 1177 Fixed an issue where copying a folder to the same location a few times caused an error.
  • 2850 Fixed an issue where incorrect values of block's peak memory usage were output to the workflow run log - the logged peak value could be less than the average usage, or could decrease during a workflow run.
  • 2759 Fixed an issue with using macOS Safari where file downloads from pSeven Enterprise stopped with an error.
  • 2562 Fixed a minor UI issue where the button sometimes appeared in the list of Composite block inputs or outputs even if you did not select any ports to delete.
  • 2642 Fixed a UI issue in AppsHub where, if you publish the same workflow a few times, the AppsHub main page displayed several app duplicates until you reload that page.
  • 2651 Fixed minor UI issues with long names of app runs in AppsHub.

pSeven Enterprise upgrade notice

Since v2023.09, pSeven Enterprise deployment requires Helm 3.10 or greater. It supports only Kubernetes versions 1.22 and above, so Kubernetes 1.22 is now also the minimum version supported by pSeven Enterprise.

If you are already using Helm 3.10 or greater, no additional actions are required.

If you are using an older version of Helm, update it before deploying pSeven Enterprise v2023.09 - otherwise the deployment is going to fail with an error from Helm. In this case, it is recommended to use the most recent release of Helm compatible with your Kubernetes version - check the Supported Version Skew section in the Helm documentation to find the compatible versions.

v2023.08

Updates

  • 2795 You can automate publishing workflows to AppsHub using the pSeven Enterprise REST API. See Workflow publishing in the REST API guide.
  • 2843 FMI support via FMPy. The fmpy module is available in Python script blocks and user blocks.
  • Design space exploration block (beta):
    • 2743 The Adaptive design for GP technique now supports mixed designs with all types of variables.
  • While block:
    • 2680 No longer in beta. No new issues have been discovered during the extended stability testing period of the block, so its current version is considered stable.
  • 2830 An update for pSeven Enterprise deployments with run resource management enabled (optional feature, disabled by default): you are no longer required to set the memory and CPU limits for blocks, so it is possible to grant a block unlimited resources. However, blocks with unlimited resources have lower priority and may be forced to stop by pSeven Enterprise, if there are not enough free resources to run other blocks that do declare their memory and CPU requirements.
  • 2860 Upgraded the embedded pSeven Core package to version 6.48. For details on recent improvements and changes, see the pSeven Core changelog.
Avoid using environment variables prefixed DA__P7__ in Python script blocks

Due to a bug in the Python script block, it is currently possible to read values of block runtime environment variables in the block's script. Those variables (with names starting in DA__P7__) are actually intended for block developers only. This bug will be fixed in future, thus the DA__P7__ variables will no longer be accessible from the user script in Python script. If you are currently using such variables in your scripts, update them beforehand to avoid future issues.

Deployment and administration

  • 2841 The "on-demand" block execution strategy is now default for new deployments, as it appears to be the preferred configuration choice in most of the known use-cases. The "all-at-once" strategy is now an option, supported for backwards compatibility.
  • 2830 Added an experimental feature to run workflow blocks natively on Kubernetes. When enabled, pSeven Enterprise creates a Pod for each running block instance, instead of placing blocks into slots provided by task executors. To enable, see the sizing.runOnKubernetes parameter in values.yaml.
  • 2206 Fixed an extension node deployment issue where the deployment stopped with a "Failed to install C++ redistributable" error, if a Microsoft Visual C++ Redistributable package is already installed on the node.
  • 1216 Fixed a security issue where a block created by a user could gain access to files of other users.
  • 2821 Fixed an issue where pSeven Enterprise could not delete some of the temporary files while cleaning up the user temporary files during a version upgrade.
  • 1648 Updated the pSeven Enterprise deployment guide to explain how to get the full version number from the Helm chart.

User block development

  • 2843 The FMPy module is now available in the block runtime environment by default, so blocks that work with FMUs can simply import fmpy.
  • 2832 Fixed a regression issue from v2022.06, which affected the compatibility of newer pSeven Enterprise versions (v2022.06 and above) with the user blocks developed for the older versions (v2022.05 and below). Due to this issue, for example, certain valid port values in those blocks - such as empty tables - were marked as invalid by pSeven Enterprise and caused errors. This issue appeared because pSeven Enterprise v2022.06 and above use the new value format, which was developed to be compatible with the old value format used in v2022.05 and below, but in fact did not provide the compatibility in full due to a few defects in implementation. Since the previously developed blocks use the old value format, those incompatibilities between the old and new formats were causing various issues with port values in those blocks.

Bugfixes

  • Design space exploration block (beta):
    • 2860 Fixed a regression issue from v2023.07 where the block could stop with an error while processing the results of Surrogate-based optimization or Adaptive design.
  • 2861 Fixed an issue which caused minute-long delays in response to certain workflow editing actions - for example, when setting up links, moving blocks, or viewing their properties.
  • 2856 Fixed an issue where the Workflow contents pane displayed incorrect values of peak memory usage, because the displayed peak value could decrease during a workflow run.
  • [2836] Fixed an issue where pSeven Enterprise displayed errors when you try to edit a workflow, if that workflow is referenced from another.
  • 2821 Fixed an issue where pSeven Enterprise could not remove some of the temporary files created while running workflows, so part of your storage space was occupied by those leftover files.

pSeven Enterprise upgrade notice

Since v2023.08, the "on-demand" block execution strategy is preferred over the "all-at-once" strategy (previous default). In new deployments, the default strategy is now "on-demand". If you are currently using the "all-at-once" strategy, consider switching to "on-demand" during the upgrade: see the sizing.blockRunStrategy parameter in the deployment configuration file values.yaml.

v2023.07

Updates

  • 2558 If you select a workflow block that can be updated to a newer block version, the Block properties pane displays an update notification.

    The Block properties pane with an outdated block selected

  • Design space exploration block (beta):

    • 2204 Updated the block UI to replace its old built-in version of the Edit value dialog with the current version used in pSeven Enterprise Studio.
  • 2816 Upgraded the embedded pSeven Core package to version 6.47. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

pSeven Enterprise v2023.07 requires PostgreSQL 12

pSeven Enterprise is no longer compatible with PostgreSQL 11 as this version of PostgreSQL is at the end of life. If you are using PostgreSQL 11, update to version 12 or 13 before you upgrade pSeven Enterprise to v2023.07. Updating from PostgreSQL 11 to 14 or greater is not recommended because of the changes in the authentication schema introduced by PostgreSQL 14.

  • 2690 If your deployment does not include Windows extension nodes (see Extension nodes), you can slightly reduce its resource consumption by disabling the pSeven Enterprise services that handle connections to the extension nodes: see the new sizing.enableExtensionNodes parameter in the deployment configuration file values.yaml.
  • 1188 Fixed an issue with Windows extension nodes where a node could not connect to other computers in its network - for example, license servers required by the software installed on that node - because all node's network traffic was redirected to pSeven Enterprise.
  • 2794 Updated the Administration guide with a note describing the logging subsystem with changes introduced in v2023.05, explaining the log cache and providing some guidelines about PostgreSQL database maintenance - see Understanding run logs storage.
  • 2346 The Open-source components page provides a list of third-party open-source software components used in pSeven Enterprise, their names, links to source, licenses, and copyright notices.

User block development

  • 2573 Ports with custom types can now specify the placeholder displayed to users when there is no value assigned to the port. To add it, set the @nullStr property in the port schema. For example, {allOf: [{$ref: "r"}, {"@nullStr": "Auto"}]} sets up a port that expects a floating point value and displays Auto when no value is set.
  • 2637 Fixed an issue with input ports of enumeration type, where users could not change the value assigned to such a port in the Block properties pane.

Bugfixes

  • 2731 Fixed an issue with workflow run launch where, if the number of ongoing runs has reached the license limit, pSeven Enterprise canceled every new run with an error. Now such runs are placed in a queue, and pSeven Enterprise starts them automatically when a license is available.
  • 2796 Fixed an issue with the Run log pane where leading whitespaces were removed from every log line, breaking indentation in log messages.
  • 2808 Fixed an issue where a workflow run stopped with an error if the workflow contains a driver block that is set up to run on a Windows extension node.
  • 1236 Fixed an issue with blocks running on a Windows extension node where string values of block runtime environment variables were encoded incorrectly if they contain Unicode characters.
  • 2767 Fixed an issue with workflows created in previous versions of pSeven Enterprise (v2023.05 and below) where, if you have used the Undo command while editing such a workflow in newer pSeven Enterprise version, that workflow could be saved in an incorrect state and become unusable, and reopening it resulted in a "Document initialization error" message.
  • 2645 Fixed an issue where a block description in the Block properties pane could be reset when you save a block.
  • 2279 Fixed minor UI issues with the Mail notifications checkbox in the user menu.

pSeven Enterprise upgrade notice

Previous versions of pSeven Enterprise are compatible with PostgreSQL 11 or greater. pSeven Enterprise v2023.07 is no longer compatible with PostgreSQL 11 and requires at least PostgreSQL 12 (version 11 is at the end of life).

If you are using PostgreSQL 12 or greater, no additional actions are required.

If you are using PostgreSQL 11, update it to version 12 or 13 before you upgrade pSeven Enterprise to v2023.07. Note that updating to PostgreSQL 14 or greater is not recommended in this case because of the changes in the authentication schema introduced by PostgreSQL 14.

v2023.06

Updates

  • 2071 Added a shortcut to preview a workflow app UI - click in the workflow edit toolbar or select the Open app UI command from the toolbar menu. As before, you can also preview the app UI by opening the index.html file from the workflow UI folder in Explorer.

    The app UI preview command in the edit toolbar menu

  • 2484 You can edit multiple workflow references at once: select the reference blocks you want to edit and use the Reference another workflow... command from the menu in the workflow edit toolbar. After you confirm the new workflow choice in the workflow reference dialog, all blocks you had selected will reference that new workflow. This might be useful, for example, when you need to switch to an updated version of a child workflow, which is referenced by several blocks in the main workflow.

    The workflow reference command in the edit toolbar menu

  • 2689 Upgraded the embedded pSeven Core package to version 6.46. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

  • 2727 Fixed a vulnerability issue where a disabled user account could gain API access to pSeven Enterprise.
  • 2595 2778 Fixed a number of CVE vulnerabilities in the pSeven Enterprise images.
  • 2513 Fixed an issue with Windows extension nodes where a node could not establish connection with pSeven Enterprise and hence was not displayed to users in pSeven Enterprise Studio, if that node was allocated to the same subnet as the Kubernetes cluster nodes.
  • 2678 Fixed an issue where the pSeven Enterprise images contained test files used in vulnerability and stability testing routines, which were recognized as security threats by protection utilities.
  • 1898 Cleaned up the administrative interface, removing various unused and disabled buttons and commands.

  • 1327 Added initial WebDAV support: users can mount their pSeven Enterprise home folders as WebDAV shares, provided that they have WebDAV access enabled.

    WebDAV support is a beta feature

    WebDAV access is a beta feature, which is currently disabled by default for all users. To enable it, set the app_auth | user | Can use WebDAV protocol user permission. Users with that permission can get the access URL using the Copy HTTP/WebDAV URL command from the Explorer pane menu.

  • 2548 Added a note about the WebSocket protocol requirement to the Reverse proxy configuration notes in the pSeven Enterprise deployment guide.

User block development

  • 1778 Fixed an issue with the example block implementation (Example user block.p7protoblock) which potentially caused instability when running workflows that include blocks based on that example: there was no timeout and no retries after sending a request from the block to pSeven Enterprise, so if the block did not receive a response (for example, due to network connectivity issues), it could wait indefinitely or raise an error, thus disrupting the workflow run.

    Review your block code as it may require an update

    If you develop and support your own blocks, it is advisable to review their implementation to avoid issues similar to the described above. For guidance, see the usage of the backoff module in the updated version of the user block example.

  • 2004 Fixed an issue where the HOME variable in the block environment was unset, which caused an error if the block had to access the directory pointed by HOME.

Bugfixes

  • 2703 Fixed a regression issue with REST API from v2023.03 where a correct authorization request returned a response with status code 500 Internal Server Error.
  • 1778 Fixed a stability issue with the blocks packaged with pSeven Enterprise, which could reveal itself if the network connectivity within the deployment Kubernetes cluster is disrupted, causing unexpected behavior and various errors when running workflows - for example, a block that never finishes execution, or an unhandled error on a block startup.
  • Design space exploration block (beta):
    • 2714 Fixed an issue with the Adaptive design for GP technique where the block stopped with an irrelevant error, if it received a NaN or ±Inf response value.
    • 2709 Fixed a rare issue where the block could unexpectedly pause at some intermediate solving iteration, causing the workflow run to wait indefinitely.
  • Python script block:
    • 2004 Fixed an issue where the HOME environment variable was unset in the script, which caused an error when using certain modules - for example, skimage.
  • 2762 Fixed an issue where pSeven Enterprise could redirect you to the sign-in page when you open a block or go to AppsHub.
  • 2307 Fixed an issue where you received a 500 Internal Server Error message when opening a workflow app UI preview using a direct link to the index.html file in the workflow UI folder, if that workflow does not exist or belongs to another user.
  • 2586 Fixed misleading error messages appearing in the workflow run log when a block was shut down due to exceeding its memory usage limit.

v2023.05

Updates

  • 2581 You can set up workflow run logging to keep the full log file, regardless of its size (default), or to keep only a certain amount of recent logs, truncating the log file from the beginning once its size exceeds the limit you set. Log truncation might be useful in workflows that write extensive logs or run for an extended time, resulting in a large log. This is a per-workflow, per-run setting: you can enable or disable it independently for each workflow in the Run setup pane, prior to starting the run.

    The log truncation setting in the Run setup pane

Deployment and administration

  • 2581 pSeven Enterprise v2023.05 increases the PostgreSQL database load, comparing to the previous versions. The increase is due to the update in the logging subsystem, which addresses several issues with workflow run logs. See the pSeven Enterprise upgrade notice below for details.
  • 2462 Fixed a denial of service issue where the appworker* services, which handle user interaction, did not close their connections to the redis service properly, so after a certain time period (typically months) the Redis database connection pool was exhausted and it refused all new connections, leading to malfunction of the appworker* services. This caused the "ERR max number of clients reached" errors to users and finally could render pSeven Enterprise unusable due to inability to edit or run workflows.

Bugfixes

  • 2581 Fixed a number of issues with the Run log pane where the log did not load properly, the "loading..." messages were displayed indefinitely, and common operations with the log (like scrolling or filtering) caused incorrect behavior. Run logs are now pre-processed on access, which improves the Run log pane stability and responsibility while viewing logs but introduces a pre-processing delay, noticeable in certain cases - for example, when you open a workflow run for the first time, or return to a workflow run that you have not opened lately.
  • 2462 Fixed an issue where the "ERR max number of clients reached" errors appeared when editing or running workflows.
  • 2721 Fixed an issue with the Composite block where it entered an extended wait, adding a noticeable delay in workflow execution, if one of its nested blocks had failed to stop properly or did not shut down within timeout.
  • 2721 Fixed an issue where the run log message informing that a block in the workflow did not shut down properly was incorrectly issued as an error (now warning).
  • 2721 Fixed an issue where a misleading error message "None has no attribute 'set_result'" appeared in the run log, if you stop the run manually.
  • 2730 Fixed a rare issue with folder sharing where the file list in Explorer displayed empty after you have shared a folder and then revoked access to it.
  • 2730 Fixed an error when opening a folder with a name containing certain characters - for example, backticks (`).
  • 2721 Fixed an issue with the Python script block that sometimes caused a delay when finishing a workflow run - that is, all blocks in the workflow you run have finished working, but you had to wait a few minutes until the run finally stops.

pSeven Enterprise upgrade notice

In pSeven Enterprise v2023.05, the logging subsystem is updated with a new log pre-processing mechanism, addressing various issues experienced by users when viewing workflow run logs. With this update, each run log opened by a user is pre-processed and loaded to the PostgreSQL database for faster access. The pre-processed log is kept in the database and gets removed if the user does not access that run log for a certain time.

Consequently, v2023.05 increases the PostgreSQL database load and resource consumption, primarily the storage space usage due to temporarily keeping the pre-processed logs. Before you upgrade to v2023.05, it is advisable to review your database server configuration and increase the amount of resources available to the database, mainly the disk space. After you upgrade, monitor the resource usage on the database server to verify that it handle the increased load.

v2023.04

Updates

  • 2592 While block (beta):
    • You can configure the block so it checks the input values against the condition and does not start the loop if the condition is not met. That mode is now default for all new While blocks created in pSeven Enterprise v2023.04 and above. Blocks in existing workflows created in previous versions keep their old behavior, which you can change using the Unconditionally run the first iteration toggle in the block configuration dialog. See the updated While block help for more details.
    • You can also use an input port value as a parameter in the condition - for example, configure the block to check a certain loop response value received at every iteration against a threshold value received to an input port once on start.
    • Updated the block help to more accurately and clearly describe its operation and setup, including its two operation modes and the new mode switch.
  • 2526 For viewing convenience, logs in the Run log pane now use a monospaced font.
  • 2663 Upgraded the embedded pSeven Core package to version 6.44. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

  • 2570 pSeven Enterprise deployment and version upgrade no longer require you to manually copy secrets (the keycloak.clientSecret, jobcluster.*, and extensionNode.* parameters) to the deployment configuration file values.yaml. The secrets are now saved to Kubernetes and loaded during the upgrade.

    This update changes the set of secrets used by pSeven Enterprise. When upgrading an existing deployment, you will have to generate new secrets and apply them once during installation to save the secrets. Future updates will load those saved secrets automatically. See the pSeven Enterprise upgrade notice below for instructions.

  • 2274 To avoid wasting storage space, pSeven Enterprise now cleans up user temporary files during a version upgrade.

  • 2686 Fixed an issue where a failed workflow run entered the Stopping state but never actually stopped, and you could not manually stop it in the admin interface.
  • 2665 Fixed an issue with the Python script block where its implementation contained files that are not required by the block but consume a certain amount of storage space for every instance of the block added to any workflow by users.
  • 2672 Fixed an issue with the SMTP server settings in the deployment configuration file values.yaml, where pSeven Enterprise did not stop the deployment if both the SSL and TLS mail protocols are enabled (those are mutually exclusive) - so you could complete the deployment with such settings, after which pSeven Enterprise did not send email notifications to users.
  • 2655 Fixed an issue where pSeven Enterprise deployment stopped with an error if there is a corrupt workflow run in some user folder.
  • 2302 Fixed an issue where the user-specified CPU limit for a block had no effect if the "on-demand" block execution strategy (experimental feature) is enabled in your deployment.
  • 2649 Updated the Kubernetes node disk space requirements in the pSeven Enterprise deployment guide.
  • 2570 Updated the pSeven Enterprise deployment and version upgrade guides with regard to changes in secrets management.

Bugfixes

  • 2628 Fixed several issues, which caused noticeable UI slowdowns when editing workflows - for example, when adding, renaming or moving blocks. If you still encounter such issues in pSeven Enterprise v2023.04, clear the workflow undo and redo history.
  • 2691 Fixed a regression issue from v2023.02 where renaming a table column in the Edit value dialog caused all data in that column to be lost.
  • 2694 Fixed a regression issue from v2023.02 where you could not get the workflow from an app you have published to AppsHub, if you have disabled that permission for other users when publishing.
  • 547 Fixed an issue with batch mode ports in Composite blocks where you could not assign a List value to the port (a batch of values to process).
  • 2668 Fixed an issue where various file operations in the Explorer pane were taking unexpectedly long time - for example, copying files or calculating the size of a folder.
  • 2686 Fixed an issue where a failed run entered the Stopping state but never actually stopped.
  • 2614 Fixed issues with the Edit value dialog where it was unresponsive when editing big lists or dictionaries.
  • 2318 Fixed an issue where you could create files (folders, workflows) with the same name in different case (for example, result and Result), which led to confusion and possible workflow run errors - for example, in workflows that contain blocks configured to run on Windows extension nodes, where file and folder names are case-insensitive.
  • Design space exploration block (beta):
    • 2663 Fixed an issue with results in optimization and other tasks with minimization or maximization responses where, if several Pareto-equal solutions were found, only one of those solutions was included into the optimal designs set.
    • 2663 Fixed an Adaptive design technique issue with initial samples containing missing response values, where the block evaluated responses for the initial sample points that violate the variable bounds or have invalid variable levels.

pSeven Enterprise upgrade notice

The new secrets management engine introduced in pSeven Enterprise v2023.04 uses a new set of secrets, which you must create and apply during the upgrade. This is required only once, when upgrading from a pre-v2023.04 version to v2023.04 or above.

  1. Before you upgrade, login to your Docker Registry and use the following commands to get the full version number and create the secrets file pseven-secrets.yaml for the new version:

    helm show chart pseven-{YYYY.MM.DD}.tgz | grep appVersion | cut -d" " -f2
docker run --rm {registry}/pseven-secretgen:{version} > pseven-secrets.yaml

    For more information about these commands, see Secrets in the pSeven Enterprise deployment guide.

  2. When setting parameters in the deployment configuration file values.yaml, do not copy the secrets related parameters keycloak.clientSecret, jobcluster.secret, jobcluster.token, and extensionNode.*. Those parameters are no longer used and must be removed from values.yaml.

  3. When installing, ensure that the secrets file pseven-secrets.yaml is in the same directory as values.yaml. To apply the new secrets, add the -f pseven-secrets.yaml option to the helm install command, for example:

    helm install pseven-rl pseven-{YYYY.MM.DD}.tgz -f values.yaml -f pseven-secrets.yaml -n pseven-ns --timeout 60m --wait --debug | tee pseven-{YYYY.MM.DD}.log

    For more information about using this command, see upgrade steps in the Upgrading pSeven Enterprise section.

After completing with the upgrade, keep a backup copy of the secrets file pseven-secrets.yaml you have created.

v2023.03

Updates

  • Several usability improvements related to workflow references:
    • 2483 You can create a workflow reference by dragging a workflow from Explorer and dropping it into the workflow you are editing.
    • 2531 A reference block in a workflow is outlined red, if it references a workflow that no longer exists.
    • 2531 Better handling of paths in the workflow reference dialog: you can use paths relative to your home folder and omit the .p7wf extension when referencing a workflow by path.
  • 1292 You can copy and paste list and dictionary data between the Edit value dialog and Excel or another spreadsheet editor that uses a similar clipboard format (tab-delimited tabular data).
  • 2221 2142 2539 All blocks packaged with pSeven Enterprise now show a warning, if you close a block configuration dialog with unsaved changes.
  • 2239 Cleaned up the error messages printed to the workflow run log when some block stops with an error: moved error details to the debug log, removed extra messages.

Deployment and administration

  • 2436 Fixed a version upgrade issue where the deployment of a new pSeven Enterprise version stopped with an error, if you have changed the keycloak.clientSecret parameter in the deployment configuration file values.yaml during the upgrade.
  • 2540 Fixed an issue where a user who is not authorized to run blocks on some extension node could circumvent that restriction by configuring a Windows-only block to run on any available host.
  • 2575 Updated information about minimum Kubernetes cluster and file storage resource requirements in the Prerequisites section of the pSeven Enterprise deployment guide (moved: see Requirements).
  • 2527 Since pSeven Enterprise is commonly deployed in a dedicated Kubernetes namespace, corresponding instructions in the deployment and admin guides are updated to reflect that.

User block development

  • 2472 Added clarifications to the user block guide about using the DEV tag to make the block run in development mode - in particular, this mode is required to disable the block code cache while testing and debugging a block on a Windows extension node. For details, see Tagging the block under development.
  • 2472 In the block prototype example (Example user block.p7protoblock) packaged with pSeven Enterprise, development mode is now enabled by default (the DEV tag is set).

  • 2564 Fixed a regression issue from v2022.06 where the Edit value dialog did not display the names of custom data types properly.

    The users of your block who have encountered this issue will have to refresh the block configuration saved with their workflows by opening the block window and saving its configuration once again. Note there is no need to make any changes in the current block configuration.

  • 1617 Fixed an issue with blocks that run third-party engineering software (CAD/CAE packages) on Windows extension nodes, where the block could not launch the software package it uses, if that package needs to read the APPDATA or USERPROFILE Windows environment variable in order to start.

Bugfixes

  • Design space exploration block (beta):
    • 2613 Fixed an issue with categorical variables in Adaptive Design, which caused an error, if you do not specify the Study target setting.
    • 2613 Fixed an issue with the Orthogonal Array technique, which caused an error, if all design variables are frozen (constant) except one continuous variable.
  • Python script block:
    • 2239 Fixed an issue where the block did not show the source of a runtime error, if that error occurs in an additional Python module, which you import in your script.
    • 1262 Fixed an issue with editing default values of variables having the Any type where string values were not recognized unless you enclose them in double quotes. Also fixed a similar issue where True and False were not recognized as Boolean values.
    • 1262 Fixed an issue an issue with editing default values of variables where the block did not accept a default value you input, if you had changed the type of a variable in its settings.
  • 2405 Fixed an issue with the Edit value dialog where you could only assign scalar values inside tables, lists, and dictionaries - for example, it did not support editing nested lists.
  • 2008 Fixed an issue where pSeven Enterprise Studio UI was becoming generally unresponsive, if you have a few workflows opened in background tabs, and those workflows contain many links or blocks with many ports.
  • 2265 Fixed issues with the Block properties pane, which caused unexpected behavior when editing port values - for example, the port value you input could be reset back to the value that was previously assigned to that port.
  • 2640 Fixed a UI issue with value input fields where, if you input an invalid value and then reset it, the input field got a persistent red highlight.
  • 2546 Fixed an issue where the AppsHub main page was taking too much time to load and looked broken, if you opened AppsHub from a device that has no Internet access, because rendering that page required downloading fonts from Google.
  • 1262 Fixed an AppsHub issue with the stock app UI where it could reset a string parameter value when you open the Edit value dialog for that parameter.
  • 2489 Fixed an AppsHub issue where an app could take a long time to load when you open the app page, if pSeven Enterprise was updated after that app had been published.
  • 2537 Fixed several issues with handling missing (N/A) values and displaying them in the UI.
  • 2305 Fixed an issue with the Block properties and Run setup panes where you could not change the port value if a Boolean value is currently assigned to that port.
  • 2580 Fixed a UI issue with dropdown menus in the Runs pane in AppsHub.
  • 2331 2547 1602 2456 Fixed various minor UI issues.

v2023.02

Updates

  • 1435 You can edit text and Markdown files in pSeven Enterprise - for example, if you want to make a quick edit in a Python script used by your workflow, you no longer have to download the script file to proceed with your edits. To open a file for editing, double-click it in Explorer, then click the Edit button in the file viewer.

    Workflow description opened for editing

  • 2447 Added a more convenient way to change a workflow reference - select the reference block and click the referenced workflow name in the Workflow reference properties pane on the right to open the workflow reference dialog.

    Click the referenced workflow name to change it

  • 1064 When you try to delete a workflow with ongoing runs or a folder containing such runs, you now get an additional warning before deletion to avoid accidentally interrupting and losing a run in progress.

  • Design space exploration block (beta):

    • 2280 For the Surrogate-based optimization and Adaptive design techniques, the block now provides estimates of the total exploration budget and initial sample size. Note that the estimates require a complete problem definition - for example, they may be unavailable, if you map certain properties of variables or responses to input ports and do not set those properties in block configuration window.

      Exploration budget estimates in the Design space exploration block

  • Python script block:

    • 1838 Minor usability improvements in the script testing UI (the Log and Results tabs).

  • 2172 Harmonized the AppsHub and Studio REST APIs (app and workflow APIs) to streamline the development process when you debug your REST client while preparing to publish the app - that is, you debug with the app's workflow, before publishing it. A client tested with a workflow in Studio should work with the app after publishing, with minimum edits in the client code. Generally you will only have to change the base API URL (pseven/.rest/v2/ to appshub/.rest/v2/).

    Related AppsHub REST API changes may break compatibility with existing clients

    Certain changes that make the app and workflow APIs consistent may lead to compatibility issues with existing code using those APIs. Review the details below and update your code accordingly.

    The response to an app details request (GET appshub/.rest/v2/apps/{app ID}/) now has the same structure as the response to a workflow details request (GET pseven/.rest/v2/workflows/{workflow ID}/):

    • Contains "parameters", "results", "runs", and "presets" as top-level keys. In the app API, these were previously available only as sub-keys of the "workflow" key, which is inconvenient when testing the same code with both the app and its source workflow.
      • The old "workflow" key with its sub-keys are kept in the app API for compatibility, though using them is not recommended. They are now deprecated and will be removed in the next API version.
    • The following keys were removed from the app details JSON: "created", "description", "image", "owner", "permissions", "view_count". Workflow details never contained these keys.

    Additionally:

    • Moved the "started", "finished", "logfile_url", and "logs" keys from the app run list to the app run details. Previously these keys were present only in the elements of the run list JSON (the response to GET appshub/.rest/v2/apps/{app ID}/runs/). Now they exist only in the run details (GET appshub/.rest/v2/apps/{app ID}/runs/{run ID}).
    • The "owner" key was removed from the app run list. App owner information is no longer available through the REST API.
    • GET requests to /pseven/.rest/v2 and /appshub/.rest/v2 both return a JSON with the "workflows_url" and "apps_url" keys. If you develop a REST client to an app, you can use the URL from either key - while testing your client with a workflow, and also after you publish the workflow as an app. To enable this, a GET request to either of those URLs always provides the list of items you can currently run, be they apps or workflows. That is, despite the key naming, in the Studio API both URLs return the list of workflows; in the AppsHub API, both return the list of apps.

  • 2473 Upgraded the embedded pSeven Core package to version 6.42. For details on recent improvements and changes, see the pSeven Core changelog.

  • 2475 To avoid page loading issues, pSeven Enterprise Help now uses your local system fonts instead of loading typefaces from Google Fonts, which required Internet access to connect to the Google font servers.

Deployment and administration

  • 1950 Added one more branding option - you can set a custom page title for pSeven Enterprise by editing title.txt in the branding customizations folder (see Branding). Your custom title will be displayed to users in the title of the browser tab where they open pSeven Enterprise, and also on top of the AppsHub main page.

    A custom title in branding

  • 2510 Fixed an issue where pSeven Enterprise incorrectly set owner of new folders and ZIP archives created by users, due to which users could exceed their storage quotas.

  • 2443 Fixed an internal issue with Windows extension node configuration, which could cause errors in user workflows that run blocks on Windows nodes due to incorrect initialization of the block runtime environment on the node.
  • 2443 Fixed mistakes (wrong and missing commands) in the Python installation verification instructions in section Node check of the extension node deployment guide.
  • 2453 Minor updates and clarifications in the pSeven Enterprise deployment, administration, and version upgrade guides, as well as in the Technical support section.

User block development

  • 2479 Fixed a regression issue from v2023.01 where the Block properties pane displayed a red highlight instead of a port value in certain cases: now such values are displayed as invalid and have a tooltip, which explains the reason.
  • 2321 Added the vector data type to the pSeven Enterprise typesystem. A vector is a homogeneous flat array of numeric or string values.

Bugfixes

  • 2486 Fixed an issue with workflow references where pSeven Enterprise did not restore the reference as expected, after you have deleted the referenced workflow and then restored it or created a new one with the same name and location as the deleted one.
  • 2496 Fixed an issue with composite blocks that have a batch input port, which is also added to workflow results (both the and port settings enabled), where a workflow run with such a block stopped with an error once the block received a matrix value to its batch input port.
  • 2172 Fixed inconsistencies of the AppsHub REST API, which caused issues when you need to develop a REST client that uses both the workflow and app REST APIs.
  • 2382 Fixed an issue with the table editor in the Edit value dialog where sometimes you could not save a valid table - for example, after you have entered an invalid cell value and then fixed it.
  • 2051 Fixed issues with behavior of the Run log pane when scrolling logs.
  • 2511 Fixed a UI issue where button and menu command tooltips never disappeared if you work with pSeven Enterprise in Firefox, unless you refresh the pSeven Enterprise tab in the browser.
  • 1955 Fixed a usability issue with the location bar in Explorer where, if you input a full path to a subfolder inside Runs, Files, or another special folder displayed in tree view, Explorer opened that folder but did not expand the tree to the subfolder you have specified in the path.
  • 2475 Fixed an issue where pSeven Enterprise Help pages were taking a few minutes to load, if you opened Help from a device that has no Internet access.
  • 2166 Fixed a minor UI issue with rendering block update notifications in the Block library pane, which were taking too much space if the block name in the pane is long enough.
  • 2381 2382 2478 2396 Fixed various minor UI issues in the Edit value dialog, Block library pane, workflow editor.

v2023.01

Updates

  • 2403 Updated the Runs pane in AppsHub:

    • You can rename completed runs.

    • Runs in the list can be grouped by adding a common prefix to their names - the prefix is used as the run group name. The prefix should be separated from the run name with a dot (.).

      An app runs list with the new and old (archived) groups

    • Updated the run list sorting and filtering options.

  • 2403 Clicking an app thumbnail on the main AppsHub page opens the app info page with a list of app runs.

    The app info page

  • 2108 Pack any of your files into a ZIP archive using the new Zip command from the menu in Explorer to free up some storage space or to spend less time downloading a single large file.

    The Zip command in Explorer

  • 2400 Redesigned the workflow reference dialog for more convenience (see Referencing workflows). You can select a workflow to be referenced from the list, which displays all workflows you have previously opened, and filter that list by typing a part of the workflow name. To select a workflow you have never opened yet, use the button or copy the workflow path into the dialog.

    A list of workflows in the add reference dialog, filtered by test in the name

  • 2072 To reduce inconvenience caused by long port names while setting up links, the Links and Autolink dialogs are now resizeable.

  • 2152 Updated and fixed the dictionary editor in the Edit value dialog.

  • Design space exploration block (beta):

    • core-315 Reworked the handling of discrete variables in the Surrogate-based optimization and Adaptive design technique algorithms, improving result quality and algorithm stability in tasks with discrete variables.

  • Python script block:

    • 2420 If you configure the block to run on a Windows extension node, the comtypes module is now available in the block's script.

  • 2420 Upgraded the embedded pSeven Core package to version 6.41. For details on recent improvements and changes, see the pSeven Core changelog.

  • 1874 Updated section Basic concepts in workflow guides to describe composite block information ports and parallelization settings (see Parallelization settings, Information ports).

Deployment and administration

  • 2175 Fixed a deployment issue where specifying a non-default value to the entrypoint.allowedHosts parameter in the deployment configuration file values.yaml caused pSeven Enterprise malfunction after the deployment.
  • 1336 Fixed a licensing issue where a license expired one day earlier than it should - at the beginning of its expiration date.
  • 1738 Fixed a vulnerability issue where a user could start uploading a large file that exceeds the user's storage quota, consuming most of the free disk space in the user data storage and causing various issues for all pSeven Enterprise users.
  • 2393 Fixed an issue where user data could exceed the user's storage quota - for example, certain files in workflow run results were quota-free.
  • 2263 Fixed an issue with run resource management where a block running on a Windows extension node could exceed its memory usage limit but was not stopped properly, potentially consuming all memory available on the node.
  • Updated the pSeven Enterprise deployment guide:

User block development

  • 2394 Implemented the code cache for blocks intended to run on Windows extension nodes. When a user first opens the configuration dialog of a Windows-only block, pSeven Enterprise copies the protoblock folder to the Windows node that hosts the block, where the protoblock is added to a user-specific local cache. This resolves certain issues previously caused by copying the protoblock over the network every time a user opens the block configuration dialog or runs a workflow containing Windows-only blocks. The protoblock that is cached locally on the node is updated only when required - for example, when pSeven Enterprise detects a newer block version in the user workflow library.

    Disable the code cache during development

    While you develop a block that runs on a Windows extension node, code cache for that block should be disabled because it interferes with testing and debugging the block. Switch your block to development mode by adding the DEV tag in the block's manifest - see section Tagging the block under development for instructions.

  • 2420 Blocks intended to run on Windows extension nodes may now use the comtypes Python module.

  • 2129 Enabled Python print() in the block code for debugging purposes.

Bugfixes

  • 2431 Fixed an issue where you could not start a new workflow run after renaming or deleting a run in progress.
  • 1738 Fixed an issue where you could upload a file larger than the available free storage space, which caused further errors. The file size is now checked before the upload.
  • 2444 Fixed an issue where sometimes you could not set up a link in the Links dialog.
  • 1746 Fixed an issue where you could not remove a workflow from favorites (the Starred list in Explorer) if that workflow is shared to you by another user.
  • 2433 Fixed a UI issue where scroll did not appear in long menus that cannot fit into the browser window.
  • 2426 Fixed an issue with the table editor in the Edit value dialog where pasting clipboard data to the dialog added an empty bottom row to the table.
  • 1570 2298 2373 Fixed various minor UI issues in the block properties pane, matrix value editor, Condition block.

  • Design space exploration block (beta):

    • core-327 p7-386 Fixed a regression issue with the Adaptive design and Adaptive design for GP techniques from v2022.11 where, if the blackbox block that evaluates responses did not support batch evaluations, the workflow stopped with an error once the Design space exploration block sent a request to the blackbox block - even if the Batch mode option is disabled.
    • 2379 Fixed an issue with the Gradient-free optimization technique where the number of linear response evaluations was much higher than expected.
    • core-283 Fixed an issue with linear responses of the Evaluation type where results could contain values of such responses calculated using an internal response model (not real evaluations).
    • 2376 2387 Fixed issues with checkboxes in the Ports and parameters dialog.

  • While block (beta):

    • 2424 Fixed an issue where the Links dialog displayed only driver ports of the While block but not its input and output ports.

v2022.12

Updates

  • 1231 Implement complex projects with interconnected workflows, where a main workflow can be composed of smaller child workflows created and maintained by different teams. The special Workflow reference block enables using any workflow as a block in one or more main workflows, without the need to copy and paste workflow parts or manually update them when the referenced workflow changes. For details, see section Referencing workflows.

    Referencing workflows

  • 2118 Get an overview of all your app runs in AppsHub using the new Runs pane added to the main AppsHub page.

    • The Runs pane is placed on the left and is collapsed by default.
    • Click a run in the pane to open it in a new tab.
    • To delete a run you no longer need, hover it with your mouse cursor and click the button that appears.
    • If you are an app owner, you will see all runs of your app in the Runs pane, including the runs started by other users.
    The Runs overview pane in AppsHub

    Runs pane functions

  • 1935 Use drag-n-drop in the tab bar to place workflow and run tabs in any order convenient for you.

  • 1614 The new While block (beta) enables the creation of processing loops in a workflow without any programming - for example, you can use it to implement a loop with condition (a while loop) or with a given number of iterations (a for loop).

    The While block

  • 1881 Improved startup times of blocks that run on a Windows extension node in the case when that node and pSeven Enterprise are located in different networks.

  • Design space exploration block (beta):

    • 2378 Enabled the batch evaluation mode support for the Gradient-free optimization technique.

    • 2320 In optimization and adaptive design, if you enable batch mode, the Exploration budget setting regards the Maximum batch size option value and limits the total number of batches output for evaluation - that is, the number of the blackbox block startups. Previously, Exploration budget in batch mode limited the total number of response evaluations requested from the blackbox block, disregarding their distribution in batches - which is usually inconvenient in tasks that use batch evaluations.

      This change applies to the following techniques: Gradient-based optimization, Gradient-free optimization, Surrogate-based optimization, and Adaptive design.

  • 2365 Upgraded the embedded pSeven Core package to version 6.40. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

  • 2366 Fixed a regression issue from v2022.10 where pSeven Enterprise was unresponsive for a certain time following the deployment, caused by excessive NFS storage checks.

    As a part of this fix, the NFS protocol version to use (NFSv3 or NFSv4) is now specified in the deployment configuration file values.yaml. For details, see the pSeven Enterprise upgrade notice below.

  • 2324 Fixed an issue where the extension node deployment stopped with an error if a certain user password length is required by Windows security policy on that node.

  • 1881 Blocks running on Windows extension nodes may now create Python cache files (.pyc) on the node, which in some cases makes the block startup noticeably faster. The cache files are created in the %USERPROFILE%\.pseven\pycache directory of the user who runs the block.

User block development

  • 2357 You can get a unique identifier of the current workflow run containing your block by reading the DA__P7__BLOCK__WORKFLOW_RUN_ID environment variable. This identifier may be useful, for example, if you need to track data generated by your block in different workflow runs.

  • 1881 Blocks are now allowed to cache the Python bytecode (create .pyc files upon startup) when running on Windows extension nodes, unless bytecode caching is disabled in the block startup script for Windows start.bat - for example, the -B option is given to the Python interpreter. It is advisable to enable bytecode caching, as this behavior is default for Python, and it also improves the block startup time on an extension node when that node and the pSeven Enterprise deployment are located in different networks.

    Existing user blocks require an upgrade to enable bytecode caching

    The block prototype example (Example user block.p7protoblock) packaged with previous versions of pSeven Enterprise disables the bytecode caching by default - in the Windows startup script start.bat, the -B option is included in the run command:

    %DA__P7__RUNENV__PYTHON3__EXECUTABLE% -B -m userblock %*

    To enable bytecode caching in your blocks when running under Windows, remove the -B option from the start.bat startup script - for example:

    :: %DA__P7__RUNENV__PYTHON3__EXECUTABLE% -B -m userblock %*
%DA__P7__RUNENV__PYTHON3__EXECUTABLE% -m userblock %*

    After that, publish the block update following the instructions in section Publishing block updates and guide the users of your block to upgrade it in their workflows as described in section Upgrading the block in the workflow of the user blocks guide.

Bugfixes

  • 2389 Fixed a regression issue with REST API from v2022.10 where a result port description was an array instead of a string.
  • 847 Fixed an issue with the file open dialog where you could not select a file located inside a shared folder, because it was impossible to open a shared folder in the dialog.
  • 2273 Fixed an issue with the file list filter in the Explorer pane where typing a partial name into the filter had no effect unless you add the * wildcards.
  • 2273 Fixed an issue with the Explorer pane where the file list filter was not reset when you navigate to a different folder, causing confusion.
  • 2317 Fixed an issue with user search in the file sharing dialog where usernames were case-sensitive.
  • 2208 Fixed an AppsHub issue with the stock app UI where it used an old version of the Edit value dialog, which had different design and functionality compared to the current version used in pSeven Enterprise Studio.
  • 2208 2326 Fixed an AppsHub issue where you could not open an app with the stock app UI that has parameters mapped to certain special ports - for example, the ports that set Exploration budget and Study target values in the Design space exploration block.
  • Design space exploration block (beta):
    • 2355 Fixed an issue with editing the Initial sample port value where the Edit value dialog opened in the JSON editing mode instead of a table editor.
    • 2210 Fixed an issue where the block did not write the intermediate result information to the run log while solving a task with categorical variables.
    • 2253 Fixed an issue where you could not set up a vector constraint that has only one bound, unless you explicitly specify that the other bound is infinite (add a +Inf or -Inf value).
    • 2212 Fixed various minor issues in the block UI.

pSeven Enterprise upgrade notice

pSeven Enterprise v2022.12 requires you to specify the NFS protocol version to use when connecting to the file storage NFS servers in the deployment configuration file values.yaml. The default is set to NFSv3, which is the correct setting if your NFS servers are configured as described in section Disabling NFSv4. If you are using NFSv4, or you are going to switch from NFSv3 to NFSv4 during the upgrade to pSeven Enterprise v2022.12, you must specify NFSv4 when preparing a deployment configuration file for the upgrade.

Previously, pSeven Enterprise determined the NFS protocol version automatically, which caused various responsiveness issues as it required testing the NFS connection when communicating with the file storage servers.

If you want to start using NFSv4, enable the NFSv4 protocol on your file storage servers before deploying the v2022.12 upgrade, and run deployment with the -⁠-⁠set fixStorages=true option. For instructions, see Enabling NFSv4 in the pSeven Enterprise upgrade notice for v2022.10.

  1. Identify the NFS protocol version enabled on each of the file storage NFS servers. You can use the test described in section Disabling NFSv4:

    1. Create a test directory on the server and try mounting it using NFSv4.

    2. If a command like the following runs without errors, the server has NFSv4 enabled:

      $ sudo mount -t nfs -o vers=4 localhost:<test directory path> <mountpoint>

    3. If the above command outputs a "Protocol not supported" error, NFSv4 is disabled.

  2. Prepare the updated deployment configuration file values.yaml as described in Upgrading pSeven Enterprise.

  3. In the prepared deployment configuration file, set the following parameters specifying the NFS protocol version enabled on a corresponding file storage server:

    • storage.userdata.nfs.version
    • storage.appdata.nfs.version
    • storage.shareddata.nfs.version

  4. If you have changed the file storage server configuration while preparing for the upgrade (enabled or disabled NFSv4), run the deployment with the storage check enabled, as noted in the upgrade guide: add the -⁠-⁠set fixStorages=true option to the helm install command - see the example in section Upgrading pSeven Enterprise. Note that the storage check may take some time depending on the amount of storage data.

v2022.11

Updates

  • Design space exploration block (beta):

    • 2284 New optimization technique - Gradient-free optimization, which can provide better quality solutions in tasks with many discrete variables. The technique uses the Nevergrad solver by Meta AI Research.

      The Gradient-free optimization technique is not enabled in SmartSelection yet, so you have to select it manually to try it.

    • 2188 If you close a block configuration dialog with unsaved changes, you get a related warning.

  • 1430 Updated the Python runtime environment version to 3.9 and updated versions of pre-installed Python packages. This version of Python is used in Python script blocks, user blocks, and on Windows extension nodes.

  • 760 Updated the top navigation bar in pSeven Enterprise Studio:
    • You can go from Studio to AppsHub using the dropdown menu at the left side of the bar.
    • Added a separate Help menu with links to the Getting started page and changelogs.
  • 2153 Updated and fixed the list editor in the Edit value dialog.
  • 2323 Upgraded the embedded pSeven Core package to version 6.39. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

  • 760 If you are logged into pSeven Enterprise Studio, you can use the new dropdown menu in the top navigation bar at its left to go to the admin interface.
  • 2294 Updated the pSeven Enterprise deployment guide with details on using a reverse proxy to secure the deployment - see sections Reverse proxy, Reverse proxy configuration notes, and updated deployment diagrams.

User block development

  • 2352 Fixed a regression issue from v2022.10 where a user block that does not handle the shutdown request from pSeven Enterprise caused incorrect workflow run behavior (the run never finished). Now, if a block did not stop within 1 minute after receiving the shutdown request, it is unconditionally forced to stop.

    Note that pSeven Enterprise will not allow any block to process the shutdown request longer than 1 minute. When implementing the shutdown request handler, you will have to make sure it can finish its job before that timeout expires.

  • 2266 Fixed an issue where pSeven Enterprise did not set the idle timeout value specified in the block manifest, when you add the block to the workflow.

Bugfixes

  • 2328 Fixed a regression issue from v2022.10 where you could not set workflow parameter values after you have created a run in the "Configuration" state.
  • 2293 Fixed an issue where you were not redirected to the sign-in page after you have logged out.
  • 2220 Fixed an issue with incorrect workflow run behavior after you rename the folder that contains your workflow or the run directory. Now you get a warning dialog when you try to rename a folder that contains a workflow run in progress.
  • 2345 Fixed an issue where adding a .p7history file to a report caused an error.
  • Fixed issues with the Runs sidebar in AppsHub:
    • 2235 If you delete an ongoing app run, it was stopped but not deleted, so you had to click again.
    • 2340 The sidebar did not show all existing app runs (created by any app user) to the app owner.
  • 2288 Fixed an issue with editing Enumeration type values where the Edit value dialog required you to input the value manually instead of selecting it from a predefined list as intended.
  • 2312 2153 Fixed various minor UI issues in the Run setup pane and the Edit value dialog.

v2022.10

Updates

  • 2215 Design space exploration block (beta):

    • Added optional per-variable and per-response result ports, which you can use to get the result values for specific input and output components of your task.
    • Added optional per-component initial sample ports.
    • Enable those ports in the Ports and parameters dialog in the block configuration.
    Optional per-component ports in the Ports and parameters dialog

  • Python script block:

    • 1689 You can now rename request (argument) and response (function) ports - for example, to display human-readable names in the Block properties pane and the Links dialog, instead of Python variable identifiers.

      Port name settings in Python script

    • 1931 Added the code autoformatting toggle in the Configuration pane, which you can use to disable the default formatting if you find that more convenient.

      Code autoformatting toggle in Python script

    • 2244 Added the support for missing (N/A) values: it is now possible to pass arrays with missing values through ports, and pass N/A or None as a value of a float (Real), int (Integer), or str (String) variable.

  • 2050 You can copy and paste table and matrix data between the Edit value dialog and Excel or another spreadsheet editor that uses a similar clipboard format (tab-delimited tabular data).

  • 2244 Added the support for tables and matrices with missing (N/A) values.
  • 2244 Added the support for string matrices.
  • 2267 Various small improvements in the table and matrix editors in the Edit value dialog.
  • 1491 Minor UI improvements in the Condition block.
  • 1199 pSeven Enterprise is now case insensitive when checking names for uniqueness (workflows, reports, blocks, ports, and so on) - for example, you cannot add two inputs named Force and force to the same block, because those names are considered identical. This helps in avoiding various usability issues related to case-sensitive name validation.
  • 2278 Upgraded the embedded pSeven Core package to version 6.38. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

  • 1112 Added experimental support for NFSv4 file storages. Disabling NFSv4 on the file storage NFS server is now optional.

    If you are going to enable NFSv4 on the file storage NFS server during an upgrade of an existing deployment (which currently uses NFSv3), you will have to deploy with the storage check enabled (the -⁠-⁠set fixStorages=true option). See Enabling NFSv4 in the pSeven Enterprise upgrade notice below for details.

  • 2098 Updated pSeven Enterprise for compatibility with Kubernetes versions 1.15.5 through 1.25.2, removing the "batch/v1beta1 CronJob is deprecated" warning that appeared during deployment on Kubernetes 1.21 and newer versions.

  • 2249 To reduce some of the security risks related to allowing users to connect to a pSeven Enterprise deployment directly, all X-Forwarded-* headers in HTTP requests are now ignored by default.

    If your deployment is protected by a trusted reverse proxy, it is advisable to enable parsing the X⁠-⁠Forwarded⁠-⁠* headers in the deployment configuration file values.yaml. For details, see Deployment configuration in the pSeven Enterprise upgrade notice below.

  • 1815 Improved stability by resolving several memory cache issues that could lead to restarts and unavailability of various pSeven Enterprise services.

  • 2202 The SSH key for extension nodes is now stored in the deployment configuration file values.yaml (the new extensionNode.blockSSHGateClientKey parameter) and is saved on each extension node during its deployment, which slightly improves startup times of workflow blocks running on extension nodes.

    When preparing the values.yaml file for pSeven Enterprise v2022.10, you will have to generate the extension node SSH key and add it to values.yaml. For instructions, see Deployment configuration in the pSeven Enterprise upgrade notice below.

  • 2248 In addition to the custom logos and Getting started page, you can set a custom favicon for pSeven Enterprise - see section Branding for details.

    There are a few changes in the branding file structure in pSeven Enterprise v2022.10. For differences, see Branding in the pSeven Enterprise upgrade notice below.

  • 1782 In user workflow and app runs that complete without errors, pSeven Enterprise performs a log files cleanup, so successful runs take slightly less disk space than in previous versions.

  • 2295 Fixed a compatibility issue with LDAP over SSL (LDAPS), which caused the sun.security.provider.certpath.SunCertPathBuilderException error during setup of user account federation with a user identity storage provider using LDAPS.

    Using LDAPS requires importing the public key of the LDAP server into pSeven Enterprise. The key must be prepared before starting the version upgrade to v2022.10. For instructions, see Using LDAPS in the pSeven Enterprise upgrade notice below.

  • 2272 Fixed a compatibility issue with the PgBouncer database connection pooler, which led to malfunction if your PostgreSQL database provider uses PgBouncer.

  • 2264 Fixed an issue where a user who runs a workflow that contains a block with a high CPU limit could leave pSeven Enterprise unusable for other users because their workflows could not start until that workflow containing a CPU-intensive block is stopped or a 10 minute timeout is exceeded.
  • 2248 Fixed a regression issue with pSeven Enterprise branding from v2022.07, where your custom logo did not apply to the pSeven Enterprise sign-in page.
  • 1112 2098 2248 2269 Various updates and fixes in the pSeven Enterprise deployment and administration guides.

User block development

  • 1964 Fixed an issue with user blocks and the REST API where you could not start a workflow run using the REST API commands if that workflow contains a user block with an empty description of some input port, and that port is a workflow parameter.

Bugfixes

  • 2215 Fixed an issue with the Design space exploration block (beta) where it did not stop as intended when receiving an invalid response from a blackbox block - for example, a matrix with incorrect number of columns.
  • 2267 Fixed an issue with the table and matrix editors in the Edit value dialog where the button was usable only to delete the bottom row.
  • 1059 Fixed an AppsHub issue where you could not get the workflow from an app due to incorrect workflow name validation.
  • 2255 Fixed a usability issue with the Explorer pane when viewing the Runs or Files special folders that contain files or subfolders with long names.
  • 1652 Fixed an issue where you could not use slashes (/ or \) in the name of a composite block.
  • 2303 Fixed a usability issue where you could not delete a link from a workflow using the Remove command from the menu in the workflow edit toolbar.
  • 865 Fixed a minor UI issue with the Explorer pane navigation bar, which displayed pSeven Enterprise reports as common folders.

pSeven Enterprise upgrade notice

pSeven Enterprise v2022.10 features a number of updates that may require additional preparation steps for the v2022.10 upgrade, depending on your current configuration. The summary of those steps is as follows:

  • In values.yaml, set the new entrypoint.usingTrustedReverseProxy parameter.
  • Generate a new SSH key for Windows extension nodes and add it to values.yaml as the new extensionNode.blockSSHGateClientKey parameter.
  • If you are going to connect pSeven Enterprise with a LDAP server using LDAPS, place the server public key certificate into the pSeven Enterprise shared data storage so it is imported during the upgrade.
  • If your deployment is branded, update the branding file structure in the shared data storage.
  • If you are going to use NFSv4 (experimental), run deployment with the -⁠-⁠set fixStorages=true option.

Each of the additional steps is described in detail in paragraphs below. Note that they are mostly done while you prepare for the upgrade, prior to running the installation. The descriptions below supplement the general updating instructions found in section Version upgrade.

Deployment configuration

When preparing the deployment configuration file values.yaml for pSeven Enterprise v2022.10, note the new entrypoint.usingTrustedReverseProxy and extensionNode.blockSSHGateClientKey parameters:

  1. Prepare the updated deployment configuration file values.yaml as described in Upgrading pSeven Enterprise.

  2. If your deployment is protected by a trusted reverse proxy that sets the X⁠-⁠Forwarded⁠-⁠* headers for user HTTP requests, and users cannot establish HTTP connections directly to pSeven Enterprise, set the entrypoint.usingTrustedReverseProxy parameter in the prepared values.yaml file to true.

    With that setting, pSeven Enterprise will consider the X⁠-⁠Forwarded⁠-⁠* HTTP headers to be trustworthy, and will use the header information for identification of clients (users). Parsing the the X⁠-⁠Forwarded⁠-⁠* headers is disabled by default as a safe setting, and should be enabled only if your pSeven Enterprise deployment is accessible to users through a trusted reverse proxy, and not also accessible directly.

  3. The prepared values.yaml file also contains a new extensionNode.blockSSHGateClientKey parameter. This key is required to establish connections with Windows extension nodes and must be specified in values.yaml.

    1. Get the full version number and generate a new file with pSeven Enterprise v2022.10 secrets using the following commands:

      helm show chart pseven-{YYYY.MM.DD}.tgz | grep appVersion | cut -d" " -f2
docker run --rm -it {registry}/pseven-secretgen:{version} > pseven-secrets-2022-10-extnodessh.txt

    2. From the generated pseven-secrets-2022-10-extnodessh.txt, copy only the extensionNode.blockSSHGateClientKey parameter to the values.yaml file you prepare for the upgrade deployment. Do not change other secrets in values.yaml (the jobcluster.* and extensionNode.* parameters, except extensionNode.blockSSHGateClientKey) - those must be copied from the existing configuration, as described in Upgrading pSeven Enterprise.

    3. Also copy the extensionNode.blockSSHGateClientKey parameter to your backup copy of pSeven Enterprise secrets.
    4. Remove the generated pseven-secrets-2022-10-extnodessh.txt file.

Using LDAPS

pSeven Enterprise v2022.10 supports connecting to user identity storage providers using LDAP over SSL (LDAPS), which requires importing the public key of the LDAP server. The easiest way to import that key is to place the public key certificate of the LDAP server into the pSeven Enterprise shared data storage before you run the deployment:

  1. Obtain the public key certificate from the LDAP server. The certificate must be exported to a DER or Base-64 encoded X.509 file. Request the certificate file from your LDAP server administrator.
  2. Locate the pSeven Enterprise shared data storage (see the storage.shareddata.nfs.* parameters in values.yaml). Create a directory named certs in the shared data storage directory specified by the storage.shareddata.nfs.path parameter.
  3. Rename the certificate file to ldaps.cert and copy it to the certs directory. The expected path to the ldaps.cert file is <shared data>/certs/ldaps.cert, where <shared data> is the path specified by the storage.shareddata.nfs.path parameter in values.yaml.

When you run the deployment, pSeven Enterprise will import the key from <shared data>/certs/ldaps.cert. Now when you set up the user identity storage provider (see section Provider setup in the administration guide), you can specify the ldaps:// prefix in the Connection URL field to use LDAPS.

Branding

There are a few changes in the branding file structure in pSeven Enterprise v2022.10. The Branding section describes the current structure; compared to previous versions, the differences are:

  • The logo-100x266.png file, as well as the <shared data>/branding/auth/ folder, are no longer used.
  • All logo customizations should now be placed into the <shared data>/branding/pseven/ folder.
  • The logo for the sign-in screen is now named block-logo-200x530.png. For best results, this logo image should be 200×530 pixels.
  • There is an additional logo used in navigation bars, named header-logo-64x64.png. This logo image should be 64×64 pixels.
  • If you want to use a custom favicon, place your favicon.ico file into the <shared data>/branding/pseven/ folder alongside the custom logos.

Enabling NFSv4

pSeven Enterprise versions prior to v2022.10 are compatible only with NFSv3 file storage servers. The experimental NFSv4 support is added only in v2022.10, so all existing deployments of prior versions are configured to use NFSv3 only, and NFSv4 is disabled on the NFS server.

If you want to start using NFSv4, enable the NFSv4 protocol on your file storage server before deploying the v2022.10 upgrade, and run deployment with the -⁠-⁠set fixStorages=true option:

  1. Before you run the deployment, revert the changes described in section Disabling NFSv4 of the pSeven Enterprise deployment guide. Use the test described there to verify that NFSv4 is now enabled:

    1. Create a test directory and try mounting it using NFSv4.

    2. Verify that a command like the following runs without errors:

      $ sudo mount -t nfs -o vers=4 localhost:<test directory path> <mountpoint>

    3. Additionally you can run the mount | grep <test directory name> command to check that the test directory is mounted using NFSv4: look for the mountvers option in the command output, it should read mountvers=4.

  2. When running the v2022.10 deployment, enable the storage check by adding the -⁠-⁠set fixStorages=true option to the helm install command, as noted in the upgrade guide - see the command example in section Upgrading pSeven Enterprise. Note that the storage check may take some time depending on the amount of storage data.

  3. Once the deployment finalization message appears, verify that the storage check has run. The message should contain the following note:

    Fixing storages enabled: deployment was run with the "--set fixStorages=true" option, fixing user permissions in data storages.

v2022.09

Updates

  • 1503 pSeven Enterprise can email you when your workflow run completes or interrupts. Enable mail notifications in the pSeven Enterprise user menu.

    Notifications toggle in the user menu

  • 1636 The Share dialog supports username autocompletion so you no longer have to input full names or email addresses of users with whom you want to share your files.

    Autocompletion suggestions in the Share dialog

    Autocompletion settings are managed by your administrators, who may decide to disable username suggestions for security reasons.

  • 1559 2121 You can specify the following block run settings in the Block properties pane:

    Block run settings in the Block properties pane

    • Memory limit (MB) - the amount of memory reserved for a block. By specifying a higher memory limit than default, you can request more memory for a block that has high memory consumption.
    • Idle timeout (sec.) - the timeout (in seconds) after which a ready block, which captures runtime resources and waits for input data, is stopped by pSeven Enterprise to free up the resources. In particular, you can set the timeout to NEVER to keep a block running until the workflow run stops - this is useful in workflows that run as a service, where certain blocks have to run continuously in order to improve the service response times.

    Block run settings are always editable, though they have effect only if the workflow runs on a pSeven Enterprise deployment where administrators have enabled the resource management features. This may be useful if you edit and configure your workflow on a test deployment with resource management disabled and then move the ready workflow to another, working deployment where resource management is enabled.

  • 1559 The Workflow contents pane on a workflow tab provides an overview of memory limits you have set for workflow blocks.

    Block memory limits overview in the Workflow contents pane

  • 2138 Updated and fixed the matrix editor in the Edit value dialog: that editor now provides all matrix editing functions that were available in previous versions (fill, transpose, and so on).

  • 882 When you create a run of a read-only shared workflow and select a folder where to create the run directory, the default location is the folder you currently have open in Explorer. In previous versions the default location was the root of your home folder, which is often inconvenient.
  • 2187 Upgraded the embedded pSeven Core package to version 6.37. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

  • 2086 Since v2022.09, pSeven Enterprise supports integration with Active Directory and LDAP services - see section Users from Active Directory or LDAP server in the administration guide for instructions.

    If you want to enable Active Directory or LDAP integration during an upgrade of an existing deployment, where users already have accounts created by means of the pSeven Enterprise admin interface, you will have to perform account matching as a part of that setup in order to prevent users from losing access to their data. For instructions, see the pSeven Enterprise upgrade notice below.

  • 2173 You can set up single sign-on (SSO) for pSeven Enterprise users, integrating with an external OpenID provider, such as a Keycloak-based identity provider. See section Setting up single sign-on (SSO) in the administration guide for instructions.

  • 1636 You can enable username and email autocompletion in pSeven Enterprise UI - for example, in the Share dialog, so the user who shares a folder to others will not have to input their full usernames or email addresses. Autocompletion is disabled by default; to enable, set the exposeUsersInfo parameter in the deployment configuration file values.yaml to true when upgrading your deployment to pSeven Enterprise v2022.09.

    Enabling autocompletion is a security risk

    A malicious user can exploit autocompletion to gather information about names and email addresses of other users. It is strongly recommended to disable this feature in public deployments, and enable it only if you are deploying pSeven Enterprise in a private environment that allows user contact sharing.

  • 2213 Fixed an issue with blocks configured to run on a Windows extension node - such a block was unable to launch a software package installed on the node, if that package requires certain environment variables (a typical requirement of various engineering software). Note that in order to avoid this issue, an additional node configuration step is needed - you must add all required variables to the node's block runtime environment as described in section Engineering software on extension nodes of the administration guide.

  • 1829 To avoid stability issues when many users actively edit blocks (open block configuration windows) and run workflows, the block edit sessions are now handled by the new appworkerblockjobs service while workflow run tasks are handled by the appworkerworkflowjobs service. Previously tasks of both kinds were handled by appworkerworkflowjobs, which caused issues for users - in particular, when a user cannot start any workflow run because other users have many block configuration windows open, thus overloading the appworkerworkflowjobs service so it cannot start new workflow run tasks.
  • 1514 Improved run resource management for Windows extension nodes to avoid issues caused by a high number of blocks running at the same time on a given Windows node.
  • 2222 Fixed a regression issue from v2022.07, which caused incompatibility with certain PostgreSQL providers that use PgBouncer as the database connection pooler: pSeven Enterprise deployment stopped with an error when initializing the Keycloak database (executing the initkeycloakdb Helm hook).
  • 2247 Fixed an issue where a < or > character in the database password (the postgres.password parameter value in values.yaml) caused a deployment error.
  • 2260 Fixed an issue with run resource management where, if there are insufficient free resources to start a workflow run, that run started and immediately failed with an "Insufficient resources to run the workflow" error, instead of being put into the run queue as intended.
  • Updated the pSeven Enterprise administration guide:
    • 2093 Added instructions for using event logging to audit activities of pSeven Enterprise users and administrators - see section Auditing user and admin activity.
    • 2181 Added instructions on how to enable users to change their username - see section Changing the username.
  • 1829 Updated the service interaction diagram in the pSeven Enterprise deployment guide: added the new appworkerblockjobs service and changed the layout to make the relationships between diagram elements more clear.

Bugfixes

  • 2140 Fixed a regression issue from v2022.08 where workflow runs failed with "The following IDs are unknown" or "Locked document must be a directory" errors.
  • 2116 2250 Fixed an issue where a prolonged workflow run created too many temporary files and auxiliary logs, due to which it occupied much disk space and caused performance and responsiveness issues when you navigate that run's results in Explorer or view run logs.
  • 2237 Fixed a regression issue from v2022.06 where a composite block with batch mode ports did not actually enable batch processing for arrays and lists sent to those ports, if the @Number of parallel workers setting is 1.
  • 2201 Fixed a regression issue from v2022.08 where uploading a file using drag and drop caused incorrect behavior of the Explorer pane.
  • 2252 Fixed a regression issue from v2022.08 where the AppsHub Runs sidebar was displayed when you open any HTML file from the Explorer pane.
  • 1829 Fixed an issue with the pSeven Enterprise load balancing system that caused confusing behavior: if other users have many block configuration windows open at the same time, you could not start any workflow run with no apparent reason. As a part of that fix, pSeven Enterprise now limits the number of block configuration windows you can open at once, and keeping them open does not affect workflow runs.
  • 2236 Fixed a regression issue from v2022.08 where pSeven Enterprise could not open a workflow run in a tab and displayed a "variable referenced before assignment" error.
  • 2254 Fixed a REST API regression issue from v2022.06: with some workflows, a correct PATCH request to set workflow run parameters returned a response with status code 500 Internal Server Error.
  • 2217 Fixed a number of issues with the Design space exploration block (beta):
    • Inconvenient data types used by request ports, which send values of variables to blackbox blocks for response evaluation.
    • Incorrect behavior when you configure a space-filling DoE technique and disable batch mode in run options: when sending generated DoE points to blackbox blocks evaluating responses, the Design space exploration block output all points as a single batch instead of the expected sequential output mode.
    • Incorrect behavior of the problem definition ports (for example, variable bounds ports): if you assign a value to such a port and connect a link to it, the block always used the assigned value while it should wait for and use a value received from the link.
    • An error when saving a block with a valid initial sample, which contains NaN or missing (N/A) response values.
    • Minor UI issue when you set the Maximum batch size option and the Gradient-based optimization technique is selected - that option does not have any effect in this case as the technique does not provide the support for real concurrency in response evaluations, but the block UI did not make it clear.
  • 2237 Fixed an issue where a workflow run continued indefinitely if the workflow contains a composite block with a batch mode port, and a value is assigned to that port.
  • 2178 Fixed an issue with the Python script block that sometimes caused a "block init was called more than once" error when starting a workflow run.
  • 1812 Fixed UI responsiveness issues in the Block properties and Composite properties panes.
  • 2197 Fixed a convenience issue with editing input values in block properties and Run setup panes, where a data editor window always popped up even if the value is a plain scalar number or a string.
  • 2041 Fixed an incomprehensible text in the error message that appears when you undo changes in a workflow but pSeven Enterprise cannot apply the undo immediately.
  • 2158 1942 1984 771 Fixed several minor UI issues in the Explorer, Block properties, and Composite properties panes.

pSeven Enterprise upgrade notice

Since version v2022.09, users can sign in to pSeven Enterprise with their accounts from an external storage such as Active Directory Domain Services or an LDAP server. To enable this feature after upgrading pSeven Enterprise, user account federation must be set up and run as described in Users from Active Directory or LDAP server. As a result, for each user account from the external storage that matches the federation settings, the associated account of the pSeven Enterprise user will be created. If any pSeven Enterprise users have previously created non-federated accounts, you must link their new federated accounts to existing authorization accounts as described below. Otherwise, upon the first sign-in with the new federated account, those users will get new authorization accounts that consume the pSeven Enterprise license and do not provide access to the existing user data associated with the old authorization accounts.

Account matching

Running federation results in creating new accounts in the pSeven Enterprise authentication service. Upon the first sign-in to pSeven Enterprise, an authorization account is created for the user account, which provides access to the user data such as their home directory, files, workflows, and apps. If the user already has an account created by means of the pSeven Enterprise admin interface (see Adding accounts), you must link the respective authorization account to the new user account, created by running federation. This ensures that having signed in with the new account, the user can access their data in pSeven Enterprise.

The authorization account of the user in question is initially linked to their old account, which was created by means of the pSeven Enterprise admin interface. Before allowing the user to sign in to pSeven Enterprise with their new account, you must complete the following steps:

  1. Disable the old account of that user in the pSeven Enterprise authentication service.
  2. Link the authorization account of that user to the new user account, which federation has created for that user.

Follow the instructions below to complete each of these steps.

To disable the old account in the pSeven Enterprise authentication service:

  1. Sign in to the pSeven Enterprise admin interface and click Manage users in the page header.
  2. Look up the old account: on the Users page of the authentication service interface that appears, enter the username or a part of the username in the Search box, and then press Enter.
  3. In the lookup results list, click the ID of the old account to open its edit page.
  4. On the Details tab of the old account edit page, turn the User Enabled switch OFF and click Save.

Copy and save the username provided in the Username field on the Details tab of the old account edit page. This username will be required to locate the authorization account.

The old account might prevent the creation of the new federated account. This happens, for example, if the account in the external storage has the same email or username as the old account in pSeven Enterprise. In this case, having copied and saved the username, you should delete the old account, and then run federation again.

To link the authorization account to the new user account:

  1. On the Users page of the authentication service interface, look up the new user account and open its edit page.
  2. Copy and save the contents of the ID field displayed on the Details tab of the edit page - this is the unique identifier of the new user account.
  3. Click the icon in the upper right corner of the authentication service interface page, and then select the Manage user permissions menu item to open the list of authorization accounts.
  4. In the list that appears, click the username of the old user account.
  5. In the Keycloak user ID field on the Change user page that appears, enter the identifier of the new user account and save the changes.

The user can now sign in to pSeven Enterprise with the new account. The account matching you performed ensures that the user keeps access to their data.

v2022.08

Updates

  • 1834 You can view and manage app runs in AppsHub using the new Runs sidebar available in all apps, including those published with a custom app UI.

    • Click at the upper left of the app page to open the Runs sidebar.
    • Click a run in the sidebar to view its results.
    • To delete runs you no longer need, select them in the sidebar and click the button that appears.
    • By default, you can view and delete only your runs. The app owner can view and delete all runs.
    The app Runs sidebar

  • 2069 2149 2124 Updated the Design space exploration block (beta), resolving a number of usability and compatibility issues. Since this version, future pSeven Enterprise updates will keep compatibility with the current Design space exploration block version.

    An optimization problem set up in a Design space exploration block

  • 2149 You can use the table editor in the Edit value dialog to create any custom table, as it now supports adding, removing, renaming and reordering table columns.

    Column settings in the table editor

  • 2136 Improved Edit value dialog usability for simple values (numbers and strings).

Deployment and administration

  • 1629 Windows extension node setup no longer requires Internet access.

  • 2151 In a deployment with the "on-demand" block execution strategy enabled, a deployment administrator can control the block shutdown timeout with the new Inactive block timeout setting.

    The "on-demand" block execution strategy is an experimental feature.

  • Updated the pSeven Enterprise administration guide:

  • 2192 Fixed a user management issue where pSeven Enterprise deployment could stop with an error if some users have changed their usernames since the previous version upgrade or restart.
  • 2185 Fixed a stability issue in workflow run management where a run that failed to start was blocking subsequent workflow runs for all users, and all those runs got stuck in the "Configuration" state until you restart pSeven Enterprise.
  • 1803 Fixed a stability issue with the user interface service (appserver) where it could run out of memory if there are users who monitor run logs during execution, which caused a service restart, creating UI issues for other Studio users.

User block development

  • 2157 You can implement a user block that runs an existing pSeven Enterprise workflow as the current user running a workflow with your block. Such user blocks work with the pSeven Enterprise REST API described in the REST API guide. pSeven Enterprise provides access to the REST API through the following block environment variables:
    • DA__P7__BLOCK__REST_API_URL - contains the REST API URL.
    • DA__P7__BLOCK__REST_API_TOKEN - contains the REST API authorization token of the current user.
  • 2185 Fixed an issue with development of Windows-only blocks, where a user who opens such a block was receiving a "Can not submit job to the cluster!" error.

Bugfixes

  • 2184 Fixed an issue where you could not sign in to pSeven Enterprise after waiting at the sign-in page for a certain time.
  • 2111 Fixed an issue where file upload, copy, and other file operations in Explorer caused a "connection already closed" error.
  • 2113 Fixed an issue with logging in long-running workflows, which caused slowdowns and unresponsiveness when working with such workflow runs:
    • Disabled extended block logging by default to reduce the amount of logs.
    • Logs displayed in the Run log pane are truncated for faster loading.
    • To get a full log, you can download the log file with the Download command from the Run log pane menu.
  • 2185 Fixed an issue with Windows-only blocks where a "Can not submit job to the cluster!" error appeared when you open the block configuration dialog.
  • 1840 Fixed an issue where sending a Matrix value to a batch mode port of some composite block caused an error, which stopped the workflow run.
  • 1803 Fixed an issue with delays in workflow loop execution, that became noticeable after a certain high number of loop iterations.
  • 1437 Fixed an issue with workflow loops where, if the loop completed but did not send some of the required responses to the loop driver block, that block waited indefinitely instead of raising an error and interrupting the workflow.
  • 2145 Fixed an AppsHub issue with the stock app UI that caused incorrect app behavior if your network connection with AppsHub got interrupted while the app runs.
  • 2133 Fixed an issue where a Python script block configured to output a NumPy array of objects (for example, an array of lists) caused an error during a workflow run.
  • 1947 Fixed a UI issue with the Workflow library pane where, if you add a block to your workflow and close its tab or hide the pane before it finishes loading the block, the added block was hidden from the pane until you add one more block.
  • 1295 Fixed an issue where an error occurred when you copy and paste a block into the same workflow, if the name of that block is a number.
  • 1564 Fixed a UI issue with the Block properties pane where it could display incorrect variable names in port tooltips for Condition blocks.
  • 562 Fixed an issue with the Explorer pane where pSeven Enterprise did now display an appropriate error message if you input a non-existing path in the Explorer address bar.

v2022.07

Updates

  • 1587 You can manage your account settings - change email, password, and other - in your user profile. To open it, use the new Edit profile command from the pSeven Enterprise user menu.

    The Edit profile command in user menu

    This feature is enabled by the new user authentication system introduced in pSeven Enterprise v2022.07. Due to changes in authentication, you will be required to verify your account once after the upgrade to v2022.07. Until you verify your account, sign-in attempts to pSeven Enterprise v2022.07 will result in an "Invalid username or password" message. To verify, click the "Forgot Password?" link in the pSeven Enterprise v2022.07 sign-in form. Upon clicking that link, you will receive an email with further instructions.

  • 1934 To make it easier to find run tabs related to a particular workflow, pSeven Enterprise opens new run tabs next to the related workflow tab, instead of opening them after the last existing tab.

    Run tabs open next to their related workflow tab

  • 1946 You can quickly show or hide all block presets, expand and collapse all block groups in the block library using the new and buttons in the Workflow library and Common library panes.

  • 2127 Upgraded the embedded pSeven Core package to version 6.36. For details on recent improvements and changes, see the pSeven Core changelog.
  • 2150 Updated the REST API guide in the pSeven Enterprise Help to describe REST API v2.

Deployment and administration

Important

When upgrading to pSeven Enterprise v2022.07, follow the instructions found in the pSeven Enterprise upgrade notice below. Otherwise, users may be unable to sign in after the upgrade due to deployment configuration errors, and fixing those errors will require you to re-deploy v2022.07.

  • 1587 pSeven Enterprise v2022.07 works with a new authentication service based on Keycloak, which improves user account security and provides additional account management options.
  • 2114 Added the number of remaining Studio and AppsHub user licenses to the Licenses page in the pSeven Enterprise admin interface.
  • 2140 Fixed a regression issue from v2022.06 where pSeven Enterprise deployment stopped with an error, if you set the entrypoint.allowedHosts parameter in the deployment configuration file values.yaml.
  • 2180 Fixed a stability issue with the user interface service (appserver) where it could become unresponsive if many users actively switch tabs in Studio, causing UI slowdowns and errors for all Studio users.
  • 2139 Updated and extended the pSeven Enterprise deployment and Administration guides following the introduction of a new authentication service based on Keycloak:
    • Updated the initial admin account setup instructions in section Admin access.
    • Updated the service interaction diagram.
    • Updated the instructions for adding new users in section Adding accounts.
    • Added the instructions for administering existing user accounts, see Managing user account data.
  • 1346 Added section Backing up and restoring user data to the administration guide.

User block development

  • 1949 Fixed an issue where loading a block prototype to the workflow library by dragging its folder from the Explorer pane onto the Workflow library pane caused a "Device or resource busy" or "Directory not empty" error.
  • 2144 Fixed an issue where a value in scientific notation (for example, 1e-08) in block manifest caused an error when you add that block to the workflow library.

Bugfixes

  • 2147 Fixed a regression issue from v2022.06 where a workflow run stopped with an error, if the workflow contains a block that outputs a large data array.

    If you encounter that issue, update blocks in the workflow library by clicking the update notification next to the block name in the Workflow library section of the Block library pane. See Updating blocks to a newer version for more details.

  • 2155 Fixed a regression issue from v2022.06 where you could not run an app in AppsHub due to a workflow run error, if app parameters have no default values.

pSeven Enterprise upgrade notice

Due to the introduction of a new authentication service based on Keycloak, an upgrade to pSeven Enterprise v2022.07 requires a few additional configuration steps, which are summarized below.

Before you upgrade to pSeven Enterprise v2022.07:

  1. It is recommended to create a backup of the pSeven Enterprise database. You may need that backup if you decide to rollback to a previous pSeven Enterprise version: the v2022.07 upgrade adds new database objects, which may cause issues if the database once upgraded to v2022.07 is used with an older version. The pSeven Enterprise database is specified by the postgres.dbname parameter in your deployment configuration file values.yaml. For database backup instructions, see Backup and Restore in the PostgreSQL server documentation.

Additional upgrade steps required for pSeven Enterprise v2022.07:

  1. Prepare the updated deployment configuration file values.yaml as described in Upgrading pSeven Enterprise.
  2. Check the entrypoint.allowedHosts and entrypoint.allowedAuthPorts parameter settings in the prepared values.yaml file (see the comments to those parameters for details). Note that entrypoint.allowedAuthPorts is a new parameter, which must be set correctly, considering the domain name and port numbers used to access your pSeven Enterprise deployment. Otherwise, you users will be unable to sign in to pSeven Enterprise after the upgrade.

  3. The prepared values.yaml file also contains a new keycloak.clientSecret parameter. This secret is required to establish connection to the new authentication service (keycloak) and must be specified in values.yaml, otherwise user authentication will be disabled, and users will be unable to sign in.

    1. Get the full version number and generate a new file with pSeven Enterprise v2022.07 secrets using the following commands:

      helm show chart pseven-{YYYY.MM.DD}.tgz | grep appVersion | cut -d" " -f2
docker run --rm -it {registry}/pseven-secretgen:{version} > pseven-secrets-2022-07-keycloak.txt

    2. From the generated pseven-secrets-2022-07-keycloak.txt, copy only the keycloak.clientSecret parameter to the values.yaml file you prepare for the upgrade deployment. Do not change other secrets in values.yaml (the jobcluster.* and extensionNode.* parameters) - those must be copied from the existing configuration, as described in Upgrading pSeven Enterprise.

    3. Also copy the keycloak.clientSecret parameter to your backup copy of pSeven Enterprise secrets.
    4. Remove the generated pseven-secrets-2022-07-keycloak.txt file.

  4. Verify your entrypoint.allowedHosts, entrypoint.allowedAuthPorts, and keycloak.clientSecret parameter settings in the updated values.yaml file you prepare. Then continue with the upgrade deployment, following the Version upgrade guide.

After the upgrade to pSeven Enterprise v2022.07:

  1. All admin accounts will have their password reset to admin. Once the upgrade deployment completes, sign in as administrator immediately and change the default password for all admin accounts. To sign in, specify an existing admin username (email address) and the admin password. For password reset instructions, see Managing user account data.

  2. The default administrator account is assigned the username admin instead of admin@admin.admin.

    If user authorization is configured for that account on an extension node (see Configuring user authorization on extension nodes), you will have to change the regular expression in the DA__P7__ALLOWED_USERS environment variable to match the new username admin. Otherwise the user admin will lose existing permission to run blocks on that specific extension node.

  3. All user accounts will have their password unset, and users will have to set a password. Notify your users that they need to pass a one-time account verification by resetting their passwords using the "Forgot Password?" link at the sign-in page. Users who have not set a password will be unable to sign in to pSeven Enterprise v2022.07, and their sign-in attempts will result in an "Invalid username or password" message. Otherwise, you can also set user passwords in the pSeven Enterprise admin interface (see Managing user account data).

Upon the v2022.07 upgrade, existing user accounts keep their usernames, so existing users should sign-in with an email address. The exception is the default administrator account, which is assigned the username admin after the upgrade. New users added after the upgrade are no longer required to use an email address as a username.

If there are no active admin accounts in your deployment, the v2022.07 upgrade will also re-enable the default admin account (username: admin, password: admin).

v2022.06

Updates

  • 2074 In the Share dialog, you can press Enter to add another user instead of clicking every time.
  • 2025 Enabled renaming block working directories in run result folders of completed workflow runs.
  • 1376 Added the support for N-dimensional NumPy array outputs in the Python script block:
    • 2-dimensional arrays are output as matrices by default, as earlier
    • Other arrays are output as nested lists by default
    • A flat (1-dimensional) array is also compatible with the matrix data type and can be converted to a single-column matrix
  • 1376 The pSeven Enterprise REST API protocol is updated to v2 with minor changes in the run parameters and run results structures (see sections Run parameters and Run results in the REST API guide for details). The new protocol version is used only if you access the REST API methods by URL /pseven/.rest/v2/. pSeven Enterprise keeps compatibility with the REST API protocol v1 (URL /pseven/.rest/v1/), so existing REST clients, which use REST API v1, will continue to work with no issues. However it is advisable to update existing clients to REST API v2, as well as use only REST API v2 in new client implementations.
  • 1376 You can write data tables for reports from a Python script block using the api.write_report_table() function from the block's embedded api module. Function parameters are write_report_table(path, data), where path is the table file full path with an optional .p7table extension, and data is the table data as pandas.DataFrame.
  • 1827 You can view shell scripts (.cmd, .bat, .sh) in pSeven Enterprise, in addition to previously supported file formats. To view a file, double-click it in Explorer, or select it and press Space.
  • 1827 Added a few commonly used shortcuts for keyboard navigation in Explorer.
  • 2038 Removed the Show active document folder command from the menu in Explorer as it is more conveniently available from the tab bar: while viewing a workflow, run, or report click in the tab bar or double-click the tab title to open the related folder in Explorer.
  • 2080 Upgraded the embedded pSeven Core package to version 6.34. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

  • 1595 Increased the number of the block configuration UI router service (blockrouter) replicas to 2 to avoid interrupting user work in case of possible blockrouter failure.
  • 1422 Fixed a deployment issue where possible errors in configuration of the database server or NFS storage server were reported only at the final deployment stage, leading to additional downtime during pSeven Enterprise version upgrades or redeployment. pSeven Enterprise now tests connection to the database and file storages at the start of the deployment.
  • 2073 Fixed an issue where Windows extension nodes could not connect to pSeven Enterprise if it is deployed on a Kubernetes cluster that uses the Calico CNI plugin.
  • 2048 Revised and updated admin guides in the pSeven Enterprise Help:
    • Added a separate Version upgrade section to help locate the appropriate instructions, replacing the upgrade sections previously included in the pSeven Enterprise administration and extension node deployment guides.
    • 1544 Expanded the administration guide with section Understanding accounts to provide a better understanding of account setup in pSeven Enterprise.
    • Moved the section describing user access settings on extension nodes from the extension node deployment guide to Administration - see Configuring user authorization on extension nodes.
    • Moved all uninstallation instructions to a separate Uninstallation guide.
    • Several other minor updates and additional explanations.

User block development

  • 1376 Redesigned the Edit value and View value dialogs to better support custom tables with predefined columns and other custom data types.
  • 2052 Fixed a regression issue from v2022.03 where pSeven Enterprise did not properly remove files from the block data directory (DA__P7__BLOCK_DATA_DIR) after they had been deleted by the block.
  • 2092 Updated section Block assets in the user blocks guide with more information about publishing and updating assets, and also using assets to add Python modules to blocks.

Bugfixes

  • 2082 Fixed a regression issue from v2022.05 where you could not download large files from pSeven Enterprise.
  • 2061 Fixed a regression issue from v2022.04 where moving a block to another workflow using cut and paste would not remove the block from the source workflow, causing errors when you edit the source workflow.
  • 1733 Fixed an issue where passing complex List or Dict values between blocks caused delays in workflow execution.
  • 1376 2135 Fixed an issue with the Python script block where a 1-dimensional NumPy array sent to a request driver port was converted to a single-column matrix by default. 1-dimensional arrays are compatible with the matrix data type, but such an array should be converted to a single-column matrix only if you pass it through a port that has its type set to matrix.
  • 1686 Fixed an issue with editing workflow notes where they could reset position or disappear if you are actively editing notes.
  • 2025 Fixed an issue with renaming subfolders in the Files special folder in workflows where you could add the reserved .p7wd extension to any folder, which caused various filesystem error messages.
  • 2123 Fixed a UI issue with the workflow editor where, if you remove some port that has a link connected to it, that link was automatically removed as intended but was still displayed in the workflow until you refresh a browser tab, which caused confusion.
  • 2036 Fixed several issues with the Run setup pane where it incorrectly displayed the current parameter preset.
  • 1957 Fixed incorrect behavior of the info pane in Explorer when selecting a shared folder (workflow, report) that is no longer available - for example, was moved or deleted by its owner.
  • 1887 Fixed an issue with the View as text command in Explorer where viewing a binary file caused UI unresponsiveness.
  • Design space exploration block (beta):
    • 2066 Fixed slowdowns when adding or removing the block from a workflow.
    • 2068 Fixed incorrect order of ports in the Block properties pane.
    • 2068 Fixed several issues that caused block execution to stop with an error.

Compatibility issues

  • 1376 Due to changes in data type conversion rules that were required to add the support for N-dimensional NumPy arrays in the Python script block, in certain cases the arrays that were previously converted to a single-column matrix may now instead be converted to flat lists if you pass them through untyped ports (the "Any" port type). Typically this causes a "'float' object is not subscriptable" error in the block that receives such an array. In affected workflows, that error appears only after you update the Python script block in the workflow library to the most recent version (block version 20). To revert to the old behavior, you can change port type in affected blocks to the matrix type - to force conversion of a 1-dimensional array to a single-column matrix.
  • 1376 The typesystem.File() constructor previously used in the Disk optimization example to create data tables for reports is no longer supported in Python script blocks, as it was replaced with the more convenient api.write_report_table() function from the block's embedded api module.

v2022.06.07

v2022.06.07 is a hotfix release for several issues related to the "Failed to send interrupt request to block" error appearing in v2022.05.

Bugfixes

  • 2096 Fixed a regression issue from v2022.05 where numerous "Failed to send interrupt request to block" error messages appeared in the Run log pane when you stop the workflow.
  • 2096 Fixed a regression issue from v2022.05 where the stop button on a running block did not work.
  • 2096 Python script block: fixed a regression issue from v2022.05 where api.interrupted() returned False even if the workflow run is stopping when the block calls this function, instead of returning True in that case, as intended.

User block development

  • 2096 Fixed a regression issue from v2022.05 where, when a user stops a workflow run, pSeven Enterprise did not send an interrupt request (the interrupt control endpoint) to running blocks, so a block could not handle the run stop properly.

v2022.05

Updates

  • 1699 To update the stock app UI in a workflow created in previous pSeven Enterprise versions or to revert UI changes in a workflow, you can use the new app UI reset command from the menu in Explorer. To update an app published with the stock UI, reset the app UI in the workflow you published as that app, then re-publish the workflow.

    The app UI reset command in the Explorer menu

  • 2018 Timestamps in the Run log pane use the time format from your system (browser).

  • 1818 Added a couple of convenience features for the tree view in the Explorer pane:
    • If you navigate between folders that are displayed in tree view (for example, run results), the pane now remembers the tree state in a few folders you have recently visited.
    • If you input a path pointing to a file or folder in a tree, the pane expands the corresponding tree node upon navigating to that path.

  • 1542 The Explorer pane uses icon and background colors to indicate status of workflow runs. The color scheme is consistent with the one used on run tabs.

    Colored run icons in Explorer indicating run status

  • 1129 Added an early preview version of the Design space exploration block, which provides methods for design of experiments and optimization.

    The Design space exploration block in the library

    This early preview version of the Design space exploration block may appear to be incompatible with future pSeven Enterprise updates and should be used for testing only.

  • 2039 Upgraded the embedded pSeven Core package to version 6.33. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

  • 1781 To prevent situation when some user opens too many blocks or starts too many workflow runs at the same time, which interferes with other users' work, you can now set limits for the number of open blocks and the number of workflow (or app) runs - common ones on the Settings page, which apply to every user by default, and individual limits for specific users in their user settings.
  • 2006 Added information about storage space used and storage quota to the Users page in the pSeven Enterprise admin interface.
  • 2022 Fixed an issue with pSeven Enterprise version upgrade where the deployment of a new version could take several hours, depending on the amount of stored user data, because it always ran a filesystem check procedure, which is often not required. That procedure is now optional, does not run by default, and you can enable it by adding -⁠-⁠set fixStorages=true to the helm install deployment command.
  • 2054 Fixed an issue where pSeven Enterprise could stop responding to user actions and sign-ins, which was caused by appserver service errors and required you to manually restart appserver. Now if any of the appserver replicas stops responding for longer than a minute, that replica is restarted automatically, which may interfere with work of some users but avoids a complete functional failure.
  • 2002 Fixed an issue where user workflow runs were blocked by the license limit if a number of runs had failed earlier - sometimes the run failure status was not set correctly, so failed runs could count as still active.
  • Updates for pSeven Enterprise deployments with run-time resource management enabled (optional feature, disabled by default):
    • 558 Added a block execution scheduler to avoid overallocation of resources when running user workflows.
    • 2021 Fixed an issue where pSeven Enterprise could cease to run user workflows until you restart the task execution service.

Bugfixes

  • 2054 Fixed an issue where you could not sign in to pSeven Enterprise or it stopped responding after certain errors - for example, when someone opens a corrupt workflow or report - which could be resolved only manually by a deployment administrator.
  • 906 Fixed a security issue in the REST API where a malicious user could set permissions on a folder owned by other user.
  • 2045 Fixed a UI issue where the Block properties and Composite properties panes incorrectly displayed unconnected optional input ports (gray dot) as required (red dot).
  • 2031 Fixed a convenience issue with the list of recent runs for shared workflows in the Start tab where the list included runs of all users who have access to that workflow instead of including only your runs.
  • 1699 Fixed an AppsHub issue with the stock app UI where an attempt to run an app that was deleted after you opened the app page produced an incomprehensible error message.
  • 1531 Fixed a UI issue where the Run setup pane did not display long values of parameters in full even if you extend pane width.
  • 1818 Fixed a UI issue where refreshing the Explorer pane in the tree view collapsed all tree nodes.
  • 807 Fixed a UI issue with editing workflow notes where the browser context menu did not appear when you right-click a note.
  • 528 Fixed a UI issue where special folders like Files or Runs were not always sorted to the top in the Explorer pane as intended.

v2022.04

Updates

  • 1808 Updated the stock app UI for AppsHub: if you open an app page directly by its URL, the app recognizes query string parameters as its arguments and automatically sets parameter values in the UI. This is useful, for example, if you integrate pSeven Enterprise apps to some other environment where users select parameter values before they launch an app: now you can pass user values to app in the query string, so users do not have to repeat parameter input in the app. See section App URL parameters for details. Note that previously published apps do not get this update automatically. To update an existing app, update the stock app UI files in the UI folder of that app's source workflow, then re-publish the workflow.

  • 1939 If a workflow run has stopped with an error, you can see error details in the run status tooltip on the run toolbar.

    Run status tooltip with error details

  • 1764 Several improvements in the Block properties and Composite properties panes:

    • If you are viewing contents of a composite block, its Composite properties pane displays port connection status regarding uplinks inside that composite block. Previously the pane was always showing status of external connections to the composite block, which was inconvenient when checking uplinks from nested blocks to ports of their parent composite block.
    • The sorting buttons work independently - each button switches the sort order in a specific list, not everywhere in the pane. For example, you can use different sort order in the input and output port lists.

    • Better alphabetical sorting: it is now case insensitive, and numbered ports are sorted in natural order - for example, port 9 is sorted before port 10.

      Port sorting

    • Sorting now applies to the order of port groups. Previously, sorting applied only to ports inside each group but never changed the group order, and port groups were always displayed before non-grouped ports in the list.

    • When you edit a port name in the Composite properties pane, leading and trailing space characters are trimmed to avoid accidentally creating ports with names that differ in the number of surrounding spaces only and look confusingly the same in the pane.
    • Port name validation in composite blocks is now case insensitive. In the Composite properties pane, it is no longer possible to create ports with names that differ by case only, like value and Value.
    • Improved pane UI responsiveness when working with large lists of ports.

  • 2000 Upgraded the embedded pSeven Core package to version 6.32. For details on recent improvements and changes, see the pSeven Core changelog.

  • Updated pSeven Enterprise Help:
    • 1865 Updated the Python script block guide with a new section explaining how to get a complete list of Python modules available in the block - see Listing modules.
    • 1865 Added a few notes on troubleshooting module import errors to section Python modules in the Python script block guide.
    • 1808 Updated section Publishing the workflow as an app with information about using query string parameters in the app URL to pass parameter values to an app (see App URL parameters).

Deployment and administration

  • 1694 pSeven Enterprise v2022.04 contains updates in the license mechanism, which add flexibility to license settings, enabling certain optimization of license usage (see below). Due to these updates, previously issued licenses are not compatible with v2022.04. You will need to request an updated license key and have it ready before you begin the update - so you can update the license immediately after deploying v2022.04. Otherwise, your users will be unable to sign in to pSeven Enterprise until you update the license key in your deployment. For further details, see the pSeven Enterprise upgrade notice below.
  • 1694 You can configure an admin account so that it does not count as a licensed user. See sections User permissions and Admin account configuration in the pSeven Enterprise administration guide.
  • 1694 pSeven Enterprise license since v2022.04 counts the number of AppsHub users and Studio users separately. AppsHub users can sign in to AppsHub and run apps from there. Studio users can sign in to both Studio and AppsHub, can edit and run workflows, publish them to AppsHub and run apps. AppsHub and Studio access require different user permissions, so you can configure user accounts to optimize license usage - see sections User permissions and End user account configuration in the pSeven Enterprise administration guide.
  • 2012 If a workflow or an app run has failed, you can now see the fail reason on the run page in the admin interface.
  • 2012 Removed the "Job info" column from the Workflow runs page in the admin interface as it was taking too much screen space. Job info is available on each run page.
  • 1171 Fixed an issue where numerous pint.util warnings appeared in logs of every Kubernetes pod.
  • 1813 Fixed an internal configuration issue, which could cause zombie processes appearing in pSeven Enterprise service containers, potentially leading to various failures that would be hard to investigate.
  • 1989 Updated the pSeven Enterprise deployment guide with notes about additional CPU requirements for the Kubernetes cluster worker nodes (moved: see Requirements).
  • 1988 Slightly cleaned up and improved the layout of the deployment diagram in the pSeven Enterprise deployment guide.
  • 1694 Updated section User management in the pSeven Enterprise administration guide.

User block development

  • 1990 Fixed an issue where names of assets in the block manifest (see Block assets) were not checked properly, which allowed using assets with names that can lead to a name clash in block environment variables that store asset paths.

Bugfixes

  • 1977 Fixed several internal issues with task management services that led to instability being one cause of incorrect behavior when running workflows - such as stopping with an error because a block failed to start, or inability to start another run after some run has stopped with an error.
  • 2013 Fixed a bug in workflow execution, which contributed to the issue where a workflow run just started by a user could get stuck in the "Submitting" state. The issue is not fixed completely yet, although that situation is more rare now.
  • 1938 Fixed an issue where a workflow run could get stuck in the "Queued to run" state if the user who started that run is near the storage quota limit.
  • 2013 Fixed the primary cause of the issue where the message "Failed to send updates to platform: [Errno 32] Broken pipe!" appeared repeatedly in the run log. That warning may still appear rarely, although it does not indicate a run error.
  • 1562 Fixed an issue where saving a block in its configuration dialog reset or unset its input values specified in the Block properties pane.
  • 1997 Fixed an issue with the Run setup pane where you could unintentionally unset a parameter, because for parameters of certain types (matrix, list, and some others) clicking outside the input field or pressing Enter while editing the parameter reset its value instead of accepting edits as intended.
  • 1868 Fixed an issue where a driver block in a running workflow started a loop that cannot complete because some of the loop body blocks do not receive all their input values, after which the run waited indefinitely for that loop. Now driver blocks check their loops before start and cease to run if required input data cannot enter the loop - this is possible, for example, if a block in the loop body receives some input from a Condition block. Also, if some block in the loop body cannot run (for example, one of its required input ports is not connected), a warning is issued to the run log pointing to the faulty block.
  • 1764 Fixed an issue with the Block properties and Composite properties panes where adding many ports to workflow parameters could cause incorrect UI behavior or unresponsiveness.
  • 1764 Fixed an issue with the Composite properties pane where you could create a port with a name that contains only space characters.
  • 2017 Fixed an AppsHub issue with the stock app UI where you could not set a value to a matrix type parameter if that parameter has no default value.
  • 1956 Fixed an issue where you could not open, move, or delete a folder copied from a run results folder.
  • 1996 Fixed a UI issue with the tab bar, which sometimes did not display the left and right shift buttons and the scrollbar when you have many tabs open.
  • 1266 Fixed a minor UI issue with workflow navigation in run tabs where you could not go inside a composite block with a double-click (like in workflow tabs) and had to always click the icon on the block to view its contents.

pSeven Enterprise upgrade notice

Since v2022.04, pSeven Enterprise license counts the number of AppsHub users and Studio users separately - the limits for them are set independently by the license. The type of a user depends on access permissions you set for that user. This also allows creating admin-only accounts, which do not have user access permissions - that is, cannot sign in to pSeven Enterprise as users. Admin-only accounts are not counted by the license, so you can create a number of such accounts as needed.

Licenses issued for previous pSeven Enterprise versions count only the total number of users, so they are not compatible with v2022.04. You will need to request an updated license key and have it ready before you begin the update - so you can update the license immediately after deploying v2022.04.

Important

Get an updated license key for pSeven Enterprise v2022.04 before you begin the upgrade. You will have to remove the existing key and add the new key immediately after upgrading to pSeven Enterprise v2022.04. Otherwise, user sign-in to pSeven Enterprise after the upgrade will be blocked until you replace the license key in your deployment.

The advisable upgrade sequence for pSeven Enterprise v2022.04 is:

  1. pSeven Enterprise licenses are bound to a specific Installation ID. Get and save your Installation ID as described in section License of the pSeven Enterprise deployment guide.
  2. Request a license update for your Installation ID. Postpone the upgrade to v2022.04 until you receive an updated license key.
  3. Having received the new license key, upgrade to v2022.04 as described in the Version upgrade guide.
  4. Immediately after the v2022.04 deployment finishes, sign in as admin and open the Licenses page of the administrative interface (URL: <admin URL>/app_auth/license/).
  5. Remove the existing license. Licenses from previous versions are not compatible with pSeven Enterprise v2022.04 and may cause issues later.
  6. Add the updated license key at the Add license page (URL: <admin URL>/app_auth/license/add/).

After replacing the license key, verify that you can sign in as a user. If you cannot sign in or encounter other issues during the upgrade, contact the DATADVANCE support team for advice (see Technical support).

After you complete the v2022.04 deployment and license update, consider also updating user permissions and configuring user and admin accounts as described in section User management of the pSeven Enterprise administration guide.

v2022.03

Updates

  • 1932 More tab bar enhancements:
    • When adding a large number of tabs, their widths are proportionally reduced in order to fit more tabs into the bar, until tab width reaches a certain minimum.
    • When tabs fill the pane, left and right shift buttons appear to enable mouse-click scrolling through tabs.
    • 1933 The tab bar menu now contains a list of all open tabs, so you can simply select a tab name from the list to navigate to the desired tab.
    • In addition to scrolling tabs with your mouse wheel, you can also use swipe gestures - for example, on Mac.
    • The new tab button is no longer stuck at the far right.
  • 1963 Upgraded the embedded pSeven Core package to version 6.31.1. For details on recent improvements and changes, see the pSeven Core changelog.
  • 912 Tooltips on links that may cause errors or unexpected behavior when running the workflow now display all warning and error details at once, instead of displaying only one of those messages.

Deployment and administration

  • 1895 1894 Updated guidelines. The pSeven Enterprise deployment guide was becoming overloaded, so it was split into several guides:
    • The pSeven Enterprise deployment guide now describes only important deployment steps. This guide was also updated to address a number of issues that may arise when installing pSeven Enterprise.
    • The Administration section contains an initial version of the support and maintenance guide.
    • Technical support contains support contacts and troubleshooting instructions.
    • Branding explains branding-related customizations.
  • 1731 Updated the sizing.* parameter descriptions in the pSeven Enterprise deployment configuration file values.yaml to better explain their role.

User block development

  • 1522 Your block can now read data from the shared data storage - see section Block assets for details.
  • 1945 Fixed an issue where pSeven Enterprise sometimes deleted folders created by the block in its data storage (DA__P7__BLOCK__DATA_DIR).
  • 1161 Fixed an issue where port values in the Results pane were always highlighted red (invalid) if the port data type is a custom table with predefined columns.

Bugfixes

  • 1721 Resolved the most prominent of stability issues that caused the "Task scheduler service restart" and "No such file or directory" errors when running a workflow. The fixes applied do not remove those errors completely yet, though significantly reduce the probability of them appearing during a workflow run.
  • 1975 Fixed an issue that caused the "Block process failed to start" error when running a workflow.
  • 1936 Fixed an issue where users who exceeded their storage quota could not stop a workflow run in progress, and that run continued with many errors in the run log.
  • 1924 Fixed an issue that caused damage to the workflow when in a situation where the user's storage quota is exceeded.
  • 1906 Fixed an issue where the block configuration dialog failed to open, displaying an empty window along with a "Block edit session failed" error message.
  • 1876 Fixed an issue with the Functions pane in the Python script block configuration dialog where pressing Enter while the cursor is in the Function name field removed one of the function arguments.
  • 1880 Fixed a regression issue from v2022.02 where moving a block to another workflow using cut and paste would not remove the block from the source workflow, until you refresh the browser tab with pSeven Enterprise or reopen that workflow.
  • 1941 Fixed UI issues with long names in the Links dialog. Long names now have an ellipsis to fit in the dialog width, and full names can be viewed by hovering over the name in the list.
  • 1893 Fixed an issue where a workflow run tab could remain in the loading or waiting state until you refresh a web browser tab with pSeven Enterprise.
  • 1179 Fixed an issue that resulted in an error message when double-clicking a workflow run tab while the run is initializing.
  • 1872 Fixed an issue with the Block library pane where a block update notification in Workflow library did not appear until you select that block in the pane.
  • 1775 Fixed an issue where the workflow run log sometimes did not update for about half a minute due to a log file initialization error.
  • 1146 Fixed an issue due to which the command for automatic layout of blocks in the workflow could place some blocks very far from the rest.
  • 1867 Fixed an issue where the search function in the Russian language version of Help ignored English words.

v2022.02

Updates

  • 1672 Updated Links and Autolink dialogs:

    • The Links dialog now supports port groups. You can collapse and expand the list of ports in each group.
    • When linking a driver block, you can select which ports to display: driver only, input (output) only, or all.
    • The Autolink dialog now suggests links in one direction only, from source to target block. Its old version, which suggested links in both directions, proved to be inconvenient for linking blocks with many similarly named ports.
    Port grouping in the Links dialog

  • 1902 Upgraded the embedded pSeven Core package to version 6.30. For details on recent improvements and changes, see the pSeven Core changelog.

Bugfixes

  • 959 Fixed an error when running a workflow with a Composite block that has a batch input port added to workflow results.
  • 1809 Fixed an issue where the recent list on the Start tab was emptied if you launch many workflow runs in succession.
  • 1900 Fixed an issue where editing a workflow after applying automatic block layout could unpredictably change the layout.
  • 1848 Fixed an issue where UI was becoming unresponsive if you move blocks in the workflow using keyboard.
  • 1792 Fixed an incorrect "You are trying to access an item that no longer exists" warning appearing when you copy blocks or notes in the workflow.
  • 1861 Fixed an issue with renaming workflow runs and items in the workflow Files folder where it was impossible to select text with a mouse during rename.
  • 1871 Fixed an issue where sometimes you could not rename a folder immediately after copying it.
  • 1702 Fixed a UI error that could be seen in the browser console when you open a block or edit port values in the Block properties pane.

v2022.01

Updates

  • 1078 Redesigned the tab bar for more convenience:

    • No more multi-row tab stack clutter. Tab titles always have the same width and are always displayed in a single line, which you can scroll with your mouse wheel, when there are too many open tabs to show all their titles at once.
    • 1540 Run tab titles use colored icons, which indicate run status.
    • Added tab bar menu with a few useful commands.
    The new tab bar

  • 1540 The main toolbar in a run tab displays current run status.

    Run status in the run toolbar

  • 1804 The Explorer pane displays your total and available storage space in its title bar.

    Storage information in Explorer

  • 1326 When you select one or a few blocks to view their log messages, the Run log pane title displays names of selected blocks, so you can always tell which logs you are viewing at the moment.

  • 718 You can rename workflow runs in Explorer - for example, to tag important or debug runs.

    Note that you cannot rename a run in progress. To create a new run with a custom name, use the New run command, which prepares a run but does not immediately start it. Then while your new run is in configuration, rename the run folder in Explorer.

  • 1722 You can view log files, Python scripts, CSV files, JSON and YAML data files in pSeven, in addition to previously supported Markdown and plain text files. To view a file, double-click it in Explorer.

  • 1864 Upgraded the embedded pSeven Core package to version 6.29. For details on recent improvements and changes, see the pSeven Core changelog.

Deployment and administration

  • 1225 User storage quotas. A deployment administrator can set file storage quota which applies to all users (as default), and override that default with user-specific quota settings. Quota management requires disk quota support on the file storage server - see section Enabling user storage quota in the pSeven Enterprise deployment guide for details.

Bugfixes

  • 1847 Fixed a workflow publishing issue where the published app could not run, if a block in the app's workflow reads files from the workflow directory.
  • 1870 Fixed an issue where refreshing a browser tab with pSeven Enterprise or AppsHub sometimes entered a loop, indefinitely redirecting you to the sign-in page and back.
  • 1753 Resolved a stability issue that could lead to errors appearing when you open a block, after which the block UI did not load - but could load successfully the next time you open the block.
  • 1873 Fixed a UI regression issue from v2021.12 where pSeven sometimes displayed a run that has already finished as a queued run or a run in progress, and you had to reopen the run tab to fix that.

v2021.12

Updates

  • 1667 Added initial support for the TensorFlow machine learning (ML) platform: in Python script blocks, you can use TensorFlow (import tensorflow) and Keras API (from tensorflow import keras) to develop and train ML models. Also added lasio (import lasio) to read and write Log ASCII Standard (LAS) files.

    Using TensorFlow, Keras API, or lasio in a Python script block that is configured to run on a Windows extension node is currently not supported.

  • 631 You can now test Python script blocks without running the workflow - see section Testing the script in the Python script block guide.

  • 1667 1659 Upgraded the embedded pSeven Core package to version 6.28. For details on recent improvements and changes, see the pSeven Core changelog.
  • 1520 Added logging to the stock app UI in AppsHub, so you can see progress of an app run, its error details, and so on. To view or download app logs, click the Log button in the app page header. Previously published apps do not get this update automatically, but you can re-publish them with the new version of the stock app UI.
  • 1465 You can now see folder (workflow, run) size in Explorer - select a folder, then expand the properties pane at the bottom and click calculate to get the selected folder size.
  • 1849 To prevent clutter in the workflow run log, the Run logs pane displays memory usage messages from blocks only if you enable showing the debug log in the pane's menu.
  • 1529 Error toast messages no longer disappear from screen by timeout - you close them manually, so there is no risk of missing an important error message. Information messages are automatically closed after a certain time, same as in previous versions.
  • 1090 Links that may cause errors or unexpected behavior when running a workflow are highlighted orange (warnings) and red (errors). Warning and error details are available in link tooltips.
  • Updated pSeven Enterprise Help:

Deployment and administration

  • 1806 Fixed a deployment issue where the command to generate secrets failed due to a configuration error.
  • 1071 Fixed an issue with pSeven Enterprise admin interface where searching for a user caused an error.

Bugfixes

  • 1799 Fixed an issue where autolinking blocks could unexpectedly remove manually created links between those blocks.
  • 1817 1468 Fixed an issue with the Run logs pane where it was not showing logs of the selected block, or was showing logs of a different block than the selected one.
  • 1654 Fixed an issue where error messages in a workflow run log were too wordy and always included full traceback, often making it harder to identify the error cause.
  • 1789 Fixed block sorting in the Block library pane.
  • 1265 Fixed various issues with keyboard navigation in the Explorer pane.
  • 1814 Fixed a UI issue with the Explorer pane, which was too wide even when resized to its minimum.
  • 1805 Fixed a UI issue with Run overview on the Start tab where it kept displaying the "Loading..." message instead of the usage tip text if there are no workflow runs that a user can view, causing confusion.

v2021.11

Updates

  • 1270 Mass uplinking and adding ports to workflow parameters is now more convenient: in the Block properties pane, you can select multiple ports in Inputs or Outputs section, then use the and buttons that appear in the section header. adds uplinks by default, however if all selected ports are uplinked already, removes their uplinks. Similarly, excludes selected ports from parameters, if all of them are parameters already. And, since an output port cannot be a parameter, the button appears in Inputs only.
  • 1734 The Block library pane loads noticeably faster in the case when the library contains blocks with many ports and a number of configuration presets.

  • 1624 During a workflow run, the Workflow contents pane shows block memory usage, and blocks also regularly report their peak and average memory consumption to the run log.

    Block memory usage in the Workflow contents pane

  • 1543 Run status indicators on the Start tab are slightly redesigned to unify their style and make color scheme consistent with the background color scheme used on run tabs.

    Run status indicators redesigned in v2021.11

  • 1727 Completed workflow runs take less storage space because intermediate data files are automatically deleted when a run finishes.

  • 1726 Changed the default parallelization ratio of batch-processing Composite blocks from 4 to 2 (the @Number of parallel workers setting). This is done to avoid situation where a workflow run that includes nested parallel Composite blocks requests too much resources by default, forcing you to manually edit settings of all Composite blocks to reduce the number of block instances created due to parallelization.
  • 1633 Partially resolved workflow run performance issues caused by using List and Dict data types to pass big volumes of data through links between blocks. As a result, delays in workflow execution were reduced but not completely removed yet.
  • 1744 Removed a distracting warning pop-up that appeared in certain cases when you add or remove an uplink in the Block properties pane.

Deployment and administration

  • A deployment administrator can now control resource usage with the following new settings:
    • 1739 The Max blocks per run setting limits the number of block instances that may be created in a single workflow run started by any user. Use it to avoid situations where a single user consumes all cluster resources by running a workflow that creates a high number of block instances - for example, due to extensive usage of Composite blocks with enabled parallel processing. Note that each block nested into Composite with parallel processing creates multiple instances when running, and the number of those instances is controlled by user through the Composite block settings.
    • 1756 The Max open blocks setting limits the total number of blocks that can be open for editing at the same time, considering all blocks opened by every one of the users. Use it to avoid situation where the number of open blocks is high enough to consume most of cluster resources, which causes block edit and save errors as well as various other issues.
  • 1716 The task executor settings in the pSeven Enterprise deployment configuration file (the sizing.htcondorexecuteResources.* parameters in values.yaml) now set resource requests for that service instead of resource limits. This improves the service stability in cases when its actual resource consumption (the amount of resources used by running blocks) exceeds the expected one (the request): the service can now use additional resources as long as they are not claimed by other services, instead of shutting down as soon as it reaches the limit. For further information about new settings, see the pSeven Enterprise upgrade notice below.
  • 1720 1761 Adjusted several deployment settings in the Helm chart to improve pSeven Enterprise stability:
    • Increased the number of user interface service (appserver) replicas from 2 to 3.
    • Increased the number of task submit service (htcondorsubmit) replicas from 1 to 2.
    • Increased amount of requested resources for some services.
  • 1710 Removed the disableFileLock parameter from the pSeven Enterprise deployment configuration file (values.yaml) as it is intended only for troubleshooting in a few rare cases, while being dangerous and essentially useless in production deployments.
  • 1795 1796 Optimized UI to lower the number of requests to the pSeven Enterprise user interface service (appserver), reducing its load.
  • 1798 Fixed an issue where the user interface service (appserver) repeatedly ran out of memory and restarted while there is a user who keeps open a tab with an ongoing workflow run. Frequent restarts caused UI issues noticeable to all users and, in some cases, errors during sign in.
  • 1717 Fixed an issue with workflow run status notifications, due to which a few concurrently running workflows could overload the pSeven Enterprise user interface service (appserver), causing UI unresponsiveness and incorrect run statuses being displayed on the Start tab.
  • 1690 Fixed an issue with published user blocks where pSeven Enterprise deployment failed if the folder name of some published block contains a space.
  • 1718 Fixed extension node deployment issues:
    • After installing Poetry, node deployment could fail to install Python packages due to an issue with Poetry cache.
    • Node deployment could fail if you specify a relative path to the installation or temporary files directory when running bootstrap.bat.

User block development

  • 1719 Fixed section Publishing a custom block in the block development guide: added missing instructions about setting access permissions for published blocks.

Bugfixes

  • 1798 Fixed an issue where watching an ongoing run in a tab disrupted general UI functionality, so the UI became unresponsive for all users, and some users also experienced errors during sign in.
  • 1711 Fixed unexpected slowdowns in workflow editing, which were noticeable when you have several workflow tabs open.
  • 1750 Fixed an issue where a large workflow with many links open in a background tab caused unexpected slowdowns and UI unresponsiveness when editing workflows in other tabs.
  • 1760 Resolved a stability issue that appeared when several users actively edit different blocks in their workflows at the same time. In particular, that issue caused errors when opening a block or saving block configuration changes.
  • 1697 Fixed an issue with the Links dialog where it did not show ports that you have recently added to one of the blocks you are linking.
  • 1793 Fixed an issue with the Links dialog where lists of ports disappeared when linking two blocks of the same type.
  • 1757 1148 Fixed issues with the Block properties pane where uplinking many ports caused slowdowns, incorrect UI behavior and errors.
  • 1149 Fixed an issue with the Block properties pane where it did not properly update the list of ports when you select different blocks in the workflow and displayed ports of a previously selected block.
  • 1783 Fixed an issue with the Block properties pane where it displayed uplinked ports as not uplinked.
  • 1150 Fixed an issue where you could not uplink multiple ports to the same port of a Composite block using buttons in the Block properties pane.
  • 1730 Fixed an issue where a block input value specified in the Block properties pane was not saved if you deselect that block or select another block immediately after input.
  • 974 Fixed an issue where input parameter values in the Run setup pane were not updated when you change parameter presets.
  • 1704 Fixed an issue where uploading large files caused a 502 Bad Gateway error.
  • 1693 Fixed delays in log output in the Run logs pane.
  • 1791 Fixed an issue where a running workflow, which contains Composite blocks with enabled parallel processing, did not stop when a block nested into one of those Composite blocks fails to start, which caused various workflow errors later on.
  • 1790 Fixed an issue with block working directory synchronization, due to which workflow runs occasionally stopped with a file access error.
  • 1790 Fixed an issue where a workflow run stopped with a "Task failed" error immediately after start.
  • 1796 Fixed a UI issue where a run tab stopped updating after you switch to another tab and back - for example, when you start several runs and switch between them.
  • 1717 1543 Fixed an issue with the Start tab where it displayed workflow run status incorrectly due to unexpected delays in status update.
  • 1784 Fixed a UI issue where the Start tab did not show the list of runs in Run overview if you do not interact with UI after opening a new Start tab.
  • 1688 Fixed some of the UI issues that appear on low-resolution displays (tablets and laptops), or when you resize the browser window with pSeven Enterprise.
  • 1749 Fixed an issue with high resolutions (for example, on 4K displays), which caused delays when switching between tabs in the pSeven Enterprise UI.
  • 1732 Fixed an issue with the Block library pane where the Workflow library section sometimes did not show block update notifications even if a newer version of a block is available from the Common library.
  • 1682 Fixed several issues that caused incorrect behavior when editing workflow annotations - for example, the text you edit could be lost or added to another annotation instead of the one you are editing.
  • 1438 Fixed a workflow copying issue where active workflow runs were "copied" with the workflow. A copy of the workflow, which was created while that workflow is running, contained runs that were incorrectly displayed as active (running) while in fact those runs in the copy are interrupted.
  • 1737 Fixed an issue where dragging a workflow in a run tab reset block startup counters to 0.
  • 1740 Fixed an issue where a workflow run that contains a Composite block with enabled parallel processing failed with an "unknown value id" error.
  • 1742 Fixed an issue where you could accidentally clear some result value in the Run setup pane, because the value reset button was displayed when you click a result despite that button is intended for input parameters only.
  • 1700 Fixed issues with the stock app UI in AppsHub:
    • The Stop button did not work after you refresh the app page.
    • The app UI did not clear the list of files in the Results pane once you start a new calculation.
  • 1801 Fixed a UI issue in the Explorer pane where a file upload progress bar remained at 0 if the cursor is away from the pane.

pSeven Enterprise upgrade notice

In the pSeven Enterprise v2021.11 deployment configuration file (values.yaml), resource settings for task executors specify resource requests instead of limits: the sizing.htcondorexecuteResources.limits.* parameters are replaced with the sizing.htcondorexecuteResources.requests.* parameters. Note that requests (sizing.htcondorexecuteResources.requests.*) are now required while limits (sizing.htcondorexecuteResources.limits.*) are optional, and the v2021.11 deployment configuration file sets only requests parameters by default.

When transferring parameter settings from an existing deployment to the v2021.11 configuration file (see Upgrading pSeven Enterprise), there are two advisable ways to update resource settings for task executors:

  • To improve pSeven Enterprise stability under load, specify requests only. Set requests to the same values that were previously specified as limits. This allows task executors to temporarily exceed the expected resource consumption, using additional resources as long as they are not claimed by other services.
  • To reproduce behavior of previous pSeven Enterprise versions, specify requests and limits, and set both to the same values that were previously specified as limits. Note that with such settings, Kubernetes pods where task executors run will be given a QoS class of Guaranteed (see QoS classes in the Kubernetes documentation).

v2021.10

Updates

  • 580 The code editor in Python script blocks now supports search and a few other common functions - see section Keyboard shortcuts in the Python script block guide.
  • 1631 Ctrl+S or Cmd+S in Python script now saves the block.
  • 1396 Newly created workflows now contain the README.md file template. When publishing a workflow to AppsHub, you can use README.md to add descriptions of the app, its parameters and results. See Adding descriptions to the stock app UI for more information.
  • Improved editing and display of parameters and results in the stock app UI in AppsHub:
    • 1341 Non-default parameter values are now displayed in bold, so you can easily tell which parameters you have already changed.
    • 1341 Each parameter now has a button, which unsets the parameter value.
    • 1625 If a result value is too long to display it in the result field (for example, the value is a matrix, a long list or string), a button appears. Clicking it opens a dialog where you can view the full value.
    • 1625 Increased value preview length in result tooltips.
  • 1341 You can now unset input values in the Block properties and Run setup panes: click a value to edit, then click in the input field.
  • 1678 Pan action in a workflow tab now works more smoothly.
  • 1316 Blocks in workflows of example apps in AppsHub are now automatically updated to current versions upon a pSeven Enterprise version upgrade.
  • 1645 Added a beta tag to the GraphQL API command in the user menu to indicate that pSeven Enterprise GraphQL API is unstable and is not yet ready to be used in production.
  • Updated pSeven Enterprise Help:
    • 1687 Added section Keyboard shortcuts to the Python script block guide.
    • 1357 Updated license information and instructions in section License of the deployment guide as the pSeven Enterprise license is now deployment specific and bound to the Installation ID.
    • 1658 The Russian language version of pSeven Enterprise Help now contains a full changelog, the same as the English version.

Deployment and administration

  • 1357 pSeven Enterprise licenses are now deployment specific. Each new deployment has a unique Installation ID (which is now generated during installation) and requires a valid license bound to this specific Installation ID. An existing Installation ID is kept upon a pSeven Enterprise version upgrade, so a license previously issued for that Installation ID remains valid after the upgrade, within its licensing period. However if you upgrade an existing deployment that does not yet have an Installation ID, you will need to request an updated license, as the upgrade will generate a new Installation ID for your deployment. For further details, see the pSeven Enterprise upgrade notice below.
  • 1660 Fixed an issue where file storage requirements (see section NFS server in the pSeven Enterprise deployment guide) were not checked properly during installation, so deployment with an unsupported file storage continued but failed after a long timeout.
  • 1703 Fixed an issue with the extension node setup package, due to which a node could appear working after installation but stopped functioning later - for example, after a reboot or after some of the pSeven Enterprise services on the node had been restarted.
  • 1637 Fixed an issue where the OpenVPN port setting for extension nodes specified in the pSeven Enterprise deployment configuration file (the entrypoint.gateOpenvpn parameter in values.yaml) did not apply, requiring you to manually edit the OpenVPN configuration file (winnode.ovpn) on every extension node if you change the entrypoint.gateOpenvpn parameter during deployment.

User block development

  • 999 It is now possible to implement and use a Windows-only block, which starts on a Windows extension node both in edit (configuration) and run (execution) modes. A user configuring such a block should select a Windows node to run on in the Block properties pane prior to opening block configuration. Thus a block in edit mode will be able to access node-specific resources - for example, Windows-only software tools installed on the selected node.

Bugfixes

  • 1566 Fixed an issue where saving a block was taking too much time.
  • 1679 Fixed unexpected slowdowns when deleting files in Explorer, deleting blocks in the workflow or in the Workflow library pane.
  • 1677 Fixed an issue where pSeven Enterprise was becoming unresponsive if you edit a complex workflow containing many blocks and links.
  • 1032 Fixed an issue where changes in workflows and apps were not showing until you delete the browser cache, which happened after a pSeven Enterprise version upgrade.
  • 1684 Fixed an issue where a workflow failed to start after you update a block in the Workflow library to a newer version.
  • 1674 Fixed slow loading of the Block properties pane.
  • 1151 Fixed an issue with the Run logs pane where it sometimes did not show the log until you reopen the pane.
  • 1504 Fixed an issue where scroll position in the Run logs pane was reset if you switch between document (workflow, report) tabs.
  • 1502 Fixed an issue with the Block properties pane, which was incorrectly highlighting an optional input port as a port required to start the block.
  • 1642 Fixed an AppsHub issue where refreshing an app page reset the app UI and unexpectedly stopped the app if it was running calculation.
  • Fixed issues with the stock app UI in AppsHub:
    • 1557 The UI sometimes did not show results of a completed calculation.
    • 1341 The UI blocked editing of a parameter if it has no default value.
    • 1642 A parameter value could be ignored if that parameter is an input port of some Composite block in the app's workflow.
    • 1642 The UI did not clear results of a previous calculation when you start a new one, causing confusion.
  • 1666 Fixed an issue with environment settings on extension nodes where pSeven Enterprise incorrectly set MKL_NUM_THREADS, NUMEXPR_NUM_THREADS, OMP_NUM_THREADS, and several other environment variables for the process it starts on the node. Those settings limited the number of CPU cores provisioned to the process, which could cause errors when running multithreaded CAD or CAE packages.
  • 1662 Fixed an issue where you could not copy a shared workflow if another user is editing a block in that workflow at the moment.
  • 1557 Fixed an issue with REST API where results of a finished run could be empty.
  • 1324 Fixed an issue where a workflow run could enter a deadlock, if the workflow contains a Python script block that declares a function but never calls it in the script, and the response port corresponding to that function has a link connected to it and actually receives function values, which the block does not use.
  • 1656 1592 Fixed several bugs that sometimes caused unexpected behavior of common file management commands in Explorer - for example, the sequence of Ctrl+C and Ctrl+V hotkeys, which commonly creates a copy of the selected file or directory, was not working as intended.
  • 1639 Fixed minor UI issues in the Composite properties pane.
  • 1172 Fixed unexpected UI behavior when quickly opening and closing block configuration dialogs.

pSeven Enterprise upgrade notice

Since v2021.10, pSeven Enterprise licenses become bound to a specific Installation ID. A unique Installation ID is automatically generated during installation, and each deployment requires a license key valid for its Installation ID. The license key contains information about the ID, and can only be used for the deployment with that ID.

Deployments of pSeven Enterprise v2021.10 and future versions will keep their once generated Installation ID upon a version upgrade, so a deployment specific license for them has to be issued only once.

Deployments of prior versions (v2021.09 and earlier) have no Installation ID. For such deployments, a new ID is generated during a version upgrade, so the previously used license becomes incompatible as it was not generated for a deployment with a specific Installation ID.

If you upgrade an existing deployment to v2021.10, you will need to request an updated license valid for your newly generated Installation ID. To get the ID, sign in as admin after v2021.10 installation finishes and copy the ID string from the admin page header (see section License in the pSeven Enterprise deployment guide for more details). This is a one-time update required to switch to the new version of license, which supports Installation ID; future upgrades will keep the same ID.

v2021.09

Updates

  • 1607 Added initial ONNX support: the Python script block now provides the ONNX Runtime (import onnxruntime), which enables the usage of ONNX models from deep learning frameworks such as PyTorch and TensorFlow as well as classical machine learning libraries such as scikit-learn, XGBoost, and other.

    Using the ONNX Runtime in a Python script block that is configured to run on a Windows extension node is currently not supported.

  • 1518 The stock app UI in AppsHub now supports uploading input files and downloading files from app results.

  • Extended the Python script block functionality:
    • 1623 Connect to MongoDB databases with PyMongo (import pymongo).
    • 1607 Retry function calls (for example, HTTP requests) with backoff (import backoff).
    • 1594 Establish SSH connections with Paramiko (import paramiko).

  • 1539 Added caching support in the Python script block. For example, if your block runs in a loop, you can cache certain data or objects to avoid initializing them every time the block runs. See Caching Python objects in the Python script block guide for details.

    Note that the cache should not be used as a persistent storage, because it may be reset between block runs.

  • 1573 Added a few common convenience features to the code editor in the Python script block:

    • code folding support
    • highlight all occurrences of a word under the caret
    • Ctrl+/ to comment the current line or selected lines

  • 1563 NumPy arrays sent to output ports of a Python script block are now required to be 2-dimensional in order to avoid errors caused by inability to convert an array to the matrix data type. The support for N-dimensional arrays may be added in future versions.

    This change may require you to update your scripts in Python script blocks - see section Compatibility issues for details.

  • 1567 Upgraded the embedded pSeven Core package to version 6.24. For details on recent improvements and changes, see the pSeven Core changelog.

  • 1058 The Run setup pane now supports tree-like grouping of parameters and results. For example, parameters named geometry.x and geometry.L are displayed as a collapsible group geometry containing inputs x and L.
  • 1525 Blocks in the Workflow library pane are now sorted by name, with different versions of the same block sorted in the version decreasing order.
  • 1487 Added block help shortcuts to the Block library pane: hover over a block in the library and click the :fontawesome-solid-question-circle: icon that appears.
  • 986 Improved AppsHub search to find search term occurrences in app descriptions. Also, you can search by username to find all apps published by a certain user.
  • 1496 Updated design of title bars in block configuration dialogs for consistency. Block title bar now always contains the Save and Help buttons. The Save button now saves configuration but does not close the dialog (use Alt+F4 to close). The Apply button is removed.
  • 1138 Added port tooltips in the Block properties and Composite properties panes, explaining the current state of each port.
  • 1515 If you open a folder in the Explorer pane and it takes a long time to load and display folder contents, the pane now shows a loading message.
  • 809 To avoid errors, a driver block can now be grouped into a Composite block only if you also group to the same Composite the entire body of a loop controlled by that driver block (select the driver block and all blocks in the loop before grouping).
  • 1416 Added a message explaining the workflow run error caused by incorrect usage of conditional branching inside a Composite block with enabled batch processing.
  • 863 Updated tooltips in the Explorer pane to better explain the use of commands, and added missing tooltips.
  • 1515 Removed the Open button from the collapsible information pane in Explorer as redundant.
  • 1547 Disabled the workflow run hotkeys F9 and F10 as they often conflict with browser hotkeys.
  • Updated pSeven Enterprise Help:
    • 1603 Updated the description of Kubernetes cluster resource requirements in the Prerequisites section of the pSeven Enterprise deployment guide (moved: see Requirements).
    • 1114 Reworked the deployment diagram in the pSeven Enterprise deployment guide and added the service interaction diagram showing pSeven Enterprise services, network connections and ports in more detail.
    • 1499 Extended the pSeven Enterprise deployment guide with detailed instructions on how to configure an NFS server for file storage - see NFS server.
    • 1517 Added a note about required ports to the Extension node deployment guide (see section Before installation).
    • 1598 Added description of pSeven Enterprise branding options - see Branding.
    • 1597 Added notes on required user permissions to the User blocks guide.
    • 1628 Updated the Python script block guide with the Working directories section describing how to configure the block working directory.
    • 1539 Added section Caching Python objects to the Python script block guide.
    • 1489 Added Python examples to the REST API guide.

Deployment and administration

  • 1577 The Technical support section now explains what information you should collect and provide when contacting support with deployment issues.
  • 1574 Optimized resource requirements for pSeven Enterprise services in the Helm chart so the services are now easier to distribute among the Kubernetes cluster resources. Requirements for non-replicated services are now set in a specific way as required to give their pods a QoS class of Guaranteed, improving their stability (see QoS classes in the Kubernetes documentation).
  • 1582 In the pSeven Enterprise deployment configuration file (values.yaml), you can now set annotations which are added only to Kubernetes jobs and pods created by jobs - see the annotations.job parameter. The annotations added by annotations.job override annotations from annotations.all.
  • 1589 Removed memory limits for post-install jobs started by Helm, which run once to check and finalize the pSeven Enterprise installation. This change is done to avoid deployment failures, which sometimes occurred because a one-time post-install job exceeded its memory limit and was stopped by Kubernetes. Also, post-install jobs are now allowed to restart up to 3 times in case of errors.
  • 1588 Increased the ephemeral storage size for the pSeven Enterprise edge router service to avoid its restarts, which sometimes occurred under a peak load because the service log size exceeded storage size and Kubernetes destroyed the service.
  • 1598 Added branding options: you can now set a custom logo on the pSeven Enterprise sign-in page and in the navigation bar, and add a custom Getting Started page. See Branding for details.

User block development

  • 1597 Getting block source code (a .p7protoblock folder) from the block library now requires the app_auth | user | Can develop blocks user permission. New and existing users will have this permission by default, and a deployment administrator can revoke it. Users without this permission will be unable to drag blocks from the block library to the Explorer pane as well as add blocks from Explorer to the Workflow library pane.
  • 1578 Fixed an issue where you could not add a block from the Explorer pane to the workflow library if the block manifest did not define default values of all ports.

Bugfixes

  • 1235 Fixed an issue where the block UI sometimes did not load when you open the block configuration dialog.
  • 1586 Fixed the "Task scheduler service restart" error, which sometimes occurred while running a workflow.
  • 1521 1640 Fixed errors and slowdowns when viewing large run logs.
  • 1541 Fixed an issue where running a workflow with a Python script block that produces many log messages resulted in a "Memory limit exceeded" error, regardless of the memory limit specified in block configuration.
  • 1385 Fixed an issue in the Python script block where function arguments in the Functions pane were listed in alphabetical order instead of their declaration order, causing confusion.
  • 1630 Fixed an issue in the Python script block where using the concurrent call syntax with 3 or more functions (for example, (F1&F2&F3)(x1, x2, x3)) caused an error.
  • 1579 Fixed an issue with the Python script block where writing a large amount of data to the block's temporary storage directory (api.tempdir) could cause an error in all running workflows.
  • 1563 Fixed an issue in the Python script block where it was possible to send an N-dimensional NumPy array to an output port, causing an error because an N-dimensional array cannot be converted to the matrix data type.
  • 1351 Fixed an issue with the variable quick add syntax in the Python script block where ranges in names of variables were sometimes parsed incorrectly.
  • 1351 Fixed an issue in the Python script block where names of variables added using the quick add syntax were not properly validated.
  • 1051 Fixed an issue where changing the value assigned to an uplinked port did not change the value assigned to the corresponding port of a parent block as intended.
  • 809 Fixed the inability to group a block into a Composite block, if there are multiple links connected to the same port of the block you are grouping.
  • 1279 Fixed an error while running a workflow containing two or more Composite blocks with similar names, which have ports added to workflow results.
  • 1381 Fixed an AppsHub issue where re-publishing a workflow changed its app ID, which required you to also update all REST clients using that app.
  • 1381 Fixed an AppsHub issue where you could overwrite an existing app by publishing another workflow under the same name, even if you are not the owner of that app.
  • 1043 Fixed an issue with the stock app UI in AppsHub where descriptions of results appeared only after you run the app.
  • 907 Fixed an AppsHub issue where you could not open an app if its description was saved with encoding different from UTF-8.
  • 584 Fixed incorrect behavior of the Publish workflow dialog - for example, it allowed you to click the Publish button multiple times, resulting in repeated publishing of the workflow.
  • 1261 Fixed an issue where you could not create a copy of a read-only shared folder.
  • 1593 Fixed an issue where deleting a workflow sometimes caused an error, after which you could not delete that workflow.
  • 1569 Fixed an issue where workflow run folders were occupying more storage space than intended because they contained hidden system files.
  • 1581 Fixed a sign-in issue where re-opening the pSeven Enterprise sign-in page after a successful sign-in did not properly redirect you to pSeven Enterprise or AppsHub.
  • 864 Fixed an issue where double-clicking a workflow, run, or report tab title did not open the corresponding folder in the Explorer pane as intended if the pane is collapsed.
  • 1523 Fixed an issue where you could not preview the app UI in a workflow (open the index.html file from the workflow UI folder in Explorer) if the workflow name or path to the workflow contain special characters.
  • 1523 Fixed an issue where you could not open an HTML file located in a run directory or the workflow Files folder.
  • 1488 Fixed incorrect status code and error message in a response to an invalid REST API file upload request.
  • 1063 Fixed an issue with renaming files and folders in the Explorer pane where it was impossible to select text with a mouse during rename because this started a drag-and-drop action.
  • 519 Fixed a UI issue with the parameter preset selector in the Run setup pane, which sometimes displayed duplicated presets.

Compatibility issues

This update requires NumPy arrays sent to output ports of a Python script block to be 2-dimensional, which in particular disables the output of 1-dimensional arrays. In previous versions, an 1-dimensional NumPy array was converted to a single-column matrix on output. In the new Python script block version coming with pSeven Enterprise v2021.09, that conversion has to be done explicitly in your Python script code.

If your workflows contain Python script blocks that output 1-dimensional NumPy arrays, you will need to update your Python code in those blocks accordingly before upgrading the Python script block in the workflow library to the latest version. Note that the Python code that converts 1-dimensional arrays to 2-dimensional before output is going to work correctly with either block version.

v2021.07

Updates

  • 1554 You can now enable or disable run-time resource management in your deployment when installing pSeven Enterprise or upgrading it to a new version. In pSeven Enterprise v2021.07, resource management is disabled by default. See the pSeven Enterprise upgrade notice below for details.
  • 1516 Adjusted resource requirements for pSeven Enterprise services in the Helm chart to their actual resource consumption based on the recent resource usage statistics gathered from real pSeven Enterprise deployments.
  • 1537 1553 1546 Improved pSeven Enterprise stability in many cases where users experienced issues when configuring blocks, editing or running workflows.
  • 1368 pSeven Enterprise administrators can now create users who have AppsHub access only. Such users may run apps from AppsHub but cannot create, edit, or run workflows.
  • 1269 When you download a workflow or a folder containing some workflows, you can now skip downloading the run results stored with workflows to speed up download and reduce its size.
  • 476 You can now upload workflows or folders in ZIP archives and extract them in pSeven Enterprise using the new Unzip command from the menu in Explorer.
  • 1497 Upgraded the embedded pSeven Core package to version 6.23. For details on recent improvements and changes, see the pSeven Core changelog.
  • 1497 The Python script block now provides optional pandas dependencies like pytables, tabulate, and other.
  • Updated pSeven Enterprise Help:
    • 1498 Added section Workflow guides, which explains pSeven Enterprise concepts and provides a general user guide.
    • 1448 1470 Added section Block guides, which provides help for the Python script and Condition blocks.
    • 1439 Added section User blocks guide, which provides an updated block development guide.
    • 1513 Minor improvements in the pSeven Enterprise deployment guide.

User block development

  • 1419 Fixed an issue where pSeven Enterprise displayed an empty Common library pane after a block with an invalid manifest had been published.
  • 1561 Fixed an issue due to which it was impossible to implement a block that allows changing default values of its ports.
  • 1439 Added the block development guide to pSeven Enterprise Help - see User blocks guide.
  • 1439 Removed the outdated User Block Example from AppsHub. Note that if you upgrade an existing deployment, this example is not removed automatically. It is recommended to delete it manually after the upgrade - see section Compatibility issues for details.

Bugfixes

  • 1545 Fixed an issue where run-time resource management did not work as intended when running a workflow that contains a block configured to run on an extension node.
  • 1546 Fixed a deployment issue where annotations specified in the pSeven Enterprise deployment configuration file (the annotations.all parameter in values.yaml) were not added to some of the Kubernetes pods during deployment.
  • 1535 Fixed an issue where saving a script with a syntax error in the Python script block produced a generic error message instead of a message that locates the error.
  • 1463 Fixed an issue where you could add a function with no arguments in the Python script block, which caused an error when running the workflow.
  • 1250 Fixed an issue where you could not change access rights of a folder you have shared earlier.
  • 1035 Fixed an issue where you could not preview the app UI in a workflow (open the index.html file from the workflow UI folder in Explorer) if the workflow had been shared to you by another user.
  • 1278 Fixed incorrect UI behavior and various issues that occurred when you can no longer access a shared workflow because its owner had deleted the workflow or revoked your access.
  • 1250 Fixed errors caused by deleting a workflow that is currently running.
  • 1450 Fixed an AppsHub issue where you could not open an app if its name contains a dot.
  • 1377 Fixed an issue where the pop-up message displayed after publishing a workflow to AppsHub contained an invalid link to the app.
  • 1423 Fixed an issue where you sometimes could not save configuration in the Example user block.
  • 996 Fixed an issue where the Ctrl+V hotkey was not working in the workflow editor.

Compatibility issues

  • 1554 pSeven Enterprise v2021.07 disables run-time resource management by default. To enable the resource management features, which were introduced in v2021.06, set the sizing.enableResourceManagement parameter in the pSeven Enterprise deployment configuration file (values.yaml) to true. For details, see the pSeven Enterprise upgrade notice below.
  • 1546 The default maximum number of users (the maxUsers parameter in values.yaml) in pSeven Enterprise v2021.07 is increased to 256 (previously the default was 128). When upgrading, it is recommended to keep the new default value, unless a non-default value was specified in your existing deployment.

  • 1439 pSeven Enterprise v2021.07 no longer installs the User Block Example to AppsHub, as this example is outdated and is completely replaced by the new block development guide (see User blocks guide) and the example implementation of a user block available from the block library. However if you upgrade an existing deployment to v2021.07, the old example is not removed from AppsHub automatically. To avoid using that outdated example, it is recommended to manually delete it from the AppsHub data storage after you upgrade to v2021.07.

    • The AppsHub data storage location is specified by the storage.appdata.* parameters in the pSeven Enterprise deployment configuration file (values.yaml). Connect to the server specified by the storage.appdata.nfs.server parameter and locate the directory specified by storage.appdata.nfs.path. This is the AppsHub data storage directory.

    • To delete the outdated User Block Example, remove the User Block Example.p7wf subdirectory and all its contents from the AppsHub data storage directory.

      Warning

      Do not remove any subdirectories other than User Block Example.p7wf from the AppsHub data storage. Those subdirectories contain more example apps, as well as apps published by pSeven Enterprise users.

pSeven Enterprise upgrade notice

pSeven Enterprise v2021.07 deployment configuration file (values.yaml) is updated with a new parameter, sizing.enableResourceManagement, which you can use to enable or disable the run-time resource management features introduced in v2021.06 (default is disabled). This setting applies once during installation and cannot be changed afterwards, unless you re-install.

If you choose to enable resource management, configure it as described in the pSeven Enterprise upgrade notice for v2021.06.

If you choose to keep resource management features disabled, you can limit the maximum load by adjusting the sizing.htcondorexecuteSlots parameter, which sets the number of tasks that may run on a task execution service instance.

v2021.06

Updates

  • 1038 Improved run-time resource management:

    • Blocks now declare their memory and CPU requirements, and pSeven Enterprise uses this information to schedule block execution.
    • pSeven Enterprise sets run-time memory and CPU limits for blocks, allowing a block to use no more resources than it requires.
    • In Python script blocks, you can set per-block resource requirements in the new Resources section of the block configuration dialog.
    • User blocks can declare resource requirements in block manifest and can also implement per-block resource settings in their configuration dialogs.
    • If a block does not declare its resource requirements, default limits apply, set by an administrator in deployment configuration.

    Warning

    The resource management updates outlined above require you to update all blocks in workflows created in previous versions of pSeven Enterprise. To update a block, click the update notification next to the block's name in the Workflow library section of the Block library pane.

    If you do not update a block, the workflow will keep using an old block version, which does not declare resource requirements, so default limits always apply. Since the default limits are rather low, you may experience various issues if you run a workflow that contains old block versions.

  • 1456 You can now reduce the size of a workflow folder by removing the history of changes - open the workflow and use the Clear history command from the menu in the workflow edit toolbar.

  • 1507 Plotly is now available in Python script blocks.

  • 1353 To access files in Python script blocks, you can now use the api.rundir and api.wfdir variables that contain, respectively, the path to the current run directory and the path to the workflow folder. Import the api module (import api) to use. Both variables are pathlib.Path objects.

    Join paths

    The join operator for pathlib paths is /. For example, to get the full path to the file named file_name.ext, located at the root of the run directory:

    import api

file_path = api.rundir / 'file_name.ext'
    Files in the block's working directory

    To work with files located inside the block's working directory, you do not need any special methods - just use relative paths. This works because the block sets its working directory as the current working directory for the script it runs.

  • 1268 Workflow run folders now occupy less storage space.

  • 1466 Improved responsiveness and loading speed of the Block properties pane.
  • 1203 Improved autolinking: the Autolink dialog suggests links between ports with partially matching names, and can match ports from different port groups.
  • 917 Recently closed workflows and reports now appear on top in the list of recent documents, so they are easier to find.
  • 1471 Helm post-install hooks and jobs no longer disappear during deployment, so you can review their logs when needed.
  • 1471 pSeven Enterprise deployment no longer creates persistent volumes that require additional privileges.
  • 1433 Upgraded the embedded pSeven Core package to version 6.22. For details on recent improvements and changes, see the pSeven Core changelog.
  • 1477 Added more details about NFS server configuration to section File storage in the pSeven Enterprise deployment guide (moved: see NFS server).
  • 1500 Reworked the Version upgrade guide to avoid issues with re-using configuration files from an existing deployment. Until further notice, the recommended method to prepare a configuration file for an update is to manually transfer settings from the existing configuration.
  • 1021 Updated the REST API guide:
  • 1436 Updated the Russian language version of pSeven Enterprise Help to bring it in line with the English language version. To select your language, use the menu in the Help's top bar.

User block development

  • 1038 Due to the resource management updates, blocks should also be updated so they declare resource requirements, and users should be additionally notified to update blocks in their workflows. See the User block upgrade notice for details.
  • 1464 Fixed an issue where users could not update a block in their workflows if the name of the folder where developers have published the updated block version was not following a specific pattern.

Bugfixes

  • 1181 Fixed an issue where pSeven Enterprise could stop responding due to a malfunctioning task scheduler service.
  • 1452 Fixed an issue where a workflow with blocks configured to run on a Windows extension node often stopped with an error, if these blocks were nested into a Composite block with parallel processing enabled.
  • 1467 Fixed performance issues caused by adding a large file to the workflow Files folder.
  • 1073 Fixed an issue where editing a block always reset its previously assigned port values to default, as well as resetting related workflow parameters.
  • 1443 Fixed an issue where pSeven did not correctly create working directories for a Composite block with working directory set to indexed and parallel processing enabled.
  • 1413 Fixed an issue where pSeven sometimes did not update a run tab properly, displaying a run as active (running) while it has already finished.
  • 802 Fixed the unexpected UI behavior after opening several workflow tabs in rapid succession.
  • 1097 Fixed errors when deleting, moving or copying a folder that contains many subfolders.
  • 1440 Fixed an issue where it was possible to create duplicate links - for example, two links connecting the same port pair.
  • 1195 Fixed an issue where pSeven displayed incorrect values of block start-up counters on a run tab.
  • 1203 Fixed a few minor issues with the Links and Autolink dialogs.
  • 1413 Fixed an issue where pSeven kept displaying long-running operation pop-ups at the bottom.
  • 1432 Fixed an issue where a side panel with debugging information (Django admin panel) appeared in AppsHub and in pSeven Enterprise admin interface.

Compatibility issues

  • 1471 pSeven Enterprise v2021.06 changes the requirements to the file storage (NFS) server: now it supports NFSv3 only and requires disabling NFSv4 - see section NFS server). in the pSeven Enterprise deployment guide for details.

    Important

    Reconfigure the NFS server before upgrading to v2021.06, otherwise the v2021.06 deployment will fail.

  • 1038 Due to the resource management updates in pSeven Enterprise v2021.06, user blocks should also be updated so they declare resource requirements. Otherwise, the general memory and CPU usage limits set in the deployment configuration will apply. The general limits may be too low for some user blocks - for example, if a block processes large data arrays, it may run out of memory. Also, users should be additionally notified to update blocks in their workflows created in previous versions of pSeven Enterprise. For details, see the User block upgrade notice below.

  • 1038 pSeven Enterprise v2021.06 uses an updated deployment configuration file, which contains new parameters related to resource management. These parameters should be adjusted when upgrading to v2021.06. For configuration details, see the pSeven Enterprise upgrade notice below.

pSeven Enterprise upgrade notice

pSeven Enterprise v2021.06 deployment configuration file (values.yaml) contains several new parameters related to resource management. To ensure correct resource allocation and to avoid issues when running workflows, these parameters should be set deliberately and with regard to resources available in your deployment.

Follow the guidelines in section Upgrading pSeven Enterprise to generate a new configuration file for v2021.06 and transfer parameter settings from your existing deployment to the new file. In addition to this, find the following sizing.* parameters in the new file and adjust their values:

  • sizing.blockrunDefaultResources.memory - default memory usage limit for a block run task.
  • sizing.blockrunDefaultResources.cpu - default CPU usage limit for a block run task.
  • sizing.htcondorexecuteReplicas - the number of task execution service instances to create.
  • sizing.htcondorexecuteResources.limits.memory - total memory usage limit for a task executor.
  • sizing.htcondorexecuteResources.limits.cpu - total CPU usage limit for a task executor.

Limits set by the sizing.blockrunDefaultResources.* parameters apply to blocks that do not declare their specific resource requirements. Do not set these default limits too high as this leads to inefficient resource usage, which causes performance issues.

Values of the sizing.htcondorexecuteReplicas and sizing.htcondorexecuteResources.limits.* parameters should be selected carefully with regard to expected load and to resources that are in fact available in your deployment. For details, see comments to these parameters in the v2021.06 configuration file (values.yaml).

User block upgrade notice

User blocks developed for prior pSeven Enterprise versions do not declare their resource requirements as expected in pSeven Enterprise v2021.06. Due to this, default run-time memory and CPU limits will apply to these user blocks unless you prepare an updated block version and add it to the block library. The default limits may be too low for some user blocks, so users may experience issues when running workflows that contain old block versions.

Before you upgrade pSeven Enterprise to v2021.06, it is recommended to update each user block, adding run-time resource requirements to the block's manifest, and upload the updated version to the block library beforehand - so the block update is readily available to users once you upgrade to v2021.06.

  • Make a copy of your .p7protoblock directory, which will be the new block version. For example, if you have a directory named TheBlock-v0.p7protoblock, copy it to TheBlock-v1.p7protoblock.

  • In the copy (TheBlock-v1.p7protoblock) open the block's manifest (manifest.yaml) and edit the following parameters:

    • Under default_preset.template, add the resources parameter specifying the block's requirements as in the following example:

      default_preset:
    template:
        # Do not copy this example as is.
        resources:
            run:
                memory: 1024
                cpu: 250
        # Other template parameters follow...

      Note that the memory and CPU requirements must be set according to the real block memory and CPU usage. Memory is specified in MB, so memory: 1024 sets the memory requirement to 1 GB. CPU requirement is specified in millicores, so cpu: 250 sets it to 0.25 vCPU.

      When adding the resources parameter, do not remove any of the existing parameters under default_preset.template.

    • After specifying the resource requirements, increase the version parameter by 1 - for example:

      # version: 2
version: 3

    Important

    Do not change the id parameter in the manifest - otherwise pSeven Enterprise will recognize your update as an entirely different block, and users will not be able to update blocks in their workflows automatically.

    Also do not change the manifest_version and protocol_version parameters - they are reserved for future use and should not be edited.

  • Save your changes to manifest.yaml and upload the new block version (TheBlock-v1.p7protoblock) to the pSeven Enterprise block library (the /protoblocks directory in the shared data storage). This updated version will appear in the Common library section of the Block library pane in the pSeven Enterprise user interface. In workflows that contain an old version of the block, the Workflow library section of the Block library pane will show the old version along with an update notification.

  • After upgrading to pSeven Enterprise v2021.06, notify your users that they need to update all blocks in their workflows created in previous versions of pSeven Enterprise - otherwise they may experience issues if they run a workflow that contains old block versions.

    To update a block in a workflow, click the update notification next to the block's name in the Workflow library section of the Block library pane.

v2021.05

Updates

  • 1355 Workflow and report downloads now run faster and have less size: when archiving the document to download, pSeven no longer adds the history of document changes to the archive. Previously, downloads always kept a full history, which could greatly increase the archive size.
  • 1200 Added a few convenience improvements related to configuring and linking blocks that have driver ports - for example, Python script blocks with function and argument ports:
    • Driver ports and generic ports are now sorted separately in the Links and Autolink dialogs.
    • Output driver ports are labeled "Request": a block uses them to request evaluations from other blocks connected to it.
    • Input driver ports are labeled "Response": they accept responses (evaluation results) from connected blocks.
  • 1429 Upgraded the embedded pSeven Core package to version 6.21. For details on recent improvements and changes, see the pSeven Core changelog.
  • 1428 Reduced the extension node setup package size. Removed a few unused versions of packaged components, which could cause issues during a node deployment.
  • 1420 Updated the pSeven Enterprise deployment guide for more clarity:
  • 1420 Updated the Extension node deployment guide, added section Allowed users (moved: see Configuring user authorization on extension nodes).
  • 1258 Added the Russian language version to pSeven Enterprise Help. To select your language, use the menu in the Help's top bar.

Note

1406 pSeven Enterprise Help is now also available at the DATADVANCE website. See www.pseven.io/permalink/pseven-enterprise-help for the latest version.

User block development

  • 1007 When block developers update a block in the common library, users can now update this block in their workflows in a few clicks - instead of manually replacing old versions of the block. pSeven shows block update notifications in the Workflow library pane; click the notification to start the update.
  • 1029 Fixed issues in block manifest validation, due to which you could add an unusable block with an invalid name to the block library, or add a block with an invalid id, which you then could not delete.
  • 1243 Fixed an issue where the Block library pane sometimes displayed wrong icons for user blocks.

Bugfixes

  • 975 Fixed an issue where any user with admin access could see workflow runs of other users on the Start tab, and an attempt to open one of these runs led to an error.
  • 1446 Fixed a rare issue where a workflow run did not start but pSeven did not show any error, so the run appeared to be waiting indefinitely.
  • 1444 Fixed a stability issue which could disable editing and running a workflow if there was an error while editing this workflow, so you had to create its copy to continue.
  • 1421 Fixed an issue where pSeven was showing an incorrect inline preview for a matrix value - for example, port values in the Block properties pane or parameter values in the Run setup pane.
  • 1348 Removed dummy test blocks from the block library.
  • 1260 Fixed Enter not working as a confirmation in the file deletion dialog.

v2021.04.20

Updates

User block development

  • 1128 Deployment administrators can now publish user blocks. Copy a user block to the directory named protoblocks in the shared data storage to make it available to all pSeven users.

    The storage server address and the shared data directory path are specified by the storage.shareddata.* parameters in the deployment configuration file (values.yaml).

Bugfixes

  • 1395 Fixed an issue in the Python script block, due to which it sometimes did not save configuration changes.
  • 1407 Fixed a bug due to which unfinished Kubernetes jobs were building up and could consume all resources, potentially leading to a general failure.
  • 1417 Fixed a bug in UDP package routing, which could cause instability of the OpenVPN connection between pSeven and Windows extension nodes.

Compatibility issues

  • 1128 Since this release, pSeven supports only NFS file storages. If your current deployment uses a persistent volume claim or a custom storage class, you will have to set up an NFS storage and transfer all files there before upgrading to v2021.04.20. Note the requirements in section NFS server of the pSeven Enterprise deployment guide.
  • 1418 If you use extension nodes, you will have to manually remove HTCondor on each node before updating it with the new node setup package - see the Extension node upgrade notice below.

Extension node upgrade notice

The version of the HTCondor service in the v2021.04.20 extension node setup package is not compatible with previous installations, so you will have to remove HTCondor on each of the extension nodes manually before upgrading.

  • Open the Windows Control Panel - Programs and Features (or run appwiz.cpl to open Programs and Features directly). Find HTCondor in the list and uninstall it. If the uninstaller prompts for a restart, reboot the node before you continue.
  • Get the new pSevenExtensionNode.zip setup package from pSeven (see Setup package).
  • Unpack pSevenExtensionNode.zip to a temporary directory. The path to this directory must not contain spaces, for example: C:\Temp\pSevenExtensionSetup.
  • From this directory, run bootstrap.bat without parameters. The setup scripts will use settings from the previous installation, which are stored into environment variables on the node.
  • When bootstrap.bat finishes, check its output for errors. If there are none, reboot. If you see errors, save the script's output from the command prompt and contact the DATADVANCE support team (see Technical support).
  • After the reboot, check the node as described in the Extension node deployment guide, section Node check.

v2021.04

Updates

  • 1361 It is now possible to restrict user access to a Windows extension node connected with pSeven - see section Configuring user authorization on extension nodes for details.
  • 1382 Added the Stop button to the stock app UI in AppsHub. You can use that button to stop a running app. Also improved the stock app UI stability in case of connection issues.
  • 1356 Workflow and report copy now works faster, and newly created copies take less space because pSeven no longer copies the history of changes with the document. The original document keeps a full history, but the history in a copy will contain only newer changes.
  • 870 Added the Getting started page, which provides a quick overview of pSeven's functions and features and links to more guides. To open it, click the Getting started button on the Start tab.
  • 1358 Since reports in pSeven are currently limited, and many of their planned features are missing, the New report dialog now contains a beta notice to avoid misunderstanding.

User block development

  • 1137 Fixed an issue where adding a user block to a workflow created duplicates of this block in the workflow library.
  • 1373 Fixed an issue where you could not copy and paste a user block in a workflow if the block contains a large amount of code or embedded data.

Bugfixes

  • 1365 Fixed slowdown issues, which could occur when running a workflow where a Composite block receives or outputs a large data array - for example, a 100×100 matrix.
  • 1373 Fixed the inability to save a Python script block that contains a very long script.
  • 1264 Fixed an issue with the stock app UI in AppsHub where the UI reported a timeout error if an app run took longer than 10 minutes.
  • 1365 Fixed a bug due to which a block in a running workflow failed to start if you have copied this block from another workflow.
  • 1390 Fixed an issue where pSeven sometimes was not showing notifications until you refresh its tab in your browser.

v2021.03

Updates

  • 1333 Convenience improvements in the Links and Autolink dialogs:
    • Quicker linking for ports with matching names. Ctrl-click and drag when creating a link to instantly open the Autolink dialog. In this case, OK in the Autolink dialog links selected port pairs and closes the dialog.
    • Switch between autolinking and manual linking. In the Links dialog, click Autolink to open the simplified dialog. In this case, OK in the Autolink dialog selects the links to create and switches back to Links, where you can continue with manual link editing.
  • 1119 The Python script block now automatically formats the script code to comply with PEP 8 and checks the code for syntax errors, when you click Apply or Save in the block's configuration dialog.
  • 1325 OpenTURNS support. You can now use the openturns module in Python script blocks and user blocks implemented in Python.
  • 1143 The Block properties pane now shows inputs and outputs sorted in the order defined by the block. You can also switch to alphabetical order using the sorting buttons in the pane. For grouped ports, alphabetical sort order applies to port names in a group but does not change the order of groups.
  • 485 When a message receive timeout is exceeded, the api.message_pop() REST API method in the Python script block now raises a TimeoutError instead of a generic exception. If you use messaging in Python script, it is recommended to update your scripts to specifically handle the TimeoutError from api.message_pop().
  • 1005 Added logging for block editing operations. When you add, reconfigure, or delete a block, pSeven writes debug information to edit log.txt in the workflow folder.

User block development

  • 485 User blocks can now access the Redis database that provides the REST API messaging functionality. This makes it possible to add your custom messaging implementation to a user block, functionally similar to the api.message_push() and api.message_pop() methods available in Python script blocks. The database hostname and port are available to blocks as the DA__P7__BLOCK__REDIS_HOST and DA__P7__BLOCK__REDIS_PORT environment variables.

Bugfixes

  • 937 Fixed various issues with incorrect run order and statuses displayed on the Start tab.
  • 1193 1264 Fixed an AppsHub issue where you could not edit some parameters in the stock app UI.
  • 1264 Fixed an issue with the stock app UI where an app could not start if there is a parameter that accepts any data type.
  • 485 Fixed a vulnerability issue in the messaging mechanism, which is in particular used by the Python script block REST API implementation. Before the fix, exploiting this vulnerability allowed a user to access messages of Python script blocks from workflows of other users, and to access messages used internally by pSeven. This could cause data loss in workflows and general malfunctioning. The fix properly isolates messaging for every user.
  • 485 Fixed a bug in the Python script block REST API, due to which api.message_pop() returned a bytearray instead of a string.
  • 1166 Fixed an issue where pSeven could show a failed workflow run as active (running) indefinitely if the run fails to start because of a block start-up error.
  • 1166 Fixed a similar issue where pSeven could not handle a block error when you open the block to edit its configuration.
  • 1166 Fixed pSeven showing a workflow run as active while it has failed due to an internal error. Now pSeven automatically cancels runs that stopped responding and shows them as failed.
  • 1095 Fixed a similar issue where pSeven did not properly stop a block if there was an error while editing its configuration. Now pSeven automatically stops a block running in the edit mode if it becomes unresponsive.

Compatibility issues

  • 485 Since api.message_pop() in a Python script block now returns a string as intended (was bytearray), you may be required to update your scripts to correctly handle the return value, depending on how it is processed.

v2021.02

Updates

  • 1298 1331 Enabled the intended functionality of the Files folder in a workflow. This special folder is a prototype of the workflow's working directory:
    • When you configure block's working directories, pSeven creates folders in Files - so you can preview the working directory structure while editing a workflow. When you move, rename, or reconfigure blocks, pSeven automatically updates the folder structure in Files.
    • You can upload data to Files and to folders inside it to make this data available to blocks during a workflow run. When a workflow starts, pSeven copies the files and folders from Files to the workflow run directory, creating the initial state of block working directories.
    • Running blocks work with copies of files in the run directory, so you can continue editing the workflow and Files as needed, even while a workflow run is active.
    • Note that if a block's working directory type is "indexed", the block creates a new (numbered) working directory every time it starts. These directories have a single prototype in Files - a folder with the [####] name suffix. When you run a workflow, pSeven copies this folder's contents to every new working directory created by the block.
    • To avoid needless copying, place read-only files somewhere outside of the Files folder.
  • 1222 Simplified behavior of the block linking dialogs. The Autolink dialog no longer opens when you create a link, and you can switch to autolinking with the Autolink button in the Links dialog. Creating or editing a link now always opens the full Links dialog.
  • 820 The Python script block now validates names of variables, functions, and arguments to avoid syntax issues and name conflicts.
  • 1106 Added a command to download the run log file, available in the Run logs pane menu.
  • 1230 REST API clients can now download workflow run logs. To get a download URL, request run details and read logfile_url from the response.

User block development

  • 1329 1330 Fixed the inability to set a port value if its data type is a custom table with predefined columns.

Bugfixes

  • 1108 Fixed licensing issues, due to which any user could start an unlimited number of workflow runs or calculations in AppsHub, potentially consuming all resources and disrupting other user's activity.
  • 1317 Fixed a bug with messaging support in AppsHub, due to which an app did not work if its workflow contained a Python script block that uses messaging (the api.message_push() and api.message_pop() REST API methods).
  • 1311 Fixed an AppsHub issue where you could not open the Edit value dialog for an app parameter that currently has an invalid value.
  • 1028 Fixed an AppsHub issue where an app could not start due to an error in processing a port that is both linked and selected as a parameter.
  • 1308 Fixed a GUI error that sometimes occurred when deleting a block from a workflow.

Compatibility issues

  • 1108 Due to bugfixes in the license mechanism, this version requires an updated license valid for v2021.02 or above.

v2021.01

Initial release.