April 4, 2017
“Build. Validate. Explore.” - Part 3: Model Explorer
Updated 18.12.2018
In Part 2 we’ve explained how to validate approximation models with Model Validator and to select the model which gives better performance on selected quality metrics. Once an appropriate model is chosen - it can be used in production or further studied with Model Explorer - an interactive visualization tool for model-based sensitivity analysis and what-if analysis.
Illustrative example
For the sake of visual interpretability let’s take a well-known “sine wave water drop” function Z = sin(R)/R, where R=sqrt(X2 + Y2). Let’s generate samples using LHS technique in the Design space exploration block with budgets of 100, 200 and 800 points, respectively and create three approximation models with Model Builder.
As expected we’ve got approximations of different precision, from very crude to pretty fine.
Let’s open these models with Model Explorer:
Model Explorer shows a matrix of 2D orthogonal slices plots of Fi(Xj) for each i, j. Each slice is defined by the direction and the origin point (an X-dimensional point through which slices are taken). By default, directions coincide with axes. And the origin is specified using sliders right below models list. Either same or different origin can be used for different models.
The uncertainty of approximation is expressed in the form of error band around each slice curve. Notice that wave 100 model is very uncertain about its predictions reasonably due to a small training set. On a contrary wave 800 is pretty confident and shape of its approximation matches well to original analytical function.
At the top left you may see a list of studied models. Right below it, there are a number of slider controls that represent the main interactive functionality - position of the origin point. Slider (and hence slices) bounds correspond to bounds of training sample, but these bounds can be changed in the configuration dialog.
Below origin point, there are slice orientation controls. The slice orientation checkbox and the slider provide an advanced feature that allows changing (i.e. rotate) slice directions. After the orientation checkbox is enabled, the direction for the first slice can be selected with the slider from the set of directions distributed in the positive orthant or can be provided manually (see the slice direction column at the right bottom). The rest of orthogonal directions are constructed automatically.
Every point on a slice curve can be highlighted to see details including input X and prediction F with accuracy estimation:
Real-world example
Let's take a real-world sample from “Static mixer optimization” example mentioned in Part 1. It is a sample of 200 points: 4 inputs: flow temperature, pressure drop, 1st flow velocity, 2nd flow velocity and 2 outputs: nozzle angle and nozzle diameter.
Let’s build 5 models with different properties and requirements: linear model, quadratic polynomial model, model that is required to fit training points exactly, a model that assumes outputs ‘Flow temperature’ and ‘pressure drop’ are correlated and model that has the accuracy evaluation (prediction uncertainty) capability in every point.
Let’s open these models with Model Explorer:
Thin bars at the right side of each slice plot shows scores of sensitivity analysis - the influence of each input Xi on each output Fj. Notice that flow temperature and pressure drop are highly dependent on the diameter of the nozzle and do not depend on the angle. Sensitivity can be scored with two techniques: Sobol indices and Screening indices. The desired technique can be selected in the configuration dialog.
Summary
Often simulation model of a real-world product or process is computationally expensive. Thus, simulation replaced with an approximation model is fast to evaluate. The main reason for using Model Explorer is that exploration and analysis of such models become interactive and easy.