April 9, 2018

pSeven beats MOPTA08 automotive benchmark

Industry: Automotive | Product: pSeven

Origin

MOPTA08 is a multidisciplinary design optimization (MDO) benchmark problem based on a real-life problem from the automotive industry. First, it was presented at the MOPTA 2008 Conference by Don Jones, a technical fellow at General Motors. It states a large-scale multidisciplinary mass optimization of a vehicle in a crash test simulation. Real simulation can optimistically compute about 60 points/day. It was highly desirable to solve the optimization problem in ≤ 1 month (30 days).

The original publication by Miguel F. Anjos is available at http://www.miguelanjos.com/jones-benchmark.

GM crash test simulation example (from wired.com)

MOPTA08 benchmark uses a blackbox based on kriging response surfaces. These response surfaces are trained on data from General Motors crash test simulation.

Problem formulation:

1 objective function to be minimized - mass
124 variables normalized to [0,1]
68 inequality constraints of form g_i (x) ≤ 0
Constraints well normalized: 0.05 means 5% over requirement, etc.
Test problem comes with the initial feasible point with objective ~251.07

Objective

A good performance would be comparable or better than derivative-free optimization algorithm - Powell’s COBYLA:

Number of evaluations = ~ 15 x Number of variables
Fully feasible solution (no constraint violations)
Objective function value ≤ 228 (at least 80% of potential reduction)

"Anything better is exciting" - states Don Jones, the author of this benchmark.

Objective function dependencies in MOPTA08 blackbox (green – feasible, red – infeasible)

Challenges

Max evaluations budget is 1860 points
Known optimum appears to have objective ~222.74
The budget is obviously very small for such big number of variables!

Solution

The significant number of design variables excludes the use of Surrogate-Based Optimization (SBO) methods so a local gradient-based method Sequential Quadratic Programming (SQP) is used.

The solution is divided into two stages N_eval = N^I_eval + N^II_eval

Stage-I: algorithm works as usual for N^I_eval evaluations
Stage-II: if a better feasible solution is not yet found, solve Constraints Satisfaction Problem (CSP) within N^II_eval evaluations

An actual budget division is assumed to be ~3:1

N^I_eval = 1460 points
N^II_eval = 400 points

MOPTA08 optimization workflow in pSeven

Results

pSeven allowed to reach feasible objective value ~227.56 in 1860 evaluations
The “effective” number of required evaluations is ~1650:

Stage-II solution (feasibility restoration) used the designs evaluated on ~1250^th iteration
In other words, the budget in pSeven could be reduced to 1650 iterations

MOPTA08 optimization history in pSeven (green – feasible, red – infeasible)

Summary

Our solution is close to being “exciting” in the terminology of this benchmark
pSeven significantly outperforms the most of the results presented in the original publication