Bo Liu - Swansea Universityeng-intranet-web.swan.ac.uk/c2ec_seminars/2015_10_BoLiuSlides.pdf ·...

Modern Optimisation Techniques and Their Applications to Simulation‐driven Engineering Design Automation

Bo LiuDepartment of Computing, Glyndwr University, UK

([email protected])

Outline

Simulation‐driven design optimisation and challenges

Surrogate model assisted evolutionary algorithms (SAEAs)

The surrogate model‐aware search framework

SAEAs based on multi‐fidelity simulation

Handling design robustness

Conclusions

Seminar

Engineering Design

Seminar

Integrated circuit

Antenna

Aerospace

Automobile

Process engineering

Design variables: W, L of each transistor, Cc

For the goal of (such as):

They are optimisation problems

Simulation‐Driven Design Optimisation

Seminar

• No time to derive analytical formulas

• Too complex to derive analytical formulas

• Need a method which can provide optimised design solutions without deeply studying the engineering design problem

• Numerical simulation + simulation‐driven optimisation!

Design Optimisation Methods: Case Study

Seminar

CST Optimiser help file

Local and Global Optimisation (1)

Seminar

local optimisation

global optimisation

search space / range

Local and Global Optimisation (2)

Seminar

• Local optimisation (SQP, NM simplex):• Advantages: efficient if there is a good initial design • Drawbacks: ad‐hoc (a good starting point) and less optimal

• Global optimisation (GA, DE, PSO):• Advantages: general, highly optimal, robust

• Why not using global optimisation methods all the time?Because numerical simulations are often computationally expensive, the simulation‐driven optimisation process leads to prohibitive time!

Example: On‐chip antenna design optimisation (standard DE): EM simulation for a candidate design: 10 minutes by ADS‐Momentum Convergence: 800 generations Population size: 40 10min x 40 x 800 = 7 months!!!

Seminar

Case Study

B. Liu, H. Aliakbarian, Z.Ma, G. Vandenbosch, G. Gielen, "An Efficient Method for Antenna Design Optimization Based on Evolutionary Computation and Machine Learning Techniques", IEEE Transactions on Antennas & Propagation, vol. 62, no. 1, pp. 7-18, 2014.

Efficiency!

A Major Challenge of Simulation‐driven Design Optimisation

Seminar

• Global optimisation algorithms often need several thousands of simulations to achieve highly optimisedsolutions for many engineering design optimisationproblems with around 20 design variables.

• New optimisation techniques are highly needed, which is able to:• Provide highly optimised designs (much better than manual design, comparable to using standard global optimisation methods)

• In a practical timeframe

Outline






Conclusions

Seminar

Evolutionary Computation (EC)

Seminar

• Evolutionary computation is a computational intelligence method for optimisation

• EC is based on natural selection, survival of the fittest (objective function)

• Different global optimisationalgorithms: GA, DE, PSO, IA, AC

• EA has strengths on black‐box (no derivatives) and multimodal (more than 1 local optima) problems

Differential Evolution

,1 ,2 ,ˆ ( ) [ , , , ]i i i MX t x x x 1, 2, , NPi

,,

,

( 1), ( ( ) ) ( ),( 1)

( ), , 1, 2, , i j

i ji j

v t if rand j CR or j randn iu t

x t otherwise j M

Seminar

Diversity vs. Optimality?

Solutions to the efficiency problem

Seminar

Efficiency enhancement

Evolutionary algorithms

Speed up the simulation

Use fewer simulations

SAEA

Change the optimiser

Introduction to SAEA

Surrogate model assisted evolutionary algorithm (SAEA): Using surrogate models to replace exact function evaluations

Seminar

• Statistical learning

• Computationally cheap

Surrogate Modeling

Seminar

Surrogate modeling:

•Gaussian Process / Kriging (GP)•Artificial neural network (ANN)•Support vector machine (SVM)•Radial basis function (RBF)•Response surface method (RSM)•…

Wrong convergence!!

Model Management

Seminar

• Surrogate model predictions have error and have to be corrected by simulations

• Surrogate model and simulations are connected, because the surrogate model is constructed by the samples obtained by simulations

• How to “cleverly” use surrogate model prediction and simulation (related to surrogate model construction)?

• Who should be used for simulation?

Model Management (pioneer methods)

Seminar

• Off‐line modelling‐based methods

• Only feasible for small scale problems

• Landscape with narrow peak?

• Generation‐based methods

• When to use exact evaluations?• Efficiency of evaluating a whole

population (how many evaluations are useful)?

Iteration 1: EM

Iteration 2: EM

Iteration 3: EM

Iteration 4:Prediction


Iteration 6:EM

Iteration 7:EM




Model Management (state‐of‐the‐art methods)

Seminar

• Elitist candidates among search operators‐visited solutions

• Directly using top candidates by EA operators

• Combining EA with local search and (multiple) local surrogate models

•Prescreening methods utilisingprediction uncertainty (GP model)

• Expected improvement• Probability of improvement• Lower confidence bound

D. Jones, A taxonomy of global optimization methods based on response surfaces, Journal of global optimization, 2001, 21(4): 345-383.

D. Jones et al., Efficient global optimization of expensive black-box functions. Journal of Global optimization, 1998, 13(4): 455-492.

M. Emmerich et al., Single-and multiobjective evolutionary optimization assisted by gaussian random field metamodels, IEEE Transactions on Evolutionary Computation, 2006, 10(4): 421-439.

D. Lim et al., Generalizing surrogate-assisted evolutionary computation, IEEE Transactions on Evolutionary Computation, 2010, 14(3): 329-355.

Z. Zhou et al., Combining global and local surrogate models to accelerate evolutionary optimization, IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 2007, 37(1): 66-76.

M. Le et al. Evolution by adapting surrogates, Evolutionary computation, 2013, 21(2): 313-340.

Ordinary GP modeling Given training data:

Correlation function:

Maximize likelihood function:

Note: solve in closed form, estimate the hyper-parameters

Best linear unbiased prediction and predictive distribution

variants: simple/blind/…

Seminar

Prescreening • With the uncertainty measurement, we can consider the quality of a candidate design in a global picture

• Even the predicted value is bad, promising solutions can still be discovered

Seminar

D. Jones, 2001. “A Taxonomy of Global Optimization Methods Based on Response Surfaces”, Journal of Global Optimization, pp. 345-383.

Prescreening•Possible promising areas but with fewer training data points can be effectively explored.

•The “guessed” promising points may not be correct. Empirical experiments show that EI is not better than LCB.

• In medium scale (20‐30d), in many cases / phases, prescreening ≈ prediction (evaluating the elitists)

Seminar

Challenges Faced by SAEA

• Many practical engineering design problems have 10‐30 design variables (medium scale) and the landscapes of which are multimodal.

• A typical numerical simulation may need 20 minutes to several hours.

• Design engineers divide the optimisation time to “a cup of tea”, “a night’s time”, “a weekend”, “a week”, “two weeks” and prohibitive.

• Existing SAEAs (often need several hundreds to thousands of simulations) still need substantial speed improvement to fit the industrial requirements.

Seminar

Outline






Conclusions

Seminar

Goal and the Key Contradiction

Model qualitySolution quality

Efficiency*x evalN

Seminar

Goal: Make SAEAs much faster without sacrificing optimality

Good Solution Quality

Good Model Quality

More Samples (exponentially with d)

Decrease Efficiency

Questions and Ideas (1)

Seminar

• Why off‐line SAEA is not fit for medium scale problems?• Do we need to model everywhere? • Can we do greedy search?

• Most available SAEAs use the standard EA structure• Standard EAs have excessive diversity

• Idea 1: only model the necessary regions!• Question 1: to what extent we should

explore the unknown parts?

Questions and Ideas (2)

Seminar

• To describe a certain landscape, a certain amount of samples is necessary, no matter what the search is and how to model it.

• What decides the number of necessary samples? landscape complexity, number of design variables and ?

Good Solution Quality

Good Model Quality

Really more Samples?(exponentially with d)

Decrease Efficiency

Summary of the Ideas

Seminar

• We need to find a search method (only prescreening is not enough) which has enough (but not excessive) diversity leading to (global) optimum without exploring unnecessary regions.

• We may improve the locations of the samples to build surrogate models with better quality using the same number of samples, which is translated to efficiency. The samples are provided by a new search method.

Seminar

Summary of the Ideas

• The search and modelling should work harmonically, reinforcing each other, rather than loosely connected

• How to make it easy to implement? • How to make it insensitive to algorithm parameters?

• Surrogate model‐aware evolutionary search (SMAS) framework

B. Liu, Q. Zhang, G. Gielen, "A Gaussian Process Surrogate Model Assisted Evolutionary Algorithm for Medium Scale Expensive Black Box Optimization Problems", IEEE Transactions on Evolutionary Computation, vol. 18, no. 2, pp. 180-192, 2014.

SMAS vs. Present SAEA SMASTraditional SAEA

Seminar

Properties of SMAS (1) Using the elitist candidates as the parent population emphasises exploitation

Only at most one candidate is different from two consecutive parent populations. The training data points describing the current search region can thus be much denser.

The exploration ability can be maintained by selecting appropriate EA operators and parameters.

Seminar

Experimental Verifications (1)

Problems: 20‐,30‐dimensional Ellipsoid (F1‐F3, opt:0), Rosenbrock (F4‐F6, opt:0), Ackley(F7‐F9, opt:0), Griewank(F10‐F12, opt:0), 30‐dimensional RS‐Rastrigin(F13, opt: ‐330), 30‐dimensional RH composition function(F14, opt:10). 1000 evaluations, 20 runs.

SMAS vs. GS‐SOMA [Lim IEEE TEVC 2010]

Seminar

Convergeat last

Experimental Verifications (2)

SMAS vs. SAGA‐GLS [Zhou IEEE TSMC 2007]

SMAS vs. MAES [Emmerich IEEE TEVC 2006]

Seminar

Seminar

Automated Design of Complex Antennas (1)

• EAs have been widely used for antenna synthesis, but the long optimisation time largely limits their applications

Example: Four-element antenna array (3.4GHz –3.8GHz, FR4 substrate)

Seminar

Automated Design of Complex Antennas (2)

• Maximise realised gain (each sampling point at least 13dB) with S11 below ‐10dB

Synthesis finished in only one night, with 71.05dB (5 sampling points total) realised gain

B. Liu, H. Aliakbarian, Z.Ma, G. Vandenbosch, G. Gielen, "An Efficient Method for Antenna Design Optimization Based on Evolutionary Computation and Machine Learning Techniques", IEEE Transactions on Antennas & Propagation, vol. 62, no. 1, pp. 7-18, 2014.

RF IC / mm‐wave IC Design

Difficulties for manual design

Passive component design is difficult Long simulation time: EM simulation (FEA), HB simulation Design experience intensive for the available step‐by‐step manual design

method

Difficulties for EA‐based automated design

Accurate lumped models (computationally cheap) over a wide bandwidth for passive components are difficult to find at high‐frequencies

Long simulation time: EM simulation, HB simulation Medium scale (15‐40 variables) Complex and tight constraints

Seminar

The GASPAD Method

Focuses on 60GHz and above RF IC

Three Main Goals of the Synthesiser Provide Highly Optimised Results General Enough to Any Circuit Configuration, Any Technology and Any

Frequency Efficient Enough for Practical Use

Update SMAS on constraint handling

Seminar

B. Liu, D. Zhao, G. Gielen, "GASPAD: A General and Efficient mm-wave Integrated Circuit Synthesis Method Based on Surrogate Model Assisted Evolutionary Algorithm", IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 33, no. 2, pp. 169-182, 2014.

Updating SMAS for Constraint Handling

New methods to model the

focused search region

New ranking methods

considering constraint

satisfaction

Separate modelling

objective and constraints

Prescreening + predicted

value

Seminar

mm‐wave IC Design Automation (1)Synthesis of a 60GHz power amplifier in a 65nm CMOS technology

(18 parameters)

Seminar

15 min / simulation

mm‐wave IC Design Automation (2)

Design parameters and their ranges

Seminar

Synthesised Result Synthesised results:Power added efficiency (@P1dB): 9.85%1dB compression point: 14.87dBmPower gain: 10.73dBK factor: 10.68 (stable)

About 2 days synthesis time. Much better performance than manual design [He 2010 RFIC]

Seminar

Benchmark tests show 10x to 100x speed improvement compared to standard EA-based methods with the increasing of severity of constraints

MEMS Synthesis: Example

Altair

Corrugated actuator: 9 parameters

From several examples, AGDEMO obtains better solutions than ANSYS DesignXplorer, obtaining about 8-13 times speed enhancement when the number of parameters is around 10.

Outline






Conclusions

Seminar

More Efficiency Improvements

Efficiency improvement of SMAS‐based methods: The more complex the landscape, the more design variables

(but within 50), the more spec constraints, the more efficiency improvement

5‐20 times efficiency improvement than standard EAs for most microwave / RF design optimisation / synthesis problems

4 months‐> 1 week

Another calculation: 30 minutes / simulation, 700 simulations, 2 weeks 1 hour / simulation, 700 simulations, 1 month Sometimes, we have to include the connector, housing, … We need more speed improvement!

Seminar

Solutions to the efficiency problem

Seminar

Efficiency enhancement

Evolutionary algorithms

Speed up the simulation

Multi‐fidelity simulation models

Use fewer simulations

SAEA

Change the optimizer

Why Multi‐fidelity Simulation Models?

Opportunities Coarse‐mesh / reduced solver iterations / simplified mesh type

numerical simulation models (e.g., FEA model) is almost universal

Coarse models often have 10‐50 times speed improvement compared to fine model

It is compatible to SAEA

A natural idea: Use the coarse model to explore the design space, filtering out many non‐optimal regions, and use the fine model to exploit the near optimal regions.

However, this is not straightforward.

Seminar

Available Method 1 Co‐Kriging assisted multi‐fidelity optimisation

Co‐kriging is a surrogate model constructed by both fine model data and coarse model data A GP model based on the coarse model data A GP model based on the residuals of the fine and coarse model data

The co‐kriging model is used as the surrogate model and is compatible to SAEAs

Seminar

A. I. Forrester, A. Sóbester, and A. J. Keane, “Multi-fidelity optimization via surrogate modelling,” Mathematical, Physical and Engineering Sciences, vol. 463, no. 2088. The Royal Society, 2007, pp. 3251–3269.

Available Method 1: Advantages and Drawbacks

Advantages: Mathematical sound and reliable

Drawbacks: Not scalable: when it goes to medium scale, the initial high‐

quality co‐kriging model needs a tremendous number of coarse and fine model samples

Seminar

Available Method 2

Use the fine model to fine tune the optimal designs obtained by the coarse model (popular for real‐world engineers)

Advantages: Scalable and good compatibility

Drawbacks: Ad‐hoc: Success depends on the fidelity of the coarse model due to the distorted landscape Sometimes, successful Sometimes, using low‐fidelity model does not translate to final

efficiency improvement Sometimes, the optimisation simply fails.

Seminar

S. Koziel and S. Ogurtsov,“Model management for cost-efficient surrogate-based optimisation of antennas using variable-fidelity electromagnetic simulations,” IET Microwaves, Antennas & Propagation, vol. 6, no. 15, pp. 1643–1650, 2012.

This Method Is NOT Reliable

• The optimal point of the coarse model may have a large distance with the optimal point of the fine model, wasting a lot of simulations

• The optimal point of the fine model may be not be accessible by fine tuning from the optimal point of the coarse model

Seminar

S. Koziel and S. Ogurtsov,“Model management for cost-efficient surrogate-based optimisation of antennas using variable-fidelity electromagnetic simulations,” IET Microwaves, Antennas & Propagation, vol. 6, no. 15, pp. 1643–1650, 2012.

Frequency shift(Missing/additional peaks)

Spatial shift

finecoarse

Ideas

Seminar

• Idea: granting a coarse model‐based optimisation based on SMAS (not waste much), mine the data to find the correct starting points.

Data Mining Process

Seminar

Goal: calibrate the distortion using as few fine simulations as possible.

Handling Distortion

Seminar

The focused distortion: cannot be obtained by local

exploitation from the optimal points in terms of the coarse model,

the landscape(s) of the coarse model maintains the general shape of that of the fine model

S11 of a microstrip antenna

Seminar

Real World Example (1)

Example: Yagi-Uda antenna Rogers RT6010

Seminar


Coarse model: 85,680 mesh cells, 2 minutes Fine model: 1,512,000 mesh cells, 40 minutes

Specific challenge: gain of the fine model is smaller than the coarse model, and the optimal solutions of the coarse model are not feasible in terms of fine model evaluation.

10GHz-11GHz

Seminar


Response: Yagi-Uda antenna SADEA-II vs. SADEA

Outline






Conclusions

Seminar

Robust design optimisation• Two kinds of robust optimisation

• Robust design• Example: mechanical engineering• Requirement: the performance shows little degradation considering process variations

• High computing overhead per simulation (e.g., FEA)

• Yield optimisation, yield‐performance trade off

• Example: IC design (high‐value manufacturing)• Requirement: accurate yield estimation • High or low computing overhead per simulation

Altair

Ordinal Optimisation Assisted DE

• The ORDE algorithm for analogue IC yield optimisation: • Ordinal optimisation for intelligent computing budget allocation

• Advanced Monte Carlo Sampling method• Integration with differential evolution

Altair

B. Liu, F. Fernández, G. Gielen, "Efficient and Accurate Statistical Analog Yield Optimization and Variation-Aware Circuit Sizing Based on Computational Intelligence Techniques", IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 30, no. 6, pp. 793-805, 2011.

seminar 59

General Ideas

Selection-based constraint handling

OCBA (Optimal Computational

Budget Allocation)

Random-scale and trigonometric mutation

seminar 60

Flow diagram

Example

Altair

Two‐stage fully differential folded‐cascode amplifier (TSMC 90nm CMOS technology): 21 parameters

From several examples, ORDE obtains comparable solutions with Differential Evolution with primitive Monte Carlo simulation, obtaining about 10 times speed enhancement.

25 minutes: yield=98.9%

Outline

Simulation‐driven design optimisation

Surrogate model assisted evolutionary algorithms (SAEAs) and challenges


Data mining techniques for multi‐fidelity SAEAs

Conclusions

Seminar

Conclusions

Improving the efficiency of SAEA is a key issue for simulation‐driven design optimisation / automation

The SMAS framework significantly enhances the efficiency of SAEA for medium scale problems because the surrogate modelling and the search work together, reinforcing each other.

The data mining assisted SMAS handles the distortion between simulation models of different fidelities, integrating the advantage of low‐fidelity simulation model in a reliable manner and is scalable.

Future works include: (1) robust FEA simulation‐embedded design automation methods and various applications, (2) introducing computational intelligence techniques to FEA solvers.

Seminar

Thank you

Thank you!

Seminar

Bo Liu - Swansea Universityeng-intranet-web.swan.ac.uk/c2ec_seminars/2015_10_BoLiuSlides.pdf ·...

Documents

Transcript of Bo Liu - Swansea Universityeng-intranet-web.swan.ac.uk/c2ec_seminars/2015_10_BoLiuSlides.pdf ·...