Frédéric Saubion LERIA Learning and Intelligent OptimizatioN Conference Autonomous Search.

Frédéric Saubion LERIA

Learning and Intelligent OptimizatioN Conference

Autonomous Search

Based on joint works on this topic with :

G. Di TolloA. Fiahlo Y. Hamadi F. Lardeux J. Maturana E. MonfroyM. SchoenauerM. Sebag

Learning and Intelligent OptimizatioN Conference

1. Introduction2. Main Ideas3. Taxonomy of AS4. Focus on examples5. Conclusion and challenges

Outline

IntroductionGeneric modeling tools for engineers

(Decision) VariablesDomains Constraints Mathematical

Solver

Solving Constraint Optimization and Satifaction Problems

IntroductionMap coloring problem

Satifaction Problems

Introduction

Optimization Problems

Travelling Salesman Problem : finda round trip across cities with minimal cost

Introduction

IntroductionSearch landscapes are difficult to explore

Many variablesComplex constraints

Problems are more and more complex

IntroductionSearch landscapes are difficult to explore

Exploration vs. Exploitation Balance

Problems are more and more complex

Boolean Variable {0,1}

IntroductionAn illustrative example : solving SAT

SAT CNF instance

Devising more and more complex Solving algorithms

Litterals

Clauses

Assignment

(1 0 0)

Introduction

How to explore the binary search space (assignments) to find a solution ?

Introduction

Use Local Search

Introduction

Basic Local Search

0 1 0 1 1

Choose a random initial assignment

Introduction

Basic Local Search

Compute the number of true and false clauses

Introduction

Basic Local Search

Try to improve by changing a value (flip)

0 1 0 1 1

0 1 1 1 1

Move to a neighbor

Introduction

Basic Local Search

Until finding a solution

Introduction

Short overview of the story : a first greedy version GSATBart Selman, Hector J. Levesque, David G. Mitchell: A New Method for Solving Hard Satisfiability Problems.AAAI 1992: 440-446

A first boat for binary seas

Introduction

10 1 0 1 1 0 0

1 1 1 1

Problem : Many possible moves (many variables)

Introduction

Restrict neighborhoodSelect a false clause C

fda a b c d e f g

0 1 0 1 1 0 0

Introduction

Get stuck in local optima

Introduction

Add pertubationsSelect a false clause C

With a random probability p Perform a random flip for C

With (1-p) Select the variable with best IMP Perform best move

If solution then stopElse go on

Parameter !

Introduction

Use restarts

False Clauses

Iterations

Parameter !

Introduction

WalkSAT : adding a noise and random restart

Henry A. Kautz, Bart Selman: Noise Strategies for Improving Local Search..AAAI 1994

Introduction

How to break ties ?

0 1 0 1 1 0 0

Introduction

Add more sophisticated heuristics Compute the age of the variableIf the best variable is not the most recent then flipElseWith a random probability p’

Perform a random flip the second best

With (1-p’) Flip the best

Parameter !

Introduction

Novelty : using more strategies to perform improvements (age of the variable)D.A. McAllester, B. Selman and H. Kautz. Evidence for invariant in local search.In Proceedings of AAAI-97, AAAI Press 1997, pages 321-326.

Introduction

And improvements go on …

Novelty +,Novelty ++, …, TNM, Sattime…

Introduction

Captain Jack : many indicators and thus selection strategies

Dave A. D. Tompkins, Adrian Balint, Holger H. Hoos: Captain Jack: New Variable Selection Heuristics in Local Search for SAT. SAT 2011: 302-316

IntroductionAdding more parameters and heuristics

More flexible algorithms Fit to different instances

Set parameters/heuristics values Understand the behavior

John Rice. The algorithm selection problem. Technical Report CSD-TR152, Computer science department, Purdue University, 1975.

The Algorithm Selection Problem

Main ideas

John Rice. The algorithm selection problem. Technical Report CSD-TR152, Computer science department, Purdue University, 1975.

The Algorithm Selection Problem

Main ideas

Tuning the parameters

Related Questions

Main ideas

Adjusting the parametersof one algorithm

Main Objectives

Main ideas

Need for more autonomous solving tools

Increasing number of works in this trend : LION, Specialsessions in EA conferences (GECCO,…) …

Ideas for More Autonomous Solvers

How to use an algorithm that includes

•Many parameters

•Many possible heuristics or components

Ideas for More Autonomous Solvers

How to use an algorithm that include

•Many parameters

•Many possible heuristics or components

How to automate all these choices ?

Off-line Automated Tuning

Run your solver on some problems

Experiment automatically parameters values

Run your solver on new problems with these parameters values

Question : Generality of the parameters ?

On-line Parameter Control

Try to react during the resolution by changing the parameter

Example : try to increase some parameter when possible

Question : How to react efficiently ?

Hyper Heuristics

Combine basic solving heuristics

Hyper Heuristics

Get new solvers

Hyper Heuristics

Question : How to learn the suitable solver ?

Portfolios Based Solvers

Use different types of solvers

Learn how to select the right solver for a given problem

Question : Reliability of the learning process ?

Evolutionary ComputationA. E. Eiben, R. Hinterding, and Z. Michalewicz. Parameter control in

evolutionary algorithms. IEEE Trans. Evolutionary Computation,

3(2) :124141, 1999.

Reactive Search Battiti R, Brunato M, Mascia F (2008) Reactive Search and Intelligent

Optimization, ORCS interf., vol 45. Springer

Hyper-HeuristicsBurke EK, Hyde M, Kendall G, Ochoa G, Ozcan E, Woodward J

Handbook of Meta-heuristics, chap A Classification of Hyper-heuristics Approaches

Portfolios methodsGagliolo M, Schmidhuber J (2008) Algorithm selection as a bandit problem with unbounded losses. Tech. rep., Tech.

report IDSIA - 07

Why introducing the concept of Autonomous Search ?

Taxonomy

Taxonomies

Classification : Solving

Solving MethodsTree-Based Search

MetaheuristicsSLS EA

line Au

Complete/incomplete search,Model representationOther optimization paradigms (e.g., ACO )

Taxonomy

Classification : Parameters

Solving MethodsTree-Based Search SLS EA

etting

line Au

Parameter type

StructuralBehavioral

Numerical/discrete valuesComponents of the solverVs. Configuration of the solver

Taxonomy

Classification : Settings

Solving MethodsTree-Based Search EA

etting

line Au

Parameter type

•Experiment-based•Feedback Control •Measures and learning techniques(reinforcement learning, statistical learning, case-base reasonning…)

Taxonomy

Related Approaches

etting

line Au

Parameter type

Taxonomy

Parameter Setting in Evolutionary Computation

Related Approaches

etting

line Au

Parameter type

Taxonomy

Parameter Setting

ParameterTuning

ParameterControl

Deterministic

Adaptive Self-adaptive

Related Approaches

etting

line Au

Parameter type

Taxonomy

Optimization of algorithms (automated tuning)

Related Approaches

etting

line Au

Parameter type

Taxonomy

Optimization of algorithms (automated tuning)

SLS Based (ParamILS)Frank Hutter, Holger H. Hoos, Kevin Leyton-Brown, Thomas Stützle: ParamILS: An Automatic Algorithm Configuration Framework. J. Artif. Intell. Res. (JAIR) 36: 267-306 (2009)

GA Based (Revac)Volker Nannen, A. E. Eiben: Efficient relevance estimation and value calibration of evolutionary algorithm parameters. IEEE Congress on Evolutionary Computation 2007: 103-110

Racing techniquesMauro Birattari, Thomas Stützle, Luis Paquete, Klaus Varrentrapp: A Racing Algorithm for Configuring Metaheuristics. GECCO 2002: 11-18

Related Approaches

etting

line Au

Parameter type

Taxonomy

Reactive SearchLearning for SLS

Related Approaches

etting

line Au

Parameter type

Taxonomy

Hyper heuristics

Hyper Heuristics

Taxonomy

Two possible views

• heuristics to choose heuristics

• heuristics to generate heuristics

Burke EK, Hyde M, Kendall G, Ochoa G, Ozcan E, Woodward JHandbook of Meta-heuristics, A Classification of Hyper-heuristicsApproaches

Proposing a general view of AS

etting

line Au

Parameter type

Taxonomy

Autonomous Search

Autonomous Search Genesis

Taxonomy

Gather works from the CSP solving community

Workshop “Autonomous Search” CP 2007, Providence (RI)

Autonomous Search Genesis

Taxonomy

Identify common concepts, goals andchallenges for future works

Requirements for an Autonomous Search SystemParameters

Taxonomy

Modify its internal components

•Parameters•Fine grain heuristics•Coarse grain solving techniques•Model representation

React to external forces and opportunities

•Search landscape analysis (quality, diversity,...)•External knowledge (prediction models, rules, ...)

CS Related Areas

Taxonomy

Solving Techniques Point of View

•Constraint Programming•Operation Research•Evolutionary Computation

Adjustment Techniques Point of View

•Reinforcement Learning•Statistical Learning•Information Theory

CS Related Areas

Taxonomy

Solving Techniques Point of View

•Constraint Programming•Operation Research•Evolutionary Computation

Adjustment Techniques Point of View

•Reinforcement Learning•Statistical Learning•Information Theory

Not limited to…

Examples of works

etting

line Au

Parameter type

StructuralBehavioralExamples

Examples of works

Examples

etting

line Au

Parameter type

Manual Empirical TuningDeciding the Size of a Tabu List

Mazure B, Sais L, Gregoire E, Tabu search for sat. In : AAAI/IAAI,pp 281285, 1997

Examples of works

Examples

etting

line Au

Parameter type

Automated Parameter Tuning(SLS based)Hutter F, Hoos H, Stutzle T Automatic algorithm configuration based on local search. AAAI 2007

Examples of works

Examples

etting

line Au

Parameter type

Portfolios ApproachesFeatures based linear regression and classiers

Xu L, Hutter F, Hoos HH, Leyton-Brown K Satzilla : portfolio-based algorithm selection for sat. JAIR 2008

Examples of works

Examples

etting

line Au

Parameter type

Learning Combinations of Well-known Heuristics

Epstein S, Freuder E, Wallace R Learning to support constraint programmers. Comput Intell 2005

Examples of works

Examples

etting

line Au

Parameter type

Discovering heuristics(variable selection in SAT SLS)

Alex S. Fukunaga : Automated Discovery of Local Search Heuristics forSatisability Testing. Evolutionary Computation 2008

Examples of works

Examples

etting

line Au

Parameter type

Automated TuningAdjusting the size of a Tabu List

R. Battiti, G. Tecchiolli : The Reactive Tabu Search. INFORMS Journal on Computing 6(2): 126-140 (1994)

Examples of works

Examples

etting

line Au

Parameter type

Controlling Variable Orderings and Values Selection inSearch Heuristics

Boussemart F, Hemery F, Lecoutre C, Sais L Boosting systematic search by weighting constraints. ECAI2004 2004

Examples of works

Examples

etting

line Au

Parameter type

Adaptive Genetic Algorithms

A.E. Eiben, Z. Michalewicz, M. Schoenauer, J. E. SmithParameter Control in Evolutionary Algorithms.In Parameter Setting in Evolutionary Algorithms

Focus on an example : how to design an AS system ?

Generic Evolutionary Algorithm for Constraint Satisfaction Problem

Population

Apply variation operator

New Population

Solution ?

Focus on an example

Generic Search Algorithm for ConstraintSatisfaction Problem

Configuration

Apply variation operator

New configuration

Solution ?

Question : How to choose the suitable operator at each iteration ?

Focus on an example

Associate a probability of application to each operator (parameter)

Select an operator according to thisprobability scheme

Focus on an example

Question : how to set the probabilities(parameters of the algorithm) ?

Focus on an example

Use a principled approach to tune yourparameter

•Search in the parameters space

•Assess the performance of each setting through runs of the algorithm on selected Instances

ParamILS, REVAC, F-Race …

Focus on an example

Second idea:

Control the probability during the run

•Evaluate the performance of each operator after application

•Adjust the parameters according to the performances

General process for control(Automated Operator Selection)

Focus on an example

What are the suitable criteria ?

-Quality -Fitness diversity -Genotypic diversity -Time -…

Focus on an example

What are the suitable criteria ?

-Quality -Fitness diversity -Genotypic diversity -Time -…

Different performnce mesearues

1 2 3 4 5 6 70

Op1Op2

Sliding Windows

Mean or Max ?

How to measure the impact ?

Focus on an example

What is the performance of the operators ?

•Fix a search policy•Dynamic policy •Values against rank •…

Focus on an example

What is the performance of the operators ?

Fix a search policy

Focus on an example

What is the performance of the operators ?No values :

Pareto rank of the operatorsArea under the curve

Estimating efficience of operators

How to reward the operators ?

Proportionally to their performance

Using UCB (Upper Confidence Bound)(reinforcement learning technique)

Exploration + Exploitation of the operators

Choosing the operator having the best UCB

Warning : UCB converge asymptotically toGain for the MAB

But here we have dynamic changes

Use of statistical test to restart learning.

Different selection processes

UCB PM Uniform0

Op1Op2Op3Op4

Instances ?

Comparisons ?

Induced new parameters ? ?

Reliability ?

How to assess the performances of your system ?

What’s next ?

etting

line Au

Parameter type

Conclusion

Many different possible approaches

Guidelines for designing new autonomousSolvers

Off-line/On-lineBehavioural parameters/componentsControl of the efficient heuristics/discovering new heuristics

Conclusion

Challenges

Comparing performances

•Autonomous vs. ad-hoc

•Off-line Tuning vs. On-line control

•Representative benchmarking

Conclusion

Challenges

Comparing performances

•Methodologies for comparisons

•New competitions Chesc (Cross-domain Heuristic Search Challenge)

G. Ochoa and her team

•Related to No Free-Lunch TheoremsMore reliable on more problems

Conclusion

Challenges

Parameters induced by the AS system

•Abstract parameters should be more easy to control (e.g., EvE balance)

•New parameters should be less sensitive than original ones

•Fewer paramaters are easier to adjust

Conclusion

Challenges

Learning

•Interactions solving-learning

•Improving learning off-line

•Short term (react) vs. long term (prediction)

•Continuous search (Arbelaez, Hamadi & Sebag)

Conclusion

Challenges

Distributed and parallel computing

•Improving algorithm’s space exploration

•Sharing information on parameters

•Sharing information on problems

Conclusion

Challenges

Towards more generic on-line control tools

•Identify generic control techniques andmeasures

•Control various components type (behavioral parameters, objective functions, heuristics…)

Conclusion

Of course all LION Proceedings …

Some books to read

Conclusion

And ;-)

Conclusion

Sorry for missing references and works

Conclusion

I will not forget important works and references I will not forget important works and references I will not forget important works and references I will not forget important works and references I will not forget important works and references I will not forget important works and references I will not forget important works and references I will not forget important works and references

Questions

Conclusion

I will not forget important works and references I will not forget important works and references I will not forget important works and references I will not forget important works and references I will not forget important works and references I will not forget important works and references I will not forget important works and references I will not forget important works and references

Frédéric Saubion LERIA Learning and Intelligent OptimizatioN Conference Autonomous Search.

Documents

Transcript of Frédéric Saubion LERIA Learning and Intelligent OptimizatioN Conference Autonomous Search.

SPRING / SUMMER 2019Lanti Jeans Trousers 13 Gaja Blouse Shorts Sori Blouse Toretto Shorts Leria Tunic Nori Leggings Lilam Dress Hoodie Summer Dress Leria Tunic Cairo Trousers Lilam

Conception des Systèmes d'Information Tassadit Amghar Frédéric Saubion.

Frédéric Darriet

Frédéric Lenoir

Workshop eTwinning: Vieira de Leria

ÉPOCA 2019 / 2020 PARECER Nº 01 SOBRE PROVA DE ATLETISMO · ASSOCIAÇÃO DISTRITAL DE ATLETISMO DE LERIA Estádio Municipal de Leiria – Apartado 552 Ec Marrazes - 2416- 904 Leria

Frédéric Frantz

Leria nº.3 - Verán 2010

VICTOR JUAREZ-LERIA TIMING-BASED LOCATION ESTIMATION FOR ...upcommons.upc.edu/bitstream/handle/2099.1/15452/MSC_TUT_v2.pdf · victor juarez-leria timing-based location estimation

FréDéRic Bazille

Frédéric AMARAL

LERIA Equipe ICLN Interaction, Connaissances et Langage Naturel.

Frédéric Houvert

Leria Etal-2014-Modelos Tridimensionales de Foraminíferos de Lukas Hottinger

Frédéric Pascal

Leria 10 - nadal 2012 - Infantil e Primaria

Pièce G3 - Dossier de mise en compatibilité du PLU de la ... filede Saubion Aire de repos de Saubion SAINT-VINCENT-DE-TYROSSE SAUBION ANGRESSE BENESSE-MAREMNE TOSSE ORX SAINT-GEOURS-DE-MAREMNE

Frédéric Chopin.pdf

VICTOR JUAREZ-LERIA TIMING-BASED LOCATION …i Abstract TAMPERE UNIVERSITY OF TECHNOLOGY Erasmus Programme JUAREZ-LERIA, VICTOR: Timing-Based Location Estimation for OFDM Signals with

DUBOIS Frédéric