RFID 181EIP Ident Device Configuration and Programming - Molex
Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... ·...
Transcript of Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... ·...
![Page 1: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/1.jpg)
My perspective on
Programming by Configuration: An “emerging” paradigm in automated algorithm design
Steven Adriaensen
AI-lab, Vrije Universiteit Brussel, Belgium
1
![Page 2: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/2.jpg)
Outline
1. Algorithm Design
– What is the algorithm design problem?
– How is it currently being solved?
2. Programming by Configuration (PbC)
– In a nutshell…
– Limitations, how they could be addressed, and my own research in this direction…
2
![Page 3: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/3.jpg)
1. Algorithm Design Problem (ADP)
There are many ways to solve a given problem.• Multiple ways to formulate a problem• Multiple (parametrized?) solvers exist.• Multiple implementations of a single solution approach.
When solving a problem we face design choices
What is the best way?• Minimizing execution time• Maximizing solution quality
“The problem of how to best solve problems”
3
![Page 4: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/4.jpg)
1. Contemporary Solution Approaches
ADPs and attempts to solve them are ubiquitous and fragmented…Algorithm Configuration, Instance-based Selection, (Dynamic) Portfolios, Parameter Control, Reactive Search, Hyper-heuristics, Search-based Software Engineering, Intelligent Compilers, Machine Learning, Reinforcement Learning, Learning Classifier Systems, Program Synthesis, Genetic Programming, Ant Programming, Logical Programming, Probabilistic Programming, Neural Turing Machines,…
How to best solve the ADP is an ADP itself! (idea: apply recursively: configuring/selecting configurators, meta-learning,…)
Research objective: Towards enabling a more unified approach to automated algorithm design, maximally exploiting the nature of the problem at hand.
4
![Page 5: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/5.jpg)
1. Manually
5
Process is...• tedious• time-consuming• costly• untraceable
Result is...• sub-optimal?• overly complex?• unreliable?
![Page 6: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/6.jpg)
1. Automated
Idea: What? Let a computer design its own programsWhy? Computers are faster, cheaper and unbiasedHow? Provide an algorithm for the ADP…
i.e. formalizing a design process.
Fully automated: Program Synthesis, Genetic Programming, Declarative Programming, and (more recently) Neural Turing Machines. Scalability issues…
6
![Page 7: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/7.jpg)
1. Semi-automated
Programming by Optimization (PbO) (Holger Hoos, 2012)1. Leave difficult decisions open at design time
2. Generate the best algorithm instance for a specific use-case automatically.
7
Program a Design Space Single Algorithm
![Page 8: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/8.jpg)
1. Who makes which design choices?
Expert Knowledge Available to make Design Decision?
Manual Fully Automatic Semi-automatic(e.g. PbO)
Low
High
![Page 9: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/9.jpg)
1. Who makes which design choices?
Expert Knowledge Available to make Design Decision?
Manual Fully Automatic Semi-automatic(e.g. PbO)
Low
High
Ad hoc design
decisions
![Page 10: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/10.jpg)
1. Who makes which design choices?
Expert Knowledge Available to make Design Decision?
Manual Fully Automatic Semi-automatic(e.g. PbO)
Low
HighInefficient
use of expert knowledge
Ad hoc design
decisions
![Page 11: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/11.jpg)
Open design choices:
1. Programming by Optimization (PbO)
11
11
How to formulate the ADP as an optimization
problem?
How to solve the resulting optimization
problem?
![Page 12: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/12.jpg)
1. Per-set Algorithm Selection Problem (set-ASP)
Given𝐴: algorithm space𝑋: input space𝐷: input distribution (“use case”)𝑝: 𝑋 × 𝐴 → ℝ: performance evaluation function
Find
𝑎∗ = argmax𝑎𝜖𝐴
𝑥𝜖𝑋
𝐷 𝑥 ∗ 𝐄[𝑝 𝑥, 𝑎 ]
12
![Page 13: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/13.jpg)
1. Set-ASP reduction
13
E.g. • configurators (ParamILS,
iRace, GGA, SMAC etc.) • Genetic Programming (GP),• generative hyper-heuristics• SBSE (program optimization)
- …
![Page 14: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/14.jpg)
1. Per-input Algorithm Selection Problem (input-ASP, Rice, 1976)
Given
𝐴: algorithm space
𝑋: input space𝐷:
𝑝: 𝑋 × 𝐴 → ℝ: performance evaluation function
Find 𝑠∗ satisfying
𝑠∗(𝑥) = argmax𝑎𝜖𝐴
𝐄[𝑝(𝑥, 𝑎)]
14
![Page 15: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/15.jpg)
1. Input-ASP reduction
15
E.g. • portfolio builders• input specific configurators
(Hydra, ISAC etc.) • context-aware compilers
- …
![Page 16: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/16.jpg)
1. Dynamic-ASP reduction
16
E.g. • (dynamic) portfolios,• parameter control, • (selection) hyper-heuristics…
![Page 17: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/17.jpg)
1. Dynamic Algorithm Selection Problem (Adriaensen et. al, IJCAI, 2016)
Given:
• f
• f
Find: A policy 𝜋 maximizing the expected future reward.
17
Design Space: Non-Deterministic TM
Desirability Execution: Rewards associated with moves performed by TM
A function mapping- input- transitions (leading up to choice point)to one of the possible next transitions.
![Page 18: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/18.jpg)
1. Reinforcement Learning Perspective
18
Algorithm Selection Problems
Reinforcement Learning (RL) Problems
Set-ASP (offline) Best-arm Identification Problem
Set-ASP (online) Multi-armed Bandit Problem
Input-ASP Contextual Bandit Problem
Dynamic ASP Markov Decision Problem
Cross-transfer:• RL literature may help you understand and solve these problems better!• RL community also needs to consider ASP methods in practical applications...
![Page 19: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/19.jpg)
Formulate the ADP as a Configuration Problem
2. Programming by Configuration
19
19E.g. ParamILS, iRace, GGA, SMAC
![Page 20: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/20.jpg)
2. Programming by Configuration
ADP ↔ ACP
20
Open design choices Parameters
Alternative decisions Range of values
Design space Configuration space
Design Configuration
C = {0,1,2} x {0,1} x {0,1}
c = (0,1,1)
![Page 21: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/21.jpg)
Success Story
Hard Combinatorial Optimization:- Spear SAT-solver: 500x speedup- SATenstein: 1.6x to 218x speedup…
Mixed Integer Programming:- IBM CPLEX: 2-500x speedup
Machine learning:- Auto-Weka: Similar/better than best with default settings.
Many more: www.prog-by-opt.net
21
![Page 22: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/22.jpg)
Limitations?
22
w.r.t. formulating the ADP as a Configuration Problem
E.g. ParamILS, iRace, GGA, SMAC
![Page 23: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/23.jpg)
Quality of the resulting design?
Question: Is it theoretically possible to always obtain the same quality of design using PbC, which solves the ADP by set-ASP reduction, as those design approaches which solve it using input-ASP or dynamic-ASP reductions?
For instance: Given unlimited resources. Can we, using tuners (e.g. ParamILS, iRace or SMAC), always design algorithms as good as those obtained by per instance tuners (Hydra/ISAC)? How about using parameter control?
23
Set-ASP ≡𝑻 input−ASP ≡𝑻 dynamic ASP?
![Page 24: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/24.jpg)
No?
“Configurators return a single algorithm to be used on all possible inputs.”
average-case performancedependant on input-distribution
(we must re-optimize whenever the use-case changes…)
“Portfolio builders return a portfolio of non-dominated algorithms.”best-case performance
Input-distribution independent
Dynamic approaches: even more powerful!?(ability to adapt to stochastic events)
24
![Page 25: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/25.jpg)
Yes!
“A (dynamic) portfolio solver is just another algorithm”Formulate the algorithm space of the set-ASP to include it.
Consequences:• Discrimination of (dynamic) portfolio solvers is misguided:
– Negative: Excluding them from competitions…– Positive: Free Lunch for (dynamic) portfolios…
• Upward reductions:– Input-ASP ≤ Set-ASP (Θ ~ family of selection mappings)– Input-ASP ≤ Dynamic-ASP (Θ ~ family of policies) RL: Configurator ~ policy search approach to MDP
• The dynamic-ASP can be solved offline
25
![Page 26: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/26.jpg)
Online > offline design?
Offline: “first design the algorithm, afterwards use it”1. Solve the ADP (“training phase”)2. Use the resulting design.
Online: “refine/design the algorithm while using it”- Given: A sequence of instances to be solved in order.- Objective:
- Minimize the cost of doing so (resource usage/quality)- Refine the design:
a) After solving an instance (cross-input learning)b) While solving an instance (within-run learning)
26
![Page 27: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/27.jpg)
Online > offline design?
Offline: “first design the algorithm, afterwards use it”1. Solve the ADP (“training phase”)2. Use the resulting design.
Online: “refine/design the algorithm while using it”- Given: A sequence of instances to be solved in order.- Objective:
- Minimize the cost of doing so (resource usage/quality)- Refine the design:
a) After solving an instance (cross-input learning)b) While solving an instance (within-run learning)
27
When have we trained enough?
Changing use case…
Which training inputs?
![Page 28: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/28.jpg)
Online > offline design?
Offline: “first design the algorithm, afterwards use it”1. Solve the ADP (“training phase”) Pure exploration2. Use the resulting design. Pure exploitation
Online: “refine/design the algorithm while using it”- Given: A sequence of instances to be solved in order.- Objective:
- Minimize the cost of doing so (resource usage/quality)- Refine the design:
a) After solving an instance (cross-input learning)b) While solving an instance (within-run learning)
28
Exploration vs. exploitation trade-off
![Page 29: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/29.jpg)
Online > offline design?
Offline: “first design the algorithm, afterwards use it”1. Solve the ADP (“training phase”) 2. Use the resulting design.3.Online: “refine/design the algorithm while using it”- Given: A sequence of instances to be solved in order.- Objective:
- Minimize the cost of doing so (resource usage/quality)- Refine the design:
a) After solving an instance (cross-input learning)b) While solving an instance (within-run learning)
29
adaptation ≠
learning
What is 𝝅?Learning curves!
Abuse RL…
![Page 30: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/30.jpg)
Semi-online
In many practical settings:
• Minimize response time > total resource usage
• Availability of (cheap, free) spare resources:– Time (overnight, in-between requests)
– Parallelism (unused cores, processors, computers)
Semi-online:
• Serve requests using the best known design
pure exploitation
• Use spare resources to refine it pure exploration
30
![Page 31: Programming by Configuration - COSEAL › wp-content › uploads › 2017 › 03 › COSEAL... · Programming by Configuration: An “emerging” paradigm in automated algorithm design](https://reader033.fdocuments.net/reader033/viewer/2022060318/5f0c74377e708231d4357b0e/html5/thumbnails/31.jpg)
Anytime ~ semi-online
Given: Anytime ADP solver (e.g. ParamILS, SMAC):
1. Start the design process in a separate thread.
2. For each request to solve 𝑥 (asynchronous)
a) Obtain 𝑎incumbent from the design process.
b) Solve 𝑥 using 𝑎incumbent
c) Return solution to the client.
d) Add 𝑥 to the set of training inputs (+ result of run)
(possibly discounting to address non-stationarity)
31