1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and...
-
Upload
nickolas-summers -
Category
Documents
-
view
228 -
download
2
Transcript of 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and...
![Page 1: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/1.jpg)
1
Multiplicative Weights Update Method
Boaz Kaminer
Andrey Dolgin
Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method: a Meta Algorithm and Applications” Unpublished. 2007.
![Page 2: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/2.jpg)
2
Motivation
Meta-Algorithm Many Applications Useful Approximations Proofs
![Page 3: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/3.jpg)
3
Content
Basic Algorithm – Weighed Majority Generalized Weighted Majority Algorithm Applications:
General Guidelines Linear Programming Approximations Zero-Sum Games Approximations Machine Learning
Summary and Conclusions
![Page 4: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/4.jpg)
4
The Basic Algorithm – Weighted Majority N ‘experts’, Each gives his estimation At each iteration, Each ‘expert’ weight is updated based
on its prediction value
N. Littlestone, and M.K. Warmuth. The Weighted Majority Algorithm. Information and Computation, 108(2):212–261,1994.
![Page 5: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/5.jpg)
5
Approximation Theorem #1 Theorem #1:
After t steps, let mti be the number of mistakes of
‘expert’ i and mt be the number of mistakes the entire algorithm has made. Then the following bound exists for every i:
![Page 6: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/6.jpg)
6
Proof of Theorem #1 Induction: Define the potential function: Each mistake at least half the total weight
decreases by 1-ε the potential function decreases by at least 1-ε/2:
Induction: Since: => Using:
1timt
iw 1,t t
iiw thus n
1 2tmt n
t tiw i
2ln(1 ) , 1/ 2x x x x
1 2 1t t
im mt tin w
![Page 7: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/7.jpg)
7
The Generalized Algorithm Denote P – The set of events/outcomes Assume matrix M: M(i,j) the penalty of ’expert’ i for
outcome j.
![Page 8: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/8.jpg)
8
The Generalized Algorithm – cont.
The expected penalty M(Dt,jt) for outcome jt:
The total loss after T rounds is
,,
t tit t t i
i tiii
w M i jj P is p M i j
w
![Page 9: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/9.jpg)
9
Approximation Theorem #2
Theorem #2:Let ε1/2.
After T rounds, for every ‘expert’ i:
![Page 10: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/10.jpg)
10
Theorem #2 - Proof
Proof:Define the potential function: t t
iiw
, / 1,1
(1 ) , 1,1
t
x
M i j
x e x
tt i
tiwp
Based on convexity of exponential function:
)1-ε(x(1-εx) for x[0,1] ,)1+ε(-x(1-εx) for x[-1,0]
![Page 11: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/11.jpg)
11
Proof of Theorem #2 – cont. Theorem #2: ε1/2. T rounds, for every ‘expert’
i:
Proof - cont: After T rounds: For every i: Use:
, / , /1t t t t
t tM D x M D xT e ne
2 21ln , ln 1 , 1/ 2
1for
![Page 12: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/12.jpg)
12
Another possible weights update rule:
For the error parameter >0#3: εmin{/4,1/2}, After T=2ln(n)/ε rounds, the
following bound on average expected loss for every i:
#4: ε=min{/4,1/2}, After T=162ln(n)/2 rounds, the following bound on average expected loss for every i:
Corollaries:
![Page 13: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/13.jpg)
13
Gains instead of losses:
In some situations, the entries of the matrix M may specify gains instead of losses.
Now our goal is to gain as much as possible in comparison to the gain of the best expert.
We can get an algorithm for this case simply by considering the matrix M’ = −M instead of M. (1+ε)M(i,j)/, when M(i j)0 (1-ε)-M(i,j)/, when M(i j)<0
#5: After T rounds, for every ‘expert’ i:
![Page 14: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/14.jpg)
14
Applications General guidelines Approximately Solving Linear Programming Approximately Solving Zero-Sum games Machine Learning
![Page 15: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/15.jpg)
15
General Guidelines for Applications
Let each ‘expert’ represent constraint The penalty for an ‘expert’ is proportional
to the satisfaction level of the constraint:The penalty focuses on ‘experts’ whose
constraints are poorly satisfied
![Page 16: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/16.jpg)
16
Linear Programming (LP)
Consider the following LP: Axb, xP Consider c=ipiAi , d=pibi . for some distribution
p1,p2,…,pm - find cTxd, xP. (Lagrangian Relaxation): solve cTxd instead of Axb
We only need to check the feasibility of one constraint rather than m!
Using this “oracle” the following algorithm either: Yields an approximately feasible solution, i.e.
Aixbi- for some small >0 or failing that, proves that the system is infeasible.
S. A. Plotkin, D. B. Shmoys, and E. Tardos. Fast approximation algorithm for fractional packing and covering problems. In Proceedings of the 32nd Annual IEEE Symposium on Foundations of Computer Science, 1991. pp. 495–504.
![Page 17: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/17.jpg)
17
Lagrange Relaxation
![Page 18: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/18.jpg)
18
General Framework for LP Each “expert” represents each of the m
constraints. The penalty of the “expert” corresponding to
constraint i for event x is Aix − bi. Assume that the oracle’s responses xP which
satisfy Aix−bi [−,] for all i, for some known parameter .
Thus the penalties lie in [−,]. Run the Multiplicative Weights Update algorithm
for T steps, using ε=/4. The answer is:
![Page 19: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/19.jpg)
19
Algorithm Results for LP If the oracle returns a feasible x for cTxd in
every iteration, then for every i:
Thus we have an approximately feasible solution Aixbi-
In some iteration, the oracle declares infeasibility of cTxd In this case, we conclude that the original system Axb is infeasible.
Number of iterations T proportional to 2.
![Page 20: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/20.jpg)
20
Packing and Covering Problems Set packing problem:
Suppose we have a finite set S and a list of subsets of S.
Then, the set packing problem asks if some k subsets in the list are pairwise disjoint (in other words, no two of them intersect).
ixi1
Set Covering: Given a universe U and a
family S of subsets of U, a cover C is a subfamily CS of sets whose union is U.
ixi1
![Page 21: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/21.jpg)
21
Fractional Set Covering Fractional Set Packing
For an error parameter ε>0: A point x is an
approximate solution to:The packing problem: if Ax(1+ε)bThe covering problem: if Ax(1-ε)b
Reminder: Running time depends on ε-1 and on - =maxi{maxxP(aix/bi)}
![Page 22: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/22.jpg)
22
Multi-commodity Flow Problems The multi-commodity flow problem is a
network flow problem with multiple commodities (or goods) flowing through the network, with different source and sink nodes: Capacity constraints: Flow conservation constraints Demand satisfaction constraints
D2S1
S2D1
![Page 23: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/23.jpg)
23
Multi-commodity Flow Problems Multi-commodity flow problems are represented by
packing/covering LPs and thus can be approximately solved using the framework outlined above LP formulation: maxppfp, s.t. epfp≤ce e Solving K shortest-path for each commodity in each iteration Unfortunately, the running time depends upon the edge
capacities (as opposed to the logarithm of the capacities) and thus the algorithm is not even polynomial-time
A modification: Weight the edges (e) The “event” consists of routing only as much flows as is allowed
by the minimum capacity edge on the path (Cpt)
The penalty incurred by edge e is M(e,pt) = cpt/ce
Start with we1=, we
t+1=wet(1+)Cpt/Ce, terminate when we
T≥1
![Page 24: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/24.jpg)
24
Applications in Zero-Sum Games Solutions Approximations Approximating the Zero-Sum game value:
Let > 0 be an error parameter. We wish to find mixed row and column strategies Dfinal,Pfinal such that:
Each “expert” corresponds to a single pure strategy Thus a distribution on the experts corresponds to a mixed
row strategy.
Y. Freund and R. E. Schapire. Adaptive game playing using multiplicative weights. Games and Economic Behavior, 29:79–103, 1999.
![Page 25: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/25.jpg)
25
Applications in Zero-Sum Games Solutions Approximations Set: =/4 Run for T= 16 ln(n)/2 Iterations For any D (mixed strategy distribution)
Specifically for D* => M(D,j)≤* for any j Thus you get a approximation to the gave value:
For the mixed strategy who reached best results Dfinal
![Page 26: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/26.jpg)
26
Applications in Machine Learning
Boosting: A machine learning meta-algorithm for performing supervised
learning. Boosting is based on the question posed by Kearns: can a set of weak learners create a single strong learner?
Adaptive Boosting (Boosting by Sampling): Fixed training set of N examples: The “experts” correspond to
samples in the training set Repeatedly run weak learning algorithm on different distributions
defined on this set: “events” correspond to the set of all hypotheses that can be generated by the weak learning algorithm
The penalty for expert x is 1 or 0 depending on whether h(x) = c(x)
Final hypothesis has error under the uniform distribution on the training set
Yoav Freund and R. E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1):119–139, August 1997.
![Page 27: 1 Multiplicative Weights Update Method Boaz Kaminer Andrey Dolgin Based on: Aurora S., Hazan E. and Kale S., “The Multiplicative Weights Update Method:](https://reader035.fdocuments.net/reader035/viewer/2022062304/56649ec65503460f94bd164d/html5/thumbnails/27.jpg)
27
Summary, Conclusions, Way-ahead
Approximation optimization algorithm for difficult constrained problems was presented
Several convergence theorems were presented Several approximation applications areas were mentioned:
Combinatorial optimization Game theory Machine learning
We believe this methodology can be used for solving constrained optimization problems through the Cross Entropy method