Counting Algorithms for Knapsack and Related Problems
description
Transcript of Counting Algorithms for Knapsack and Related Problems
![Page 1: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/1.jpg)
Counting Algorithms for Knapsack and Related Problems
1
Raghu Meka(UT Austin, work done at MSR, SVC)
Parikshit Gopalan (Microsoft Research, SVC)
Adam Klivans (UT Austin)Daniel Stefankovic (Univ. of Rochester)
Santosh Vempala (Georgia Tech)Eric Vigoda (Georgia Tech)
![Page 2: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/2.jpg)
Can we Count?
2
Count proper 4-colorings?
533,816,322,048!
O(1)
![Page 3: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/3.jpg)
Can we Count?
3
Count the num. of sols. to a 2-SAT instance?
Count the number of perfect matchings?Counting ~ Random Sampling
Volume estimation, statistics, statistical physics.
Above problems are #P-hard#P ~ NP in the counting world
![Page 4: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/4.jpg)
Approximate Counting for #P
4
#P introduced by Valiant in 1979.Don’t expect to solve #P-hard problems
exactly. Duh.How about approximating?
Want relative error: compute p such that
![Page 5: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/5.jpg)
Approximate Counting for #P
5
Triggered counting through MCMC:Permanent/Matchings: Jerrum, Sinclair 1988;
Jerrum, Sinclair, Vigoda 2001Volume estimation: Dyer, Frieze, Kannan
1989; Lovasz, Vempala 2003Does counting require randomness?
Approximate Counting ~ Random Sampling
Jerrum, Valiant, Vazirani 1986
![Page 6: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/6.jpg)
Deterministic Approximate Counting for #P?
Derandomizing simple complexity classes is important.Primes is in P – Agarwal, Kayal, Saxena 2001SL=L – Reingold 2005
Most previous work through samplingNeed new techniques for countingEfficiency?
Examples: Weitz 06, Bavati et al. 07,
Ultimate Goal: Derandomize BPP …
6
![Page 7: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/7.jpg)
Our Work
7
Techniques of independent interestSimilar results for multi-dimensional
knapsack, contingency tables.Efficient algorithm for learning functions of
halfspaces with small error.
First deterministic approximate counting algorithm for
Knapsack. Near-linear time sampling.
![Page 8: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/8.jpg)
Weight could be exponential
Knapsack
8
Applications: Optimization, Packing, Finance, Auctions
![Page 9: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/9.jpg)
Counting for Knapsack
9
Estimate
Reference Complexity
Dynamic programmingDyer et al. 1993 RandomizedMorris and Sinclair 1999
Randomized
Dyer 2003 Randomized
![Page 10: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/10.jpg)
Counting for Knapsack
10
Efficient sampling: after a preprocessing phase each sample takes time O(n).
Deterministic algorithm for knapsack in time
.
![Page 11: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/11.jpg)
Multi-Dimensional Knapsack
11
Given , estimate
![Page 12: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/12.jpg)
Multi-Dimensional Knapsack
12
Near linear-time sampling after preprocessing.
Previously: randomized analogues due to Morris and Sinclair, Dyer.
Thm: Deterministic counting algorithm for k-dimensional knapsack in
time
![Page 13: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/13.jpg)
Counting Contingency Tables
13
Dyer: randomized poly. time when rows constant.This work: deterministic poly. time when rows
constant
Right-handed
Left-handed
TOTALS
Males 43 9 52Females 44 4 48TOTALS 87 13 100
TOTALS
? ? 52? ? 48
TOTALS 87 13 100
![Page 14: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/14.jpg)
Learning Results: Halfspaces
14
Applications: Perceptrons, Boosting, Support Vector Machines
![Page 15: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/15.jpg)
Functions of Halfspaces
15
Intersections Depth 2 Neural Networks
![Page 16: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/16.jpg)
Learning Functions of Halfspaces
16
Input: Uniformly random
examples and labels.
Output: Hypothesis
agreeing with f. Query algorithm to learn
functions of k halfspaces in time .
First algorithm for intersection of two halfspaces.
![Page 17: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/17.jpg)
17
Explicitly construct a small-width approximating branching program.
Motivated by monotone trick of M. and Zuckerman 2010.
Main Technique: Approximation by Branching Programs.
![Page 18: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/18.jpg)
Read Once Branching Programs
18
• Layered directed graph
• vertices per layer• Edges between
consecutive layers• Edges labeled • Input: • Output: Label of final
vertex reached
n layers
![Page 19: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/19.jpg)
Counting for ROBPs
19
Can count number of accepting solutions in
time by dynamic programming.
n layers
![Page 20: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/20.jpg)
Knapsack computable by ROBPs
20
n layers
Can we use counting for ROBPs? No – width too large.Our observation: Yes – reduce width by approximating.
![Page 21: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/21.jpg)
Knapsack and Monotone ROBPs
21
n layers
Order vertices by partial sums
![Page 22: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/22.jpg)
Approximating with Small Width
22
Intuition: Only need to know when acc. prob. increase.
![Page 23: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/23.jpg)
Approximating ROBP: Rounding
23
Say we know when jumps occur. How about edges?
Approximating: error-factor per layer is
Round edges
![Page 24: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/24.jpg)
Problem: Finding probabilities is another knapsack instance.
Solution: Build ROBP backward one layer at time.When rounding layer i, already know the
following layers.
Computing an Approximating ROBP
24
Build ROBP backwards with binary search.
![Page 25: Counting Algorithms for Knapsack and Related Problems](https://reader033.fdocuments.net/reader033/viewer/2022061615/5681642a550346895dd5ebc3/html5/thumbnails/25.jpg)
Thank You
25