An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and...
-
Upload
nathan-bradford -
Category
Documents
-
view
218 -
download
0
Transcript of An Introduction to Parallel Computation and Ρ-Completeness Theory Ray Greenlaw, Jim Hoover, and...
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
1
Joint work with:
An Introduction toParallel Computation
and Ρ-Completeness Theory
by
Raymond Greenlaw
An Introduction toParallel Computation
and Ρ-Completeness Theory
by
Raymond Greenlaw
Jim HooverComputing Science Department
University of Alberta
Larry Ruzzo Department of Computer
ScienceUniversity of Washington
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
2
OutlineOutline
• Introduction• Parallel Models of Computation• Basic Complexity• Evidence that NC Ρ• Ρ-Complete Problems• Comparator Circuit Class• Open Problems
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
3
OutlineOutline
• Introduction• Parallel Models of Computation• Basic Complexity• Evidence that NC Ρ• Ρ-Complete Problems• Comparator Circuit Class• Open Problems
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
4
IntroductionIntroduction
• Sequential computation:
Feasible ~ n O(1) time(polynomial time).
• Parallel computation:
Feasible ~ n O(1) time and ~ n O(1) processors.
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
5
IntroductionIntroduction
• Goal of parallel computation:Develop extremely fast ((log n)O(1) ) time algorithms using a reasonable number of processors.
• Speedup equation: (best sequential time)(number of processors)
Allow a polynomial number of processors.
(parallel time).
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
6
IntroductionIntroduction
Roughly,• A problem is feasible if it can be solved by a
parallel algorithm with worst case time and processor complexity n
O(1).• A problem is feasible highly parallel if it can be
solved by an algorithm with worst case time complexity (log n)
O(1) and processor complexity n
O(1).• A problem is inherently sequential if it is
feasible but has no feasible highly parallel algorithm for its solution.
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
7
OutlineOutline
• Introduction• Parallel Models of Computation• Basic Complexity• Evidence that NC Ρ• Ρ-Complete Problems• Comparator Circuit Class• Open Problems
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
8
Parallel Models of ComputationParallel Models of Computation
• Parallel Random Access Machine Model• Boolean Circuit Model• Circuits and PRAMs
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
9
Parallel Random Access MachineParallel Random Access Machine
RAM Processors
Global Memory Cells
P0 P1 P2
C0 C1 C2
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
10
Boolean Circuit ModelBoolean Circuit Model
1 0 1 10
AND OR
AND
ANDOR
OR
NOT
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
11
Circuits and PRAMSCircuits and PRAMS
Theorem:
A function f from {0,1}* to {0,1}* can be computed by a logarithmic space uniform Boolean circuit family {n } with depth (n) є (logn)O(1) and size (n ) є
n O(1)
if and only if
f can be computed by a CREW-PRAM M on inputs of length n in time t(n) є (logn)O(1) using p(n) є n
O(1).
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
12
OutlineOutline
• Introduction• Parallel Models of Computation• Basic Complexity• Evidence that NC Ρ• Ρ-Complete Problems• Comparator Circuit Class• Open Problems
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
13
Basic ComplexityBasic Complexity
• Decision, Function, and Search Problems• Complexity Classes• Reducibility• Completeness
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
14
Decision, Function, and Search ProblemsDecision, Function, and Search Problems
4 4
4 4
3 3 366
Spanning Tree-DGiven: An undirected graph G = (V,E) with weights from N labeling edges in E and a natural number k.Problem: Is there a spanning tree of G with cost less than or equal to k ?
Spanning Tree-FGiven: Same (no k ).Problem: Compute the weight of a minimum cost spanning tree.
Spanning Tree-SGiven: Same.Problem: Find a minimum cost spanning tree.
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
15
Complexity ClassesComplexity Classes
Definitions:
P is the set of all languages L that are decidable in sequential time n O(1).
NC is the set of all languages L that are decidable in parallel time (logn)O(1) and processors n O(1).
FP is the set of all functions from {0,1}* to {0,1}* that are computable in sequential time n O(1).
FNC is the set of all functions from {0,1}* to {0,1}* that are computable in parallel time (logn)O(1) and processors n O(1).
NC k, k 1, is the set of all languages L such that L is recognized by a uniform Boolean circuit family {n } with size(n
) = n O(1) and depth (n ) = O((logn)k ).
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
16
Definitions:
A language L is many-one reducible to a language L’, written L P L’ , if there is a function f such that x є L if and only if f(x) є L’.
L is P many-one reducible to L’, written L P L’ , if the function f is in FP.
For k 1, L is NC k many-one reducible to L’, written L NC kL’, if the function f is in FNC k.
L is NC many-one reducible to L’, written L NC L’, if the function f is in FNC.
Many-One ReducibilityMany-One Reducibility
m
m
m
m
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
17
Many-One ReducibilityMany-One Reducibility
m
Lemma:
The reducibilities m, P, NC k(k 1), and
NC are transitive.
– If L’ є P and L P L’, L
NC k L’ (k > 1), or L
NC L’ then L є P.
– If L’ є NC k (k > 1), and L NC kL’ then L є NC k.
– If L’ є NC and L NC L’ then L є NC .
m m m
mm m
m
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
18
Turing ReducibilityTuring Reducibility
Definition:
L is NC Turing reducible to L’, written L NC L’, if and only if
the L’-oracle PRAM on inputs of length n uses time (logn)O(1) and processors n O(1).
Lemma:
The reducibility NC is transitive.
– If L NC L’ then L
NC L’.
– If L NC L’ and L’ є NC then L є NC.
– If L NC L’ and L’ є P then L є P.
T
T
m T
T
T
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
19
CompletenessCompleteness
Definitions:
A language L is P-hard under NC reducibility if L’ NC L for
every L’ є P.
A language L is P-complete under NC reducibility if L є P and L is P-hard.
Theorem:If any P-complete problem is in NC then NC equals P.
T
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
20
OutlineOutline
• Introduction• Parallel Models of Computation• Basic Complexity• Evidence that NC Ρ• Ρ-Complete Problems• Comparator Circuit Class• Open Problems
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
21
Evidence That NC PEvidence That NC P
• General Simulations Are Not Fast• Fast Simulations Are Not General• Natural Approaches Provably Fail
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
22
Evidence That NC PEvidence That NC P
Gaps in Simulations
ModelResource
1Resource
2Max R2C NC
Min R2C P
DTMDTMATMPDAUCPRAM
Time = n O(1)
Time = n O(1)
2Space = n O(1)
2Space = n O(1)
Size = n O(1)
Procs = n O(1)
SpaceReversalslog(Treesize)log(Time)DepthTime
log n(log n)O(1)
(log n)O(1)
(log n)O(1)
(log n)O(1)
(log n)O(1)
n O(1)
n O(1)
n O(1)
n O(1)
n O(1)
n O(1)
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
23
OutlineOutline
• Introduction• Parallel Models of Computation• Basic Complexity• Evidence that NC Ρ• Ρ-Complete Problems• Comparator Circuit Class• Open Problems
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
24
P-Complete ProblemsP-Complete Problems
There are approximately 175 P-complete problems (500 with variations).
Categories:– Circuit complexity– Graph theory– Searching graphs– Combinatorial
optimization and flow– Local optimality– Logic
– Formal languages– Algebra– Geometry– Real analysis– Games– Miscellaneous
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
25
Circuit Value ProblemCircuit Value Problem
Given: An encoding of a Boolean circuit , inputs x1,…,xn, and a designated output y.
Problem:Is output y of TRUE on input x1,…,xn?
Theorem: [Ladner 75]The Circuit Value Problem is P-complete under
NC
1
reductions.
m
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
26
P-Complete Variations of CVPP-Complete Variations of CVP
– Topologically Ordered [Folklore]– Monotone [Goldschlager 77]– Alternating Monotone Fanin 2, Fanout 2
[Folklore]– NAND [Folklore]– Topologically Ordered NOR [Folklore]– Synchronous Alternating Monotone Fanout
2 CVP [Greenlaw, Hoover, and Ruzzo 87]– Planar [Goldschlager 77]
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
27
NAND Circuit Value ProblemNAND Circuit Value Problem
Given: An encoding of a Boolean circuit that consists solely of
NAND gates, inputs x1,…,xn, and a designated output y.
Problem: Is output y of TRUE on input x1,…,xn?
Theorem: The NAND Circuit Value Problem is P-complete.
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
28
NAND Circuit Value ProblemNAND Circuit Value Problem
Proof:Reduce AM2CVP to NAND CVP. Complement all inputs. Relabel all gates as NAND.
01 0 0 1
1
1
OR
AND
OROR
1
1
10 1 1 0
1
1
NAND
NAND
NANDNAND
1
0
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
29
Graph TheoryGraph Theory
– Lexicographically First Maximal Independent Set [Cook 85]
– Lexicographically First ( + 1)-Vertex Coloring [Luby 84]
– High Degree Subgraph [Anderson and Mayr 84]
– Nearest Neighbor Traveling Salesman Heuristic [Kindervater, Lenstra, and Shmoys 89]
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
30
Lexicographically First Maximal Independent Set
Lexicographically First Maximal Independent Set
Theorem: [Cook 85]LFMIS is P-complete.
Proof:Reduce TopNOR CVP to LFMIS. Add new vertex 0. Connect to all false inputs.
1 2 3 4 5
NOR9
NOR8
NOR7
NOR6
1 1 0 0 0
0
1
1 2 3 4 5
6 7
0
8
9
0
0
X X
X
X
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
31
Searching GraphsSearching Graphs
– Lexicographically First Depth-First Search Ordering [Reif 85]
– Stack Breadth-First Search [Greenlaw 92]
– Breadth-Depth Search [Greenlaw 93]
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
32
Context-Free Grammar EmptyContext-Free Grammar Empty
Given: A context-free grammar G=(N,T,P,S).Problem: Is L(G) empty?
Theorem: [Jones and Laaser 76], [Goldschlager 81], [Tompa 91]CFGempty is P-complete.
Proof: Reduce Monotone CVP to CFGempty. Given construct G=(N,T,P,S) with N, T, S, and P as follows:
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
33
Context-Free Grammar EmptyContext-Free Grammar Empty
N = {i | vi is a vertex in }
T = {a }S = n, where vn is the output of .
P as follows:
1. For input vi, i a if value of vi is 1,
2. i jk if vi vj Λ vk, and
3. i j | k if vi vj V vk .
Then the value of vi is 1 if and only if i , where є {a }+.*
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
34
CFGempty ExampleCFGempty Example
x1 = 0, x2 = 0, x3 = 1, and x4 = 1.
G = (N, T, S, P), whereN = {1, 2, 3, 4, 5, 6, 7 }T = {a }S = 7P = {3 a, 4 a, 5 1 | 2, 6 34, 7 56}
1 2 3 4
5 6
7
x3 x4x1 x2
OR
AND
AND
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
35
OutlineOutline
• Introduction• Parallel Models of Computation• Basic Complexity• Evidence that NC Ρ• Ρ-Complete Problems• Comparator Circuit Class• Open Problems
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
36
Comparator Circuit ClassComparator Circuit Class
Definition: [Cook 82]Comparator Circuit Value Problem (CCVP)
Given: An encoding of a circuit composed of comparator gates, plus inputs x1,…,xn, and a designated output y.Problem: Is output y of TRUE on input x1,…,xn?
Definition: CC is the class of languages NC reducible to CCVP.
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
37
CC-Complete ProblemsCC-Complete Problems
– Lexicographically First Maximal Matching [Cook 82]
– Telephone Connection [Ramachandran and Wang 91]
– Stable Marriage [Mayr and Subramanian 92]
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
38
OutlineOutline
• Introduction• Parallel Models of Computation• Basic Complexity• Evidence that NC Ρ• Ρ-Complete Problems• Comparator Circuit Class• Open Problems
An Introduction to Parallel Computation and Ρ-Completeness TheoryRay Greenlaw, Jim Hoover, and Larry Ruzzo
39
Open ProblemsOpen Problems
Find an NC algorithm or classify as P-complete.
– Edge Ranking– Edge-Weighted Matching– Restricted Lexicographically First Maximal
Independent Set– Integer Greatest Common Divisor– Modular Powering