ICC Module Computation –Theory of Computation Feedback · ICC Module Computation –Theory of...
Transcript of ICC Module Computation –Theory of Computation Feedback · ICC Module Computation –Theory of...
ICC Module Computation – Theory of Computation
1
Feedback75-100% : 41%50- 75%: 44%
<50%: 15%
Next Friday: no new topic, instead we will revise the topics learned so far!
ICC Module Computation – Theory of Computation
2
Complexity Example Behavior Example Algorithm
Θ(log n) Look at only log n elements of the input Binary search (Recherche dichotomique)
Θ(n) Look at each input element once Linear search
Θ(n・log n) Split input into half (which can be done at most log n times)At each split look at each element once
Merge sort (tri fusion)
Θ(n2) Look at each pair of elments (i,j) Insertion sort
Θ(n3) Look at each triple of element (i,j,k) Floyd’s shortest path algorithm
Θ(2n) Look at all subset of a list orall permutation of a list orall paths in a graph
Create a list of binary numbers of length n, e.g., input: 3output:{{0,0,0},{0,0,1},{0,1,0},{0,1,1},
{1,0,0},{1,0,1},{1,1,0},{1,1,1}}
Representative Algorithms
In this table “look at element x” means to do a fix number instructions for element x, e.g., compare x with 3, add x to another variable,…
ICC Module Computation – Theory of Computation
3
Information, Computation, and Communication
Theory of Computation
ICC Module Computation – Theory of Computation
4
TopicsStudy the two big questions of the theory of computation:§ Which problems can we solve with an algorithm?§ Which problems can we solve efficiently with an
algorithm?
Important notions:§ Countability§ Decidability (video)§ Complexity classes: P and NP (video)
ICC Module Computation – Theory of Computation
5
§ Previously, we have studied algorithms that work regardless of the input data.
§ Each algorithm solves the same question for different input data each time.
§ Running the algorithm on one input gives us, the answer to one instance of the problem
§ Running an algorithm is not the solution of the problem, the solution is the algorithm itself
Algorithms & Problems
ICC Module Computation – Theory of Computation
6
§ sort({4, 6, 2,10}) … one instance§ max({4, 6, 2,10}) … one instance§ add(3, 5) … one instance§ add(x, y) … infinitely many instances
(if x and y are not bounded)§ add(x, y), where x, y from {1, 2, 3, 4, 5}
…finitely many instances
Problem Instances
ICC Module Computation – Theory of Computation
7
§ If the set of problem instances is finite, it is always possible to resolve the problem by building a Look-Up Table (LUT): we store the answer to every instance. The algorithm consults this table and prints the solution found inside the table – no computation involved!
§ Only the problems involving infinite sets of instances require real work. They are the ones that interest us in today's class.
Infinity of Instances
ICC Module Computation – Theory of Computation
8
§ add(x,y) where x, y from {1, 2, 3, 4, 5}Example: Look-Up Table
1 2 3 4 5
1 2 3 4 5 6
2 3 4 5 6 7
3 4 5 6 7 8
4 5 6 7 8 9
5 6 7 8 9 10
xy
Another example: multiplication table until 10
ICC Module Computation – Theory of Computation
9
§ add(x,y) where x, y from {1, 2, 3, 4, 5}Example: Look-Up Table
xy
1 2 3 4 5
1 2 3 4 5 6
2 3 4 5 6 7
3 4 5 6 7 8
4 5 6 7 8 9
5 6 7 8 9 10
xy
ICC Module Computation – Theory of Computation
10
§ What is the complexity?Example: Look-Up Table
xy x
y
ICC Module Computation – Theory of Computation
11
§ Notation: Countability§ Observation:
the natural numbers, N = {1, 2, 3,...}, define counting.
§ Definition: an infinite set S is countable if and only if there exists a bijection f : N → S (a 1-to-1 mapping)
§ Function f defines the enumeration of S.
Comparing Infinite Sets
ICC Module Computation – Theory of Computation
12
§ The integers, ℤ = {… ,−3,−2,−1, 0, 1, 2, 3… }, are a countable set.
§ We define a function 𝑓:ℕ → ℤ as follows:• if i is even, then 𝑓 𝑖 = !
"• if i is odd, then 𝑓 𝑖 = − !#$
"
Example: Integers
𝑖 ∈ ℕ 1 2 3 4 5 6 …
f(𝑖) ∈ ℤ 0 1 -1 2 -2 3 …
ICC Module Computation – Theory of Computation
13
§ Is the set of even numbers countable?§ Yes, we define a bijection as follows:
𝑓 𝑖 = 2 % 𝑖
Countability: Example
𝑖 ∈ ℕ 1 2 3 4 5 6 …
Even numbers f(𝑖) 2 4 6 8 10 12 …
ICC Module Computation – Theory of Computation
14
§ Is the set of rational numbers countable? ℚ =
𝑝𝑞| 𝑝, 𝑞 ∈ ℕ 𝑎𝑛𝑑 𝑞 ≠ 0
Countability: Another Example
1 2 3 4 …
1 1/1 1/2 1/3 1/4 ..
2 2/1 2/2 2/3 2/4 …
3 3/1 3/2 3/3 3/4 …
4 4/1 4/2 4/3 4/4 …
… … … … … …
pq
ICC Module Computation – Theory of Computation
15
§ Is the set of rational numbers countable? ℚ =
𝑝𝑞| 𝑝, 𝑞 ∈ ℕ 𝑎𝑛𝑑 𝑞 ≠ 0
Countability: Another Example
1 2 3 4 …
1 1/1 1/2 1/3 1/4 ..
2 2/1 2/2 2/3 2/4 …
3 3/1 3/2 3/3 3/4 …
4 4/1 4/2 4/3 4/4 …
… … … … … …
pq
You can use the same order to counter pairs of integers (i,j)
ICC Module Computation – Theory of Computation
16
§ Is the set of rational numbers countable? ℚ =
𝑝𝑞| 𝑝, 𝑞 ∈ ℕ 𝑎𝑛𝑑 𝑞 ≠ 0
Countability: Another Example
1 2 3 4 …
1 1/1 1/2 1/3 1/4 ..
2 2/1 2/2 2/3 2/4 …
3 3/1 3/2 3/3 3/4 …
4 4/1 4/2 4/3 4/4 …
… … … … … …
pq
You can use the same order to counter pairs of integers (i,j)
ICC Module Computation – Theory of Computation
17
§ Every algorithm has a finite description, which is a text written in a chosen alphabet. It is easy to enumerate all the possible texts.
§ Let’s start by enumerating the texts of one character (26):
§ Let’s continue with the texts of two characters (26・26 = 676)
§ Now, let’s move to the texts of 3 characters:
§ The set of algorithms is smaller than the set of all possible texts because some texts are not algorithms. Therefore, the set of algorithms is countable.
The Set of Algorithms is Countable
𝑖 ∈ ℕ 1 2 3 4 5 6 …
Text a b c d e f …
𝑖 ∈ ℕ 27 28 29 30 31 32 …
Text aa ab ac ad ae af …
𝑖 ∈ ℕ 703 704 705 706 707 708 …
Text aaa aab aac aad aae aaf …
ICC Module Computation – Theory of Computation
18
§ Let’s look at a subset of problems, namely all problems that take an integer as input and return true or false.
§ This is the set of all Boolean functions 𝑓 𝑖 ∶ ℕ → 0, 1
The Set of Problems is not Countable
ICC Module Computation – Theory of Computation
19
Consider the set of Boolean functions of an integer variable B = 𝑓 | 𝑓 𝑖 ∶ ℕ → 0, 1
Each function is represented by a list of binary values:
If the set B were countable, we could enumerate the functions:f1, f2, f3, f4, f5, f6, f7, . . .Let’s put them in a table format:
n 1 2 3 4 5 6 7 …f1 0 1 1 0 0 0 1 . . .f2 0 0 0 0 1 0 1 . . .f3 1 1 0 0 1 1 1 . . .f4 0 1 0 1 0 1 1 . . .f5 1 0 1 0 0 0 0 . . .f6 1 1 0 1 1 0 1 . . .f7 0 0 0 1 1 1 1 . . .
The Set of Boolean Functions is Not Countable
n 1 2 3 4 5 6 7 . . .f (n) 0 0 0 1 0 0 1 . . .
ICC Module Computation – Theory of Computation
20
Consider the set of Boolean functions of an integer variable B = 𝑓 | 𝑓 𝑖 ∶ ℕ → 0, 1
Each function is represented by a list of binary values:
If the set B were countable, we could enumerate the functions.f1, f2, f3, f4, f5, f6, f7, . . .
The Set of Boolean Functions is Not Countable
Observe the diagonal:a list of binary values
n 1 2 3 4 5 6 7 . . .f (n) 0 0 0 1 0 0 1 . . .
n 1 2 3 4 5 6 7 …f1 0 1 1 0 0 0 1 . . .f2 0 0 0 0 1 0 1 . . .f3 1 1 0 0 1 1 1 . . .f4 0 1 0 1 0 1 1 . . .f5 1 0 1 0 0 0 0 . . .f6 1 1 0 1 1 0 1 . . .f7 0 0 0 1 1 1 1 . . .
ICC Module Computation – Theory of Computation
21
§ If B is countable, we can enumerate the functions: f1, f2, f3, f4, f5, f6, f7, . . .
§ The diagonal is a list of binary values 0 0 0 1 0 0 1 …
§ It can be seen as another Boolean function
6𝑓 𝑖 ∶ ℕ → 0, 1that can be written as
6𝑓 𝑖 = 𝑓!(𝑖)
The Set of Boolean Functions is Not Countable
n 1 2 3 4 5 6 7 …f1 0 1 1 0 0 0 1 . . .f2 0 0 0 0 1 0 1 . . .f3 1 1 0 0 1 1 1 . . .f4 0 1 0 1 0 1 1 . . .f5 1 0 1 0 0 0 0 . . .f6 1 1 0 1 1 0 1 . . .f7 0 0 0 1 1 1 1 . . .
ICC Module Computation – Theory of Computation
22
§ Let’s define a new Boolean function 𝑓∗ 𝑖 = 1 − 6𝑓 = 1 − 𝑓!(𝑖)𝑓∗ ∶ 1 1 1 0 1 1 0 . . .
§ 𝑓∗ does not appear in the enumeration because for every number i, we have 𝑓∗ 𝑖 ≠ 𝑓!(𝑖)
⇒ B is not countable.For all enumerations, there is always a function 𝑓∗ ∈ 𝐵that is not enumerated.
The Set of Boolean Functions is Not Countable
n 1 2 3 4 5 6 7 …f1 0 1 1 0 0 0 1 . . .f2 0 0 0 0 1 0 1 . . .f3 1 1 0 0 1 1 1 . . .f4 0 1 0 1 0 1 1 . . .f5 1 0 1 0 0 0 0 . . .f6 1 1 0 1 1 0 1 . . .f7 0 0 0 1 1 1 1 . . .
ICC Module Computation – Theory of Computation
23
What have we learned?§ All algorithms (whatever the language) are countable.§ Not all functions are countable. There are more
functions (and therefore problems) than algorithm.
In reality, most functions cannot be computed using an algorithm!
(There are simply not enough possible algorithm.)
Not Enough Algorithms!
ICC Module Computation – Theory of Computation
24
§ Hilbert's Tenth Problem: "Does a given Diophantine equation have an integer solution?”E.g., the equation 3x2 – 2xy – y2z – 7=0 has an integer solution: x=1, y=2, z=-2By contrast, the equation x2 + y2 + 1=0 has no such solution.
§ Tiling Problem: "Can this set of domino-like tiles tile the plane?” § Post Correspondence Problem:
"Given two lists of strings, is there a list of indices such that the concatenationsfrom both lists are equal?”
§ Security: "Does this computer program have any security vulnerabilities?” (Identifying buffer overflows is undecidable)
§ Deadcode: does a program contain a piece of useless code? (Rice’s Theorem)
§ Program Equivalent: do two programs calculate the same thing? (Rice’s Theorem)
Some Undecidable Problems
ICC Module Computation – Theory of Computation
25
§ Yes, if you can find an algorithm to solve P1
§ No, if you can find another problem P2 that is know to be undecidable (e.g., the Halting problem) and you can “reduce P2 to P1”, i.e.,• Show that if you had an algorithm for P1, then you
could also solve P2.• This shows that P1 is at least as difficult as P2. • Since P2 is undecidable, so is P1.
Is Problem P1 is decidable?
ICC Module Computation – Theory of Computation
26
§ Several problems do not have an algorithmic solution.
§ Some of these problems are artificial and present no interest, but others are important in practice, such as questions on program characteristics.
§ However, there may exist algorithms that solve common (but not all) instances of such problems.
Summary
ICC Module Computation – Theory of Computation
27
TopicsStudy the two big questions of the theory of computation:§ Which problems can we solve with an algorithm?§ Which problems can we solve efficiently with an
algorithm?
Important notions:§ Countability§ Decidability (video)§ Complexity classes: P and NP (video)
ICC Module Computation – Theory of Computation
28
§ How can we know if our algorithm is the most efficient?
§ We determine the complexity of the problem§ Like for the algorithms, we do it for the
computation time (time) and the storage requirements (space).
§ The complexity of a problem is the complexity of the best algorithm that solves that problem.
Complexity: Efficiency Measurement
ICC Module Computation – Theory of Computation
29
§ The complexity class which has become a synonym for the set of efficient solutions is P, and contains all problems for which there is an algorithm that solves any instance of this problem in polynomial time.
§ Problems in P:• Binary search: log n• Find one’s friends in a social network: n• Sorting: n log n• The shortest path in a graph: n log n• Alignment of two DNA sequences: n2
Solution in Polynomial Time: P
ICC Module Computation – Theory of Computation
30
§ The complexity class characterizing the problems whose solutions can be verified efficiently (in polynomial time) is NP (Nondeterministic Polynomial)
§ Example of Problems in NP:• Find a subset of integers whose sum is 0 in a list• Finding a DNA sequence that best fits a collection of
fragments of the sequence• Determine if a mathematical statement has a short
proof
Verification in Polynomial Time: NP
ICC Module Computation – Theory of Computation
31
§ The difference between P and NP (both defined in terms of polynomial time) is the same as the difference between solving a problem (compute a solution) and verifying correctness of the solution.
? * ? = 91Is 7 * 13 = 91 correct?Sort the list [3,4,5,2]
Is the list [3,4,5,2] sorted?
From P to NPPolynomial (P) vs Nondeterministic Polynomial (NP)
ICC Module Computation – Theory of Computation
32
§ The biggest theoretical problem of today (in Computer Science and in Mathematics):
Is P equal to NP?
§ Option (1): we find a problem in NP which is not in P
§ Option (2): we show that an NP-hard (NP difficile) problem, e.g., the Clique problem, can be solved in a polynomial way
P vs NP
Clique = group in which everyone knows everyone
Millennium problem: the Clay Math Institute named the P versus NPproblem as one of the seven most important open questions in mathematics and offers a 1 million-dollar prize for solving it.
ICC Module Computation – Theory of Computation
33
§ Yes, if you can find an algorithm that solves any instance of this problem in polynomial time.
Is a Problem in P?
§ Yes, if you can find an algorithm that verifiesthat a solution is correct in polynomial time.
Is a Problem in NP?
ICC Module Computation – Theory of Computation
34
§ Yes, if you can find another problem P2 that is known to be NP-hard (e.g., the Clique problem) and you can reduce P2 to P1 in polynomial time, i.e.,• Show that if you had an algorithm for P1, then you
could also solve P2 using only polynomially many steps to convert a solution for P1 to a solution for P2.
§ This shows that P1 is at least as difficult as P2.§ Since P2 is NP-hard, so is P1.
Is a Problem P1 NP-hard (NP difficile)?
ICC Module Computation – Theory of Computation
35
Questions ?