Priority Queues and Heapsort (9.1-9.4) Priority queues are used for many purposes,

such as job scheduling, shortest path, file compression …

Recall the definition of a Priority Queue: operations insert(), delete_max() also max(), change_priority(), join()

How would I sort a list, using a priority queue?for (i=0; i<n; i++) insert(A[i]);

for (i=0; i<n; i++) cout << delete_max();

How would I implement a priority queue? how fast a sorting alg would your implementation

yield? can we do better?

Priority Queue Implementations insert max delete change join

priority

sorted array n 1 n n n

unsorted array 1 n 1 1 n

heap lg n lg n lg n lg n n

binomial queue lg n lg n lg n lg n lg n

(best) 1 lg n lg n 1 1

Heaps How can we build a data structure to do this? Hints:

we want to find the smallest element quickly we want to be able to remove an element quickly Tree of some sort?

Heap: a full binary tree (all leaves at the same level, on left) each element is at least as large as its children (note: this is not a BST!)

How to delete the maximum? How to add a number to a heap? How to build a heap out of a list of numbers?

Insert Implicit representation

XTOGSMNAERAI(children of i at 2i and 2i+1)

How would I do an insert()? add to the end of the array repeat: if larger than parent, swap

template <class Item>void insert(Item a[], Item newItem, int items){ n = ++items; a[n] = newItem; while (n>1 && a[n/2] < a[n]) { exch(a[n], a[n/2]); n/=2; }}

XTOGSMNAERIPXTOGSPNAERIMXTPGSONAERIM

Runtime?

(log n)

DeleteMax How would I delete X?

Move last element to root If larger than either child,

swap with larger child

Item DeleteMax(Item a[], int items){ exch(a[1], a[items--]); reHeapify(a, items); return a[items+1];}void reHeapify(Item a[], int items){ int n=1; while (2*n <= items) { int j = 2*n; if (j<items && a[j] < a[j+1]) j++; if (a[n] >= a[j]) break; exch(a[n],a[j]); n=j;} }

XTOGSMNAERAIITOGSMNAERAXTIOGSMNAERAXTSOGIMNAERAX

Runtime?

(log n)

BuildHeap (top down) Given an array, e.g.

ASORTINGEXAMPLE, how do I make it a heap?

Top-down: for (i=2; i<=items; i++) insert(a,a[i],i-1)

Runtime: (n log n)

Can we do better?

BuildHeap (bottom up) Suppose we use the reHeapify() function

instead and work bottom-up. For (i=items/2; i>=1; i--)

reHeapify(a)

ASORTINGEXASORXINGETAXORSINGETAXORTINGESXAORTINGESXTORAINGESXTORSINGEA

Runtime? 1+1+1+1+…+2+2+..+4+.. n/4 + 2(n/8) + 3(n/16) + 4(n/32) + … n(1/4 + 2/8 + 3/16 + 4/32 + …) n * 1

(n)! Top-down was (n log n); bottom up is (n)!

cool!

Heapsort BuildHeap() for (i=1; i<=n; i++) DeleteMax();

Runtime?(n log n)

Almost competitive with quicksort

Priority Queue Operations insert(), max(), deleteMax() Could implement with heap Runtime for each operation?

insert(), deleteMax() – O(log n) max() – O(1)

Example Application Suppose you have a text, abracadabra. Want

to compress it. How many bits required?

at 3 bits per letter, 33 bits. Can we do better? How about variable length codes? In order to be able to decode the file again,

we would need a prefix code: no code is the prefix of another.

How do we make a prefix code that compresses the text?

Huffman Coding Note: Put the letters at the leaves of a binary

tree. Left=0, Right=1. Voila! A prefix code.

Huffman coding: an optimal prefix code Algorithm: use a priority queue.

insert all letters according to frequencyif there is only one tree left, done.else, a=deleteMin(); b=deleteMin(); make tree t out of a and b with weight a.weight() +

b.weight(); insert(t)

Huffman coding example abracadabra frequencies:

a: 5, b: 2, c: 1, d: 1, r: 2 Huffman code:

a: 0, b: 100, c: 1010, d: 1011, r: 11 bits: 5 * 1 + 2 * 3 + 1 * 4 + 1 * 4 + 2 * 2 = 23

Finite automaton to decode – (n) Time to encode?

Compute frequencies – O(n) Build heap – O(1) assuming alphabet has constant

size Encode – O(n)

Huffman coding summary Huffman coding is very frequently used (You use it every time you watch HTDV or

listen to mp3, for example) Text files often compress to 60% of original

size In real life, Huffman coding is usually used in

conjunction with a modeling algorithm… E.g. jpeg compression: DCT, quantization, and

Huffman coding Text compression: dictionary + Huffman

coding

Finite Automata and Regular Expressions How can I decode some Huffman-encoded text

efficiently?(hand-design a dfa to recognize)

Or: how can I find all instances of aardvark, aaardvark, aaaardvark, etc. or zyzzyva, zyzzzyva, zyzzzzyva, etc. in Microsoft Word? Unix? (grep)

All words with 2 or more As or Zs? Important topic: regular expressions and finite

automata. theoretician: regular expressions are grammars that

define regular languages programmer: compact patterns for matching and

replacing

DFA for abracadabra Huffman code: A=0, B=100, C=1010, D=1011,

E=11 DFA:

state read out new state

0 0 A 0

0 1 1

1 0 2

1 1 R 0

2 0 B 0

2 1 3

3 0 C 0

3 1 D 0 (Actually, this looks just like the original tree,

doesn’t it.)

Regular Expressions Regular expressions are one of

a literal character a (regular expression) – in parentheses a concatenation of two R.E.s the alternation (“or”) of two R.E.s, denoted + the closure of an R.E., denoted * (i.e 0 or more occurrences) Possibly additional syntactic sugar

Examplesabracadabraabra(cadabra)* = {abra, abracadabra, abracadabracadabra, …

}(a*b + ac)d(a(a+b)b*)*t(w+o)?o [? means 0 or 1 occurrences]aa+rdvark [+ means 1 or more occurrences]

Finite Automata Regular language: any language defined by a R.E. Finite automata: machines that recognize regular

languages. Deterministic Finite Automaton (DFA):

a set of states including a start state and one or more accepting states

a transition function: given current state and input letter, what’s the new state?

Non-deterministic Finite Automaton (NDFA): like a DFA, but there may be

more than one transition out of a state on the same letter (Pick the right one non-deterministically, i.e. via lucky guess!)

epsilon-transitions, i.e optional transitions on no input letter

RE NDFA Given a Regular Expression, how can I build a

DFA? Work bottom up. Letter:

Concatenation:

Or: Closure:

RE -> NDFA Ex Construct an NDFA for the RE

(A*B + AC)D

AA*A*BA*B + AC(A*B + AC)D

NDFA -> DFA Keep track of the set of states you are in. On each new input letter, compute the new

set of states you could be in.

The set of states for the DFA is the power set of the NDFA states.

I.e. up to 2n states, where there were n in the DFA.

Recognizing Regular Languages Suppose your language is given by a DFA.

How to recognize? Build a table. One row for every (state,input

letter) pair. Give resulting state. For each letter of input string, compute new

state When done, check whether the last state is an

accepting state. Runtime?

O(n), where n is the number of input letters Another approach: use a C program to

simulate NDFA with backtracking. Less space, more time. (egrep vs. fgrep?)

Examples Unix grep Perl$input =~ s/t[wo]?o/2;

$input =~ s|<link[^>]*>\s*||gs;

$input =~ s|\s*\@font-face\s*{.*?}||gs;

$input =~ s|\s*mso-[^>"]*"|"|gis;

$input =~ s/([^ ]+) +([^ ]+)/$2 $1/;

$input =~ m/^[0-9]+\.?[0-9]*|\.[0-9]+$/;

($word1,$word2,$rest) =

($foo =~ m/^ *([^ ]+) +([^ ]+) +(.*)$/);

$input=~s|<span[^>]*>\s*<br\s+clear="?all[^>]*>\s*</span>|<br clear="all"/>|gis;