Post on 27-Jun-2020
1
Chapter 7: Sorting
(Insertion Sort, Shellsort)
CE 221
Data Structures and Algorithms
Izmir University of Economics
Text: Read Weiss, § 7.1 – 7.4
2
Preliminaries• Main memory sorting algorithms
• All algorithms are Interchangeable; an
array containing the N elements will be
passed.
• “<“, “>” (comparison) and “=“ (assignment)
are the only operations allowed on the
input data : comparison-based sorting
Izmir University of Economics
3
Insertion Sort• One of the simplest sorting algorithms
• Consists of N-1 passes.
• for pass p = 1 to N-1 (0 thru p-1 already known to be sorted)
– elements in position 0 trough p (p+1
elements) are sorted by moving the element
left until a smaller element is encountered.
Izmir University of Economics
4
Insertion Sort - Algorithm
N*N iterations, hence, time complexity = O(N2) in the worst case.This bound
is tight (input in the reverse order). Number of element comparisons in the
inner loop is p, summing up over all p = 1+2+...+N-1 = Θ(N2). If the input is
sorted O(N).Izmir University of Economics
5
A Lower Bound for Simple Sorting
Algorithms
• An inversion in an array of numbers is
any ordered pair (i, j) such that a[i] > a[j].
• In the example; 9 inversions.
• Swapping two adjacent elements that are
out of place removes exactly one inversion
and a sorted array has no inversions.
• The running time of insertion sort O(I+N).
Izmir University of Economics
6
Average Running Time for
Simple Sorting - I
• Theorem: The average number of inversions
for N distinct elements is N(N-1)/4.
• Proof: It is the sum of the number of inversions
in N! different permutations divided by N!. Each
permutation L has a corresponding permutation
LR which is reversed in sequence. If L has x
inversions, then LR has N(N-1)/2 – x inversions.
As a result ((N(N-1)/2) * (N!/2)) / N! = N(N-1)/4
is the number of inversions for an average list.
Izmir University of Economics
7
• Theorem: Any algorithm that sorts by exchanging adjacent elements requires Ω(N2) time on average.
• Proof: Initially there exists N(N-1)/4 inversions on the average and each swap removes only one inversion, so Ω(N2) swaps are required.
• This is valid for all types of sorting algorithms (including those undiscovered) that perform only adjacent exchanges.
• Result: For a sorting algorithm to run subquadratic (o(N2)), it must exchange elements that are far apart (eliminating more than just one inversion per exchange).
Average Running Time for
Simple Sorting - II
Izmir University of Economics
8
Shellsort - I
• Shellsort, invented by Donald Shell, works
by comparing elements that are distant;
the distance decreases as the algorithm
runs until the last phase (diminishing
increment sort)
• Sequence h1, h2, ..., ht is called the
increment sequence. h1 = 1 always. After
a phase, using hk, for every i, a[i] ≤ a[i+hk].
The file is then said to be hk-sorted.
Izmir University of Economics
9
Shellsort - II• An hk-sorted file that is then hk-1-sorted
remains hk-sorted.
• To hk-sort, for each i in hk,hk+1,...,N-1,
place the element in the correct spot
among i, i-hk, i-2hk. This is equivalent to
performing an insertion sort on hk
independent subarrays.
Izmir University of Economics
10
Shellsort - III
Increment sequence by Shell: ht=floor(N/2),
hk=floor(hk+1/2) (poor)
Izmir University of Economics
11
Worst-Case Analysis of Shellsort - I
• Theorem: The worst case running time of
Shellsort, using Shell’s increments, is
Θ(N2).
• Proof: part I: prove Ω(N2)= Why?
smallest N/2 elements goes from position
2i-1 to i during the last pass. Previous
passes all have even increments.
Izmir University of Economics
2
11N/
i)(i
12
• Proof: part II: prove O(N2)
• A pass with increment hk consists of hk
insertion sorts of about N/hk elements.
One pass,hence, is O(hk(N/hk)2). Summing
over all passes which is O(N2).
• Shell’s increments: pairs of increments
are not relatively prime.
• Hibbard’s increments: 1, 3, 7,..., 2k-1
Worst-Case Analysis of Shellsort - II
Izmir University of Economics
ti ihNO 1
2 )/(
13
Worst-Case Analysis of Shellsort - III• Theorem: The worst case running time of Shellsort, using Hibbard’s
increments, is Θ(N3/2).
• Proof: (results from additive number theory)
- for hk>N1/2 use the bound O(N2/hk) // hk=1, 3, 7,..., 2t-1
- hk+2 done, hk+1done, hk now.
- a[p-i] < a[p] if
- but hk+2=2hk+1+1 hence gcd(hk+2, hk+1) = 1
- Thus all can be expressed as such .
- Therefore; innermost for loop executes O(hk) times for each N-hk
positions. This gives a bound of O(Nhk) per pass.
Izmir University of Economics
0,21 baforbhahi kk
kh
kh
kh
khi 428)1
2)(1
1(
)()()()(
)/1()/(
2/2/3
2/
2
2/
12/
22/
112/
22/
1
NhSinceNOh
NONhO
hNhNOhNNhO
t
t
t
t
tk
k
t
k
k
t
tk
k
t
k
k
Homework Assignments
• 7.1, 7.2, 7.3, 7.4
• You are requested to study and solve the
exercises. Note that these are for you to
practice only. You are not to deliver the
results to me.
Izmir University of Economics 14