I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.
-
date post
21-Dec-2015 -
Category
Documents
-
view
220 -
download
2
Transcript of I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.
![Page 1: I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649d5a5503460f94a3a6d8/html5/thumbnails/1.jpg)
I/O-Algorithms
Lars Arge
University of Aarhus
March 7, 2005
![Page 2: I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649d5a5503460f94a3a6d8/html5/thumbnails/2.jpg)
Lars Arge
I/O-algorithms
2
I/O-Model
• Parameters
N = # elements in problem instance
B = # elements that fits in disk block
M = # elements that fits in main memory
T = # output size in searching problem
• I/O: Movement of block between memory and disk
D
P
M
Block I/O
![Page 3: I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649d5a5503460f94a3a6d8/html5/thumbnails/3.jpg)
Lars Arge
I/O-algorithms
3
Fundamental Bounds [AV88]
Internal External
• Scanning: N
• Sorting: N log N
• Permuting
– List rank
• Searching: NBlog
BN
BN
BMlog
BN
log,minBN
BN
BMNN
N2log
![Page 4: I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649d5a5503460f94a3a6d8/html5/thumbnails/4.jpg)
Lars Arge
I/O-algorithms
4
Until now: Data Structures• B-trees
– Trees with fanout B, balanced using split/fuse
– query, space, update
• Weight-balanced B-trees
– Weight balancing constraint rather than degree constraint
– Ω(w(v)) updates below v between consecutive operations on v
• Persistent B-trees
– Update current version (getting new version)
– Query all versions
• Buffer-trees
– amortized bounds using buffers and lazyness
)(log NO B)(log BT
B NO )(BNO
)log( 1BN
BMBO
![Page 5: I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649d5a5503460f94a3a6d8/html5/thumbnails/5.jpg)
Lars Arge
I/O-algorithms
5
Until now: Data Structures• Special cases of two-dimensional range search:
– Diagonal corner queries: External interval tree
– Three-sided queries: External priority search tree
query, space, update
• Same bounds cannot be obtained for general planar range searching
q3
q2q1q
q
)(log NO B)(log BT
B NO )(BNO
![Page 6: I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649d5a5503460f94a3a6d8/html5/thumbnails/6.jpg)
Lars Arge
I/O-algorithms
6
Until now: Data Structures
• General planer range searching:
– External range tree: query, space,
update
– O-tree: query, space, update
q3
q2q1
q4
)(loglog
logN
NBN
BB
BO)(log BT
B NO
)(BNO)( B
TB
NO )(log NO B
)(loglog
log2
NN
BB
BO
![Page 7: I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649d5a5503460f94a3a6d8/html5/thumbnails/7.jpg)
Lars Arge
I/O-algorithms
7
Techniques• Tools:
– B-trees
– Persistent B-trees
– Buffer trees
– Logarithmic method
– Weight-balanced B-trees
– Global rebuilding
• Techniques:
– Bootstrapping
– Filtering
q3
q2q1
q3
q2q1
q4
(x,x)
![Page 8: I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649d5a5503460f94a3a6d8/html5/thumbnails/8.jpg)
Lars Arge
I/O-algorithms
8
Point Enclosure Queries• Dual of 2d range searching problem
– Report all rectangles containing query point (x,y)
• Internal memory:
– Can be solved in O(N) space and O(log N + T) time
x
y
![Page 9: I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649d5a5503460f94a3a6d8/html5/thumbnails/9.jpg)
Lars Arge
I/O-algorithms
9
Point Enclosure Queries• Similarity between internal and external results (space, query)
– in general tradeoff between space and query I/O
Internal External
1d range search (N, log N + T) (N/B, logB N + T/B)
3-sided 2d range search (N, log N + T) (N/B, logB N + T/B)
2d range search
2d point enclosure (N, log N + T)
BT
NN NN log,
logloglog B
TBN
NBN N
BB
B log,loglog
log
TNN , BT
BN
BN ,
(N/B, log N + T/B)?2B
(N/B, log N+T/B)
(N/B1-ε, logB N+T/B)
![Page 10: I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649d5a5503460f94a3a6d8/html5/thumbnails/10.jpg)
Lars Arge
I/O-algorithms
10
Rectangle Range Searching• Report all rectangles intersecting query rectangle Q
• Often used in practice when handling
complex geometric objects
Q
![Page 11: I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649d5a5503460f94a3a6d8/html5/thumbnails/11.jpg)
Lars Arge
I/O-algorithms
11
R-Trees• Herman!
![Page 12: I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649d5a5503460f94a3a6d8/html5/thumbnails/12.jpg)
Lars Arge
I/O-algorithms
12
Geometric Algorithms• We will now (shortly) look at geometric algorithms
– Solves problem on set of objects
• Example: Orthogonal line segment intersection
– Given set of axis-parallel line segments, report all intersections
• In internal memory many problems is solved using sweeping
![Page 13: I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649d5a5503460f94a3a6d8/html5/thumbnails/13.jpg)
Lars Arge
I/O-algorithms
13
Plane Sweeping
• Sweep plane top-down while maintaining search tree T on vertical segments crossing sweep line (by x-coordinates)
– Top endpoint of vertical segment: Insert in T
– Bottom endpoint of vertical segment: Delete from T
– Horizontal segment: Perform range query with x-interval on T
![Page 14: I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649d5a5503460f94a3a6d8/html5/thumbnails/14.jpg)
Lars Arge
I/O-algorithms
14
Plane Sweeping
• In internal memory algorithm runs in optimal O(Nlog N+T) time
• In external memory algorithm performs badly (>N I/Os) if |T|>M
– Even if we implements T as B-tree O(NlogB N+T/B) I/Os
• Solution: Distribution sweeping
![Page 15: I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649d5a5503460f94a3a6d8/html5/thumbnails/15.jpg)
Lars Arge
I/O-algorithms
15
Distribution Sweeping
• Divide plane into M/B-1 slabs with O(N/(M/B)) endpoints each
• Sweep plane top-down while reporting intersections between
– part of horizontal segment spanning slab(s) and vertical segments
• Distribute data to M/B-1 slabs
– vertical segments and non-spanning parts of horizontal segments
• Recurse in each slab
![Page 16: I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649d5a5503460f94a3a6d8/html5/thumbnails/16.jpg)
Lars Arge
I/O-algorithms
16
Distribution Sweeping
• Sweep performed in O(N/B+T’/B) I/Os I/Os
• Maintain active list of vertical segments for each slab ( <B in memory)
– Top endpoint of vertical segment: Insert in active list
– Horizontal segment: Scan through all relevant active lists
* Removing “expired” vertical segments
* Reporting intersections with “non-expired” vertical segments
)log(BT
BN
BN
BMO
![Page 17: I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649d5a5503460f94a3a6d8/html5/thumbnails/17.jpg)
Lars Arge
I/O-algorithms
17
Distribution Sweeping• Other example: Rectangle intersection
– Given set of axis-parallel rectangles, report all intersections.
![Page 18: I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649d5a5503460f94a3a6d8/html5/thumbnails/18.jpg)
Lars Arge
I/O-algorithms
18
Distribution Sweeping
• Divide plane into M/B-1 slabs with O(N/(M/B)) endpoints each
• Sweep plane top-down while reporting intersections between
– part of rectangles spanning slab(s) and other rectangles
• Distribute data to M/B-1 slabs
– Non-spanning parts of rectangles
• Rcurse in each slab
![Page 19: I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649d5a5503460f94a3a6d8/html5/thumbnails/19.jpg)
Lars Arge
I/O-algorithms
19
Distribution Sweeping
• Seems hard to perform sweep in O(N/B+T’/B) I/Os
• Solution: Multislabs
– Reduce fanout of distribution to
– Recursion height still
– Room for block from each multislab (activlist) in memory
)( BM
)(logBN
BMO
![Page 20: I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649d5a5503460f94a3a6d8/html5/thumbnails/20.jpg)
Lars Arge
I/O-algorithms
20
Distribution Sweeping
• Sweep while maintaining rectangle active list for each multisslab
– Top side of spanning rectangle: Insert in active multislab list
– Each rectangle: Scan through all relevant multislab lists
* Removing “expired” rectangles
* Reporting intersections with “non-expired” rectangles I/Os)log(
BT
BN
BN
BMO
![Page 21: I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649d5a5503460f94a3a6d8/html5/thumbnails/21.jpg)
Lars Arge
I/O-algorithms
21
Distribution Sweeping• Distribution sweeping can relatively easily be used to solve a
number of other problems in the plane I/O-efficiently
• By decreasing distribution fanout to for c≥1 a number of higher-dimensional problems can also be solved I/O-efficiently
))((1
cBMO
![Page 22: I/O-Algorithms Lars Arge University of Aarhus March 7, 2005.](https://reader036.fdocuments.net/reader036/viewer/2022062407/56649d5a5503460f94a3a6d8/html5/thumbnails/22.jpg)
Lars Arge
I/O-algorithms
22
Other Results
• Other geometric algorithms results include:
– Red blue line segment intersection
(using distribution sweep, buffer trees/batched filtering, external fractional cascading)
– General planar line segment intersection
(as above and external priority queue)
– 2d and 2d Convex hull:
(Complicated deterministic 3d and simpler randomized)
– 2d Delaunay triangulation