Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.
-
Upload
james-wilkins -
Category
Documents
-
view
238 -
download
0
Transcript of Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.
![Page 1: Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.](https://reader036.fdocuments.net/reader036/viewer/2022081503/56649f045503460f94c18b67/html5/thumbnails/1.jpg)
Dense Image Over-segmentation on a GPU
Alex Rodionov4/24/2009
![Page 2: Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.](https://reader036.fdocuments.net/reader036/viewer/2022081503/56649f045503460f94c18b67/html5/thumbnails/2.jpg)
Outline
1) Problem/Motivation
2) Goals
3) Overview of Algorithm
4) Implementation
5) Results
6) Future work & Conclusions
![Page 3: Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.](https://reader036.fdocuments.net/reader036/viewer/2022081503/56649f045503460f94c18b67/html5/thumbnails/3.jpg)
Problem
![Page 4: Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.](https://reader036.fdocuments.net/reader036/viewer/2022081503/56649f045503460f94c18b67/html5/thumbnails/4.jpg)
Applications
Computer vision (object recognition, intelligent segmentation, ...) Apply Algorithm X to a grid of superpixels vs. a grid
of pixels Image compression
![Page 5: Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.](https://reader036.fdocuments.net/reader036/viewer/2022081503/56649f045503460f94c18b67/html5/thumbnails/5.jpg)
Goals
Run at interactive framerates Fast enough for interactive use (video) Segment a 640x480 image in 200ms or less (5+
FPS)
![Page 6: Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.](https://reader036.fdocuments.net/reader036/viewer/2022081503/56649f045503460f94c18b67/html5/thumbnails/6.jpg)
The Algorithm
0) Preprocess image (to grayscale, smooth) 1) Calculate speed map 2) Place N seed points throughout the image 3) Initialize a distance function to these seeds 4) Evolve distance function one timestep 5) Superpixel boundaries: pixels where distance
func == 0
![Page 7: Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.](https://reader036.fdocuments.net/reader036/viewer/2022081503/56649f045503460f94c18b67/html5/thumbnails/7.jpg)
0) Preprocess
![Page 8: Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.](https://reader036.fdocuments.net/reader036/viewer/2022081503/56649f045503460f94c18b67/html5/thumbnails/8.jpg)
1) Calculate Speed
- Function of image gradient magnitude (edge strength)
![Page 9: Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.](https://reader036.fdocuments.net/reader036/viewer/2022081503/56649f045503460f94c18b67/html5/thumbnails/9.jpg)
2) Place seeds
![Page 10: Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.](https://reader036.fdocuments.net/reader036/viewer/2022081503/56649f045503460f94c18b67/html5/thumbnails/10.jpg)
3) Init Distance Function
Pixel value = distance to closest seed
![Page 11: Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.](https://reader036.fdocuments.net/reader036/viewer/2022081503/56649f045503460f94c18b67/html5/thumbnails/11.jpg)
4) Evolve the function
- Superpixels grow over time- Evolution done by partial differential equation- d(pixel)/dt function of spatial derivatives, speed, and proximity to other superpixels
![Page 12: Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.](https://reader036.fdocuments.net/reader036/viewer/2022081503/56649f045503460f94c18b67/html5/thumbnails/12.jpg)
5) Extract boundary
Zero-crossings of distance function define boundary
![Page 13: Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.](https://reader036.fdocuments.net/reader036/viewer/2022081503/56649f045503460f94c18b67/html5/thumbnails/13.jpg)
Result
![Page 14: Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.](https://reader036.fdocuments.net/reader036/viewer/2022081503/56649f045503460f94c18b67/html5/thumbnails/14.jpg)
GPU Implementation
Original implementation of TurboPixels done in MATLAB, parts accelerated with C
GPU Implementation is C++ accelerated with CUDA
Not all parts of original algorithm mappable to GPU (algorithms not parallel!)
![Page 15: Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.](https://reader036.fdocuments.net/reader036/viewer/2022081503/56649f045503460f94c18b67/html5/thumbnails/15.jpg)
GPU Implementation
Example: Distance Transform Foreach pixel, get distance to nearest pixel-of-
importance Used in initializing distance function Used during evolution to get nearest superpixel
boundary point Original algorithm used Fast Marching Method
to calculate (uses global data structure, not parallel)
Replace with a GPU-friendly substitute
![Page 16: Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.](https://reader036.fdocuments.net/reader036/viewer/2022081503/56649f045503460f94c18b67/html5/thumbnails/16.jpg)
GPU Implementation
Assignment map Array of superpixel IDs Keeps track of superpixel coverage, ownership Prevents merging of superpixels
![Page 17: Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.](https://reader036.fdocuments.net/reader036/viewer/2022081503/56649f045503460f94c18b67/html5/thumbnails/17.jpg)
Performance Optimizations
Use CUDA arrays,textures when kernel performs random or neighbor accesses
Little use of shared memory Kernels either read once+write once, or have
complex access patterns not easily done with shmem
Loop unrolling, aligning image array sizes, ...
![Page 18: Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.](https://reader036.fdocuments.net/reader036/viewer/2022081503/56649f045503460f94c18b67/html5/thumbnails/18.jpg)
Results
![Page 19: Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.](https://reader036.fdocuments.net/reader036/viewer/2022081503/56649f045503460f94c18b67/html5/thumbnails/19.jpg)
Sample Result
Platform: NVIDIA GTX280 Image size: 640x480 Time: 443ms (2.25FPS) Timesteps: 122 Superpixels: 1000
Software implementation: 30 sec on 481x321 image
![Page 20: Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.](https://reader036.fdocuments.net/reader036/viewer/2022081503/56649f045503460f94c18b67/html5/thumbnails/20.jpg)
Conclusions
Implemented a TurboPixels-like image oversegmentation algorithm on a GPU
Performance goal of 5fps on 640x480 not quite attained
Achieved significant speedup over software implementation (although one written mostly in MATLAB...)
![Page 21: Dense Image Over-segmentation on a GPU Alex Rodionov 4/24/2009.](https://reader036.fdocuments.net/reader036/viewer/2022081503/56649f045503460f94c18b67/html5/thumbnails/21.jpg)
Future Work
Algorithmic optimizations: Evolve area around the expanding boundary,
instead of evolving everything Use variable-length timesteps to reduce number of
timesteps and amount of work Application to video:
Once it runs fast enough, then what? Modify algorithm to take advantage of inter-frame
coherence