JoeStam GPU Computer Vision Novideo
of 17
Transcript of JoeStam GPU Computer Vision Novideo
-
7/29/2019 JoeStam GPU Computer Vision Novideo
1/17
NVIDIA Confidential
GPU Computer Vision The next generation of machines that seeJoe Stam [email protected]
GPU Computer Vision The next generation of machines that see
Joe Stam [email protected]
-
7/29/2019 JoeStam GPU Computer Vision Novideo
2/17
NVIDIA Confidential
Agenda
Why GPUs for Computer Vision?ExamplesDemos
Discussion, future research topics
-
7/29/2019 JoeStam GPU Computer Vision Novideo
3/17
NVIDIA Confidential
Why use GPUs for Computer VisionObvious Answer:
Many vision algorithms are highly
parallel, and GPUs have lots of cores!
-
7/29/2019 JoeStam GPU Computer Vision Novideo
4/17
NVIDIA Confidential
Data-Parallel architecture
Same math to many pixels
leverage redundancy
-
7/29/2019 JoeStam GPU Computer Vision Novideo
5/17
NVIDIA Confidential
Massive Memory Bandwidth
Nearly 100 GB/s ofon-card memory
bandwidth
Dedicated Memorynot shared with other
tasks
-
7/29/2019 JoeStam GPU Computer Vision Novideo
6/17
NVIDIA Confidential
Latency Hiding
Even with massive bandwidth, memory is very slowrelative to compute
Caches are not big enough for many vision
applications (especially with video)
GPUs
hide memory latency through fast thread
context switch
-
7/29/2019 JoeStam GPU Computer Vision Novideo
7/17
NVIDIA Confidential
Shared Memory
Acts like a local cache, allowing pixels to access
their neighbors with extremely high speed
Intermediate results can also be shared between
pixels
-
7/29/2019 JoeStam GPU Computer Vision Novideo
8/17
NVIDIA Confidential
Custom Hardware - Textures
Fast, Cached access toimage data
Hardware for:
Bilinear InterpolationType conversion
Boundary handling
Excellent for
geometrical distortions
-
7/29/2019 JoeStam GPU Computer Vision Novideo
9/17
NVIDIA Confidential
Video Decoder / Encoder
-
7/29/2019 JoeStam GPU Computer Vision Novideo
10/17
NVIDIA Confidential
Examples: Panaroma StitchingPanorama generated from three 3840x2880 images completed in
0.577s on a GTX280 GPU
Work by James Fung at NVIDIA
-
7/29/2019 JoeStam GPU Computer Vision Novideo
11/17
NVIDIA Confidential
Panaroma Algorithm
Keypoint Detection& Extraction
(SIFT)
KeypointMatching
RecoverHomography
(RANSAC)
ProjectiveTransform
Multi-Band Blend
CreateLaplacian Pyramid
LeftImage
RightImage
-
7/29/2019 JoeStam GPU Computer Vision Novideo
12/17
NVIDIA Confidential
Panaroma PerformanceComparison of GPU vs. Photoshop CS3 Photomerge
Tool
CPU GPU
Xeon E54402.83 GHz Quad-Core
4 GB RAM
Tesla C1060
4 GB Framebuffer
3.3 s 0.668 s
Time to Stitch 3 Images (11 Megapixels / image)
-
7/29/2019 JoeStam GPU Computer Vision Novideo
13/17
NVIDIA Confidential
Stereo Vision
B =baseline distance
Object
L R
FIGURE 1
http://www.minoru3dwebcam.com/ -
7/29/2019 JoeStam GPU Computer Vision Novideo
14/17
NVIDIA Confidential
Computationally Insane
Need to find correspondence between left and right image forevery pixel
For 640 x 480 pixels, 11x11 block match, this is 2 Billion
computations for 50-pixel search
(with no optimization)
-
7/29/2019 JoeStam GPU Computer Vision Novideo
15/17
NVIDIA Confidential
GPU Strategy
THREAD0
THREAD1
THREAD2
THREAD3
THREAD4
THREAD5
THREAD6
THREAD7
THREAD8
THREAD9
THREAD10
THREAD11
THREAD12
THREAD13
THREAD14
THREAD15
L R
2
THREAD 0
col(0..4)
THREAD 1
col(1..5)
THREAD 2
col(2..6)
THREAD0
THREAD1
THREAD2
THREAD3
0 1 2 ...
Heavily leverages SMEM for intermediate computationsto share work between threads
-
7/29/2019 JoeStam GPU Computer Vision Novideo
16/17
NVIDIA Confidential
Whats Next: GPUs enable practical3D vision!
Ushers in a whole new class of Human-Machine
Interface & Gaming Concepts
-
7/29/2019 JoeStam GPU Computer Vision Novideo
17/17
NVIDIA Confidential
And the future..
Biologically inspired vision
Talk to Nicolas
