OpenCV Introduction
description
Transcript of OpenCV Introduction
![Page 1: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/1.jpg)
OpenCV IntroductionHang Xiao
Oct 26, 2012
![Page 2: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/2.jpg)
History 1999 Jan : lanched by Intel, real time machine vision library for UI, optimized code
for intel
2000 Jun : OpenCV alpha 3。 2000 Dec : OpenCV beta 1 for linux
2006 : the first 1.0 version supports Mac OS
2008 mid : obtain corporate support from Willow Garage
2009 Sep : OpenCV 1.2( beta2.0
2009 Oct : Version 2.0 released。 2010 Dec : OpenCV 2.2。 2011 Aug : OpenCV 2.3。 2012 Apr : OpenCV 2.4.
![Page 3: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/3.jpg)
Overview Goals
Develop a universal toolbox for research and development in the field of Computer Vision
Algorithms More than 350 algorithms, 500 API
Programming language C/C++, C#, Ch , Python, Ruby, Matlab, and Java (using JavaCV)
OS support Windows, Android, Maemo, FreeBSD, OpenBSD, iOS, Linux and Mac
OS.
Licence BSDlisence, free for commercial and non-commmercial
![Page 4: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/4.jpg)
Overview - Applications 2D and 3D feature toolkits Egomotion estimation Facial recognition system Gesture recognition Human–computer interaction (HCI) Mobile robotics Motion understanding Object identification Segmentation and Recognition Stereopsis Stereo vision: depth perception from 2 cameras Structure from motion (SFM)Motion tracking
![Page 5: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/5.jpg)
Overview - A statistical machine learning library Boosting (meta-algorithm)
Decision tree learning
Gradient boosting trees
Expectation-maximization algorithm
k-nearest neighbor algorithm
Naive Bayes classifier
Artificial neural networks
Random forest
Support vector machine (SVM)
![Page 6: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/6.jpg)
Outline
Image Analysis Structural Analysis Object Recognition Motion Analysis and Object Tracking 3D Reconstruction
![Page 7: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/7.jpg)
Outline
Image Analysis Structural Analysis Object Recognition Motion Analysis and Object Tracking 3D Reconstruction
![Page 8: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/8.jpg)
Image Analysis Thresholds Statistics Pyramids Morphology Distance transform Flood fill Feature detection Contours retrieving
![Page 9: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/9.jpg)
Image Thresholding
Fixed threshold; Adaptive
threshold;
![Page 10: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/10.jpg)
Image Thresholding Examples
Source picture Fixed threshold Adaptive threshold
![Page 11: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/11.jpg)
Statistics
min, max, mean value, standard deviation over the image
Norms C, L1, L2 Multidimensional histograms Spatial moments up to order 3 (central,
normalized, Hu)
![Page 12: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/12.jpg)
Multidimensional Histograms Histogram operations calculation, normalization, comparison, back project
Histograms types: Dense histograms Signatures (balanced tree)
EMD algorithm The EMD computes the distance between two distributions, which are
represented by signatures. The signatures are sets of weighted features that capture the distributions.
The features can be of any type and in any number of dimensions, and are defined by the user.
The EMD is defined as the minimum amount of work needed to change one signature into the other
![Page 13: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/13.jpg)
EMD – a method for the histograms comparison
.),(
,
,),(
),(
,1,,1,
,
,
jiji
ij
jiij
jijiij
ji
qandpelementsthebetweendistancetheqpd
tscoefficienweightf
f
qpdfQPEMD
historamstwoQjQqPiPp
![Page 14: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/14.jpg)
Image Pyramids
Gaussian and Laplacian pyramids
Image segmentation by pyramids
![Page 15: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/15.jpg)
Image Pyramids
Gaussian and Laplacian
![Page 16: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/16.jpg)
Pyramid-based color segmentation
On still pictures And on movies
![Page 17: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/17.jpg)
Morphological Operations Two basic morphology operations using structuring element:
erosion dilation
More complex morphology operations: opening : erosion + dilation closing : dilation + erosion morphological gradient : the difference between the dilation and the erosion of an
image top hat : the difference between an input image and its opening black hat : the difference between the closing and its input image
![Page 18: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/18.jpg)
Morphological Operations Examples Morphology - applying Min-Max. Filters and its combinations
Opening IoB= (IB)BDilatation IBErosion IBImage I
Closing I•B= (IB)B TopHat(I)= I - (IB) BlackHat(I)= (IB) - IGrad(I)= (IB)-(IB)
![Page 19: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/19.jpg)
Distance Transform
Calculate the distance for all non-feature points to the closest feature point
Two-pass algorithm, 3x3 and 5x5 masks, various metrics predefined
![Page 20: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/20.jpg)
Flood Filling
Simple Gradient
![Page 21: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/21.jpg)
Feature Detection Fixed filters (Sobel operator, Laplacian); Optimal filter kernels with floating point coefficients
(first, second derivatives, Laplacian) Special feature detection (corners) Canny operator Hough transform (find lines and line segments) Gradient runs
![Page 22: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/22.jpg)
Canny Edge Detector
![Page 23: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/23.jpg)
Hough TransformDetects lines in a binary image
• Probabilistic Hough Transform• Standard Hough
Transform
![Page 24: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/24.jpg)
Another Sample of the Hough Transform Using
Source picture Result
![Page 25: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/25.jpg)
Contour Retrieving
The contour representation: Chain code (Freeman code) Polygonal representation
Initial PointChain code for the curve: 34445670007654443
Contour representation
![Page 26: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/26.jpg)
Hierarchical representation of contours
Image Boundary
(W1) (W2) (W3)
(B2) (B3) (B4)
(W5) (W6)
![Page 27: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/27.jpg)
Contours Examples
Source Picture(300x600 = 180000 pts total)
Retrieved Contours (<1800 pts total)
After Approximation(<180 pts total)
And it is rather fast: ~70 FPS for 640x480 on complex scenes
![Page 28: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/28.jpg)
Outline
Image Analysis Structural Analysis Object Recognition Motion Analysis and Object Tracking 3D Reconstruction
![Page 29: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/29.jpg)
Structural Analysis Contours processing Approximation Hierarchical representation Shape characteristics Matching
Geometry Contour properties Fitting with primitives PGH: pair-wise geometrical histogram for the
contour.
![Page 30: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/30.jpg)
Contour Processing
Approximation: RLE algorithm (chain code) Teh-Chin approximation (polygonal) Douglas-Peucker approximation (polygonal);
Contour moments (central and normalized up to order 3) Hierarchical representation of contours Matching of contours
![Page 31: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/31.jpg)
Hierarchical Representation of Contours
A contour is represented with a binary tree
Given the binary tree, the contour can be retrieved with arbitrary precision
The binary tree is quasi invariant to translations, rotations and scaling
![Page 32: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/32.jpg)
Contours matching
Matching based on hierarchical representation of contours
![Page 33: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/33.jpg)
Geometry
Properties of contours: (perimeter, area, convex hull, convexity defects, rectangle of minimum area)
Fitting: (2D line, 3D line, circle, ellipse) Pair-wise geometrical histogram
![Page 34: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/34.jpg)
Pair-wise geometrical histogram (PGH)
),( jip
.),(/),()(
,),(/),()(
,)](),2(),1(),(),2(),1([
iic
jjr
TcccrrrPGH
jipjipijE
jipjipjiE
MEEENEEEf
![Page 35: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/35.jpg)
Outline
Image Analysis Structural Analysis Object Recognition Motion Analysis and Object Tracking 3D Reconstruction
![Page 36: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/36.jpg)
Object Recognition Eigen objects Hidden Markov Models
![Page 37: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/37.jpg)
Eigen Objects
..},,,{
,,1},,,,{
),(/1
,/1),()(
,cov}{,,1,},,{
21
111
1
21
vectorseigenandvalueseigenand
basiseigenmmieeee
uuve
umuuuuuc
matrixariancecCnmmiRuuuu
im
iiii
in
iii
lk
kl
iki
il
m
k
klll
jl
ll
ilij
ij
nin
iii
![Page 38: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/38.jpg)
Eigen objects (continued)
![Page 39: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/39.jpg)
Hidden Markov Model
Definitions
},...,,{ 21 NsssS },...,,{ 21 MoooO
Oqt )}|(\{ 1 itjtijij sqsqPaaA
)}|(\{ jiijij soPbbB
)}(\{ 0 iii sqP
- The set of states
- The set of measurements
- The state at time t
- The transition probability matrix
- The conditional probability matrix
- The starting states distribution
![Page 40: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/40.jpg)
Embedded HMM for Face Recognition
Model-
- Face ROI partition
![Page 41: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/41.jpg)
Face recognition using Hidden Markov Models
One person – one HMMStage 1 – Train every HMM
Stage 2 – Recognition
Pi - probability
Choose max(Pi)
…1
n
i
![Page 42: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/42.jpg)
Outline
Image Analysis Structural Analysis Object Recognition Motion Analysis and Object Tracking 3D Reconstruction
![Page 43: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/43.jpg)
Motion Analysis and Object Tracking
Background subtraction Motion templates Optical flow Active contours Estimators
![Page 44: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/44.jpg)
Background Subtraction
Background model (normal distribution) Background statistics functions:
Average Standard deviation Running average
![Page 45: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/45.jpg)
Motion Templates
Object silhouette Motion history images Motion history gradients Motion segmentation algorithm
silhouette MHIMHG
![Page 46: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/46.jpg)
Motion Segmentation Algorithm
Two-pass algorithm labeling all motion segments
![Page 47: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/47.jpg)
Motion Templates Example
Motion templates allow to retrieve the dynamic characteristics of the moving object
![Page 48: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/48.jpg)
Optical Flow Block matching technique Horn & Schunck technique Lucas & Kanade technique Pyramidal LK algorithm 6DOF (6 degree of freedom) algorithm
y
xt
yyx
yxx
II
IbIII
IIIGyxX
bXG
dtdyyIdtdxxItItyxIdttdyydxxI
,,
,),,(
,
);/(/)/(//);,,(),,(
2
2
Optical flow equations:
![Page 49: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/49.jpg)
Pyramidal Implementation of the optical flow algorithm
J image I image
Image Pyramid Representation
Iterative Lucas – Kanade Scheme
Generic Image
(L-1)-th Level
L-th Level
Location of point u on image uL=u/2L
Spatial gradient matrix
Standard Lucas – Kanade scheme for optical flow computation at level L dL
Guess for next pyramid level L – 1
Finally,
Image pyramid building
Optical flow computation
2
2
,
,
yyx
yxx
III
IIIG
)(21 LLL dgg
00 gdd
dUV
![Page 50: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/50.jpg)
6DOF Algorithm
N
i ROI
Tit
N
i ROIi
Ti IIdsII
sXIsII
i11
///
).(sX
Parametrical optical flow equations:
![Page 51: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/51.jpg)
Active Contours
Snake energy: Internal energy: External energy: Two external energy types:
extEEE int
curvcont EEE int
conimgext EEE
min
,)(
,
imgcurvcont
img
img
EEEE
IgradE
IE
![Page 52: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/52.jpg)
Estimators
Kalman filter ConDensation filter
![Page 53: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/53.jpg)
Kalman object tracker
![Page 54: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/54.jpg)
Outline
Image Analysis Structural Analysis Object Recognition Motion Analysis and Object Tracking 3D Reconstruction
![Page 55: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/55.jpg)
3D reconstruction
Camera Calibration View Morphing POSIT
![Page 56: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/56.jpg)
Camera Calibration Define intrinsic and extrinsic camera parameters. Define Distortion parameters
],[],,,[,,,
100
00
,][
3
2
1
333231
232221
131211
vupZYXPttt
Trrrrrrrrr
Rcfcf
A
PRTAp
yy
xx
.
)],2/(2[)(~)],2/(2[)(~
222
212
42
21
221
42
21
yxr
yyrpxprkrkcvvv
xxrpyprkrkcuuu
y
x
![Page 57: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/57.jpg)
Camera CalibrationNow, camera calibration can be done by holding checkerboard in front of the camera for a few seconds.
And after that you’ll get:3D view of etalon Un-distorted image
![Page 58: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/58.jpg)
View Morphing
![Page 59: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/59.jpg)
POSIT Algorithm
Perspective projection:
Weak-perspective projection:
iiiiii YZfyXZfx )/(,)/(
./,, ZfsYsyXsx iiii
![Page 60: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/60.jpg)
OpenCV Websites http://opencv.org
OpenCV official webpage.
http://opencvlibrary.sourceforge.net/ OpenCV documentation and FAQs.
![Page 61: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/61.jpg)
OpenCV Examples adaptiveskindetector : detect skin area fback_c : dense Franeback optical flow contours : calculate contours on different levels delaunay : delaunay triangle find_obj : SURF Detector and Descriptor using either FLANN or brute
force matching on planar objects morphology : open/close, erode/dilate motempl :motion templates mser_sample : Maximal Extremal Region interest point detector polar_transforms : illustrates Linear-Polar and Log-Polar image transforms pyramid_segmentation : color pyramid segmentation
![Page 62: OpenCV Introduction](https://reader036.fdocuments.net/reader036/viewer/2022082213/5681638d550346895dd4822b/html5/thumbnails/62.jpg)
Thanks !!!