Machine Vision made easy with Ruby - ShRUG June 2010
-
Upload
jan-wedekind -
Category
Technology
-
view
3.278 -
download
2
Transcript of Machine Vision made easy with Ruby - ShRUG June 2010
![Page 1: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/1.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdf
Machine Vision made easy with Ruby
1/33c© 2010 Jan Wedekind
Machine Vision made easy with Ruby
Jan Wedekind
Mon Jun 14 19:00:00 BST 2010
![Page 2: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/2.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdf
Motivation I/II
2/33c© 2010 Jan Wedekind
Andrew Wilson: Robust Computer Vision-Based Detection of Pinching for One andTwo-Handed Gesture Input
http://research.microsoft.com/˜awilson
![Page 3: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/3.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdf
Motivation II/II
3/33c© 2010 Jan Wedekind
• subtract background from input image
• threshold resulting difference image
• connected component labeling
• discard components touching border of image
• select component of significant size
• extract centroid
• . . .
http://bit.ly/b9sCgw
![Page 4: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/4.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdfHornetseyeInput/Output
4/33c© 2010 Jan Wedekind
![Page 5: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/5.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdfHornetseye
Malloc Objects
5/33c© 2010 Jan Wedekind
m = Malloc.new 10
m.write ’0123456789’
# "0123456789"
m.read 5
# "01234"
( m + 2 ).read 5
# "23456"
![Page 6: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/6.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdfHornetseye
Array Operations I/II
6/33c© 2010 Jan Wedekind
operation loop body loop variable
write element r[b] =a -
read element r =a[b] -
write sub-array r[b+i]=a[i] i
read sub-array r[i] =a[i+b] i
fill r[i] =a i
index array r[i] =i i
unary function r[i] =f(a[i]) i
binary function r[i] =f(a,b[i]) i
binary function r[i] =f(a[i],b) i
binary function r[i] =f(a[i],b[i]) i
accumulate r =f(r,a[i]) i...
......
![Page 7: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/7.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdfHornetseye
Array Operations II/II
7/33c© 2010 Jan Wedekind
operation loop body loop variable
......
...
warp/mask r[i] =a[b[i]] i
unmask r[b[i]]=a[i] i
downsampling r[i] =a[b*i] i
upsampling r[b*i] =a[i] i
integral r[i] =r[i-1]+a[i] i
map r[i] =b[a[i]] i
histogram r[a[i]]=r[a[i]]+1 i
weighted hist. r[a[i]]=r[a[i]]+b[i] i
correlation r[i] =r[i]+a[i+j]*b[j] i,j
![Page 8: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/8.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdf
InstallationKubuntu 8.04
8/33c© 2010 Jan Wedekind
• Install required packages: sudo aptitude install ruby1.8 ruby1.8-dev \irb1.8 imagemagick librmagick-ruby1.8 g++ ccache libboost-dev libxine-dev \libxine1-all-plugins libdc1394-13-dev xorg-dev libfftw3-dev libopenexr-dev \bison flex texinfo
• Install libJIT:wget http://vision.eng.shu.ac.uk/jan/libjit-0.1.3pre.tar.bz2tar xjf libjit-0.1.3pre.tar.bz2cd libjit-0.1.3pre./configure && make && sudo make install
• Download hornetseye-x.x.tar.bz2 from Rubyforge
• Install Hornetseye:tar xjf hornetseye-*.tar.bz2cd hornetseye-*./configure.ruby18 && make && sudo make install
![Page 9: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/9.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdf
InstallationMicrosoft Windows
9/33c© 2010 Jan Wedekind
• Run Ruby one-click installer
• Reboot
• Unpack Ghostscript fonts to c:\gs (fonts should end up in c:\gs\fonts)
• Run ImageMagick installer
• Download RMagick Rubygem and install using the commandgem install rmagick-2.6.0-x86-mswin32.gem
• Download NSIS installer for Hornetseye from Rubyforge and run it
• Optionally install NArray, MPlayer, Qt4 (requires MinGW), and Qt4QtRuby
![Page 10: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/10.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdfInstallation
Future: Installation with Rubygems?
10/33c© 2010 Jan Wedekind
gem install mallocgem install multiarraygem install hornetseye-xinegem install hornetseye-video4linuxgem install hornetseye-video4linux2gem install hornetseye-x11. . .
![Page 11: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/11.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdf
Interactive Ruby
11/33c© 2010 Jan Wedekind
![Page 12: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/12.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdfGesture Recognition Example
Background Image
12/33c© 2010 Jan Wedekind
input = DC1394Input.new ’’ , 0, 0,
DC1394Input::FORMAT VGA NONCOMPRESSED,
DC1394Input::MODE 320x240 YUV422
bg = input.read sint
![Page 13: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/13.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdfGesture Recognition Example
Input Image
13/33c© 2010 Jan Wedekind
img = input.read ubyte
![Page 14: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/14.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdf
Gesture Recognition ExampleDifference
14/33c© 2010 Jan Wedekind
diff = img - bg
![Page 15: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/15.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdfGesture Recognition Example
Thresholding
15/33c© 2010 Jan Wedekind
binary = diff <= THRESHOLD
![Page 16: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/16.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdfGesture Recognition Example
Components
16/33c© 2010 Jan Wedekind
components = binary.components
n components = components.max + 1
![Page 17: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/17.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdfGesture Recognition Example
Compute Area of Components
17/33c© 2010 Jan Wedekind
area = components.hist n components
![Page 18: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/18.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdfGesture Recognition Example
Impose Size Constraint
18/33c© 2010 Jan Wedekind
mask area = area.between? RANGE.min * SIZE,
RANGE.max * SIZE
![Page 19: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/19.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdf
Gesture Recognition ExampleBorder Pixel
19/33c© 2010 Jan Wedekind
border = MultiArray.int( *SHAPE ).fill! 1
border[ 1 ... SHAPE[ 0 ] - 1, 1 ... SHAPE[ 1 ] - 1 ] = 0
![Page 20: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/20.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdfGesture Recognition ExampleReject Components touching Border
20/33c© 2010 Jan Wedekind
mask border = components.
hist weighted( n components, border ).eq 0
![Page 21: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/21.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdfGesture Recognition Example
Remaining Component(s)
21/33c© 2010 Jan Wedekind
mask = mask area.and mask border
map = mask.to ubyte.integral * mask.to ubyte
target = components.map map
![Page 22: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/22.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdfGesture Recognition Example
Centre of Gravity
22/33c© 2010 Jan Wedekind
index = MultiArray.int( *SHAPE ).indgen!
x, y = index % SHAPE[ 0 ], index / SHAPE[ 0 ]sum target = target.sum.to f
x target = x.mask( target.to bool ).sum / sum target
y target = y.mask( target.to bool ).sum / sum target
x target, y target
![Page 23: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/23.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdf
Other ExamplesPresentation Software
23/33c© 2010 Jan Wedekind
http://www.youtube.com/watch?v=wNFr7RNWeCs
![Page 24: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/24.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdfOther ExamplesCamshift Tracking
24/33c© 2010 Jan Wedekind
http://www.wedesoft.demon.co.uk/hornetseye-api/files/camshift-txt.html
![Page 25: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/25.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdf
Other ExamplesBarcode Reader
25/33c© 2010 Jan Wedekind
http://www.wedesoft.demon.co.uk/hornetseye-api/files/barcode-txt.html
![Page 26: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/26.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdf
Other ExamplesLucas Kanade Tracker
26/33c© 2010 Jan Wedekind
http://www.wedesoft.demon.co.uk/hornetseye-api/files/lktracker-txt.html
![Page 27: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/27.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdfOther Examples
Planar Marker Tracking
27/33c© 2010 Jan Wedekind
http://rubyconf2009.confreaks.com/
19-nov-2009-13-15-computer-vision-using-ruby-and-libjit-jan-wedekind.html
![Page 28: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/28.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdfFuture Work
Feature Locations and Descriptors
28/33c© 2010 Jan Wedekind
http://www.wedesoft.demon.co.uk/hornetseye-api/files/features-txt.html
![Page 29: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/29.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdfFuture WorkInspiration: Probabilistic Feature-based On-line
Rapid Model Acquisition
29/33c© 2010 Jan Wedekind
http://mi.eng.cam.ac.uk/˜qp202/my_papers/BMVC09
http://mi.eng.cam.ac.uk/˜twd20/
![Page 30: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/30.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdfFuture Work
Inspiration: Bounded Hough Transform
30/33c© 2010 Jan Wedekind
http://www.ptgrey.com/newsletters/dec2004.html
http://www.ptgrey.com/newsletters/images/GreShaJas04.pdf
![Page 31: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/31.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdfFuture Work
Inspiration: SceneLib
31/33c© 2010 Jan Wedekind
http://www.doc.ic.ac.uk/˜ajd/
![Page 32: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/32.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdfFuture Work
Inspiration: Avatar
32/33c© 2010 Jan Wedekind
http://seqmag.com/2010/01/making-of-avatar/
http://seqmag.com/2010/01/exclusive-45minute-making-of-avatar/
![Page 33: Machine Vision made easy with Ruby - ShRUG June 2010](https://reader033.fdocuments.net/reader033/viewer/2022060205/55a1821b1a28abd6098b477e/html5/thumbnails/33.jpg)
http://vision.eng.shu.ac.uk/jan/shrug7.pdf
Thanks
33/33c© 2010 Jan Wedekind
CreditsAiden Lockwood, Aleksey Demakov, Annemie Wedekind, Arul Nirai
Selvan, Ashley Moran, Balasundram Amavasai, Beverly Inkson, ChinweLucy Ozoegwu, Damien Douxchamps, Daniel Martın Marın, Geraud De LaMensbruge, Gerhard Wedekind, Hussein Abdul-Rahman, Jacques Penders,
Jag Gill, Jing Jing Wang, Jon Travis, Jong Peng, Juan Roldan, JulienDemarest, Julien Faucher, Julien Lacheray, Ken Dutton, Kim Chuan Lim,
Kirill Kononenko, Klaus Treichel, Manuel Boissenin, Martin Howarth,Matthias Stumpf, Michael Doronin, Ralph Gay, Richard Dale, Sonia
Fernandez Rodrıguez, Tan Kang Song, Ushakiran Soutapalli, Volkan Karaca,Warren Jasper, Zineb Saghi, ...
http://www.wedesoft.demon.co.uk/hornetseye-api/
http://rubyforge.org/projects/hornetseye/
http://sourceforge.net/projects/hornetseye/
http://launchpad.net/hornetseye/
http://raa.ruby-lang.org/project/hornetseye/
http://www.ohloh.net/p/hornetseye/