Models and Matching
-
Upload
fordlovers -
Category
Documents
-
view
444 -
download
0
Transcript of Models and Matching
![Page 1: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/1.jpg)
Stockman CSE/MSU Fall 2008 1
Models and Matching
Methods of modeling objects and their environments;
Methods of matching models to sensed data for recogniton
![Page 2: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/2.jpg)
Stockman CSE/MSU Fall 2008 2
Some methods to study Mesh models (surface) Vertex-edge-face models (surface) Functional forms: superquadrics (surface) Generalized cylinders (volume) Voxel sets and octrees (volume) View class models (image-based) Recognition by appearance (image-based) Functional models and the Theory of
affordances (object-oriented)
![Page 3: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/3.jpg)
Stockman CSE/MSU Fall 2008 3
Models are what models do
![Page 4: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/4.jpg)
Stockman CSE/MSU Fall 2008 4
What do models do?
![Page 5: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/5.jpg)
Stockman CSE/MSU Fall 2008 5
Vertex-edge-face models
Polyhedra and extensions;Model the surface of objects
![Page 6: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/6.jpg)
Stockman CSE/MSU Fall 2008 6
Vertex-Edge-Face model
![Page 7: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/7.jpg)
Stockman CSE/MSU Fall 2008 7
Sample object
All surfaces are planar or cylindrical
![Page 8: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/8.jpg)
Stockman CSE/MSU Fall 2008 8
Matching methods Hypothesize point correspondences Filter on distances Compute 3D alignment of model to
data Verify positions of other model points,
edges, or faces. You can now do this! LOTS of work in the literature on this!
Can work for many industrial objects (and human faces perhaps!)
![Page 9: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/9.jpg)
Stockman CSE/MSU Fall 2008 9
Triangular meshes
Very general and used by most CAD systems.
![Page 10: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/10.jpg)
Stockman CSE/MSU Fall 2008 10
Texture-mapped mesh dog
Courtesy of Kari PuliWith each triangle is a mapping of its vertices into pixels [r, c] of a color image. Thus any point of any triangle can be assigned a color [R, G, B]. There may be several images available to create these mappings.
3D SURFACE MODEL
SURFACE PLUS TEXTURE
![Page 11: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/11.jpg)
Stockman CSE/MSU Fall 2008 11
Meshes are very general
They are usually verbose and often are too detailed for many operations, but are often used in CAD. (Volumetric cube models are actually displayed here: made from many views by Kari Pulli.)
![Page 12: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/12.jpg)
Stockman CSE/MSU Fall 2008 12
Modeling the human body for clothing industry and …
Multiple Structured light scanners used: could this be a service industry such as Kinkos?
Actually cross sections of a generalized cylinder model.
![Page 13: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/13.jpg)
Stockman CSE/MSU Fall 2008 13
Mesh characteristics
+ can be easy to generate from scanned data
![Page 14: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/14.jpg)
Stockman CSE/MSU Fall 2008 14
Making mesh models
![Page 15: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/15.jpg)
Stockman CSE/MSU Fall 2008 15
Marching cubeshttp://www.exaflop.org/docs/marchcubes/ (James Sharman)
"Marching Cubes: A High Resolution 3D Surface Construction Algorithm",William E. Lorensen and Harvey E. Cline,Computer Graphics (Proceedings of SIGGRAPH '87), Vol. 21, No. 4, pp. 163-169.
Raster scan through image F(r, c).Look for adjacent pixels, one above threshold and one below threshold. Interpolate real coordinates for f(x, y) = t in between
![Page 16: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/16.jpg)
Stockman CSE/MSU Fall 2008 16
Marching in 3D space F(s, r, c)
Some voxel corners are above threshold t and some are below.
![Page 17: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/17.jpg)
Stockman CSE/MSU Fall 2008 17
PhD work by Paul Albee 2004 Used Argonne National Labs scanner High energy, high resolution planar
Xrays penetrate object rotating on a turntable
Computer aided tomography synthesizes a 3D volume of densities with voxel size of about 5 microns
![Page 18: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/18.jpg)
Stockman CSE/MSU Fall 2008 18
Segmentation of Scutigera a tiny crablike organism
Slice j of material densityF( sj, r, c ) “thresholded”
volume
![Page 19: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/19.jpg)
Stockman CSE/MSU Fall 2008 19
Some common 3D problems analyze blood vessel structure in head capture structure and motion of
vertebrae of spine analyze porosity and structure of soil analyze structure of materials automatic segmentation into regions automatic correspondence of 3D points at two instants of of time 3D volume visualization and virtual tours
![Page 20: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/20.jpg)
Stockman CSE/MSU Fall 2008 20
Scanning technique abstraction
CCD camera (row) material sample X-ray planes scintillator
Pin head
rotate
X-rays partly absorbed by sample; excite scintillator producing one row in the camera image; rotate sample a few degrees and produce another row; 3D reconstruction using CT
![Page 21: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/21.jpg)
Stockman CSE/MSU Fall 2008 21
Scutigera: a tiny crustacean
• organism is smaller than 1 mm• scanned at Argonne • volume segmented and meshed by Paul Albee• roughly ten million triangles to represent the surface• anaglyph created for 3D visualization (view with stereo glasses)
![Page 22: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/22.jpg)
Stockman CSE/MSU Fall 2008 22
Presentation of Results to User
Can explore the 3D data using rotation/translation Can create stereo images from 3D data
![Page 23: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/23.jpg)
Stockman CSE/MSU Fall 2008 23
Physics-based models
Can be used to make meshes;Meshes retain perfect
topology;Can span spots of bad or no
data
![Page 24: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/24.jpg)
Stockman CSE/MSU Fall 2008 24
Physics-based modeling
![Page 25: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/25.jpg)
Stockman CSE/MSU Fall 2008 25
Forces move points on the model; halt at scanned data
![Page 26: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/26.jpg)
Stockman CSE/MSU Fall 2008 26
Fitting an active contour to image data
![Page 27: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/27.jpg)
Stockman CSE/MSU Fall 2008 27
Balloon model for closed object surface
Courtesy of Chen and Medioni
![Page 28: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/28.jpg)
Stockman CSE/MSU Fall 2008 28
Balloon evolution• balloon stops at data points• mesh forces constrain neighbors• large triangles split into 4 triangles• resulting mesh has correct topology• hard CS part is detecting when balloon should be stopped by data point
![Page 29: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/29.jpg)
Stockman CSE/MSU Fall 2008 29
Physics-based models
Can also model dynamic behavior of solids (Finite Element Methods)
![Page 30: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/30.jpg)
Stockman CSE/MSU Fall 2008 30
Tagged MRI: 3D interest points can be written to body!
The MRI sensor tags living tissue and can sense its movement. Motion of a 3D tetrahedral finite elements model can then be analyzed. FMA model attempts to model the real physics of the heart. Work by Jinah Park and Dimitry Metaxes.
![Page 31: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/31.jpg)
Stockman CSE/MSU Fall 2008 31
Algorithms from computer graphics make mesh models from blobs Marching squares applied to some
connected image region (blob) Marching cubes applied to some
connected set of voxels (blob) See a CG text for algorithms: see
the visualization toolkit for software
![Page 32: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/32.jpg)
Stockman CSE/MSU Fall 2008 32
The octree for compression
![Page 33: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/33.jpg)
Stockman CSE/MSU Fall 2008 33
Generalized cylinders
![Page 34: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/34.jpg)
Stockman CSE/MSU Fall 2008 34
Generalized cylinders
• component parts have axis• cross section function describes variation along axis• good for articulated objects, such as animals, tools• can be extracted from intensity images with difficulty
![Page 35: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/35.jpg)
Stockman CSE/MSU Fall 2008 35
Extracting a model from a segmented image region
Courtesy of Chen and Medioni
![Page 36: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/36.jpg)
Stockman CSE/MSU Fall 2008 36
Interpreting frames from video Can we match a frame region to a
model? What about a sequence of frames? Can we determine what actions
the body is doing?
![Page 37: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/37.jpg)
Stockman CSE/MSU Fall 2008 37
Generalized cylinders
![Page 38: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/38.jpg)
Stockman CSE/MSU Fall 2008 38
View class models
Objects modeled by the distinct views that they can
produce
![Page 39: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/39.jpg)
Stockman CSE/MSU Fall 2008 39
“aspect model” of a cube
![Page 40: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/40.jpg)
Stockman CSE/MSU Fall 2008 40
Recognition using an aspect model
![Page 41: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/41.jpg)
Stockman CSE/MSU Fall 2008 41
View class model of chair
2D Graph-matching (as in Ch 11) used to evaluate match.
![Page 42: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/42.jpg)
Stockman CSE/MSU Fall 2008 42
Side view classes of Ford Taurus (Chen and Stockman)
These were made in the PRIP Lab from a scale model. Viewpoints in between can be generated from x and y curvature stored on boundary. Viewpoints matched to real image boundaries via optimization.
![Page 43: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/43.jpg)
Stockman CSE/MSU Fall 2008 43
Matching image edges to model limbs
Could recognize car model at stoplight or gate or in car wash.
![Page 44: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/44.jpg)
Stockman CSE/MSU Fall 2008 44
Appearance-based models
Using a basis of sub images;Using PCA to compress bases;
Eigenfaces (see older .pdf slides 14C)
![Page 45: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/45.jpg)
Stockman CSE/MSU Fall 2008 45
Function-based modelingObject-oriented;
What parts does the object have;What behaviors does it have;
What can be done with it?(See plastic slides of Louise Starks’s work.)
![Page 46: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/46.jpg)
Stockman CSE/MSU Fall 2008 46
Louise Stark: chair model Dozens of CAD models of chairs Program analyzes model for * stable pose * seat of right size * height off ground right size * no obstruction to body on seat * program would accept a trash can (which could also pass as a container)
![Page 47: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/47.jpg)
Stockman CSE/MSU Fall 2008 47
Theory of affordances: J.J. Gibson An object can be “sittable”: a large
number of chair types, a box of certain size, a trash can turned over, …
An object can be “walkable”: the floor, ground, thick ice, bridge, ...
An object can be a “container”: a cup, a hat, a barrel, a box, …
An object can be “throwable”: a ball, a book, a coin, an apple, a small chair, …
![Page 48: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/48.jpg)
Stockman CSE/MSU Fall 2008 48
Minski’s theory of frames(Schank’s theory of scripts) Frames are learned expectations –
frame for a room, a car, a party, an argument, …
Frame is evoked by current situation – how? (hard)
Human “fills in” the details of the current frame (easier)
![Page 49: Models and Matching](https://reader034.fdocuments.net/reader034/viewer/2022051707/58ed6be51a28ab8f2c8b46bf/html5/thumbnails/49.jpg)
Stockman CSE/MSU Fall 2008 49
Make a frame for my house Item 1 Item 2 Item 3 Item 4 Item 5 Item 6