`Dynamism - Technionron/PAPERS/mobacl_ECCV2002.pdf · 2016-04-24 · Fig. 1. `Dynamism of a Dog on...

15

Transcript of `Dynamism - Technionron/PAPERS/mobacl_ECCV2002.pdf · 2016-04-24 · Fig. 1. `Dynamism of a Dog on...

Page 1: `Dynamism - Technionron/PAPERS/mobacl_ECCV2002.pdf · 2016-04-24 · Fig. 1. `Dynamism of a Dog on Leash' 1912, b y Giacomo Balla. Albrigh t-Kno x Art Gallery, Bu alo. 2 Related w

`Dynamism of a Dog on a Leash'orBehavior lassi� ation by eigen-de ompositionof periodi motions.Roman Goldenberg, Ron Kimmel, Ehud Rivlin, and Mi hael RudzskyComputer S ien e Department, Te hnion|Israel Institute of Te hnologyTe hnion City, Haifa 32000, ISRAELAbstra t. Following Futurism, we show how periodi motions an be represented by a smallnumber of eigen-shapes that apture the whole dynami me hanism of periodi motions.Spe tral de omposition of a silhouette of an obje t in motion serves as a basis for behavior lassi� ation by prin iple omponent analysis. The boundary ontour of the walking dog, forexample, is �rst omputed eÆ iently and a urately. After normalization, the impli it repre-sentation of a sequen e of silhouette ontours given by their orresponding binary images, isused for generating eigen-shapes for the given motion. Singular value de omposition produ esthese eigen-shapes that are then used to analyze the sequen e. We show examples of obje tas well as behavior lassi� ation based on the eigen-de omposition of the binary silhouettesequen e.1 Introdu tionFuturism is a movement in art, musi , and literature that began in Italy at about1909 and marked espe ially by an e�ort to give formal expression to the dynami energy and movement of me hani al pro esses. A typi al example is the `Dynamismof a Dog on a Leash' by Gia omo Balla, who lived during the years 1871-1958 inItaly, see Figure 1 [2℄. In this painting one ould see how the artist aptures in onestill image the periodi walking motion of a dog on a leash. Following Futurism, weshow how periodi motions an be represented by a small number of eigen-shapesthat apture the whole dynami me hanism of periodi motions. Singular value de- omposition of a silhouette of an obje t serves as a basis for behavior lassi� ationby prin iple omponent analysis. Figure 2 present a running horse video sequen eand its eigen-shape de omposition. One an see the similarity between the overlappedeigen-shapes 2( ) and another futurism style painting "The Red Horseman" by CarloCarra [2℄. The boundary ontour of the walking dog, for example, is omputed eÆ- iently and a urately by the fast geodesi a tive ontours [15℄. After normalization,the impli it representation of a sequen e of silhouette ontours given by their or-responding binary images, is used for generating eigen-shapes for the given motion.Singular value de omposition produ es the eigen-shapes that are used to analyze thesequen e. We show examples of obje t as well as behavior lassi� ation based on theeigen-de omposition of the sequen e.

Page 2: `Dynamism - Technionron/PAPERS/mobacl_ECCV2002.pdf · 2016-04-24 · Fig. 1. `Dynamism of a Dog on Leash' 1912, b y Giacomo Balla. Albrigh t-Kno x Art Gallery, Bu alo. 2 Related w

Fig. 1. `Dynamism of a Dog on a Leash' 1912, by Gia omo Balla. Albright-Knox Art Gallery, Bu�alo.2 Related workMotion based re ognition re eived a lot of attention in the last several years. Thisis due to the general re ognition of the fa t that the dire t use of temporal datamay signi� antly improve our ability to solve a number of basi omputer visionproblems su h as image segmentation, tra king, obje t lassi� ation, et ., as wellas the availability of a low ost omputer systems powerful enough to pro ess largeamounts of data.In general, when analyzing a moving obje t, one an use two main sour es ofinformation to rely upon: hanges of the moving obje t position (and orientation) inspa e, and obje t deformations.Obje t position is an easy-to-get hara teristi , appli able both for rigid and non-rigid bodies that is provided by most of the target dete tion and tra king systems,usually as a enter of the target bounding box. A number of te hniques [17℄, [16℄,[11℄, [26℄ were proposed for the dete tion of motion events and for the re ognitionof various types of motions based on the analysis of the moving obje t traje toryand its derivatives. Dete ting obje t orientation is a more hallenging problem whi his usually solved by �tting a model that may vary from a simple ellipsoid [26℄ to a omplex 3D vehi le model [18℄ or a spe i� air raft- lass model adapted for noisyradar images as in [9℄.While obje t orientation hara teristi is more appli able for rigid obje ts, it isobje t deformation that ontains the most essential information about the natureof the non-rigid body motion. This is espe ially true for natural non-rigid obje tsin lo omotion that exhibit substantial hanges in their apparent view, as in this ase the motion itself is aused by these deformations, e.g. walking, running, hoping, rawling, ying, et .There exists a large number of papers dealing with the lassi� ation of movingnon-rigid obje ts and their motions, based on their appearan e. Lipton et al. des ribea method for moving target lassi� ation based on their stati appearan e [19℄ andusing the skeletonization [13℄. Polana and Nelson [24℄ used lo al motion statisti s

Page 3: `Dynamism - Technionron/PAPERS/mobacl_ECCV2002.pdf · 2016-04-24 · Fig. 1. `Dynamism of a Dog on Leash' 1912, b y Giacomo Balla. Albrigh t-Kno x Art Gallery, Bu alo. 2 Related w

(a)(b)( ) (d) (e)Fig. 2. (a) running horse video sequen e, (b) �rst 10 eigen-shapes, ( ,d) �rst and se ond eigen-shapesenlarged, (e) `The Red Horseman', 1914, by Carlo Carra, Civi o Museo d'Arte Contemporanea, Milan. omputed for image grid ells to lassify various types of a tivities. An originalapproa h using the temporal templates and motion history images (MHI) for a tionrepresentation and lassi� ation was suggested by Davis and Bobi k in [3℄. Cutlerand Davis [10℄ des ribe a system for real-time moving obje t lassi� ation based onperiodi ity analysis. It would be impossible to des ribe here the whole spe trum ofpapers done in this �eld and we refer the reader to the following surveys [5℄, [14℄ and[21℄.The most related to our approa h is a work by Ya oob and Bla k [28℄, wheredi�erent types of human a tivities were re ognized using a parameterized represen-tation of measurements olle ted during one motion period. The measurements wereeight motion parameters tra ked for �ve body parts (arm, torso, thigh, alf and foot).In this paper we on entrate on the analysis of the deformations of moving non-rigid bodies in an attempt to extra t hara teristi s that allow us to distinguishbetween di�erent types of motions and di�erent lasses of obje ts.3 Our approa hOur basi assumption is that for any given lass of moving obje ts, like humans, dogs, ats, and birds, the apparent obje t view in every phase of its motion an be en odedas a ombination of several basi body views or on�gurations. Assuming that aliving reature exhibits a pseudo-periodi motion, one motion period an be usedas a omparable information unit. Then, by extra ting the basi views from a largetraining set and proje ting onto them the observed sequen e of obje t views olle tedfrom one motion period, we obtain a parameterized representation of obje t's motionthat an be used for lassi� ation.

Page 4: `Dynamism - Technionron/PAPERS/mobacl_ECCV2002.pdf · 2016-04-24 · Fig. 1. `Dynamism of a Dog on Leash' 1912, b y Giacomo Balla. Albrigh t-Kno x Art Gallery, Bu alo. 2 Related w

Unlike [28℄ we do not assume an initial segmentation of the body into parts anddo not expli itly measure the motion parameters. Instead, we work with the hangingapparent view of deformable obje ts and use the parameterization indu ed by theirform variability.In what follows we des ribe the main steps of the pro ess that in lude,{ Segmentation and tra king of the moving obje t that yield an a urate externalobje t boundary in every frame.{ Periodi ity analysis, in whi h we estimate the frequen y of the pseudo-periodi motion and split the video sequen e into single-period intervals.{ Frame sequen e alignment that brings the single-period sequen es above to astandardized form by ompensating for temporal shift, speed variations, di�erentobje t sizes and imaging onditions.{ Parameterization by building an eigen-shape basis from a training set of possibleobje t views and proje ting the apparent view of a moving body onto this basis.3.1 Segmentation and Tra kingAs our approa h is based on the analysis of deformations of the moving body, thea ura y of the segmentation and tra king algorithm in �nding the target outlineis ru ial for the quality of the �nal result. This rules out a number of available oreasy-to-build tra king systems that provide only a enter of mass or a bounding boxaround the target and alls for more pre ise and usually more sophisti ated solutions.Therefore we de ided to use the geodesi a tive ontour approa h [4℄ and spe i�- ally the `fast geodesi a tive ontour' method des ribed in [15℄, where the segmenta-tion problem is expressed as a geometri energy minimization. We sear h for a urveC that minimizes the fun tionalS[C℄ = Z L(C)0 g(C)ds;where ds is the Eu lidean ar length, L(C) is the total Eu lidean length of the urve,and g is a positive edge indi ator fun tion in a 3D hybrid spa ial-temporal spa ethat depends on the pair of onse utive frames I t�1(x; y) and I t(x; y). It gets smallvalues along the spa ial-temporal edges, i.e. moving obje t boundaries, and highervalues elsewhere.In addition to the s heme des ribed in [15℄, we also use the ba kground infor-mation whenever a stati ba kground assumption is valid and a ba kground im-age B(x; y) is available. In the a tive ontours framework this an be a hievedeither by modifying the g fun tion to re e t the edges in the di�eren e imageD(x; y) = jB(x; y)� I t(x; y)j, or by introdu ing additional area integration terms tothe fun tional S(C):S[C℄ = Z L(C)0 g(C)ds+ �1 Zinside(C) jD(x; y)� 1j2da+ �2 Zoutside(C) jD(x; y)� 2j2da;

Page 5: `Dynamism - Technionron/PAPERS/mobacl_ECCV2002.pdf · 2016-04-24 · Fig. 1. `Dynamism of a Dog on Leash' 1912, b y Giacomo Balla. Albrigh t-Kno x Art Gallery, Bu alo. 2 Related w

where �1 and �2 are �xed parameters and 1, 2 are given by: 1 = averageinside(C)[D(x; y)℄ 2 = averageoutside(C)[D(x; y)℄The latter approa h is inspired by the `a tive ontours without edges' model proposedby Chan and Vese [6℄ and for es the urve C to lose on a region whose interior andexterior have approximately uniform values inD(x; y). A di�erent approa h to utilizethe region information by oupling between the motion estimation and the tra kingproblem was suggested by Paragios and Deri he in [22℄.Figure 3 shows some results of moving obje t segmentation and tra king usingthe proposed method.

Fig. 3. Non-rigid moving obje t segmentation and tra king.3.2 Periodi ity analysisHere we assume that the majority of non-rigid moving obje ts are self-propelled alive reatures whose motion is almost periodi . Thus, one motion period, like a step of awalking man or a rabbit hop, an be used as a natural unit of motion and extra tedmotion hara teristi s an by normalized by the period size.The problem of dete tion and hara terization of periodi a tivities was addressedby several resear h groups and the prevailing te hnique for periodi ity dete tion andmeasurements is the analysis of the hanging 1-D intensity signals along spatio-temporal urves asso iated with a moving obje t or the urvature analysis of featurepoint traje tories [23℄, [20℄, [25℄, [27℄. Here we address the problem using global har-a teristi s of motion su h as moving obje t ontour deformations and the traje toryof the enter of mass.

Page 6: `Dynamism - Technionron/PAPERS/mobacl_ECCV2002.pdf · 2016-04-24 · Fig. 1. `Dynamism of a Dog on Leash' 1912, b y Giacomo Balla. Albrigh t-Kno x Art Gallery, Bu alo. 2 Related w

By running frequen y analysis on su h 1-D ontour metri s as the ontour area,velo ity of the enter of mass, prin ipal axes orientation, et . we an dete t thebasi period of the motion. Figures 4 and 5 present global motion hara teristi sderived from segmented moving obje ts in two sequen es. One an learly observethe ommon dominant frequen y in all three graphs.0 20 40 60 80 100 120 140

−0.2

−0.1

0

0.1

0.2

rad

0 20 40 60 80 100 120 140600

800

1000

1200

1400

sq. p

ixel

s

0 20 40 60 80 100 120 140−4

−2

0

2

4

pixe

ls/fr

ame

frame number

Walking man − global motion parameters

Principal axis inclination angle

Contour area

Center of mass − vertical velocity Fig. 4. Global motion hara teristi s measured for walking man sequen e.The period an also be estimated in a straightforward manner by looking forthe frame where the external obje t ontour best mat hes the obje t ontour in the urrent frame. Figure 6 shows the deformations of a walking man ontour during onemotion period (step). Samples from two di�erent steps are presented and ea h verti alpair of frames is phase syn hronized. One an learly see the similarity betweenthe orresponding ontours. An automated ontour mat hing an be performed in anumber of ways, e.g. by omparing ontour signatures or by looking at the orrelationbetween the obje t silhouettes in di�erent frames. Figure 7 shows four graphs of inter-frame silhouette orrelation values measured for four di�erent starting frames takenwithin one motion period. It is learly visible that all four graphs nearly oin ideand the lo al maxima peaks are approximately evenly spa ed. The period, therefore, an be estimated as the average distan e between the neighboring peaks.

Page 7: `Dynamism - Technionron/PAPERS/mobacl_ECCV2002.pdf · 2016-04-24 · Fig. 1. `Dynamism of a Dog on Leash' 1912, b y Giacomo Balla. Albrigh t-Kno x Art Gallery, Bu alo. 2 Related w

0 20 40 60 80 100 1200

0.05

0.1

0.15

0.2

rad

0 20 40 60 80 100 1207000

7500

8000

8500sq

. pix

els

0 20 40 60 80 100 120−2

−1

0

1

2

pixe

ls/fr

ame

frame number

Walking cat − global motion parameters

Principal axis inclination angle

Contour area

Center of mass − vertical velocity Fig. 5. Global motion hara teristi s measured for walking at sequen e.3.3 Frame sequen e alignmentOne of the most desirable features of any lassi� ation system is the invarian e to aset of possible input transformations. As the input in our ase is not a stati image,but a sequen e of images, the system should be robust to both spa ial and temporalvariations.Spatial alignment: S ale invarian e is a hieved by ropping a square boundingbox around the enter of mass of the tra ked target silhouette and re-s aling it to aprede�ned size (see Figure 8).One way to have orientation invarian e is to keep a olle tion of motion samplesfor a wide range of possible motion dire tions and then look for the best mat h.This approa h was used by Ya oob and Bla k in [28℄ to distinguish between di�erentwalking dire tions. Although here we experiment only with motions nearly parallelto the image plane, the system proved to be robust to small variations in orienta-tion. Sin e we do not want to keep models for both left-to-right and right-to-leftmotion dire tions, the right-to-left moving sequen es are onverted to left-to-rightby horizontal mirror ip.Temporal alignment: A good estimate of the motion period allows us to om-pensate for motion speed variations by re-sampling ea h period subsequen e to a

Page 8: `Dynamism - Technionron/PAPERS/mobacl_ECCV2002.pdf · 2016-04-24 · Fig. 1. `Dynamism of a Dog on Leash' 1912, b y Giacomo Balla. Albrigh t-Kno x Art Gallery, Bu alo. 2 Related w

(t) (t+3) (t+6) (t+9) (t+12)(� ) (�+3) (�+6) (�+9) (�+12)Fig. 6. Deformations of a walking man ontour during one motion period (step). Two steps syn hronizedin phase are shown. One an see the similarity between ontours in orresponding phases.prede�ned duration. This an be done by interpolation between the binary silhou-ette images themselves or between their parameterized representation as explainedbelow. Figure 9 presents an original and re-sampled one-period subsequen e afters aling from 11 to 10 frames.Temporal shift is another issue that has to be addressed in order to align the phaseof the observed one- y le sample and the models stored in the training base. In [28℄it was done by solving a minimization problem of �nding the optimal parametersof temporal s aling and time shift transformations so that the observed sequen e isbest mat hed to the training samples. Polana and Nelson [24℄ handled this problemby mat hing the test one-period subsequen e to referen e template at all possibletemporal translations.Assuming that in the training set all the sequen es are a urately aligned, we�nd the temporal shift of a test sequen e by looking for the starting frame that bestmat hes the generalized (averaged) starting frame of the training samples, as they alllook alike. Figure 10 shows (a) - the referen e starting frame taken as an average overthe temporally aligned training set, (b) - a re-sampled single-period test sequen eand, ( ) the orrelation between the referen e starting frame and the test sequen eframes. The maximal orrelation is a hieved at the seventh frame, therefore the testsequen e is aligned by y li ally shifting it 7 frames to the left.3.4 ParameterizationIn order to redu e the dimensionality of the problem we �rst proje t the obje t imagein every frame onto a low dimensional base that represents all possible appearan esof obje ts that belong to a ertain lass, like humans, four-leg animals, et .Let n be number of frames in the training base of a ertain lass of obje tsand M be a training samples matrix, where ea h olumn orresponds to a spatiallyaligned image of a moving obje t written as a binary ve tor. In our experimentswe use 50 � 50 normalized images, therefore, M is a 2500 � n matrix. Matrix M

Page 9: `Dynamism - Technionron/PAPERS/mobacl_ECCV2002.pdf · 2016-04-24 · Fig. 1. `Dynamism of a Dog on Leash' 1912, b y Giacomo Balla. Albrigh t-Kno x Art Gallery, Bu alo. 2 Related w

0 10 20 30 40 50 600.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

frame offset

inte

r−fr

ame

corr

elat

ion

Fig. 7. Inter-frame orrelation between obje t silhouettes. Four graphs show the orrelation measured forfour initial frames.(a) (b)Fig. 8. S ale alignment. A minimal square bounding box around the enter of the segmented obje t silhou-ette (a) is ropped and re-s aled to form a 50� 50 binary image (b).is de omposed using Singular Value De omposition as M = U�V T , where U is anorthogonal matrix of prin ipal dire tions and the � is a diagonal matrix of singularvalues. The prin ipal basis fUi; i = 1::kg for the training set is then taken as k olumns of U orresponding to the largest singular values in �. Figure 11 presents aprin ipal basis for the training set formed of 800 sample images olle ted from morethan 60 sequen es showing dogs and ats in motion. The basis is built by taking thek = 20 �rst prin ipal omponent ve tors.We assume that by building su h representative bases for every lass of obje tsand then �nding the basis that best represents a given obje t image in a minimaldistan e to the feature spa e (DTFS) sense, we an distinguish between variousobje t lasses. Figure 12 shows the distan es from more than 1000 various images ofpeople, dogs and ats to the feature spa e of people and to that of dogs and ats.

Page 10: `Dynamism - Technionron/PAPERS/mobacl_ECCV2002.pdf · 2016-04-24 · Fig. 1. `Dynamism of a Dog on Leash' 1912, b y Giacomo Balla. Albrigh t-Kno x Art Gallery, Bu alo. 2 Related w

Fig. 9. Temporal alignment. Top: original 11 frames of one period subsequen e. Bottom: re-sampled 10frames sequen e.In all ases, images of people were loser to the people feature spa e than to theanimals' feature spa e and vise a versa. This allows us to distinguish between thesetwo lasses. A similar approa h was used in [12℄ for the dete tion of pedestrians intraÆ s enes.If the obje t lass is known (e.g. we know that the obje t is a dog), we anparameterize the moving obje t silhouette image I in every frame by proje tingit onto the lass basis. Let B be the basis matrix formed from the basis ve torsfUi; i = 1::kg. Then, the parameterized representation of the obje t image I is givenby the ve tor p of length k as p = BTvI , where vI is the image I written as a ve tor.The idea of using a parameterized representation in motion-based re ognition ontext is ertainly not a new one. To name a few examples we mention again thework of Ya oob and Bla k [28℄. Cootes et al. [8℄ used similar te hnique for des ribingfeature point lo ations by a redu ed parameter set. Baumberg and Hogg [1℄ usedPCA to des ribe a set of admissible B-spline models for deformable obje t tra king.Chomat and Crowley [7℄ used PCA-based spatio-temporal �lter for human motionre ognition.Figure 13 shows several normalized moving obje t images from the original se-quen e and their re onstru tion from a parameterized representation by ba k-proje tionto the image spa e. The numbers below are the norms of di�eren es between theoriginal and the ba k-proje ted images. These norms an be used as the DTFS esti-mation.Now, we an use these parameterized representations to distinguish between dif-ferent types of motion. The referen e base for the a tivity re ognition onsists oftemporally aligned one-period subsequen es, whereas the moving obje t silhouettein every frame of these subsequen es is represented by its proje tion to the prin ipalbasis. More formally, let fIf : f = 1::Tg be a one-period, temporally aligned set ofnormalized obje t images, and pf ; f = 1::T a proje tion of the image If onto theprin ipal basis B of size k. Then, the ve tor P of length kT formed by on atenationof all the ve tors pf ; f = 1::T , represent a one-period subsequen e. By hoosing abasis of size k = 20 and the normalized duration of one-period subsequen e to be

Page 11: `Dynamism - Technionron/PAPERS/mobacl_ECCV2002.pdf · 2016-04-24 · Fig. 1. `Dynamism of a Dog on Leash' 1912, b y Giacomo Balla. Albrigh t-Kno x Art Gallery, Bu alo. 2 Related w

(a) (b)

1 2 3 4 5 6 7 8 9 10330

340

350

360

370

380

390

400

410

420

Frame number

Cor

rela

tion

with

the

aver

age

star

ting

fram

e

( )Fig. 10. Temporal shift alignment: (a) - average starting frame of all the training set sequen es, (b) -temporally shifted single- y le test sequen e, ( ) - the orrelation between the referen e starting frame andthe test sequen e framesT = 10 frames, every single-period subsequen e is represented by a feature point ina 200-dimensional feature spa e.In the following experiment we pro essed a number of sequen es of dogs and ats in various types of lo omotion. From these sequen es we extra ted 33 samplesof walking dogs, 9 samples of running dogs, 9 samples with galloping dogs and 14samples of walking ats. In Figure 14 we depi t the resulting feature points proje tedfor visualization to the 3-D spa e using the three �rst prin ipal dire tions . One aneasily observe four separable lusters orresponding to the four groups.Another experiment was done over the `people' lass of images. Figure 15 presentsfeature points orresponding to several sequen es showing people walking and run-ning parallel to the image plane and running at oblique angle to the amera. Again,all three groups lie in separable lusters.The lassi� ation an be performed, for example, using the k-nearest-neighboralgorithm. We ondu ted the `leave one out' test for the dogs set above, lassifyingevery sample by taking them out from the training set one at a time, and the three-nearest-neighbors strategy resulted in 100% su ess rate.

Page 12: `Dynamism - Technionron/PAPERS/mobacl_ECCV2002.pdf · 2016-04-24 · Fig. 1. `Dynamism of a Dog on Leash' 1912, b y Giacomo Balla. Albrigh t-Kno x Art Gallery, Bu alo. 2 Related w

Fig. 11. The prin ipal basis for the `dogs and ats' training set formed of 20 �rst prin ipal omponentve tors.

0 5 10 15 20 25 300

5

10

15

20

25

30

Distance to the "people" feature space

Dis

tanc

e to

the

"dog

s &

cat

s" fe

atur

e sp

ace

dogs and catspeople

Fig. 12. Distan es to the `people' and 'dogs and ats' feature spa es from more than 1000 various imagesof people, dogs and ats.

Page 13: `Dynamism - Technionron/PAPERS/mobacl_ECCV2002.pdf · 2016-04-24 · Fig. 1. `Dynamism of a Dog on Leash' 1912, b y Giacomo Balla. Albrigh t-Kno x Art Gallery, Bu alo. 2 Related w

5.23 4.66 4.01 3.96 3.42 4.45 4.64 6.10 5.99 6.94 5.89Fig. 13. Image sequen e parameterization. Top: 11 normalized target images of the original sequen e.Bottom: the same images after the parameterization using the prin ipal basis and ba k-proje ting to theimage basis. The numbers are the norms of the di�eren es between the original and the ba k-proje tedimages.4 Con luding remarksWe presented a new framework for motion-based lassi� ation of moving non-rigidobje ts. The te hnique is based on the analysis of hanging appearan e of moving ob-je ts and is heavily relying on high a ura y results of segmentation and tra king byusing the fast geodesi ontour approa h. The periodi ity analysis is then performedbased on the global properties of the extra ted moving obje t ontours, followedby video sequen e spatial and temporal normalization. Normalized one-period sub-sequen es are parameterized by proje tion onto a prin ipal basis extra ted from atraining set of images for a given lass of obje ts. A number of experiments showthe ability of the system to analyze motions of humans and animals, to distinguishbetween these two lasses based on obje t appearan e, and to lassify various typeof a tivities with a lass, su h as walking, running, galloping. The `dogs and ats'experiment demonstrate the ability of the system to dis riminate between these twovery similar by appearan e lasses by analyzing their lo omotion.Referen es1. A Baumberg and D Hogg. An eÆ ient method for ontour tra king using a tive shape models. In InPro . IEEE Workshop on Motion of Non-Rigid and Arti ulated Obje ts, pages 194{199, Austin, 1994.2. J R Beniger. The Arts and New Media site. In www.us .edu/s hools/annenberg/as /proje ts/ omm544/, University of South California, Annenberg S hool for Communi ation.3. A Bobi k and J Davis. The representation and re ognition of a tion using temporal templates. IEEETrans. on PAMI, 23(3):257{267, 2001.4. V. Caselles, R. Kimmel, and G. Sapiro. Geodesi a tive ontours. IJCV, 22(1):61{79, 1997.5. C Cedras and M Shah. Motion-based re ognition: A survey. IVC, 13(2):129{155, Mar h 1995.6. T F Chan and L A Vese. A tive ontours without edges. IEEE trans. on Image Pro essing, 10(2):266{277, February 2001.7. O Chomat and J Crowley. Re ognizing motion using lo al appearan e, 1998.8. T F Cootes, C J Taylor, D H Cooper, and J Graham. A tive shape models: Their training andappli ation. CVIU, 61(1):38{59, January 1995.9. N J Cutaia and J A O'Sullivan. Automati target re ognition using kinemati priors. In Pro eedingsof the 33rd Conferen e on De ision and Control, pages 3303{3307, Lake Buena Vista, FL, De ember1994.

Page 14: `Dynamism - Technionron/PAPERS/mobacl_ECCV2002.pdf · 2016-04-24 · Fig. 1. `Dynamism of a Dog on Leash' 1912, b y Giacomo Balla. Albrigh t-Kno x Art Gallery, Bu alo. 2 Related w

dog walking dog running dog gallopingcat walking

Fig. 14. Feature points extra ted from the sequen es with walking, running and galloping dogs and walking ats and proje ted to the 3-D spa e for visualizationwalking running running−45

Fig. 15. Feature points extra ted from the sequen es showing people walking and running parallel to theimage plane and at 45 degrees angle to the amera. Feature points are proje ted to the 3-D spa e forvisualization

Page 15: `Dynamism - Technionron/PAPERS/mobacl_ECCV2002.pdf · 2016-04-24 · Fig. 1. `Dynamism of a Dog on Leash' 1912, b y Giacomo Balla. Albrigh t-Kno x Art Gallery, Bu alo. 2 Related w

10. R Cutler and L Davis. Robust real-time periodi motion dete tion, analysis, and appli ations. PAMI,22(8):781{796, August 2000.11. S A Engel and J M Rubin. Dete ting visual motion boundaries. In Pro . Workshop on Motion:Representation and Analysis, pages 107{111, Charleston, S.C., May 1986.12. U Franke, D Gavrila, S Gorzig, F Lindner, F Paetzold, and C Wohler. Autonomous driving goesdowntown. IEEE Intelligent System, 13(6):40{48, 1998.13. H Fujiyoshi and A Lipton. Real-time human motion analysis by image skeletonization. In Pro . of theWorkshop on Appli ation of Computer Vision, O tober 1998.14. D M Gavrila. The visual analysis of human movement: A survey. CVIU, 73(1):82{98, January 1999.15. R Goldenberg, R Kimmel, E Rivlin, and M Rudzsky. Fast geodesi a tive ontours. IEEE Trans. onImage Pro essing, 10(10):1467{75, O tober 2001.16. K Gould, K Rangarajan, and M Shah. Dete tion and representation of events in motion traje tories. InAdvan es in Image Pro essing and Analysis, hapter 14. SPIE Opti al Engineering Press, June 1992.Gonzalez and Mahdavieh (Eds.).17. K Gould and M Shah. The traje tory primal sket h: a multi-s ale s heme for representing motion hara teristi s. In Pro . Conf. on Computer Vision and Pattern Re ognition, San Diego, CA, pages79{85, 1989.18. D Koller, K Daniilidis, and H-H Nagel. Model-based obje t tra king in mono ular image sequen es ofroad traÆ s enes. International Journal of Computer Vision, 10:257{281, 1993.19. A Lipton, H Fujiyoshi, and R Patil. Moving target lassi� ation and tra king from real-time video. InIn Pro . IEEE Image Understanding Workshop, pages 129{136, 1998.20. F Liu and R W Pi ard. Finding periodi ity in spa e and time. In Pro . of the 6th Int. Conf. onComputer Vision, pages 376{383, Bombay, India, 1998.21. D M Moeslund and E Granum. A survey of omputer vision-based human motion apture. CVIU,81(3):231{268, Mar h 2001.22. N Paragios and R Deri he. Geodesi a tive regions for motion estimation and tra king. In Pro . of the7th Int. Conf. on Computer Vision, pages 688{694, Kerkyra, Gree e, 1999.23. R Polana and R C Nelson. Dete ting a tivities. Journal of Visual Communi ation and Image Repre-sentation, 5:172{180, 1994.24. R. Polana and R.C. Nelson. Dete tion and re ognition of periodi , nonrigid motion. IJCV, 23(3):261{282, June 1997.25. S M Seitz and C R Dyer. View invariant analysis of y li motion. Int. Journal of Computer Vision,25(3):231{251, De ember 1997.26. J M Siskind and QMorris. A maximum-likelihood approa h to visual event lassi� ation. In Pro eedingsof the Fourth European Conferen e on Computer Vision, pages 347{360, Cambridge, UK, April 1996.27. P S Tsai, M Shah, K Keiter, and T Kasparis. Cy li motion dete tion for motion based re ognition.Pattern Re ognition, 27(12):1591{1603, De ember 1994.28. Y Ya oob and M J Bla k. Parameterized modeling and re ognition of a tivities. CVIU, 73(2):232{247,February 1999.