Event prediction
description
Transcript of Event prediction
![Page 1: Event prediction](https://reader036.fdocuments.net/reader036/viewer/2022062315/5681632e550346895dd3a9ce/html5/thumbnails/1.jpg)
Event prediction
CS 590v
![Page 2: Event prediction](https://reader036.fdocuments.net/reader036/viewer/2022062315/5681632e550346895dd3a9ce/html5/thumbnails/2.jpg)
Applications
• Video search• Surveillance– Detecting suspicious activities– Illegally parked cars– Abandoned bags
• Intelligent environments– Healthcare: fall detector– Healthcare: hand-washing prompter
![Page 3: Event prediction](https://reader036.fdocuments.net/reader036/viewer/2022062315/5681632e550346895dd3a9ce/html5/thumbnails/3.jpg)
Common approaches
• Compare keypoint (joint) trajectory shapes• Various spatio-temporal features with
classifier
![Page 4: Event prediction](https://reader036.fdocuments.net/reader036/viewer/2022062315/5681632e550346895dd3a9ce/html5/thumbnails/4.jpg)
Trajectories of keypoints
• Yilmaz and Shah, 2005 – UCF• Joint trajectories in XYT space• Compare trajectory shapes to classify actions
![Page 5: Event prediction](https://reader036.fdocuments.net/reader036/viewer/2022062315/5681632e550346895dd3a9ce/html5/thumbnails/5.jpg)
Scene context
• Idea: transfer event information only from similar images
• Use context to determine expected motion– E.g. climber on a rock wall vs. climber on a
building
![Page 6: Event prediction](https://reader036.fdocuments.net/reader036/viewer/2022062315/5681632e550346895dd3a9ce/html5/thumbnails/6.jpg)
Features for matching images: GistOliva and Torralba, 2001
8 orientations 4 scalesx 16 bins 512 dimensions
•Apply oriented Gabor filters over different scales•Average filter energy in each bin
•Used for scene recognition•Similar to SIFT (Lowe 1999)
![Page 7: Event prediction](https://reader036.fdocuments.net/reader036/viewer/2022062315/5681632e550346895dd3a9ce/html5/thumbnails/7.jpg)
Retrieving similar images from large image databases
Hays and Efros, 2007Image completion using Flickr images
![Page 8: Event prediction](https://reader036.fdocuments.net/reader036/viewer/2022062315/5681632e550346895dd3a9ce/html5/thumbnails/8.jpg)
Transfer of knowledge: SIFT-flow
car sidewalk treeSystem
C. Liu, J. Yuen, A. Torralba, 2009
• Can “copy and paste” segmentation labels from similar labeled example to unknown
![Page 9: Event prediction](https://reader036.fdocuments.net/reader036/viewer/2022062315/5681632e550346895dd3a9ce/html5/thumbnails/9.jpg)
Task
• Predict motion from static images• Predict semantic event from static images
![Page 10: Event prediction](https://reader036.fdocuments.net/reader036/viewer/2022062315/5681632e550346895dd3a9ce/html5/thumbnails/10.jpg)
Approach: high-level
• Model video as trajectories of keypoints– Cluster of trajectories for each object
• Global similarity measure
![Page 11: Event prediction](https://reader036.fdocuments.net/reader036/viewer/2022062315/5681632e550346895dd3a9ce/html5/thumbnails/11.jpg)
Tracking key points
• KLT feature tracker– Solve for the displacement d that optimizes a
dissimilarity metric, per pair of consecutive frames
• Cluster trajectories into objects– By average distance between them
![Page 12: Event prediction](https://reader036.fdocuments.net/reader036/viewer/2022062315/5681632e550346895dd3a9ce/html5/thumbnails/12.jpg)
Comparing track clusters
• Create spatial histogram of cluster velocities• Sum of histogram intersections is the similarity
score
• 1-level spatial pyramid matching
![Page 13: Event prediction](https://reader036.fdocuments.net/reader036/viewer/2022062315/5681632e550346895dd3a9ce/html5/thumbnails/13.jpg)
Predicting local motion
• Average motion over N nearest neighbors
![Page 14: Event prediction](https://reader036.fdocuments.net/reader036/viewer/2022062315/5681632e550346895dd3a9ce/html5/thumbnails/14.jpg)
Event prediction
• Match query with similar track clusters from database
• Cluster retrieved tracks to reduce redundant results
![Page 15: Event prediction](https://reader036.fdocuments.net/reader036/viewer/2022062315/5681632e550346895dd3a9ce/html5/thumbnails/15.jpg)
Results: local motion prediction
![Page 16: Event prediction](https://reader036.fdocuments.net/reader036/viewer/2022062315/5681632e550346895dd3a9ce/html5/thumbnails/16.jpg)
Results: event prediction
![Page 17: Event prediction](https://reader036.fdocuments.net/reader036/viewer/2022062315/5681632e550346895dd3a9ce/html5/thumbnails/17.jpg)
Results: event prediction
![Page 18: Event prediction](https://reader036.fdocuments.net/reader036/viewer/2022062315/5681632e550346895dd3a9ce/html5/thumbnails/18.jpg)
Unusual events
![Page 19: Event prediction](https://reader036.fdocuments.net/reader036/viewer/2022062315/5681632e550346895dd3a9ce/html5/thumbnails/19.jpg)
Overall discussion
• Still need training data from scenes similar to target
• Possible corruption by unusual events in the training set?
• Applications– Unusual event detector– Knowledge transference between environments
![Page 20: Event prediction](https://reader036.fdocuments.net/reader036/viewer/2022062315/5681632e550346895dd3a9ce/html5/thumbnails/20.jpg)
The end