Human Motion Detection and TrackingA Technical Presentation

Members● William Juszczyk● Michael Lazar● Ryan Lattrel● Mark Birdsall● Camden Smith

Design Team 2Moving Human Electromagnetic

Scattering SimulatorFacilitator● Dr. Wen Li

Sponsor● Air Force Research

Laboratory

● Introduction● History● Methods of detection● Modeling techniques● Applications● Summary● Questions

Presentation Outline

● Human motion tracking is a subset motion tracking

● Accurately identifying human activity has become important in a large number of fields

● Developments in camera technologies and data processing make complex system possible

Introduction

History ● Infrared Search and Track (IRST) - First IR Detection● Motion Controllers

a. SEGA Activator - first Motion Controller (1993)b. Nintendo Wii-Remote - using accelerometers

(2006)● Depth Sensing Technology,

Xbox Kinect first systemto use DST.

Picture - SEGA Activator. Sits on the groundwhile the player stands in middle.

http://mytechroom.co.uk/kinectivator/

Methods of Detection

Methods of Detection● There are many ways to detect and track

humans○ Sound○ Vibration○ Ultrasonic sonar○ Infrared

● All have unique advantages and disadvantages

Sound● Monitor acoustic

sound signatures● Footsteps have

broadband frequency○ Generated by foot

striking the ground● Works best

indoors(http://www.tpub.com/neets/book10/NTX1-14.GIF)

Vibration● Detection with an

accelerometer● Difficult to analyze

○ Varies with site● Can work in non-

ideal conditions○ Outdoors○ Wind or noise

(https://en.wikipedia.org/wiki/File:Kinemetrics_seismograph.jpg)

Ultrasonic Sonar● Uses sound waves● Can detect moving

objects● Wide field of vision

○ Infrared is more limited

○ Sound can travel further

(http://www.cypress.com/?rID=82)

Infrared● Detect emitted

energy○ Unlike ultrasonic

sonar, there is no transmitter

● Sees changes of temperature around a point

● Can be calibrated to track movement

(https://upload.wikimedia.org/wikipedia/commons/2/25/ParowozIR.jpg)

Microsoft Modeling

Algorithm Approaches● Calibration pose

○ Old theory○ Problems?

● Microsoft approach○ Robust solution○ Scan 3D image○ This research led to Kinect SDK

http://leonardodavinci.stanford.edu/submissions/clabaugh/history/leonardo.html

Joint Labeling● Single depth

image● Body part labeling● 3D joint proposals● Body parts are

then treated per pixel http://research.microsoft.

com/pubs/145347/BodyPartRecognition.pdf

How Decisions are Made● Generate "realistic" depth images● Use a decision forest

○ A decision forest is a group of decision trees● Speed issues

○ Each pixel run in parallel on the decision forest○ On Xbox 360 GPU: 5ms per frame

http://research.microsoft.com/pubs/145347/BodyPartRecognition.pdf

Alternative Modeling

2D Model● biped model● 6 rectangles in the model

○ head○ torso○ left/right thigh○ left/right calf

● Detection and tracking algorithm in a 3 step loop○ Detecting human candidate○ Validating model of a human○ Tracking of the model in subsequent frames

Detecting Human Candidate● Input is output of a pixel-based

motion detector● A rectangular mask is compared

to the different parts of the image○ If there is motion detected that can be

fit within the mask, then this is a region of interest (ROI)

○ The mask is scaled based on vertical location of ROI

● Assumes people are roughly the same height

http://www.sciencedirect.com/science/article/pii/S0031320308000071

Validating the Model● Biped model is initialized at each ROI

○ Determines position and orientation of model

○ Pose of model that fits ROI best is calculated using a mean-shift algorithm

○ Location of torso found first, then legs, then head

● Model is refined○ Vertical position of each model part needs

to be shifted to better fit the image○ Non-humans will be thrown out.

http://cmp.felk.cvut.cz/demos/Tracking/TrackHumansKorc/

Tracking of Model● Pose of model from previous frame is kept● Algorithm tries to anticipate the human pose

in the current frame based on the previous frame.

● Runs the same algorithm from the Validation phase to fit model to image.

● There are limitations due to occlusion.

Applications

● Automotive○ Motivated by Safety○ MobilEye PCW○ Warning Systems○ Active vs. Passive

● Surveillance○ More Efficient○ Improved reliability○ Used in airports, subway stations, office buildings,

private use

Applications

http://www.mobileye.com/technology/applications/pedestrian-detection/pedestrian-collision-warning/

● Content based video categorization● Sports video analysis● Motion capture for film and television● Gesture recognition for smart devices

More Applications

Summary● History

○ Infrared ○ motion controllers

● Methods of Detection○ Sound, vibration, sonar, and infrared

● Modeling Humans○ Microsoft's approach○ Two dimensional model

● Applications○ Auto○ Security

Questions?

ReferencesMotion Detection History: 1) http://illumin.usc.edu/165/motion-sensors/2) http://en.wikipedia.org/wiki/Infra-red_search_and_trackMethods:1) http://www.doityourself.com/stry/ultrasonic-vs-infrared-wireless-motion-detectors2) http://www.cypress.com/?rID=823) https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5935481&tag=14) http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA4787955) http://research.microsoft.com/pubs/145347/BodyPartRecognition.pdf