Reactive Pedestrian Path Following from Examples

Computer Animation and Social Computer Animation and Social Agents 2003Agents 2003

Ronald A. MetoyerRonald A. Metoyer

Jessica K. HodginsJessica K. Hodgins

Computer Animation and Social Computer Animation and Social Agents 2003Agents 2003

Ronald A. MetoyerRonald A. Metoyer

Jessica K. HodginsJessica K. Hodgins

Introduction

Need a system to model the movement of many people walking and interacting

Want to maintain control over the path each individual takes

Hard to deal with collision avoidance with many characters

Easy to use

Need a system to model the movement of many people walking and interacting

Want to maintain control over the path each individual takes

Hard to deal with collision avoidance with many characters

Easy to use

Previous Work

ReynoldsReynolds

• Boid Model for flocks, schools, and herdsBoid Model for flocks, schools, and herds

Pedestrian ModelsPedestrian Models

• Fluid flow modelFluid flow model

• Inter-pedestrian interaction models (Helbing and Inter-pedestrian interaction models (Helbing and Molnar)Molnar)

– Social interaction based on + and – potential fieldsSocial interaction based on + and – potential fields

– Lane formation in halls, queuing, turn takingLane formation in halls, queuing, turn taking

ReynoldsReynolds

• Boid Model for flocks, schools, and herdsBoid Model for flocks, schools, and herds

Pedestrian ModelsPedestrian Models

• Fluid flow modelFluid flow model

• Inter-pedestrian interaction models (Helbing and Inter-pedestrian interaction models (Helbing and Molnar)Molnar)

– Social interaction based on + and – potential fieldsSocial interaction based on + and – potential fields

– Lane formation in halls, queuing, turn takingLane formation in halls, queuing, turn taking

2D Character Intelligence

Exploit fact that humans have to move on a Exploit fact that humans have to move on a 2D plane (for the most part)2D plane (for the most part)

Basic level of intelligenceBasic level of intelligence• ReactiveReactive path following, obstacles, other pedestrians path following, obstacles, other pedestrians

Social Forces ModelSocial Forces Model• ReactiveReactive control utilizes control utilizes potential fieldspotential fields

• Obstacles are repulsiveObstacles are repulsive

• Goals are attractiveGoals are attractive

Exploit fact that humans have to move on a Exploit fact that humans have to move on a 2D plane (for the most part)2D plane (for the most part)

Basic level of intelligenceBasic level of intelligence• ReactiveReactive path following, obstacles, other pedestrians path following, obstacles, other pedestrians

Social Forces ModelSocial Forces Model• ReactiveReactive control utilizes control utilizes potential fieldspotential fields

• Obstacles are repulsiveObstacles are repulsive

• Goals are attractiveGoals are attractive

Potential Fields

Modeling

Point mass dynamicsPoint mass dynamics• Update equation is:Update equation is:

• Where the force Where the force ffxx is obtained from the potential field is obtained from the potential field

• dtdt is the simulation steps is the simulation steps

• mm is the mass of the character is the mass of the character

Although goal locations can be specified, it is Although goal locations can be specified, it is desirable to allow a definable path for the desirable to allow a definable path for the character to followcharacter to follow• People are experts in drawing a path through a scene in the People are experts in drawing a path through a scene in the

absence of moving obstaclesabsence of moving obstacles

• Can also be generated through automatic processCan also be generated through automatic process

Point mass dynamicsPoint mass dynamics• Update equation is:Update equation is:

• Where the force Where the force ffxx is obtained from the potential field is obtained from the potential field

• dtdt is the simulation steps is the simulation steps

• mm is the mass of the character is the mass of the character

Although goal locations can be specified, it is Although goal locations can be specified, it is desirable to allow a definable path for the desirable to allow a definable path for the character to followcharacter to follow• People are experts in drawing a path through a scene in the People are experts in drawing a path through a scene in the

absence of moving obstaclesabsence of moving obstacles

• Can also be generated through automatic processCan also be generated through automatic process

Path Diagram

User draws a spline path for User draws a spline path for character to followcharacter to follow

The path is converted into forces The path is converted into forces by the following:by the following:

Character will attempt to follow Character will attempt to follow the direction of the path, but as the direction of the path, but as it gets more off track, it’ll be it gets more off track, it’ll be pulled back strongerpulled back stronger

Direction Primitives

Intelligence model will produce correct 2D Intelligence model will produce correct 2D animation in terms of obstacle avoidance, animation in terms of obstacle avoidance, but not necessarily natural lookingbut not necessarily natural looking

Alert user to potential collisions and ask how Alert user to potential collisions and ask how to resolve themto resolve them

Navigation PrimitivesNavigation Primitives

• Yield, Cut-in-front, Go-around-right, Go-around-left, Yield, Cut-in-front, Go-around-right, Go-around-left, No-actionNo-action

• Chosen based on traffic planning researchChosen based on traffic planning research

Intelligence model will produce correct 2D Intelligence model will produce correct 2D animation in terms of obstacle avoidance, animation in terms of obstacle avoidance, but not necessarily natural lookingbut not necessarily natural looking

Alert user to potential collisions and ask how Alert user to potential collisions and ask how to resolve themto resolve them

Navigation PrimitivesNavigation Primitives

• Yield, Cut-in-front, Go-around-right, Go-around-left, Yield, Cut-in-front, Go-around-right, Go-around-left, No-actionNo-action

• Chosen based on traffic planning researchChosen based on traffic planning research

Direction Primitives (Cont.)

Focus on two tasks a pedestrian performsFocus on two tasks a pedestrian performs

• MonitoringMonitoring

– Observing other pedestrians in the area to Observing other pedestrians in the area to determine their navigational intentsdetermine their navigational intents

• YieldingYielding

– Act of adjusting velocity (Magnitude or Direction) Act of adjusting velocity (Magnitude or Direction) to avoid a potential collisionto avoid a potential collision

Focus on two tasks a pedestrian performsFocus on two tasks a pedestrian performs

• MonitoringMonitoring

– Observing other pedestrians in the area to Observing other pedestrians in the area to determine their navigational intentsdetermine their navigational intents

• YieldingYielding

– Act of adjusting velocity (Magnitude or Direction) Act of adjusting velocity (Magnitude or Direction) to avoid a potential collisionto avoid a potential collision

LearningUse previous direction primitive choices to aid the user Use previous direction primitive choices to aid the user

in future decisionsin future decisions• Direction PrimitiveDirection Primitive

• Feature vector that describes current sceneFeature vector that describes current scene

– Is the path around left blocked by other pedestrians or obstacles (Y or N)Is the path around left blocked by other pedestrians or obstacles (Y or N)

– Is the path around right blocked by other pedestrians or obstacles (Y or N)Is the path around right blocked by other pedestrians or obstacles (Y or N)

– Relative speed of the colliding pedestrian (5)Relative speed of the colliding pedestrian (5)

– Approach direction of the colliding pedestrian (8)Approach direction of the colliding pedestrian (8)

– Colliding pedestrian’s distance to collision (5)Colliding pedestrian’s distance to collision (5)

– Pedestrian’s distance to collision (5)Pedestrian’s distance to collision (5)

– Desired travel direction (3)Desired travel direction (3)

Use previous direction primitive choices to aid the user Use previous direction primitive choices to aid the user in future decisionsin future decisions• Direction PrimitiveDirection Primitive

• Feature vector that describes current sceneFeature vector that describes current scene

– Is the path around left blocked by other pedestrians or obstacles (Y or N)Is the path around left blocked by other pedestrians or obstacles (Y or N)

– Is the path around right blocked by other pedestrians or obstacles (Y or N)Is the path around right blocked by other pedestrians or obstacles (Y or N)

– Relative speed of the colliding pedestrian (5)Relative speed of the colliding pedestrian (5)

– Approach direction of the colliding pedestrian (8)Approach direction of the colliding pedestrian (8)

– Colliding pedestrian’s distance to collision (5)Colliding pedestrian’s distance to collision (5)

– Pedestrian’s distance to collision (5)Pedestrian’s distance to collision (5)

– Desired travel direction (3)Desired travel direction (3)

Learning (Cont.)Naïve Bayes ClassifierNaïve Bayes Classifier

• Five primitives are hypothesesFive primitives are hypotheses

• Seven variables are inputsSeven variables are inputs

• Potential collisions are classified into one of the 5 primitivesPotential collisions are classified into one of the 5 primitives

AdvantagesAdvantages• Outperforms neural networks and machine learning algorithms Outperforms neural networks and machine learning algorithms

in most real life casesin most real life cases

DisadvantagesDisadvantages• Limited by the fact that it can only deal with discrete dataLimited by the fact that it can only deal with discrete data

Naïve Bayes ClassifierNaïve Bayes Classifier• Five primitives are hypothesesFive primitives are hypotheses

• Seven variables are inputsSeven variables are inputs

• Potential collisions are classified into one of the 5 primitivesPotential collisions are classified into one of the 5 primitives

AdvantagesAdvantages• Outperforms neural networks and machine learning algorithms Outperforms neural networks and machine learning algorithms

in most real life casesin most real life cases

DisadvantagesDisadvantages• Limited by the fact that it can only deal with discrete dataLimited by the fact that it can only deal with discrete data

3D Motion Generation

Use motion captureUse motion capture

Create a directed graph of poses to get a Create a directed graph of poses to get a probability matrix for transitions from one probability matrix for transitions from one pose to anotherpose to another

Use motion captureUse motion capture

Create a directed graph of poses to get a Create a directed graph of poses to get a probability matrix for transitions from one probability matrix for transitions from one pose to anotherpose to another

Results

Compared the Naïve Bayes algorithm to Compared the Naïve Bayes algorithm to actual choices made by usersactual choices made by users

• Claim 72% accuracy as opposed to a random choice Claim 72% accuracy as opposed to a random choice which would be 20% naturallywhich would be 20% naturally

• This doesn’t mean much, because all it is really This doesn’t mean much, because all it is really testing is their ability to train a Bayes classifiertesting is their ability to train a Bayes classifier

Compared the Naïve Bayes algorithm to Compared the Naïve Bayes algorithm to actual choices made by usersactual choices made by users

• Claim 72% accuracy as opposed to a random choice Claim 72% accuracy as opposed to a random choice which would be 20% naturallywhich would be 20% naturally

• This doesn’t mean much, because all it is really This doesn’t mean much, because all it is really testing is their ability to train a Bayes classifiertesting is their ability to train a Bayes classifier

Limitations

Requires (utilizes) a lot of human interventionRequires (utilizes) a lot of human intervention

There is no motion capture data of a person There is no motion capture data of a person stopped, so it appears the person is stopped, so it appears the person is spinning around when standing stillspinning around when standing still

Requires (utilizes) a lot of human interventionRequires (utilizes) a lot of human intervention

There is no motion capture data of a person There is no motion capture data of a person stopped, so it appears the person is stopped, so it appears the person is spinning around when standing stillspinning around when standing still

Videos

Basic System

Renderer

DynamicsController

Integrator

U/I

State

State

StateTorques

Desired State

StateTorques

Cart / Pole

Cart / Pole

Apply torque to cart’s wheelsApply torque to cart’s wheels

• Balance poleBalance pole

• Accomplish desired locationAccomplish desired location

• Accomplish desired velocityAccomplish desired velocity

Extra CreditExtra Credit

• Swing-up taskSwing-up task

Apply torque to cart’s wheelsApply torque to cart’s wheels

• Balance poleBalance pole

• Accomplish desired locationAccomplish desired location

• Accomplish desired velocityAccomplish desired velocity

Extra CreditExtra Credit

• Swing-up taskSwing-up task

Basic input file for Cart / Pole

language = Clanguage = C

gravity = 0 0 -9.80665gravity = 0 0 -9.80665

prefix = cartpoleprefix = cartpole

# cart is a truck-sized object, 20 x 4 x 3 feet = 6x1.5x1 # cart is a truck-sized object, 20 x 4 x 3 feet = 6x1.5x1 metersmeters

# with car-like density of 170 kg / m^3# with car-like density of 170 kg / m^3

# therefore, truck-like mass of 1800kg = 4000 lbs # therefore, truck-like mass of 1800kg = 4000 lbs

language = Clanguage = C

gravity = 0 0 -9.80665gravity = 0 0 -9.80665

prefix = cartpoleprefix = cartpole

# cart is a truck-sized object, 20 x 4 x 3 feet = 6x1.5x1 # cart is a truck-sized object, 20 x 4 x 3 feet = 6x1.5x1 metersmeters

# with car-like density of 170 kg / m^3# with car-like density of 170 kg / m^3

# therefore, truck-like mass of 1800kg = 4000 lbs # therefore, truck-like mass of 1800kg = 4000 lbs

body = cart body = cart joint = slider joint = slider jname = posjname = pos

mass = 1530mass = 1530

inertia = 414.37500000 4717.50000000 4876.87500000inertia = 414.37500000 4717.50000000 4876.87500000

bodyToJoint = 0 0 0bodyToJoint = 0 0 0

pin = 1 0 0pin = 1 0 0

body = cart body = cart joint = slider joint = slider jname = posjname = pos

mass = 1530mass = 1530

inertia = 414.37500000 4717.50000000 4876.87500000inertia = 414.37500000 4717.50000000 4876.87500000

bodyToJoint = 0 0 0bodyToJoint = 0 0 0

pin = 1 0 0pin = 1 0 0

# A 300 lb = 136 kg ladder that is roughly # A 300 lb = 136 kg ladder that is roughly # 15 x 1.5 x 0.5 feet = 4.6x.45x.15 meters # 15 x 1.5 x 0.5 feet = 4.6x.45x.15 meters

body = ladder body = ladder inboard = cart inboard = cart joint = pin joint = pin jname = thetajname = theta

mass = 52.785mass = 52.785

inertia = 0.98971875 93.17652187 93.96829687inertia = 0.98971875 93.17652187 93.96829687

bodyTojoint = -2.3 0 0bodyTojoint = -2.3 0 0

inbToJoint = -3.0 0 0.75inbToJoint = -3.0 0 0.75

pin = 0 1 0pin = 0 1 0

# A 300 lb = 136 kg ladder that is roughly # A 300 lb = 136 kg ladder that is roughly # 15 x 1.5 x 0.5 feet = 4.6x.45x.15 meters # 15 x 1.5 x 0.5 feet = 4.6x.45x.15 meters

body = ladder body = ladder inboard = cart inboard = cart joint = pin joint = pin jname = thetajname = theta

mass = 52.785mass = 52.785

inertia = 0.98971875 93.17652187 93.96829687inertia = 0.98971875 93.17652187 93.96829687

bodyTojoint = -2.3 0 0bodyTojoint = -2.3 0 0

inbToJoint = -3.0 0 0.75inbToJoint = -3.0 0 0.75

pin = 0 1 0pin = 0 1 0

Swinger

More complicated simulation of girl on a More complicated simulation of girl on a swingswing

• Hands are rigidly attached to ropeHands are rigidly attached to rope

• Butt is rigidly attached to seatButt is rigidly attached to seat

• You control torques at shoulder, elbow, hips, and kneeYou control torques at shoulder, elbow, hips, and knee

More complicated simulation of girl on a More complicated simulation of girl on a swingswing

• Hands are rigidly attached to ropeHands are rigidly attached to rope

• Butt is rigidly attached to seatButt is rigidly attached to seat

• You control torques at shoulder, elbow, hips, and kneeYou control torques at shoulder, elbow, hips, and knee

Swinger

State machineState machine

• Swinging has discrete modes, or statesSwinging has discrete modes, or states

– Define when they begin and endDefine when they begin and end

– Define what movements are required for each stateDefine what movements are required for each state

State machineState machine

• Swinging has discrete modes, or statesSwinging has discrete modes, or states

– Define when they begin and endDefine when they begin and end

– Define what movements are required for each stateDefine what movements are required for each state

Discrete event simulations

Very important!!!Very important!!!

• Each simulation has a simulation timestep, DTEach simulation has a simulation timestep, DT

• Smaller timestep required for larger forcesSmaller timestep required for larger forces

– Numerical imprecision of integratorNumerical imprecision of integrator

• Make sure your simulations are precise by dropping DT Make sure your simulations are precise by dropping DT by an order of magnitude and confirm behavior is the by an order of magnitude and confirm behavior is the samesame

Very important!!!Very important!!!

• Each simulation has a simulation timestep, DTEach simulation has a simulation timestep, DT

• Smaller timestep required for larger forcesSmaller timestep required for larger forces

– Numerical imprecision of integratorNumerical imprecision of integrator

• Make sure your simulations are precise by dropping DT Make sure your simulations are precise by dropping DT by an order of magnitude and confirm behavior is the by an order of magnitude and confirm behavior is the samesame