SummerCourse-France-2018-Presentation-v0.3 - Modo de...

Summer School on Cooperative Interacting Automobiles, Loire Valley, France. September 2018

Intelligent Interaction between Vehicles and VRUs

Miguel Ángel Sotelo

SPAIN

Summer School on Cooperative Interacting Automobiles, Loire Valley, France. September 2018 2

Motivation

Proposed approach

Pedestrian pose measurement

Dimensionality reduction and Prediction

Experimental results

Conclusions and Future Work

Content


Motivation


Motivation

• Pedestrian Path Prediction in the Automotive:

– Further improvement in state-of-the-art ADAS by meansof action classification based on body language reading

– Walking, Stopping, Starting, Bending-in

– Improvement of accuracy in 30-50 cm:

– Difference between effective and non-effectiveintervention in emergency braking systems

– Initiation of emergency braking 0.16 s in advance canpotentially reduce severity of accidents injuries by 50%

– Early recognition of pedestrians stopping/walking actionscan provide more accurate last-second active interventions

Strong gains are expected in the performance and reliability of active

pedestrian protection systems






Motivation

Proposed approach





Content


Proposed Approach

Global Scheme

Off-line Motion Capture System

Recovery of 3D pose and position

Lateral predicted position

On-line

Probabilistic Training

Knowledge of

Pedestrians

dynamics

Stereo Cameras

Transformation to latent space +

prediction

Geometric processing

3D Pedestrian Pose

Estimation


Motivation

Proposed approach





Content


Pedestrian Pose Measurement

Pedestrian Skeleton considered in this research



Body parts detection using Deep Learning

St: Confidence MapsLt: Parts Affinity Maps (PAF) (Z. Cao. CVPR 2017)




Confidence maps of the right wrist (first row) and PAFs(second row) of right forearm across stages


Motivation

Proposed approach





Content


Dimensionality Reduction

Major Goal

Learning the pedestrians motion dynamics by reducingthe dimensionality of input data and to make it moremanageable and interpretable in a low dimensional latentembedding

Method considered

GPDM (Gaussian Process with Dynamical Model)

Input data-set

CMU (41 body joints obtained with motion capture syst.)



CMU Sequence Example



GPDM (1/3)

1. Nonlinear probabilistic generalization of PCA: marginalization of the mapping (W) and optimization of the latent variables (X)

2. Conditional density in GPDM for a centered observed data Y, latent variable X, and kernel hyperparameters θ

where W is a scaling matrix and K is a kernel matrix with:

where δi,j is the Kronecker delta function



GPDM (2/3)

3. Dynamic mapping from latent coordinates:

where Xout=[X2, … , XN], d is the model dimension and KX is the kernel built from {X1, … , XN-1} with:

where βi are hyperparameters

4. Minimization of –ln p(X,θ,β,W|Y) using SCG



GPDM (3/3)

Prediction

1. Predicted latent position in the next frame (kX is a column vector)

2. Reconstructed 3D pose (kY is a column vector)


Learning Pedestrian Motion

Pedestrian walking 6 steps (using B-GPDM)

Green: projection of pedestrian motion sequence onto the subspace.

Variance in color code: cold (small) to warm (big)


Experimental Results

Video sequence showing prediction results


Motivation

Proposed approach





Content


General Method - Overview


Datasets

Frame Rate of training data

CMU data-set: 120 fps (frames per second)

Feature Vector

UAH: 11 body joints (3D pose + 3D motion: 11x6)

CMU: 41 (we consider only the 11 joints in common with UAH)

Pedestrian Behaviors

Walking (toward the curb to enter the road lane)

Stopping (at the curb)

Starting (from the curb)

Standing

Prediction Horizon

Up to 1.0 s



UAH Dataset - Breakdown


Manual Labelling of Pedestrian Poses

Standing-Starting Transition

Criteria: movement of leg forward



Starting-Walking Transition

Criteria: pedestrian places foot on the ground after one stride



Walking-Stopping Transition

Criteria: pedestrian places foot on the ground in the final stride before full stop



Stopping-Standing Transition

Criteria: pedestrian comes to a full stop


Activity Recognition

Clues for prediction of “Starting” behaviors



Clues for prediction of “Stopping” behaviors



Pedestrian activity modelled as HMM




Prior

Emission (Likelihood)

α: SSE for pedestrian poseβ: SSE for pedestrian displacement



Recognition results for different features and #joints



Probabilistic Action Classification



Detection Delay - Summary



UAH Prototype vehicle - DRIVERTIVE



Experiments in UTAC proving ground (France) – BRAVE H2020 Project




Expected Contributions to Euro NCAP

- Recommendations to Euro NCAP WG on AD on how to measure performance of VRU prediction systems

- Enhanced performance in Euro NCAP tests by including predictions


Intelligent Interface with VRUs

GRAIL – GReen Assistant Interface Light


Use Case 1 – Pedestrians (GRAIL-1)

The pedestrian stops and looks at the ego-vehicle

Ego-vehicle decelerates until full stop and indicates to thepedestrian that s/he can cross.

The pedestrian continues and completely crosses.


Use Case 1 – Pedestrians (GRAIL-2)

The ego-vehicle decelerates, the pedestrian turns his/her head to the left indicating a possible intention to cross

The ego-vehicle comes to a full stop while indicating the pedestrian that s/he can cross.

The pedestrian crosses.


Use Case 2 – Pedestrians (passing-by)

Pedestrian walking along the road

The ego-vehicle decelerates

Passes the pedestrian slowly.


Use Case 3 – Pedestrians (avoidance)

25%

Crossing scenario with impact point at 25%.

AES activation to avoid the impact.


Use Case 4 – Cyclists (passing-by)

It’s possible to overtake so deceleration takes place

and overtaking with the correct distance with the bicycle.


Use Case 5 – Cyclists (traffic context)

The ego-vehicle decelerates and waits to the oncoming traffic to pass

After that it overtakes.



GRAIL – GReen Assistant Interface Light



GRAIL will be demonstrated at IEEE IROS 2018

First week of October, Madrid


Motivation

Proposed approach





Content



Conclusions

- Accurate path pedestrian prediction is possible up to 1.0 s using body language traits and probabilistic dimensionality reduction techniques.

- Predictions are carried out in the latent space, yielding results with a potential accuracy of 15-20 cm at a time horizon of 1.0 s

- The generative solution is not scalable to thousands of examples. A new approach is needed for massive learning.

Future Research Challenges

- Taking a discriminative approach to the prediction problem: Deep Learning? Recurrent Neural Networks?

- VRU recognition (standing pedestrian, seating pedestrian, cyclist, motorcyclist).

- Context-based action prediction:

- Gaze direction, group behavior, type of road.

- Use of Probabilistic Graphical Models (Bayesian Networks).


Merci beaucoup!

Miguel Ángel Sotelo

[email protected] University of Alcalá, SPAIN

Intelligent Interaction between Vehicles and VRUs

SummerCourse-France-2018-Presentation-v0.3 - Modo de...

Documents

Transcript of SummerCourse-France-2018-Presentation-v0.3 - Modo de...