Tracking of Animals Using Airborne...

Tracking of AnimalsUsing Airborne CamerasClas Veibäck

1 Introduction2 Background3 Constrained Motion Model4 Uncertain Timestamp Model5 Mode Observations6 Conclusions

LINK-SIC Presentation Clas Veibäck November 7, 2016 2

Introduction

• Tracking of animals

• Overhead cameras

• Applicable to other target

• Applications to demonstrate

theory

Introduction

theory

Introduction

theory

Introduction

theory

Main Contributions

Uncertain Timestamps

Constrained Motion Model Mode Observations

k− 1 k k+ 1

Main Contributions

Constrained Motion Model

Mode Observations

k− 1 k k+ 1

Main Contributions

Constrained Motion Model Mode Observations

k− 1 k k+ 1

Dolphin Application

• Dolphinarium at Kolmården

Wildlife Park

• Fisheye camera with

occlusions

• Reflections and changing

light conditions

• Constrained to basin

Dolphin Application

Wildlife Park

occlusions

light conditions

Dolphin Application

Wildlife Park

occlusions

light conditions

Dolphin Application

Wildlife Park

occlusions

light conditions

Bird Application

• Recording of birds in Emlen

funnels

• Detect take-off times

• Estimate take-off directions

Bird Application

funnels

Bird Application

funnels

Orienteering Application

• GPS trajectory

• Control position known

• Improve position estimate

• GPS trajectory

Target Tracking

Tracker

Presentation

Pre-processing Association TrackManagement

Estimation

Sensor Data

Target Model

Linear Gaussian state-space model

xk = Fkxk−1 +wk,

wk ∼ N (0,Qk),

yj = Hjxj + vj , vj ∼ N (0,Rj),

x0 ∼ N (x0,P0).

Target Model

xk = Fkxk−1 +wk, wk ∼ N (0,Qk),

x0 ∼ N (x0,P0).

Target Model

yj = Hjxj + vj ,

vj ∼ N (0,Rj),

x0 ∼ N (x0,P0).

Target Model

x0 ∼ N (x0,P0).

Target Model

x0 ∼ N (x0,P0).

Camera Model

• Camera extrinsics

• Camera intrinsics

• Projection

• Perspective compensation

• Lens distortion

Camera Model

• Projection

• Lens distortion

Camera Model

• Projection

• Lens distortion

Camera Model

• Projection

• Lens distortion

Camera Model

• Projection

• Lens distortion

Dolphin Camera z

Bird Camera z

• Targets constrained to

region

• Feasible predictions

• Similar behaviour

region

Turning Model

ω(x) = dr(x)

(βd + βa

(p⊥ · l(n)

))w(x, n) dn

• Nearly constant speed

• Influence by edges

• Avoid collision with edges

• Align with edges

• Preferred direction

• Polygon region

Turning Model

ω(x) = dr(x)

(βd + βa

(p⊥ · l(n)

))w(x, n) dn

• Polygon region

Turning Model

ω(x) = dr(x)

(βd + βa

(p⊥ · l(n)

))w(x, n) dn

w(x, n) =1

‖p− n‖2

• Polygon region

Turning Model

ω(x) = dr(x)

(βd + βa

(p⊥ · l(n)

))w(x, n) dn

• Polygon region

Turning Model

ω(x) = dr(x)

(βd + βa

(p⊥ · l(n)

))w(x, n) dn

• Polygon region

Turning Model

ω(x) = dr(x)

(βd + βa

(p⊥ · l(n)

))w(x, n) dn

Clockwise

Counterclockwise

• Polygon region

Turning Model

ω(x) = dr(x)

N∑i=1

(βd + βa(p⊥ · li)

)wi(x)

• Polygon region

Potential Field

Dolphin - Model Simulation

Dolphin - Model Comparisons

Detection regionConstraint regionConstant Velocity modelCoordinated Turn modelConstrained Motion model

Dolphin - Complete Framework

Uncertain Timestamp Model

• Traditional measurements

• Observations sampled at an

uncertain time

• Crime scene investigations

• Traces from animals

• Bottlenecks in mines

uncertain time

Consider a linear Gaussian state-space model,

yj = Hyjxj + vy

j , vyj ∼ N (0,Ry

x0 ∼ N (x0,P0),

Extend the model with

z = Hzxτ + vz, vz ∼ N (0,Rz), τ ∼ p(τ).

Consider a linear Gaussian state-space model,

yj = Hyjxj + vy

j , vyj ∼ N (0,Ry

x0 ∼ N (x0,P0),

z = Hzxτ + vz, vz ∼ N (0,Rz), τ ∼ p(τ).

Simple Uncertain Time Scenario

Consider the simple model,

xk = xk−1 + wk, wk ∼ N (0, Q),

yj = xj + vyj , vyj ∼ N (0, Ry)

for k ∈ {1, . . . , N} and two measurements y1 and yN .

z = xτ + vz, vz ∼ N (0, Rz), τ ∼ p(τ).

Consider the simple model,

xk = xk−1 + wk, wk ∼ N (0, Q),

yj = xj + vyj , vyj ∼ N (0, Ry)

for k ∈ {1, . . . , N} and two measurements y1 and yN .

z = xτ + vz, vz ∼ N (0, Rz), τ ∼ p(τ).

0 0.2 0.4 0.6 0.8 1

Positio

z (first case)z (second case)

The measurements are

• y1 = 0,

• yN = 1.

The two cases of

observations are

• z = 0.5 with flat prior.

• z = 1.5 with non-flatprior.

0 0.2 0.4 0.6 0.8 1

Positio

z (first case)z (second case)

The measurements are

• y1 = 0,

• yN = 1.

The two cases of

observations are

• z = 0.5 with flat prior.

• z = 1.5 with non-flatprior.

Posterior Distributions - Time

The posterior distribution of the uncertain time is

wτ , p(τ |Y, z) ∝ p(τ)N (z| zτ , Sτ ).

Posterior Distributions - Time

p(τ |Y, z)maxX p(X , τ |Y, z)p(τ)

0 0.5 1

babili

×10-3

0 0.2 0.4 0.6 0.8 1

×10-3

Posterior Distributions - States

The posterior distribution of the state is

p(xk|Y, z) =

N∑τ=1

wτ · N (xk|xτk,P

Posterior Distributions - States

0 0.2 0.4 0.6 0.8 1

Estimators - Minimum Mean Squared Error

XMMSE |Y, z

XMMSE |Y

0 0.5 1

Positio

0 0.5 1

Positio

Orienteering

• Adjusted trajectory

• Single control point

Orienteering

• Adjusted trajectory

• Single control point

Mode Observations

k− 1 k k+ 1

Mode Observations

k− 1 k k+ 1

• Jump Markov model

• Model target behaviour

• Probability of switching

• Direct mode observation

Mode Observations

k− 1 k k+ 1

Mode Observations

k− 1 k k+ 1

Mode Observations

k− 1 k k+ 1

Jump Markov ModelThe jump Markov linear model is

xk = Fk(δk)xk−1 +wk, wk ∼ N (0, Qk(δk)),

yj = Hj(δj)xj + vj , vj ∼ N (0, Rj(δj)).

The mode is modelled as

p(δk | δk−1) = Πδk,δk−1

k , δk ∈ S.

Augmented is the direct mode observation,

zk ∼ p(zk | δk).

k , δk ∈ S.

zk ∼ p(zk | δk).

k , δk ∈ S.

zk ∼ p(zk | δk).

k , δk ∈ S.

zk ∼ p(zk | δk).

Posterior Distribution - Mode Sequence

The posterior probability of a mode sequence {δij}kj=1 is

computed recursively by

wik ∝ wi

k−1 ·Πδik,δ

ik−1

k · N (yk | yik, S

ik) · p(zk | δik).

Posterior Distribution - Mode Sequence

The posterior probability of a mode sequence {δij}kj=1 is

computed recursively by

wik ∝ wi

k−1 ·Πδik,δ

ik−1

k · N (yk | yik, S

ik) · p(zk | δik).

Posterior Distribution - State

The posterior distribution of the state is given by

p(xk | {yj}kj=1, {zj}kj=1) =

|S|k∑i=1

wik · N (xk | xi

k, Pik).

Bird - ModelThe modes are S = {s, f}.

The state variables are x =

The state-space model is

xk = xk−1 +wk, wk ∼ N (0, Q(δk)),

(h(pk)

)+ vk, vk ∼ N (0, R),

xk = xk−1 +wk, wk ∼ N (0, Q(δk)),

(h(pk)

)+ vk, vk ∼ N (0, R),

xk = xk−1 +wk, wk ∼ N (0, Q(δk)),

(h(pk)

)+ vk, vk ∼ N (0, R),

Bird - Model ExtensionThe direct mode observation is the radial position

zk =√x2k + y2k,

where p(zk|δk) is given by:

Radius(mm)

0 100 200 300

babili

ensity

Bird - Model ExtensionThe direct mode observation is the radial position

zk =√x2k + y2k,

where p(zk|δk) is given by:

Radius(mm)

0 100 200 300

babili

ensity

Bird - Results

Time(s)

0 1000 2000 3000

Mode P

robabili

Cage 1

Time(s)

0 1000 2000 3000

Mode P

robabili

Cage 4

Time(s)

0 1000 2000 3000

Mode P

robabili

Cage 2

Time(s)

0 1000 2000 3000

Mode P

robabili

Cage 3

• Estimated modes

• Extracted takeoffs

• Takeoff directions

• Matched takeoffs

Bird - Results

Time(s)

0 1000 2000 3000

Cage 1

Time(s)

0 1000 2000 3000

Cage 4

Time(s)

0 1000 2000 3000

Cage 2

Time(s)

0 1000 2000 3000

Cage 3

• Estimated modes

Bird - Results

Cage 1 Cage 4

Cage 2 Cage 3

• Estimated modes

Bird - Results

Cage 1 Cage 4

Cage 2 Cage 3

• Estimated modes

Bird - Complete Framework M1

Conclusions

Theory is presented on

• a constrained motion model

• an uncertain timestamp model

• mode observations

Theory is demonstrated to work in applications

Conclusions

Questions?www.liu.se

Tracking of Animals Using Airborne...

Documents

Transcript of Tracking of Animals Using Airborne...