Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision...
Transcript of Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision...
![Page 1: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/1.jpg)
1
Associating People Dropping
off and Picking up Objects
Dima Damen, David HoggComputer Vision Group – University of Leeds
BMVC07
![Page 2: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/2.jpg)
2/25BMVC 07
The Task
![Page 3: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/3.jpg)
3/25BMVC 07
Tracking people and detecting objects
The video is a sequence of periods of activity
![Page 4: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/4.jpg)
4/25BMVC 07
Tracking people and detecting objects
The video is a sequence of periods of activity
![Page 5: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/5.jpg)
5/25BMVC 07
Associating drops with picks
![Page 6: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/6.jpg)
6/25BMVC 07
Discriminating drops from picks - people
![Page 7: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/7.jpg)
7/25BMVC 07
Discriminating drops from picks - people
![Page 8: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/8.jpg)
8/25BMVC 07
Discriminating drops from picks - objects
Masked edges
‘before’ reference image
‘after’ reference image
![Page 9: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/9.jpg)
9/25BMVC 07
Possible assignments for a period of activity
Assignments set 1:
drop(Person1, Bike1)
none(Person2)
drop(Person3, Bike1)
drop(Person4, Bike1)
none(Person5)
drop(Person6, Bike2)
Person
1
2
3
4
5
6
Bike
1
2
![Page 10: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/10.jpg)
10/25BMVC 07
Possible assignments for a period of activity
Assignments set 1:
drop(Person1, Bike1)
none(Person2)
drop(Person3, Bike1)
drop(Person4, Bike1)
none(Person5)
drop(Person6, Bike2)
Person
1
2
3
4
5
6
Bike
1
2
![Page 11: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/11.jpg)
11/25BMVC 07
Tree of hypotheses: sequences of assignments
Period of activity
1
2
![Page 12: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/12.jpg)
12/25BMVC 07
Tree of hypotheses: sequences of assignments
Period of activity
1
2
![Page 13: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/13.jpg)
13/25BMVC 07
Tree of hypotheses: sequences of assignments
Period of activity
1
2
![Page 14: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/14.jpg)
14/25BMVC 07
Tree of hypotheses: sequences of assignments
Period of activity
1
2
3
![Page 15: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/15.jpg)
15/25BMVC 07
Tree of hypotheses: sequences of assignments
Period of activity
1
2
3
K-best – k-min-cost
![Page 16: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/16.jpg)
16/25BMVC 07
Tree of hypotheses: sequences of assignments
Period of activity
1
2
3
K-best – k-min-cost
![Page 17: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/17.jpg)
17/25BMVC 07
Tree of hypotheses: sequences of assignments
Period of activity
1
2
3
K-best – k-min-cost
![Page 18: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/18.jpg)
18/25BMVC 07
Tree of hypotheses: sequences of assignments
![Page 19: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/19.jpg)
19/25BMVC 07
Tree of hypotheses: sequences of assignments
![Page 20: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/20.jpg)
20/25BMVC 07
Constrained optimisation
![Page 21: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/21.jpg)
21/25BMVC 07
Post-segmentation
![Page 22: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/22.jpg)
22/25BMVC 07
Experiments & Results
� 3 experiments
� 1 hour (45 events)
� 50 minutes (22 events)
� Full day (9 hours and 30mins) (40 events)
% of correct connections
83.59
70.37
75.86
Unconstrained
96.093
92.592
93.101
ConstrainedExp #
![Page 23: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/23.jpg)
23/25BMVC 07
Application to bicycle theft detection
� 8×8×8 scale-normalized equal-bin-size colour
histogram
� Scale-by-max
� Median histogram
Predicted
18317Non-Thief
310Thief
Non-ThiefThiefActual
![Page 24: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/24.jpg)
24/25BMVC 07
Summary
� Deal with ambiguity in the visual data through the use of global constraints on what is possible.
� Comparison with unconstrained and partially-constrained solutions (in the paper).
� Ambiguities in the observations are expressed as multiple hypotheses.
� Hypotheses can then be verified or invalidated by future observations.
![Page 25: Associating People Dropping off and Picking up Objects · Dima Damen, David Hogg Computer Vision Group – University of Leeds BMVC07. BMVC 07 2/25 The Task. BMVC 07 3/25 Tracking](https://reader030.fdocuments.net/reader030/viewer/2022041208/5d6655ed88c993493f8b48b7/html5/thumbnails/25.jpg)
25/25BMVC 07
Thank you for listening
Contact Info:Dima DamenComputer Vision Group – University of [email protected]