Video MiningLearning Patterns of Behaviour

via an Intelligent Image Analysis System

Introduction• All large archives of video are now available in

repositories, there is hidden much potentially useful knowledge

Latent, Useful, Interesting Information

Data mining Techniques

Large archives of

videos

23年 4月 19日 2Jang Hee Kyun

Introduction• Video mining is more challenging, due to lack of

explicit structure in the raw data in video archives• Ex) Surveillance, Analysis of expert’s activities

Studying animal behaviour, Emulate robot behaviour

• Each applications there are often patterns of activity

• They can be classified in order to gain more general insights into agent and object movements and behaviour

23年 4月 19日 3Jang Hee Kyun

Video Mining• Interesting in that the raw data in videos is

expressed in a way that is not directly amenable to the use of conventional mining techniques

• There is a lot of variation in the details of the patterns and it is important to abstract out the key features of a behaviour

• Unwanted or irrelevant details and noise can be filtered out

23年 4月 19日 4Jang Hee Kyun

Video Mining• Video mining of patterns of behaviour and their

inter-play, has to deal with very dynamic situations

• Techniques that have been developed for the extraction of temporal rules from collections of time series data

• We now want to identify patterns of data that are unusual, and discover inter-relationships between the patterns of different agents and objects

23年 4月 19日 5Jang Hee Kyun

Discover Rules

• First Stream• Detect ‘abnormal ‘ or ‘interesting ’ behaviour• The ability to learn what is ‘abnormal ’ or ‘ interesting ’ • Ultimate goal is that they will use only innate knowledge

• Second Stream• Summarisation for behavioural pattern detection/

matching in the second stream using AI (Artificial Intelligence) and DM (Data Mining) algorithms for time series analysis

23年 4月 19日 Jang Hee Kyun 6

Discover Rules

• It will be able to operate without the direct intervention of a user, and be able to control its own focus of attention to some extent

• This will in turn influence how it operates in related situations in the future

23年 4月 19日 Jang Hee Kyun 7

Background and approach• We use our own system, ModTrack, for vehicle

detection and tracking• “Independent Moving Object Detection Using a Colour

Background Model” by F. Campbell – West, P. Miller, H. Wang

• DM (Data Mining) aspects• Tracking system• Identify abnormal behaviour• Infer unusual pattern of activity

• AI (Artificial Intelligence) aspects• Learning how another agent learns and making use of

the results

23年 4月 19日 Jang Hee Kyun 8

Method of representation and analysis• Our objective is to reverse engineer what we

observe in the real world by using a vision or imaging system• We need to emulate the behaviour of the real world

actors

• How does a robot adjust its knowledge about the behaviour of a light using the adaptive learning paradigm?

23年 4月 19日 Jang Hee Kyun 9

Method of representation and analysis• We use the robot’s intention not only as a

consideration for our decision making, but also as a guide for our accumulation of observations

• We make a qualitative assessment by distinguishing suggestion and confirmation

23年 4月 19日 Jang Hee Kyun 10

Example

23年 4月 19日 Jang Hee Kyun 11

Method of representation and analysis1. As it detects sequences of such atomic

movements the system records them

2. Behaviour pattern detection• Classify these behavioural seuqences• Classifier is important requirement

3. Represent activities rule set• Behaviour matching and prediction

23年 4月 19日 Jang Hee Kyun 12

Conclusions and Summary

• ModTrack was used to obtain the behavioural traces of the robot/agent

• Using representation we can build a new representation of what is happening with the raw data

• We have shown how detailed behaviours from video can be coarsened and mined to obtain useful knowledge

23年 4月 19日 Jang Hee Kyun 13