Video Analysis in Autonomous Systems: Data Analytics Challenges

School of somethingFACULTY OF OTHERSchool of ComputingFACULTY OF ENGINEERING

Video Analysis in Autonomous Systems: Data Analytics Challenges

Krishna Dubba

Institute for Artificial Intelligence and Biological Systems

*

School of ComputingFACULTY OF ENGINEERING

Leeds Activity Analysis Group

Computer Vision (Prof. David Hogg)

Knowledge Representation and Reasoning (Prof. Tony Cohn)

*


Motivation:

“We are drowning in data yet starving for knowledge” ~ John Naisbitt

*


Motivation:● Are computers drowning in (video) data?

○ CCTV cameras○ Personal digital video cameras○ Video content on TV and Internet○ In future: Google glass, autonomous cars, personal

robots

*


TrixiUniversity of Hamburg

LUCIELeeds University Cognitive Intelligent Entity

*


Motivation:● Are computers starving for knowledge?

*


Motivation:● Applications:

○ Security and Surveillance○ Intelligent autonomous systems (robots, cars etc.)○ Content based video retrieval (instead of text tags)○ Automatic script and commentary generation for videos

*


Nature of Data:● Images ● Each pixel in image is a tuple (R,G,B)

*


Nature of Data:● Videos (series of images)

*



Third Person View

*



Third Person View Ego-Centric View

*


Nature of Data● Sensor data such as laser, depth data etc (Kinect).

*


Nature of Data:

● Text (annotations, additional information from web)

● Verbal instructions

*


Challenges:● Supervised, unsupervised and semi-supervised learning

*


Challenges:● Supervised, unsupervised and semi-supervised learning● Data comes from multiple sources and mainly aimed at

humans - Multidisciplinary approach

*



humans - Multidisciplinary approach● Real time analysis: GPU processing

○ LUCIE has three kinects attached and needs a separate computer for each kinect.

*





● Integrating low-level representation and high level reasoning: Statistical Relational Models like Markov Logic Networks

*





● Integrating low-level representation and high level reasoning: Statistical Relational Models like Markov Logic Networks

● Online learning and how learning affects the state of the system.

*

Thank You


Video Analysis in Autonomous Systems: Data Analytics Challenges

Data & Analytics

Transcript of Video Analysis in Autonomous Systems: Data Analytics Challenges