24/10/02AutoArch Overview Computer Vision and Media Group: Selected Previous Work David Gibson,...

24/10/02 AutoArch Overview

Computer Vision and Media Group:

Selected Previous Work

David Gibson, Neill Campbell

Colin Dalton

Department of Computer Science

University of Bristol


Duck: The AutomaticGeneration of 3D Models

• Generating 3D computer models is difficult

• Put object on turntable

• Take 8 pictures of it from different angles

• Crank the handle…

• No skilled user or expensive equipment

• Make avatars by spinning person on chair


Cog and Stepper

• Automatically inject ‘life’ into computer animations

• 3D swathe through 4D space time

• Where space is 3D computer model

• Or just to make things look strange!


Casablanca: Motion Ripper

• Computer animation driven by film

• Animator labels a small number of points

• System then tracks these points over all frames

• Motions are extracted and used to drive animation


Laughing ManMotion Ripper Part 2

• Automatic video creation

• Points are marked and tracked

• System learns the motions

• System generates new motions which are different but ‘correct’

• Forever!


AutoArch: The Automatic Archiving of Wildlife Film

Footage

David Gibson, Neill Campbell

David Tweed, Sarah Porter

Department of Computer Science

University of Bristol


Motivation

• BBC Natural History Unit

• Manual archiving/meta data generation

• Reuse problematic– Inefficient/time consuming– Expensive– Limited access

• Obvious need to automate


Objectives

• Generate efficient visual representations– Video segmentation– Visual browsing/summarisation– Visual searching

• Generate as much meta data automatically– Camera motions/effects– Scene structure– Scene content


System Overview

ShotSegmentation

VisualSummarisation

MotionAnalysis

Colour/TextureAnalysis

Meta data extraction algorithms

Catalogue Entry

Visualisation based algorithms

Visualisation and Searching


Video Segmentation


Visual Summarisation

• Key frame extraction


Visual Summarisation Tree

Ent

ire

shot

Level of detail


Visual Searching

• Layered 2D representation

of high D clip space


Motion Analysis using point tracking

•Camera Motion Estimation•Event/Area of Interest Detection•Gait Analysis•Foreground/Background Separation•Combine with Colour and Texture for Classification•See cheetah track avi


Camera Pan

BCD0111.09_0085.epslines = 47, curls = 98, shorts = 5long lines = 47, mode = 95.00, mean = 95.21, std = 4.15zoom centre = (603.01, 63.65), val = -0.2356zoom residual per line = 22.92zoom residual #2 per line = 28.92Average line vector: 109.94 -8.27

pan/tilt angle: 94.30, vector: (109.94 -8.27)pan/tilt residual per line = 21.67pan/tilt residual #2 per line = 33.38percentage of lines within 5% of mode: 89.36


Camera Zoom

BCD0113.15_0067.epslines = 142, curls = 1, shorts = 7long lines = 134, mode = 340.00, mean = 227.24, std = 128.76

zoom centre = (182.97, 55.52), val = 0.2063zoom residual per line = 4.86zoom residual #2 per line = 6.90Average line vector: -3.81 17.28pan/tilt angle: 347.57, vector: (-3.81 17.28)pan/tilt residual per line = 13.85pan/tilt residual #2 per line = 16.13percentage of lines within 5% of mode: 17.16


Tracking Failure

This could be an interestingevent in its self: flocking,herding, close up of lots ofactivity, shot grouping, etc.


Event/Area of InterestDetection


Frequency Analysis:Gait Detection

FFT

After trajectory segmentation


Foreground/BackgroundExtraction

Feature space #1

Feat

ure

spac

e #2

Foregroundmodel

Backgroundmodel

Which pixelsare foreground?


Animal Identification

Give models a name:

= cheetah

= elephant

= zebra

= lion


Some Problems

• Noise in images

• Noise in measurements

• Camouflage

• Occlusion

• Answer: Need higher level models

• See next few slides


Model Based Tracking


Lion Tracking

• Synchronise horse model with lion points• Move and deform horse model to lion points• See avi• To do: Improve spatial deformation, especially for

legs, using colour and texture


Multiple Object Tracking


Conclusions

• Visualisation is very powerful

• Combined with text is even better!

• Assists searching and communication

• Lots of meta data can be auto generated

• Assists archiving

• Help to prioritise manual archiving

• Can be applied to any visual media

24/10/02AutoArch Overview Computer Vision and Media Group: Selected Previous Work David Gibson,...

Documents

Transcript of 24/10/02AutoArch Overview Computer Vision and Media Group: Selected Previous Work David Gibson,...