ORION Project-teamMonique THONNAT

INRIA Sophia Antipolis

Creation: July 1995

Multidisciplinary team: artificial intelligence, software engineering, computer

vision

Evaluation May 2006

Orion 2

Team Presentation Research Directions Cognitive Vision 2002-2006 Reusable Systems 2002-2006 Objectives for the next Period

Contents

Evaluation May 2006

Orion 3

4 Research Scientists: François Bremond (CR1 Inria)

Sabine Moisan (CR1 Inria, HDR)

Annie Ressouche (CR1 Inria)

(team leader) Monique Thonnat (DR1 Inria)

1 External Collaborator: Jean-Paul Rigault (Prof. UNSA Inria secondment)

4 Temporary Engineers: Etienne Corvee, Ruihua Ma, Valery Valentin, Thinh Van Vu

7 PhD Students: Bui Binh, Bernard Boulay, Naoufel Kayati,

Le Thi Lan, Mohamed Becha Kaaniche,

Vincent Martin, Marcos Zuniga

Team presentation (May 2006)

Evaluation May 2006

Orion 4

Objective: Intelligent Reusable Systems for Cognitive

Vision

Cognitive Vision: Interpretation of static images Video understanding

Reusable Systems: Program Supervision LAMA Software platform

Research directions

Evaluation May 2006

Orion 5

Cognitive Vision: Image interpretation (ECVision European network on cognitive

vision, EUCognition) vs. computer vision (INRIA CogB)

Video understanding (USC Los Angeles, Georgia Tech. Atlanta, Univ. Central Florida, NUCK Taiwan, Univ. Kingston UK, INRIA Prima)

Reusable Systems: Program supervision: e.g., scheduling (ASPEN and CASPER at

JPL), image processing (Hermès at Univ. Caen, ExTI at IRIT)…

Platform approach: e.g., ontology management (Protegé at Stanford), frameworks for multi agents (Aglets, Jade, Oasis at LIP6), distributed object community (Oasis at INRIA Sophia)…

Orion team positioning

Evaluation May 2006

Orion 6

Objective: semantic interpretation of static 2D images

Recognition of object categories (versus individuals) Recognition of scenes involving several objects with spatial reasoning Intelligent management of image processing programs

Towards a cognitive vision platform

Cognitive Vision : Image Interpretation 2002-2006

Evaluation May 2006

Orion 7

Scientific achievements:

Knowledge acquisition: A visual concept ontology with 144 spatial, color and

texture concepts [MVA04]

Learning: Visual concept detectors [IVC06]

Image segmentation parameters [ICVSa06]

Cognitive vision platform Architecture [ICVS03]

Object class recognition algorithm [CIVR05]

Cognitive Vision : Image Interpretation 2002-2006

Evaluation May 2006

Orion 8

Self Assessment:

Strong points: Visual concept ontology as user-friendly intermediate layer

between image processing and application domain

Automatic building of the visual concept detectors

Still open issues: Learning for image segmentation

Temporal visual concept ontology

Cognitive Vision: Image Interpretation 2002-2006

Evaluation May 2006

Orion 9

Objective: Real time recognition of interesting behaviors

How? Data captured by video surveillance cameras

Original video understanding approach mixing: computer vision: 4D analysis (3D + temporal analysis)

artificial intelligence: a priori knowledge (scenario, environment)

software engineering: reusable VSIP platform

Cognitive Vision: Video Understanding 2002-2006

Evaluation May 2006

Orion 10

Cognitive Vision: Video Understanding 2002-2006

SegmentationSegmentation ClassificationClassification TrackingTracking Scenario RecognitionScenario Recognition

Alarms

access to forbidden

area

3D scene modelScenario models A priori Knowledge

Objective: Interpretation of videos from pixels to alarms

Evaluation May 2006

Orion 11

Scientific achievements:

Multi-sensor video understanding: 2 to 4 video cameras overlapping or not [IDSS03,JASP05]

Video cameras + optical cells + contact sensors [AVSS05]…

Learning: parameter tuning[MVAa06]

frequent temporal scenarios models [ICVSb06]

Temporal scenario: a new real time recognition algorithm [IJCAI03,ICVS03]

a new representation language [MVAb06,ECAI02,KES02]

Cognitive Vision: Video Understanding 2002-2006

Evaluation May 2006

Orion 12

Industrial impact:

Strong impact in visual surveillance (metro station, bank

agency, building access control, onboard train, airport) 4 European projects (ADVISOR, AVITRACK, SERKET, CARETAKER)

5 industrial contracts with RATP, ALSTOM, SNCF, Credit Agricole,

STMicroelectronics

2 transfer activities with BULL (Paris), VIGITEC (Brussels)

Creation of a start-up Keeneo July 2005 (8 persons) for

industrialization and exploitation of VSIP library.

Cognitive Vision: Video Understanding 2002-2006

Evaluation May 2006

Orion 13

Cognitive Vision: Video Understanding 2002-2006

Intelligent video surveillance of Bank agencies

Toulouse - 3rd June 2004

Orion14

“Unloading Global Operation”

Cognitive Vision: Video Understanding 2002-2006

Toulouse - 3rd June 2004

Orion15

Airport Apron Monitoring “Unloading Operation” European AVITRACK project

Cognitive Vision: Video Understanding 2002-2006

Evaluation May 2006

Orion 16

Self Assessment:

Strong points: Video understanding approach: real time, effective

techniques used by external academic and industrial teams

Launch of an evaluation competition for video surveillance

algorithms (ETISEO) with currently 25 international teams

Still open issues: Learning

Multi sensor

Cognitive Vision:Video Understanding 2002-2006

Evaluation May 2006

Orion 17

Reusable Systems: original approach for the reuse

of programs with program supervision techniques

Program supervision: Automate the (re)configuration and execution of programs selection, scheduling, execution, and control of results

Knowledge-based approach: knowledge modeling, planning techniques, …..

Reusable Systems: Program Supervision

Evaluation May 2006

Orion 18

Reusable Systems: LAMA Platform

Reusable Systems: Reuse of tools to design knowledge-

based systems (KBS)

LAMA Software Platform:

Set of toolkits to facilitate design and evolution of KBS elements:

engines, GUI, knowledge languages, learning and verification facilities…

Software Engineering approach: genericity, frameworks, objects and components

LAMA

ProblemSolving

KBS

providegeneric

components and tools

raise newissues, to beabstracted into new

components

Virtuous Circle

Evaluation May 2006

Orion 19

Reusable Systems: LAMA Platform

Task dedicated Engine

KnowledgeBase

KBS User

Expert

Task dedicated GUI

Task dedicated Languagewith compiler

& KB verification

Blo

cks

Java graphic library for GUIs

Verification library for knowledge bases

Compilers/verifiers generatorsfor knowledge

description languages

Framework for engine design &

knowledge representation

support and task specific layers

LAMA Designer

Program Supervision

Object Recognition

ModelCalibration

Evaluation May 2006

Orion 20

Reusable Systems: Program Supervision 2002-2006

Scientific achievements: Improvement of the Pegase engine (Pegase+)

Multithreading, extensions to the YAKL language [ECAI02] Distributed program supervision

Supervision Web server, multi-agent techniques, interoperability Pegase/Java/agents [TC06]

Cooperation with image and video understanding Object recognition task using program supervision

[ICTAI03] Interoperability with VSIP: program supervision for video

understanding [ICVSc06]

Evaluation May 2006

Orion 21

Reusable Systems: LAMA Platform 2002-2006

Scientific achievements: Enforcing LAMA safe usage

Verification of LAMA component extensions relying on Model Checking approach [Informatica01, SEFM04]

Encompassing new tasks Classification and object recognition in images: new

engine and new knowledge representation language [ICTAI03]

Model calibration in hydraulics: new engine/language (PhD co-directed with INPT and CEMAGREF) [KES03, JH05]

Evaluation May 2006

Orion 22

Reusable Systems: Self Assessment

Strong points: Real time performance (Pegase+ and video)

Using program supervision costs less than 5% of overall processing time

LAMA genericity at work Different tasks (supervision, classification, calibration) in

various application domains (hydraulics, biology, astronomy, video surveillance…)

Shorter development time and safer code Reuse of concepts as well as code

Several variants of a task sharing common concepts Extensibility and commitment to Standards

Evaluation May 2006

Orion 23

Creation of a new INRIA project-team PULSAR

Perception Understanding and Learning Systems for Activity

Recognition

Theme:

CogC Multimedia data: interpretation and man-machine interaction

Multidisciplinary team: artificial intelligence, software engineering, computer vision

Objective: Research on Cognitive Systems for Activity Recognition

Focus on spatiotemporal activities of physical objects

From sensor output to high level interpretation

Objectives for the next period 1/5

Evaluation May 2006

Orion 24

PULSAR Scientific objectives:

Two research axes: Scene Understanding for Activity Recognition

Generic Components for Activity Recognition

PULSAR Applications: Safety/security (e.g. intelligent surveillance)

Healthcare (e.g. assistance to the elderly)

Objectives for the next period 2/5

Evaluation May 2006

Orion 25

PULSAR: Scene Understanding for Activity

Recognition Perception: multi-sensors, finer descriptors

Understanding: uncertainty, 4D coherency,

ontology for AR

Learning: parameter setting, event detector,

activity models, program supervision KB (risky

objective)

Objectives for the next period 3/5

Evaluation May 2006

Orion 26

PULSAR Generic Components for Activity

RecognitionFrom LAMA Platform to AR platform: Model extensions:

modeling time and scenarios handling uncertainty

User-friendliness and safeness of use: theory and tools for component frameworks scalability of verification methods

Architecture improvement: parallelization, distribution, concurrence real time response domain specific software and graphical interface plugging

Objectives for the next period 4/5

Evaluation May 2006

Orion 27

Short term objectives:

Scene Understanding for Activity Recognition Perception: gesture analysis Understanding:

ontology-based activity recognition uncertainty management

Learning: primitive event detectors learning

Generic Components for Activity Recognition Model of time and scenarios Internal concurrency and distributed architecture

Objectives for the next period 5/5