OrionImage Understanding for Object

Recognition

Monique Thonnat

INRIA Sophia Antipolis

2

Orion research project

A multi-disciplinary team at the frontier of computer vision, knowledge-based systems and software engineering

Research on Automatic Image Understanding Video Interpretation :

subway and bank monitoring, airport activity monitoring ...

Natural Object Recognition : galaxy classification, pollen categorization...

3

INRA/INRIA Cooperation

Main goals: Automatic recognition of biological organisms in

their natural environment In particular early rose disease diagnosis by

image understanding for Integrated Pest Management

Partners: INRIA Orion Team

Monique Thonnat and Celine Hudelot (PhD) INRA Sophia Antipolis, URIH

Paul Boissard

4

INRA/INRIA Cooperation

Why Image Understanding ? Plant disease diagnosis = visual observation which aims at

inferring disease presence by the observation of signs and symptoms

TO BE ABLE TO REASON : signs and symptoms interpretation in terms of diseases

TO BE ABLE TO SEE : Focusing on relevant criteria

Star shape network of white and thin filaments (5-10 μ)Presence of elliptical white blobs in the centre of the networkClimatic Context: High humidity, Temperature : 25 °C

Early powdery mildew infection in propitious conditions

Early diagnosis:Microscopic image (x64) of rose leaf part

5

Natural Object Recognition : main issues

Powdery mildew :State of infection : earlyVegetal support : red leaf

Powdery mildew :State of infection : very earlyVegetal support : green leaf

Two white flies close to their eggs

Need of domain knowledge

Intelligent management of image processing programs

Complexity and variability of object appearance

Variability of contexts

Scene knowledge and spatial reasoning

Multiple objectsand various object types

6

Natural Object Recognition : proposed solution

A platform with 3 dedicated tasks Semantic data interpretation

pathologist knowledge (domain taxonomy and terminology)

Ontological engineering to facilitate knowledge acquisition

Data management Matching between numerical image data and symbols Scene analysis using spatial reasoning

Image processing numerical object description program supervision techniques : to automate the

management of an image processing library

7

Input : User Request Fungi infection?

Image + Context

Variety : LeonidasLeaf : youngSeason: summerTemp: 24° CHumidity: 80...

Example

InterpretationDomain concept tree traversal to build visual object hypotheses

Leaf Scene

VegetalPart

Disease

Insects

Virus

Fungi

White fly

Penicillium

Powdery mildew

Dispersed

Clump

Aphid

Subpart

Subclass

AcaridVery Early

Pellet

1

Data Management Symbolic request to image processing

request

3

Goal: segmentationContraints:Image entity = ridge

Object.width = [1..3]Object.intensity > 150

Input Data: Image : input imageMask : area of interest

Image Processing Request

4

Image Processing:request processing

by program supervision techniques

5Image Data

Ridge 1

numerical descriptors

Ridge 3+ Numerical descriptors

Ridge 2+

NumericalDescriptors

6

InterpretationDomain concept tree traversal to build visual object hypotheses

Visual Object Hypothesis

2

Group of :Geometry: star shape network of { Geometry: line Thickness : thin width [7..10 m] very thin width [5..7 m] Straightness : almost straight Lightness: bright}Spatial Relation: Connected}

1

8

Example

Image Data

Ridge 1

numerical descriptors

Ridge 3+ Numerical descriptors

Ridge 2+

NumericalDescriptors

Data Management Visual object

hypothesis verification and instantiation

7

Interpretation : classificationMatching between visual object instances and domain concepts

9

Interpretation : diagnosis

Post classification rules activation

11

Visual Object InstanceNetwork of lines

Line 1

Line 2 Line 3

Line 5 Line 4

ECEC

ECEC

Line line1Thickness:=thin (0.8)Straightness:= straight (0.5)Lightness:=bright (0.7)Connected (line2)Connected (line4)+ link to image data

8

Diagnosis

Early powdery mildew infection on young leaf

12

Recognised domain concept

10

Freely dispersed mycelium

9

Conclusion

A generic platform for automatic recognition of natural objects

a formalism and an ontology for knowledge base building 3 dedicated reusable engines

semantic interpretation image/symbol matching and spatial reasoning management of a generic image processing library

Evaluation and validation (on going) with microscopic images of greenhouse rose leaf diseases

Future works machine learning for image/symbol matching (Nicolas Maillot)

and for image segmentation (Vincent Martin) biological long term objective : continuous disease monitoring in

greenhouse

10

Object Recognition Platform

Visual ConceptOntology

Image DataOntology

Visual Object Instances

Image Processing

Data Management

Data Management and Scene Analysis

Knowledge Base

Data ManagementEngine

Interpretation Application Domain

Knowledge BaseInterpretation 

Engine

Program Utilization Knowledge Base

Library of Image Processing Programs

Program Supervision

Engine

Visual Object

Hypotheses

Image Processing Requests

Image Numerical data

User Request Interpretation results