1

Generic Image Structures in Integrated Media

Nick Rossiter & Michael Heather, University of Northumbria at

Newcastlenick.rossiter@unn.ac.uk

2

Two main direction in Image Research

• Image contents:– modelled as set of attributes– at fairly high level of abstraction– but with little scope for free or ad hoc queries

• Feature extraction/object recognition subsystems:– automated object recognition– but difficult methods, computationally expensive and

domain specific

3

Problems with Current Directions

• General models (e.g. databases) not used

• Concentration on customised methods

• Interoperability is difficult as it requires– more general techniques – meta and metameta data

• Importance of text not always recognised

4

Theoretical Needs

• Emphasis on powerobjects rather than atomic objects with flexible searching on clusters and groups

• Construction of universal relations for new connections intra-schema (local universe) and inter-schema (global universe) - i.e. for integration of images with text across heterogeneous databases

• Joining of type/domain attributes for different image representations (pixel, graph, Postscript)

5

Set Theoretic Approaches

• For example object-oriented methods– problems in unifying these

• Recent developments for universal description– MOF (Meta-Object Facility)– RDF (Resource Description Framework)– promising but aimed more at business data

• In general lack universal type system

6

Category Theory and Types

• Categories provide a theory of types.

• Typing is an inherent feature of every image recognition with two basic categories of data, the source and the medium– e.g. for an old master the source will be a human

painter, whereas the medium may well be a painting in oils which will again import certain characteristics to the image and be specifiable in the retrieval process

7

Characteristic Features of Image Data

• Differentiating between background and foreground e.g. weather forecasting

• Use of colour for differentiation e.g. sunsets• Texture e.g. human faces (qualia)

– periodicity– directionality– randomness.

• Semantic interrogation - less work

8

S l

s x m s

S XIMG M W/IMG

rs x m

m

MFigure 1: Pullback of types source m along image s.

S = category Source, M = category Medium, W/IMG = subcategory of W containing components of image, S XIMG M = product of category S and M over IMG

9

l

s x m S s l

s x m s (s)

-1 S XIMG M W/IMG r

s x m m

s x m

*m (m)-1

M

Figure 2: Pullback showing fuller collection of arrows

10

arrow selects of from comments W given S source analysis

(s)-1 S given W source construction

m W given M medium analysis

(m)-1 M given W medium constructionl

s x m S given S X M source nature

s x m M given S X M image qualia S X M given W real-world image query

s S X M given S image creativity*

m S X M given M medium typel

s x m S some S X M source collectionr

s x m M some S X M medium collection W some S X M component collection e.g. pixels W all S X M component combinations

Figure 3: Nature of each pullback arrow of Figure 2

11

Conclusions 1

• Full complexity of image recognition is shown:– Figure 2 (pullback with fuller collection of arrows)

– Figure 3 (table showing nature of each pullback arrow)

• Binary relation between two categories seems adequate for representing image relationships

• The highest type of arrow, natural transformation, can represent characteristics like creativity (s) or image quality (s x m)

12

Conclusions 2

• Open systems are exposed to real-world complexity

• Category theory facilitates the integration of different models – universal type system– multi-level meta information – assists workers who use a selection of models

(hierarchical, object-oriented) to represent aspects such as texture