CH 11 Multimedia IR: Models and Languages Presented by: Mohammed Al-Rashidi Supervised by: Dr....
-
date post
21-Dec-2015 -
Category
Documents
-
view
226 -
download
0
Transcript of CH 11 Multimedia IR: Models and Languages Presented by: Mohammed Al-Rashidi Supervised by: Dr....
CH 11 Multimedia IR: Models and Languages
Presented by: Mohammed Al-Rashidi
Supervised by:Dr. Mourad Ykhlef
Multimedia IR system
The architecture of a Multimedia IR system depends on two main factors The peculiar characteristics of multimedia data The kinds of operations to be performed on such
data
Multimedia IR system
Support variety of data Different kinds of media
Text, images (both still and moving), graphs, and sound Mix of structured and unstructured data
Metadata Semi-structured data
Data whose structure may not match, or only partially match, the structure prescribed by the data schema
The system must typically extract some features from the multimedia objects
Multimedia IR system
Data retrieval Exploiting data attributes and the content of
multimedia objects Basic steps
1. Query specification Fuzzy predicates (Find all images similar to a car) Content-based predicates (Find all objects containing an
apple) Object attributes (Find all red images) Structural predicates (Find all multimedia containing a
video clip).
Multimedia IR system
2. Query processing and optimization Query is parsed and compiled into an internal form
3. Query answer
4. Query iteration
Combine DBMS and IR technology DBMS: data modeling capabilities IR system: similarity-based query capabilities
Data modeling
Main tasks1. A data model should be defined by which the
user can specify the data to be stored into the system
Support conventional and multimedia data types Provide methods to analyze, retrieve, and query such
data
2. Provide a model for the internal representation of multimedia data
Object-oriented DBMS
Provide rich data model More suitable for modeling both multimedia data types and
their semantic relationships Class
Attributes +operations Inheritance
Drawback The performances of storage techniques, query
processing, and transaction management is not comparable to that of relational DBMSs
Highly non-standard
Object-relational DBMS
Extend the relational model Represent complex data types Maintain the performance and the simplicity of
relational DBMSs and related query languages Define abstract data types
Allows one to define ad hoc data types for multimedia data
Internal representation
Using attributes is not sufficient Feature
Information extracted from objects Multimedia object is represented as a set of
features Features can be assigned manually,
automatically, or using a hybrid approach
Internal representation
Values of some specific features are assigned to a object by comparing the object with some previously classified objects
Feature extraction cannot be precise A weight is usually assigned to each feature value
representing the uncertainty of assigning such a value to that feature 80% sure that a shape is a square
SQL3
Support extensible type system Provide constructs to define user-dependent abstr
act data types, in an object-oriented like manner Collection data types
Sets, multisets, and lists The elements of a collection must have compatibl
e types
MULTOS
MULTimedia Office Server Client/server Support filing and retrieval of multimedia objects
Each document is described by a logical structure, a layout structure, and a conceptual structure
Documents having similar conceptual structures are grouped into conceptual types
Document
Place Date Receiver+ Sender
AddressName
CityStreet Country
AddressName
CityStreet Country
Letter_body
spring component type
Conceptual structure of the type Generic_Letter
Complete conceptual structure of the type Business_Product_Letter
Document
Place Date Receiver+ Sender
AddressName
CityStreet Country
AddressName
CityStreet
Company_LogoImage
Country
Product_PresentationText
Product_DescriptionText
Product_CoseText
Signature
Letter_body
Query languages
Relational/object-oriented database system Exact match of the values of attributes
Multimedia IR system Similarity-based approach
Considers the structure and the content of the objects Content-based query
Retrieve multimedia objects depending on their globe content
Query languages
In designing a multimedia query language, three main aspects require attention How the user enters their request to the system Which conditions on multimedia objects can be
specified in the user request How uncertainty, proximity, and weights impact
the design of the query language
Request specification
Interfaces Browsing and navigation Specifying the conditions the objects of interest
must satisfy, by means of queries Queries can be specified in two different
ways Using a specific query language Query by example
Using actual data (object example)
Search Using Sketch in QBIC
Sketch entry
Results of search
Search by Color in QBIC
Color selection15% yellow and 13% blue
Results of search
Search by User Drawn Pattern in QBIC
Pattern
Results of search
Conditions on multimedia data
1. Attribute predicates Concern the attributes for which an exact value is
supplied for each object Exact-match retrieval
2. Structural predicates Concern the structure of multimedia objects Can be answered by metadata and information about
the database schema “Find all multimedia objects containing at least one
image and a video clip”
Conditions on multimedia data
3. Semantic predicates Concern the semantic content of the required data,
depending on the features that have been extracted and stored for each multimedia object
“Find all the red houses” Exact match cannot be applied
Thanks
Any questions