Multimedia Systems and Communication Research

Department of Electrical and Computer Engineering Multimedia Systems Lab

University of Illinois at ChicagoChicago, Illinois, USA

Ashfaq Khokhar

Major Related Research ThrustsMultimedia Representation, Analysis, Communication,

and Manipulation (Ansari, Schonfled and Khokhar) Content based Indexing and Retrieval Classification of Spatio-Tempral Image and Video Events Motion Tracking Digital Right Management Parallel Implementations on GPU and multicore processors

Heterogeneous Sensor Networks (Ansari, Zefran, and Khokhar) Approximate Spatio-Temporal Query Processing, Information Fusion, and Triggers Motion Control Algorithms Cross Layer Power Efficient Routing Solutions

2

3

Multimedia Representation, Analysis, Communication, and Manipulation Event/object retrieval and classification from

video databases is an extremely challenging problem.

Query and/or stored video data undergo transformation due to camera or object motion (e.g. affine mapping).

Query and/or stored video data contain partial information (e.g. due to video occlusions).

4

Our Worko Scalable content based indexing and retrieval

system for video events, images, and audio clips.

o Classification of motion events, facial expressions, gestures

o Tracking of multiple moving objectso Localized Null Spaceo Kernel Particle Filterso Hierarchical Distributed Indexing Structureso Distributed Hidden Markov Models

Proposed Localized Null Space

Zero elements

Zero elements

N-3

N

Traditional Null Space

5

Structure of Localized Null Space

Illustration of the structure of the traditional Null Space and the proposed Localized Null Space.

Zero elements

Zero elements

3 Non-Zero elements

N-3

Zero elements

Zero elements

K-3Non-Zero elements for W1

K

N-K-3Non-Zero elements for W2

3

Zero elements

Zero elements

N-K

6

Benefits of LNS

)ˆ(,,

ixiuiu rf Can be viewed as consisting of multiple

subspace, therefore can be dynamically split for retrieval of partial queries.

Can be used to merge multiple Null Spaces into an integrated Null Space.

Has the same complexity as the traditional null space.

7

Trajectory and part of the rotated trajectory with identical localized null space representations.

LNS Example

8

Application of LNS in Face Recognition

24 different poses used for each face from the UMIST database.

9

Application of LNS in Face Recognition

Visual illustration of classification accuracy based on Localized Null Space Invariants when the query image is missing vertical or horizontal sections.

Multi-foveation videos

Pixel foveation

DCT foveation

Cyclic Motion Tracking

(Click to play)

Full body, Background clutter Occlusion

Heterogeneous Sensor Networks

Joint work with Northwestern Univ.

12

Proposed SolutionHierarchical Novel Scalable

AbstractionsHybrid StructureRank Order Filters for Value Field

AbstractionMulti-resolution Binary Maps for

Sensor Location Abstraction

Sensor Networks: In-network Hybrid Query Processing

Example Query: Retrieve all the prairie regions in DuPage county that are near river and have between 15% and 45% of salinity decline.

Solution RequirementsLess CommunicationLess Maintenance CostLess StorageLess Query LatencyMore Accurate Results

Existing distributed solutions are incapable of handling value and location queries with equal efficiency!

Our Solution: Novel Hierarchical Abstractions

9 6 5 12 3 18 3 17Sensed Values

Sensor node

9 6 5 12 3 18 3 17Sensed Values

Sensor node

Local cluster head

9 6 5 12 3 18 3 17

9 6 5 12 3 18 3 17Gathering data

Sensed Values

Sensor node

Local cluster head

9 6 5 12 3 18 3 17

3 3 5 6 9 12 17 18

9 6 5 12 3 18 3 17Gathering data

Sorting gathered data

Sensed Values

Sensor node

Local cluster head

9 6 5 12 3 18 3 17

3 5 12 18

3 3 5 6 9 12 17 18

9 6 5 12 3 18 3 17Gathering data

Sorting gathered data

Regular sampling

Sensed Values

Sensor node

Local cluster head

3 5 12 18 4 9 21 25Data sample

Intermediate Level i+1 node

Intermediate Level i nodes

|..|..|..|

3 5 12 18 4 9 21 25

|..|..|..|CompressionData sample

Intermediate Level i+1 node

Intermediate Level i nodes

|..|..|..|

3 5 12 18 4 9 21 25

|..|..|..|

|..|..|..||..|..|..|

CompressionData sample

Gather samples

Intermediate Level i+1 node

Intermediate Level i nodes

|..|..|..|

3 5 12 18 4 9 21 25

|..|..|..|

|..|..|..||..|..|..|

3 5 12 18 4 9 21 25

CompressionData sample

Decompression

Gather samples

Intermediate Level i+1 node

Intermediate Level i nodes

|..|..|..|

3 5 12 18 4 9 21 25

|..|..|..|

|..|..|..||..|..|..|

3 5 12 18 4 9 21 25

3 4 5 9 12 18 21 25

CompressionData sample

Decompression

Gather samples

Merge samples

Intermediate Level i+1 node

Intermediate Level i nodes

|..|..|..|

3 5 12 18 4 9 21 25

|..|..|..|

|..|..|..||..|..|..|

3 5 12 18 4 9 21 25

3 4 5 9 12 18 21 25

3 5 18 25

CompressionData sample

Decompression

Gather samples

Merge samples

Regular samplingIntermediate

Level i+1 node

Intermediate Level i nodes

• Small and fixed size update messages across the hierarchical structure

• Immediate exact response for extreme values (minimum and maximum)

• Low latency, error bounded responses for range queries.

• Small and fixed size update messages across the hierarchical structure

• Fast response for coarse view queries• Low latency, energy efficient responses for

fine detailed queries

What Can be Done for Nokia

Parallel implementation of complete image processing pipeline on GPUs and multi-core platforms

Scalable solutions for recognition/classification, and content based indexing and retrieval of images, audio, and video events.Solutions will work under affine transformations

In network indexing and querying solutions for approximate query processing

16