T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing ...
-
Upload
avice-miles -
Category
Documents
-
view
217 -
download
0
Transcript of T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing ...
![Page 1: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/1.jpg)
TOPIC 5: LEVEL 0
David L. Hall
![Page 2: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/2.jpg)
TOPIC OBJECTIVES
Introduce concepts of level 0 processing Identify categories of techniques for level 0
processing (e.g., signal and image processing) Extend the concept of traditional signal and
image processing and conditioning to meta-data generation
![Page 3: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/3.jpg)
LEVEL 0 PROCESSING
![Page 4: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/4.jpg)
COMMENTS ON LEVEL 0 PROCESSING
Level 0 processing concerns processing each source of data independently to obtain the most useful information possible; we want to “squeeze out” useful information
In general, input data to a fusion system may include scalar or vector data, signal data, image data, or textual information. Each of these classes of information may entail an entire range of potentially applicable techniques (e.g., signal processing, image processing, text-based processing)
It is beyond the scope of this lecture (and indeed this entire course) to address these subjects – however, we seek to make the student aware of the need to consider such processing
Finally, an emerging new area is the generation of semantic meta-data (e.g., automatic semantic labeling of images) as a new powerful process to improve data fusion.
![Page 5: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/5.jpg)
METASENSOR PROCESSING
METASENSOR PROCESSING
CA
TEG
OR
YFU
NC
TIO
N
TEC
HN
I QU
E
Sensor Validation
and Calibration
Parametric
Modeling
• Spectral analysis• Wavelet processing• Short-time Fourier transform• AR modeling• System I/D analysis
Entity
Detection
• Array processing - Synthetic aperture (SA) - Inverse SA• Moving target indicator (MTI) • Hypothesis testing• Pre-detection fusion• Distributed fusion
• Phase and gain calibration• Sensor data validation
Non-Parametric
Analysis
• Time-series analysis• Dynamical systems analysis• Image analysis• Neural networks• Trend analysis• Thresholding and integration• Energy detection
![Page 6: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/6.jpg)
LEVEL 0 PROCESSING BY SOURCE TYPE
Scalar orVectorSensor
Signalsensor
ImagingSensor
Text-based information
yt
Semantic processing
Signal processing
Image processing
Data Conditioning
Original Source Data plus transformed & added meta-data
![Page 7: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/7.jpg)
SOURCE TYPE: SCALAR & VECTOR
DATA Representative techniques
Scaling methods and transformations Coordinate transformations and rotations Smoothing, filtering, averaging Thresholding Feature extraction and representations
y y + z Transformation
Original data Original data Alternate representations
![Page 8: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/8.jpg)
SOURCE TYPE: SIGNAL DATA Representative Parametric
Modeling Techniques Spectral analysis Wavelet processing Short-time Fourier transform Auto-Regression modeling System Identification analysis
Representative Non-Parametric Modeling Techniques
Time-series analysis Dynamical systems analysis Neural networks Trend analysis Threshold & integration Energy detection
Transform + characterization vector, y
Original signalTransformed signal
![Page 9: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/9.jpg)
EXAMPLES OF SIGNAL PROCESSING
Signal processing and enhancement are standard on most radios & receivers High & low frequency filtering Boost of selected frequency ranges “Coloring” of signals
Active Noise Cancelation devices Active cancellation of background
noise (e.g., airline engine noise)
Extensive commercial tools are available such as MATLAB, Mathematica, etc
![Page 10: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/10.jpg)
SOURCE TYPE: IMAGE DATA Classification Feature Extraction Pattern Recognition Projection Multi-Scale Signal Analysis
Principal Components Analysis Independent Component Analysis Self-Organizing Maps Hidden Markov Models Neural Nets
+ Meta-Data
Original Image Transformed image
ImageProcessing
An enormous amount of methods exist to process and transform image data from the individual pixel level, to object level to complete image level. Commercial tools such as Photoshop Pro make many of these available to casual users
![Page 11: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/11.jpg)
EXAMPLES OF IMAGE PROCESSING
http://web.uct.ac.za/depts/physics/laser/hanbury/intro_ip.html
Contrast enhancement
Image sharpening using wavelets
http://www.phasespace.com.au/wavelet_ex.htm
Deconvolution
http://www.phasespace.com.au/wavelet_ex.htm
![Page 12: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/12.jpg)
SOURCE TYPE: TEXT DATA
Representative techniques Key word extraction Name disambiguation Link of semantics to parametric data (e.g. location) Syntactic processing Thesaurus processing Semantic distance calculations Links to other documents/
Textual input via human reports or web info.
Textual input via human reports or web info.
Meta Data - key words - lexicon data - identified parameters - links to related documents
+TextualProcessing
![Page 13: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/13.jpg)
CONCEPT OF SEMANTIC META DATA GENERATION
Signal & images represent a data level view of patterns, features & characteristics
In human pattern recognition, we identify labels to represent signals & data
New techniques are being developed to train computers to perform a similar function
![Page 14: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/14.jpg)
PROBLEM: HOW TO EFFECTIVELY QUERY ON A VERY LARGE COLLECTION OF SATELLITE IMAGERY FOR
SEMANTICALLY MEANINGFUL REGIONS ?
Use semantic categorization combined with CBIR on small “patches” of the images (Dr. James Wang)
![Page 15: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/15.jpg)
QUERY FORMULATION
Query “patch” – pertaining to some semantics, e.g. mountains
Satellite Image Database Ranked
ResultsPurpose ?
Geography - Find mountainous regions with snow-caps (low-level semantics).
Forestry – Find forests of a certain density, analyze deforestation (mid-level semantics).
Military – Find air-bases in certain regions of the world (high-level semantics).
![Page 16: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/16.jpg)
LIMITATIONS OF CURRENT APPROACHES
Difficulty in defining classes Continuum of variety Subjectivity
No major improvement in classification accuracy G. G. Wilkinson, “Results and Implications of a Study of Fifteen Years of
Satellite Image Classification Experiments,” IEEE Trans. On Geoscience and Remote Sensing, 2005.
Recognition of higher-order semantics More focus on spectral than spatial dimensions
Scalability of complex querying and browsing
Flexibility to handle diverse applications
![Page 17: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/17.jpg)
AUTOMATIC SEMANTIC CATEGORIZATION
![Page 18: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/18.jpg)
SUMMARY OF APPROACH
Practical implementation on large-scale archives
Flexibility to adapt to various applications of satellite imagery, e.g., Geography, Military, Metallurgy, Agriculture etc.
Exploiting spectral and spatial information using a generative model for semantic categorization (supervised learning)
Handling of untrained classes (supervised learning)
Using a scalable CBIR system for efficient querying and browsing (unsupervised)
![Page 19: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/19.jpg)
OVERVIEW OF THE SYSTEM
Classification Generative Classifier: Two-dimensional Multi-resolution Hidden Markov Models
(2-D MHMM) Handling untrained classes
Discriminative Classifier: Support Vector Machines Querying for fast retrieval
Integrated Region Matching (IRM) measure used in SIMPLIcity system
K trained 2D-MHMMs
40 x K training patches
Query patch
Patch Class
Random samples from trained and untrained classes
K Likelihood scores Trained
biased SVM
Database search using IRM
Query Results
![Page 20: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/20.jpg)
DETAILS OF THE ARCHITECTURE
Efficient database design
Pre-computation of classes using 2D-MHMM for faster response
Suitable user interface and visualization (Common CBIR issue)
![Page 21: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/21.jpg)
TRAINING PROCESS
![Page 22: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/22.jpg)
AUTOMATIC SEMANTIC CATEGORIZATION
![Page 23: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/23.jpg)
RETRIEVAL RESULTS
http://riemann.ist.psu.edu/image
![Page 24: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/24.jpg)
RETRIEVAL RESULTS
http://riemann.ist.psu.edu/image
![Page 25: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/25.jpg)
TOPIC 5 ASSIGNMENTS
Preview the on-line topic 5 materials Read Farid paper (2008)
![Page 26: T OPIC 5: L EVEL 0 David L. Hall. T OPIC O BJECTIVES Introduce concepts of level 0 processing Identify categories of techniques for level 0 processing.](https://reader035.fdocuments.net/reader035/viewer/2022062517/56649e855503460f94b86fb3/html5/thumbnails/26.jpg)
DATA FUSION TIP OF THE WEEK
Level 0 processing is a vital part of the data fusion process - in essence pre-processing or conditioning data from individual sources and sensors. This kicks off all subsequent processing. It is necessary to do the best job possible at this stage without either introducing spurious information into the observed data, nor failing to extract and enhance the data “at the source”. It is not possible to use “down-stream” data fusion processing to make up for failures at the level 0 stage.