Object detection, tracking and event recognition: the ETISEO experience
Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object...
Transcript of Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object...
![Page 1: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/1.jpg)
Object Detection and Recognition in Complex Environmental Conditions
Vijayan K. Asari
University of Dayton
Dayton, Ohio, USA
VISUAL 2016
13 November 2016
![Page 2: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/2.jpg)
Overview
Sensor Data Acquisition
Sensor Data Exploitation
Decision and Feedback
![Page 3: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/3.jpg)
Data Acquisition
Pre-processing
Feature Extraction
Classification
Decision
Scene Analysis and Understanding
Biometric Identification
Human Activity Recognition
Brain Activity Analysis
Hardware Acceleration
Wide Area Surveillance
Distortion correction, enhancement, haze/fog
removal, rain removal, stabilization. Spatial domain features
Spectral domain features
Statistical features
Phase features
Classical and neural network
based approaches.
Face recognition.
Human action and activity recognition.
Object detection/tracking on WAMI data.
Emotion recognition by EEG analysis.
Brain machine interface.
Object Detection and Recognition: Processing Pipeline
![Page 4: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/4.jpg)
Focus Areas
Biometrics
Vision-Guided Robotics
Perception Beyond Visible Spectrum
Image and Video Preprocessing
Wide Area Surveillance
Brain Activity Analysis
Enhancement
Super-resolution
Haze removal
Rain removal
Stabilization
Object detection
Object recognition
Object tracking
3D reconstruction
Change detection
Face recognition
Human action and
activity recognition
Expression analysis
Emotion recognition
Robotic navigation
Path planning
Object following
Behavior analysis
Threat analysis
LiDAR data analysis
Hyperspectral data
IR/thermal data
Satellite imagery
EEG data analysis
Emotion recognition
Brain machine
interface
Source localization
Neurofeedback
![Page 5: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/5.jpg)
Underexposed, dark, dark and bright (shadows), bright, overexposed regions
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
AINDANE MWIS Power function LTSN STTF
Enhancement of Low Lighting and Over Exposed Images
![Page 6: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/6.jpg)
Intensity computation (NTSC)
Nonlinear function
Dynamic Range Compression
),(114.0),(5867.0),(2989.0),( yxIyxIyxIyxI BhGhRh
)2/
),(()/2(),(q
yxnIArcSinyxenhI
Dark pixels Bright pixels
![Page 7: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/7.jpg)
Adaptive Estimation of Control Parameter
q < 1 Provide various nonlinear curves if the
pixels are dark.
q = 1 Provides a curve if the pixel has
sufficient intensity.
q > 1 Provide various nonlinear curves if the
pixels are bright.
Depending on the mean value of its neighborhood
Multi-level Gaussian function
w1 = 5 w2 = 20 w3 = 240
1
0
1
0
),(),(),(M
m
N
n
iM ynxmGnmIyxIi
2
22 )(
.),( iw
yx
i eKyxG
Window size depends on the resolution and object size in an image.
![Page 8: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/8.jpg)
The function for the q value can be designed as
and are empirically determined.
5.0,1
5.0,1
5.0,1
n
n
n
M
M
M
ifI
ifI
Iif
q
21)/(*),( ccyxITanqn
M
1c 2c
Criteria for estimation of q
25.21 c 0085.02 c
Adaptive Estimation of Control Parameter
For q values which are closer to 0 the noise in the extreme dark regions will also be
enhanced.
Hence, the q values corresponding to the mean value below 0.2 is considered as extreme
dark regions and q for those pixels can be calculated as
;2),(2log
yx
nM
Iq
![Page 9: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/9.jpg)
Nonlinear Enhancement Module
( , )( , )
( , )
convI x yE x y
I x y
),(),(255),( yxE
enh yxxIyxS
where i represents red, green, blue spectral band
),((
),(
,),(
yxn
I
yxenh
I
ienhI yxI i
Contrast Enhancement
Color restoration
![Page 10: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/10.jpg)
Input image Enhanced image
Enhancement of Low Lighting and Over Exposed Images
![Page 11: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/11.jpg)
Hazy image Weather degraded image
Weather Degraded Image: Poor contrast, distorted color
Estimation of approximate thickness of haze in the scene and enhancement using a single nonlinear function.
An adaptive estimation of control parameter from its neighborhood information.
Hazy image
Enhancement of Hazy/Foggy Images
![Page 12: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/12.jpg)
Transmission mapOriginal Haze-free
Transmission mapOriginal Haze-free
Enhancement of Hazy/Foggy Images
![Page 13: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/13.jpg)
Enhancement of Hazy/Foggy Images
![Page 14: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/14.jpg)
Original Images Enhanced Images
Enhancement of Hazy/Foggy Images
![Page 15: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/15.jpg)
Scene Visibility Improvement: Rain Removal
![Page 16: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/16.jpg)
Action Recognition
Face Recognition System
Face Detection
Expression Recognition
Iris Recognition
Biometric Data Analysis for Human Identification
![Page 17: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/17.jpg)
Human Detection
Feature
Extraction Classification
Class1 (Positive)
Class2 (Negative)
Input images
Dataset for training the classifier
Sliding window; Stride= 8 pixels
Off-line phase
On-line phase
![Page 18: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/18.jpg)
Framework of the Human Detection System
* HOP= Histogram of Oriented Phase * HOG= Histogram of Oriented Gradient * CSLBP= Central Symmetric Local Binary Pattern.
Pedestrian /
Non-pedestrian
Gradient magnitude
and orientation
Computation
(for gray image)
Linear
SVM
classifier
Concatenation of the entire image Histograms
CPGT Features
Input image (RGB)
Fusing local region's histograms
CPGT
feature
vectors of
training
datasets
PC. / R PC. / G PC. / B
Select the maximum phase
congruency (one channel).
HOG of local regions
Local region= 16x16 pixels
(4x4 cells)
Cell= 4x4 pixels.
(16 histograms)
HOP of local regions
Local region= 16x16 pixels
(4x4 cells)
Cell= 4x4 pixels.
(16 histograms)
Non-
maximum
suppression
Compute CSLBP
Values
(for gray image)
Histogram of CSLBP
for local regions
(16x16 pixels)
(1 histograms)
Chromatic domain phase features with gradient and texture (CPGT)
![Page 19: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/19.jpg)
CPGT Detector Results
![Page 20: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/20.jpg)
CPGT Detector Results
![Page 21: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/21.jpg)
Face Detection – quickly and
efficiently locates all faces in a
given image region.
Face Features– calculates
unique features of each person
in the face database that can be
used for accurate classification.
Feature Classification –
compares features of face
regions obtained from the
detection process with face
feature data computed from the
training stage to determine the
identity of individuals.
Face Recognition System
Overview of a face recognition system
Face recognition in video
![Page 22: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/22.jpg)
Face Recognition: Appearance Variations
Expression
Lighting
Pose
Occlusion
![Page 23: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/23.jpg)
Face Recognition – UD Research
• Images at various lighting conditions are enhanced to a uniform lighting environment.
• In order to reduce the search space for faces in an image frame, the human skin regions are extracted using the color information.
• Search for faces in all skin regions by using a feature matrix developed by a training process.
• Detected faces are tracked in consecutive frames by statistical analysis performed using the concept of particle filter.
• Manifold learning technique for face recognition.
Image Enhancement
Skin Segmentation
Face Detection
Face Tracking
Face Recognition
![Page 24: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/24.jpg)
Skin colors are forming a nonlinear pipe in the RGB
space. It is possible to describe the skin color
mathematically using the nonlinear manifold.
Skin Color Segmentation
![Page 25: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/25.jpg)
Face Detection
2429 faces
4549 non-faces
Training with faces and non-faces.
Dimensionality reduction.
Classification.
![Page 26: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/26.jpg)
Face Detection in Enhanced Images
Original image
Enhanced image
Skin segmented image
Detected faces
![Page 27: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/27.jpg)
Lighting Invariant Face Detection
![Page 28: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/28.jpg)
Pose Invariant Face Detection
![Page 30: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/30.jpg)
Face Recognition: A Modular Approach
Person #?Face Identity Classifier
Face Database
Person#1
Person#2
Feature ExtractionTransform to Subspace (high
dimensional vector space to a low dimensional feature space)
Transform to Subspace
Probe
![Page 31: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/31.jpg)
Face Recognition: Object Pose and Orientation Variation
Face images are from UMIST face database
![Page 32: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/32.jpg)
Original image
Generated synthetic 2D images
Synthetic Database using Single Training Image
3D Face Model
lighting pose
![Page 33: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/33.jpg)
Face Recognition – Moving Forward!
![Page 34: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/34.jpg)
Pedestrian tracking
Small boat detection
Whale blow detection in IR video
Object detection and tracking on WAMI data
Object Detection, Tracking, and Identification: Wide Area Motion Imagery Data and IR Data Analysis
![Page 35: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/35.jpg)
Wide Area Aerial Imagery Data Analysis
• CLIF – Columbus Large Image Format.
• Data from electro-optic sensors mounted on an aerial platform flying at 7000 feet.
• Six cameras with partially overlapping fields of view.
• Frame size: 4008×2672 pixels at 2 fps.
Objects of interest – cars, vans, trucks
![Page 36: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/36.jpg)
Moving Object Detection
Original frame Difference image After gradient suppression
![Page 37: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/37.jpg)
Object Tracking
• Feature tracking using Dense SIFT
• Extract SIFT features for every pixel .
• Dense feature set gives a better representation of the object.
• Matching is based on the criteria that ratio of distances to first and second closest match should be greater than a particular threshold.
50 100 150 200 250
50
100
150
200
![Page 38: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/38.jpg)
Vehicle Tracking
Tracking multiple objects in a scene with enhancement
![Page 39: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/39.jpg)
Object Tracking with Enhancement and Super-resolution
![Page 40: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/40.jpg)
Track pedestrian movement in long range data (CLIF data)
Pedestrian Tracking
Car15x15 pixels
Pedestrian2x7 pixels
![Page 41: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/41.jpg)
Pedestrian Tracking
![Page 42: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/42.jpg)
Classification Problem on CLIF Data
Low resolution
Poorly defined contour
No color information
Trucks and cars: Intensity distributions are significantly
different
- Enhancement is an important preprocessing step
- Some fuzziness in the intensity distribution
Classifiable with Linear SVM
![Page 43: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/43.jpg)
Tracking with Classification
![Page 44: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/44.jpg)
Moving Object Classification
Detecting and classifying moving targets into two classes.
![Page 45: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/45.jpg)
Moving Object Classification with Enhancement
The number of detections significantly improves with super-resolution and enhancement.
![Page 46: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/46.jpg)
Whale Blow Detection in IR Video: Objective
• Detect and track movement of whales during migration
• Detect presence of whales by detecting whale blows
• Estimate pod size using timing constraints of whale blows
• Track whale movement based on their characteristic movement patterns
IR Video: Frame Size: 340 × 280 pixels, Frame Rate: 30 fps
![Page 47: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/47.jpg)
Characteristics of Whale Blows
• Blow appears as a distinct change in the environment.
• Whale blow is brighter than the background.
• Distinctive shape when the blow is full-size.
• Two whale blows will not have same base.
• Presence of significant distance between two whales.
• Temporal characteristics of the blow.
• Rise period and fall period.
• Characteristic variation in blow shape.
![Page 48: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/48.jpg)
Whale Blow Detection
Video with whale blow
![Page 49: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/49.jpg)
Whale Blow Detection
With textural variations on the surface
![Page 50: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/50.jpg)
Whale Blow Detection
Multiple whale blows
![Page 51: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/51.jpg)
Image AnalysisAerial Imagery
Object Classification
Threat
Localization
Automated System Cast Shadows Low illumination
Partial Occlusion Small Scale Orientation
Color Transformation and Enhancement
Key Frame Selection
Key Region Localization
Local Feature Extraction
Object Detection and Identification
KML File Mapping
Oil/Gas Pipeline Right-of-Way Automated Monitoring for Pipeline Encroachment and Machinery Threat Detection
![Page 52: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/52.jpg)
Part-based Model for Robust Classification
• The purpose of developing a part-based model is to cope with partial occlusion and large appearance variations.
Raw ImageImage
Enhancement
BackgroundElimination
Part-basedModel for Object
Recognition
Tractor
![Page 53: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/53.jpg)
Part-based Model for Robust Classification
Object Partitioning PartsFeature
Extraction andClustering
HistogramRepresentation
…
Significant Features
![Page 54: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/54.jpg)
Ringlet Part-Based Model
Method: Using Ring Histogram for each part of objects• Invariant to rotation• Still contains spatial information• Still contains partial occlusion ability
Object Ring Based Partitioning Parts
Feature Extractionand Clustering
HistogramRepresentation
![Page 55: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/55.jpg)
Raw Image– Non Occlusion
![Page 56: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/56.jpg)
Part-based Detection – Non Occlusion
Final Detection Output
Backhoe
Most significant partsLess significant parts
![Page 57: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/57.jpg)
Raw Image– Partial Occlusion
![Page 58: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/58.jpg)
Object is occluded by tree
Part-based Detection – Partial Occlusion
Tractor
Final Detection Output
Most significant partsLess significant parts
![Page 59: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/59.jpg)
Threat Detection Results
![Page 60: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/60.jpg)
Control
Independent components
Feature
Extraction
Classification
Decision
EEG Data
Medical Applications Security Efficiency
Stress detection, fatigue assessmentIntentions, motivesBrain machine interface
Brain Signal Analysis: Emotion Recognition and Brain Machine Interface
![Page 61: Object Detection and Recognition in Complex Environmental Conditions … · 2016-12-25 · Object Detection and Recognition in Complex Environmental Conditions Vijayan K. Asari University](https://reader033.fdocuments.net/reader033/viewer/2022041822/5e5f07496c7d1b148c214fea/html5/thumbnails/61.jpg)
ThanksSensing, Processing and Automatic Decision Making in Real Time
www.visionlab.udayton.edu