Advanced Image Processing IIRS-2008
Transcript of Advanced Image Processing IIRS-2008
-
8/10/2019 Advanced Image Processing IIRS-2008
1/20
ADVANCED IMAGE
PROCESSING
Sanjay K. GHOSH
Professor of Civil Engg
IIT ROORKEE
email: [email protected]@yahoo.co.in
IMAGE PROCESSING AND
ANALYSIS
Act of examining images for the purpose of identi fying
objects and judging their signi f icance
Image analyst studies the remotely sensed data and
attempts through logical process
detection,
identification classification
measurement
Evaluate the significance of physical and cultural
objects, their patterns and spatial relationship.
-
8/10/2019 Advanced Image Processing IIRS-2008
2/20
Representation of Data. Photograph
Image
The data is in digital form
where the area is
subdivided into equal size
picture elements or pixels.
The information is
collected in narrow
wavelength range referredas a BAND
FCC OF ROORKEE AREA IRS LISS III DATA
-
8/10/2019 Advanced Image Processing IIRS-2008
3/20
IKONOS DATA OF ROORKEE DATA
PROCESSING & ANALYSIS
INTERPRETATION
Visual - Human based
Digital - Computer assisted
-
8/10/2019 Advanced Image Processing IIRS-2008
4/20
COMPARISON
VISUAL ANALYSIS
Single band or as FCC
Subjective
Slow
Analyst Bias
DIGITAL ANALYSIS
Multi Image
Objective
Fast with many options
Free of Analyst bias
Elements of Image Interpretation
Primary Elements
Black and White Tone
Color
Stereoscopic Parallax
Spatial Arrangement of Tone& Color
Size
Shape
Texture
Pattern
Based on Analysis of
Primary Elements
Height
Shadow
Contextual ElementsSite
Association
-
8/10/2019 Advanced Image Processing IIRS-2008
5/20
DIGITAL IMAGE PROCESSING
Image classification and analysis
digitally identify and
classify pixels
supervised
unsupervised
-
8/10/2019 Advanced Image Processing IIRS-2008
6/20
Image Classification and Analysis
Spectral pattern recognition
Digital image classification uses the spectral
information represented by the digital numbers in one
or more spectral bands, and attempts to classify each
individual pixel based on this spectral information
The resulting classified image is comprised of a mosaic of
pixels, each of which belong to a particular theme, and is
essentially a thematic "map" of the original image.
Common classification procedures
Supervised classification
Unsupervised classification
-
8/10/2019 Advanced Image Processing IIRS-2008
7/20
Supervised classification
Training areasthe analyst identifieshomogeneous representativesamples of the differentsurface cover types
To determine the numerical"signatures
Once the computer hasdetermined the signatures foreach class, each pixel in theimage is compared to these
signatures and labeled as theclass it most closely"resembles" digitally
Unsupervised classification
reverse of supervised
classification
Spectral classes are grouped
first
Then matched to information
classes the analyst specifies how
many groups or clusters
It is iterative in nature
not completely without
human intervention
-
8/10/2019 Advanced Image Processing IIRS-2008
8/20
Comparison
PROBLEM OF MIXED PIXEL With coarse resolution data, the occurrence of mixed pixels
had been intense, and it was thought that this aspect will
reduce with increase in spatial resolution.
However, this problem has remained same in magnitude
with increase in spatial resolution.
With coarse resolution, the chances of two or more classes
contributing to a mixed pixel were high but the number of
such pixels was small. With improved spatial resolution, the number of classes
within a pixel has reduced but the number of mixed pixels
has increased.
In a way, the problem of mixed pixels remained, may be its
direction of impact has changed.
-
8/10/2019 Advanced Image Processing IIRS-2008
9/20
PROBLEM OF MIXED PIXEL Consider a simple land area consisting of two classes,
namely, water and land (Fig.1).
Two pixels belong to only one class, i.e., pixel 1 haswater and pixel 4 has land, and these are called as pure
pixels.
Pixels 2 and 3 has varying composition of land andwater, and are called mixed pixels.
A mixed pixel displays a composite spectral responsethat may be dissimilar to the spectral response of eachof its component classes, and therefore, the pixel maynot be allocated to any of its constituent classes.
Therefore, an error is likely to occur in theclassification of the image.
Convention statistical based image classification (alsoknown as hard classification) which assumes that the
pixels contain pure information, would identify thepixel to one and only one class.
Thus pixel 2 may be classified as water and pixel 3 asland (Fig.1b).
Depending upon the proportion of mixed information,it may result into a loss of pertinent information
present in a pixel and subsequently in an image.
Pixel 2
Pixel 3 Pixel 4
Land
Water
0 1
(a) Actual land cover (b) Hard classification
(i) Water (ii) Land
(c) Fraction Image
Land
Land Water
Water
Pixel 2
Pixel 3 Pixel 4
Pixel 1
Pixel 1 Pixel 2
Pixel 3 Pixel 4
Pixel 1 Pixel 2
Pixel 3 Pixel 4
Mixed pixels have to be accommodated in theclassification process in some way, by making useof sub-pixel or soft classification methods basedon certain heuristic and logical reason has to beadopted.
The output from these methods is a set of class
membership values for each pixel known as soft,fuzzy or sub-pixel classification outputs whichrepresent the probability fraction or proportionimages (Fig.1c).
These soft outputs strongly relate to actual extentsof the classes on ground.
-
8/10/2019 Advanced Image Processing IIRS-2008
10/20
Soft classification methods
Spectral mixture analysis.
Fuzzy set theory.
Artificial neural network.
Linear Mixture Model (LMM) Widely used for the decomposition of the class proportion of
mixed pixels.
The method assumes that the spectral response of a pixel is alinear sum of the mean spectral responses of the various landcover classes weighted by their relative proportion on theground
The model can be mathematically expressed as
whereMij is the end member spectra representing the meanclass spectral response ofjth land cover class in the ithband,
fj are the proportions ofjth land cover class in a pixel,
ei is the error term for ithband, which expresses the difference
between the observed spectral response and the model derivedspectral response of the pixel.
=
+=c
jiijji eMfx
1
-
8/10/2019 Advanced Image Processing IIRS-2008
11/20
Linear Mixture Model (LMM)
It may be noted that class proportions of a mixed
pixel are not negative and that the sum of all theclass proportions is equal to one, and can beexpressed as
Andfj 0 for allj land cover classes.
The end member spectra matrix M represents thespectral responses of classes, and may be calculated
by taking the average spectral response of that classhaving pure pixels, or estimated from laboratory andfield measurements of the classes, or by performing
principal component analysis.
=
=c
j
jf1
1
When applied to remote sensing of semi-vegetated areasthe linear mixture model approach assumes that end-members can be frequently be recognized from the imageitself ('image end-members').
Disregarding theoretical considerations, such as the factthat the model assumes a single-scattering approach, it isthe difficulty in locating end-member spectra that presentthe main difficulty to the user.
Logic indicates that an end- member proportion can not benegative and, if the model is properly specified, that thesum of the proportions of end-members at a given pointmust be less than or equal to unity.
-
8/10/2019 Advanced Image Processing IIRS-2008
12/20
It is possible to build these constraints into the linearmixture model so that the result derived for every
individual pixel satisfy these logical requirements. It is, however, more practical to consider the
unconstrained model which simply computes, from alibrary of end member spectra, the end-member
proportions at a given point.
If the model fits perfectly then there should be no end-member proportions less than zero or greater thanunity, and the sum of the proportions at a given point.
If the model fits perfectly then there should be no end-member proportion less than zero or greater than
unity, and the sum of the proportions should notexceed 1.0.
Furthermore a root mean squared error may not showany systematic pattern.
Only by using and unconstrained model is it possibleto check that these conditions are met.
One constraint imposed by linear unmixing is that thenumber of end-members cannot exceed the number ofspectral bands available.
Even so, the selection of end-members which iscrucial to the successful application of the linearmixing model in fraught with difficulties.
-
8/10/2019 Advanced Image Processing IIRS-2008
13/20
Fuzzy c-Means (FCM) FCM is an iterative clustering method employed to
partition pixels of remote sensing images into differentclass membership values.
The key is to represent the similarity that a pixel shareswith each cluster with a function (membership function)whose value lies between zero and one.
Each pixel will have membership in every cluster.
Memberships close to unity signify a high degree ofsimilarity between the pixel and that cluster.
The net effect of such a function of clustering is toproduce fuzzy c-partitions of a given data.
A fuzzy c-partition of the data is the one whichcharacterizes the membership of each pixel in all theclusters by a membership function that ranges from zero toone.
Possibilistic c-Means (PCM) The main motivation behind the use of PCM relates to the
relaxation of the probabilistic constraint of FCM.
Formulation of PCM is based on a modified FCM objective
function whereby an additional term called as regularizing term
is included.
It is similar to FCM as PCM clustering is also an iterative
process where the class membership values are obtained by
minimizing the generalized least-square error objective function
where is a parameter that depends on the distribution of pixels
in the cluster j and is assumed to be proportional to the mean
value of the intra cluster distance
= = = =
+=N
i
c
j
c
j
mN
i
ijjAji
m
ijm ivxVUJ1 1 1 1
2
)()(),(
j
-
8/10/2019 Advanced Image Processing IIRS-2008
14/20
Neural Network Based Methods Artificial neural networks have the capability to generalize
the relation between the evidence (e.g., remote sensingdata) and the conclusion (e.g., landcover classification)without developing any mathematical models.
Thus, unlike statistical parametric methods, they do notassume that the data follows a distribution.
The neural network contains interconnected layers eachcontaining a number of units, symbolizing the biologicalconcept of a neuron.
The interconnections carry weights, which are adjusted inan interactive learning process to provide neural networksolution.
The learning process may be supervised or unsuperviseddepending on whether training data are required or not.
Accordingly, a number of supervised an unsupervisedneural network algorithms have been developed.
Supervised Neural Network
Number of units in the input layer is equal the number of bandsused for the classification.
Unlike input layer, hidden and output layers process the data.The output layer produces the neural network results.
The number of units in the output layer is generally equal to thenumber of classes to be mapped.
Class 1
Band1
Band2
Band3
Band4
input Layer (i) Input Layer (s) Output Layer (j)
Remote Sensing Data Land Cover Classes
Class 2
Class 3
Class 4
Class 5
Wi
s
Ws
j
Typically, a supervisedneural network consists ofthree layers; an inputlayer, a hidden layer andan output layer.
The input layer receivesthe data (i.e., the multi-spectral remote sensingimage data).
=i
isiWxsnet sjWiOsO =
-
8/10/2019 Advanced Image Processing IIRS-2008
15/20
Supervised Neural Network
Therefore, the number of units in the input and output
layers are fixed by the application designed. Selection of the number of hidden layers and their units is
a critical step for the successful operation of the neuralnetwork.
Using too few units in the hidden layer may result intoinaccurate classification as the network may not bepowerful enough to process the data.
On the other hand, by using a large number of hiddenunits, the computational time becomes large. It may alsoresult into the network being over-trained.
The optimum number of units in the hidden layer is oftendetermined by trial and error, though some empiricalrelations do exist.
Back Propagation Neural Network (BPNN) The BPNN is a generalized least squares algorithm that adjusts the
connection weights between units to minimize the mean square errorbetween the network output and the target output.
The target output is known from reference data.
Data provided to input unit are multiplied by the connection weightsand are summed to derive the net input to the unit in the hidden layer.
where, xi is a vector of magnitude of the ith input (i.e., spectralresponse of pixel),
Wis is matrix of the connection weights between ith input layer unit and
sth hidden layer unit.
Each unit in sth hidden layer computes a weighted sum of its inputs,and passes the sum via an activation function to the units in the jth
output layer through weight vectorWsj.
=i
isis Wxnet
-
8/10/2019 Advanced Image Processing IIRS-2008
16/20
There is a range of activation functions to transform the data fromhidden layer unit to an output layer unit. These include pure linear,tangent, hyperbolic, sigmoid functions , etc.
Although, the use of these functions may lead to difference inaccuracy of classification. Generally, sigmoid function has beenwidely used, and may be defined as
where is the output from the sth hidden layer unit, and is a gainparameter that controls the connection weights between the hiddenlayer unit and the output layer unit .
Outputs from the hidden units are multiplied with the connectionweights, and are summed to produce the output of thejth unit in theoutput layer
where Oj is the network output for the jth output unit (i.e., the land
cover class) and Wsj is the weight of the connection betweensth hidden
layer unit andjth output layer unit.
]exp/[ snet
+= 11Os
sjsj WOO =
An error functionE, determined from a sample of target (known)outputs and network outputs, is minimized iteratively. Theprocess continues untilEconverges to some minimum value, andthe adjusted weights are obtained.
E=
where Tj is the target output vector, Oj is the network outputvector, and c is the number of classes.
The target vector is determined from the known class allocationsof the training pixels, which are coded in binary form. Forexample, a pixel belonging to class 3 shall be coded as 0 0 1 0 0 atthe five output units.
The collection of known class allocations of all pixels will formthe target vector.
=
c
j
jj OT
1
2)(50.0
-
8/10/2019 Advanced Image Processing IIRS-2008
17/20
Target output coding for BPNN
0
0
1
0
0
Band 1
Band 2
Band 3
Band 4
Remote Sensing Data
Input Layer Input LayerOutput Layer
Class Allocation
Learning algorithms such as backpropagation have parameters
(e.g momentum and learning rate) that mush be selected. These
can significantly influence the performance of a network. What
values should be selected and should be they be varied in training.
Learning
parameters
There are a range of learning algorithms available.
Backpropagation is the most widely used but can be slow and
faster variants, which make assumptions about the error surface,
are popular. Which should be used.
Learning
algorithm
Determines the capacity of the network to learn and generalize. In
general, large network may learn more accurately but have poorer
generalization ability than a small network. Larger networks are
also slower to train. How many hidden units and layers should be
used?
Number
of hidden
unit & layers
CommentParameter
/ issue
-
8/10/2019 Advanced Image Processing IIRS-2008
18/20
The initial weight settings of the pre-trained network can significantly
influence network performance. Typically, these are generally set
randomly, but within what range?
Initial weights
There is a need to ensure that the network has learnt to correctly
identify class membership from the training data but is not
overtraining and so has acceptable generalization ability. How is thisto be assessed? Should verification sets be used?
When/how to
terminate
training
The training error is generally negatively related to the number of
training iterations. The accuracy of generalizations may be non-
monotonically related to the intensity of training: typically the
accuracy of generalization increases as the network gradually learns
the underlying relationship with greater accuracy but will eventually
decline as the network becomes over trained. How many iterations of
the learning algorithm should be used?
Number of
training
iterations
There is usually one input unit associated with each discriminating
variable but other approaches may be used. Also the data input t o the
neural network generally have to be rescaled for the analysis,
typically to a 0 to 1 or -1 to 1 scale. What method should be used to
achieve this and what allowance should be made for data to extend
beyond the range observed in the training set?
Data input
and scaling
CLASSIFICATION ACCURACYASSESSMENT
The accuracy assessment is a critical step in any mapping process, andthus is an essential component that allows a degree of confidence tobe attached to maps for their effective use.
Traditionally, the accuracy of classification has been assessed usingerror matrix based measures.
Here, each pixel in the image is assumed pure, containing one classper pixel on the ground.
Thus, in essence, the continuum of variation found in the landscape isdivided into a finite set of classes such that pixels representing theseclasses became pure, and the error matrix based measures may beused.
However, these classes become less separable as the class mixtureincreases, and therefore, the error matrix based measures may beinappropriate.
Alternate accuracy measures are, therefore, sought to evaluate theaccuracy of soft classification which represents the class mixture in ameaningful way.
-
8/10/2019 Advanced Image Processing IIRS-2008
19/20
CLASSIFICATION ACCURACY
ASSESSMENT Euclidean distance
L1 distance
the cross-entropy
correlation coefficients
fuzzy error matrix (FERM)
All these measures may be treated as indirect methods ofassessing the accuracy of soft classification because theaccuracy evaluation is interpretative rather than arepresentation of actual value as denoted by the traditionalerror matrix based measures.
Correlation Coefficient CC The correlation coefficient CCmay also be used to indicate the
accuracy on individual class basis estimated from a soft classificationoutput and a soft reference data.
The higher the correlation coefficient, the higher is the classificationaccuracy of a class.
where
is the covariance between the two distributions (i.e. the soft classifiedoutput and the soft reference data) and
are the standard deviations of both the distributions.
ijij
ijijCovCC
21
),( 21
=
),( ij2
ij
1
Cov
ijij 21 ,
-
8/10/2019 Advanced Image Processing IIRS-2008
20/20
THANK
YOU