What is pattern recognition (lecture 4 of 6)

ERI SUMMER TRAININGCOMPUTERS & SYSTEMS DEPT.

Dr. Randa ElanwarLecture 4

Content

� Feature types

� Feature selection methods

� Classifier types� Template Matching

Statistical methods

2

ERI Summer training (C&S) Dr. Randa Elanwar

� Statistical methods

� Stochastic Processes

� Structural Matching

� Neural Networks

� Rule-based methods

� The real pattern recognition system

Feature types3

� After having the big picture about how classification is done, we need to discuss some points in more details to have deeper understanding

� Features Types

� Three different groups of feature extraction techniques exist: templates, structural decomposition and series expansion.


Feature types

� 1. Template Matching, i.e. using a template (as a mask) to compute certain features. For example:

4


� using a grid to compute the ratio of foreground black pixels to the back ground white pixels, or the count of black pixels per column, or the number of connected pixels. The template can be circular mask with sectors and the foreground black pixels area could be the feature, and so on.

Feature types5

� Template Matching is very sensitive to pattern/sample size, rotation and translation variations as well as to noise. I.e., two typical patterns may have different feature vectors if one of them is bigger in size, rotated, or translated a little. Thus, it has limited application.

� 2. Structural Decomposition i.e., describing the geometry of the pattern shape. For example � (1) knowledge about the contour of the object like height

contour and chain code features, � (2) knowledge about what sort of components make up that

object like end points, T-joints and X-joints, loops, curves, etc.


Feature types

� 3. Series Expansion i.e., representing a signal by a linear combination of a series of simpler well-defined functions, where the coefficients of the linear combination are considered as features.

� If these functions are orthogonal to each other, then the

6


� If these functions are orthogonal to each other, then the coefficients provide non-redundant (distinct) information (features) like: Moments, Fourier transform, Gabor transform and wavelets.

� There is an infinite number of potential features that one can extract from a finite 2D pattern. However, best features are those relevant to the problem domain.

Feature types

� Usually, features used are selected and tested in groups till the best group of features leading to the best classification results are selected eventually.

� As we mentioned before, increasing the number of features to over fit the data classification in the feature space is a

7


to over fit the data classification in the feature space is a wrong practice because once we have a new set of data the decision boundary will not be able to classify them correctly.

� Moreover, as we illustrated, increasing features more than needed will lead to introducing noise to the system and cause samples overlap in the feature space.

Feature selection methods8

� At this stage, we are not going to go into deep details describing the features selection techniques but for knowledge these are the techniques names only:

� Suboptimal solution sets , Sequential forward selection (SFS), Generalized SFS, Sequential backward selection (SBS), Generalized SBS, etc.


Feature selection methods9

� All of them go around the same idea either initialized by a small set of features and adding to it gradually or initializing by all features in one set and taking away from it gradually according to the calculation of probability of correct classification.calculation of probability of correct classification.

� Now it's time to have a closer look to other classifiers than NN for broader vision of machine learning.


Classifier types

� As we mentioned before, recognition involves classifying each unknown object into one of a finite number of categories (or classes). There are different methods of classification to implement the above recognition approaches:

� Template Matching.

10


� Template Matching.� Statistical methods.� Stochastic Processes (Markov Chains).� Structural Matching (Trees, Chains, etc.).� Neural Networks.� Rule-based methods.

� Many recent methods mix several techniques together in order to obtain improved reliability.

Template Matching11

� determine the degree of similarity between two vectors (Groups of pixels, shapes, curvatures, etc) in the feature space using some minimum distance approach (e.g. Euclidean distance).

� Examples: Direct Template Matching, String Matching, Elastic Template Matching.


Statistical methods12

� concerned with statistical decision functions and a set of optimal criteria, which determine the probability of the observed pattern belonging to a certain class.

� Examples: k-Nearest-Neighbor (k-NN), and Bayesian Classifier (Maximum likelihood).


Stochastic Processes

� is a doubly stochastic process (probabilistic models) offered for modeling patterns. One of the most important properties is the existence of efficient algorithms to automatically train the models without any need of segmenting patterns.

13


any need of segmenting patterns.

� Example: Hidden Markov Model (HMM)

Structural Matching14

� patterns are represented as unions of smaller sub-patterns (primitives). It can find the relationship among them.

� Examples: grammatical methods and graphical methods.


Rule-based methods

� Sometimes called "Ad-hoc rules", they use abstract descriptions of patterns to recognize them.

� For example, a rule of recognizing the letter ‘x’ might be ‘two lines that cross over and are at approximately 45 and 135 degrees’.

15


� The problem with these methods is that it is not possible to design an exhaustive set of rules that model all possible ways of forming a letter. They are still not as good as the performance of statistical methods.

� They are however very useful in disambiguating between certain class pairs using very few parameters. One example is the disambiguation between a ‘v’ and ‘u’ based on the presence or absence of a hook at the end

The real pattern recognition system16


The real pattern recognition system

� 1. Data acquisition and sensing:� Measurements of physical variables (capturing image,

recording speech or video etc.)

� Important factors (parameters) in this step that affects the patterns due to data acquisition: bandwidth, resolution,

17


patterns due to data acquisition: bandwidth, resolution, sensitivity, distortion, SNR, latency, etc.

� Data collected is divided to 3 sections (train, test, validation). The training data is that used for learning models. From time to time you need to make a test to adjust the system parameters (thresholds) or rules. In such case, validation data is used. When the system performs quite well (without over fitting), test section is used for final evaluation.


� 2. Pre-processing:� an essential task that precedes the tasks of image representation

and recognition. Its importance is derived from the fact that the discrimination power is directly proportional to the digital image quality, in the sense that, the higher the image quality the less confusions we have and thus more powerful classification we can make. make.

� Some common operations performed as preprocessing are: binarization, the task of converting gray-scale image into a binary black-white image; noise removal, the extraction of the foreground textual matter by removing textured background, salt and pepper noise or interfering strokes; image enhancement and restoration, the task of converting the image to be more suitable than the original image for a specific application ; morphological image processing, the task of extracting image components that are useful in the representation and description of region shape.



� The basic morphological algorithms are: boundary extraction; region filling; extraction of connected components; thinning and thickening.

� And the most common morphological operations used are

19


� And the most common morphological operations used are dilation, where the value of the output pixel is the maximum value of all the pixels in the input pixel's neighborhood; erosion, where the value of the output pixel is the minimum value of all the pixels in the input pixel's neighborhood; opening, which smoothes contours and eliminates small islands and sharp peaks; and closing, which smoothes contours, fuses narrow breaks and eliminates small holes.


� Segmentation of pattern image is sometimes considered as preprocessing operation and sometimes is a separate stage.

21


separate stage.

� 3. Feature extraction:

� Finding a new representation in terms of features


� 4. Model learning and estimation:� Learning a mapping between features and pattern groups

and categories.

� 5. Classification:� Using features and learned models to assign a pattern to a

class.

� 6. Post-processing (extra optional stage to improve performance):� Evaluation of confidence in decisions.� Exploitation of context to improve performance.� Combination of multiple classifiers decisions.


What is pattern recognition (lecture 4 of 6)

Science

Transcript of What is pattern recognition (lecture 4 of 6)