Objective of Computer Vision The field of computer vision can be divided into two areas – Image...

Objective of Computer Vision

• The field of computer vision can be divided into two areas

– Image enhancement

– Image analysis

Here we concentrate on fast methods typical for robot soccer and robot theatre applications

Binary image Binary image processingprocessing

Image with two gray levels 0 and 1 It contains the basic terms and

concepts used in machine vision. Its techniques are used in all aspects

of a vision system. Small memory requirements Fast execution time

Gray level image vs. binary image

Image EnhancementImage Enhancement

POINT POINT OPERATIONSOPERATIONS

Point OperationsPoint Operations

Region and Region and segmentationsegmentation

Region ( )A subset of an image

SegmentationGrouping of pixels into regions such

that

ThresholdingThresholding

Thresholding Thresholding :

A method to convert a gray scale image into a binary image for object-background separation

: Thresholded gray image Obtained using a threshold T for the original gray image

.

: Binary image Equivalent to .

Three types of thresholding

where Z is a set of which elements are integer-valued intervals.

Original image and its histogram

Thresholding

HHistogram EqualizationHistogram Equalization

VVision Systemision Systemof Soccer Robot of Soccer Robot

Without going first to details we will discuss a set of techniques for robot soccer

Image notation for soccerImage notation for soccer

Image : a two-dimensional array of pixels

Pixel a[i, j]

Pixel B[i, j]

Geometric Geometric propertiesproperties

In many cases, some simple features of regions are useful to: determine the locations of objects, and to recognize objects.

Geometric properties:SizePositionOrientation

Size and positionSize and position Given an m x n binary image,

Size (area) A : zero-order moment

Position : the center of area

Average in x

Average in y

Total size is number of black dots

Pixel B[i, j]

ExampleExample

Size filter for Noise Size filter for Noise removalremoval

It can effectively remove noise after component algorithm labeling.

If objects of interest have sizes greater than T, all components below T are removed by changing the corresponding pixels’ values to 0.

A noisy binary image and the resulting image after size filtering (T = 10)

How to get How to get

the position and angle of robotthe position and angle of robot Get frame-grabber, color CCD camera and computer. Understand how you can read the image data from frame-

grabber. Find the position of a colored object in 2-D image. Determine the robot by determining its two colored objects Calculate the position and angle of the robot from the

positions of two colored objects.

Position of colored objectPosition of colored object

1. Setting of ranges for YUV

[Ymin,Ymax], [Umin,Umax], [Vmin, Vmax]

2. Thresholding

3. Labeling (grouping)

4. Size filtering (noise elimination)

5. Finding the center of a colored object

Calculations for Calculations for soccer fieldsoccer field

Finding robots in the Finding robots in the fieldfield

From size of soccer field and finding robot 1 with camera I can calculate its center in centimeters

Without special

Robot position and headingRobot position and heading

This cross sign is easy to recognize

Using two colored objectsUsing two colored objects

Robot color and team colorRobot color and team color

This slide explains labeling robots and teams

Window tracking methodWindow tracking method Processing only the data within a small window Getting a fast vision processing

Line Line OrientationOrientation

Finding Orientation using the axis of elongation

This is called line orientation

Having the shape we want to find its axis of elongation

Line equation : : the minimum distance between the line and origin : the angle from x-axis to the line

The distance, d, from any (x, y) within the object to the line :

which satisfies

Minimize

Our task is to find values of angle theta and rho for which this formula is minimum. This provides best fit to line equation

Having the shape we want to find its axis of elongationHaving the shape we want to find its axis of elongation

The center of object : Let

By the least-squares fitleast-squares fit of the line,

Step 1: calculate a,b,c coefficientsStep 1: calculate a,b,c coefficients

Step 2: calculate angle, line orientationStep 2: calculate angle, line orientation

How to calculate line How to calculate line orientation?orientation?

Pixel B[i, j]

The center of object : Let

Given is object. Find its line orientation

1. Calculate center

2. Calculate a, b, and c.

By the least-squares fit of the line,1. Calculate center

2. Calculate a, b, and c.

3. Calculate theta

Binary algorithms to measure shapes Binary algorithms to measure shapes and sizes from top cameraand sizes from top camera

Several definitions Neighbors

4-neighbors (4-connected)


Path A sequence of neighbors

Foreground : The set of all unity valued pixels in an image

Connectivity A pixel is said to be connected to if there

is a path from to consisting entirely of pixels of .

Connected components A set of pixels in which each pixel is connected to all

other pixels.

Main definitionsMain definitions

Component Component labelinglabeling

Component labeling algorithm

It finds all connected components in an image and assigns a unique label to all the points in a component.

One of the most common operations in machine vision

Recursive connected components algorithm

Sequential connected components algorithm

The points in a connected component form a candidate region for an object.

An image and its connected component image

RecursiveRecursive algorithm for algorithm for connected component labelingconnected component labeling

Recursive connected components algorithm

1. Scan the image to find an unlabeled unity valued pixel and assign it a new label L.

2. Recursively assign a label L to all its unity valued neighbors.

3. Stop if there are no more unlabeled unity valued pixels.

4. Go to step 1.

Pseudo code for the recursive algorithm

Recursive call of procedure LabelRecursive call of procedure Label

SequentialSequential algorithm algorithm for for connected component labelingconnected component labeling

Sequential connected components algorithm using 4-connectivity

1. Scan the image from left to right and top to bottom.

2. If the pixel is unity valued , then(a) If only one of its upper or left neighbors has a label,

then copy the label.

(b) If both have the same label, then copy the same label.

(c) If both have different labels, then copy the upper pixel’s label and enter the labels in an equivalence table as equivalent labels.

(d) Otherwise assign a new label to this pixel and enter this label in the equivalence table.

3. If there are more pixels to consider, then go to step 2.

4. Find the lowest label for each equivalent set in the equivalence table.

5. Scan the picture. Replace each label by the lowest label in its equivalent set.

Sequential algorithm for connected component Sequential algorithm for connected component labeling labeling continuedcontinued

Pseudo code for step 2 in the Pseudo code for step 2 in the sequential sequential algorithmalgorithm

An example for the sequential algorithm

Initial labeling

relabeling

Connected Component LabelingConnected Component Labeling

10

01

2

10

01

2

Algorithm

1. Image is A. Let A = -A;

2. Start in upper left and work L to R, Top to Bottom, looking for an unprocessed (-1) pixel.

3. When one is found, change its label to the next unused integer. Relabel all of that pixel’s unprocessed neighbors and their neighbors recursively.

4. When there are no more unprocessed neighbors, resume searching at step 2 -- but do so where you left off the last time.

Connected Component LabelingConnected Component Labeling

Digital GeometryDigital Geometry

Neighborhood Connectedness Distance Metrics

Picture Element or Pixel

Pixel value I(I,j) =0,1 Binary Image0 - K-1 Gray Scale ImageVector: Multispectral Image

32

I(i,j) (0,0)

i

j

Connected ComponentsConnected Components Binary image with multiple 'objects' Separate 'objects' must be labeled individually

This image has 6 Connected Components

Finding Connected ComponentsFinding Connected Components Two points in an image are 'connected' if a path can be

found for which the value of the image function is the same all along the path.

P1

P2

P3

P4

P1 connected to P2

P3 connected to P4

P1 not connected to P3 or P4




General Labeling AlgorithmGeneral Labeling Algorithm Pick any pixel in the image and assign it a label Assign same label to any neighbor pixel with the

same value of the image function Continue labeling neighbors until no neighbors

can be assigned this label Choose another label and another pixel not

already labeled and continue If no more unlabeled image points, stop.

Who's my neighbor?

Example of using this algorithm for labelingExample of using this algorithm for labeling

Lab. Im. - 4th Component Final Labeling

Image 'Label' Image

Lab. Im. - 1st Component Lab. Im. - 2nd Component Lab. Im. - 3rd Component

Who is my Neighbor?Who is my Neighbor? Consider the definition of the term 'neighbor' Two common definitions:

Consider what happens with a closed curve. One would expect a closed curve to partition the plane into two connected regions.

Four Neighbor Eight Neighbor

Alternate Neighborhood DefinitionsAlternate Neighborhood Definitions

Neither neighborhood definition satisfactory!

4-neighborconnectedness

8-neighborconnectedness

Assuming 4-neighbor I have four different

neighbors

Assuming 8-neighbor

I have just one neighbor

Possible Solutions to Neighbor problemPossible Solutions to Neighbor problem

1. Use 4-neighborhood for object and 8-neighborhood for background

requires a-priori knowledge about which pixels are object and which are background

2. Use a six-connected neighborhood:

Digital DistancesDigital Distances Alternate distance metrics for digital images

i

m

n

j

i

m

n

j

i

m

n

j

Euclidean Distance City Block Distance Chessboard Distance

= (i-n) 2 + (j-m) 2= |i-n| + |j-m| = max[ |i-n|, |j-m| ]

manhattan

Connected Components /Connected Components /Image LabelingImage Labeling

Goal: To find clusters of pixels that are similar and connected to each other

How it works:Assign a value to each pixelDefine what similar values mean

e.g., 10 +/- 2

Determine if like pixels are connected

Component Component LabelingLabeling

Definitions Neighbors



Connected components A set of pixels in which each pixel is connected to all

other pixels.

4- connected 8-connected

An image and its connected coAn image and its connected component imagemponent image


1 1 1 1 1 1

1 0 0 1 1 1

1 1 1 0 1 1

1 2 2 0 0 1

1 2 2 0 0 1

A A A A A A

A B B A A A

A A A C A A

A D D C C A

A D D C C A

Cross produces 4 segments


1 1 1 1 1 1

1 0 0 1 1 1

1 1 1 0 1 1

1 2 2 0 0 1

1 2 2 0 0 1

A A A A A A

A B B A A A

A A A B A A

A C C B B A

A C C B B A

rectangle produces 3 segments

After thresholding an image, we want to know something about the regions found ...

Binary Image ProcessingBinary Image Processing

How many objects are in the image?

Where are the distinct “object” components?

“Cleaning up” a binary image?

Recognizing objects through their response to image masks

Describing the shape/structure of 2d objects

Pattern

Counting ObjectsCounting Objectsexternal corners internal corners

1

0 1

1 1

1 0

11

1 1

0 0

1 1

10

0 0

1 1

0 0

00

1 0

0 0

0 1

0

You can characterize a shape by counting

its internal and its external corners

This gives you two parameters

SummarySummary A simple procedure to get robot position and heading:

Thresholding, labeling, size filtering, center of area calculation, window tracking method

Trade-off between: real-time vision processing and robustness

Specific frame grabber required

Problems for studentsProblems for students1. Binary image processing and basic operations on binary

images.2. List and explain briefly operations used in image enhancement3. What are point operations. Give examples.4. Explain contrast stretching operation. Write simple program.5. Definition of image segmentation.6. Segmentation based on thresholding7. Types of thresholding. Give examples of each with practical

applications.8. The general idea of histogram equalization. Show examples.9. Geometric positions of objects in robot soccer.10. Size and position of objects in robot soccer.

Problems for studentsProblems for students11. How to calculate quickly the center of the mass of an

object?

12. How to calculate line orientation?

13. Describe and program size filter for noise removal. Emphasize speed not quality.

14. How to calculate position and angle of a robot using the ceiling camera?

15. Use of colors in vision of robot soccer.

16. Window tracking methods in robot soccer.

17. Use of Line orientation in robot soccer.

Problems for studentsProblems for students18. Binary image processing algorithms in robot soccer.

19. Component labeling20. Recursive connected components algorithm21. Sequential connected components algorithm22. Algorithms to find connected components.23. Alternate definitions of neighbors in connected component

s algorithms. Applications in robot soccer.24. The concept of digital distance and its use.25. Connected component labeling in binary images.26. Counting objects. What are applications in robot soccer?

Objective of Computer Vision The field of computer vision can be divided into two areas – Image...

Documents

Transcript of Objective of Computer Vision The field of computer vision can be divided into two areas – Image...