Overview of Computer and Machine VisionComputer Vision References Computer Vision by Linda G....

TM

Freescale™ and the Freescale logo are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © Freescale Semiconductor, Inc. 2009.

Overview of Computer and Machine Vision

Eric GregoriSr. Software Field Application Engineer

July 2009

TMFreescale™ and the Freescale logo are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © Freescale Semiconductor, Inc. 2009. 1

Agenda

►"The goal of computer vision is to make useful decisions about real physical objects and scenes based on sensed images." Computer Vision

by Linda Shapiro and George Stockman.

1


Computer Vision References

►Computer Vision

by Linda G. Shapiro and George C. Stockman

►Machine Vision theory,algorithms, practicalities

by E.R. Davies

►Algorithms for Image Processing and Computer Vision

by J.R. Parker


Block Diagram of Typical Embedded Computer Vision SystemChip camera

CCIR-601

CCIR-656

inerface

framebuffer

Bayer colorspace

conversion

dma

LCD

dma

LCD controller

dma

LCD bus

ARM11 core

With FPU

And DSP extensions

AHBbus

i.MX31 SoC


IP Camera


i.MX27 IP Camera Kit

►

For $1995 (budgetary):•

Form-factor IP camera with optics and plastics

Includes mini-tripod•

RJ45 cable•

Serial cable (female DB-9)•

Power supply•

Quickstart

guide•

IP-CAM Developer’s CD:Hardware and software reference manualDesign files (Gerber) and schematicsBOM for CPU board and imagerFreescale Linux BSP with integrated drivers (sensor, Wi-Fi), middleware and applications

•

Access to discussion forum www.ip-cam.org•

Part number is MCIMX27IPCAM•

FCC and CE compliant•

Royalty-free reference design•

Available for order August 2008

►

Free (“as is”):•

Design files (Gerber) and schematics•

BOM for CPU board and imager•

Access to discussion forum: www.ip-cam.org•

www.freescale.com/imx27ipcamera

http://www.ip-cam.org/

http://www.ip-cam.org/


i.MX27 H.264 Internet Camera


i.MX27(L)

►Specifications:CPU:

ARM926EJ-S, 400MHzProcess:

90nmCore Voltage:

1.2-1.5V

►Key i.MX27 Features and Advantages:16 KB L1 I-Cache and D-Cache16-channel DMAMulti-standard video codecs at D1 resolution (i.MX27 only)

Video pre- and post-processing, scalingSecurity

Crypto AcceleratorSecurity controller with encrypted RAM storageElectronically blown fuse boxHigh-assurance boot

Real-time OS/SW integrity checkerDynamic Process and Temperature Compensation (DPTC)Connectivity

Ethernet 802.3 MACUSB 2.0 OTG 480MbpsUSB 2.0 Host 12Mbps, USB 2.0 Host 480Mbps PCMCIA/CF, Audio MUXMCC, SD, IrDA, 8x8 keypad, CMOS sensor interfaceATA-6, Memory Stick (i.MX27 only)

Availability:►Package types

•404-ball, 17x17, MAPBGA, 0.65mm pitch

CPUPlatform

Std System

Connectivity

Multimedia Accelerator

System Control

Security

ARM926EJ-S

MemoryControl

ICache

DCache

InternalControl

MMU

BusControl

RTC

PWM

Watch Dog

PCMCIA/CF

UART x6

SSI/I2S x2

I2C x2HS USB OTG

Host x2

IrDA

DDR/SDRAMNAND FlashVsync Flash

EIM

Audio Mux

1-Wire

MMC/SD x3

10/100 Ethernet

Hardware Video CodecsMPEG4/H.263/H.264

Pre and Post Processing

JTAG/ICEM

CLKMgt.

Bootstrap

Sahara v2

RTICSCC

RNGA

CSPI x3

GPIO8x8 Keypad

ATA-6

Memory Stick Pro

LCD Control

i.MX27(L) Applications Processor

Timer x6

GPIO

DMA

IIM

Ext Memory I/F

DDR/SDRAMNAND FlashVsync Flash

EIM

Camera I/F

Not available on i.MX27L


Hardware Overviewi.MX27 CPU PCB

Image Sensor PCB

USB 2.0 / OTG

SD Slot

12VDC / POE

10/100/POEWiFi antenna


i.MX27 IP Camera Features

►Video Compression•

Compression: H.264 and MPEG-4 Part 2 •

Resolution: D1 (720 x 480, 720 x 576) •

Frame Rate: 30fps •

Automatic and Variable Bit Rate Support ►Network Connectivity

•

Interface: 10/100 Ethernet or 802.11g •

Protocols: TCP/IP, DHCP, HTTP, UDP, FTP ►Software Features

•

Freescale Linux®

2.6 Board Support Package (BSP) •

User space applications and middleware for camera configuration and data streaming

•

Configuration webpage for camera setup •

Multi-camera viewing on target system •

Media player client for compressing and rendering captured data on target system


Optics and Sensors

► Tamron

M13VM246•

F/1.2 varifocal

lense•

Focus range: 0.3 -

∞•

Zoom: 1 -

2.5•

Aperture: 1.2 –

close•

Angle: W 111x 83.5degT 47 x 35deg

Note: These are the optics tested to date. Platform is easily accommodates alternate lensing, filtering, etc.

► Aptina

(Micron) CMOS Sensor•

MT9D131 2MP•

30 fps @ 800x600•

1/3 inch format (4:3)•

1600 x 1200 (2.8um x 2.8um)•

On-chip ePTZ

and ROI support

► Sunex

DSL 227•

F/2.0 fish eye•

149deg wide angle•

M12 x 0.5 mount


i.MX31 PDK


i.MX PDK Product Development Board

De-bug Module

Personality Module

CPU Module connected beneath

Camera

SD card containing labs

Pre-loaded with WinCE6


Hardware Overview

De-bug Module

Provides Functions required for hardware and software development, but would not reside on a final product:

•Debug Ethernet•Debug Serial Port•JTAG•Reset, Interrupt, Boot Switches•Debug LEDs•CodeTest

Interface•Power Source•Current/Power Monitoring

i.MX31 Processor ModuleCan be attached to:

•De-bug module for software development•Personality module for demonstration•Both De-bug and Personality

Personality ModuleProvides the devices most commonly tailored to meet a specific target product or customer requirement:

•VGA Touch-screen Display•Buttons•Connectivity•User I/O•Communications•Camera•Storage•External connectors


i.MX31(L) Applications ProcessorSpecifications:•

CPU:

ARM1136, 532MHz•

Process:

90nm•

Core Voltage:

1.2-1.6V

Key i.MX31 Features and AdvantagesARM1136 with 128Kb L2 CacheIntegrated 3D graphics processor

Eliminates 2 chip solutionEliminates separate memory subsystem

Integrated HW Image Processing Unit (IPU)CMOS/CCD InterfaceResize, CSC, Deblock, Dering, Blending

Vector Floating Point Co-Processor (VFP)Smart DMA RISC-based DMA controllerConnectivity

WLAN, BT, GPS via external chipsetHS USB, ATA-6, MMC/SDIO, MS-Pro, Compact Flash266MHz Mobile DDR, NAND/NOR, Mobile SDRAM, SRAM

Scale - shipped multi-million unitsBreadth – design wins in PMP, PND, Cellular, IP Telephony, POS terminals

Availability:•

Package types•

457-ball, 14x14, MAPBGA, 0.5mm pitch•

473-ball, 19x19, MAPBGA, 0.8mm pitch

i.MX31(L)

Not available on i.MX31L

Special Functions

MemoryInterface

System Control

Expansion

ConnectivityInternal

External

CPU Complex

Std System I/O

Multimedia &Human

Interface

IPU

GraphicsAccelerator

Security HW

NANDF Ctl

VSync

SDRAM/DDR

PSRAM

SmartMedia

Bootstrap

JTAG, ETM

System Reset

PLL &Power Mgmt

SIM

ATA

2 x MMC / SD

PCMCIA / CF

2 x MemoryStick - Pro

USB OTG HS

5 x UART

2 x USB Host

3 x CSPI

Audio Mux

1-Wire

2 x SSI/I2S

Fast IrDA

3 x I2C

VFP

ARM1136 CPU Smart SpeedSwitch (MAX)

ROMPatch ETM

i-cache d-cache L2-cache

3 x Timers

GPIO

RTC

PWM

WD Timer

RAM, ROM

eDMA

Camera I/F

MPEG-4Encoder

Keypad

Blending

Display/TV Ctl

Pre & PostProcessing

Inversion andRotation


Overview of Common Chroma Standards


RGB Format

►Red Green Blue

►RGB565 –

5-bits Red, 6-bits Green, 5-bits Blue (16-bits)

►RGB666 –

6-bits Red, 6-bits Green, 6-bits Blue (18-bits)

►RGB888 –

8-bits / color (24-bits)


YUV

►Y = 0.30R + 0.59G + 0.11B►U = 0.493(B-Y)►V = 0.877(R-Y)

►In humans, there are many more red and green receptors then blue receptors.

►The eyes see green more clearly then either red or blue.


YUV and YCbCr Formats

►Y –

intensity►UV,CbCr

–

Chroma

information

►YUV is the format native to TV broadcast and composite video signals. It separates the brightness information (Y) from the color information (U and V or Cb

and Cr). The color information consists of red and blue color difference signals, this way the green component can be reconstructed by subtracting from the brightness component.

►YCbCr

is the digital version of YUV. Although the two are used interchangeably, there are differences caused primarily of the quantization effect of the digital domain.

►YCbCr

is not a absolute color space. The formulas depend on the specification your are using.


YUV

►Y = 0.30R + 0.59G + 0.11B►U = 0.493*(B-Y)►V = 0.877*(R-Y)

►U = -0.15R –

0.29G + 0.44B►V = 0.62R -

0.52G -

0.1B


YCbCr ( Wikipedia )

►

Y'CbCr

is not an absolute color space. It is a way of encoding RGB information, and the actual color displayed depends on the actual RGB colorants used to display the signal. Therefore a value expressed as Y'CbCr

is only predictable if standard RGB colorants are used, or if an ICC profile is attached or implied which is used to translate value for the colorants in use.

►

Y'CbCr

signals (prior to scaling and offsets to place the signals into

digital form) are called YPbPr, and are created from the corresponding gamma-adjusted RGB (red, green and blue) source using two defined constants Kb and Kr as follows:

►

YPbPr

(analog version of Y'CbCr) from R'G'B'====================================================Y' = Kr * R' + (1 -

Kr -

Kb) * G' + Kb * B'Pb = 0.5 * (B' -

Y') / (1 -

Kb)Pr = 0.5 * (R' -

Y') / (1 -

Kr)....................................................R', G', B' in [0; 1]Y' in [0; 1]Pb in [-0.5; 0.5]Pr in [-0.5; 0.5]

where Kb and Kr are ordinarily derived from the definition of the corresponding RGB space.


YCbCr ( 8 bit 601 )

►YCbCr

(601) from "digital 8-bit R'G'B' “

Y' = 16 + 1/256 * ( 65.738 * R'd

+ 129.057 * G'd

+ 25.064 * B'd)Cb

= 128 + 1/256 * ( -

37.945 * R'd

-

74.494 * G'd

+ 112.439 * B'd)Cr = 128 + 1/256 * ( 112.439 * R'd

-

94.154 * G'd

-

18.285 * B'd)........................................................................R'd, G'd, B'd

in {0, 1, 2, ..., 255}Y' in {16, 17, ..., 235}

with footroom

in {1, 2, ..., 15}headroom in {236, 237, ..., 254}sync. in {0, 255}

Cb, Cr in {16, 17, ..., 240}


Demonstrate Y, Cb, and Cr Components of Image


YCbCr Components

YCr Cb


YCbCr.exe

►Use INC to select either Y, Cr, or Cb

view. The average filter can be turned on or off.

The INC button changes between the views Y, CB, CR

Y Cb Cr


YCbCr.exe

►Use filt

button to turn on or off a averaging filter. Button text indicates CURRENT status of filter.

The filter status is OFF

Click button to turn filter ON.


Thresholding and Histograms


Thresholding

►Thresholding

(or gray-level segmentation) is a conversion from a gray level image to a bi-level (monochrome / black and white) image.

►A 1-bit image (bi-level) should contain all of the essential information concerning the number, position, and shape of objects while containing a lot less information.

►The information of interest is set to 1, the rest is set to 0.


Thresholding

►Thresholding

is done by selecting a value T, and comparing each pixel to that value.

►Pixels in the image I are classified on whether they are greater

then or less then T.

►If I(I,j) > T then the pixel is in class 1

►If I(I,j) < T then the pixel is in class 0


Thresholding►Thresholding

( T = 128 )

Y > 128Cr > 128 Cu > 128


Methods for Selecting T

►Constant threshold•

Assumes constant lighting and image properties (not very practical)

►Computed threshold•

The threshold is computed using pixel data in the image.HistogramUsing edge pixelsIterative


Thresholding Using a Constant

►A value for T is chosen in advanced (possibly at compile time)

►Can only be used if the lighting and image content is tightly controlled (for example a image scanner)

►This method may work on the bench, but is not recommended for real world applications.


Finding the Cone

►Thresholding

is a tool to highlight an object of interest in an image.

►Tracking an object of a particular color is a common computer vision exercise (for example the white or yellow lines on the road).

►By converting the image to a binary image, the background information in a image can be removed.


Base Image


Same Picture Cr > 128

►Notice the cone is highlighted, along with the basketball.

►But the image still contains a bunch of noise.


Thresholding Using Iterative Selection

►A random initial threshold is selected.

►The threshold is dynamically altered until the required characteristic appears in the image.

►This method works very well in a environment with relatively static lighting.

►This method requires a learning process.

►This method can be slow, and does not adapt to dynamic lighting conditions.


Thresholding Using Iterative Selection


Threshholding Using Histograms


Image Histogram

► “The image histogram plots the number of pixels in the image (vertical axis) with a particular brightness value (horizontal axis).”

Wikipedia

►The shape of the histogram can be used to dynamically adjust the

threshold

►This method eliminates the learn procedure required by the iterative method.

# of pixels

256 Gray Levels

Area under curve is sum of all pixels


Histograms of Y Cr Cb

YCr Cb


Using Histogram to Detect the Presence of an Object


Demonstrate the Effects of Threshold


Thresholding (T = 128)

Y > 128Cr > 128 Cu > 128


threshold_Cr_constant.exe

►Binary Cr image with user configurable threshold. The average filter can be turned on or off.

►Use INC button to change the threshold.►The threshold indicator only updates every second.

Threshold

Use INC button to change


threshold_Cr_itterative.exe

►The software learns the threshold via a iterative process. Be sure the cone is not in view while it is calibrating. Notice the threshold changing.


Demonstrate Histograms


RGB_histogram.exe

►Displays RGB histogram. The RED is the number of red pixels in the image. The GREEN is the number of green pixels in the image,

and so on. 0

31graylevels


YCbCr_histogram.exe

►Display YCbCr

histogram. Y is GREEN, Cb

is BLUE, and Cr is RED.


threshold_Cr_histogram.exe

►The threshold is automatically determined based on the image histogram. The algorithm looks for the secondary peak.

►Green bar indicates MAX detected, Blue bar indicates calculated threshold.

No

Cone

Cone

Notice when the cone is detected the system starts tracking it.

Green line in histogram indicates secondary peak

Blue line indicates calculated threshold.


Color Tracking using the IP Camera


Motion Tracking/Detection


Motion Detection► With video, motion is a change between successive video frames.► Motion detection is simply the process of measuring the change between

successive frames.► The simplest method to do this is simply to subtract the previous frame from the

current frame. ► Using simple frame subtraction, with a threshold of 20, the algorithm catches

the author blinking his eyes.► Notice the rest of the image is in the background ( black ) only

the motion is brought to the foreground.


Motion Detection

►

From Computer Vision

by Linda G.Shapiro

and George C.Stockman

►

Let It[r,c] be a monochrome image( Y )►

And It-theta[r,c] be monochrome image taken theta seconds from It►

tau is an intensity threshold►

Iout[r,c] is the binary output image; B is a bounding box

►

Step to detect motion1)

For all pixels [r,c]in

the input imagesa)

Set Iout[r,c] = 1 if( It[r,c]-It-theta[r,c] > tau )b)

Set Iout[r,c] = 0 otherwise2)

Perform connected component extraction on Iout3)

Remove small regions assuming they are noise4)

Perform a closing of Iout

using a small disk to fuse neighboring regions.5)

Compute the bounding boxes of all remaining regions of changed pixels.6)

Return Iout[r,c] and the bounding boxes B of regions of changed pixels.


Motion Detection in English

1)

For all pixels [r,c]in

the input imagesa)

Set Iout[r,c] = 1 if( It[r,c]-It-theta[r,c] > tau )b)

Set Iout[r,c] = 0 otherwise

►

Take 2 consecutive images, from the current image go though each

pixel and subtract the gray scale value from the previous image.

►

If the absolute value of the result is greater then tau then set

the pixel in the output image to 1 ( this will create a binary image ).

It-theta( last frame ) It( current frame ) Binary result



2)

Perform connected component extraction on Iout

►

Connected component extraction associates pixels in a image within groups, and label the groups accordingly.

►

Groups are identified as being surrounded by background, 0 for binary images.

2

1

1

2

0

1

1

1 1

0

0

0 0

0

0 0 0

0

0

0

0

0

2

3

3

3

30 0 0

0 0

0

0

0 0

0 0

0

0

0

0

0

0

0

0

0 3

0

0

0

0

0 0

0

0

0

0

0

0

0

0

0 0

4



3)

Remove small regions, assuming they are noise.

►

Groups containing less then T are removed

►

Setting T to 4 removes regions with less then 4 pixels

1

1

0

1

1

1 1

0

0

0 0

0

0 0 0

0

0

0

0

0

3

3

3

30 0 0

0 0

0

0

0 0

0 0

0

0

0

0

0

0

0

0

0 3

0

0

0

0

0 0

0

0

0

0

0

0

0

0

0 0

0 0 0

0



4)

Compute the bounding boxes of all remaining regions of changed pixels.

5)

Return Iout[r,c] and the bounding boxes B of regions of changed pixels.


Demonstrate Motion Tracking


motion_detecion.exe

► Image subtraction of the Y component. Use the INC button to increment the threshold. A threshold of 1 or 2 works well in most cases. Be patient, the threshold indicator takes a second to display the new threshold.

Noise in image at left is caused by low threshold.

Use INC button to increase threshold to about 2.

Image at right is with threshold of 2.


motion_Cr_detection.exe

►Detect moving cones, suggested threshold is 1. Use INC to increment the threshold. Be patient, the threshold display takes a second to display the new threshold.

No motion

motion

Use INC button to select threshold of 1 or 2.


Image Filtering and Using Convolution Matrices


Image Convolution

►

Convolution is the process of applying a mask to a image.

►

The mask is made up of weights.

►

Each weight is multiplied by the pixel in the image under the mask.

►

The result is summed together.

1 1 1

1 1 1

1 1 1

M =

0

1

2

0 1 2

51 11 71

15 10 81

31 15 91

11 14 18

17 51 41

16 21 61

12 61 10

31 17 41

14 18 15

15 91 71

13 11 19

11 15 91I =

0 1 2 3 4 50

1

2

3

4

5


Image Convolution

1 1 1

1 1 1

1 1 1

M =

-1

0

1

-1 0 1

O[1,1] = I[0,0]*M[-1,-1]

+ I[ 1,0]*M[0,-1]

+ I[2,0]*M[1,-1]

+

I[0,1]*M[-1,0]

+ I[1,1]*M[0,0]

+ I[2,1]*M[1,0]

+

I[0,2]*M[-1,1]

+ I[1,2]*M[0,1]

+ I[2,2]*M[1,1]

51 11 71

15 10 81

31 15 91

11 14 18

17 51 41

16 21 61

12 61 10

31 17 41

14 18 15

15 91 71

13 11 19

11 15 91I =

0 1 2 3 4 5

0

1

2

3

4

5


The Unity Kernel

►The unity kernel performs pixel averaging when used as a convolution mask.

+1 +1►O[r,c] = (Σ Σ ln[r+1,c+j]/9

i=-1j=-1

►This formula defines a unity 3x3 kernel-1 0 1

-1 1 1 1M 0 1 1 1 / 9

1 1 1 1


The Unity Kernel

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 01 0 0 0 0 0 0 0 0 0 0 1 0 5 5 5.4 0 0 0 0 0 0

I 2 0 0 49 0 0 0 0 0 0 0 O 2 0 5 5 5.4 0 0 0 0 0 03 0 0 0 0 0 0 0 0 0 0 3 0 5 5 5.4 0 0 0 0 0 04 0 0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 05 0 0 0 0 0 0 0 0 0 0 5 0 0 0 0 0 0 0 0 0 06 0 0 0 0 0 0 0 0 0 0 6 0 0 0 0 0 0 0 0 0 07 0 0 0 0 0 0 0 0 0 0 7 0 0 0 0 0 0 0 0 0 08 0 0 0 0 0 0 0 0 0 0 8 0 0 0 0 0 0 0 0 0 09 0 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 0

0 = I * M-1 0 1

-1 1 1 1M 0 1 1 1 / 9

1 1 1 1


The Unity Kernel

1 4 7

10

S1

S80

10

20

30

40

50

1 4 7

10

S1

S80

10

20

30

40

50

I -> input Image O -> Output

PixelY


Gaussian Filtering

►G(x,y) = ce^(-(x^2+y^2)/(2t^2)►C –

determines peak at center

►T^2 –

determines the spread

►The Gaussian filter is a image smoothing filter.

►The effect is to take sharp edges out of a image

-1 0 1-1 1 2 1

M 0 2 4 2 / 11 1 2 1


Gaussian Filtering

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 01 0 0 0 0 0 0 0 0 0 0 1 0 10 20 10 0 0 0 0 0 0

I 2 0 0 10 0 0 0 0 0 0 0 O 2 0 20 40 20 0 0 0 0 0 03 0 0 0 0 0 0 0 0 0 0 3 0 10 20 10 0 0 0 0 0 04 0 0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 05 0 0 0 0 0 0 0 0 0 0 5 0 0 0 0 0 0 0 0 0 06 0 0 0 0 0 0 0 0 0 0 6 0 0 0 0 0 0 0 0 0 07 0 0 0 0 0 0 0 0 0 0 7 0 0 0 0 0 0 0 0 0 08 0 0 0 0 0 0 0 0 0 0 8 0 0 0 0 0 0 0 0 0 09 0 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 0

0 = I * M-1 0 1

-1 1 2 1M 0 2 4 2 / 1

1 1 2 1


Gaussian Filtering1 4 7

10

S1

S80

10

20

30

40

50

1 4 7

10

S1

S80

10

20

30

40

50

PixelY

I -> input ImageO -> Output


Edge Detection Using Convolution Matrices


Edge Detection

►Edge detection is used for image segmentation.

►Edge detection is the process of locating the edge pixels, and edge enhancement will increase the contrast between the edges and the

background so that the edges become more visible.


Turn on Your Wayback Machine - Derivatives

►The first derivative is a common and effective mechanism for detecting edges in a picture.

►A edge in a monochrome image is a change in contrast between pixels.


Real Edge

255

0

0

First derivative

infinity


Real Edge

Amplitude is proportional to sharpness of edge.

0First derivative

0

0

255

64

128

192


Prewitt Templates/Matrice/Kernels

-1 0 1Px

= -1 0 1

-1 0 1The template is multiplied by 3x3 portion of the image I.Let x,y

be a pixel in image I

Px

= 1*I[x+1][y-1] + 1*I[x+1][y] + 1*I[x+1][y+1] +-1*I[x-1][y-1] + -1*I[x][y-1] + -1*I[x+1][y-1]


Prewitt Mask or Kernel

0 42 84 126 168 210 252 252

252

2520 0 0

-1 0 1

-1 0 1

-1 0 1

-1 0 1

-1 0 1

-1 0 1

-1 0 1

-1 0 1

-1 0 1

0 0 126 252

-1 0 1

-1 0 1

-1 0 1

-1 0 1

-1 0 1

-1 0 1

252

-1 0 1

-1 0 1

-1 0 1

252

-1 0 1

-1 0 1

-1 0 1

252

-1 0 1

-1 0 1

-1 0 1

252

-1 0 1

-1 0 1

-1 0 1

126

-1 0 1

-1 0 1

-1 0 1

0

-1 0 1

-1 0 1

-1 0 1

0


Prewitt Edge Detection ( x-axis )

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 01 0 0 0 0 0 0 0 0 0 0 1 0 1 3 6 -1 -7 -2 0 0 0

I 2 0 0 1 3 7 2 0 0 0 0 O 2 0 2 6 12 -2 -14 -4 0 0 03 0 0 1 3 7 2 0 0 0 0 3 0 3 9 18 -3 -21 -6 0 0 04 0 0 1 3 7 2 0 0 0 0 4 0 3 9 18 -3 -21 -6 0 0 05 0 0 1 3 7 2 0 0 0 0 5 0 3 9 18 -3 -21 -6 0 0 06 0 0 1 3 7 2 0 0 0 0 6 0 3 9 18 -3 -21 -6 0 0 07 0 0 1 3 7 2 0 0 0 0 7 0 2 6 12 -2 -14 -4 0 0 08 0 0 0 0 0 0 0 0 0 0 8 0 1 3 6 -1 -7 -2 0 0 09 0 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 0

0 = I * M-1 0 1

-1 -1 0 1M 0 -1 0 1 / 1

1 -1 0 1


Prewitt Edge Detection

1 4 7

10

S1

S80

10

20

1 4 7

10

S1

S8-30

-20

-10

0

10

20

PixelY


Prewitt Edge Detection – Wide Edge

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 01 0 0 0 0 0 0 0 0 0 0 1 0 1 3 6 4 0 0 -5 -7 0

I 2 0 0 1 3 7 7 7 7 2 0 O 2 0 2 6 12 8 0 0 -10 -14 03 0 0 1 3 7 7 7 7 2 0 3 0 3 9 18 12 0 0 -15 -21 04 0 0 1 3 7 7 7 7 2 0 4 0 3 9 18 12 0 0 -15 -21 05 0 0 1 3 7 7 7 7 2 0 5 0 3 9 18 12 0 0 -15 -21 06 0 0 1 3 7 7 7 7 2 0 6 0 3 9 18 12 0 0 -15 -21 07 0 0 1 3 7 7 7 7 2 0 7 0 2 6 12 8 0 0 -10 -14 08 0 0 0 0 0 0 0 0 0 0 8 0 1 3 6 4 0 0 -5 -7 09 0 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 0

0 = I * M-1 0 1

-1 -1 0 1M 0 -1 0 1 / 1

1 -1 0 1


Prewitt Edge Detection – Wide Edge

1 4 7

10

S1

S80

10

20

1 4 7

10

S1

S8-30

-20

-10

0

10

20


Demonstrate Prewitt Edge Detection


prewitt_sobel_color.exe

1 4 7

10

S1

S80

10

20

1 4 7

10

S1

S8-30

-20

-10

0

10

20

1 4 7

10

S1

S80

10

20

MAX output of algorithm

MIN output of algorithm

GREEN in image indicates positive results ( slope ),

RED indicates negative results ( slope )

TOP = input edge

BOT = Prewitt result


prewitt_soble_gray.exeThe MIN/MAX indicators are the output of the algorithm.The image is a gray scale ( not a true binary output ).

On the left, a threshold of 2.

On the right a threshold of 11.

Notice the algorithm MAX and MIN.


prewitt_sobel_binary.exe

Use the INC button to adjust the threshold for the binary image.


Sobel Edge Detection



►Sobel

edge detection is done using an X and Y kernel.

►It’s very similar to Prewitt, but includes a Gaussian distribution.

►The complete Sobel

algorithm includes a Gaussian filter before applying the mask’s.

-1 0 1

-2 0 2

-1 0 1

Sx

=

1 2 1

0 0 0

-1 -2 -1

Sy

=


Sobel Edge Detection - Sx

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 01 0 0 0 0 0 0 0 0 0 0 1 0 1 3 6 4 0 0 -5 -7 0

I 2 0 0 1 3 7 7 7 7 2 0 O 2 0 3 9 18 12 0 0 -15 -21 03 0 0 1 3 7 7 7 7 2 0 3 0 4 12 24 16 0 0 -20 -28 04 0 0 1 3 7 7 7 7 2 0 4 0 4 12 24 16 0 0 -20 -28 05 0 0 1 3 7 7 7 7 2 0 5 0 4 12 24 16 0 0 -20 -28 06 0 0 1 3 7 7 7 7 2 0 6 0 4 12 24 16 0 0 -20 -28 07 0 0 1 3 7 7 7 7 2 0 7 0 3 9 18 12 0 0 -15 -21 08 0 0 0 0 0 0 0 0 0 0 8 0 1 3 6 4 0 0 -5 -7 09 0 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 0

0 = I * M-1 0 1

-1 -1 0 1M 0 -2 0 2 / 1

1 -1 0 1


Sobel Edge Detection - Sx

1 4 7

10

S1

S80

10

20

1 4 7

10

S1

S8-30

-20

-10

0

10

20


Sobel Edge Detection - Sy

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 01 0 0 0 0 0 0 0 0 0 0 1 0 -1 -5 -14 -24 -28 -28 -23 -11 0

I 2 0 0 1 3 7 7 7 7 2 0 O 2 0 -1 -5 -14 -24 -28 -28 -23 -11 03 0 0 1 3 7 7 7 7 2 0 3 0 0 0 0 0 0 0 0 0 04 0 0 1 3 7 7 7 7 2 0 4 0 0 0 0 0 0 0 0 0 05 0 0 1 3 7 7 7 7 2 0 5 0 0 0 0 0 0 0 0 0 06 0 0 1 3 7 7 7 7 2 0 6 0 0 0 0 0 0 0 0 0 07 0 0 1 3 7 7 7 7 2 0 7 0 1 5 14 24 28 28 23 11 08 0 0 0 0 0 0 0 0 0 0 8 0 1 5 14 24 28 28 23 11 09 0 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 0

0 = I * M-1 0 1

-1 1 2 1M 0 0 0 0 / 1

1 -1 -2 -1


Sobel Edge Detection - Sy

1 4 7

10

S1

S80

10

20

1 4 7

10

S1

S8-30

-20

-10

0

10

20



►The Sx

and Sy

results are combined by summing the absolute values.

►This results in the negative values becoming positive.►This removes the edge direction information, creating 2 positive

peaks.

1 4 7

10

S1

S8-30

-20

-10

0

10

20

Sx

* I

1 4 7

10

S1

S8-30

-20

-10

0

10

20

Sy

* I


Demonstrate Sobel Edge Detection


prewitt_sobel_color.exe

►RED indicates a negative number out of the algorithm, and GREEN indicates a positive number out of the algorithm. Both PREWITT and SOBEL are supported.

Prewitt on left,

Sobel

on right.

The Sobel

algorithm detect more edges ( it has more gain ).


prewitt_soble_gray.exe

►The PREWITT/SOBEL button selects the algorithm. The name on the

button is the current algorithm running. The MIN/MAX indicators

are the output of the algorithm.

►The image is a gray scale ( not a true binary output ).

Prewitt on left,

Sobel

on right.

The Sobel

algorithm detect more edges ( it has more gain ).


prewitt_sobel_binary.exe

►The MIN/MAX are the output of the algorithm. The display is a binary output, with the threshold being set by the INC button.


Viola Jones Face Detector


Viola-Jones Face Detector

99


Viola-Jones Face Detector

►

The Viola-Jones framework has 2 parts1.

The first part is used to create a “cascade of classifiers”This is done using a learning algorithm called AdaBoostThis learning process is very slow and processor intensiveThe learning process only need to be done once

2.

The second part actually runs in the embedded systemThe image is passed through the “cascade of classifiers”Feature computation is accelerated using a “integral image”Move detector around the image (e.g., 1 pixel increments)Scale the filter, not the image


Cascade of classifiers

“.. a cascade of classifiers which achieves increased detection performance while radically reducing computation time. The key insight is that smaller, and therefore more efficient, boosted classifiers can be constructed which reject many of the negative sub-windows while detecting almost all positive instances. Simpler classifiers are used to reject the majority of sub-windows before more complex classifiers are called upon to achieve low false positive rates.”

Think of a classifier as a filter.

The first classifier is the simplest and weakest, with a false positive rate of about 40%.

Each filter after is more complex, but has a lower false positive rate.


Classifiers are created using features

“More specifically, we use three kinds of features. The value of a two- rectangle feature is the difference between the sum of the pixels within two rectangular regions. The regions have the same size and

shape and are horizontally or vertically adjacent (see Figure 1). A three-rectangle feature computes the sum within two outside rectangles subtracted from the sum in a center rectangle. Finally a four-rectangle feature computes the difference between diagonal pairs of rectangles.”


► Each strong classifier consists of features within the 24 x 24 pixel box (x) :

► type

► size

► location ( x, y coordinates within box )

► sum ( white –

black )

A Strong Classifier is a group of features in a 24 x 24 pixel box

24 pixels

24 pixels

Strong Classifier

h(x) =

1

If pf(x) < pθ

where: p = parity

θ

= threshold

0 otherwise f = feature

h(x) = single feature / weak classifier

Feature /

weak Classifier


AdaBoost Learning Algorithm – Selecting the features


AdaBoost Learning Algorithm – Selecting the features

► “Recall that there are 45,396 rectangle features associated with each image sub-window, a number far larger than the number of pixels. Even though each feature can be computed very efficiently, computing the complete set is prohibitively expensive. Our hypothesis, which is borne out by experiment, is that a very small number of these features can be combined to form an effective classifier. The main challenge is to find these features.”

►“In our system a variant of AdaBoost

is used both to select the features and to train the classifier”


AdaBoost Learning Algorithm

► “The AdaBoost

learning algorithm is used to boost the classification performance of a simple learning algorithm (e.g., it might be used to boost the performance of a simple perceptron). It does this by combining a collection of weak classification functions to form a stronger classifier. In the language of boosting the simple learning algorithm is called a weak learner.”

► “In order for the weak learner to be boosted, it is called upon to solve a sequence of learning problems. After the first round of learning, the examples are re-weighted in order to emphasize those which were incorrectly classified by the previous weak classifier. The final strong classifier takes the form of a perceptron, a weighted combination of weak classifiers followed by a threshold.”


Strong Classifier creation using a variant of AdaBoost


AdaBoost

Where: ά

= weighted value

AdaBoost

uses a iterative process to adjust the weights of each weak classifier to create a strong classifier.

∑ά1

ά2

ά3

h1(x)

h2(x)

h3(x)

f(x)


AdaBoost Example

► Training Data: 71 -> 129 = good(1), everything else is bad(-1)

► Weak Classifier 1: 51 -> 89 = good, everything else is bad




► After Adaboost: •

h(x) = 0.0200046*C1(x) + 0.0191364*C2(x) +0*C3(x)+ 0.0195084*C4(x)

•

51 -> 129 = good(1), everything else bad(-1)


AdaBoost Example

► After Adaboost: •

h(x) = 0.0200046*C1(x) + 0.0191364*C2(x) +0*C3(x)+ 0.0195084*C4(x)

•

51 -> 129 = good(1), everything else bad(-1)

h(60) = 0.020046*(1)+0.0191364*(-1)+0.0195084(1) = 0.020418 = 1


Practical Use Example of Computer Vision


Practical Use Example of Computer Vision

►A practical example of Computer Vision is in the Automotive Market.►Using edge detection, we can filter an image, and highlight the

edges of the road.►This technology was used in the Darpa

Urban Challenge.

Input Image I Sobel

output automatically scaled using a histogram


Eye tracking - eye_tracking.exe

►Put your eye into the box. The click the INC button once. The software will automatically adjust the threshold to highlite

the center of your eye. The it will track your eye.

►The sobel

algorithm works best.►This works best without a strong background light source.


Done


Q&A

►Thank you for attending this presentation. We’ll now take a few moments to review the audience questions, and then we’ll begin the question and answer session.

115

Overview of Computer and Machine VisionComputer Vision References Computer Vision by Linda G....

Documents

Transcript of Overview of Computer and Machine VisionComputer Vision References Computer Vision by Linda G....