Dip lect2-Machine Vision Fundamentals

DDigital igital IImage mage PProcessing & rocessing &

MMachine achine VVision ision Lecture # 2 Lecture # 2

By By

Dr. Abdul Rehman Abbasi Dr. Abdul Rehman Abbasi

Interesting FactInteresting Fact

Vision accounts for about ____ of Vision accounts for about ____ of the data flowing into the human the data flowing into the human

central nervous systemcentral nervous system

70%70%

FeatureFeature Machine VisionMachine Vision Human VisionHuman Vision

Spectral range Gamma rays to microwaves

(10-11 - 10-1 m)

Visible light (4x10-7 -7x10-7m)

Spatial Resolution 4x106 pixels (area scan, growing rapidly), 8192 (line-scan)

Effectively approximately 4000x4000 pixels

Sensor size Small (approx.5x5x15 mm3) Very large

Quantitative Yes. Capable of precise measurement of size, area

No

Ability to cope with unseen events

Poor Good

Performance on repetitive tasks Good Poor, due to fatigue and boredom

Intelligence Low High (subjective)

Light level variability Fixed, closely controlled Highly variable

Light level (min) Equivalent to cloudy moonless night

Quarter-moon light

(Greater if dark-adaptation is extended)

Strobe lighting and lasers Possible (good screening is needed for safety)

Unsafe

Consistency Good Poor

Capital cost Moderate Low

Running cost Low High

FeatureFeature Machine VisionMachine Vision Human VisionHuman Vision

Inspection cost, per unit Low High

Ability to “program” in situ Limited.. Special interfaces make task easier

Speech is effective

Able to cope with multiple views in space and/or time

Versatile Limited

Able to work in toxic, biohazard areas

Yes Not easily

Non-standard scanning methods

Line scan, circular scan, random scan, spiral-scan, radial scan

Not possible

Image storage Good Poor without photography or digital storage

Optical aids Numerous available Limited

Spectral Range Spectral Range

Application features that make your vision system attractive includesApplication features that make your vision system attractive includes

1.1. Inaccessible part (a robot in the way for example)Inaccessible part (a robot in the way for example)

2.2. Hostile manufacturing environmentHostile manufacturing environment

3.3. Possible part damage from physical contactPossible part damage from physical contact

4.4. Need to measure large number of featuresNeed to measure large number of features

5.5. Predictable interaction with lightPredictable interaction with light

6.6. Poor or no visual access to part features of interestsPoor or no visual access to part features of interests

7.7. Extremely poor visibilityExtremely poor visibility

8.8. Mechanical/electrical sensors provide the necessary Mechanical/electrical sensors provide the necessary datadata

Machine Vision ProcessMachine Vision Process

Frame Grabber

Prepro-cessing

Vision Engine

Operation Interface

I/O

Vision software tool and

algorithms

Custom user

Software

Illumination

Image

sensorObject

IMAGE FORMATION IMAGE CAPTURE IMAGE PROCESSING

Data collection

Illumination

Process control

Server

Image

Acquisition

Object

PC

Industrial Manufacturing cell with vision system

Image formation: Image formation:

Right illumination, optical sensor such as high resolution cameras, line scan cameras, Right illumination, optical sensor such as high resolution cameras, line scan cameras, frame grabber . Transformation of the visual image of a physical object and its intrinsic frame grabber . Transformation of the visual image of a physical object and its intrinsic characteristics into set of digitized data that can be used by the image processing unit.characteristics into set of digitized data that can be used by the image processing unit.

Image processing: Image processing:

Image processing consists of image grabbing, image enhancement, feature extraction Image processing consists of image grabbing, image enhancement, feature extraction and output formatting. Function of image processing is to create a new image by and output formatting. Function of image processing is to create a new image by altering the data in such a way that the features of interests are enhanced and the altering the data in such a way that the features of interests are enhanced and the noise is reduced.noise is reduced.

Image analysis:Image analysis:

The main function of image analysis is the automatic extraction of explicit information The main function of image analysis is the automatic extraction of explicit information regarding the content of the image, for example, the shape, size or the range data and regarding the content of the image, for example, the shape, size or the range data and local orientation information from several two dimensional images. It utilizes several local orientation information from several two dimensional images. It utilizes several redundant information representations such as edges, boundaries, disparities shading redundant information representations such as edges, boundaries, disparities shading etc. Most commonly used techniques are: template matching, statistical pattern etc. Most commonly used techniques are: template matching, statistical pattern recognition and the Hough transform.recognition and the Hough transform.

Decision making: Decision making:

Decision making is concerned with making a decision based on the description in the Decision making is concerned with making a decision based on the description in the image and using AI to control the process or taskimage and using AI to control the process or task

Generic Model of Machine Vision Generic Model of Machine Vision

• Scene constraintsScene constraints

• Image acquisitionImage acquisition

• PreprocessingPreprocessing

• SegmentationSegmentation

• Feature ExtractionFeature Extraction

• Classification and/or InterpretationClassification and/or Interpretation

• ActuationActuation

Scene ConstraintsScene Constraints• Scene refers to the environment in which the task is taking place and into Scene refers to the environment in which the task is taking place and into

which the machine ‑vision system is to be placed.which the machine ‑vision system is to be placed.

• The aim of the scene constraint sub‑system is to reduce the complexity of The aim of the scene constraint sub‑system is to reduce the complexity of the subsequent subsystems to a manageable level. This is achieved by the subsequent subsystems to a manageable level. This is achieved by proper exploitation of a priori constraints such as: knowledge of limited proper exploitation of a priori constraints such as: knowledge of limited number of objects possible in the scene, knowledge of their surface finish number of objects possible in the scene, knowledge of their surface finish and appearance etc. We can also impose new constraints such as: and appearance etc. We can also impose new constraints such as: replacement of ambient light with carefully controlled lighting.replacement of ambient light with carefully controlled lighting.

• As is clear from the terminology itself the Scene refers to the industrial As is clear from the terminology itself the Scene refers to the industrial environment in which the manufacturing is being done and the machine environment in which the manufacturing is being done and the machine vision system is to perform the required task in that environment. vision system is to perform the required task in that environment.

• The aim of this module is to reduce the complexity of all the subsequent The aim of this module is to reduce the complexity of all the subsequent sub-systems to a manageable level which is achieved by exploitation of sub-systems to a manageable level which is achieved by exploitation of existing constraints and imposition of new ones.existing constraints and imposition of new ones.

Scene Constraints Scene Constraints Two types of scene constraints can be appliedTwo types of scene constraints can be applied

1.1. Inherent or natural constraintsInherent or natural constraints

2.2. Imposed constraints Imposed constraints

Inherent constraintsInherent constraints

• Characteristics of the materialCharacteristics of the material

• Inherent featuresInherent features

• Limitations, in the range of objectsLimitations, in the range of objects

• Inherent positional limitationsInherent positional limitations

Imposed constraintsImposed constraints

• Control of object featuresControl of object features

• Control of object positionControl of object position

• Control of lighting conditionsControl of lighting conditions

•

Control of Lighting Conditions-1Control of Lighting Conditions-1

Control of Lighting Conditions-2Control of Lighting Conditions-2

Lighting SourcesLighting Sources

• LED illumination units LED illumination units

• Metal halide light sources (“cold light sources” Metal halide light sources (“cold light sources” transmitted over fibre-optic cables)transmitted over fibre-optic cables)

• Laser illumination unitsLaser illumination units

• Fluorescent light (high-frequency)Fluorescent light (high-frequency)

• Halogen lampsHalogen lamps

Light Source Type Advantages DisadvantagesLED Array of light-emitting diodes

Can form many configurations within the arrays; single color source can be useful in some application

Some features hard to see with single color source; large array required to light large area

Fiber-Optic Illuminators Incandescent lamp in housing; light carried by optical fiber bundle to application

Fiber bundles available in many configurations; heat and electrical power remote from application; easy access for lamp replacement

Incandescent lamp has low efficiency, especially for blue light

Fluorescent High-frequency tube or ring lamp

Diffuse source; wide or narrow spectral range available; lamps are efficient and long lived

Limited range of configurations; intensity control not available on some lamps

Strobe Xenon arc strobe lamp, with either direct or fiber bundle light delivery

Freezes rapidly moving parts; high peak illumination intensity

Requires precise timing of light source and image capture electronics. May require eye protection for persons working near the application

Laser Applications Laser Applications

Image Acquisition Image Acquisition

• Translation from the light stimuli falling onto the photo sensors Translation from the light stimuli falling onto the photo sensors of a camera to a stored digital value within the computer’s of a camera to a stored digital value within the computer’s memory. memory.

• Each digitized picture is typically of 512x512 pixels resolution, Each digitized picture is typically of 512x512 pixels resolution, with each pixel representing a binary, grey or color value.with each pixel representing a binary, grey or color value.

• To ensure that no useful information is lost a proper choice of To ensure that no useful information is lost a proper choice of spatial and luminescence resolution parameters must be made.spatial and luminescence resolution parameters must be made.

• Depending on particular application cameras with line scan or Depending on particular application cameras with line scan or area scan elements can be made use of for image acquisition. area scan elements can be made use of for image acquisition.

• While area scan sensors have lower spatial resolution but they While area scan sensors have lower spatial resolution but they provide highly standardized interfacing to computers and do not provide highly standardized interfacing to computers and do not need any relative motion between the object and the camera; need any relative motion between the object and the camera; the line scan sensors need relative motion to build 2-D image.the line scan sensors need relative motion to build 2-D image.

Preprocessing-1Preprocessing-1• To produce a form of the acquired image which is better suited To produce a form of the acquired image which is better suited

for further operations the processes (contrast enhancement, for further operations the processes (contrast enhancement, and adjustment, filtering to remove noise and improve quality) and adjustment, filtering to remove noise and improve quality) modify and prepare pixel values for digitized image. modify and prepare pixel values for digitized image.

• Fundamental information of the image is not changed during Fundamental information of the image is not changed during this module.this module.

• The initially acquired image has direct pixel by pixel relation to The initially acquired image has direct pixel by pixel relation to the original scene and thus lies in the spatial domain. the original scene and thus lies in the spatial domain.

• Transformations from spatial to frequency domain can be done Transformations from spatial to frequency domain can be done using Fourier transforms, which is although not very using Fourier transforms, which is although not very computationally efficient operation. computationally efficient operation.

Preprocessing-2Preprocessing-2

• Low level processing for image improvement such as histogram Low level processing for image improvement such as histogram manipulations (grey level shifting or equalization) involves noisy images manipulations (grey level shifting or equalization) involves noisy images clean up and highlight features of particular interest. clean up and highlight features of particular interest.

• With the use of some transformations pixels are shared among grey levels With the use of some transformations pixels are shared among grey levels which would enhance or alter the appearance of the image.which would enhance or alter the appearance of the image.

• Histogram manipulations provide simple image improvement operations, Histogram manipulations provide simple image improvement operations, either by grey level shifting or, equalization. either by grey level shifting or, equalization.

• An image histogram is easily produced by recording the number of pixels at An image histogram is easily produced by recording the number of pixels at a particular grey level. a particular grey level.

• If this shows a bias towards the lower intensity grey levels, then some If this shows a bias towards the lower intensity grey levels, then some transformation to achieve a more equitable sharing of pixels among the grey transformation to achieve a more equitable sharing of pixels among the grey levels would enhance or alter the appearance of the image. Such levels would enhance or alter the appearance of the image. Such transformations will simply enhance or suppress contrast, and stretch or transformations will simply enhance or suppress contrast, and stretch or compress grey levels, without any alteration in the structural information compress grey levels, without any alteration in the structural information present in the image.present in the image.

Preprocessing-3Preprocessing-3

• Another important class of spatial domain algorithms is Another important class of spatial domain algorithms is designed to perform pixel transformation, whose final value is designed to perform pixel transformation, whose final value is calculated as a function of a group of pixel values (or calculated as a function of a group of pixel values (or 'neighborhood') in some specified spatial location in the original 'neighborhood') in some specified spatial location in the original image. image.

• Many filtering algorithms for smoothing (low pass) and edge Many filtering algorithms for smoothing (low pass) and edge enhancement (high pass) are firmly in this category. enhancement (high pass) are firmly in this category.

• This introduces the basic principle of 'windowing operations' in This introduces the basic principle of 'windowing operations' in which a 2‑D (two‑dimensional) mask, or window, defining the which a 2‑D (two‑dimensional) mask, or window, defining the neighborhood of interest is moved across the image, taking neighborhood of interest is moved across the image, taking each pixel in turn as the centre, and at each position the each pixel in turn as the centre, and at each position the transformed value of the pixel of interest is calculated.transformed value of the pixel of interest is calculated.

Low Pass Filter (5x5 median)Low Pass Filter (5x5 median)

Image Segmentation-1 Image Segmentation-1

• Acquired image is broken up into meaningful regions or segments, i.e. Acquired image is broken up into meaningful regions or segments, i.e. partitioning of image. partitioning of image.

• Segmentation is not concerned with what the image represents. Broadly two Segmentation is not concerned with what the image represents. Broadly two approaches are employed:approaches are employed:

ThresholdingThresholding based on some predetermined criterion ( based on some predetermined criterion (global global thresholds thresholds the entire image into single threshold value or the entire image into single threshold value or local local thresholds partitions thresholds partitions image into sub-images and determines for each of them) and image into sub-images and determines for each of them) and

• Edge-basedEdge-based methods (uses digital versions of standard finite operators methods (uses digital versions of standard finite operators which accentuates intensity changes, which gives rise to peak in the first which accentuates intensity changes, which gives rise to peak in the first derivative or a zero crossing in second derivative, which can be detected derivative or a zero crossing in second derivative, which can be detected and properties such as position, sharpness, and height of peak infer the and properties such as position, sharpness, and height of peak infer the location, sharpness and contrast of intensity changes in the image). Edge location, sharpness and contrast of intensity changes in the image). Edge elements can be used to form the complete boundaries as shown in the elements can be used to form the complete boundaries as shown in the Figure 1.5.Figure 1.5.

Image Segmentation-2Image Segmentation-2

The classical approach to edge‑based segmentation begins with edge The classical approach to edge‑based segmentation begins with edge

enhancement which makes use of digital versions of standard finite enhancement which makes use of digital versions of standard finite difference operators, as in the first‑order gradient operators (e.g. difference operators, as in the first‑order gradient operators (e.g. Roberts Sobel) or in the second‑order Laplacian operator. Roberts Sobel) or in the second‑order Laplacian operator.

The difference operation accentuates intensity changes, and transforms The difference operation accentuates intensity changes, and transforms this the image into a representation from which properties of these this the image into a representation from which properties of these changes can be extracted more easily. changes can be extracted more easily.

A significant intensity change gives rise to a peak in the first derivative or A significant intensity change gives rise to a peak in the first derivative or a zero crossing in the second derivative of the smoothed intensities. a zero crossing in the second derivative of the smoothed intensities.

These peaks, or zero crossings, can be detected easily, and properties These peaks, or zero crossings, can be detected easily, and properties such as the position, sharpness, and height of the peaks infer the such as the position, sharpness, and height of the peaks infer the location, sharpness and contrast of the intensity changes in the image. location, sharpness and contrast of the intensity changes in the image.

Edge elements can be identified from the edge‑enhanced image and Edge elements can be identified from the edge‑enhanced image and these can then be linked to form complete boundaries of the regions of these can then be linked to form complete boundaries of the regions of interestinterest

Feature Extraction Feature Extraction

• During this phase the inherent characteristics or During this phase the inherent characteristics or features of different regions within the image are features of different regions within the image are identified, which are checked against predetermined identified, which are checked against predetermined standards. standards.

• This description should be invariant to position, This description should be invariant to position, orientation and scale of the object. orientation and scale of the object.

• A number of basic parameters such as minimum A number of basic parameters such as minimum enclosing rectangle, centre of area (e.g. centre may be enclosing rectangle, centre of area (e.g. centre may be considered as object oriented origin and series of considered as object oriented origin and series of feature descriptors can be developed), may be derived feature descriptors can be developed), may be derived from an arbitrary shape and can be used for from an arbitrary shape and can be used for classification and position information classification and position information

Image Classification (Analysis) Image Classification (Analysis)

• The classification sub-system is concerned with pattern recognition or image classification. This The classification sub-system is concerned with pattern recognition or image classification. This process utilizes some or all of the extracted features to make a decision about to which process utilizes some or all of the extracted features to make a decision about to which category of objects the unknown object belongs. category of objects the unknown object belongs.

• There are three main techniques for classification There are three main techniques for classification • Template matching Template matching • Statistically based approaches Statistically based approaches • Neural network approach Neural network approach

• Template matching is used in situations where the objects to be identified have well defined and Template matching is used in situations where the objects to be identified have well defined and highly 'differentiated’ features, for example standard alphanumeric character fonts. In such highly 'differentiated’ features, for example standard alphanumeric character fonts. In such cases an unknown character is compared with a set of templates or masks, each of which fits cases an unknown character is compared with a set of templates or masks, each of which fits just one character uniquely. just one character uniquely.

• Statistical techniques can be selected to provide optimum classification performance for more Statistical techniques can be selected to provide optimum classification performance for more varied industrial applications. varied industrial applications.

• If the vision task is well constrained then classification may be made via a simple tree If the vision task is well constrained then classification may be made via a simple tree searching algorithm where classification proceeds by making branching choices on the basis of searching algorithm where classification proceeds by making branching choices on the basis of single feature parameters. In more complex cases, n features are combined to create a 'feature single feature parameters. In more complex cases, n features are combined to create a 'feature vector' which places a candidate object within the n-dimensional feature space. Provided that vector' which places a candidate object within the n-dimensional feature space. Provided that the features have been properly chosen to divide the allowable range of candidate objects into the features have been properly chosen to divide the allowable range of candidate objects into well separated 'clusters', then classification merely consists of dividing the space with one or well separated 'clusters', then classification merely consists of dividing the space with one or more 'decision surfaces', such that each decision surface reliably separates two clusters.more 'decision surfaces', such that each decision surface reliably separates two clusters.

Industrial Vision: Image acquisitionIndustrial Vision: Image acquisition

CCD cameraCCD camera

DigitalizationDigitalization

Data acquisition cardsData acquisition cards

Vision software Vision software

Cameras and sensors; lensesCameras and sensors; lenses

Image acquisition: CCD cameraImage acquisition: CCD camera

GatesGates

Photon

SiliconSilicon

ChargesCharges

PotentialPotentialWellWell

Image acquisition: CCD sensorImage acquisition: CCD sensor

Image acquisition: digitalisationImage acquisition: digitalisation

AD

8 BitGreyvalueVolts

0...255

Greyvalue

Volts

0.7

255

0.348

127

brighter

DigitalisationDigitalisation

camera image 8 bit grayscale digital image

Pixel mask

Pixel = Picture Element

DigitalisationDigitalisation

Grayscale image& numeric representation

255 255 255 255 253 88 74 73 72 72 75 175 255 255 255

255 255 255 255 250 82 75 74 73 74 73 190 255 255 255

255 255 255 255 231 80 73 72 72 72 76 197 255 255 255

255 255 255 255 232 83 73 73 73 76 75 172 255 255 255

255 255 255 255 226 79 75 74 74 76 75 184 255 255 255

255 255 255 255 220 84 75 73 76 79 74 159 255 255 255

255 255 255 255 224 83 76 74 77 75 75 156 255 255 255

255 255 255 255 207 90 75 76 78 77 81 172 255 255 255

255 255 255 255 252 107 75 75 80 79 79 162 255 255 255

255 255 255 255 249 136 77 76 89 81 99 217 255 255 255

255 255 255 255 255 183 78 75 80 81 120 248 255 255 255

255 255 255 255 255 249 86 76 74 84 201 255 255 255 255

255 255 255 255 255 255 115 77 77 98 251 255 255 255 255

255 255 255 255 255 255 193 80 78 143 255 255 255 255 255

255 255 255 255 255 255 217 85 78 173 255 255 255 255 255

255 255 255 255 255 255 248 97 79 220 255 255 255 255 255

255 255 255 255 255 255 255 119 80 224 255 255 255 255 255

255 255 255 255 255 255 255 255 255 255 255 255 255 255 255

255 255 255 255 255 255 255 255 255 255 255 255 255 255 255

255 255 255 255 255 255 255 255 255 255 255 255 255 255 255

255 255 255 255 255 255 255 255 255 255 255 255 255 255 255

255 255 255 255 255 255 255 255 255 255 255 255 255 255 255

Video sourcesVideo sources

The video source can be:The video source can be:• Video cameraVideo camera• CamcorderCamcorder• Video recorder (VCR)Video recorder (VCR)• Television broadcastsTelevision broadcasts• X-ray equipmentX-ray equipment• Scanning Electron Microscope (SEM)Scanning Electron Microscope (SEM)• CT scannerCT scanner

Composite video = signal containing bothComposite video = signal containing both video data (luminance + colour) video data (luminance + colour) and the timing and the timing (synchronisation)(synchronisation) information. It is the standard which information. It is the standard which interconnects almost all video equipment (TVs, laserdisc, videorecorders, interconnects almost all video equipment (TVs, laserdisc, videorecorders, camcorders) at home.camcorders) at home.

Examples of composite video standards: Examples of composite video standards: • RS-170:RS-170:

• used in North America and Japanused in North America and Japan• Monochrome signalMonochrome signal• Spatial resolution: 640 pixels x 480 linesSpatial resolution: 640 pixels x 480 lines• Frequency: 60 fields/second (equivalent to 30 frames/second)Frequency: 60 fields/second (equivalent to 30 frames/second)

• NTSC/RS-330NTSC/RS-330• used in North America and Japanused in North America and Japan• Equivalent to RS-170 but colour information is superimposed on the Equivalent to RS-170 but colour information is superimposed on the

monochrome signal.monochrome signal.• NTSC = National Television System CommitteeNTSC = National Television System Committee

Signal types for Image acquisition boardsSignal types for Image acquisition boards


More composite video standards: More composite video standards: • CCIRCCIR

• used in Northern Europeused in Northern Europe• Monochrome signalMonochrome signal• Spatial resolution: 768 pixels x 576 linesSpatial resolution: 768 pixels x 576 lines• Frequency: 50 fields/second (equivalent to 25 frames/second)Frequency: 50 fields/second (equivalent to 25 frames/second)• CCIR = Comité Consultatif International RadioCCIR = Comité Consultatif International Radio

• PALPAL• used in Northern Europeused in Northern Europe• Equivalent to CCIR but colour information is superimposed on the Equivalent to CCIR but colour information is superimposed on the

monochrome signal.monochrome signal.• PAL = Phase Alteration LinePAL = Phase Alteration Line

• SECAMSECAM• used in France, Russia and the Sovjet Republic Statesused in France, Russia and the Sovjet Republic States• Equivalent to CCIR but colour information is superimposed on the Equivalent to CCIR but colour information is superimposed on the

monochrome signal.monochrome signal.• SECAM = Séquencial Couleur Avec MemoireSECAM = Séquencial Couleur Avec Memoire


S-Video (also called Y/C video): luminance (Y) and chrominance (C) are separate S-Video (also called Y/C video): luminance (Y) and chrominance (C) are separate signals. The Y signal contains timing (synchronisation) information. S-video can be signals. The Y signal contains timing (synchronisation) information. S-video can be transported over 4 pin mini DIN connector, or over SCART connector.transported over 4 pin mini DIN connector, or over SCART connector.

Some image sources produce “nonstandard” video signals:Some image sources produce “nonstandard” video signals:• Video and timing information can vary in format as well as in single or multiple Video and timing information can vary in format as well as in single or multiple

signals. They do not adhere to particular spatial resolutions, signal timing signals. They do not adhere to particular spatial resolutions, signal timing schemes, signal characteristics … Consult the documentation provided with schemes, signal characteristics … Consult the documentation provided with your video source.your video source.

Progressive scan (25-30 frames/sec) cameras produce non interlaced signals.Progressive scan (25-30 frames/sec) cameras produce non interlaced signals.

All previous camera signals are All previous camera signals are analogueanalogue..

DIGITAL CAMERAS: DIGITAL CAMERAS: No frame grabber required!No frame grabber required!• Cameras with FireWire (IEEE 1394) interface. Cameras with FireWire (IEEE 1394) interface.

Supported by Apple, Windows XPSupported by Apple, Windows XP• Cameras with USB interfaceCameras with USB interface

FugaFuga

AllegroAllegro

Image acquisition boardsImage acquisition boards• The video capture device is often called The video capture device is often called frame grabberframe grabber card. card.• Frame grabber puts a pixel mask over the image: the card converts the Frame grabber puts a pixel mask over the image: the card converts the

analogue image (or images) supplied by a video source into a digital array analogue image (or images) supplied by a video source into a digital array (or arrays) of data points.(or arrays) of data points.

• It is a plug in card (PCI) with It is a plug in card (PCI) with AD AD convertor. The ADC must have video convertor. The ADC must have video speed: 20 MHz or higher (30 or 25 video frames per second, 300 kB [640 x speed: 20 MHz or higher (30 or 25 video frames per second, 300 kB [640 x 480 x 8 bit] per frame.480 x 8 bit] per frame.

• Other features: Other features: • input multiplexer (to select one of the 4 inputs)input multiplexer (to select one of the 4 inputs)• Colour notch filter = chrominance filter (to acquire monochrome signals Colour notch filter = chrominance filter (to acquire monochrome signals

from colour sources)from colour sources)• Programmable gain stage (to match the signal into the ADC input Programmable gain stage (to match the signal into the ADC input

range)range)• Timing and acquisition control (to synchronise grabbing with sync Timing and acquisition control (to synchronise grabbing with sync

pulses of incoming signal: PLL or Digital Clock Synchronisation)pulses of incoming signal: PLL or Digital Clock Synchronisation)• Camera control stage (to send to the camera or to receive from the Camera control stage (to send to the camera or to receive from the

camera setup and control signals, e.g. horizontal and vertical sync camera setup and control signals, e.g. horizontal and vertical sync signals, pixel clock and reset signals)signals, pixel clock and reset signals)

• Most cards provide digital I/O for input or output operations, to Most cards provide digital I/O for input or output operations, to communicate with external digital devices (e.g. industrial process). This communicate with external digital devices (e.g. industrial process). This saves a separate I/O board.saves a separate I/O board.

Image acquisition boardsImage acquisition boards (continued) (continued)• Plug-in cards (image grabber, frame grabber card) for analogue camerasPlug-in cards (image grabber, frame grabber card) for analogue cameras

• Are plugged in at a VME or PCI busAre plugged in at a VME or PCI bus• Are delivered with Windows 98 or NT driversAre delivered with Windows 98 or NT drivers• Accept cameras according to the EIA (30 frames/sec) or CCIR (25) Accept cameras according to the EIA (30 frames/sec) or CCIR (25)

standardsstandards• Good cards have their own processor (DMA data transfer to PC) and Good cards have their own processor (DMA data transfer to PC) and

large RAMlarge RAM• Others (cheaper ones) use the PC processorOthers (cheaper ones) use the PC processor• They accept the signals: S video, composite video TV or VCR signals They accept the signals: S video, composite video TV or VCR signals

(NTSC/PAL/Secam)(NTSC/PAL/Secam)• Some cards have camera control outputSome cards have camera control output

Image acquisition - CamerasImage acquisition - Cameras

• Sensor types:Sensor types:• LineLine• ArrayArray

• Interface standardsInterface standards::• CCIR / RS-170 (B&W, 50-60 fields/sec.)CCIR / RS-170 (B&W, 50-60 fields/sec.)• PAL / SECAM / NTSC (Colour)PAL / SECAM / NTSC (Colour)• Progressive scan (25-30 frames/sec.)Progressive scan (25-30 frames/sec.)• FireWire (IEEE 1394)FireWire (IEEE 1394)• USBUSB

• Sensor technology:Sensor technology:• CCD (Charge Coupled Device)CCD (Charge Coupled Device)• CMOS (Complementary Metal Oxide CMOS (Complementary Metal Oxide

Semiconductor). A CMOS camera produces a Semiconductor). A CMOS camera produces a 1000*1000 pixel image1000*1000 pixel image

Spatial resolutionSpatial resolution• The number of rows (N) from a video source generally corresponds The number of rows (N) from a video source generally corresponds

one-to-one with lines in the video image. The number of columns, one-to-one with lines in the video image. The number of columns, however, depends on the nature of the electronics that is used to however, depends on the nature of the electronics that is used to digitize the image. Different frame grabbers for the same video camera digitize the image. Different frame grabbers for the same video camera might produce M = 384, 512, or 768 columns (pixels) per line. might produce M = 384, 512, or 768 columns (pixels) per line.

• a CCIR / PAL image source can result in max 768 x 576 pixel imagea CCIR / PAL image source can result in max 768 x 576 pixel image

• a RS-170 / NTSC source can result in max 640 x 480 pixel imagea RS-170 / NTSC source can result in max 640 x 480 pixel image

• Depending on video source or camera used, the spatial resolution can Depending on video source or camera used, the spatial resolution can range from 256 x 256 up to 4096 x 4096.range from 256 x 256 up to 4096 x 4096.

• Most applications use only the spatial resolution required. For fast Most applications use only the spatial resolution required. For fast image transfer and manipulation, often 512 x 512 is used. For more image transfer and manipulation, often 512 x 512 is used. For more accurate image processing, 1024 x 1024 is common.accurate image processing, 1024 x 1024 is common.

• The pixel aspect ratio (pixel width : pixel height) can be different from The pixel aspect ratio (pixel width : pixel height) can be different from 1:1, typical 4:3. Some frame grabbers don’t convert video data into 1:1, typical 4:3. Some frame grabbers don’t convert video data into square pixels but into rectangle ones. This creates the effect of a circle square pixels but into rectangle ones. This creates the effect of a circle appearing ovular, and squares appearing as rectangles.appearing ovular, and squares appearing as rectangles.

Spatial resolutionSpatial resolution• Example 768 x 512 (aspect ratio 3 : 2)

Brightness resolutionBrightness resolution• Brightness resolution = bit depth resolution: number of gray levels Brightness resolution = bit depth resolution: number of gray levels

(monochrome) or number of colours(monochrome) or number of colours

• RS-170 / NTSC image: 8 bits = 256 gray levelsRS-170 / NTSC image: 8 bits = 256 gray levels

• A standard RS-170 image is 307 kB large: 640 x 480 x 8bit.A standard RS-170 image is 307 kB large: 640 x 480 x 8bit.

768 pixels

512 rows

2

3

520 rows max CCIR

Interlaced / non interlaced formatsInterlaced / non interlaced formats• A video signal consists of a series of lines. Horizontal sync pulses

separe the lines from each other.

• All composite video sources (RS-170/NTSC, CCIR/PAL) and some nonstandard video sources transmit the lines in interlaced format: first the odd (first field), afterwards the even lines (second field).

• Vertical sync pulses separate the fields from each other.

• Some nonstandard video sources transmit the lines in non-interlaced format = progressive scan. Only one field, containing all the lines, is transmitted.

• Progressive scan is recommended for fast moving images.

• If one is planning to use images that have been scanned from an interlaced video source, it is important to know if the two half-images have been appropriately "shuffled" by the digitization hardware or if that should be implemented in software. Further, the analysis of moving objects requires special care with interlaced video to avoid "zigzag" edges.

Photo Sensors Photo Sensors

Dip lect2-Machine Vision Fundamentals

Engineering

Transcript of Dip lect2-Machine Vision Fundamentals