UNIVERSITI TEKNIKAL MALAYSIA MELAKA

VISION INSPECTION SYSTEM FOR

PRODUCT DEFECT

Thesis submitted in a accordance with the requirements of the

Universiti Teknikal Malaysia Melaka for the Degree of

Bachelor of Manufacturing Engineering (Robotics & Automation) with Honours

By

SYAHRIL ANUAR BIN IDRIS

B050410219

Faculty of Manufacturing Engineering

May 2008

APPROVAL

This thesis submitted to the senate of UTeM and has been accepted as partial fulfillment

of the requirements for the degree of Bachelor of Manufacturing Engineering (Robotic

and Automation) with Honours. The members of the supervisory committee are as

follow:

..

Supervisor

DECLARATION

I hereby, declare this thesis entitled Vision Inspection System for Product Defect is the

results of my own research except as cited in the reference.

Signature : .

Authors Name :

Date :

i

ABSTRACT

Vision system is one of the most advance systems in industrial automation to replace

conventional method of inspection done by human operator. Visual inspection requires

high-speed, high-magnification, 24-hour operation, and repeatability of measurements.

All of these tasks extend roles traditionally occupied by human beings, whose degree of

failure is classically high through distraction, illness and circumstance.The use of vision

to enhance the operation of an automated work cell has moved from research laboratory

to the factory floor. In order to design the vision inspection system, lots of aspect should

be taken into consideration. This includes tools used, the environment, the materials and

most important things are the purpose of the inspection. The pharmaceutical industry

relies heavily on automated process-control and quality-assurance systems to ensure that

batch production is carried out repeatable, reliably, and accurately. Thus vision inspection

system for pharmaceutical industry requires critical consideration in develop automated

inspection system. This project focuses on the development of vision inspection system

to detect product defect for pharmaceutical product.

ii

ABSTRAK

Sistem penglihatan adalah salah satu sistem maju dalam automasi industri untuk

mengantikan pemeriksaan cara lama yang di lakukan oleh operator manusia. Pemeriksaan

mengunakan penglihatan memerlukan kelajuan, ketelitian, berkerja 24 jam dan

pengukuran berulang-ulang. Kerja-kerja ini melebihi tahap kerja yang di lakukan oleh

manusia, yang mempunyai kecuaian yang tinggi kalau di ganggu, sakit dan sebagainya.

Pengunaan sistem penglihatan untuk memperbaiki operasi sel kerja automatik yang

sebelum ini hanya di gunakan dalam makmal perlu dipindahkan untuk digunakan dalam

kilang. Untuk mereka sistem pemerikasaan menggunakan penglihatan, banyak aspek

yang perlu di ambil kira. Ini termasuk penggunaaan alatan, keadaan sekeliling, bahan

yang digunakan dan yang paling penting tujuan pemeriksaan. Industri perubatan

bergantung pada proses kawalan automatik dan sistem kepastiaan kualiti untuk

memastikan produksi berjalan lancar. Oleh sebab itu sistem pemeriksaan mengunakan

penglihatan untuk industri perubatan memerlukan pertimbangan yang kritikal dalam

menghasilkan sistem pemeriksaan automatik. Projek ini memberi fokus kepada

penghasilan sistem pemeriksaan menggunakan penglihatan untuk mengesan kerosakan

pada produk perubatan.

iii

ACKNOWLEDGEMENTS

Alhamdullillah, Im grateful that by the power of Allah, Most Gracious, Most Merciful, I

able to complete this project. My thank goes to my parents, who taught me the value of

hard work by their own example, who support me both on financial and inspiration. They

are my source of inspiration that lead me to working hard in gaining knowledge. Never to

forget, to thanks all of my brothers and sisters, because without them I would not be able

to finish my project.

I also would like to thank a few people who have made this project a success, for without

them, it would be possible to achieve what I have right now. Firstly, I would like to thank

my supervisor, Muhammad Hafidz Fazli Bin Md. Fauadi who never ever endlessly to

support me, gives opinion, share information and even sacrifice his time just to help me

finish this project. . I wish to express my honest thanks to him for the support during the

whole project. The encouragement and enthusiasm that was given to me to carry out my

project work greatly appreciated. Not to forget other lecturers of BMFA that helps me in

gaining the information which helps me in completing the project. The effort shows by

the lecturers really touch my heart.

Finally, I would like to thank all whos directly and indirectly helped me completing my

thesis in time.

Thank you, as I really appreciate all the helps I gains...

iv

TABLE OF CONTENTS

Abstract i

Acknowledgement iii

Table of Contents iv

List of Figures viii

List of Tables xi

List of Abbreviations, Symbols, Specialized Nomenclature xii

1. INTRODUCTION

1.1. Backgrounds 1

1.2. Project Overview 3

1.3. Problem Statement 3

1.4. Objectives 4

1.5. Scopes 5

2. LITERATURES REVIEW

2.1. Vision System 6

2.1.1. Human Vision vs Machine Vision 8

2.1.2. Machine Vision Fundamentals 11

2.1.3. Machine Vision Components 15

2.1.3.1.Camera 16

2.1.3.2.Lighting System 17

2.1.3.3.Part Sensor 20

2.1.3.4.Frame Grabber 21

2.1.3.5.Camera Controller/ PC Platform 21

2.1.3.6.Inspection Software 22

2.1.3.7.Interfaced Circuits 22

2.1.4. Pixels 22

2.1.5. Resolution 24

v

2.1.6. Brightness Adaptation and Discrimination (Contrast) 25

2.2. Machine Vision System 26

2.2.1. Machine Vision Application 28

2.3. Vision Inspection System 29

2.3.1. Design of Vision Inspection System 29

2.3.1.1.Determine Inspection Goals 30

2.3.1.2.Estimation of the Inspection Time 31

2.3.1.3.Identification of Features or Defects 32

2.3.2. Optimizing The Performance of Vision System 33

2.3.2.1.Selection of Lighting and Material Handling Technique 33

2.3.2.2.Selection of Optics 34

2.3.2.3.Selection of Image Acquisition Hardware 35

2.3.2.4.Development of Strategy 36

2.3.2.5.Calibration and Testing Inspection Strategy 36

2.3.2.6.Development of Operator Interface (GUI or HMI) 37

2.4. Inspection & Quality Control/ Quality Assurance 37

2.4.1. Type of Inspection 39

2.5. Pharmaceutical Industry 40

2.5.1. Vision System in Pharmaceutical Industry 41

3. METHODOLOGY

3.1. Introduction 44

3.1.1. Introduction 46

3.1.2. Literature Review 46

3.1.3. Methodology 46

3.1.4. Result 46

3.1.5. Discussion 47

3.1.6. Summary and Conclusions 47

3.2. Project Design 47

3.2.1. Problem Statement 47

3.2.2. Study and Research 48

vi

3.2.2.1.Internet 48

3.2.2.2.Books, Manuals & Magazines 49

3.2.2.3.Journals & Articles 49

3.2.3. Selection & Analysis Tools 49

3.2.3.1.Product Selection 49

3.2.3.2.Programming Software Selection 50

3.2.3.3.Camera Selection 53

3.2.4. Categorized Product Defect 54

3.2.5. Design & Development Inspection System 54

3.3. Project Tools & Equipment 55

3.3.1. Personal Computer 55

3.3.2. USB WebCam CLIPTEC FLAXCAM ZW 811 56

3.3.3. MATLAB 7.0.4. Software 57

3.3.4. Product 58

4. DESIGN & DEVELOMENT

4.1. Designing Inspection Platform 59

4.2. Types of Defects 60

4.3. Design Image Acquisition 62

4.4. Design GUI 65

4.5. Image Processing Method 67

4.5.1. Edge Detection Operators 67

4.5.2. Threshold Grayscale 69

5. RESULT & ANALYSIS

5.1. Result 70

5.1.1. Improve Inspection Platform 70

5.1.2. Image Enhancing 71

5.1.3. Image Processing 74

5.1.4. Image Interpretation 78

5.1.5. Display Result 81

vii

5.2. Final Outcome Evaluations 82

6. CONCLUSIONS

6.1. Discussions 83

6.1.1. Environment Control 83

6.1.2. Value of Final Result 85

6.1.3. Performance of Camera 86

6.2. Conclusions 88

6.3. Future Work 89

REFERENCES

APPENDICES

A. Checklist form to design Vision Inspection System

B. Mechanical drawings of Product (Medical Container STREPSILS Cough

Syrup)

C. Mechanical drawings of Inspection Platform

viii

LIST OF FIGURES

1.1 Vision Inspection System Design 2

2.1 Vision System Field 7

2.2 Structure of Human Eye 8

2.3 Operation of Machine Vision 11

2.4 Edge Detection Route 14

2.5 Basic Machine Vision Components 16

2.6 Variances of lighting components 17

2.7 Front Lighting 18

2.8 Back Lighting 19

2.9 Structured Lighting 20

2.10 Frame Grabber attach to PC 21

2.11 Ways of Reconstructing an Image 23

2.12 The Contrast of Black and White from Imaging Pixels 25

2.13 Machine Vision System Use for Manufacturing 26

2.14 Field of View for Videotape Inspection 34

2.15 Inspection of Product 38

2.16 Inspection of Medical Container 40

2.17 Vision Inspection System Use in Production Line of Medical Container 42

3.1 Flow Chart for Design &Development of vision Inspection System 45

3.2 Personal Computer/ Camera Controller 55

3.3 Web Cam 56

3.4 Product 58

ix

4.1 Inspection Platforms 59

4.2 Sample Product 60

4.3 Crack Defect 61

4.4 Cap Defect 61

4.5 Size Defect 61

4.6 Acquire Device Data 62

4.7 Input of Camera 63

4.8 Create Preview Window 63

4.9 Call Preview Window 63

4.10 Actual Product 64

4.11 Defect Product (Crack) 64

4.12 GUIDE Layout Editor 65

4.13 Image Processing Interface 66

4.14 Sobel Matlab Syntax 67

4.15 Canny Matlab Syntax 68

4.16 Threshold Matlab Syntax 69

5.1 Platform of Inspection using Solidworks 70

5.2 RGB to Gray Image Processing Syntax 71

5.3 Syntax for Windowing 71

5.4 Windowing According to Defect 72

5.5 Imtool Overview 72

5.6 Filtering Syntax 73

5.7 Filtering on Crack Window 73

5.8 Filtering on Cap Window 73

5.9 Filtering on Size Window 74

5.10 Crack Defect Using Canny Method 74

5.11 Result of Edge Detection of Canny Operator for Crack Defect 75

5.12 Sobel Matlab Operator 75

5.13 Result of Edge Detection of Sobel Operator for Cap Defect 76

5.14 Matlab Syntax for Thresholding 76

x

5.15 Sample Image 77

5.16 No Size Defect Image 77

5.17 No Size Defect But With Cap Defect 77

5.18 Images with Size Defect 78

5.19 Syntax for Counting White Pixel of Sample Image 79

5.20 Syntax for Counting White Pixels for Real Time Image 80

5.21 Syntax for Compare Sample Image with Real Time Image 80

5.22 Result of Evaluation 81

5.23 Syntax for Display Defect Types 81

6.1 Effect of Glare Due To Glass Reflection 84

6.2 Image Enhancing To Remove Glare 84

6.3 Image for Size Defect 86

6.4 Crack Defect Captured by Webcam 87

xi

LIST OF TABLE

2.1 Evaluation of capabilities 10

2.2 Evaluation of Camera 14

3.1 Selection of Image Processing Software 48

5.1 Final Evaluation for 3 types of samples 82

xii

LIST OF ABBREVIATIONS, SYMBOLS, SPECIALIZED

NOMENCLATURE

ADC - Analog to Digital Converter

AI - Automated Inspection

BGA - Ball Grid Array

CCD - Couple Charged Device

CMOS - Complementary Metal Oxide Semiconductor

CRT - Cathode Ray Tube

DOF - Depth of Field

DPI - Dots per Inch

FDA - Food and Drug Administration

FOV - Field of View

GMPs - Good Manufacturing Practices

GUI - Graphic User Interface

HMI - Human Machine Interface

I/O - Input / Output

LCD - Liquid Crystal Display

LED - Light Emitting Diode

MMX - Matrix Math Extensions

MTF - Modulation Transfer Function

MV - Machine Vision

NG - Not Good

OK - Okay

PC - Personal Computer

PCI - Peripheral Component Interconnect

PPI - Pixel per Inch

RGB - Red Green Blue

USB - Universal Serial Bus

VGA - Video Graphic Accelerator

1

CHAPTER 1

INTRODUCTION

1.1 Backgrounds

The use of vision to enhance the operation of an automated work cell has moved from

research laboratory to the factory floor. Vision systems can be programmed to perform

narrowly defined tasks such as counting objects on a conveyor, reading serial numbers,

and searching for surface defects. Vision systems are being used increasingly with robot

automation to perform the following task:

Part identification: Commercially available vision system store data for different

parts in active memory and use the data to distinguish between parts as they enter

the work cell.

Part location: Vision technology allows the user to locate randomly placed parts

on an X-Y grid. The vision system measures the X and Y distances from center of

the camera coordinate system to the center of the randomly placed part.

Part orientation: Every part must be gripped in a specified manner by the end of

arm tooling. The vision system supplies the orientation information and data that

are used to drive the gripper into the correct orientation for part pickup

Part inspection: Vision systems are used to check parts for dimensional accuracy

and geometrical integrity.

Range Findings: In some applications the system uses two or more cameras to

measure the X, Y, Z location of the part.

2

Manufacturers usually used vision systems for visual inspections. Visual inspection

systems are used to check parts for dimensional accuracy and geometrical integrity.

These parts are measured by camera and the dimensions are calculated by the image

processing program; at the same time, the vision system checks the parts for any missing

holes or changes in the part geometry (Asfahl, 1992). Data collect from the vision system

will be analyzed and the defect of the product can be categorized. Refer Figure 1.1 for the

communication of vision inspection system.

Figure 1.1: Vision Inspection System Design (Hardin, 2000)

An inspection is, most generally, an organized examination or formal evaluation exercise.

It involves the measurements, tests, and gauges applied to certain characteristics in

regards to an object or activity. The results are usually compared to specified

requirements and standards for determining whether the item or activity is in line with

these targets.

Machine Interface Vision Processing Operator Interface

3

1.2 Project Overview

This project will be focusing on inspections system using the vision sensor. The product

to demonstrate the inspection system is a medical glass bottle container that used to fill

the cough syrup. The product was chosen due to the high quality inspection prefer for this

kind of product. Using the available programming software to programming the

inspections system, this project will result in the suitable program for inspections defect

on the medical glass bottle container.

1.3 Problem Statement

In manufacturing process, there will always be a defect in the product. Even though this

defect is only 1% from the batch, it is still a defect. For instance, when the product that

needs to be exported has slightly 1% of defect, the manufacturing company is putting

their profits in line where a great loss to the company can happen due to this 1% of

defect. Worst, the company has to bear other consequences such as lost of clients and

even being sued for negligence. Therefore this defect should not reach the user, as the last

end product should be 100% perfect. Where, the defect can be solved using the visual

inspection, ultra wave inspection and other type of inspection.

This project has chosen the medical containers as the product. This is because the

pharmaceutical industry relies heavily on automated process-control and quality-

assurance systems to ensure that batch production is carried out repeatable, reliably, and

accurately. Moreover, there is probably no other industry where proper quality assurance

and quality control can literally make the difference between cure and casualty

(Haystead, 1997).

4

Visual inspection requires high-speed, high-magnification, 24-hour operation, and

repeatability of measurements. All of these tasks extend roles traditionally occupied by

human beings, whose degree of failure is classically high through distraction, illness and

circumstance. Although humans may display finer perception over the short period and

greater flexibility in classification and adaptation to new defects and quality assurance

policies, but they make error and can not be hoped 100% perfect with their job (Asfahl,

1992).

The eyes inspection can be reliable for short period of time but in making the system

automated, non-stop inspections system and high reliability result are needed in the

system. It is impossible for the operators to inspect the product for 8 hours or at least 2

hours straight without making any flaws in its inspections.

1.4 Objectives

In order to develop a successful project, objectives should be achieved. The objectives

are required to assist and guide the development of the project. The objectives for this

project are to:

categorize the defect (defect by vision system, defect by product) and store the

defect data in a database

design and develop a system to automatically inspect a product defect

5

1.5 Scopes

a) Vision System

The system are use only for inspection of the medical container

STREPSILS cough syrup bottle only

The inspection will detect defect of that bottle only

b) Camera

The camera is choosen based on the application and economical expense.

The chosen camera will be use only to acquire picture to be process by the

image processing software.

c) Product

The product selected to be use for the project is only one type, which is

empty medical glass container STREPSILS cough syrup bottle

This vision inspection system will be only focusing defect of the bottle on

that type of product only.

d) Programming Image Processing Software

Image processing software selected is use to process the image and

compare with the original image.

The program for the inspection is program only to inspect the defect of the

product only.

6

CHAPTER II

LITERATURE REVIEW

2.1 Vision System

Vision can be describe as ability to see the features of objects that look at, such as color,

shape, size, details, depth, and contrast. Vision is achieved when the eyes and brain work

together to form pictures of the world around. Vision begins with light rays bouncing off

the surface of objects. These reflected light rays enter the eye and are transformed into

electrical signals. Millions of signals per second leave the eye via the optic nerve and

travel to the visual area of the brain. Brain cells then decode the signals into images,

providing sight to the human (Microsoft Encarta 2006).

In industry sector, vision is use with machine to automate the system. It consists of

establishments that supply technology used in manufacturing industries as a substitute for

the human vision function. It is made up of suppliers of systems that embody techniques

leading to decisions based on the equivalent functionality of vision, but without operator

intervention.

The characteristic of these techniques include non-contact sensing of electromagnetic

radiation, direct or indirect operation of an image, the use of a computer to process the

sensor data and analyze that data to reach a decision with regard to the scene being

examined, and an understanding that the ultimate function of the system performance is

control (process control, quality control, machine control or robot control) (Zuech, 2000).

7

Machine vision system can be characterized as newly approach field and diverse. The

machine system is now being used in many fields such as medical, industrial quality

control, military, astronomy field and other fields. Figure 2.1 shows the field of vision

system.

Figure 2.1: Vision System Field (Zuech, 2000)

Machine vision is distinct from computer vision. Computer vision extends to topics

related to autonomous robotics and machine representation of human vision. Machine

vision refers to automated imaging systems including a wide range of computing

disciplines aggregated to form a complete solution to visual problems and can be

considered a superset composed of computer vision and elements such as equipment

control, data basing, network systems, interfacing and machine learning (Zuech, 2000).

8

2.1.1. Human Vision vs Machine Vision

Living creature eye can be defined as light-sensitive organ of vision in humans

(Microsoft Encarta 2006). The eyes of various species vary from simple structures

that are capable only of differentiating between light and dark to complex organs, such as

those of humans and other mammals, which can distinguish minute variations of shape,

color, brightness, and distance. The actual process of seeing is performed by the brain

rather than by the eye. The function of the eye is to translate the electromagnetic

vibrations of light into patterns of nerve impulses that are transmitted to the brain

(Microsoft Encarta 2006).

The eye is nearly sphere, with an average diameter of approximately 20mm. Three

membranes enclose the eye: the cornea and sclera outer cover, the choroids, and the

retina. The cornea is a tough, transparent tissue that covers the anterior surface of the eye.

Continuous with the cornea, the sclera is an opaque membrane that encloses the

remainder of the optic globe. Refer Figure 2.2 for the structure of human eye.

Figure 2.2: Structure of Human Eye (Encarta Encyclopedia, 2006)

9

The innermost membrane of the eye is the retina, which lines the inside of the walls

entire posterior portion. When the eye is properly focused, light from an object outside

the eye is imaged on the retina. Pattern vision is afforded by the distribution of discrete

light receptors over the surface of the retina. They are two classes of receptors: cones and

rods. The cones in each eye number between 6 and 7 million. They are located primarily

in the central portion of the retina, called the fovea, and highly sensitive to color.

The fovea itself is a circular indentation in the retina of about 1.5mm in diameter. Fovea

can be view as a square sensor array of size 1.5mm x 1.5mm. The density of cones in that

area of the retina is approximately 150,000 elements per mm2. Based on these

approximately, the number of cones highest acuity in the eye is about 337,000 elements.

Just in terms of raw resolving power, a couple-charged-device (CCD) imaging chip of

medium resolution can have this value of elements in a receptor array no larger than 5mm

x 5mm (Gonzalez, 1992).

Machine vision has been defined by the Machine Vision Association of the Society of

Manufacturing Engineers and the Automated Imaging Association as the use of devices

for optical, non contact sensing to automatically receive and interpret an image of a real

scene in order to obtain information and/or control machines or process. It uses a lens to

capture a picture of the object and focus it onto a sensor plane. The quality of the lens

will influence the quality of the images. Distortions and aberrations could affect the size

of features in image space. Vignette in a lens can affect the distribution of light across the

image plane. Magnification of the lens has to be appropriate for the application. As much

as possible the image of the object should fill the image plane of the sensor. The imaging

sensor that use in the machine vision system will basically dictate the limit of

discrimination of detail that will be experienced with system (Zuech, 2000).

Untitled