COLOUR SEGREGATION USING REAL TIME IMAGE

PROCESSING AND PLC BASED AUTOMATION

By:

UJJWAL SIDDHARTH

JITENDER KATARIA

TUSHAR DEORE

GUIDED BY:PROF. HARJEET KAUR

PROBLEM STATEMENT SEGREGATION OF INDIVIDUAL COLOURS

IN COLOUR SPACE?

INTRODUCTION: The Project deals with an automated

material handling system. It synchronizes the movement of punch

mechanism. It aims in classifying the colored objects

which are coming on the conveyor by punching and placing the objects in its respective pre-programmed place.

NOTABLE RESEARCH PAPER: Khojastehnazhand,et al., (2010) Grading systems give many kinds of information such as size, colour, shape,

defect, and internal quality. Among these colour and size are the most important features for accurate

classification and/or sorting of citrus such as oranges, lemons and tangerines.

Basically, two inspection stages of the system can be identified: external fruit inspection and internal fruit inspection. The former task is accomplished through processing of colour images, while internal inspection requires special sensors for moisture, sugar and acid contents.

In this paper, an efficient algorithm for grading lemon fruits is developed and implemented in visual basic environment. The system consists of two CCD cameras, two capture cards, an appropriate lighting system, a personal computer and other mechanical parts.

The algorithm initially extracts the fruit from the background. The samples of different grades of lemon are situated in front of the cameras and are calibrated off-line. Then information on the HSI colour values and estimated volumes of fruits are extracted and saved in a database.

By comparing the information during sorting phase with the available information inside the database, the final grade of the passing fruits is determined. This algorithm can be easily adapted for grading and/or inspection of other agricultural products such as cucumber and eggplant. 

TECHNICAL GUIDANCE

BASIC OVER VIEW

PUNCH ZONE

CAMERA

REMOTE PC

ARM

PLC

PLACE ZONECONVYER PLACE ZONE

PUNCH MECHANISM

INTERFACE UNIT

Functional block

conveyer

Camera

Remote PC

Punch Mechanism

PLC

PLACE ZONE

PUNCH ZONE

INTERFACE UNIT

DOMAINS COVERED:

Real time image processing [Matlab] PLC [ladder diagram] Interface unit[Assembly Language] Relay circuitry Punch mechanism [motor control] Conveyer Mechanism

WHY IMAGE PROCESSING?Image processing is used because: Colour Sensors are not sensitive to shades of

same colour. Here we can use image processing, For example In metal processing plant, metal grade ore processing can be done by using image processing. This is because ore and metal are almost of the same colour but if we process the image we can separate ore and metal.

Sensors need calibration while there is no need for calibration in image processing. Hence in case certain parameters in a process change we not only have to update the software but also calibrate the hardware. But in image processing only changes have to be made in the software of the program.

Additional sensors are not required for safety because image processing is real time hence an integrated system can be developed.

In case if we are picking from a heap of

metal ore then we need an array of sensors which is more costly compared to image processing.

Certain processes which deal with high temperature or pressure or any unfavorable condition. Instead of designing specific sensors we can simply use image processing.

Also in above processes mechanical wear-tear takes place of sensors due to exposure to such high temperatures etc.

WHY MATLAB? The above mentioned image processing is done using Matlab in the

following way: The Image Processing Toolbox software is a collection of functions that

extend the capability of the MATLAB numeric computing environment. The toolbox supports a wide range of image processing operations,

including1. Spatial image transformations2. Morphological operations3. Neighborhood and block operations4. Linear filtering and filter design5. Transforms6. Image analysis and enhancement7. Image registration8. Deblurring9. Region of interest operations Many of the toolbox functions are MATLAB M-files, a series of MATLAB

statements that implement specialized image processing algorithms. Thus, colour segregation can be done by using the MATLAB image

processing tool box and we will use the highlighted operations extensively for it.

COLOR SENSING MECHANISM: The resolution of the image should be found out. Generally is in terms

of:[640x480] or [320x240] etc[10]. Then, the colour is expressed as an RGB triplet (r,g,b), each

component of which can vary from zero to a defined maximum value. If all the components are at zero the result is black; if all are at maximum, the result is the brightest representable white.

SETTING THRESHOLD LEVELS : Since a RGB image model will contain red, green & blue as the constituent colours. Hence, if we are successfully able to distinguish between the three; rest are just shades of these colours. Hence, we establish a condition for separate identification of these colours. One condition per base colour(R or G or B) means three conditions in all. Also these threshold values for condition are so set that we are looking for pure base colour to avoid over lapping of base colours.

IDENTIFICATION AND SETTING UP OF MARKER : Now we define a 3-dimentional matrix which will correspond to the pixel value of the current image displayed. Once a pixel value corresponds to any of the three conditions defined above, it should place a marker on it. And start tracking it wherever it moves on the screen. In case multiple pixel values satisfy the condition then multiple markers should appear. For continuous tracking a loop should be used.

WORK SO FAR

MULTIPLE color detectionFiltering out different

colors

IMAGE SEGMENTATION We have found out that instead of treating image as a whole entity it is

sometimes better to segment & process. Imagine a situation where we have to pick up objects from a heap. For this we are acquiring a real tim image from a camera source and if we

divide the image, say into 9 cells [3x3] and process it one after another. We focus on one block at a time and pick up the required objects and

then move to next block. This prevents the picking mechanism from getting confused and also

since the mechanism is sequential in order, process automation is easier. Also since, multiple objects are processed simultaneously faster

processing takes place. Algorithm can be modified according to the domain of processing

required.

EXAMPLE(3:1)

PROBLEM STATEMENT:

?????????

APPLICATION OF IMAGE SEGMENTATION:

No COLOR detected Color detected

WHAT WE WOULD CONTRIBUTE Picking up object from heap: Image

segmentation reduces coding part for picking mechanism as the image is processed segment wise.

Color segregation: Objects of different colors can be processed sequentially. A particular color object can be separated from rest. Most of the industrial products are colored hence there is a large scope for segregation of these products.

PLC

 A programmable logic controller (PLC) or programmable controller is a digital computer used for automation of electromechanical processes, such as control of machinery on factory assembly lines, amusement rides, or light fixtures. In our case PLC understands: Logic 0 – 0 V Logic 1- 24 V

WHY PLC?Functions of a PLC:1. Process synchronization between Microcontroller, conveyer belt and robotic arm.2. HMI based control of process. (START/STOP/ABORT)3. Error report generation.4. Inventory report.5. Control via LAN using a remote PC.6. Active process report. (Via HMI)

RELAYS: A relay is an electrically operated switch. Many

relays use an electromagnet to operate a switching mechanism mechanically, but other operating principles are also used.

Relays are used where it is necessary to control a circuit by a low-power signal (with complete electrical isolation between control and controlled circuits), or where several circuits must be controlled by one signal.

The first relays were used in long distance telegraph circuits, repeating the signal coming in from one circuit and re-transmitting it to another.

Relays were used extensively in telephone exchanges and early computers to perform logical operations.

DIAGRAM

WHY RELAYS?6V (Sensor output to PLC)

24V(PLC output to Motor)

24V(PLC output to Indicators)

6V(Process control to PLC)

24(timed/PWM) V(PLC to DC)

CONNECTION MODULESData is transmitted serially in one direction over a pair of wires. Data going out is labeled Tx (indicating transmission) while data coming in is labeled Rx (indicating reception). To create a two way communication system a minimum of three wires are needed Tx, Rx and GND (ground). Crossing over Tx & Rx between the two systems lets each unit talk to the opposite one.

MOTORS: Motor Details Punch mechanism: 5V, 20 rpm Conveyer: 24V, 30 rpm  The DC motors are used to control the arm and turn table movement

are connected to controller circuit and receives signals from micro-controller. There are IR sensors installed in order to accurately identify ground and drop places

An electric motor converts electrical energy into mechanical energy. DC motor design generates an oscillating current in a wound rotor, or armature, with a split ring commutator, and either a wound or permanent magnet stator. A rotor consists of one or more coils of wire wound around a core on a shaft; an electrical power source is connected to the rotor coil through the commutator and its brushes, causing current to flow in it, producing electromagnetism. The commutator causes the current in the coils to be switched as the rotor turns, keeping the magnetic poles of the rotor from ever fully aligning with the magnetic poles of the stator field, so that the rotor never stops (like a compass needle does) but rather keeps rotating indefinitely (as long as power is applied and is sufficient for the motor to overcome the shaft torque load and internal losses due to friction, etc.).

CONVEYER BELT MECHANISM

Here, the conveyor motor receives power and signal from the central supply through rectifier and control circuit. The control circuit consisting of an potentiometer will allow the user to manually control the speed of conveyor belt by the regulatory knob. Polyester is used as a belt material. 

A conveyor belt consists of two or more pulleys, with a continuous loop of material - the conveyor belt - that rotates about them.

One or both of the pulleys are powered, moving the belt and the material on the belt forward. The powered pulley is called the drive pulley while the unpowered pulley is called the idler.

There are two main industrial classes of belt conveyors; those in general material handling such as those moving boxes along inside a factory and bulk material handling such as those used to transport industrial and agricultural materials, such as grain, coal, ores, etc. generally in outdoor locations.

PUNCH MECHANISM

APPLICATION

CONCLUSION The project works successfully and separates

different coloured objects using colour sensor. The camera based colour detection result was

converted chiefly to the command that drive the handling systems which drive the pick and place robot to pick up the object and place it into its designated place.

There are two main steps in colour sensing part, objects detection and colour recognition.

The system has successfully performed handling station task, namely pick and place mechanism with help of colour sensor.

REFERENCES [1].IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 45, NO. 8, AUGUST 2007 Walled LTSA Array for Rapid, High Spatial Resolution, and Phase-Sensitive Imaging to Visualize Plastic Landmines Soichi Masuyama and Akira Hirose, Member, IEEE. [2] CAN bus based on ARM core and the gangue separation system ;Xian-Min Ma; Xiao-Ru Song Machine Learning and Cybernetics, 2005. Proceedings of 2005 International Conference on Volume: 2 Publication Year: 2005 , Page(s): 988 - 992 Vol. 2 [3] Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying Chad Carson, Member, IEEE, Serge Belongie, Member, IEEE Hayit Greenspan, Member, IEEE, and Jitendra Malik, Member, IEEE : IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE  [4]. Khojastehnazhand, M., Omid, M., and Tabatabaeefar, A., “Development of a lemon sorting system based on colour and size” Journal of Plant Science, Vol. 4, No. 4, pp. 122-127, 2010. 

QUESTIONS?

/* COMMENTS */

THANK YOU!