1. and (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha

2. presents (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 3. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 4. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 5. The word robot originally was supposed to mean a slave It is a machine which performs a variety of tasks, either using manual external control or intelligent automation A manually controlled car or a ASIMOV trying to kick a football are all robots (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 6. Robotics is a multi disciplinary field of engineering encompassing the vistas of Mechanical design Electronic control Artificial Intelligence It finds its uses in all aspects of our life automated vacuum cleaner Exploring the Red planet Setting up a human colony there :D (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 7. ROBOTS CONTROL AUTONOMOUS MANUAL APPLICATIONS INDUSTRIAL MEDICAL INTERFACE HARDWARE SOFTWARE INTERLINKED (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 8. Locomotion System Actuators Power Supply System Transmission System Switches Sensory Devices For Feedback Sensor Data Processing Unit (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 9. A mobile robot must have a system to make it move. Ob. This system gives our machine the ability to move forward, backward and take turns It may also provide for climbing up and down Or even flying or floating J (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 10. Each type of locomotion requires different number of degrees of freedom More degrees of freedom means more the number of actuators you will have to use Although one actuator can be used to control more than one degree of freedom (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 11. Wheeled Legged Climbing Flying Floating Snake-Like (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 12. The kind of locomotion most frequently used in robotics at the undergrad level This involves conversion of electrical energy into mechanical energy (mostly using motors) The issue is to control these motors to give the required speed and torque (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 13. We have a simple equation for the constant power delivered to the motor: P = X Note that the torque and angular velocity are inversely proportionally to each other So to increase the speed we have to reduce the torque (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 14. The dc motors available have very high speed of rotation which is generally not needed At high speeds, they lack torque For reduction in speed and increase in pulling capacity we use pulley or gear systems (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 15. Differential Drive Dual Differential Drive Car-type Drive Skid-steer Drive Synchronous Drive (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 16. Simplest, easiest to implement and most widely used. It has a free moving wheel in the front accompanied with a left and right wheel. The two wheels are separately powered When the wheels move in the same direction the machine moves in that direction. Turning is achieved by making the wheels oppose each others motion, thus generating a couple (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 17. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 18. In-place (zero turning radius) rotation is done by turning the drive wheels at the same rate in the opposite direction Arbitrary motion paths can be implemented by dynamically modifying the angular velocity and/or direction of the drive wheels Total of two motors are required, both of them are responsible for translation and rotational motion (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 19. Simplicity and ease of use makes it the most preferred system by beginners Independent drives makes it difficult for straight line motion. The differences in motors and frictional profile of the two wheels cause them to move with slight turning effect The above drawback must be countered with appropriate feedback system. Suitable for human controlled remote robots (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 20. Uses synchronous rotation of its wheels to achieve motion & turns It is made up of a system of 2 motors. One which drive the wheels and the other turns the wheels in a synchronous fashion The two can be directly mechanically coupled as they always move in the same direction with same speed (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 21. The direction of motion is given by black arrow. The alignment of the machine is shown by red arrow (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 22. The use of separate motors for translation and wheel turning guarantees straight line motion without the need for dynamic feedback control This system is somewhat complex in designing but further use is much simpler (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 23. Actuators, also known as drives, are mechanisms for getting robots to move. Most actuators are powered by pneumatics (air pressure), hydraulics (fluid pressure), or motors (electric current). They are devices which transform an input signal (mainly an electrical signal)) into motion (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 24. Widely used because of their small size and high energy output. Operating voltage: usually 6,12,24V. Speed: 1-20,000 rpm.. Power: P = X (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 25. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha The stator is the stationary outside part of a motor. The rotor is the inner part which rotates. Red represents a magnet or winding with a north polarization. Green represents a magnet or winding with a south polarization. Opposite, red and green, polarities attract. Commutator contacts are brown and the brushes are dark grey. 26. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Stator is composed of two or more permanent magnet pole pieces. Rotor composed of windings which are connected to a mechanical commutator. The opposite polarities of the energized winding and the stator magnet attract and the rotor will rotate until it is aligned with the stator. Just as the rotor reaches alignment, the brushes move across the commutator contacts and energize the next winding. A yellow spark shows when the brushes switch to the next winding. 27. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha It is an electric motor that can divide a full rotation into a large number of steps. The motor's position can be controlled precisely, without any feedback mechanism. There are three types: Permanent Magnet Variable Resistance Hybrid type 28. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Stepper motors work in a similar way to dc motors, but where dc motors have 1 electromagnetic coil to produce movement, stepper motors contain many. Stepper motors are controlled by turning each coil on and off in a sequence. Every time a new coil is energized, the motor rotates a few degrees, called the step angle. 29. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 30. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Full Step Stepper motors have 200 rotor teeth, or 200 full steps per revolution of the motor shaft. Dividing the 200 steps into the 360's rotation equals a 1.8 full step angle. Achieved by energizing both windings while reversing the current alternately. 31. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Servos operate on the principle of negative feedback, where the control input is compared to the actual position of the mechanical system as measured. Any difference between the actual and wanted values (an "error signal") is amplified and used to drive the system in the direction necessary to reduce or eliminate the error Their precision movement makes them ideal for powering legs, controlling rack and pinion steering, to move a sensor around etc. 32. Suitable power source is needed to run the robots Mobile robots are most suitably powered by batteries The weight and energy capacity of the batteries may become the determinative factor of its performance (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 33. For a manually controlled robot, you can use batteries or voltage eliminators (convert the normal 220V supply to the required DC voltage 12V , 24V etc.) (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 34. Gear Belt Pulley Chain Sprocket Rack and Pinion Pick Place Mechanisms (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 35. Gears are the most common means of transmitting power in mechanical engineering Gears form vital elements of mechanisms in many machines such as vehicles, metal tooling machine tools, rolling mills, hoisting etc. In robotics its vital to control actuator speeds and in exercising different degrees of freedom (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 36. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 37. To achieve torque magnification and speed reduction They are analogous to transformers in electrical systems It follows the basic equation: 1 x r1 = 2 x r2 Gears are very useful in transferring motion between different dimension (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 38. An arrangement of gears to convert rotational torque to linear motion Same mechanism used to steer wheels using a steering In robotics used extensively in clamping systems (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 39. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 40. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 41. It allows for mechanical power, torque, and speed to be transmitted across axes If the pulleys are of differing diameters, it gives a mechanical advantage In robotics it can be used in lifting loads or speed reduction Also it can be used in a differential drive to interconnect wheels (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 42. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 43. Sprocket is a profiled wheel with teeth that meshes with a chain It is similar to the system found in bicycles It can transfer rotary motion between shafts in cases where gears are unsuitable Can be used over a larger distance Compared to pulleys has lesser slippage due to firm meshing between the chain and sprocket (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 44. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 45. For picking and placing many mechanisms can be used: vHook and pick vClamp and pick vSlide a sheet below and pick vMany other ways vLots of Scope for innovation (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 46. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 47. Image Processing is a tool for analyzing image data in all areas of natural science It is concerned with extracting data from real-world images Differences from computer graphics is that computer graphics makes extensive use of primitives like lines, triangles & points. However no such primitives exist in a real world images. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 48. Increasing need to replicate human sensory organs Eye (Vision) : The most useful and complex sensory organ (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 49. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 50. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 51. Automated visual inspection system Checking of objects for defects visually Remote Sensing Satellite Image Processing Classification (OCR), identification (Handwriting, finger prints) etc. Detection and Recognition systems (Facial recognition..etc) Biomedical applications (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 52. Camera, Scanner or any other image acquisition device PC or Workstation or Digital Signal Processor for processing Software to run on the hardware platform (Matlab, Open CV etc.) Image representation to process the image (usually matrix) and provide spatial relationship A particular color space is used to represent the image(c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 53. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Image Acquisition Device (Eg. CCD or CMOS Camera) Image Processor (Eg. PC or DSP) Image Analysis Tool (Eg. Matlab or Open CV) Machine Control Of Hardware through serial or parallel interfacing 54. Using a camera Analog cameras Digital cameras CCD and CMOS cameras Captures data from a single light receptor at a time CCD Charge Coupled Devices CMOS Complementary MOSFET Sensor based (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 55. Digital Cameras CCD Cameras High quality, low noise images Genarates analog signal converted using ADC Consumes high power CMOS Cameras Lesser sensitivity Poor image quality Lesser power Analogue cameras require grabbing card or TV tuner card to interface with a PC (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 56. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Colored pixels on CCD Chip 57. Matlab Open CV (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 58. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 59. Two types: Vector and Raster Vector images store curve information Example: Indias flag Three rectangles, one circle and the spokes We will not deal with vector images at all (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 60. Raster images are different They are made up of several dots (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 61. If you think about it, your laptops display is a raster display Also, vector images are high level abstractions Vector representations are more complex and used for specific purposes (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 62. Raster Matrix Vector Quadtrees Chains Pyramid Of the four, matrix is the most general. The other three are used for special purposes. All these representations must provide for spatial relationships (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 63. Computers cannot handle continuous images but only arrays of digital numbers So images are represented as 2-D arrays of points (2-D matrix)(Raster Represenatation) A point on this 2-D grid (corresponding to the image matrix element) is called PIXEL (picture element) It represents the average irradiance over the area of the pixel (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 64. Each pixel requires some memory Color depth : Amount of memory each pixel requires Examples 1-bit 8-bit 32-bit 64-bit (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 65. Pixels are tiny little dots of color you see on your screen, and the smallest possible size any image can get When an image is stored, the image file contains information on every single pixel in that image i.e Pixel Location Intensity The number of pixels used to represent the image digitally is called Resolution More the number of pixels used, higher the resolution Higher resolution requires more processing power (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 66. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 67. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 68. MATLAB stands for MATrix LABoratory, a software developed by Mathworks Inc (www.mathworks.com). MATLAB provides extensive library support for various domains of scientific and engineering computations and simulations (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 69. When you click the MATLAB icon (from your desktop or Start>All Programs), you typically see three windows: Command Window, Workspace and Command History. Snapshots of these windows are shown below (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 70. This window shows the variables defined by you in current session on MATLAB (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 71. Command History stores the list of recently used commands for quick reference (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 72. This is where you run your code (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 73. This is where you run your code (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 74. In MATLAB, variables are stored as matrices (singular: matrix), which could be either an integer, real numbers or even complex numbers These matrices bear some resemblance to array data structures (used in computer programming) (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 75. Let us start with writing simple instructions on MATLAB command window To define an integer, Type a=4 and hit enter >>a=4 To avoid seeing the variable, add a semicolon after the instruction >>a=4; (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 76. Similarly to define a 2x2 matrix, the instruction in MATLAB is written as >> b=[ 1 2; 3 4]; If you are familiar with operations on matrix, you can find the determinant or the inverse of the matrix. >> determin= det(b) >> d=inv(b) (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 77. Images as we have already seen are stored as matrices So now we try to see this for real on MATLAB We shall also look into the basic commands provided by MATLABs Image Processing Toolbox (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 78. Once you have started MATLAB, type the following in the Command Window >> im=imread(sample.jpg'); This command stores the file image file sample.jpg in a variable called im It takes this file from the Current- Directory specified Else, entire path of file should be mentioned (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 79. You can display the image in another window by using imshow command >>figure,imshow(im); This pops up another window (called as figure window), and displays the image im (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 80. The imview command can also be used in order toview the image imview(im); Difference is that in this case you can see specific pixel values just by moving the cursor over the image (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 81. To know the breadth and height of the image, use the size function, >>s=size(im); The size function basically gives the size of any array in MATLAB Here we get the size of the IMAGE ARRAY (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 82. Now that we have our image stored in a variable we can observe and understand the following: How pixels are stored? What does the values given by each pixel indicate? What is Image Resolution? (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 83. Have a look at the values stored Say the first block of 10 x 10 >>im(1:10,1:10); Or Say view the pixel range 50:150 on both axis >> figure,imshow(im(50:150,50:150)); (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 84. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 85. 1-bit = BLACK or WHITE 8-bit = 28 different shades 24-bit = 224 different shades 64-bit images High end displays Used in HDRI, storing extra information per pixel, etc (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 86. This is another name for 1-bit images Each pixel is either White or Black Technically, this is a black & white image (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 87. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 88. Another name for 8-bit images Each pixel can be one of 256 different shades of gray These images are popularly called Black & White. Though, this is technically wrong. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 89. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 90. Again, each pixel gets 8 bits But each of the 256 values maps to a color in a predefined palette If required, you can have different bit depths (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 91. We wont be dealing with indexed images (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 92. 8-bits is too less for all the different shades of colors we see So 24-bits is generally used for color images Thus each pixel can have one of 224 unique colors (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 93. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 94. Now, a new problem arises: How do you manage so many different shades? Programmers would go nuts Then came along the idea of color spaces (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 95. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 96. A color space can be thought of as a way to manage millions of colors Eliminates memorization, and increases predictability Common color spaces: RGB HSV YCrCb or YUV YIQ (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 97. Youve probably used this already (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 98. Each pixel stores 3 bytes of data The 24-bits are divided into three 8-bit values The three are: Red, Green and Blue i.e the primary colours Mixing of primary colours in right proportions gives any particular colour Each pixel has these 3 values (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 99. 1 byte = 8 bits can store a value between 0-255 We get pixel data in the form RGB values with each varying from 0-255 That is how displays work So there are 3 grayscale channels (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 100. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 101. Advantages: Intuitive Very widely used Disadvantages: Image processing is relatively tough (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 102. HSV makes image processing easier Again, 24 bits = three 8-bit values or 3 channels The 3 channels are: Hue Saturation (Shade of Colour) Value (Intensity) (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 103. The Hue is the tint of color used It represents the colour of the pixel (Eg. Red Green Yellow etc) The Saturation is the amount of that tint It represents the intensity of the colour (Eg. Dark red and light red) The Value is the intensity of that pixel It represents the intensity of brightness of the colour (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 104. RGB image converted to HSV (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha RGB HUE SATURATION VALUE 105. Advantages: The color at a pixel depends on a single value Illumination independent Disadvantages: Something (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 106. Intuitively RGB might seem to be the simpler and better colour space to deal with Though HSV has its own advantages especially in colour thresholding As the colour at each pixel depends on a single hue value it is very useful in separating out blobs of specific colours even when there are huge light variations Thus it is very useful in processing real images taken from camera as there is a large amount of intensity variation in this case Hence, ideal for robotics applications (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 107. Widely used in digital video Has three 8-bit channels: Y Component: Gives luminance or intensity Cr Component: It is the RED component minus a reference value Cb Component: It is the BLUE component minus a reference value Hence Cr and Cb components represent the colour called Color Difference Components (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 108. Advantages: Used in video processing Gives you a 2-D colour space hence helps in closer distinguishing of colours Disadvantages: (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 109. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 110. The camera returns images in a certain color space You might want to convert to different color spaces to process it Colour space conversions can take place between RGB to any other colour space and vice versa (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 111. Since cameras usually input images in rgb We would like to convert these images into HSV or YCrCb Conversions: RGB->HSV HSV->RGB RGB->YCrCb YCrCb->RGB (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 112. RGB -> HSV (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 113. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha HSV RGB YCrCb 114. >>h = rgb2hsv(im) This converts the RGB image to HSV The new colour space components can be seen using >> imview(h) >> imview(h(:,:,1)) HUE >> imview(h(:,:,2)) Saturation >> imview(h(:,:,3)) Value (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 115. >>R = hsv2rgb(im) This converts the HSV image to RGB The new colour space components can be seen using >> imview(R) >> imview(R(:,:,1)) Red >> imview(R(:,:,2)) Green >> imview(R(:,:,3)) Blue (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 116. >> Y = rgb2ycbcr(im); This converts the RGB image to YCbCr The new colour space components can be seen using >> imview(Y) >> imview(Y(:,:,1)) Luminance >> imview(Y(:,:,2)) Differenced Blue >> imview(Y(:,:,3)) Differenced Red (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 117. >> R = ycbcr2rgb(im); This converts the YCbCr image to RGB The new colour space components can be seen using >> imview(R) >> imview(R(:,:,1)) Red >> imview(R(:,:,2)) Green >> imview(R(:,:,3)) Blue (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 118. Formulae for conversion are very complex But the best thing is, you dont need to remember these formulae Matlab and OpenCV have built-in functions for these transformations :-) (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 119. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 120. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 121. OpenCV is a collection of many functions that help in image processing You can use OpenCV in C/C++, .net languages, Java, Python, etc as well We will only discuss OpenCV in C/C++ (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 122. It is blazingly fast Quite simple to use and learn Has functions for machine learning, image processing, and GUI creation (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 123. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 124. Download the latest OpenCV package from http://sourceforge.net/projects/opencv / Install the package, and note where you installed it (like C:Program FilesOpenCV) (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 125. Now, we need to tell Microsoft Visual Studio that weve installed OpenCV So, we tell it where to find the OpenCV header files Start Microsoft Visual Studio 2008 (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 126. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 127. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 1 2 128. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Type these paths into the list 129. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Type these paths into the list 130. Right now, Visual Studio knows where to find the OpenCV include files and library files Now we create a new project (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 131. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 132. Accept all default settings in the project Youll end up with an empty project with a single file (like Mybot.cpp) Open this file, well write some code now (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 133. Add the following at the top of the code #include #include This piece of code includes necessary OpenCV functionality (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 134. Now, we get to the main() function int main() { The main function is where for program execution begins (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 135. Next, we load an image IplImage* img = cvLoadImage("C:hello.jpg"); The IplImage is a data type, like int, char, etc (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 136. Comes built-into OpenCV Any image in OpenCV is stored as an IplImage thingy It is a structure (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 137. Opens filename and returns it as an IplImage structure Supported formats: Windows bitmaps - BMP, DIB JPEG files - JPEG, JPG, JPE Portable Network Graphics - PNG Portable image format - PBM, PGM, PPM Sun rasters - SR, RAS TIFF files - TIFF, TIF OpenEXR HDR images - EXR JPEG 2000 images - jp2 (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 138. Now we show this image in a window cvNamedWindow("myfirstwindow"); cvShowImage("myfirstwindow", img); This uses some HighGUI functions (comes along with OpenCV) (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 139. Creates a window with the caption title This is a HighGUI function You can add controls to each window as well (track bars, buttons, etc) (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 140. Shows img in the window with caption title If no such window exists, nothing happens (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 141. Finally, we wait for an input, release and exit cvWaitKey(0); cvReleaseImage(&img); return 0; } (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 142. Waits for time milliseconds, and returns whatever key is pressed If time=0, waits till eternity Here, weve used it to keep the windows from vanishing immediately (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 143. Erases img from the RAM Get rid of an image as soon as possible. RAM is precious J Note that you send the address of the image (&img) and not just the image (img) (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 144. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 145. Right now, Visual Studio knows where OpenCV is But it does not know, whether to use OpenCV or not We need to tell this explicitly (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 146. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 147. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 148. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 149. Got errors? Check if the syntax is correct Copy all DLL files in *OpenCVbin into C:WindowsSystem32 (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 150. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 151. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 152. src is the original image dst is the destination code is one of the follow: CV_BGR2HSV CV_RGB2HSV CV_RGB2YCrCb CV_HSV2RGB CV_2 (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 153. src should be a valid image. Or an error will pop up dst should be a valid image, i.e. you need a blank image of the same size code should be valid (check the OpenCV documentation for that) (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 154. Allocates memory for an image of size size, with bits bits/pixel and chan number of channels Used for creating a blank image Use cvSize(width, height) to specify the size (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 155. Example: IplImage* blankImg = cvCreateImage(cvSize(640, 480), 8, 3); (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 156. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 157. Wired Motor Driving module Interface with PC (Parallel/Serial) Wireless The Motor-driving module The Wireless Receiver Circuit The Wireless Transmitter Circuit Interface with PC (Parallel/Serial) (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 158. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 159. IC 7805 Voltage Regulator L293D Motor Driver MCT2E Opto-Coupler Parallel Port Male-Connector RF-RX Connector (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 160. Its a three terminal linear 5 volt regulator used to supply the board and other peripherals Prescribed input voltage to this component is about 7-9 Volts (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 161. Voltage fluctuations can be controlled by using low pass filter capacitors across output and input Higher input voltage can be applied if heatsink is provided (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 162. Used to control Dc and Stepper Motors Uses a H-Bridge which is an electronic switching circuit that can reverse direction of current Its a Dual-H bridge Basically used to convert a low voltage input into a high voltage output to drive the motor or any other component Eg: Microcontroller Motor Driver Motor (5 Volts) (12 Volts) (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 163. Different Motor Driver ICs L293D 600mA Current Rating Dual H-bridge (Dc and Stepper Motors) L298N 1 Amp Current Rating Dual H-bridge (Dc and Stepper Motors) L297-L298 (Coupled) For stepper motor overdriving Dual H-bridge (Dc and Stepper Motors) 2 Ics in parallel ULN2003/ULN2803 500mA Current Rating For unipolar stepper motors (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 164. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 165. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 166. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 167. Output Current: 600 mA Output Voltage Wide Range 4.5 V 36 V (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 168. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 169. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha There are many situations where signals and data need to be transferred from one subsystem to another within a piece of electronics Relays are too bulky as they are electromechanical in nature and at the same time give lesser efficiency In these cases an electronic component called Optocoupler is used 170. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha They are generally used when the 2 subsystems are at largely different voltages These use a beam of light to transmit the signals or data across an electrical barrier, and achieve excellent isolation 171. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha In our circuit, Opto-isolator (MCT2E) is used to ensure electrical isolation between motors and the PC parallel port during wired connection The Viz-Board has four such chips to isolate the four data lines (pin 2, pin 3, pin 4, pin 5) coming out of the parallel port 172. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 173. Along with the Viz-Board 2 extensions have been provided i.e The Rf Transmitter Module The Rf Reciever Module (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Transmitt er Receive r 174. Radio frequency modules are used for data transmission wirelessly at a certain frequency It sends and receives radio waves of a particular frequency and a decoder and encoder IC is provided to encode and decode this information Wireless transmission takes place at a particular frequency Eg. 315Mhz Theses modules might be single or dual frequency (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 175. Antenna is recommended on both of them - just connect any piece of 23 cm long to the Antenna pin The kit has a dual frequency RF module with frequencies 315/434 Mhz (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 176. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 177. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 178. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 179. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 180. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 181. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 182. The encoder IC encodes the parallel port data and sends it to the RF transmitter module for wireless transmission They are capable of encoding information which consists of N address bits and (12-N) data bits (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 183. The HT12E Encoder IC has 8 address bits and 4 data bits A DIP-Switch can be used to set or unset the address bits A0-A7 (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 184. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha A0-A7Address Bits AD8-AD11Data Bits 185. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha A0-A7Address Bits AD8-AD11Data Bits 186. The decoder IC decodes the RF transmitter data and sends it to the parallel port for wireless transmission They are capable of encoding information which consists of N address bits and (12-N) data bits (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 187. The HT12D Decoder IC has 8 address bits and 4 data bits A DIP-Switch can be used to set or unset the address bits A0-A7 (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 188. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha A0-A7Address Bits D8-D11Data Bits 189. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha A0-A7Address Bits D8-D11Data Bits 190. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 191. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 192. Serial Port Parallel Port USB (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 193. Data is transferred serially i.e packets are sent one after the other through a single port (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 194. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 195. Data is transferred in parallel through different data pins at the same time Communication is pretty fast Found in old printer ports (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 196. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 197. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 25th pin : Ground 2nd-12th pin : I/O lines 198. Parallel port is faster than serial A mass of data can be transmitted at the same time through parallel ports Though parallel and serial ports are not found these days in laptops Desktops and old laptops have these ports (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 199. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Direct Output from parallel port Output from motor driver 200. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 201. Camera, object and source positions (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 202. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Image sampling and quantization 203. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Continuous image projected on an array sensor Result of image sampling and quantization 204. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Sampling: Digitizing the coordinate values (spatial resolution) Quantization: Digitizing the amplitude values (intensity levels) 205. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 1 bit /pixel B bits/pixel 2B gray levels 1 byte = 8 bits > 256 levels 2 possible values 2 gray levels -> 0 or 1 (binary image) 206. All this sampling and quantization puts in extra noise on the image! Noise can be reduced by Using hardware Using software: filters (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 207. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 208. Why do we need to enhance images? Why filter images? (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 209. Large amounts of external disturbances in real images Due to different factors like changing lighting and other real-time effects To improve quality of a captured image to make it easier to process the image (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 210. First step in most IP applications Used to remove noise in the input image To remove motion blur from an image Enhancing the edges of an image to make it appear sharper (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 211. Generally used types Of Filtering Averaging Filter Mean Filter Median Filter Gaussian Smoothing Histogram Equalization (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 212. The Averaging filter is used to sharpen the images by taking average over a number of images It eliminates noise by assuming that different snaps of the same image have different noise patterns (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 213. Noise is gaussian in nature i.e follows a gaussian curve Hence, summing up noises infinite times approaches zero (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 214. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 215. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 216. This is extremely useful for satellites that take intergalactic photographs The images are extremely faint, and there is more noise than the image itself Millions of pictures are taken, and averaged to get a clear picture (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 217. The Mean is used to soften an image by averaging surrounding pixel values (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Center pixel = (22+77+48+150+77+158+0+77+219)/9 218. The center pixel would be changed from 77 to 92 as that is the mean value of all surrounding pixels This filter is often used to smooth images prior to processing It can be used to reduce pixel flicker due to overhead fluorescent lights (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 219. This replaces each pixel value by the median of its neighbors, i.e. the value such that 50% of the values in the neighborhood are above, and 50% are below This can be difficult and costly to implement due to the need for sorting of the values However, this method is generally very good at preserving edges (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 220. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 221. Its performance is particularly good for removing short noise The median is calculated by first sorting all the pixel values from the surrounding neighborhood into numerical order and then replacing the pixel being considered with the middle pixel value If the neighborhood under consideration contains an even number of pixels, the average of the two middle pixel values is used (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 222. Used to `blur' images and remove detail and noise The effect of Gaussian smoothing is to blur an image The Gaussian outputs a `weighted average' of each pixel's neighborhood, with the average weighted more towards the value of the central pixels (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 223. A Gaussian provides gentler smoothing and preserves edges better than a similarly sized mean filter (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Before Blurring After Blurring 224. It is very useful in contrast enhancement Especially to eliminate noise due to changing lighting conditions etc Transforms the values in an intensity image so that the histogram of the output image approximately matches a specified histogram (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 225. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 226. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Filters and histograms 227. Imfilter function is used for creating different kinds of filters In MATLAB B = imfilter(A,H,option) filters the multidimensional array A with the multidimensional filter H The array A can be a nonsparse numeric array of any class and dimension The result B has the same size and class as A (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 228. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Options in imfilter Convolution is same as correlation except that the h matrix is inverted before applying the filter 229. h = ones(5,5) / 25; imsmooth = imfilter(im,h); Here a mean filter is implemented using the appropriate h matrix imshow(im), title('Original Image'); figure, imshow(imsmooth), title('Filtered Image') (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 230. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 231. FSPECIAL is used to create predefined filters h = FSPECIAL(TYPE); FSPECIAL returns h as a computational molecule, which is the appropriate form to use with imfilter (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 232. FSPECIAL is used to create predefined filters h = FSPECIAL(TYPE); FSPECIAL returns h as a computational molecule, which is the appropriate form to use with imfilter (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 233. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 234. The process of adjusting intensity values can be done automatically by the histeq function >>im = imread('pout.tif'); >>jm = histeq(im); >>imshow(jm) >>figure, imhist(jm,64) (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 235. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Original Image 236. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Histogram Equalized Image 237. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 238. Things arent as simple as they were in Matlab C/C++ needs a bit of syntax and formalities (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 239. Well try doing the following right now Gaussian filter Median filter Bilateral filter Simple blur Averaging filter (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 240. Start Microsoft Visual Studio 2008 I assume you have OpenCV installed (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 241. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 242. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 243. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 244. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 245. #include #include (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 246. #include #include int main() { (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 247. #include #include int main() { IplImage* img = cvLoadImage(C:noisy.jpg); (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 248. #include #include int main() { IplImage* img = cvLoadImage(C:noisy.jpg); IplImage* imgBlur = cvCreateImage(cvGetSize(img), 8, 3); IplImage* imgGaussian = cvCreateImage(cvGetSize (img), 8, 3); IplImage* imgMedian = cvCreateImage(cvGetSize (img), 8, 3); IplImage* imgBilateral = cvCreateImage(cvGetSize (img), 8, 3); (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 249. cvSmooth(img, imgBlur, CV_BLUR, 3, 3); cvSmooth(img, imgGaussian, CV_GAUSSIAN, 3, 3); cvSmooth(img, imgMedian, CV_MEDIAN, 3, 3); cvSmooth(img, imgBilateral, CV_BILATERAL, 3, 3); (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 250. cvNamedWindow(original); cvNamedWindow(blur); cvNamedWindow(gaussian); cvNamedWindow(median); cvNamedWindow(bilateral); cvShowImage(original, img); cvShowImage(blur, imgBlur); cvShowImage(gaussian, imgGaussian); cvShowImage(median, imgMedian); cvShowImage(bilateral, imgBilateral); cvWaitKey(0); return 0; } (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 251. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 252. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 253. Blur: The plain simple Photoshop blur Gaussian: The best result (preserved edges and smoothed out noise) Median: Nothing special Bilateral: Got rid of some noise, but preserved edges to a greater extend (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 254. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 255. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 256. Your OpenCV installation comes with detailed documentation *OpenCVdocsindex.html Scroll down, and youll see OpenCV Reference Manuals (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 257. Try looking up cvSmooth in the CV Reference Manual (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 258. Now try looking up cvEqualizeHist (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 259. There are no built-in functions for this So, well code it ourselves And this will be a good exercise for getting better at OpenCV (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 260. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 261. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 262. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 263. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 264. #include #include (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 265. #include #include int main() { (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 266. #include #include int main() { IplImage* imgRed[25]; IplImage* imgGreen[25]; IplImage* imgBlue[25]; Holds the R, G and B channels separately for each of the 25 images (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 267. IplImage* imgBlue[25]; for(int i=0;ise=strel(shape,parameters) (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 303. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 304. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 305. Dilation in matlab is done using the following command: >>bw2=imdilate(bw,st) Bw = Original image St = Structuring element (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 306. Erosion in matlab is done using the following command: >>bw2=imerode(bw,st) Bw = Original image St = Structuring element (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 307. Opening in matlab is done using the following command: >>bw2=imopen(bw,st) Bw = Original image St = Structuring element (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 308. Closing in matlab is done using the following command: >>bw2=imclose(bw,st) Bw = Original image St = Structuring element (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 309. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 310. cvErode(src, dst) cvDilate(src, dst) Opening & closing: use the appropriate sequence (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 311. By default, OpenCV uses the zero structuring element (all are zeros) You can explicitly specify your structuring element as well Check the OpenCV Documentation for more information (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 312. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 313. Computers can manipulate images very efficiently But, comprehending an image with millions of colors is tough Solution: Figure out interesting regions, and process them (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 314. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 315. Each pixel is checked for its value If it lies within a range, it is marked as interesting (or made white) Otherwise, its made black Figuring out the range depends on lighting, color, texture, etc (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 316. Demo thresholdRGB Demo thresholdHSV (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 317. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 318. MATLAB provides a facility to execute multiple command statements with a single command. This is done by writing a .m file Goto File > New > M-file For example, the graythresh function can be manually written as a m-file as: (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 319. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 320. Observe that, comments (in green) can be written after the symbol %. A commented statement is not considered for execution M-files become a very handy utility for writing lengthy programs and can be saved and edited, as and when required We shall now see, how to define your own functions in MATLAB. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 321. Functions help in writing organized code with minimum repetition of logic Instead of rewriting the instruction set every time, you can define a function Syntax: Create an m-file and the top most statement of the file should be the function header function [return values] = function- name(arguments) (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 322. The inbuilt graythresh function in matlab is used for thresholding of grayscale images It uses the Otsus Method Of thresholding A sample thresholding opreation has been shown in the next slide (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 323. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Image thresholded for the colour blue 324. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha The Real thing J 325. Thresholding of a grayscale image can be done in MATLAB using the following commands: >> level=graythresh(imGRAY); >> imBW = im2bw(imGRAY,level); >> imview(imBW); (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 326. The graythresh command basically gives an idea as to what exactly the threshold value should be Graythresh returns a value that lies in the range 0-1 This gives the level of threshold which is obtained by a complex method called the Otsus Method of Thresholding (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 327. This level can be converted into pixel value by multiplying by 255 Lets say, level=.4 Then threshold value for the grayscale image is: 0.4 x 255 =102 (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 328. What this indicates is that for the given image the values below 102 have to be converted to 0 and values from 103-255 to the value 1 Conversion from grayscale to binary image is done using the function: >>imBW = im2bw(imGRAY,level); (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 329. Here level is the threshold level obtained from graythresh function This function converts pixel intensities between 0 to level to zero intensity (black) and between level+1 to 255 to maximum (white) (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 330. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 331. In order to threshold an RGB colour image using the graythresh function, the following have to be done: Conversion of the RGB image into its 3 grayscale components Subtracting each of these components from the other 2 to get the pure colour intensities Finding level for each of the grayscale using graythresh Thresholding the image using imbw and the level (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 332. Commands: Im=Imread(rgb.jpg); R = im(:,:,1); --Red G = im(:,:,2); --Green B = im(:,:,3); --Blue (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 333. Ronly=R-B-G; --Pure RED Gonly=G-R-B; --Pure GREEN Bonly=B-G-R; --Pure BLUE (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 334. Levelr=graythresh(Ronly); Levelg=graythresh(Gonly); Levelb=graythresh(Bonly); (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 335. Rthresh=im2bw (Ronly,levelR); Gthresh=im2bw(Gonly,levelG); Bthresh=im2bw(Bonly,levelB); (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 336. Using a manually designed thresh_tool function to adjust the levels as required To get a feel of how levels vary (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 337. s=size(im); temp=im; thresh=128; for i=1:s(1,1) for j=1:s(1,2) if temp(i,j)level1) temp(i,j)=1; else temp(i,j)=0; end end end imview(temp); (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 341. To this function we give the input arguments as the upper and lower bounds of the threshold levels These levels can be obtained by having a look at the range of hue values for the particular colour (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 342. Now that you know the basics (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 343. cvThreshold(src, dst, threshold, max, type) type: CV_THRESH_BINARY CV_THRESH_BINARY_INV And several others (check documentation) (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 344. cvInRangeS(src, scalarLower, scalarUpper, dst); scalarLower = cvScalar(chan1, chan2, chan3, chan4); scalarUpper = cvScalar(chan1, chan2, chan3, chan4); (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 345. Ultra basics: motors, drives, etc Digital image representation Color spaces Inter-conversion of color spaces Electronics Filtering Thresholding Morphology (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 346. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 347. After thresholding, we get a binary image We want useable information like centers, outlines, etc There geometrical properties can be found using many methods. Well talk about moments and contours only. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 348. Moments are a mathematical concept intensity*xxorder*yyorder (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 349. Consider xorder=0 and yorder=0 for a binary image So youre just summing up pixel values This means, youre calculating the area of the white pixels (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 350. Now consider xorder=1 and yorder=0 for a binary image You sum only those x which are white So youre calculating the numerator of an average (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 351. The number of points where the pixel is white is the area of the image So, dividing this particular moment (xorder=1, yorder=0) by the earlier example (xorder=0, yorder=0) gives the average x This is the x coordinate of the centroid of the blob (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 352. Similarly, for xorder=0 and yorder=1, youll get the y coordinate (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 353. The order of a moment = xorder+yorder So, the area is a zero order moment The centroid coordinate = a first order moment / the zero order moment (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 354. There are entire books written on this topic You can find complex geometrical properties, like the eccentricity of an ellipse, radius of curvature of objects, etc Also check for Hu invariants if youre interested (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 355. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Centroid Area etc 356. These are pixels of an image that are conencted to each other forming separate blobs in an image They can be seperated out and labelled (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 357. >>L = bwlabel(BW,n) Returns a matrix L, of the same size as BW, containing labels for the connected objects in BW n can have a value of either 4 or 8, where 4 specifies 4-connected objects and 8 specifies 8-connected objects; if the argument is omitted, it defaults to 8 (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 358. >>L = bwlabel(BW,n) Returns a matrix L, of the same size as BW, containing labels for the connected objects in BW n can have a value of either 4 or 8, where 4 specifies 4-connected objects and 8 specifies 8-connected objects; if the argument is omitted, it defaults to 8 (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 359. STATS = regionprops(L,properties) Measures a set of properties for each labeled region in the label matrix L The set of elements of L equal to 1 corresponds to region 1; the set of elements of L equal to 2 corresponds to region 2; and so on (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 360. 'Area' The actual number of pixels in the region 'Centroid'-- The center of mass of the region. Note that the first element of Centroid is the horizontal coordinate (or x-coordinate) of the center of mass, and the second element is the vertical coordinate (or y-coordinate) 'Orientation' -- Scalar; the angle (in degrees) between the x-axis and the major axis of the ellipse that has the same second-moments as the region. This property is supported only for 2-D input label matrices (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 361. BW = imread('text.png'); L = bwlabel(BW); stats = regionprops(L,'all'); (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 362. Label into an RGB image for better vizualization RGB = label2rgb(L) (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 363. Binary area open remove small objects BW2 = bwareaopen(BW,P) Removes from a binary image all connected components (objects) that have fewer than P pixels, producing another binary image, BW2. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 364. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 365. OpenCV supports functions to calculate moments upto order 3 CvMoments *moments = (CvMoments*)malloc(sizeof (CvMoments)); cvMoments(img, moments, 1); (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 366. cvGetSpatialMoment(moments, xorder, yorder) cvGetCentralMoment(moments, xorder, yorder) Central = spatial/area (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 367. Example (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 368. For robotics purposes, moments are fine till have one single object If we have multiple objects in the same binary image (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 369. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 370. You can think of contours as an approximation of a binary image (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 371. You get polygonal approximation of each connected area (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 372. The output you get for the previous binary image is: Four chains of points Each chain can have any number of points In our case, each chain has four points (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 373. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Contour plotting 374. Contour plot of an image im can be made in MATLAB using the command: im = imread(img.jpg'); imcontour(im,level) Level=number of equally spaced contour levels if level is not given it will choose automatically (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 375. OpenCV linked lists to store the chains Well see some code to find out the squares in the thresholded image you saw (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 376. CvSeq* contours; CvSeq* result; CvMemStorage *storage = cvCreateMemStorage(0); The chains are stored in contours result is a temporary variable storage is for temporary memory allocation (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 377. cvFindContours(img, storage, &contours, sizeof(CvContour), CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE, cvPoint(0,0)); img is a grayscale thresholded image storage is for temporary storage All chains found would be stored in the contours sequence The rest of the parameters are usually kept at these values Check the OpenCV documentation for details information about the last four variables (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 378. while(contours) { result = cvApproxPoly(contours, sizeof(CvContour), storage, CV_POLY_APPROX_DP, cvContourPerimeter(contours)*0.02, 0); The previous command makes contours point to the first chain Were approximating the contour right now After this command, result stores the approximate contour as a polygon (many points) (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 379. if(result->total==4) { CvPoint *pt[4]; for(int i=0;ih_next; } Do whatever you want to do with the four points Then, we move onto processing the next contour (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 381. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 382. MATLAB has an image acquisition toolbox which helps capture images Now-a-days most of the cameras are available with USB interface Once you install the driver for the camera, the computer detects the device whenever you connect it (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 383. In MATLAB, you can check if the support is available for your camera MATLAB has built-in adaptors for accessing these devices An adaptor is a software that MATLAB uses to communicate with an image acquisition device (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 384. COMMANDS: >> imaqhwinfo >> cam=imaqhwinfo; >> cam.InstalledAdaptors (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 385. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 386. To get more information about the device, type >>dev_info = imaqhwinfo('winvideo',1) Instead of winvideo, if imaqhwinfo shows another adaptor, then type that adaptor name instead of winvideo. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 387. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 388. You can preview the video captured by the image by defining an object and associate it with the device >>vid=videoinput(winvideo,1,RGB24 _320x240) (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 389. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 390. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Now to see the video insert the following command: >> preview(vid) 391. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha You should see a window pop-up, that displays what your camera is capturing 392. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha The camera may support multiple video formats. To see for yourself all the supported formats, type >>dev_info = imaqhwinfo('winvideo',1); >>celldisp(dev_info.SupportedForma ts); Check out for yourself the display of other formats, by replacing `RGB24_320x240` with other formats, in the definition of the object vid 393. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Now to capture an image from the video, define the object vid as described before and use getdata to capture a frame from the video >>start(vid); % This command initiates capturing of frames and stores the frames in memory >>im=getdata(vid,1); >>imview(im); 394. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha You can store the captured image as a .jpg or .gif file using imwrite function >>imwrite(im,'testimage.gif'); The image will be stored in MATLAB71work folder 395. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 396. Static Processing Step Processing Real-Time Processing (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 397. Take a single picture of the arena and process it Find out critical regions and points Apply some geometry and mathematical calculations Then blindly follow a specified path (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 398. Advantages Simplest to implement Can be fairly accurate with stepper motors Disadvantages Bot goes blind because only one pic determines the bot motion Accuracy is very low especially with DC motors (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 399. Take images of the arena in discrete intervals (say Eg. 10secs/image) Process the images and find out critical regions and points Check bot orientation at these intervals and try to correct Partial feedback mechanism is implemented (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 400. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Advantages Quite simple to implement Can be very accurate with stepper motors Disadvantages Bot goes blind for a particular period of time Accuracy is compromised with DC motors Dynamic environmental changes cannot be accounted for 401. Take images of the arena continuously at a particular frame rate Process the images and find out critical regions and points Check bot orientation at every frame and correct Complete feedback mechanism is implemented (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 402. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha Advantages Very accurate for both DC and Stepper motors Gives dynamic feedback and accounts for changing environment Can give bot orientation at each point of time Disadvantages Requires more processing power Requires more memory for taking so many images 403. Real-time image processing is the best approach for any real application Dynamic feedback systems give excellent accuracy and precision and hence is the best approach (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 404. Image processing is an important tool in many applications Problem is that one needs to acquire images and pre-process them before doing actual IP Sometimes it may be required that offline image processing is not possible i.e. one needs to proceed with real-time IP or even more, processing of the video itself (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 405. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 406. Every time you want to capture an instantaneous image, you have to stop the video, start it again and use the getdata function To avoid this repetitive actions, the Image Acquisition toolbox provides an option for triggering the video object when required and capture an instantaneous frame (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 407. Every time you want to capture an instantaneous image, you have to stop the video, start it again and use the getdata function To avoid this repetitive actions, the Image Acquisition toolbox provides an option for triggering the video object when required and capture an instantaneous frame (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 408. Create an m-file with following sequence of commands: (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 409. vid=videoinput('winvideo',1); triggerconfig(vid,'manual'); set(vid,'FramesPerTrigger',1 ); set(vid,'TriggerRepeat', Inf); start(vid); (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 410. for i=1:5 trigger(vid); im= getdata(vid,1); figure,imshow(im); end stop(vid); delete(vid); clear vid; (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha 411. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha In the above code, object im gets overwritten while execution of each of the interations of the for loop To be able to see all the five images, replace im with im(:,:,:,i) 412. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha In the above code, object im gets overwritten while execution of each of the interations of the for loop To be able to see all the five images, replace im with im(:,:,:,i) 413. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha triggerconfig sets the object to manual triggering, since its default triggering is of type immediate In immediate triggering, the video is captured as soon as you start the object vid The captured frames are stored in memory. Getdata function can be used to access these frames But in manual triggering, you get the image only when you trigger the video 414. (c) 2009-2010 Electronics & Robotics Club, BITS-Pilani, Goa | Ajusal Sugathan & Utkarsh Sinha FramesPerTrigger decides the number of frames you want to capture each time trigger is executed TriggerRepeat has to be either equal to the number of frames you want t