Optical 3D Measurement in the Quality Assurance of Formed Sheet Metal Parts

Kari Jyrkinen*, Mikael Ollikainen**, Ville Kyrki*, Juha P. Varis** and Heikki Kälviäinen*

*Laboratory of Information Processing, Department of Information Technology,

Lappeenranta University of Technology Lappeenranta, Finland

Kari.Jyrkinen@lut.fi Ville.Kyrki@lut.fi Heikki.Kalviainen@lut.fi

**Laboratory of Sheet Metal Technology, Department of Mechanical Engineering, Lappeenranta University of Technology

Lappeenranta, Finland Mikael.Ollikainen@lut.fi Juha.Varis@lut.fi

ABSTRACT Structures based on fabricated sheet metal parts are widely used in the electronics and telecommunication industries. Particularly in the telecommunication industry, volumes are increasing rapidly at the same time as market situation is changing to more global direction. Moreover, the range of products is increasing.

Surveys in Finland have shown the need to invest in AMT (Advanced Manufacturing Technologies) in the sheet metal industry in the 1990's. Quality costs are conceived to be 2-3 percent of the net revenue of the sheet metal part fabricating industry. Since increasing quality costs are seen as a threat, better and faster quality control systems are therefore needed.

A photogrammetric 3D machine vision system and its suitability for the quality assurance of formed sheet metal parts are studied in this paper. Different kinds of formed sheet metal parts have been measured using multiple camera views and results have been analyzed. The focus is on components used in telecommunication industry. Research is connected to the field study carried out in three Finnish "state of the art" case factories.

Experiments indicate that the studied machine vision methods are applicable to sheet metal part quality assurance. The methods are accurate enough to achieve tolerances required in the sheet metal parts. It is also possible to measure complex parts relatively easy. Limitations can be found in edge detection, especially when measuring hole diameters.

Keywords: Defective sheet metal parts, Visual inspection, 3D measurements, Quality assurance, Machine vision, Image analysis

1. INTRODUCTION In sheet metal industry, markets have changed from local regional business to more global worldwide markets. It is important to minimize production costs and to maintain high quality of products to

survive in global competition. Those companies that take full advantage of modern technology will succeed. Surveys in Finland [13][14] have shown the need to invest in AMT (Advanced Manufacturing Technologies) in the sheet metal industry in the 1990's. Quality costs are conceived to be 2-3 percent of the net revenue of the sheet metal part fabricating industry. Since increasing quality costs are seen as a threat, better and faster quality control systems are therefore needed. The focus has been on AMT and less attention has been paid to the utilization of human resources. In many companies an appreciable portion of profit within reach is wasted due to the poor quality of planning and workmanship.

One of the promising technologies is the use of machine vision in visual inspection tasks without human interaction. The goal of the paper is to study possibilities of optical 3D measurements in automatic quality assurance. A photogrammetric 3D machine vision system and its suitability for the quality assurance of formed sheet metal parts are examined. This research is related to the previous work of local 2D measurements using image analysis by the authors [1].

In this paper the production chain of sheet metal part based constructions is studied from the point of view of automated visual inspection. There are production errors in high volume production work phases where costs caused by errors, many times human activity based ones, are considerable and they are inherited to the next work phases.

Two different parts are illustrated in Figure 1. The first object is naturally more three-dimensional than the second one. The objective of this paper is to evaluate the suitability of a photogrammetric machine vision system for quality assurance of 3D objects.

The paper is organized as follows: In Section 2 sheet metal production and quality assurance are discussed. The use of machine vision is introduced in Section 3. An overview of 3D measurement methods is given and developed

1

methods based on structured light and photogrammetry are suggested. Experiments and results are shown in Section 4. The experiments are focused on measuring distances, angles, and diameters in respect of measuring times. Finally, discussion and conclusions are given in Section 5.

Figure 1. Metal parts: three-dimensional and two-

dimensional characteristics.

2. QUALITY ASSURANCE IN SHEET METAL PRODUCTION

Sheet metal production and quality assurance are discussed in this section. Problems with current quality assurance are considered from the point of view of product chain, and observations are given based on research done so far. 2.1 Production Chain and Quality Assurance Constructions based on fabricated sheet metal parts are used in a wide range of different types of products. Typically, these constructions can be found, for example, in consumer goods (e.g. white brand products), means of transportation (e.g. cars and elevators), mechanical engineering (e.g. machine cabinets and covers) and electronic equipments, such as telecommunication cabinets and computer housings. One typical product, a telecommunication cabinet, is illustrated in Figure 2.

Figure 2. Telecommunication cabinet (Source NOKIA Networks, Ltd.).

Manufacturing has not historically been

utilized as a competitive weapon. However, the market place of the twenty-first century will demand that manufacturing assumes a crucial role in a new competitive field. Japan, for example, has succeeded in world markets by focusing its attention on the importance of superior manufacturing systems and techniques. Thus, manufacturing may be the "sleeping giant" within firms and prove to be a formidable competitive weapon in the global marketplace [2].

An increasing market turbulence and customer demands compels manufacturing companies to manufacture high-quality and customized products within short lead-times and at condescending expenses. These competition requirements are important for customer loyalty and long-term survival but they can eat deep into profits. The solution for stable profits and long-term survival, therefore, lies in the continuous development of manufacturing resource performance and the elimination of threats amongst them. Improved production efficiency and flexibility are the keywords for the most manufacturing companies. Two potential resources in the area of manufacturing are AMT [2][3][4][5] and empowered employees [3][6][7].

It is extensively accepted that human intelligence and human beings in an organization are the key factors in manufacturing systems and for their success [5][6][8][9][10]. Bohnhoff et al. [11] points out that the design of a technical system will always be the design of a human-machine system. They also believe that the unmanned factory appears to be impossible to implement. Corbett [12] states that the unmanned factory will not become widespread reality due to inability of most companies to manage the complexities of

2

system integration. He also comments that the operation and maintenance of a complex manufacturing system requires skilled and trained personnel.

Surveys in Finland [13] [14] have shown the need to invest in the new AMT in the Finnish sheet metal industry in the 1990's. The need to produce growing amount of customized products within short lead-times and at condescending expenses mainly for the electronics and telecommunication industry has driven the metal fabricating industry to find new ways of improving production through advanced manufacturing technology. In this run the focus has been on hard technology and less attention is paid to the workforce empowerment [15]. Because of that, not much attention has been paid to production flow wholeness and quality assurance.

In many manufacturing companies an appreciable portion of profit within reach is wasted due to the poor quality of planning and workmanship [16]. In many cases potential savings are high and assuring quality should reach the same importance as improving efficiency and flexibility. However, there is much information to be collected about production errors in the production flow in manufacturing companies. Such data can be used as a tool when production flow performance and revenue improvement activities are planned.

Information about the production flow and errors of constructions based on sheet metal parts and used by electronics and telecommunication industry is available very limited in published papers. There is much information to collect about production errors in the production flow in manufacturing companies. Such data can be used as a tool when production flow performance and revenue improvement activities are planned. This requires systematic data collection and proper tools to analyze collected data.

2.2 Studies on Production Flow Few papers can be found in literature about the production flow of constructions based on fabricated sheet metal parts. A literature review exposed no written papers handling the production flow of constructions used in electronics and telecommunication industry. Following papers concentrating to the frame of reference has been presented:

Bitzel et al. [17] describes the sheet metal process flow in general in their book. Also the production chain of a sheet metal parts based cross member of a flatbed laser machine is described as an industrial example. Berkhahn and Miyakawa [18] show general sheet metal fabrication processes in their paper. They also show examples of sheet metal parts used in a machine tool. The process flow of elevator car constructions is showed in a paper of Kanamouri et al. [19]. One selected

example [17] of described sheet metal processes is showed in Figure 3.

Figure 3. One sheet metal process described. [17]

Ollikainen [20] has listed production activities in sheet metal part fabricating industry in his paper. According to Ollikainen activities include following work phases in manufacturing function of the company:

NC-programming. • •

•

• •

Part fabricating operations; 2D-parts, bending, joining, assisting work phases. Surface treatment operations; pre-treatments, surface treatments, after-treatments. Assembly operations. Packing and transportation arrangements.

3

Warehousing operations. •

•

•

•

In the electronics and telecommunication industry products have become more global and very much attention is paid for product usability, durability, and quality. Many standards must be fulfilled. Production chain can therefore be very complicated in individual sheet metal parts. Ollikainen [21] has described the eventful production chain of a front panel of telecommunication cabinet plug-in unit.

It has been noticed that in literature review no written papers could be found about error distribution or origins of production errors in the production flow of constructions based on fabricated sheet metal parts. 2.3 Case Factories All three case factories are well known Finnish factories. It is generally accepted that these factories represent advanced activities in their manufacturing operations. Factories A and C are parts of larger consolidated companies. All case factories manufacture products for global distribution. The turnover of the consolidated companies is representing quite a remarkable part of the annual Finnish turnover in sheet metal fabricating industry. Branches of manufacturing activities in case factories are listed below:

Factory A manufactures electromechanical locks. Factory B manufactures sheet metal parts based constructions for electronics, telecommunication, and automotive industry. Factory C manufactures custom outdoor and indoor enclosures for telecom applications such as wireless base stations, switching systems, and network access equipment.

The production flow in each case factory is different. Different fabricating methods are used. Also batch sizes and annual production figures are different in each case factory. A common factor for every factory is mechanical constructions based on sheet metal parts and used in electronics and telecommunication industry.

2.4 Results from the Field Study The number of the parts tracked in the field study was 732724 pieces. A total of 84011 production errors were reported. The share of the origins of the production errors is presented in Figure 4. From this picture we can see that most of the production errors are based on human errors in the production flow. Still one result from the field study was that most of the human errors are caused in mass production phases and manually operated work phases. Typically these human based production errors in mass production phases are caused by defective process settings and faulty maintenance. Situation is alarming in mass production phases, where production volumes are high and cycle times

are very short. The risk of numerous defective parts is obvious and the risk that defective parts are inherited to the next work phases in clear. This advocates the use of intermittent on line-inspection.

68,8

84,8

70,4

21,7

9,616,0

0,65,4

11,18,9

0,2 2,50

10

20

30

40

50

60

70

80

90

100

FACTORY A FACTORY B FACTORY C

% O

f all

prod

uctio

n er

rors

in e

ach

fact

ory

Human activity based errors-categoryManufacturing technology based errors-categoryMaterial based errors-categoryOther errors-category

Figure 4. Origins of production errors.

3. INDUSTRIAL MACHINE VISION We will begin this section by discussing machine vision in general and its role as an advanced manufacturing technology. Then we will review the different types of methodologies that can be used for optical measurements of three-dimensional objects, focusing on methods that could be used in the quality control of sheet metal products. Finally, we present in more detail the methods that have been employed in this study, particularly the use of photogrammetry and structured light in the measurement of sheet metal objects. 3.1 Machine Vision Computer vision means the analysis of a scene using a camera connected to a computer. The term machine vision refers specifically to application oriented image analysis. Machine vision has become an important tool in industry mostly because the level of automation has continually increased. For example, in Finland the machine vision program of TEKES (Finnish National Technology Agency) started in 1992. When it finished in 1996, annual machine vision sales had grown 130 % from 140 million to 320 million FIM [22].

The benefits of automation, including flexible manufacturing and lower labor costs, can be utilized even more efficiently when combined with machine vision. Another reason to use machine vision is the possibility of a 100 % inspection rate, that is, the inspection of every product. In addition, the reliability of machine

4

vision systems compared to humans is often significantly greater, especially in repetitive tasks like product inspection. The reason for this is, naturally, human fatigue. Machine vision is also a preferable measurement method when product damage is hard to avoid since it is a non-contact method, unlike most of common measurement methods. It is also a convenient technique to use in hazardous environments, such as nuclear power plants, where human presence is undesirable.

Machine vision has basically three industrial application sectors. These are recognition, guidance, and inspection. Recognition can be understood as searching for matching or similar images. For example, the searching of criminal records can be understood as a recognition task. Classification of products is another important group of recognition tasks. The recognition of three-dimensional objects from two-dimensional images has also attracted a great deal of research attention.

Guidance tasks primarily include robot guidance. Vision is nowadays one of the most important sources of information for industrial robots. Applications include a wide range of areas from chip placement to autonomous vehicles. Stereo and three-dimensional vision techniques are increasing in importance also in this area.

Inspection is a process to determine if a product conforms to a given set of specifications. It is a quality assurance task. It usually incorporates measurements of individual part features such as surface quality and geometric dimensions. Industries which routinely utilize machine vision in inspection include the electronics and automobile industries. The increased miniaturization and complexity of printed circuit boards and integrated circuits has promoted the development of a large number of machine vision applications. In Finland, the application areas of machine vision include the telecom and electronics industry, pulp and paper industry, graphic arts, burning processes, condition monitoring, food and beverage production, and wood and metal surface quality control.

A machine vision system consists of several elements. In practice, most vision systems encompass the same components. These components are as follows: (a) image acquisition, (b) pre-processing, (c) feature extraction and representation, and (d) classification and matching. The components and their relationships are presented in Figure 5. Each component in turn can include hardware, software, and methodological elements.

Image

acquisition

Pre-processing

Feature extraction

Classification

Figure 5. System model. 3.2 3D Measurements The objectives of three-dimensional measurements range from three-dimensional object recognition to precise inspection of dimensions of a 3D object. Traditionally, the 3D measurements have been performed by equipment based on mechanical contact. However, nowadays the trend is towards optical non-contact measurements. Many of the optical measurements employ machine vision, but there are also techniques such as laser range finders that are not strictly considered machine vision methods as they do not use any camera or related device.

In 3D measurements, the location of certain features are determined in three-dimensional world coordinates. However, this is not enough, because the specifications of products usually determine relationships between features rather than the features themselves. For example, the distance of two holes may be specified to be within a margin, rather than specifying the locations of individual holes. The 3D inspection process can be specified as in Figure 6. As in all kinds of precise measurements, calibration is an essential step in 3D machine vision. Calibration means the adjustment of the scale of an instrument to the true values. In machine vision, this means the determination of the locations of the cameras and optic distortion. The measurement phase contains the steps according to general machine vision system model presented in Figure 5. In three-dimensional machine vision an object cannot be seen entirely from one image due to occlusion. For that reason, several images have to be measured and their results combined. Finally, the inspection

5

ends when the measurements are compared to the specifications.

Calibration

Measurement

Result merging

Model comparison

Figure 6. 3D inspection process.

Most optical three-dimensional measurements are based on triangulation. Using triangulation the three-dimensional coordinates of a point can be calculated from a pair of two-dimensional projections with known calibration. That is, for a feature to be located in the three-dimensional world coordinates, at least two cameras must be able to see it. The principle is illustrated in Figure 7.

Optical methods for 3D measurement can be classified based on several criteria. They can be categorized as active or passive based on the features they use. Passive methods rely on features present in the measured objects themselves, while active ones produce the features by projecting a regular pattern of light on the object. A taxonomy of different methods is presented in Figure 8. While the ultimate goal of computer vision research is to understand the environment in natural lighting, in practice the passive systems do not perform very well in measurement tasks as they are not usually very precise. For that reason, active systems are more popular in industrial applications.

All of the active methods presented in Figure 8 have sufficient accuracy for the measurement of sheet metal parts. Measurement time is one of the governing factors in the application. The fastest methods are usually capable of measuring the coordinates of several points at the same time. This capability is mostly related to structured light where a high-resolution measurement grid can be projected to the object. Sometimes the active methods suffer if the object surface is highly reflective. Particularly, in metal surfaces the specular reflections can present

problems. Also the surfaces which absorb light well can be troublesome.

plpr

z

z ~ 1/(pr-pl)

Figure 7. Triangulation.

Active

Optical methods

Passive

Photogrammetry

Shape from shading

Shape from texture

Shape from focus

Imaging radar

Interferometry

Structured light

Photometric stereo

Figure 8. Optical 3D measurement methods.

3.3 Structured Light and Photogrammetry Photogrammetry, literally meaning “measurement of photographs”, refers to methods which employ triangulation to infer three-dimensional information from two-dimensional images. Structured light covers the methods to produce more measurement features to an object by projecting a light pattern on it. The pattern can be a single point, a grid of points, a line, several lines, or some more complex pattern. When the position of the light source and a camera, or the positions of two cameras are known, the coordinates of a reflected features can be calculated using triangulation. For this reason, structured light is sometimes referred to as active triangulation. Structured light is one of the most industrially important methods in 3D machine vision.

The Mapvision 4D photogrammetric measurement machine [23] will be now presented (see Figure 9). The system consists of four cameras, control unit, scanner, and light source. The halogen light source and the scanner are used to project a four-times-four point matrix on the surface of the measured object. The point matrix can be moved using two controllable mirrors. The cameras are then used to measure the 3D locations of the projected points. At least two cameras must be able to see each point to allow 3D measurement. The use of more cameras increases accuracy and

6

allows the assessment of the measurement errors for each measured point. The 3D position of each point is determined using epipolar geometry. If an object needs to be measured from several directions, the base under the object can be rotated, and the measurements can be combined by measuring reference points located on the base.

Figure 9. Mapvision 4D measurement machine.

The manufacturer, Mapvision Ltd., states

that the measurement accuracy of the equipment is 5 micrometers and the operating range is 200x200x130 mm which is sufficient for smaller sheet metal parts. The manufacturer also produces models with a larger operating range. Measurement speed is up to 80 points per second which is adequate if each product does not need to be measured. However, the speed is superior to the operating speed of CMMs (Coordinate Measuring Machine). Setup time of the measurement is from 30 seconds to 5 minutes for a calibrated measurement system. The calibration of the system takes from 1 hour to 2 days.

4. EXPERIMENTS AND RESULTS Next experiments are introduced and results are shown. Mapvision 4D, the measuring system described in Section 3, was applied to the problem domain. The experiments are focused on measuring distances, angles, and diameters in respect of measuring times. 4.1 Experimental Setup Some real sheet metal components in production were measured by Mapvision 4D to test the applicability of the device for the sheet metal industry. CAD models and some measurement data were available for comparisons. The objective was to find out whether the measurement device is accurate enough and would it be suitable for quality control in the production process of participating companies. Sheet metal surface can strongly reflect light and objects have to be measured fast enough

so that the device can be used in the production process.

The measurement time is proportional to the amount of measured points and the theoretical measurement speed is 80 points per second. Although in reality we do not achieve exactly this speed, we can measure well enough points to analyze some features quite fast. The measured points can be compared to the CAD model fast, if there is software written, which transforms the points to the same coordinate system with the CAD image. In this case there were no such programs available, and the most time consuming part was to fit the data to the CAD model. This restricted the size of test series and the amount of different kind of measurements.

The setup time of basic measurement is one to two minutes if the measurement is set up from the beginning. If the object is also rotated, some five minutes is needed for the setup. To repeat a similar measurement for the same kind of objects takes as much time as you need to change the part inside the machine and to push a button to restart the measurement.

The three-dimensional coordinates of every point measured are stored in a text file. Two adjacent overlapping images are in most cases enough to count the coordinates of a point. Using more images it is possible to evaluate the accuracy of a measured point. This evaluation is expressed as a quality index (QI) and it is stored into the same file with the amount of cameras used for the measurement. QI defines the quadratic distance from epipolar lines to the computational location of the point. An example of measured data is shown in Table 1. Table 1. An example of the output data of Mapvision 4D. Point X [mm] y [mm] Z [mm] QI [mm] Number

of cameras

8287 -42.428605 68.969598 -5.578050 0.005703 4 8288 -38.864094 70.791597 -5.556362 0.015460 3 8289 -33.746616 80.917152 -5.567319 0.022083 3 8290 -40.146464 70.764543 -5.587679 0.011495 4

4.1 Diameters An edge cannot be precisely measured because it is not precisely defined. Only the points on both sides the edge can be measured. Machined edges are never exactly sharp so it is almost an impossible task to measure the position of an edge exactly. On the other hand surfaces are easy to measure and the edge can be then defined as a transversal of two planes. In practice, edges can be seen from the measured point cloud and also somehow measured.

To find a diameter of a hole one should find out the border of a hole – it is not exactly defined, in the same way as edges. A part of a sheet metal component was measured ten times to find

7

out the accuracy and repeatability. The object was removed and put back between different measurements. The first five measurements contain 63700 points, and the other five were measured with a bit better accuracy, 130000 points. The first five measurements took 16 minutes 20 seconds, and respectively, the last five ones 33 minutes 20 seconds. The results were filtered using the QI value 0.05 mm; so only points with approximated error of 0.05 millimeters and smaller were used. The most inaccurate points are near edges and borders, so filtering removes especially points from the border of the holes. The border lines of holes were fitted to the border points half-automatically using I-DEAS Imageware Software. A point cloud of one measurement with measurement results can be seen in Figure 10 and the results of the measurements in Table 2.

Figure 10. A point cloud of one measurement with dimensioning. Table 2. Measurements of a diameter.

Measurement Diameter [mm] 1 4.53 2 4.53 3 4.57 4 4.56 5 4.51 6 4.55 7 4.50 8 4.61 9 4.55

10 4.54 The designed diameter is 4.0 ± 0.1 mm. A 2D video measuring device SmartScope ZIP 250 gave 4.21 mm as the real diameter. The average of all measurements is 4.55 mm and the standard deviation 0.03 mm. The measurement results are very near each other and the measurement accuracy of 63700 points seems to be already good enough compared to the better accuracy tests. The measurement results of the diameter are too large. This is due to the difficulty to measure the exact place of an edge or a border. For the purposes of the sheet metal industry it is in many cases enough

to verify that all the holes exist and they are in right places and that can be easily done. That was clearly seen when a measurement result was compared to a CAD model, in which a hole was missing. Holes are punched to the sheet metal before bending. If the right diameters are essential, these features can be measured using 2D measurement devices. Those are usually faster and more accurate when they can be used. 4.2 Distances The center of the hole cannot be measured directly, but it can be approximated using the information of the borders of the holes. Although the diameters of the holes are systematically too large in all measurements and the border is not then exactly found, the border is situated evenly wrong from the center of the hole. As we see from previous measurements, the results are very near in all measurements, so we can approximate the center point quite reliably.

The distance between the holes, which was measured, can be seen in Figure 10. The measurement data is the same than in Section 4.1 and the results are presented in Table 3. The designed distance is 19.10 ± 0.3 mm. The average of the measurement is 19.06 mm with the standard deviation of 0.0814 mm. The real distance measured by SmartScope is 19.14. The real distance is inside the tolerance and so are all measurements, respectively. Table 3. Distance of centers of two holes.

Measurement Distance [mm] 1 18.95 2 19.08 3 19.12 4 19.02 5 19.02 6 18.97 7 18.99 8 19.18 9 19.17

10 19.09 4.3 Angles Different kinds of angles cannot be measured with two-dimensional measuring devices, such as the SmartScope. After the bending is made, it is important to verify that the product is according to the specification. A part of a bracket was measured ten times to define the angle between the bracket body and the bracket wing. A sample point cloud can be seen in Figure 11.

In theory six points would be enough to fit two planes and to measure the angle between the planes, but in practice it is easier to distinguish, which point belongs to which plane, when there

8

were more points available. 32500 points were measured each time and the measurement took 8 minutes 20 seconds. Between the first five measurements the object was removed and put back, and during the last five measurements the object was not moved. Measured points were filtered and only points with Quality Index 0.05 and better were selected. Results of the measurements are shown in Table 4.

Figure 11. A part of measured point cloud of a bracket. Table 4. The angle between body and wing of a bracket.

Measurement Angle [°] 1 79.80174 2 79.83737 3 80.00693 4 79.63801 5 79.69168 6 79.44317 7 79.52221 8 79.54170 9 79.47286

10 79.44444

The angle is 87.0 degrees according to the specification. The angle was observed to be out of tolerance also with CMM. The average of all measurements is 79.64 degrees, the average of measurements, when the object is moved, is 79.80 degrees, and without moving, 79.48 degrees. The maximum differences are +0.37/-0.20. The standard deviation is 0.16 degrees, which also indicates the accuracy of the measurement.

The results differ less when the object is not moved between the measurements but there is no significant difference. The difference of average values in the first and last measurement part is most probably due to fitting which was made by hand. The first and last parts were fitted separately. The

accuracy could have been even better, if fitting had been done more carefully. However, the results are in most cases already good enough. A difference of a degree in such a small part may not be significant. Bigger deviation will be noticed in every case. 4.4 Measuring Times In theory 80 points per second can be measured. Five different scenes can be processed in a second and 16 points is projected to the surface of the measured object at a time. In our tests the source of light was a bit broken and only 13 points were projected at a time. This fact will taken account in this section.

Only the measured values of those points, which are seen at least by three different cameras, will be measured and stored in the file. In the standard measurement points are divided equally inside a square, which corners can be defined by the user. If there are holes in the measured object, for example, points projected to the holes will be lost. Only the points accurate enough can be selected for further processing by using the QI value to filter the points. This reduces the number of the usable points. Measurement efficiency of one test series can be seen in Table 5. Table 5. Amount of usable points in measurements. Measurement Percentage

of points measured compared to max. possible

Percentage of points fulfilling accuracy recuirement

Percentage of points used compared to optimum

Measurement speed in practice [points per second]

1 75.40 95.80 72.23 57.79 2 78.50 96.60 75.83 60.66 3 80.20 96.90 77.71 62.17 4 81.00 97.50 78.98 63.18 5 79.40 97.10 77.10 61.68 6 84.70 96.70 81.90 65.52 7 84.30 97.20 81.94 65.55 8 85.00 85.00 72.25 57.80 9 86.60 86.80 75.17 60.14 10 75.90 90.40 68.61 54.89

The values are from the measurement in Section 4.1. On the average 18.9 % of projected measurement points could not be measured. Filtering with QI value 0.05 removed 6 % of the points in average. Altogether 76.2 % of all potential measurement points were used for comparison, which means that the measurement speed was about 61 points per second. The object was positioned quite optimally and using more difficult objects the result is worse. In the measurement in Section 4.3, when there were surfaces in different angles, less points was got with a bit worse QI values. A part of the points projected do not hit the object. The actual speed to get usable points for fitting can then be under 40 points per second, one half of the optimum. The best measuring times can be achieved by carefully positioning the object and

9

cameras and by correctly setting illumination and measurement parameters. 4.5 Comparison to the CAD Model The most important benefit is achieved when the measured point cloud is compared to the CAD model. A bracket, a part of which was measured in Section 4.3, was measured very thoroughly. To be able to measure a three-dimensional object, it has to be rotated. In this test we measured the object in 16 different positions and there were 130000 possible measurement points per position. Altogether over a million points were measured, a bit over a half of possible maximum amount. Measurement took almost nine hours.

There were far enough points for comparing to the CAD model, so the data was filtered with very small QI value, 0.003. Still 82.4 % of the points fulfilled this requirement. The data was imported into I-DEAS Imageware Inspection software and fitted to the CAD model by hand. This could be easily automated. The comparison can be seen in Figure 12. Errors are shown as different colors in a color bar. The leftmost color represents the error of 1 mm and the rightmost color represents the error of –1 mm. In the center the error is zero. The twisted wing can be easily noticed and every missing feature could be seen as easily. Checking the tolerances could be automated and the system could be integrated as a part of a production process.

Figure 12. Comparison of the measured point cloud of a bracket to the CAD model after fitting.

5. DISCUSSION AND CONCLUSIONS The quality assurance of formed sheet metal parts has been studied in this paper. It is essential to detect defects in parts caused by errors in high volume production. Usually, these errors are done by humans in the production chain, and they are inherited to the next work phases. Moreover, there are other errors caused, for example, by damaged tools and machines. There are local defects and more global ones, and besides metal parts can look two-dimensional of three-dimensional. There are several details to inspect: are details like holes in correct places, are they mutual positions correct,

are dimensions of an object within tolerances, are 3D shapes according to CAD models, etc. It was assumed that a machine vision system could be used in automated visual quality inspection. This opinion was encouraged by the previous successful study with 2D objects [1]. A photogrammetric 3D machine vision system was used in experiments with metal parts that were fundamentally three-dimensional. The Mapvision 4D measurement machine was selected and ground truth was measured with a 2D video measuring device, called SmartScope ZIP 250. Three experiments were performed in order to find the details of a metal part: diameters, distances, and angles. Mapvision succeeded well with angles and distances, where 2D SmartScope cannot be used. However, with diameters there were problems since edges of a hole could not be detected accurately. One of the reasons is that most part of a projected point must be reflected from an object to get an accurate measurement. Thus, surfaces can be fitted very accurately, and those characteristics, which can be measured using the surfaces, can be measured accurately without a bias. Measuring times vary according to points used in the estimation. The real measuring speed varies also according to the shape and complexity of the object, and the desired resolution. In the test with diameters the measuring speed was around 60 points per second, and in the test with angles 40 points per second.

Another experiment was to compare the measurement of the whole object to the corresponding CAD model. The results are quite accurate, but unfortunately measuring times are very high.

It can be concluded that the results verify that a machine vision system can be used in quality assurance. Measuring is accurate enough. However, measuring times may be too high for the on-line inspection of every metal part. Moreover, measuring devices should become more popular, so that their prices would decrease in order to enable more profitable return on investment. Another issue for further research is how this kind of device should be integrated to the manufacturing system, keeping in mind space limitations in a factory. However, it seems that these kind of intelligent vision-based systems are needed to survive in the global market competition in future.

ACKNOWLEDGEMENTS Finnish National Technology Agency (TEKES) and East Finland Graduate School in Computer Science and Engineering (ECSE) are gratefully acknowledged for financial support. The authors would like to thank Mr. Jari Selesvuo for useful comments.

10

REFERENCES [1] Kyrki, V. and Kälviäinen, H., “High

Precision 2D Geometrical Inspection,” Proceedings of 15th International Conference on Pattern Recognition, Barcelona, Spain, September 3-8, 2000, vol. 4, pp. 779-782.

[2] Mechling, G., Pearce, J. and Busbin, J., “Exploiting AMT in small manufacturing firms for global competitiveness,” International Journal of Operations & Production Management, vol. 15, no. 2, 1995, pp. 61-76.

[3] Adeleye, E., Yusuf, Y., Sivayoganathan, K. and Al-Dabass, D., “An Exploratory study of the impact of process automation and employee empowerment on competi-tive capabilities and performance,” 16th International Conference of Production Research (ICPR), 2001.

[4] Udo, G. and Ehie, I., “Advanced manufacturing technologies - Determinant of implementation success,” International Journal of Operations & Production Management, vol. 16, no. 12, 1996, pp. 6-26.

[5] Tranfield, D., Stuart, S., Ley, C., Bessant, J. and Levy, P., “Changing organizational design and practices for computer-integrated technologies,” International Journal of Technology Management, Special Issue on Manufacturing Strategy, vol. 6, no. 3/4, 1991, pp. 211-221.

[6] McEwan, A. and Sackett, P., “The human factor in CIM systems: worker empowerment and control within a high-volume production environment,” Computers in Industry, vol. 36, no. 1-2, 1998, pp. 39-47.

[7] Johnson, R. and Thurston, E., “Achieving empowerment using the Empowerment Strategy Grid,” Leadership & Organization Development Journal, vol. 18, no. 2, 1997, pp. 64-73.

[8] Zhou, Y. and Chuah, K., “Human intelligence: the key factor for succesful intelligent manufacturing,” Integrated Manufacturing Systems, vol. 11, no. 1, 2000, pp. 30-41.

[9] Cook, J. and Cook, L., “Achieving competitive advantages of advanced manufacturing technology,” Benchmarking for Quality Management & Technology, vol. 1, no. 2, 1994, pp. 42-63.

[10] Williams, D., “Examing the CIM equation,” The Fabricator, vol. 22, no. 4, 1992, pp. 47-49.

[11] Bohnhoff, A., Brandt, D. and Henning, K., “The dual design approach as a tool for

the interdisplinary design of human-centered systems,” The International Journal of Human Factors in Manufacturing, vol. 2, no. 3, 1992, pp. 289-301.

[12] Corbett, M., “Strategic options for CIM,” Computer Integrated Manufacturing Systems, vol. 1, no. 2, 1988, pp. 75-81.

[13] Varis, J., ”Joustavan konepaja-automaation mahdollisuudet levytuote-tuotannossa,” (in Finnish), Lappeenranta University of Technology, Finland, 1988.

[14] Ollikainen, M., ”Levytuotetuotannon nostaminen uusilla levytyökoneilla, oheislaitteilla ja laitejärjestelmillä,” (in Finnish), Lappeenranta University of Technology, Finland, 1990.

[15] Ollikainen, M. and Varis, J., “The Utilization of Workforce Potential in the Finnish Sheet Metal Industry,” (in Finnish), Recearch Report 31, Lappeenranta University of Technology, Finland, 2001.

[16] Porter, L. and Rayner, P., “Quality costing for total quality management,” International Journal of Production Economics, vol. 27, no. 1, 1992, pp. 69-81.

[17] Bitzel, H., Borscherd, J., Muller, J., Neidhart, F., Parey, K., Rau, A., Riecke, S., Schmidt, A., Trentmann, G., Vorländer, G., and Zimmermann, K., The fascinating world of sheet metal, TRUMPF GmbH + Co, 1996.

[18] Berkhahn, G. and Miyakawa, N., “Developing a unit-Based production system,” The Fabricator, vol. 23, no. 10, 1993, pp. 22-27.

[19] Kanamori, Y., Kojima, K., Okazaki, T., and Hashigushi, N., “A FMS to process elevator's sheet metals,” IEEE Symposium on Emerging Technologies & Factory Automation, v3, 1995, Proceedings of the 1995 INRIA/IEEE Symposium on Emerging Technologies and Factory Automation. Part 3 (of 3), Oct 10-13 1995, Paris, France, pp. 375-380.

[20] Ollikainen, M., “Improvement of the operations is a customer-oriented sheet metal component production,” Lappeenranta University of Technology, Finland, 2000.

[21] Ollikainen, M., “DFM(A)-aspects of fabricating front panels for telecommunication cabinet plug-in units and use in similar product types,” Research Report 30, Lappeenranta University of Technology, Finland, 2001.

[22] Machine Vision 1992-1996, Finnish National Technology Agency (TEKES),

11

TEKES Technology Programme Report 15/96. Final Report, 1996.

[23] Leikas, E. and Valtonen, T., ”The 4th dimension in measuring,” Machine Vision News, 7:6, 2002.

12