Multi-view based template matching method for surface defect detection of circuit board

In order to avoid defects (e.g. dust, scratch, static electricity) caused by manual detection, this research proposes a non-contact defect detection approach for circuit board through use of template matching method. In the proposed approach, images of both circuit board to be detected and circuit board selected as template are obtained respectively from multiple views. Then defect index (DI) of each view is constructed based on corresponding matching results between the image to be detected and the template image. Finally, total defect index is obtained by integrating all DI from multiple views, which can characterize the existence of defects in the circuit board (e.g. lack or wrong installation of components). The proposed approach is experimentally investigated to detect two surface defects on a circuit board. Results show that the two defects can be accurately detected with no false alarm by use the proposed method.


Introduction
In recently years, the density of components on the surface of circuit board is gradually increased with the miniaturization design of electronic instruments. Many kinds of defects may occur in the surface of circuit board in manufacturing process, such as lack of installation, wrong installation, physical damage and corrosion, which seriously threaten the reliability of electronic products. As a non-destructive method, automatic optical inspection (AOI) has played an important role in defect detection of circuit board. The thermal feature of components in circuit board was extracted through infrared thermal imaging method in Literatures [1-2], then pattern classification was carried out based on support vector machine to identify abnormal states of components in circuit board. In general, this method can only be applied when circuit board is powered on, which cannot detect defects in the production and debugging process. The artificial neural network junction model was established in Literature [3], where the defects originated from wires and pads in circuit board can be identified. In order to achieve high recognition accuracy, this kind of defect detection method based on machine learning needs a large number of databases. However, for circuit board in multi-types with small batch, the valid number of databases is insufficient, which severely limits the practical use of this method. In contrast, template matching method is widely used in image recognition because it only counts the similarity between images, instead of depending on the specific content [4][5][6]. A circuit board defect detection system was designed in Literature [7] based on template matching algorithm. In the literature, a component library was established in advance where the component templates were contained. Then images in the library were matched by the circuit board to be detected. However, it is difficult to ensure that the images of all components in circuit board to be detected are completely consistent with that in the component library. Aiming at the shortcomings of above research, this research proposes a new detection method for defects of circuit board based on template matching algorithm. The image of circuit board to be detected is obtained from multiple views and then was matched with the template image obtained in corresponding views respectively. Then defect index (DI) is constructed based on template matching results. Finally, the total defect index is obtained by integrating all DI from multiple views, which can characterize the existence of defects exactly. The proposed method is non-destructive and without the need of a large number of databases.

Multiple-view based template matching method
In this section, the principle of template matching method is firstly introduced. Then defect index (DI) is constructed according to template matching method. Finally, a defect detecting method is proposed by integrating all DI from multiple views.

The principle of template matching method
Template matching method is to find out the region with greatest similarity to the template image by matching the pixels of input image with those of template image. The schematic diagram of template matching method is shown in Figure 1. Set the image to be detected as S and the template image as T. Make T start from the upper-left corner (i=1, j=1) of S, and match the gray value T (X,Y) of each point in T with the gray value S(x+x', y+y')of each point in Sij. The correlation coefficient between Sij and T is calculated by the standardized correlation matching method as [8] Then translate T to next region in S and repeat the same step until image T compared with all the sub-regions in S. Finally, the region Sij with largest correlation coefficient is the one with greatest similarity to template image T.

Defect index
In this research, the image of an intact circuit board is selected as template image T, and the image of circuit board to be detected is selected as S. Firstly, the template image T and the image S are preprocessed to make them have the same size of pixel, then they are divided into sub-regions of same size as Tij、Sij (assuming the side length is 1). For simplicity, each template image Tij is only matched to Sij which has the same location in the image. In order to characterize the existence of defects, this research further defines defect index (DI) as DI i, j 1 R i, j According to the definition, the larger the difference between the template image and the image to be matched, the smaller the correlation coefficient R (i, j), as well as the larger the defect index DI (i, j). When the DI of a sub region exceeds a preset threshold, the region can be judged as a defect region.

Integration of DI
In general, the template image Tij cannot be accurately matched by Sij due to the influence of sampling conditions, which will cause a large value of DI. For this reason, a larger threshold should be preset. However, once the threshold is increased, the detection accuracy will be seriously affected, resulting in a large number of false alarms. To solve this problem, this research proposes a method to detect defects based on the integration of DI from multiple views. As the influences of sampling conditions on the matching results from different views are not relevant, so make the DI of each region obtained from multiple views added respectively as i, j ∑ i, j , 1 i n where i, j represents the DI obtained from the i-th view. The influence of defect region on DI can be significantly enhanced and the defect region can be identified accurately with a reasonable threshold by using this method.

Experiment and results
In this section, the proposed method was examined experimentally by detecting two defects in a circuit board.

Description of the experiment
The circuit board selected in the experiment (Ultrasonic Ranging Module-KS 103, 42mm long and 20mm wide) is shown in Figure 2. The left one is an intact circuit board. The other one contains two surface defects as wrong installation of R10 (package size is 1206) and lack installation of C6 (package size is 1206) respectively, which was marked as the circuit board to be detected. In our experiment, four groups (case1 to case4) of images from different views were acquired. In the process of image acquisition, the camera (CMOS image sensor, with a resolution of up to 4032 pixels and 3024 pixels) was fixed and over against the two circuit boards. Then the images of the two circuit boards are acquired in four cases respectively with a rotation of 90 degree clockwise in each case. Then the size of the acquired images was adjusted to 1200 pixels * 600 pixels.  Table 1 shows the template matching results corresponding to different defect judgment thresholds in the four cases. It can be seen from the table that with the increase of defect judgment threshold, the number of false alarms is gradually decreased. However, when the threshold is raised to a certain extent (e.g. threshold=0.25), the true defects are not identified. This result shows that the defect region does not necessarily lead to the largest defect index due to the influence of errors in traditional single-view template matching. The template matching result obtained from a certain view (case1) is shown in Figure 3. The regions circled by red box (40 pixels * 40 pixels, which is close to the size of components) in the figure can be judged as defect region. It can be seen that although two different types of defect regions on the surface of circuit board are detected, there are 15 false alarms.