A Novel coarse-to-fine Localization Algorithm for Automated Vickers Hardness Measurement

Vickers hardness testing is one of the most useful methods to determine the hardness of materials. To calculate the hardness of materials, the key is to measure the diagonal length of the Vickers indentation on the surface accurately. However, since this length is extremely minuscule, there are many challenges to achieve accurate measurement. Especially, when the indentation corner is cracked, the precise position of the corner cannot be obtained by conventional methods. In this paper, we proposed a method of coarse-to-fine localization to accurately locate the indentation corner. The coarse localization process can be used to determine the position and size of the indentation. During fine localization, the linear equations of the indentation edges are calculated by the line fitting method. The capabilities of the proposed method are compared to manual measurement and results are presented.


Introduction
To get the hardness of the material in the Vickers hardness test, a standard indentation is firstly required to apply a specified pressure F on the surface of the material [1]. After that, the diagonal length d of the indentation is measured and the Vickers hardness value (HV) can be calculated as follows: Where g is the acceleration of gravity. Usually, the lengths of these indentations are at micron-level. The traditional measurement process is completed by human operators to adjust the scale plate with a microscope. The disadvantages of this manual measurement are low efficiency and high labor intensity. Moreover, it is difficult to ensure consistency of the positions of the indentation corners among operators. In comparison, the use of image processing technology for indentation image automated measurement can improve the stability of measurements and greatly reduce the labor force [2]. The majority of the solutions use such classic methods as morphological operation [3,4],areamap [5],wavelet algorithm [6],space transform [3],corner detection [7],template or shape matching [8] and active contours [9]. Recently, a method for the detection of Brinell or Vickers indentation edges using convolutional neural networks (CNNs) has been proposed [10][11][12]. Normally, an industrial camera equipped with specific micro-optical lenses and an auxiliary light source is used to obtain a clear indentation image. However, affected by the surface flatness and 2 material, there may be a large difference in the light compensation effect. Also, there may be scratches and etchings on the surface of materials. These factors mentioned above may affect the accuracy of indentation measurement. Another thing that deserves special attention is the influence of image resolution. As known, the indentation measurement with the use of image processing technology can only measure the target at the pixel level. Generally, the diagonal length of Vickers indentation is extremely small (20-1400 μm). If using an indentation image with low resolution, will cause a large measurement error. Hence, the accuracy of indentation measurement is greatly limited by the image resolution. To improve the accuracy, it can make use of the position of indentation corner on indentation to calculate the Vickers hardness value. The mentioned position can be obtained by image corner detection methods. As a result, the measurement accuracy can be improved from pixel level to sub-pixel level. However, due to long-term use or non-standard operation in practice, the indentation corner may be worn or even damaged. At this time, the precise position of the indentation corner cannot be obtained by using conventional methods. Moreover, it will greatly increase the difficulty of detection. In this paper, we propose a coarse-to-fine localization method to locate the indentation corner accurately. The coarse localization can obtain the minimal bounding-box of indentation in the image. During fine localization, the linear equations of the prismatic edges of the Vickers indentation can be obtained by the line fitting method. Then, the intersection point of the edges can be calculated by the linear equations. Even if the indentation corner is abraded, its real position on indentation can still be estimated using the edge lines. This can greatly improve the accuracy of Vickers hardness calculation.

Methodology
The process flow of the proposed method is shown in Fig. 1，which roughly consists of two phases: Coarse localization and Fine localization measurement. The former is used to determine the Boundingbox of indentation in the image, and the latter is used to locate the precise position of the Vickers indentation corner on indentation.

1. Coarse localization measurements
The main process of coarse localization is as follows: firstly, to reduce the influence of material and light on image contrast, we used Contrast Limited Adaptive Histogram Equalization (CLAHE) [13] to correct the contrast of the original image. Then, the corrected image is processed by mean filtering with 5*5 cores to reduce the influence of noises. Next, the filtered image is thresholded into a binary image. Due to the less light reflected at the indentation, the image of the material surface with indentation often shows obvious light and dark changes. For such images, using variance classification methods, such as the Otsu method [14], can achieve better threshold segmentation results. Furthermore, to separate the surface scratches in the background from the Vickers indentation, the link between indentation and scratch in the binary image will be broken by morphological dilation and erosion algorithm. Finally, the bounding-box of the indentation can be determined by calculating the largest   Generally, the results obtained by rough positioning operation are suitable when the high precision measurement is not required in practical applications. To meet the requirements of higher accuracy, the procedure of fine localization measurement will be further introduced.

Fine localization measurements
Compared with the sharp corner of the Vickers indentation, the edges are easier to be located in the indentation. Since the intersection of the edges represents the actual position of the corner whether it is worn or not, the exact analytical solution for the position of the indentation corner can be calculated by the above linear equations of edges.

√ ∑ ̄
Where ̄ represents the arithmetic mean of the intercepts of N straight lines. Moreover, considering that the inner point p far from the center is more likely to introduce larger error, the Gaussian weight w i is introduced to improve the fitting accuracy. Then, the above formulas (5) and (6) can be arranged as the following weighted forms: The above formula (7) and (8)

Experimental Results and Analysis
We measured diagonal lengths and Vickers hardness values of different materials and compared the results from the proposed method and manual measurement. The experimental platform is shown in Fig. 6. The single gray output industrial camera is used in the experiment, and the output image resolution is 765*576 pixels. For manual measurement, a micrometer eyepiece with a minimum resolution of 0.1μm is used. By switching the objective lens with different magnification, it can be used to measure different sizes of indentation. The manual measurement process is completed in strict accordance with the national standard <GBT 4340.1 Vickers hardness test method>. For the sake of objectivity, the test process was independently completed by two manual operators, and five indentations were measured under the same hardness test condition. The final manual measurement data were obtained by averaging these results. The data of indentation diagonal length and Vickers hardness obtained by two methods are shown in Table 1. The results show that the maximum bias of the diagonal length obtained by two methods is about 3μm, and the average absolute difference of Vickers hardness is about 0.4%. Analysis of the data shows that the results of manual measurement are generally higher than those of our method. Moreover, with the increase of test force, the deviation of results of the two methods shows the trend of gradual increase. Two possible reasons are accounting for this result (observation): 1) Due to the wear of the Vickers indentation corner, the diagonal length measured by hand is smaller, which makes the calculated hardness value larger. 2) When switching the objective lens, the measurement error arises because of the manual adjustment of the micrometer eyepiece. Overall, the values measured by the proposed method are in good agreement with the manual measurement. Furthermore, different from the manual measurement method which is limited by the accuracy of measuring tools, our method inferred the position of corner points by the way of edge line fitting, so higher accuracy of the decimal point can be obtained in the final diagonal measurement results. To verify the robustness of the proposed method to illumination variation, we tested our method under different conditions of illumination. As shown in Fig. 7

Conclusions
To replace the manual Vickers hardware measurement, we proposed a coarse-to-fine localization algorithm to achieve the accurate calculation of the diagonal length of the indentation The preliminary experimental results show that the values measured by the proposed method are in good agreement with the manual measurement. Moreover, the proposed method is less affected by the actual position of the indentation corner. Because the diagonal length of indentation is calculated automatically, the accuracy and stability of the proposed method are better than that of manual measurement. Furthermore, it has strong robustness to illumination variation. Nowadays, more and more hardness testing instruments have begun to be compatible with various testing methods such as Vickers and Brinell. Although the edge linear fitting method in this method is suitable for Vickers angular indentation, it is not suitable for the measurement of circular indentation in Brinell hardness.As can be seen from the method of Tanaka et al., convolutional neural network has