Method for rectifying image deviation based on perspective transformation

A new method for rectifying image deviation of circular instrument based on perspective transformation is presented in the paper, and the correction of circular instrument image in substation environment is realized. First of all, the digital image processing technology is used to pre-process the site image. Secondly, Canny operator is used for edge detection. According to the edge feature points, the equipment area is detected and the regional parameters can be computed. Then the perspective transformation is used to correct the image, and the positive image of the circular instrument image is obtained. Finally, the corrected tilt image is done by the rotation operation. Experimental results show that the algorithm can realize image rectification, which is simple with fast speed and high precision. The proposed method is helpful for the further recognition.


Introduction
Substation intelligent patrol by robot with portable devices instead of human is main stream in power substation. Limitations of environment is obvious during human patrol, especially for the outdoor equipment during the worse weather condition [1,2] . In addition, some substation instrument located too high to obtain information. Intelligent patrol of power substation is synchronous monitor, which collects the information by the intelligent robot and then transmit it to the console. The camera located in intelligent robot is used to capture the scene by the path that the patrol mission. We can analyze the images or movies to acquire information of devices.
One of the main detected devices in the outdoor substation is circle instrument, which is mainly divided into pointer and digital display [3] . For the pointer instrument, the performance of display recognition largely depends on the location of the center of the circle. The more accuracy the center is, the higher precision accuracy. Moreover, the perspective view of the picture is related with the recognition precision. The picture of pointer instruments is ideal circle. But for the reason of substation scene, angle of view of camera cannot be front view, as shown in Figure 1. The Therefore, developing a suitable method for fast, accurate, stable and reliable image rectification system is significant.

Ellipse fitting
In the analysis of the distort image, the first step is to detect the equipment in the image, so that we can extract the equipment area completely and get the shape feature of the region. The circle meter is the object of this paper, pre-procession such as RGB to grey, binaryzation, edge detection and other image processing technology is applied to get the edge feature points. Then according to the edge feature points, the ellipse fitting algorithm is used to find the circular dial area, and then the parameters of the long axis, short axis, center and tilt angle are obtained.

Canny edge detection
The Canny edge detection operator was a multi-level edge detection algorithm developed by John F. Canny in 1986, and belongs to the method of smoothing the derivative. Canny established the computational theory of edge detection, which is regarded as the best edge detection algorithm. The three evaluation criterion of optimal detection are as follows [4] : (1) Low error rate: the most possible actual edges are identified, and the error of noise is reduced as much as possible at the same time; (2) High positioning accuracy: the identification edges should be as close as possible to the actual edges in the image; (3) Minimum response: image can only be identified once, and the possible noise should not be identified as the edge.
Canny operator to find the marginal point of the specific algorithm steps are as follows: Step1: Smoothing image with Gaussian filter; Step2: Calculating of gradient magnitude and direction by first order partial differential finite difference method; Step3: The non-maximum suppression of the gradient amplitude; Step4: Using double threshold algorithm to detect and connect edges. The goal of Canny operator is to find an optimal edge detection algorithm. In order to meet these requirements Canny uses the calculus of variation method, which is a way to find a function that satisfies a particular function [5] . In view of the above information, the Canny operator is chosen for edge detection in this paper.

Hough circle detection
Ellipse fitting is a classical problem in data processing. It has important applications in the fields of image processing, machine vision, pattern recognition and so on [6,7] . The basic idea of ellipse fitting method is to find an ellipse for a set of sample points on a given plane, and make it as close as possible to the sample points [8] . That is to say, a set of data in the image is fitted with ellipse equation as the model, so that a certain elliptic equation can satisfy the data. Common algorithms for elliptic fitting include: least squares fitting, Hough transform fitting, five-point fitting and so on. In the two-dimensional plane coordinate system, the ellipse can generally be expressed as an algebraic form of the curve equation, as shown in the following equation (1): (1) Where A, B, C, D, E, F are the parameters to be determined for the elliptic equation. According to the mathematical knowledge, we can get the parameters in equation (1) when we know the coordinates of five points or more than five points on the ellipse.
The ellipse can be obtained by using the Hough transform. According to the elliptic equation, we can get the information about the long axis, the short axis, the center and the tilt angle of the ellipse. Therefore, when the ellipse is corrected to a circle, the device image also changes from the oblique image to the front image.

Correction algorithm
Substation environment is more complex, and there is many equipment in the station. There are different conditions of equipment installation level. Therefore, the equipment in which the preset position cannot be satisfied the front image in the area where the inspection robot can travel. It brings difficulties to the processing, analysis and identification of the equipment. Therefore, the detection and correction of the tilt target device is an important part, and it is great significance to design correction algorithms.

Perspective transformation
Perspective transformation is turn the projection of an image into a new visual plane, also known as projection mapping.  Perspective transformation refers to the use of perspective center, image points, the target point of the conditions of three-point collinear, according to the law of rotation of the shadow plane (perspective) around the trajectory (perspective axis) rotation of a certain angle, and destruction of the original projection light beam, which can still remain the same projection geometry on easel [9] .

Perspective matrix
A perspective matrix is a matrix of H whose size is 3*3, which satisfies a given point P1 is given, and all the elements are multiplied by a constant a, the perspective matrix aH and matrix H are the same. So the new perspective matrix is only turn the homogeneous point P1 into the homogeneous point ap2, ap2 and P2 corresponding to the same point on the image. Perspective matrix should only have 8 free elements, and generally make the lower right corner of the element h33 = 1 to normalize [10] .
We know that 8 unknowns need 8 equations to solve. The reason why the four pairs of points can be solved is that a pair of points provides the two equations. We assume that there are two points on the image 11 If there are four pairs of non collinear points, the set of equations can be up to 8 lines, and there is a unique solution. If more than four pairs of points, such as n pairs of points, then the equation can be up to 2n lines, and the least squares method or SVD decomposition is used to solve the matrix H.

Algorithm design and analysis
All the data in the experiment are the circular instrument images taken by the substation inspection robot during the inspection process. The process diagram of the algorithm is shown as follows: We take one of the site picture ( Figure 1) as an example to illustrate the method in detail. Preprocession of the image is fundamental for the further rectification. The site RGB picture is transferred to grey picture and then the canny edge detection is applied for the ellipse detection. Five ellipse parameters is obtained by the Hough detection, which contains the axial length, the focus point and lean angle. As is shown in Figure3, the exterior ellipse is detected, which is described in the red line, and the centre is labelled in yellow point. Because we know nothing about the original circular instrument, in this paper the hypothesis circle point coincides with the centre of the ellipse and the radius length is half the long axial length. So the information of the original circular instruments is obtained. All the matched points are known and the homography matrix can be computed by the known matched points. Perspective transformation is applied to obtain the rectification image and the front view image is illustrated in Figure 3. For the site image is not set right, the tilt must be rectified. Because the lean angle is obtained in the ellipse fitting stage, so the rotation matrix can be computed by the centre of a circle and lean angle, then the rectified image is the result, as is shown in Figure 5. From the result image we can see that the meter instrument in the site picture is reconstruct by the proposed method.

Conclusion
In this paper the edge feature points of the device are extracted by image gray scale, binaryzation, edge detection and so on. The Hough transform is used to fit the ellipse, and the regional feature of the tilting device is obtained. The image correction is realized by the perspective transformation, and the front view of the target device image is obtained. The algorithm successfully solves the phenomenon that the recognition rate of the algorithm is reduced due to the tilt of the substation equipment, and improves the device identification performance of the substation inspection robot.
On the other hand, the construction of inspection robot platform not only improves the work efficiency of the staff, but also ensures the safety of the personnel in the station. The experimental results show that the proposed algorithm is simple, fast and robust, and the successful implementation of the algorithm promotes the unattended process of substation.