Substation Equipment Fault Identification Based on Infrared Image Analysis

As a key part of the power system, the substations undertake important work. With the development and construction of smart grids, the status data, image monitoring data, and environmental meteorological data of power systems are gradually being integrated and shared on a unified platform. Traditional models based on theoretical analysis are difficult to deal with multi-dimensional, massive data set information. Under this background, starting from the inherent law of the data itself, the use of machine learning methods combined with the infrared image data of the equipment can achieve intelligent identification and early warning of substation equipment failures. This paper first uses the convolutional neural network to identify the substation equipment in the picture, and then combines the infrared image to perform image registration. Finally, the deep belief network is used to determine whether the device is in wrong condition. The overall substation equipment fault identification is tested on real data, and the experimental results show that the proposed method has high accuracy.


Introduction
As an important part of the entire power system, it is of great significance to maintain the normal operation of substation equipment. When performing fault diagnosis and analysis on substation equipment, it is required that the constructed model can promptly display the potential fault information existing in the equipment [1]. Relying on the fact that a large number of monitoring devices acquire data in real time during the operation of the substation, these collected data can be compared with historical data previously stored, and then a rough judgment can be made on the operating state [2].
Since most transformer failures occur due to long-term accumulation, failures occur more suddenly during transformer operation. Therefore, it is of great significance to timely and accurately discover the potential operation hazards of the transformer for maintaining the stable operation of the system [3][4]. Generally speaking, the potential faults of the transformer can be divided into internal faults and external faults. The external faults of the cabinet mainly include faults in the external connection of the conductor, faults in the cooling device and oil circuit system, and faults in magnetic leakage. The poor external connection failure of the conductor is due to the fact that the connection point of the transformer box and the external wires are not strong enough, resulting in a larger resistance at this point, and then more heat is emitted from this point. The failure of the cooling device and the oil circuit system mainly occurs in the cooler, explosion-proof tube or some external oil circuit system. These fault features can be clearly reflected in the infrared image. When the transformer has a magnetic leakage, overheating and loss will occur, making the corresponding temperature of the screw abnormally high, and ultimately disturbing the transformer's proper operating state.
The internal faults of the cabinet are mainly due to the insufficiency and defects of internal devices such as coils, iron cores and leads. The current method does not directly detect the fault inside the cabinet. However, different internal fault defects will lead to different thermal state distribution on the surface of the cabinet. Therefore, by analysing the thermal state distribution diagram displayed on the outside of the cabinet, it is also possible to roughly analyse the type of faults existing in the device. For dry-type transformers, when a short circuit occurs between the iron cores or the iron cores are connected at multiple positions and large points, thermal power will be generated due to the short circuit, and some of the heat will be directly radiated out, which can be directly detected by infrared imaging. For wet voltage devices, because the winding and iron core will be placed in the middle of the oil tank, the surroundings are filled with transformer oil. When a local fault occurs inside and heat is generated, it is generally not displayed on the thermal image due to the cooling and diffusion effect of the oil. Therefore, other methods such as dissolved gas in oil are needed for analysis [5][6].
There have been some studies on the analysis of the operation status of substation equipment based on image analysis. [7] uses a combined SVM and wavelet multi-resolution analysis to evaluate the condition of overhead power distribution system. This principle can also be extended for assessing the condition of both insulators and sagging of the lines. [8] takes the convolution neural networks to extract features of infrared images, and then uses the vector of locally aggregated descriptors to aggregate them. [9] takes the BP neural network to detect the fault occurring in solar photovoltaic system. [10] uses the K-means algorithm to cluster the infrared images into five regions, then the statistical characteristic in each region is detected, which are taken as the input of SVM.
The current research mainly considers the decomposition of the device image or only uses the RGB image data to classify the operating states. However, there is no specific implementation plan for the complete closed-loop analysis system. In addition to this, the joint analysis of RGB and infrared images is not considered. To address this problem, this paper first uses convolutional neural network (CNN) to identify the substation equipment in the picture, and then combines the infrared image for image registration. Finally, the deep belief network (DBN) is used to determine whether the device is faulty.

Image pre-processing
Because there are many types of faults in electrical equipment, and there are certain differences between each type of fault. In order to use infrared image data to locate the fault, the image needs to be segmented. Image segmentation is the use of data such as grayscale, depth, texture, and colour to divide a picture into one or more regions.
For the image segmentation of infrared data of electrical equipment, there are mainly the following characteristics: First, the infrared radiation signal is relatively weak. If the picture is collected in a remote place, the obtained picture will be blurred. Secondly, infrared signals will be disturbed by many external environmental factors, so usually only the approximate shape of the object can be obtained. Finally, there may be multiple electrical devices in an infrared photo, and there will be overlap between them, affecting the accuracy of the analysis. In response to these problems, this paper first uses mean filtering to denoise infrared pictures, then uses SIFT feature matching to stitch the transformer images, and finally uses the OSTU method to segment the pictures.
Mean filtering is a linear filtering method that averages the pixel values of adjacent areas. Taking the target point as the centre, calculate the eight adjacent pixels to get the mean value.
The OSTU algorithm uses the special nature of the grey value of the image to segment and divide the image. Calculate the variance between the target threshold and non-target threshold classes to measure the degree of difference between these two. Let X be an L-level grayscale image, and divide all pixels in the image into target class C 0 and non-target class C 1 by threshold k. The grayscale values of all pixels in the C 0 domain are taken the value range in [0, k-1], and the grey value range of all . The maximum variance between classes is then: The feature points detected in the scale space have scale invariance. According to [11], at the image (x, y), L(x, y) is the scale of the feature point, and the gradient length and direction are (2) and (3).
Calculate the gradient and direction of the pixels in the neighbourhood of the feature point, and divide the gradient direction into 36 parts. A statistical gradient histogram at every 10 degrees is used to calculate the weighted gradient value in each direction. The direction with the largest value is the direction of the feature point at that location. By assigning the main directions to the feature points, the algorithm detects that the key points have translation, scale and rotation without deformation.
Finally, feature vectors are established at feature points to facilitate the measurement of similarity in the feature matching stage. Feature vectors not only contain information about feature points and neighboring pixels, they can also eliminate the effects of changes in brightness to a certain extent.
In order to ensure the rotation-invariant feature, the image needs to be rotated first so that the main direction of the feature point becomes the coordinate axis. According to [11], the rotation coordinate transformation is (4).
Then take the feature point as the centre, calculate the gradient value and direction in the area. Calculate the gradient and direction of a total of 8 directions of horizontal, vertical and diagonal. After obtaining the feature vector, the picture is further stitched.

Image classification based on convolutional neural network
Common types of substation equipment are shown in Figure 1, which mainly include insulators, highvoltage bushings, sleeve, oil tanks, oil conservator, lightning arrester, current transformers, voltage transformers and isolating switches. This article uses the CNN network to classify these types. The basic structure of CNN includes input layer, convolution layer, activation layer, pooling layer, fully connected layer and output layer. The CNN network structure of this paper is shown in Figure 2.

Figure 2. CNN framework
The calculation formula of the convolution layer is: The activation layer is a non-linear mapping of the results of the convolution layer. This article uses the commonly used ReLU activation function.
As a down-sampling layer, the pooling layer is usually located in the middle of continuous convolutional layers, which can reduce the features while ensuring the local invariance of the features. Common pooling methods include max pooling and average pooling. This article uses the max pooling method.

Transformer fault diagnosis
By segmenting and extracting images and selecting appropriate features, intelligent diagnosis of the operating status of the equipment can be achieved, and errors and uncertainties caused by manual reading of infrared images can be avoided. This paper uses the DBN network to achieve intelligent classification of different equipment failures.
Restricted boltzmann machines (RBM) is a probabilistic graphical model of random neural networks. The output of RBM neuron has only two states: activated and inactive, and each output state has a probability to be determined. With the improvement of computer computing level and the continuous improvement and optimization of algorithms, RBM has been practically applied in various machine learning algorithms. RBM is a relatively simple two-layer network model. The network result is shown in Figure 3, which includes the visible layer and the hidden layer. Since RBM is an energy-based probability model, it is necessary to first define an RBM energy function, and then introduce the corresponding distribution function through this defined energy function. The formula for calculating the energy function of RBM is as follows: DBN is composed of multi-layer stacked RBM network and the last layer is logistic regression. The network structure is shown in Figure 4.  Figure 4. Structure of DBN The training process of the DBN network includes two processes: pre-training and fine-tuning. First, unsupervised pre-train the RBM layer by layer and deeply mine the hidden feature information of the data. Only one layer of RBM is trained in each training epoch. After the training is completed, the next layer of RBM is trained and stacked, and then the label combined with the sample is used to adopt a back propagation algorithm for supervised tuning to achieve classification.

Performance of substation equipment classification
The original submission equipment dataset consists of 1620 monitoring image sets of RGB image and infrared pictures, which is provided and annotated by the state grid power supply company. To promote the invariance and robustness of the network, some shift and rotation actions are made to the original pictures. After data augmentation, the dataset is enlarged to 16200 sets of images. Eighty percentage of the dataset is used for model training. For validation, we use the other ten percent images of the dataset. And the rest twenty percent is configured as the testing dataset. Figure 5 shows the results of two examples of substation equipment identification. In Figure 5(a), the blue, red, and yellow boxes respectively identify the bushing shell, oil tank, and oil conservator of the transformer. However, some small bushing shells in the picture are not recognized. Some equipment can not be classified because of the small proportion of the device in the picture. In Figure  5(b), the yellow box and the blue box identify the sleeve and bushing shell respectively. It can be seen from the figure that the algorithm can identify each device more accurately and has higher recognition accuracy.
(a) case 1 (b) case 2 To assess our proposed network, we adopt 4 different criterions to give a comprehensive judgement. Each kind of power equipment is denoted as one category, and the faulty part component is tagged as another category separately. Pixel accuracy (PA), mean pixel accuracy (MPA), mean intersection over union (mIoU) and frequency weighted intersection over union (FWIoU) are written as below.
Where, is the number of pixels belonging to the class , however, predicted to be class . is the number of pixels belonging to the class and predicted to be class .
In order to demonstrate the superior ability of proposed method, three other method that threshold segmentation (TS), area extraction (AE) and edge detection (ED) are used for comparison which is shown in Table 1. Compared to those models, the proposed methods improve the segmentation quality on behalf of PA and MPA more than 5%.
The extracted hidden information is used as the input of the DBN network, and the 9 types of equipment failures are used as the output. The classification results of DBN, SVM and BPNN during a certain test are shown in Table 2. It can be seen from the table that BPNN has a high accuracy in the classification of transformer faults, and it performs best in both the recall rate and accuracy. BPNN has the worst classification effect. In order to better verify the effectiveness of the method proposed in this paper, we test the overall process of substation equipment fault analysis based on infrared images. First, image recognition is performed on the input RGB image, then the infrared image is used for image registration, and finally the trained convolutional network is used to determine the type of failure incorporating the temperature variation of the device. Figures 6 and 7 are the actual case results of the tested equipment. It can be seen from the figures that the method in this paper can accurately extract the target of the substation equipment existing in the picture. Combined with the infrared image, the image registration can be achieved more accurately. In the device extracted in Figure 6, the lowest temperature was detected to be 39.02°C and the highest temperature was 39.28°C. The difference between the two was less than 1%, and according to the analysis results of the CNN network, it can be judged that the device is currently in normal condition. In the equipment proposed in Figure 7, the lowest temperature was found to be 39.42°C and the highest temperature was 41.68°C. The difference between the two was more than 5%. At the same time, the results output by the CNN network also determined that the equipment was in an abnormal operating mode. At this time, an early warning signal needs to be sent out in time to notify the staff to conduct further equipment operation status investigation.

Conclusion
Aiming at the problem of substation equipment failure analysis, this paper comprehensively considers RGB image and infrared image information. First, the convolutional neural network is used to identify the device in the picture, then the infrared image is used for image registration, and finally the DBN network is used to determine whether the device is in a fault state. Through the analysis of specific examples, the method proposed in this paper has a better analysis performance, which proves the effectiveness of the proposed method