Deep Learning Based External-force-damage Detection for Power Transmission Line

With the accelerating pace of modernization, engineering construction has frequently threatened the safety of the power transmission lines. The external-force-damage has gradually become a major factor affecting the safety of transmission lines. To overcome the shortcomings of traditional detection techniques, this paper proposes a deep learning based external-force-damage detection method for transmission lines. First, thousands of surveillance images are collected and annotated with bounding boxes for construction objects. Second, a deep learning based object detection model is trained to localize the construction objects. The conducted experiments show that the proposed method can detect the construction objects in surveillance images accurately and is robust to illumination and climate variations.


Introduction
With the rapid development of national economy, the demand for electricity in all walks of life grows more vigorous and the layout of power transmission lines becomes denser. However, the increasing engineering construction of railways, highways, quarries, mines and other buildings has inevitably caused a large number of mechanical construction in the protection zone of transmission line. Deconstruction by external forces, such as tower crane hitting the transmission line, would cause large-scale power outage which not only threaten the safety of personal life but also increase the difficulty of maintenance for the power supply department. A lot of manpower and material resources may be spent for the power supply department to patrol the transmission line in order to eliminate potential dangers.
Currently, the common power transmission line inspection methods can be divided into three categories: manual inspection, UAV inspection and video surveillance. Manual inspection requires a lot of manpower and material resources, which is inefficient and cannot adapt to the grid with increasing scale. UAV inspection does not need much human resources, but the endurance of UAV may restrict the inspection scope and we cannot monitor the transmission line for a long time. Video surveillance is a more mature inspection method in recent years. The surveillance camera installed on the power transmission tower can capture the images of surrounding environment in real time and transmit them back to the monitoring center. The staff can judge whether there is a safety hazard on the transmission line by checking the images and thus greatly reduces the manpower and material resources.
Although video surveillance can avoid the inconvenience in manual and UAV inspection methods, it's still far from practical application as the amount of surveillance data is too large to process if only by human observation.. In recent years, the research community has brought up some video analysis algorithms towards automatic external-force-damage detection. Zhang [1] [3] et al. proposed a detection system of potential accidents around transmission lines. They detect crane and tower crane in the sky with background modelling method, identify fire by considering colour and texture information and localize engineering vehicles with convolution neural networks. Chen [4] et al. combined moving object segmentation methods and support vector machine (SVM) to recognize crane, they track the lifting arm with Kalman filter and detect its angle with Hough transform so as to report different levels of alarm signals. He [5] et al. proposed a warning system to handle transmission line ice cover, disconnection, engineering vehicle intrusion and other abnormalities in surveillance by combining object extraction algorithm based on adaptive threshold and identification methods based on subtraction and texture characteristics.
Although all the methods mentioned above realize the basic early warning functionality against external-force-damage, most of them still use the traditional image processing and machine learning algorithms which need carefully tuning of the model parameters in order to cope with complex illumination and climate conditions, as well as varied transmission line environments. On the other hand, deep learning algorithms, especially the convolutional neural network (CNN), gradually dominates most of the computer vision and image processing tasks due to the increasing amount of data and computing resources. A typical characteristic of CNNs is that they can learn the feature of objects or images from large amount of data and are more robust to varying scenes and illumination. Currently, the transmission line surveillance systems can produce massive image data every day, making full use of these data to train a CNN-based method for external-force-damage detection becomes applicable.
In this paper, we propose a deep learning based external-force-damage detection method for transmission line. First, we collect about 5700 surveillance images from 223 scenes of transmission line and generate our dataset by eliminating the similar images and annotating the construction objects. Then, we train a Faster RCNN [6] model to detect the construction objects. The results on test dataset shows that our model can detect the targets on transmission line surveillance images effectively and is robust to illumination and climate changes.

Constructing dataset
Since the number of surveillance data is too large and images from the same scene usually look similar. We first eliminate the similar images from the same scene to avoid overfitting during model training phase. We collect around 5700 surveillance images of transmission line and divide them into 223 scenes. For each surveillance scene, images are captured from different seasons in a year as well as different time in a day in order to make the detection model learn robust feature against illumination and climate changes. After eliminating the similar ones, we finally generate 2312 surveillance images of transmission line.
We categorize the construction objects that may cause external-force-damage into three classes: crane, tower crane and excavator (we merge bulldozer and excavator into one class due to limited number of samples). An example the annotated surveillance images is illustrated in figure 1.

Deep learning based detection model
Traditional object detection algorithms rely on hand-crafted features which are vulnerable to the deformation of objects and illumination changes. In recent years, with the growing number of available training data, CNN-based object detection methods have surpassed the traditional methods by a large margin. In our paper, since we have already annotated enough training data, a CNN-based object detector is trained to localize the construction objects in transmission line surveillance images.
We choose Faster RCNN with VGG16 [7] for feature extraction. Each module is introduced in the following in detail.

Construction object detection with Faster RCNN
Faster RCNN is a general object detection method proposed by Ren [6] et.al in 2015. It first extracts the image feature with a backbone network, such as the VGG network introduced in section 3.1, then the region proposal network (RPN) predicts region proposals that may contain construction objects. Finally, the region features are generated by RoI pooling and fed to classification and bounding box regression. The overall pipeline is shown in figure 3.

Region proposal network.
Region proposal network (RPN) is designed for generating proposals that may contain target objects. Compared to the traditional proposal generation methods, such as EdgeBox [9], RPN could reach higher recall with less proposals, which makes it much more efficient. For an input image, RPN first generates evenly distributed anchor points on the image, each corresponding to several anchor boxes that provide the scale prior of target objects. For each anchor box, RPN predicts its objectiveness, i.e. the probability of containing target objects and the offset of object proposal with regard to this anchor box.
In order to make full use of the training data, we apply K-means clustering algorithm on the training ground-truth objects to generate the anchor boxes. The detailed flow diagram of K-means clustering is illustrated in figure 4. ij-th bin and ij n is the number of points in it.
After RoI pooling, the input region features of varying size are fixed to kk  , which is then fed into the subsequent region classification and regression module.

Region classification and regression module.
Region classification and regression module are responsible to predict the classification probability distribution as well as refine the bounding box for each region proposal. Suppose

Experiment setting
We divide the annotated 2312 surveillance images into two splits, with 1734 images for training and the rest for testing.
Our model is implemented based on Faster R-CNN in Tensorflow [10]. The input images are resized to 1000x750 before fed into the network. SGD with mini-batch is used for training. The batch size is set to 1, the momentum is 0.9 and the weight decay is 0.0005. All the convolutional layers are initialized with weights trained on the ImageNet dataset, while the rest of layers are initialized with Gaussian distribution.
We conduct our experiments on GeForce GTX 1080Ti. We train our model for 70k iterations, with learning rate 10 -3 for the first 50k iterations and 10 -4 for the rest.

Experiment results
The overall loss in training phase is shown in figure 5.  figure 5 we can see that the training loss decreases gradually and stabilizes at last, which indicates that our model has been effectively trained.
After training, we evaluate our model on the test dataset in terms of mean Average Precision (mAP), the same as the standard PASCAL VOC protocol [11], where a detection is considered to be true positive if its IoU (intersection over union) with a ground-truth box is higher than 0.5. The evaluation results are shown in Table 1. As we can see, our deep learning based construction object detection method can reach 90% mAP on the test dataset, which demonstrates the effectiveness of our method. The qualitative results are illustrated in figure 6.  6. Qualitative detection results in the test dataset From figure 6 we can see that our model could localize the construction objects in transmission line surveillance images accurately. The first row demonstrates the detection results in normal scenes, the second illustrates results on snowy scenes and the last row shows the detection results on foggy day and during dawn time. We can see can our model are robust to the changes of illumination and climate.

Conclusion
In this paper we have proposed a deep learning based external-force-damage detection method for transmission line, which can be applied to the video surveillance scenario and save the manpower as well as material resources. We first collect thousands of transmission line surveillance images and annotate the construction objects, then we train a Faster R-CNN based method to localize the construction objects in images. Experiment results on test dataset demonstrates that our model could localize the construction objects accurately and is robust to the changes of illumination and climate.