Style transfer and recognition of infrared images of transformers under different lighting based on CycleGAN

In order to solve the problem that the transformer in the infrared image of the transformer is difficult to be accurately detected due to strong light irradiation under different lighting conditions, a method for unifying the illumination and equipment identification of the infrared image of the transformer based on CycleGAN and yolov3 is proposed. CycleGAN is used to unify the illumination intensity of the infrared image of the transformer, and an adversarial loss function and a cycle consistency loss function are introduced to ensure high-quality generation of infrared images, and the expanded infrared sample library is richer. Finally, the yolov3 network is trained using the uniformly illuminated image set to achieve accurate identification of transformers. The experimental results show that the infrared image of the transformer that has been unified by illumination can identify the transformer with an accuracy of 95.2%, which is 3.1% higher than before enhancement.


Introduction
The purpose of image infrared diagnosis is to select the working status and existing faults of the transformer through temperature distribution [1].How to identify infrared images efficiently and accurately is the basis for ensuring the safe and reliable operation of transformers [2].
Infrared identification technology has the advantages of non-contact, fast response, wide measurement range, and intuitive measurement results.Its application in power system operation and maintenance is becoming increasingly important [3].Through real-time monitoring and analysis of heat map data and analysis of high-temperature areas in the heat map, fault points can be accurately located and the personal safety of operators can be greatly protected [4].
Due to actual operation and maintenance, strong sunlight will produce a large amount of infrared light.The shadowing effect causes the background light source to be stronger than other heat sources.The continuous illumination causes the ambient temperature to rise, making it difficult to accurately detect target objects in infrared images.In this regard, in [5], a radiation transmission model is established based on the infrared radiation imaging process, and a new inversion algorithm is proposed to extract radiation information from infrared images under different illumination, which helps to improve infrared recognition; In [6], ANSYS software is used to conduct finite element analysis of the temperature field of the transformer under different ambient temperatures.After isolating the transformer area, the influence of the ambient temperature on the temperature measurement during infrared image collection is considered; In [7], the Unet network is used as the image reconstruction network model to reconstruct the real scene in the light interference area, which can eliminate the light interference in the infrared image of the photovoltaic array.In general, existing research has paid less attention to the impact of the infrared light generated by the illumination itself on the infrared image of the transformer.
In response to the situation above, a method is proposed in this paper for transformer infrared image style transfer based on cycle-consistent generative adversarial networks [8].This method uses infrared images of transformers under different illumination as samples, and infrared images of transformers are generated under the same illumination through a style transfer network.It is used for transformer fault recognition training by yolov3, which expands the sample data and enhances the accuracy of the recognition model.The results show that under different lighting conditions, this method can effectively expand sample data, optimize the detection model, and improve the accuracy of transformer fault identification.

Cycle-consistent generative adversarial networks
Cycle-consistent generative adversarial networks (CycleGAN) achieve inter-domain image conversion by optimizing the loss function of the generator and discriminator, allowing the generator and discriminator to cooperate and maintain content similarity.The specific loss function is as follows: Equation ( 1) is the loss function of the generator, where EB~P data(B) and EA~P data(A) represent samples sampled from B and A, and DG is used to determine the probability of whether the image generated by the generator comes from Set B to ensure the normal transfer of image style.Equation ( 2) is the cycle consistency loss function to prevent excessive migration of samples.Equation ( 3) is the overall loss function.

Transformer infrared image style transfer method
CycleGAN is used to perform style transfer on infrared images of transformers under different lighting conditions, and unify them to the same lighting conditions for model training and fault identification, thereby improving detection accuracy.As shown in the Figure 1, The specific steps are as follows: Step 1: Data preprocessing.The existing transformer infrared atlas is labeled and divided into two atlases A and B under different illumination and the same illumination.
Step 2: The deep learning framework is used to build the CycleGAN model and A and B data sets are used for training.

Transformer infrared image recognition method under different lighting conditions
According to the existing infrared images of transformers, it is difficult to directly obtain the corresponding lighting conditions.For this reason, each image has a specific shooting date, period, and location, and is corresponding to the weather, shooting period, and lighting conditions.As shown in the Figure 2, the specific identification process is as follows: Step 1: Image preprocessing.CycleGAN is called to perform style migration on the image set and unify all infrared images under the lighting of the same period, and the resize function is used to adjust them to the appropriate size.
Step 2: Model training.The uniformly illuminated infrared image sample set is used to train the yolov3 network.
Step 3: Transformer fault identification.By inputting the uniformly illuminated image to be identified into the trained Yolo network, the fault of the transformer can be accurately identified.

Calculation example description
In this experiment, the Windows operating system is adopted as the experimental platform.The CPU is Gen Intel(R) Core(TM), the GPU is NVIDIA GeForce RTX 4080, and the memory size is 32 GB.
The learning framework is Pytorch 1.8.0, the programming language is Python 3.8, the learning rate of the generator and discriminator is 2×10 -5 , and the number of training rounds is 200.The specific environment is shown in Table 1.

Migration effect assessment
In order to evaluate the quality and diversity of the generated infrared images, two indicators, PSNR (Peak Signal-to-Noise Ratio) and structural similarity index measure (SSIM), are introduced.Among them, PSNR is calculated by calculating the mean square error value (MSE) of each channel of the three channels of the RGB image and then averaging it to calculate PSNR.The higher the value is, the closer the infrared image generated will be to the original data and the greater the authenticity.The specific calculation formula is as follows: = 10 log (2 − 1) Formula (4) and Formula (5) correspond to MSE and PSNR, respectively.Among them, f(x, y) is the original image pixel, and g(x, y) represents the image to be detected.
The SSIM index comprehensively evaluates the quality of the generated image from three aspects: brightness, contrast, and overall structure.The closer the value is to 1, the higher the quality of the generated image will be.The specific calculation formula is as follows: In formula (6), l(x, y) uses the mean to estimate the brightness, c(x, y) uses the variance to estimate the degree, and s(x, y) uses the covariance to estimate the structural similarity.
In this article, VAE and WGAN•GP are used to compare two-generation networks with the selected network.The average value of the calculation results of the infrared image evaluation index generated by each method is shown in Table 2.The infrared image reconstructed by each method is shown in Figure 3.After the CycleGAN network used in this article performs style transfer on infrared images, the SSIM value of the new image generated reaches 0.986, which is closer to 1. PSNR reaches 29.21 dB, which is significantly higher than the other two-generation networks, and its value is close to 30, indicating better image quality.The specific migration effect is shown in Figure 3. GAN by introducing the Wasserstein distance and gradient penalty methods, but there is still a loss of detailed features.As shown in Figure 4, the cycle consistency generative adversarial network adopted this time introduces the adversarial loss function and the cycle consistency loss function, and the generated infrared image quality is higher and the effect is better.

Effect of lighting on model recognition
In order to verify the impact of lighting on infrared images of transformers and the effective improvement of model recognition by infrared style transfer on CycleGAN proposed in this article, two indicators, Accuracy and Recall, were used to evaluate the model recognition effect.Accuracy refers to the proportion of the total number of correct predictions by the model.In binary classification, the following equation ( 7) can be calculated based on positive and negative classes: The Recall rate is to solve the problem of the proportion of all positive category samples that are correctly identified as positive categories.The specific calculation formula is shown in equation ( 8): In the two formulae above, TP represents a true example, TN represents a true negative example, FP represents a false positive example, and FN represents a false negative example.As can be seen from Table 3, after migrating to the same lighting, the Accuracy of the Yolo network used for infrared identification of transformers increased by 3.1%, and the Recall rate increased by 2.4%.It can be seen that light intensity does have an impact on infrared recognition.

The impact of different generation networks on model recognition.
Three generative networks, VAE, WGAN•GP, and CycleGAN, are used to unify the illumination of the samples, and then for the training of the yolov3 network respectively.The trained network is used to detect the same infrared image of the transformer.The identification effect is shown in the Table 4.

Conclusion
In order to solve the problem that strong light irradiation produces a large amount of infrared light under different lighting conditions, making it difficult accurately detect the transformer in the infrared image of the transformer, a method is proposed in this paper for illumination unification and equipment identification of the transformer infrared image based on CycleGAN and yolov3.The main conclusions are as follows: 1) The Cycle-consistent generative adversarial networks can unify the infrared images of transformers under different illumination into the same illumination.The generated image SSIM and PSNR indicators are 0.968 and 26.21 dB, respectively, which are higher than those of other generation models.It shows that the cycle-consistent generative adversarial model can reconstruct infrared images that are closer to real illumination.
2) CycleGAN unifies the lighting and uses it for yolov3 model training, which can effectively improve the recognition accuracy and recall rate.Compared with other generation models, the images generated by CycleGAN are better at optimizing the recognition accuracy.

Figure 1 .
Figure 1.Style transfer of infrared images under different lighting conditions.

Figure 3 .
Figure 3.Comparison of effects before and after migration.

3. 3 . 1 .
Model recognition effect under different lighting training.The infrared images are transferred to the samples before and after the same illumination and used for network training of yolov3.The specific recognition effect of the model after training is shown in Figure 4 and Table3below.

Table 2 .
Generate infrared image indicator comparison.

Table 3 .
Recognition effect before and after migration.

Table 4 .
The impact of different generation networks on recognition.

Table 4 ,
after using the unified illumination images generated by CycleGAN for training, the Accuracy of transformer fault diagnosis reaches 95.2%, and the Recall rate reaches 96.3%, both of which are better than VAE and WGAN•GP networks.