Automatic bridge crack detection device based on quadrotor UAV

For the bridge located in deep valley, with high tower and long span structure, the traditional artificial crack detection can not meet the needs of today. In order to solve this problem, a new bridge crack detection equipment is designed based on quadrocopter unmanned aerial vehicle. According to this equipment, the developed system contains: reconnaissance unmanned aerial vehicle system, acquisition and transmission system, crack positioning system, and graphics operations system. Reconnaissance UAV system is improved based on quadrocoper UAV to meet the requirements of flight power, flight control, flight safety and signal stability. In the integrated transmission system, the designed UAV equipped with Hawkeye 5S motion camera, and 4G communication is used to complete the real-time image transmission; The crack location system uses BDS / GPS dual-mode combined single-point positioning module to realize the accurate location of bridge surface cracks; Graphics operations system, based on C++ and python language, designed a superpotent actual-time video streaming graphics operations method, and used median filter, image grayscale, histogram equalization, threshold segmentation and other methods to process the crack image. The designed bridge crack detection equipment can effectively solve the problem of low crack detection efficiency, and realize the intelligence and automation of crack identification. It has practical engineering application value and promotes the development of bridge crack detection technology.


Introduction
In recent years, with the rapid development of China's bridge industry, remarkable achievements have been made, and China has become a big bridge country in the world. However, with the increasing number of bridges in our country, the maintenance task of bridges is also increasing. Crack is one of the important hidden dangers threatening the safety of bridges. It is an important index to evaluate the safety of concrete components. It not only affects the beauty of bridge structure, but also leads to the corrosion of steel bars in concrete, which accelerates the aging of concrete, weakens the bearing capacity of bridges, and reduces the stability of bridges. Because many bridges are located in deep valley and have high tower and long-span structure, the traditional manual measurement has low safety factor, slow detection speed and small detection range, so it is very difficult for large-scale measurement. With the emergence of digital image processing and intelligent crack recognition technology, the disadvantages of the traditional detection methods are eliminated. Using UAV for bridge crack detection can well avoid the above problems, so as to realize the intellectualization and automation of bridge appearance detection [1] .
At present, the recognition and detection of bridge crack image mainly adopts camera system and its equipment [2] . Chen Yao [3] proposed using wall climbing robot instead of manual to detect bridge crack image, and using Wiener filtering method to remove blur, which achieved good results, but the detection efficiency of wall climbing robot is not high; He Zhiyong [4] and others have planned the detection flight route of UAV, focused on the image shooting of the cracks at the bottom of the bridge, and verified the reliability of crack detection using UAV; Wei Sihang [5] and others proved the superiority of UAV image compared with traditional manual detection by taking pictures of the structure surface; Xu [6] and other methods based on machine learning realized the classification of cracks; Tan Xiaojing [7] and others used Matlab language to compile crack recognition program, and elaborated the principle of crack information recognition by digital image processing technology. However, there are some problems in the current schemes: ①The crack detection efficiency is too low and the detection speed is too slow; ②There is no automatic bridge crack detection device that fully meets the requirements; ③The production cost is too high, and the intelligence and automation of crack detection are not fully realized; In view of the defects of the above scheme, this paper designs a bridge crack detection equipment with high automation, high speed and accurate positioning based on quadrotor UAV. This equipment mainly includes two aspects: software algorithm and hardware equipment. The software algorithm includes: UAV accurate positioning of bridge cracks, crack image processing system; Hardware equipment includes: four rotor survey UAV system, image acquisition system, remote wireless transmission system. In the survey UAV system, based on the dynamic model of quadrotor UAV, build a survey UAV model to meet the needs; In image acquisition and transmission, the eagle eye 5S motion camera is used to collect the crack image, and the real-time image transmission is completed through 4G communication; In image processing, based on C++ and python language, an efficient real-time video stream image processing method is designed, and histogram equalization, edge enhancement and other methods are used to complete the real-time image processing. Figure 1 shows the specific technical route.

2.1.UAV selection
Four rotor survey UAV is widely used in various fields. Its four rotors are equal in size, located in the same horizontal plane, and symmetrically distributed in left and right, front and back, and its motors are symmetrically distributed correspondingly. Four rotor UAV has good flight control performance and good maneuverability. Compared with multi rotor UAV, it has lower energy consumption, stronger endurance and better adaptability. Therefore, this paper builds the hardware platform of

The dynamic model of quadrotor reconnaissance UAV
The dynamic model of quadrotor reconnaissance UAV can be used to describe the change of position parameters of reconnaissance UAV system with time parameters. Ignoring the small deformation of its internal structure, the quadrotor UAV can be regarded as a rigid body. Under this assumption, six parameters including translation and rotation can be used to accurately describe the space position and attitude of survey UAV.
The spatial position and attitude relative to the geodetic coordinate system can be expressed in the following spatial coordinate form of quadrotor reconnaissance UAV • ξ is the space position of the aircraft relative to the geodetic coordinate system; • η is the attitude of the aircraft relative to the geodetic coordinate system; • ψ ,θ ,φ they are Euler yaw angle, Euler pitch angle and Euler roll angle. The dynamic model of quadrotor UAV is established by Eulerian Lagrange method, which describes the force and moment of quadrotor UAV in flight: sin , cos sin , cos cos , • f is the vertical resultant force acting on the survey UAV; • , and represent the torques around φ，θ and ψ respectively.

Flight module design
When quadrotor reconnaissance UAV system is applied to bridge detection, it has high requirements for flight module, flight control, image acquisition, flight power, safety and signal stability.

Flight dynamics
The wheelbase is 450mm and equipped with carbon fiber frame with 14 inch carbon fiber propeller； The battery adopts Grignard lithium battery with continuous discharge rate of 10C and capacity of 25000 MAH；M3508 brushless DC motor with torque up to 3N·m is selected；C620 is selected as the electronic governor. The weight of the designed UAV can reach 6.2kg, which can fully meet the load requirements of the bridge detection instrument, and its flight time can reach 50 min, with the ability of long endurance.

Flight safety
STM32F103 MCU is used as the safety coprocessor to deal with emergencies, so as to ensure flight safety. At the same time, this paper configures a safe power supply for the UAV to prevent the problem of battery exhaustion. In order to ensure the UAV equipment can work normally when some components fail, reduce the failure probability and improve the system reliability, this paper adds micro accelerometer and micro gyroscope to achieve the purpose of redundancy design. camera is not perpendicular to the image captured by the bridge, the steering gear on the three axes of the pan tilt is used to adjust the deflection angle on the x, y and z axes of the camera, so as to adjust the camera attitude to adapt to the flight attitude.

PTZ parameters
In order to achieve the above PTZ parameters, the overall design of PTZ system is carried out in this paper. The PTZ system consists of four parts: controller, attitude feedback element, actuator and frame structure. The controller module is mainly composed of STM32F203 in 64 pin LQPF package; Inertial sensor MPU6050 is used as attitude feedback component, which integrates three-axis gyroscope and accelerometer; The actuator is composed of servo actuator, M2006 brushless motor and C610 motor driver. According to the special requirements of bridge crack detection, servo motor pan tilt and brushless motor pan tilt are also designed in this paper, which have lower overall center of gravity and better self stability. The joint between the base and the fuselage is a deceleration ball, which can reduce the influence of the fuselage shaking on the pan tilt. Because the quadrotor reconnaissance UAV will affect the PTZ in the flight process, this paper adds a motion compensation controller to compensate.

Design of control module
The main control board of the UAV is STM32F427, the architecture is ARM cortex M4, the model of magnetometer is HMC5883L, and the model of accelerometer / gyroscope is MPU6050. The digital pressure sensor model isBMP085, ATK-NEO-6M-V23, and the GPS module and other sensors are used to obtain the deflection angle and three-axis acceleration of the aircraft in real time, so as to realize the stable hovering and corresponding attitude adjustment of the survey UAV.

Motion processor
In this paper, MPU-6050 is used as motion processing sensor. MPU6050 is a very popular space motion sensor chip, which can obtain the current three acceleration components and three rotation angular velocities of the device. It has the characteristics of small size, powerful function and high precision. Figure 2 shows the circuit connection diagram of MPU6050.

Magnetometer
During the flight process of quadrotor reconnaissance UAV, heading maintenance and control are needed. Magnetometer is used to measure the size and direction of geomagnetic field, and also to measure the component of magnetic field intensity vector of quadrotor reconnaissance UAV in the system, so as to be used for attitude control and heading maintenance. HMC5883l magnetometer with digital interface is used in this paper. It is a surface mount weak magnetic sensor chip with high integration and good effect. Because HMC5883L has a very high resolution magnetoresistive sensor, it can fully meet the needs of bridge crack detection. Figure 3 shows the circuit connection diagram of HMC5883L.

Digital pressure sensor
In this paper, BMP085 digital pressure sensor is used. Its power consumption is as low as 5uA in standard mode, and its measurement accuracy is high in high linear mode. Its resolution is 0.03hpa (0.25m). It needs only 7.5ms and no external clock circuit. It can be used in mobile devices. It is very suitable for use in reconnaissance UAV. In this chip, temperature compensation module, data storage module and control module are integrated. Figure 4 is the schematic circuit diagram of BMP085.

Image acquisition system for UAV
The designed UAV is equipped with Hawkeye 5S motion high-resolution camera to collect the crack image and realize the full non dead angle shooting. The designed quadrotor reconnaissance UAV has the advantages of long endurance, stable hovering, stable signal receiving, adjustable attitude and precise positioning, which can meet the requirements of flight power, flight control, flight safety and signal stability in image acquisition. Eagle eye 5S motion camera has 2400W high-definition zoom camera with good imaging quality and supports JPEG image output. The field operation shows that the UAV signal is stable, the picture is not obviously shaking and the resolution is high when the UAV image acquisition system is used to detect the bridge cracks. Figure 5 shows the field image of the engineering structure taken by the UAV system..

Remote wireless transmission system
This paper uses 4G network wireless communication to complete the task of real-time image transmission. 4G network wireless communication transmission has the characteristics of high stability, high reliability and low cost. It can realize point-to-point communication and point to multipoint communication. Through 4G wireless transmission module, the RF and baseband of the images collected by the survey UAV are integrated on a PCB board to complete the functions of wireless receiving, transmitting and baseband signal processing. Through high-speed networking, the data stream is transmitted to the computer in real time, which lays the foundation for subsequent image processing, improves the detection efficiency and speed, and realizes the intellectualization of crack detection.

Image graying
The acquired bridge deck crack image contains other unnecessary background, and there are some problems such as insufficient contrast and fuzzy details, which will have a certain impact on the later recognition. In order to facilitate the following necessary steps such as threshold segmentation and get the desired image which can better reflect the image features, it is necessary to carry out the preliminary grayscale transformation on the image. By using the API encapsulated in opencv library, RGB format is transformed into GRAY format to realize image graying.

Median filtering
The collected bridge deck crack image generally has complex noise, which will cause bright and dark interference to the image, greatly reduce the image quality [8] , and affect the crack recognition. Therefore, denoising should be carried out before further image processing. Median filter is a kind of nonlinear spatial filter, which replaces the gray value of the pixel with the median value of the gray value in the neighborhood of the pixel. The response is based on the gray order of the pixels in the image area surrounded by the image filter, which determines the gray value of the central pixel. Median filter is very effective to deal with impulse noise and salt and pepper noise. The key is to choose the appropriate window shape and size. If the filtering window is too large, the edge will be blurred, and if it is too small, the denoising effect will be poor. In the first mock exam, a multi structure median filter algorithm is adopted to solve the defect of single template in general median filtering. The structural elements in morphology are introduced into median filter, and 4 shapes are adopted. The original gray-scale image is filtered successively by 3 structural elements. Due to the use of a variety of structural elements, a variety of noise can be effectively filtered.

Histogram equalization
Due to the influence of environmental conditions such as light and background color difference, the crack image collected by reconnaissance UAV is difficult to distinguish from the background, which brings difficulties to image analysis. Histogram equalization is a commonly used correction method. Through some transformation, the original image whose gray histogram is too concentrated in a certain gray range is transformed into a uniform distribution in a larger gray range, and the gray difference between each part is increased, so as to increase the overall contrast of the image and achieve the purpose of enhancing the details of the image, so as to further process the image. In this paper, an adaptive image histogram equalization processing is adopted. According to the local statistical characteristics of the image pixels, the gray value of the pixel is transformed by the function. The transformation function is determined by the histogram of a certain size of sub image around the pixel, and the image enhancement effect is obtained.

Edge enhancement
Gray processing can only simply enhance part of the image domain, but the background of this project is the bridge crack, which is in the outdoor, or even bad environment. In the process of image acquisition, it is often affected by various noises, which easily leads to many important information at the edge of the contour can not be read and recognized, so it is necessary to enhance the image edge. Spatial domain enhancement is based on the linear convolution operation, which completes the calculation process by sliding the template in the image and constantly calculating the new value of each pixel according to the value of the domain pixel. In this paper, Laplacian operator with better edge enhancement is introduced to filter out the lower noise and enhance the important information of the image.

Threshold segmentation
Threshold detection algorithm is the most basic and commonly used technology in bridge crack image detection because of its small amount of calculation, fast operation speed, stable performance and simple implementation. Because the simple and direct threshold segmentation method is generally only suitable for the image with uniform background gray, uniform illumination and high contrast, the segmentation effect is not ideal for the image with uneven illumination or background with mottled points, which can not meet the needs of practical engineering. Therefore, this paper combines the gradient information Prewitt operator with Otsu method to achieve better fracture image segmentation effect. Figure 6 shows the effect of image preprocessing.

5.Experimental results and analysis
The model of quadrotor reconnaissance UAV designed in this paper is shown in Figure 7.  Figure 7. Survey UAV model The photos taken by the equipment show that the system can be used for hovering detection and fixed-point inspection, and the photos taken are stable and high resolution. At the same time of image acquisition, the device can also locate the cracks independently and accurately, and the difference between the measured position and the actual value is very small. In addition, the image processing effect of this system is good, which can lay a foundation for the subsequent crack recognition and the extraction of length and width.

6.Conclusion
a. quadrotor UAV is designed to meet the requirements of bridge crack detection, which can meet the requirements of flight power, flight control, flight safety and signal stability. b. Aiming at the limitation of traditional manual crack detection, a bridge crack image acquisition device carried by UAV is designed, which realizes the automation and intelligence of crack detection and greatly speeds up the speed of crack detection.