Precise Unmanned Aerial Vehicle Visual Positioning Based on Neural Network

Unmanned aerial vehicle (UAV) positioning provides a means for human beings to achieve ubiquitous positioning and target tracking in low-altitude areas. The current UAVs mainly rely on satellite navigation and positioning, and airborne inertial devices (gyro) to complete positioning, which makes its signal easy to be blocked and leads to errors accumulation. However, visual positioning is characterized with high local positioning accuracy and good continuity. Therefore, visual positioning is integrated with UAV positioning in this study. First of all, UAV spatial reference acquisition and neural network are studied to analyze the shortcomings of the current UAV positioning technology in application and the necessity of integrating neural network. Second, a precise visual positioning algorithm based on neural network is presented. Based on the neural network model with image matching and position determination, the UAV visual positioning algorithm can be optimized and the redundancy of the target image can be reduced. What’s more, by establishing the experimental environment with Matlab, it is proved that the positioning accuracy can be improved with the algorithm proposed in this study.


1.
Introduction Unmanned aerial vehicle (UAV) positioning [1] is an integrated technology which can determine the virtual relative position and absolute physical position between the UAV and the target in two ways by a comprehensive space-time sensing device, providing a means for human beings to achieve ubiquitous positioning and target tracking in low-altitude areas.
With new carrier load, information technology has been applied to new space, new platforms and new nodes. New technologies, such as space navigation, machine learning, micro-nano materials, highperformance batteries, broadband communications, high-definition image processing, have provided foundations for UAVs to overcome difficulties in complex environments.
The high-accuracy UAV positioning has become a hot-spot in industry and academia across the world. In 2018, the United States Department of Defense released Unmanned Systems Integrated Roadmap (2017-2042), which proposed developing integrated artificial intelligence, advancing collision avoidance system design, and optimizing UAV target determination and tracking. In 2015, the United States Congress invested nearly $27 million to support the National Oceanic and Atmospheric Administration to develop the UAV project which can acquire more accurate location information and provide real-time mapping of coastal hydrological features. In 2019, EU Sky and Space Intergroup released EU Aeronautics Flying Beyond 2020, proposing the U-space concept of the Unmanned Traffic Management System (UTM) in order to construct an air fence depending on the location information. Japan Ministry of Land, Infrastructure, Transport and Tourism has also proposed developing a precise UAV positioning system which can be managed in collaboration by 2020. In this system, the flight location of each UAV can be displayed in real time and the routes are planned in advance to avoid collision. It can be observed that it has become the most urgent need to improve the UAV positioning accuracy in application.
In order to improve the UAV positioning accuracy, real-time differential dynamic positioning equipment, real-time route design platform, ultra-wideband (UWB) real-time positioning system and intelligent positioning system can be applied to improve UAV positioning accuracy so that UAVs can maintain the security and stability, serve special operations and emergency rescue. It is expected that UAV detection market will reach $14 billion by 2022.
The current UAV positioning mainly relies on satellite navigation and positioning [2] and airborne inertial devices (gyro), whose main principles are to measure the arrival time difference between electromagnetic wave transmissions and to match geomagnetic field features respectively. However, there are many problems as follows.
1) In the heavily blocked area, the satellite navigation and positioning signal will be dramatically weakened, reflected and scattered, resulting that the positioning deviation is large, and the requirements of positioning in the buildings or in complex environment cannot be met.
2) The long-time flight of UAVs requires continuous tracking of the flight state. By accurately correcting the initial state in advance, the airborne inertial devices can track the changes of continuous motion. When the state changes, the accurate initial state can be applied in matching. However, its continuous accurate positioning accuracy is low.
Visual positioning can match the location according to the differences between the features of the prior image and the real-time captured image. Different from satellite navigation and inertial positioning, it features high local positioning accuracy and good continuity. Therefore, visual positioning is integrated with UAV positioning in this study. First of all, UAV spatial reference acquisition and neural network are studied to analyze the shortcomings of the current UAV positioning technology in application and the necessity of integrating neural network. Second, a precise visual positioning algorithm based on neural network [3] is presented. By designing the neural network model with image matching and position determination, the UAV visual positioning algorithm can be optimized and the redundancy of the target image can be reduced. Finally, by building experimental environments with Matlab, it is proved that the positioning accuracy can be improved with the algorithm proposed in this study.

Technologies Related to UAV Positioning
With accurate spatial location acquisition, UAVs can effectively perform path planning, obstacle avoidance, task completion, and so on. According to the load of UAVs, the commonly used positioning methods include satellite navigation and positioning, inertial navigation, and visual positioning. Satellite navigation and positioning can be applied within the effective coverage of satellite signals, which can estimate the relative distance in accordance with the propagation time of the signal from the satellite to the receiver. As Figure 1 shows, assuming that the coordinates of the receiver are( , , ), and the time difference between the satellite 1 and the receiver is ∆ 1 , an equation can be obtained: is the light speed. With three satellites which can be accepted by the receiver, the equations of the receiver can be expressed as: By solving the equations (2), the coordinates of the receiver ( , , ) can be obtained. In addition, the error can be calculated as follows: Where ( ′ , ′ , ′ ) are the real coordinates of the receiver. According to engineering experience, the accuracy of satellite navigation and positioning generally ranges from 5 to 10 meters.
It can be seen that when UAVs are in flight state, satellites can be used to provide point-to-point navigation in remote path planning. However, when UAVs are in the area where the satellite signals are blocked, the precise location cannot be obtained because the completeness of equations (2) will be destroyed. In order to overcome the problem that UAVs cannot complete positioning when the satellite signal is blocked, visual positioning can be applied to determine the relative spatial location of UAVs. The principle of visual positioning is to pre-store the image features of the target captured from various angles and distances. When positioning, the real-time captured images will be compared with the stored images to obtain the angles and distances corresponding to the matching results. Assuming that the coordinates of the UAV are ( , , ), the image captured at this time is , the image feature extracted at this time is ∈ , and the image feature database stored by the UAV is Γ = { 1 , 2 , ⋯ , }, the image feature can be searched one by one in the process of positioning, which can be presented as: min{ − } , ≤ (4) When the condition (4) is satisfied, the best matching condition will be formed, and the coordinates of the UAV are ( ′ , ′ , ′ ) = ( , , ) → (5) According to the above analysis, the accuracy of visual positioning depends on the mapping granularity of the image feature database and the accuracy, and the quality of real-time image acquisition. Under these two conditions, the visual positioning accuracy can reach centimeter level (the mapping granularity of the image feature and the distance~=centimeter). As the quality of real-time image acquisition goes down, the accuracy of visual positioning will also decrease.

The Shortcomings of the Current Positioning Technology in UAV Application
Since the UAV is a dynamic load, its positioning conditions will change in conformity with the changes of the motion state, which will cause the input of the original positioning system to change, resulting in a decrease of positioning accuracy. When applying satellite positioning, the satellite's line of sight (LOS) will be taken as the reference of the UAV positioning accuracy. According to equations (2), the distance accuracy between a single satellite and the receiver can be determined by the time difference∆ 1 . When the satellite signal is blocked from reaching the receiver, ∆ 1 tends to reach ∞; when the satellite signal reaches the receiver through reflection, the actual time difference ∆ will be greater than the rectilinear propagation time difference ∆ 1 , and thereby the error will increase. At the same time, the dynamics of UAVs will also cause image deviations in visual photography. Assuming that the error caused by image distortion may lead to the feature deviation, the expression can be obtained as → ′ ， → ′ . In this condition, in conformity with the equation (4), the error can be expressed as follows: = ( , , ) → − ( ′ , ′ , ′ ) → ′ .
From the above analysis, in order to improve the accuracy of the UAV positioning, many methods can be applied, such as combining satellite positioning and visual positioning, reducing the image distortion of visual positioning, and improving the granularity of the image feature database [5].

The Necessity of integrating Neural Network and UAV Positioning
The integration of satellite and visual positioning can improve the UAV positioning accuracy in the signal blocked area. To be specific, visual positioning is the main approach, and more detailed image feature and location mapping database can improve the positioning accuracy. The image feature and location mapping database is derived from model processing based on behavioral learning, so the distortion of the input image needs to be reduced, and the reference and rules need to be more detailed. Neural network is a human-like logic processing model, which can effectively improve the normalization and accuracy of behavioral learning. It has been widely used in information processing and big data. Therefore, this paper will introduce the neural network to reduce image distortion in UAV visual positioning and improve the granularity of the rule database.

Neural Network Model with Image Matching and Location Determination
According to the analysis in Section Two, the dynamic motion of UAVs will lead to the change of the input condition of positioning, and thereby make the completeness of the positioning equations be destroyed. Therefore, separately applying the satellite navigation or visual positioning cannot meet the positioning requirements when UAVs fly continuously indoor and outdoor, whereas the integration of satellite and visual positioning can effectively compensate for the blind zone of each method. In this paper, with visual positioning as the reference and the prior positioning result of satellite navigation as the calibration and pre-compensation, the neural network is used to learn the image distortion and the mapping granularity in image matching in order to improve the UAV positioning ability in the signal blocked area. The neural network model with image matching and position determination is shown in Figure 3.
As Figure 3 shows, the neural network model mainly contains input layer, hidden layer and output layer. The model is designed as follows.
The input layer of the neural network is mainly used to process a large number of original images acquired by the UAV cloud platform in real time, and normalize the gray scale and size, and extract image features. In this paper, some heterogeneous factors, such as time variation, spatial variation, and causal variation, are classified as the input features ( ). At the same time, the positioning information of satellite navigation is described as ( ), and the other information, mainly covering the specification parameters of UAVs, are presented in the constant matrix ( ) = { 1 , 2 , ⋯ , }. Apart from that, the output condition is described as ( ).

Figure 3. Neural network integrating vision and location
(2) The hidden layer, lying between input layer and output layer, consists of numerous neurons and links, and is responsible for data processing. In image processing, hierarchical features need to be extracted from the influence factor matrix. In this paper, only one hidden layer is selected to abstract the general rule of UAV image acquisition. Since the range of each extraction will be gradually expanded in the multi-layer feature extraction, the extraction result of the hidden layer is highly abstract. Under this circumstance, regression analysis is also conducted to process the results. By only taking visual positioning into account, each node in the hidden layer can be expressed as: Where ℎ ( )is the response function of each node, represents that the node has sub-results, and represents that the node has input features. The equation (6) can be adjusted to And the lowest image matching error can be obtained by the equation input layer and the hidden layer, the optimized output of UAV image recognition, integrated with satellite navigation, can be obtained in the output layer.

The Optimization of Visual Positioning Algorithm Based on Neural Network
When applying visual positioning, positioning accuracy depends on the mapping granularity of image feature database [6] and accuracy, and the quality of real-time image acquisition. Under this circumstance, the influence of neural network model on image distortion and positioning accuracy should be firstly considered. The specific algorithm is as follows: Step 1: When the UAV flies in the open area where the number of the visible satellites is greater than 4, it should continuously record the positioning coordinates( 0 , 0 , 0 ).
Step 2: When the UAV enters the signal blocked area, visual positioning begins to work. At this time, the cloud platform can collect regional images and extract the initial image feature 1 .
Step 3: The initial positioning coordinates 1 -> ( ′ , ′ , ′ ) can be obtained by comparing the features of the real-time image and the reference image.
Step 4: The error can be calculated by calibrating the initial positioning coordinates and the positioning coordinates of the satellite navigation at the last moment, (10) In addition, the error of multiple measurements, 1⁄ ∑ ∆ , is set as the boundary condition.
Step 5: Taking the above conditions as the training values of the neural network model, the response function of the nodes ℎ ( )|∆ can be obtained.
Step 6: When the UAV flies randomly, the trained neural network model can provide accurate error compensation value. Therefore, the positioning coordinates of UAV can be determined.

Establishment of Experimental Environment and UAV Positioning Model
In order to verify the optimization effect of the neural-network-based precise visual positioning algorithm on positioning accuracy in the blocked area, this paper adopts Matlab platform for simulation. In the experiment, the satellite module and the UAV module which is made up of satellite positioning and visual positioning are established. In addition, three experimental environments are set, including unblocked area, lightly blocked area and heavily blocked area. What's more, 35 points are tested. The positioning accuracy of satellite navigation without neural network correction and the accuracy of the neural-network-based positioning are measured. Furthermore, the positioning errors are analyzed. Figure 4 shows the coordinate points recorded by the UAV system constructed in this study, and Figure  5 presents the error of the neural-network-based positioning.  Figure 4. UAV location test points As Figure 4 shows, the UAV location points are in a relatively stable state, with a certain deviation from the actual ones. Some location points are close to the actual ones, while the difference between some points and the actual ones is large. The variance of the overall location deviation is small. Figure 5 shows the positioning error distribution of the algorithm in this paper. It can be seen that the positioning errors of all 35 test points are less than 0.28m, and the test points with the positioning errors less than 0.1m account for about 50% of the total. Thus, when UAVs are under flight state, the positioning accuracy can be still high even though the environments and conditions change.  Figure 6 shows the positioning accuracy of the satellite navigation without neural network correction and the accuracy of the neural-network-based positioning. It can be seen that the accuracy of the neuralnetwork-based positioning is better than that of the positioning algorithm without neural network correction, and its error is relatively stable with less fluctuation. As the environment changes, when the satellite signal is blocked (the 8th test), the positioning error of the positioning algorithm without neural network correction will increase to more than 0.35m, while the positioning error of the algorithm proposed in this paper is only 0.06m. Therefore, the algorithm proposed in this study can effectively ensure the continuous flight of UAVs.

5.
Conclusion By introducing visual positioning into UAV positioning, this paper proposes a precise visual positioning algorithm based on neural network. It is proved that the neural network model with image matching and location determination can optimize UAV visual positioning algorithm, reduce the redundancy of target images, effectively solve the problem that the signal is easy to be blocked in the flight of UAVs, and decrease error accumulations. In accordance with the simulation results, the positioning error of the new algorithm is less than 0.28m in all the 35 test points, and the number of the test points with the positioning error less than 0.1m accounts for about 50% of the total.