Overview of Obstacle Avoidance Algorithms for UAV Environment Awareness

In complex indoor and outdoor environment, obstacle avoidance of UAV (Unmanned Aerial Vehicles) is a challenging problem. In order to realize the obstacle detection, autonomous positioning and trajectory planning of UAV flight mission in large outdoor scene, there are three main technical problems: Firstly, UAV needs to have the ability to quickly detect a variety of obstacles in outdoor scene. Secondly, in order to realize the autonomous navigation of UAV, it needs to establish the coordinates of obstacles in three-dimensional space. Thirdly, based on the above two conditions, UAV can independently plan flight path to avoid obstacles. This paper mainly introduces the use of RGB-D camera, lidar, monocular camera and binocular camera in UAV obstacle avoidance, and compares them from the sensor types, advantages disadvantages and application range. Secondly, the path planning strategy of UAV is discussed, and the existing problems and current research results of UAV trajectory planning strategy are described. Finally, it is pointed out that real-time computing, multi-sensor fusion and integrated obstacle avoidance between multi aircraft should be the research direction of autonomous obstacle avoidance navigation for UAV in the future.


Introduction
Due to its great application value, UAV is widely used in military investigation, express transportation, agriculture, photography, surveying and mapping and other industries. According to different classification of flight platform configuration, UAV can be devided into five types: fixed wing, rotor, airship, parafoil and flapping wing. According to the classification of application, UAV can be devided into military and civil. Military UAV is mainly used for reconnaissance, electronic countermeasures, target aircraft and so on. Civil UAV can be mainly used for monitoring, agriculture, meteorology, surveying and mapping, etc. If classified by scale, UAV can be devided into micro, light, small and large. It is because of the huge application prospect of UAV that it has attracted extensive attention of researchers in recent years. However, the existing UAV technology is not enough to meet the technical requirements of different operation scenarios. Especially the perception and obstacle avoidance ability of UAV is an important part to ensure the safe operation of UAV. Therefore, this paper mainly summarizes the research results of UAV perception and obstacle avoidance. The contributions of this paper are summarized as follows: 1) In the second section, several common means of environment perception for UAV to complete obstacle avoidance task are described, and the advantages and disadvantages and application fields of each method are listed. The detection principle and classification of binocular stereo vision algorithm are emphasized 2) In the third section, several common UAV trajectory planning algorithms are described. This paper describes the main reasons for the failure of UAV trajectory planning and how to maximize the success rate of UAV trajectory planning in practical use.
3) In the fourth section, describes the development prospect of UAV obstacle detection algorithm and the problems to be solved in the future.

Uav environment awareness system
UAV face many challenges in autonomous obstacle avoidance in large outdoor scenes. In the past decades, many scholars have done a lot of research work to help UAVs achieve autonomous navigation in known and unknown environments. For example, lidar [1], monocular camera [2], binocular camera [3], RGB-D sensors [4] and IMU(Inertial Measurement Unit) are used to realize realize the tasks of obstacle identification, distance measurement and 3D reconstruction in unknown environment.

RGB-D camera
RGB-D cameras provide both color images and per-pixel depth estimates. The richness of this data and the recent development of low-cost sensors have combined to present an attractive opportunity for mobile robotics research. Abraham Bachrach, Samuel Prentice, Ruijie He of computer science and Artificial Intelligence Laboratory of Massachusetts Institute of technology [4] studied the visual odometer map system based on RGB-D camera for autonomous navigation of UAV in GPS failure environment, and gave its application in autonomous navigation of UAV combined with specific experiments. The overall block diagram of the system is shown in Figure 1. The system decouples realtime local state estimation from global SLAM (Simultaneous Localization and Mapping). Local state estimation is calculated by visual odometer. In order to correct the drift in these local estimation, the estimator periodically combines the position correction provided by SLAM algorithm. This structure allows SLAM algorithm to use state estimation of SLAM algorithm to control vehicle more processing time than that of SLAM algorithm. The system inputs RGB-D data into visual mileage algorithm and matches the detected features. By using advanced navigation algorithm and hardware platform, the system can achieve robust flight in cluttered three-dimensional environment using only airborne sensor data. However, RGB-D depth camera is mainly used for indoor scanning, but it is not suitable in the chaotic outdoor environment.

Lidar
Accurate sensing, obstacle detection and modeling of cluttered, unstructured scenes, such as natural outdoor environments, is a diffcult research challenge. Nicolas Vandapel, James Kuffner, Omead Amidi of Carnegie Mellon University [1] develops techniques for enabling efficient three-dimensional(3-D) path planning among obstacles represented by dense point data sets. These data sets are typical of the data returned by Light Amplification for Detection and Ranging devices (LADAR). It is further proposed an approach that for small scale UAV navigation within and below the tree canopy using a priori aerial LADAR data. 3-D safety tunnels are constructed by exploring segmented data. Finally, they demonstrated their approch using aerial data collected by the CMU autonomus helicopter ( Figure 2). Although the accuracy of lidar used in UAV navigation is high, carrying lidar sensors on MAV will increase the load and increase the power consumption of UAV.

Monocular camera
Stephane Ross and others from Carnegie Mellon University Robotics Institute [2] use monocular camera to sense the environment around the UAV, use IMU to detect the lateral drift of the UAV and use Dagger algorithm to generate strategies to simulate expert behavior through repeated iterative training. In addition to color features, nine additional features are added. They use a cheap, commercially available quad-rotor helicopter, namely the Parrot ARDrone, as their airborne platform ( Figure 3). Although the monocular camera is used to realize the obstacle detection and obstacle avoidance function of UAV, as far as monocular camera is concerned, the information obtained by the obstacle detection system of UAV Based on single camera is limited, and sometimes the obstacle detection is lost, which leads to the decrease of the reliability of navigation system.

Binocular camera
Stereo vision is considered as a more robust obstacle avoidance method [5]. Its main advantage is that it can detect objects in the direction of motion and provide absolute distance estimation for these objects. Stereo vision technology is widely used in fixed wing [6], quadrotor [7] and multi rotor [8] autonomous navigation. A stereo camera is usually composed of two cameras. Due to the slightly different viewing angles, the scene point M is projected to different positions m1 and m2 in the left and right camera images. The pixel level matching between the two input images is found to get the corresponding parallax and the depth map. When the camera parameters are known, the corresponding pixels can be found in the camera image to calculate the depth of the corresponding points and reconstruct their 3D position. Two key factors need to be solved in computer stereo vision are speed and precision.
In the past decades, a lot of research has been done to improve the parallax accuracy or the complexity of the algorithm. More advanced stereo vision algorithms can be divided into convolutional neural network (CNN) stereo vision algorithm [9,10,11] and traditional stereo vision algorithm [12,13,14]. The former generally describes disparity estimation as a binary classification problem and learns the probability distribution of all disparity values. For example, Chang J R [8] used pyramid stereo matching network for multi-scale context aggregation for depth estimation. Chao Zhou [14] proposed an unsupervised stereo matching learning framework in order to solve the problems of insufficient training data and difficult depth annotation of ground truth. Although the above methods have obtained some high-precision disparity maps, they usually need a lot of labeled training data to learn. Therefore, if there is no trained disparity map, it is difficult to carry out subsequent network training. Moreover, using CNN to predict parallax map is still a complex task. It takes several seconds or even minutes to get the detection results on the most advanced graphics card. Traditional stereo vision algorithms are mainly divided into local, global and semi global. Local algorithms usually select a series of blocks from the target image and match them with the constant blocks selected from the reference image, and then determine the parallax by finding the drift distance with the highest correlation or the lowest cost. This optimization method is also known as WTA (Winner Take All) method. Different from local algorithms, global algorithms usually transform stereo matching problem into energy minimization problem, and then use complex optimization methods to solve the problem, such as belief propagation (BP), graph cuts (GC), etc. These techniques are usually based on the evolution of Markov random field (MRF). Semi global matching (SGM) [12] algorithm calculates the cost in all directions of the image, and takes the cost aggregation as the approximate value of the global energy function. WTA method is used to calculate the parallax, and further parallax thinning is done by means of consistency check and sub-pixel interpolation, which greatly improves the accuracy and efficiency of stereo matching.
Combined with the previous content, the influence of different types of sensors on UAV obstacle avoidance is summarized, as shown in Table 1. RGB-D depth camera is mainly used for indoor scanning, but it is not suitable for clutter outdoor environment; monocular camera has low fault tolerance rate of obstacle detection, which leads to low reliability; although lidar is used for UAV navigation with high accuracy, carrying lidar sensor on micro air vehicle will increase the load, thus increasing the power consumption of the aircraft. Although binocular camera has strong robustness, how to reduce the complexity of airborne computer obstacle detection is still a problem to be considered.

UAV trajectory planning system
When UAV flies in unknown environment, it needs to avoid obstacles in a very short time. At this time, the flight trajectory replanning of UAV is very important for autonomous navigation of UAV. The existing trajectory re planning methods are mainly divided into two categories: hard constraint method and gradient based optimization method.

Trajectory Planning Method with Hard Constraints
Mellinger D [15] first proposed the concept of hard constraint method, The main idea is to express the flight trajectory of UAV as piecewise polynomial and solve the quadratic programming (QP) problem. According to the test results of Richter C, Bry A, Roy N [16], the closed minimum capture trajectory can be obtained by adding intermediate waypoints iteratively to ensure the safety of navigation trajectory. Ding W [17] and Liu s [18] proposed a dynamic search method based on B-spline to find the initial trajectory, and then used the elastic band optimization method to optimize the initial trajectory. Using uniform B-spline method can ensure the effectiveness of flight trajectory, but this method can not meet the trajectory re planning problem of aircraft in high-speed flight state. And the above methods have a common disadvantage, that is, the time allocation of trajectory is given by a simple heuristic algorithm. However, improper time allocation will significantly reduce the quality of flight trajectory. In addition, only by iterating and adding more constraints to solve the quadratic programming problem, can the feasible solution be obtained. To solve these problems, Gao F [19] proposes a path search algorithm with reasonable time allocation, which ensures the safety of trajectory and the feasibility of dynamics through optimization. The hard constraint method guarantees global optimization by convex formula. However, in free space, the calculated trajectory is too close to the obstacle due to the neglect of the distance from the obstacle. In addition, the calculation method of dynamic constraints is too conservative, which is obviously not suitable for UAV trajectory planning in fast flight.

Gradient Based Trajectory Planning
GTO (Gradient Based Trajectory Optimization) has been widely used in UAV trajectory planning. In GTO, trajectory replanning of UAV is usually described as a nonlinear optimization problem to balance smoothness, safety and dynamic feasibility. It can easily transform the infeasible trajectory into the feasible one, and has high efficiency and low memory requirement, so it has been widely used. Gao F [20] improves the success rate by providing a high-quality initial path, which is found through the sampling based path search, but the above method is only suitable for the trajectory planning problem of low-speed aircraft. Usenko V [21] further parameterizes the flight trajectory of UAV into uniform Bspline curve. The results show that the continuity and locality of B-spline function are particularly useful for trajectory re planning. Zhou B [22] further uses the convex hull characteristics of B-spline curve to greatly improve the efficiency and robustness of flight trajectory optimization. However, in the complex outdoor environment, due to the poor initial trajectory, this method still can not achieve robust trajectory planning. So far, there are still local minima in gradient based trajectory planning.
In order to solve the local trajectory replanning problem of UAV in a short time range in complex three-dimensional environment and fast flight state, Zhou B [23] proposed a dynamic path search method based on heuristic search and linear quadratic minimum time control in discrete control space. It uses the dynamic path search method to find the safe, dynamic and feasible initial trajectory with the shortest time. By using the convex hull property of B-spline function, combined with the gradient information of Euclidean distance field and dynamic constraints, the trajectory is optimized by B-spline function, and the smoothness of the trajectory is improved. Finally, the optimal trajectory is expressed as a non-uniform B-spline curve, and the iterative time adjustment method is used to ensure the dynamic feasibility and non conservatism of the trajectory.
Furthermore, the reasons for the failure of existing trajectory planning methods are summarized [24]. The main reason for the failure of aircraft trajectory planning is the problem of trajectory initialization (Fig 4). Typical GTO methods incorporate the gradients of a Euclidean signed distance field (ESDF) in a collision cost to push the trajectory out of obstacles. This cost is combined with the smoothness and dynamic feasibility cost to form an objective function, whose gradients iteratively deform the trajectory into smooth and safe one. Yet there are some "valleys" or "ridges" in the ESDF, around which the gradients differ greatly. Consequently, if a trajectory is in collision and crosses such regions, the gradients of ESDF will change abruptly at some points. This can result in gradients of the objective function pushing different parts of the trajectory in opposing directions and fails the optimization.

Fig. 4 An example of radar UAV obstacle avoidance
In order to solve this problem, Zhou B [24] designed a path guided optimization (PGO) method including multiple topological paths, which generates multiple parallel local optimal trajectories through path guided optimization. In this step, the optimistic assumption strategy is adopted. Because the previous trajectory planning method is based on optimistic assumption, it lacks the ability of environment perception, which limits its application in complex and high-speed environment. To solve this problem, Zhou B [25] further adds environment awareness strategy into its trajectory planning framework to ensure the safe and fast flight of UAV from two aspects: Firstly, a risk perception trajectory optimization method combined with optimistic hypothesis planner is proposed to ensure that the obstacles in the unmapped area can be detected as early as possible, Secondly, the yaw angle of UAV is included in the trajectory planning framework to ensure the safety and fast flight of UAV The optimal yaw angle sequence with maximum information gain and smoothness is searched in discrete state space, and the flight trajectory is further smoothed by optimization strategy. The planned movement of yaw angle enables the UAV with limited field of vision to actively explore the unknown space and obtain more relevant information for future flight.

Development trend of UAV obstacle avoidance
Through the discussion in Section 2 and section 3, it can be found that as two important parts of UAV obstacle avoidance task, environment sensing sensor and UAV path planning algorithm play an important role in UAV autonomous navigation. In recent years, UAV autonomous obstacle avoidance technology has made some achievements, but there are still many problems worth exploring: 1) In the autonomous obstacle avoidance navigation of UAV, how to improve the real-time computing performance of UAV is very important. Therefore, the research on the real-time obstacle detection system and path planning algorithm of UAV with robustness, fast computing speed and small memory occupation is the direction of further efforts.
2) Due to the diversity and complexity of UAV operating environment, it is unreliable to only rely on a single sensor to sense the surrounding environment, especially in the case of GPS failure. Therefore, exploring the autonomous obstacle avoidance navigation system of UAV Based on multi-sensor fusion should be the next problem to be solved.
3) The existing research work only takes a single UAV as the research object. In the cluster control based on multi vision perception system, how to obtain the relative position, relative attitude and other information between the sub perception systems, so as to predict the danger in advance and calculate the scheduled flight path of different UAVs, will be the development direction of UAV autonomous obstacle avoidance navigation system in the future.

Conclusion
This paper mainly discusses the research status and future development direction of UAV obstacle avoidance. For large UAV equipment, lidar combined with IMU is mostly used as the UAV sensing system. If the UAV is in indoor activities and needs fine environmental information, RGB-D camera can be considered as the UAV data acquisition platform. In order to complete some special tasks in outdoor complex scenes, such as fast flight obstacle avoidance, we need to consider the power consumption, real-time performance and robustness and choose binocular camera as the visual input system of UAV. In the face of UAV multi scene, multi-target and complex outdoor scenes, we need to consider how to research and design better visual perception algorithm, visual recognition algorithm and visual recognition algorithm. In addition, the hardware system with smaller quality and stronger performance is also the research direction in the future.