Research on the Method of 3D Registration Technology

Three-dimensional registration technology is the foundation, key and difficult point of constructing augmented reality system. This paper first reviews the three-dimensional registration technology in augmented reality, and compares the advantages and disadvantages of each method. Then, from the improvement of feature descriptor, the combination of registration technology and tracking technology, online learning-based registration technology and deep learning-based registration technology, the improvement strategy of registration method is analysed and summarized, which is the three-dimensional registration technology. In-depth research provides ideas. Finally, the problems existing in the research of 3D registration technology are pointed out, and the future research directions are prospected.


Introduction
Based on computer graphics technology and visualization technology, Augmented Reality (AR) adds positioning virtual objects in 3D space, which can integrate real scene and virtual scene information, and has real-time interaction. Since the concept of induced maintenance based on augmented reality was put forward, AR's research in the field of maintenance has gradually deepened. Benbelkacem et al.
[1] designed a maintenance system using AR technology in pump maintenance to solve the maintenance problem of complex photovoltaic water pumps; the United States Air Force Research Laboratory and Columbia University and other six units carried out augmented reality-based maintenance and repair projects. To verify and evaluate the feasibility and effectiveness of AR in the field of equipment maintenance support [2], and completed the development of the wheeled armored vehicle auxiliary maintenance system in 2011. On the whole, the AR-based equipment maintenance system has the following advantages: (1) shortened the training time of personnel, reduced the consumption of equipment wear and fuel and other resources, and saved maintenance costs; (2) reduced the misoperation of relevant personnel The risk provides a safe working environment; (3) It can realize human-computer interaction and improve training efficiency. Especially in the harsh conditions such as battlefield repair, the support personnel do not need to frequently switch between the maintenance manual and the equipment, which can ensure the task is completed accurately and efficiently.
The 3D registration tracking technology is the core technology of augmented reality. Its purpose is to accurately calculate the position and posture of the camera in real time and accurately. Based on this, the position of the virtual information to be added in the projection plane is obtained. Blended with real scenes [3]. As a difficulty in building an AR system, the effect of 3D registration directly affects the performance of the system. In view of this, this paper compares the advantages and disadvantages of 3D registration tracking technology in augmented reality, and analyzes the improvement strategy of registration tracking technology, which provides an idea for the in-depth study of 3D registration tracking technology.

Research status of 3D registration technology
According to different equipment's, 3D registration technology can be divided into sensor-based registration technology, vision-based registration technology and hybrid registration technology, as shown in Figure 1. Among them, sensor-based registration technology includes electromagnetic tracking technology, optical tracking technology, ultrasonic tracking technology, inertial tracking technology, etc.; vision-based registration technology can be divided into identification methods based on artificial markers and based on A non-identifying registration method for natural features, wherein the model-based method has been widely used in augmented reality unidentified three-dimensional registration tracking systems, such as Kinect Fusion, LSD-SLAM (Large-Scale Direct Monocular Simultaneous Localization and Mapping). Hybrid registration technology combines hardware registration technology with visual registration technology to separate feature extraction from subsequent points, including magnetic field and visual based registration techniques and inertial and visual based registration techniques, neural network and visual registration methods. And a registration method based on feature operator and vision, a TLD-based augmented reality tracking registration method, and an augmented reality tracking registration algorithm for FAST key points. Different registration methods have their own characteristics. In practice, they should be combined with specific

Sensor-based registration technology
Combined with the development of recent years. According to the sensor-based registration technology, the position and rotation angle of the camera are measured by the sensor, and the external parameter matrix of the camera is obtained. It mainly includes the following methods: 2.1.1. Electromagnetic tracking technology. As a position and attitude measuring device, the electromagnetic tracker is widely used in augmented reality, and the position and direction of the observer are calculated by sensing the intensity and direction of the low frequency magnetic field emitted by the corresponding emitter to the space. Can be divided into AC electromagnetic tracker and DC electromagnetic tracker. The electromagnetic tracking technology has good real-time performance, and can be well adapted to the field of view transformation and occlusion. However, due to the small field range, the method is poorly adaptable to distance changes and is easily distorted when current and magnet materials are present around the scene. Therefore, electromagnetic tracking is only suitable for small-scale systems without metal materials and electrical equipment.
2.1.2. Optical Tracking Technology. The optical tracking technology places the light source in a real scene according to certain rules. The system receives the image through the camera and uses the light received by the photoreceptor to determine the position, velocity and speed of the object. The method has good real-time performance, but the out-of-stock of the obstacle is large and the cost is high.
2.1.3. Ultrasonic tracking technology. Ultrasonic tracking technology uses multiple ultrasonic transmitters to transmit ultrasonic waves, and calculates information such as the position, velocity and direction of the tracking object based on the time difference, phase difference and sound pressure difference of the ultrasonic signals arriving at the receiver. Ultrasonic tracking costs are relatively low, but are susceptible to other ultrasonic sources in the surrounding environment. In addition, the technology has a low refresh rate and a long delay time, and the real-time performance needs to be further improved. Inertial tracking uses a gyroscope (and accelerometer) to measure the angular velocity of a moving object and integrates the measured value over a period of time to obtain the position and orientation of the object. The inertial tracking device is light and easy to carry, suitable for outdoor and dynamic scene detection. However, due to the accumulated error of zero drift and random error of the currently used MEMS sensor, inertial tracking is not suitable for long-term tracking.

Vision-based registration technology
It is mainly divided into methods based on markers and no markers.

Marker-based registration technology.
The marker-based registration method refers to placing some artificial markers (such as two-dimensional code and circle) whose relative positions are known in the scene to be registered, and then combining the projection principle to process the image containing the logo. Camera tracking.
The signature-based registration method is simple in principle, and can extract image features quickly and efficiently. It has high registration accuracy and good stability, and is the most mature registration method currently developed. However, there are still some shortcomings: (1) in the pose solution process, the marker is likely to cause positional interference, causing the registration result to be jittery; (2) in the scene where the marker is not convenient to be placed (such as augmented reality based indoor navigation), the maintenance of large-scale weapons and equipment, etc., the registration method based on the marker is not applicable; (3) the algorithm requires that the marker must appear completely within the camera's viewing angle to be accurately registered, with certain limitations.

Unsigned registration technology.
The unidentified method is mainly based on the registration of natural features, without the need to manually set the mark, but to take some natural features in the real scene as a reference for three-dimensional registration, and estimate the position of the camera in the world coordinate system through the extraction and matching of natural features. Then complete the registration. In the early stage, the texture and contour information of the target were mainly used. The degree of similarity of the thoughts by comparison, scoring, and classification reached a high score, and the higher the computer score was obtained. In recent years, neural networks have been used to extract high-level information, semantic information and other features of the target. In summary, natural feature-based registration techniques can be divided into feature descriptor-based methods and classifier-based methods. Among them, the common feature descriptors include SIFT (Scale-In-variant Feature Transform) operator [4], SURF (Speeded up Robust Features) operator [5], and FAST (Features from Accelerated Segment Test) operator [6], ORB (Oriented FAST and Rotated BRIEF) operator [7], KAZE operator [8] and so on. In comparison, SIFT and SURF operators use high-dimensional vector features to describe the key information of objects, but the complexity is high, the calculation is large, and the real-time performance is poor. The FAST operator is faster, but the scale information of the image cannot be extracted. It is difficult to adapt to scenes with large changes in image illumination and scale changes; the ORB operator has greatly improved the feature extraction speed, but it needs to use the non-extreme constraint processing to detect the pseudo corner points in the segment detection phase, which increases The complexity of registration; KAZE operator uses the nonlinear diffusion method to construct the scale space, which can extract features effectively, but the computation is large, and the feature cannot be extracted quickly when there is motion blur in the image. The classifier-based registration method treats the matching of feature points as a classification problem, and has the advantage of fast recognition speed. For example, classifiers such as random tree, random forest, and random fern [9] have been applied in augmented reality three-dimensional registration, but it is difficult to adapt to the situation with high target complexity.
Natural feature-based registration directly acquires the position of the camera by extracting and matching natural features, which is relatively flexible, and registration can be performed smoothly as long as two adjacent frames have enough matching feature point pairs. However, there is a registration error in this method, and the error will increase as the system running time increases. In severe cases, the registration will fail.
The vision-based registration technology will reduce the number of feature points under the influence of observer's pose change and illumination, noise, etc., which will affect the accuracy of registration. Under severe conditions, registration failure may also occur.
Early mark-based augmented reality technology relied on the existence of a label image frame and could not be applied to the environment of natural features. Therefore, the application environment tends to be a mark-free augmented reality technology. From the application point of view, the signature-based registration is generally applicable to a scenario in which the identifier is easily set and the registration real-time requirement is high. The registration based on the natural feature is mainly applicable to the scenario where the outdoor inconvenience is pre-set. In order to overcome the shortcomings of single-registration technology, scholars began to study hybrid registration technology, combining vision-based registration and sensor-based registration technology to take advantage of various registration methods. For example, firstly, the FAST corner point is extracted, and when the subsequent point matching is performed, the feature of generating the feature vector using the integral image by the SURF descriptor improves the efficiency of calculating the image gradient and reduces the time consumption. Common hybrid registration techniques can be divided into magnetic field and visual based registration and inertial and visual based registration. From the perspective of fusion methods, Extended Kalman Filter (EKF), Unscented Kalman Filter (UKF) [10] and Particle Filter (PF) [11] have been applied. However, the EKF fusion algorithm often needs to linearize the user's motion model, which will reduce the accuracy of registration, and even cause the filter to diverge, which is difficult to adapt to the occasions of complex motion and high registration accuracy requirements. The UKF algorithm improves the accuracy of registration, but it is still based on the premise that the observer does uniform motion or uniform acceleration. The defects of this model cannot adapt to the sudden movement of the observer.

Mixed Registration Technology
On the whole, hybrid registration technology enhances the robustness and usability of the registration method, but the cost of the technology is higher, the fusion algorithm is more complex, and the adaptability is poor for occasions with complex actions and high requirements for tracking accuracy.

3D registration technology improvement strategy
From the perspective of algorithm execution, the three-dimensional registration method can be divided into several stages: feature extraction, feature description and execution registration algorithm. The following highlights the improved strategies for feature descriptors and registration techniques.

Improvements to feature descriptors
In order to improve the validity of feature descriptors, scholars have extended the construction of subscale space from linear to nonlinear. For example, the SIFT and SURF feature descriptors use linear diffusion to construct the scale space, and the edge features of the image cannot be accurately extracted. When the picture is fuzzy, the feature extraction effect is not ideal. The KAZE feature descriptor uses the nonlinear diffusion method to construct the scale space, which improves the effectiveness of feature extraction to some extent, but the real-time performance of the algorithm needs to be further improved.
In order to improve the real-time performance of descriptor extraction features, scholars have proposed binary-based feature descriptors, such as the BRIEF (Binary Robust Independent Elementary Features) descriptor [12], FREAK (Fast Retina Key point) descriptor [13]. FREAK descriptors are searched by glance search and matched from coarse to fine. They have good real-time performance and good adaptability to rotation, scale, illumination changes and image blur. In order to overcome the fact that the BRIEF descriptor does not have rotational invariance, Lu Hongbo et al. [14] also proposed a rotation-invariant binary local descriptor RMBD; Wang J et al. [15] proposed a CS-FREAK description based on the FREAK descriptor. The sub-field increases the intensity information of the neighborhood of the central symmetric sampling point and improves the robustness and realtime performance of the FREAK descriptor.
In order to improve the robustness of feature descriptors, Liu Jia et al. [16] combined the AGAST (Adaptive and Generic Accelerated Segment Test) method with the FREAK method, making full use of the robustness of the AGAST method for illumination transformation. In addition, using the filtering algorithm to preprocess the image can also improve the accuracy of feature extraction and reduce the impact of environmental noise on the registration method.

Combination of registration technology and tracking technology
In order to improve the efficiency of the registration algorithm, scholars combine the tracking technology with the registration technology, pre-determine the approximate region of the registration target through the tracking algorithm, and then perform feature extraction and matching in the region to finally realize three-dimensional registration. For example, Yan Yuruo and others use the tracking algorithm based on compressed sensing [17] to track the target and use the nonlinear scale space to register in the tracking region, which improves the speed of feature extraction and the robustness of the algorithm to changes such as illumination and occlusion. Yan Xingya et al. [18] used the optical flow method to track the target in real time, and used the ORB operator to extract features to ensure the real-time performance of the registration algorithm. The registration method of the Tracking Learning Detection (TLD) tracking algorithm has good stability and robustness. In addition, the "tracking-by-detection" real-time tracking model based on classification idea makes some classification algorithms can also be used for target tracking, which provides more reference for the combination of target tracking and registration methods.

Online learning-based registration technology
Online learning can make full use of the calculation results in the online scene feature matching process to achieve effective learning of the scene, which can improve the registration accuracy. For example, the online reconstruction registration SFM (Structure from Motion) [19] method achieves higher registration accuracy, but the stability and real-time performance of the algorithm need to be further improved. Gui Zhenwen et al. [20] in the natural feature-based registration method, the multifeature tracking technology combining online learning and optical flow tracking has the characteristics of good real-time performance and strong stability.

Deep learning-based registration technology
In recent years, the idea of deep learning has triggered a revolution in the field of feature extraction and mining, which also provides new ideas for the improvement of registration technology. For example, A Zhen et al. [21] proposed a three-dimensional tracking registration technique combining semantic target grayscale and depth information. Firstly, the improved SSD depth convolutional neural network is used to semantically segment the image and combine the grayscale constraints of the image. The geometric constraint realizes the estimation of successive poses, which improves the adaptability of the registration algorithm to the lack of feature points and mismatch under weak texture conditions. Qiao Yanjun et al [22] combined mobile augmented reality with deep learning, using deep convolutional neural network for image classification and target detection, and combined with residual neural network, proposed a solution for detecting geographic entity targets in outdoor augmented reality.

Combine the specific application background to choose the appropriate registration method
The research of any technology cannot be separated from its application background, the background is different, and the requirements for registration technology are not the same. For example, mobile computing has limited computing and storage capabilities, and the corresponding registration algorithm should reduce the amount of computation to improve execution speed under certain conditions of accuracy and robustness. Conversely, if the large server can be reconstructed based on the GPU (Graphic Processing Unit) calculation, and then the feature points are tracked, a better registration effect can be achieved, but the cost is too high. Therefore, coordination between real-time and hardware conditions, robustness and reliability, accuracy and computational complexity is required from an engineering perspective.

Study registration methods in complex environments to improve the robustness of registration methods
The outdoor environment is more complicated. In practice, there are often obstacles such as occlusion and terrain visibility, and illumination changes. To ensure the accuracy of registration results in complex environments, improving the robustness of the registration method will be an important development in the future direction. At present, scholars have initially explored the use of cone splitting [23] registration strategy to solve registration problems under restricted conditions; in order to avoid pose jitter or disturbance caused by lack of sufficient texture features on the surface of the object, non-introduction can be introduced. Iterative perspective n point problem solving method [24] to improve the accuracy and speed of registration.
It should be noted that the study of multi-method hybrid registration technology is the easiest place to break through, such as the current algorithm based on SURF combined with optical flow method, or the use of sensors to obtain approximate position, reduce the duration of early capture, improve realtime, and then pass The visual registration method further obtains the precise position and improves the stability. The impact of the actual environment on the registration effect is compound, and various improvement methods are often used in combination to obtain the desired effect. Therefore, the study of registration methods in complex environments is the key to their application.

Studying Tracking Registration Techniques for Unknown Scenes
Tracking registrations for studying unknown scenes have a wider range of applications than registrations for known specific scenarios. Simultaneous Localization and Mapping (SLAM) can locate its own position under the condition that the scene is unknown, and establish a threedimensional map of the surrounding environment to ensure the good consistency between the virtual object and the real scene. Since the introduction of this technology has attracted a lot of scholars' attention, then improved algorithms such as EKF-SLAM, IEKF-SLAM, V-SLAM [25] have been proposed to obtain more effective registration results. With the deepening of registration technology research, the advantages of registration technology based on unknown scenes will become more and more obvious.

Conclusion
This paper makes an in-depth analysis of the augmented reality registration technology from the research status, improvement strategy and research prospects. First of all, no algorithm can be universally applied as a consensus, and must be combined with specific problems for analysis and design. From the application status quo, the development of signature-based registration technology is relatively mature, and related products have been available, but its application scope is limited; registration technology based on natural features should be widely applied, and it is also necessary to solve complex environments, especially illumination. Changes and three-dimensional registration problems under occluded conditions. With the continuous development of 3D registration technology, the induced maintenance based on augmented reality will play an important role in the modern information warfare of the battlefield environment and increasingly complex weapons and equipment.