High precision map crowdsource update technology and SLAM technology - Application in autonomous driving

In response to the challenges faced by real-time dynamic updates of high-precision maps for autonomous driving in terms of high precision, high reliability, and high safety, this paper summarizes the difficulties and challenges currently faced by high-precision map updates. Accelerate the large-scale commercialization of autonomous driving high-precision maps. Thus, improving the safety and stability of intelligent vehicles and providing critical support for high-level autonomous driving. Firstly, the definition and connotation of autonomous driving maps were described, and the data characteristics and functional applications of autonomous driving maps were pointed out. Secondly, the development status and trends of high-precision map updates were reviewed, and the advantages and disadvantages of centralized map updates and crowdsourced map updates were summarized. Pointing out that crowdsource updates have become a new trend in the development of map updates. Once again, the basic architecture and key core modules of current crowdsource updates were summarized, and an analysis was conducted on key core technologies, summarizing the current status and trends of key technologies involved in crowdsource updates. The results indicate that the current technology still faces 7 main challenges, including map modeling, high-precision positioning, 3D reconstruction, fusion updates, data security, rapid review, and standard laws and regulations to address these challenges.


Introduction
With the significant increase in car ownership, issues related to traffic safety and congestion have attracted widespread attention.The "new four modernizations" trend, characterized by electrification, intelligence, networking, and sharing, is emerging globally.Intelligent connected vehicles, as an essential technological means to improve traffic safety and transportation efficiency, have become the focus of global attention in the field of vehicles and transportation [1].At present, various countries are accelerating the layout of the intelligent vehicle industry, and foreign automobile industry giants, including the United States, Germany, Japan, etc. are actively promoting autonomous driving.Many countries have proposed building a road traffic geographic information system that covers the national road network, developing standardized intelligent vehicle basic maps, and providing real-time dynamic data services [2].Moreover, it is clear that the basic map of intelligent vehicles is one of the key basic technologies that must be broken through.At the same time, China has clarified the high-precision

The Connotation of High Precision Maps
With the continuous improvement of automobile intelligence, the requirements for map accuracy and freshness are also gradually increasing.The development of in-car electronic maps has gone through navigation electronic maps, advanced driving assistance system (ADAS) maps, and high-precision maps for autonomous driving.The comparison in Table 1, shows that the map accuracy has developed from the meter level to the decimeter level and even to the centimeter level.In terms of update frequency, from annual to quarterly updates, gradually upgrading to daily updates, and even real-time updates [4].Regarding map content, it gradually upgrades from road-level to lane-level static data to lane-level dynamic and static real-time data.The continuous increase in data content has led to the volume of highprecision map data for autonomous driving exceeding that of ordinary maps by more than 10 ^ 5 times.

Basic architecture for high-precision map crowdsource update
At present, there is relatively little research on high-precision map crowdsource updating, and existing studies mainly focus on reconstructing and updating individual map elements.Tchuente et al. proposed an updated architecture for traffic signs, including modules for collection, storage, monitoring, fusion, and update release.Use the car camera as the main sensor for data collection, storage, and recognition, and then send the car data to the cloud platform for storage and processing [5].Cloud-based noise filtering and clustering processing are performed on the detection results of multiple vehicle data, and the confidence level of clustering results is statistically analyzed.Match with high-precision base maps, project the newly created elements onto the corresponding layers of map elements and regularly send them to the vehicle end to update the map changes (Figure 1).At present, there is relatively little research on the architecture of high-precision map crowdsource update systems, and there is still no relatively complete unified system architecture [7].Based on the basic principles of high-precision maps and existing framework models, the basic architecture of the crowdsource update system has been sorted out.On the basis of interdisciplinary integration, an autonomous driving map model that supports dynamic map updates is the fundamental support.The key technologies of vehicle and road end focus more on real-time road information acquisition and real-time perception of map elements.In high-precision positioning at the vehicle end, match and analyze the vehicle end perception results with high-precision base maps to achieve high-precision map change estimation.The cloud focuses on data cleaning and mining based on a large amount of data uploaded from the vehicle and roadside, analyzing data uncertainty to establish a reliable model of map elements [8].Through high-precision matching and fusion of multi-source data, high-precision reconstruction of map update features is carried out.Furthermore, conduct a quality evaluation based on the multidimensional characteristics of high-precision maps.Finally, after security review, the updated high-precision map data will be released to the vehicle through a secure transmission channel.A highprecision real-time dynamic map for autonomous driving is formed while ensuring the effectiveness, real-time performance, safety, and high-precision characteristics of high-precision map data updates.

Key technologies for high-precision map crowdsource update
The critical technologies for high-precision map crowdsource update, mainly focused on mass production of intelligent vehicle visual data, mainly involve map models, vehicle positioning, visual reconstruction, multi-source data fusion and update, and security.Among them, the map model is the basis for updating the crowdsource of autonomous driving maps [9].The high-precision positioning of the vehicle end is a prerequisite for the matching and updating of various map elements.The highprecision 3D reconstruction of map features based on visual sensors is the core of crowdsource updating.The accurate estimation of maps by integrating multi-source data is the key to highly reliable updates of autonomous driving maps.Data security technology is the guarantee for updating crowdsource maps.

Automatic driving high-precision map update expression model
Autonomous driving high-precision maps not only provide precise perception, positioning, and lane prediction for intelligent vehicle decision-making and control but also provide dynamic real-time traffic information, plan paths in advance, and effectively avoid congestion and traffic obstacles.At present, high-precision map updates are gradually receiving close attention from international research institutions and enterprises.The high-precision map expression model for autonomous driving is the prerequisite and foundation of map updates, which not only needs to be combined with the actual functional application requirements of autonomous driving but also needs to support high-precision map incremental updates [10].
In recent years, there has also been some progress in high-precision map models.He Yong et al. proposed a high-precision map data storage model that uses object relational databases to store map data.The first layer expresses road network information, and the second layer expresses lane network, the third layer is lane line, and the fourth layer is traffic sign layer, including information such as signal lights, road signs, and ground road signs.Kang et al. proposed a simple and open-source high-precision map model based on the node edge model, and designed a representation and storage method for road networks and landmarks.There is still room for improvement in efficient data management, and it is clearly pointed out that the support for efficient updating of map data is an important direction for future model improvement [11].
In summary, there is currently research on the types of elements, geometry, attributes, and expression methods of autonomous driving map models.For model design, most autonomous driving map models adopt a hierarchical structure expression framework, which is extended on the basis of navigation electronic map models.In terms of data content, it includes static road environment information and adds dynamic real-time elements to enrich the expression content.Due to higher requirements for accuracy and content richness of autonomous driving map data, it can also lead to more complex map model structures.Moreover, it needs to support real-time incremental updates of autonomous driving maps.Therefore, it is necessary to closely combine the actual needs of autonomous driving and design a reasonable map model to ensure the compatibility and flexibility of the autonomous driving map model.Especially in the face of complex road traffic scenarios, it is necessary to establish a unified autonomous driving map model that supports all elements, flexible expansion, and real-time updates.

High precision positioning based on vehicle vision data
The mainstream solution for updating autonomous driving maps, especially with mass-produced intelligent car visual sensors, requires the real-time perception of vehicle elements to be sent to cloud platforms for efficient fusion and update.However, the accuracy of the spatial location of map elements based on visual acquisition at the vehicle end directly affects the accuracy of map updates.The vehicle positioning capability has also become an important prerequisite for high-precision map crowdsource updates.Therefore, high-precision positioning based on vehicle vision data is an important foundation for achieving crowd source updates on autonomous driving maps.How to use the existing visual sensors in the vehicle and roadside for high-precision positioning is a key issue that needs to be addressed in map crowdsource update solutions.The following will mainly elaborate on visual positioning methods.
For vehicle end positioning, professional mobile measurement systems are equipped with highprecision GPS positioning equipment and high-precision inertial navigation systems, which obtain highprecision positioning information through data post-processing.However, for mass-produced intelligent vehicles from various sources, the vehicle is equipped with low-cost and high-precision positioning equipment, making it difficult to directly obtain high-precision positioning information.The problem of vehicle localization is essentially a state estimation problem.For this problem, the main algorithms can be divided into two directions: filtering and optimization.Filtering based algorithms can be explained from the perspective of maximum a posteriori probability.The central idea is to use historical observations and states as prior knowledge of the current state at the moment, and then use Bayesian formulas to model the probability distribution of the current state.Classic algorithms include Kalman filtering, extended Kalman filtering, and particle filtering.Optimization based algorithms are considered from the perspective of maximum likelihood.Taking the vision based positioning problem as an example, by constructing the camera pose and the position constraint relationship of landmarks over a period of time.Establish an optimization equation based on the Gaussian assumption.Optimize the camera's pose through Newton or Gaussian Newton methods.
Among them, optimization based methods require a large amount of computation, and early positioning work mainly relied on filtering algorithms for camera pose determination and positioning.MonoSLAM was the earliest to implement a real-time visual based positioning and mapping system [12].Parallel Tracking and Mapping (PIAM) has implemented a real-time positioning system based on optimization algorithms for the first time [13].This work divides the entire system into the front-end camera visual feature tracking and the back-end pose optimization and mapping parts, and parallelizes them to ensure real-time performance.This framework has become the mainstream design approach for real-time location and map building (SLAM) systems based on optimization algorithms in the future.Strasdat et al. proposed an optimization algorithm based on keyframes, which can achieve higher accuracy than filtering algorithms in equivalent computational complexity [14].ORB-SLAM improves the feature extraction efficiency of the front-end on the basis of PIAM [15][16].At the same time, a more comprehensive keyframe processing and backend map management mechanism has been designed to achieve a robust open-source visual SLAM system.At present, mainstream vision based localization methods mainly rely on pixel features in environmental images for localization, but pixel based localization usually only provides an estimation of a relative pose.In order to obtain absolute positioning information of vehicles, the industry has begun to explore map assisted vehicle end positioning technology.
At present, map based auxiliary positioning methods are gradually becoming a new development trend.The early problem of map based localization was due to the lack of high-precision maps, and visual maps were directly used.Pink uses aerial photographs to construct a visual map, and then extracts lane line edges from the image [17].Match with visual maps, combined with iterative nearest point algorithm and least squares optimization to regress the vehicle's posture.In Courbon's algorithm, images are first taken at different locations on the road to establish a database of visual information [18].By comparing and analyzing the images captured by the current vehicle with the visual information in the database, the vehicle's motion trajectory is obtained to assist in vehicle navigation.With the development of SLAM technology, some research work has begun to use visual SLAM to construct sparse visual feature point maps [19,20].And subsequent vehicles rely on these feature points for positioning on the map.In order to overcome the disadvantage of visual feature points not being robust to factors such as lighting and perspective, as well as being susceptible to changes in lighting and seasons, researchers have attempted to use semantic elements in maps (such as static map elements such as lane lines) to locate vehicles.Some researchers attempt to match the lane lines detected in the image with the lane lines in the visual map to achieve vehicle localization [21,22].Wu et al. proposed a method for binocular camera positioning using road markings (such as arrows, pedestrian crossings, speed limits, etc.), which is based on template matching to automatically detect road markings and estimate the vehicle's pose based on the corner feature points of the road markings [23].But this method assumes that the pitch angle of the vehicle is known on the ground of the platform, and the accuracy still needs to be improved.Barth et al. constructed offline maps of intersections based on street level images and GPS data, including location information and stop line data [24].Then, based on offline map data, the position and attitude of the vehicle can be obtained, with an accuracy of up to decimeters.The use of map based localization algorithms has gradually shifted from visual feature maps to road marking maps with semantic information, greatly improving the robustness of algorithm localization.With the development of high-precision map making technology, the accuracy of maps has been greatly improved, and information has become more abundant, greatly promoting high-precision positioning algorithms based on map matching.Due to the rich geographic spatial information in high-precision maps, existing work has used different high-precision map features to associate with visual features when constructing map matching constraints.Most research work uses original high-precision point cloud maps to locate cameras.Wolcott et al. synthesized camera images using point cloud data from high-precision maps and compared the images collected by the camera with the synthesized images by introducing Normalized Mutual Information (NMI) [25].Use maximum likelihood estimation to obtain the camera's pose on the map.Ding et al. incorporated the constraint of associating visual feature points with high-precision point cloud maps into the local beam adjustment (BA) optimization, and designed a robust, tightly coupled optimization method to improve positioning accuracy [26].On the other hand, some works use vectorized high-precision semantic maps to constrain the posture of vehicles, greatly reducing the amount of high-precision data compared to the original map point cloud.Schreiher et al. attempted to use lane line maps produced by LiDAR [27].By extracting elements of lane lines and curbs from the camera, a camera observation model of map elements is established.Finally, the vehicle is located using Kalman filtering, combined with visual constraints and inertial measurement units.Cai et al. obtained the lateral distance between the vehicle and the lane line from the results of monocular visual perception, and then used a Kalman filtering framework to constrain and fuse GPS, monocular lateral distance measurement, and high-precision maps to obtain high-precision positioning results [28].
Based on the above analysis, it can be concluded that the fusion of prior information and visual information from high-precision maps makes high-precision positioning at the centimeter level possible.This provides strong support for high-precision map updates based on crowdsource vision solutions.The development direction of high-precision positioning in the future will focus more on low-cost and be more closely related to perception technology.However, currently, low-cost, highly stable, and highprecision positioning technologies for vehicle terminals in complex traffic scenarios still face challenges.

3D feature reconstruction based on multi-source visual data
The high-precision dynamic update of autonomous driving maps requires the implementation of threedimensional high-precision reconstruction of local changing map elements on the basis of high-precision base maps to ensure the reconstruction accuracy of changing elements.The difficulty of updating autonomous driving maps mainly focuses on high-precision three-dimensional reconstruction of lowcost monocular visual data.The lack of depth information in visual data poses a challenge for highprecision 3D maps and feature reconstruction [29].
Although there have been studies on map construction based on visual data, the object of construction is still relatively single.Further exploration is needed for the reconstruction and updating of most features in high-precision maps.At the same time, most of the publicly available map reconstruction accuracy in the academic community is still at the meter level.Compared to high-precision point cloud maps obtained using map collection vehicles, there is still a significant gap in accuracy.Based on the current development trend, high-precision map reconstruction work that integrates multi-source visual data requires more precise multi element modeling and richer map construction work.

Change estimation and update technology based on multi-source data
The accurate estimation and update of map change elements is the key to ensuring the reliability of highprecision map dynamic updates based on crowdsources.At present, although visual based geospatial change estimation has a certain research foundation, most studies are based on the same perspective for detection and the scene is relatively single [30].Typical methods for estimating change elements mainly rely on image registration and similarity to determine changes.For example, Alcantarilla and others use monocular vision data from the same perspective to input coarsely registered image pairs into a deep deconvolution network for pixel-by-pixel scene change estimation.IHOMAS et al. used 3D voxel models to store the probability distribution of object surfaces to describe the scene, and matched the 3D linear features in the image to estimate the changes in low altitude perspective data.Suzuki et al. proposed extracting multi-scale features from pre-trained deep convolutional networks and predicting semantic labels of changing regions for real-time perception of scene changes.Optimize, align, and split crowdsourcing vehicle trajectories into corresponding interval observation data.Substitute the calculated feature vectors into the optimal policy objective function and define the probability distribution of the change threshold to determine the change.But this method requires frequent updates of navigation map mapping relationships.Li et al. considered the pose uncertainty of the camera in perspective projection and treated the markings in high-precision maps and crowdsourced images as observation results [31].Considering motion noise, construct a posterior conditional distribution of markings.Use statistical hypothesis testing to evaluate their consistency in the same image coordinate, set a threshold for conditional probability, and choose to retain high probability values.Kim et al. combined the characteristics of laser radar beam divergence and multiple echoes, and based on probability theory and evidence theory, compared the environmental information collected by laser radar with existing map data [32].Determine whether there are new or deleted state elements in the point cloud layer of high-precision maps.But this method is only applicable to crowdsourcing vehicles equipped with LiDAR.Heo et al. proposed a high-precision map change detection algorithm based on depth metric learning, which combines adversarial learning to reduce the domain spacing between images and high-precision maps [33].Using pixel level local change detectors to detect change areas, but cannot recognize change categories and objects, and further expansion is needed for vertical map features.
From the above analysis, it can be seen that the existing map change element estimation and judgment models consider ideal scenarios.Not fully considering the inherent characteristics of the data and the uncertainty caused by many errors.The confidence assessment model for map element changes is relatively simple and still needs further exploration [34].At the same time, high-precision maps, as an important basic service of autonomous driving technology, currently most map management and updates mainly adopt a centralized map management mode.Once the central system fails in this way, the highprecision map services obtained by all vehicles will be severely affected.A more effective approach is to manage high-precision maps in a distributed manner, storing high-precision maps in a distributed manner on crowdsourced vehicles.It is crucial to ensure the accuracy and authenticity of high-precision map update features when facing conflicts with multi-source data.This is also a key issue that must be addressed in high-precision map updates.

crowdsource update data security guarantee technology
Map data involves national security.At present, the traditional offline approval process for maps has a low degree of automation and a long cycle.Difficult to meet the real-time update needs of autonomous driving maps.How to efficiently, safely, and compliantly encrypt and quickly review autonomous driving map data while updating the map in real-time from multiple sources is currently a challenge faced by multiple source updates.The details are as follows: (1) In terms of data confidentiality processing, navigation electronic maps must undergo confidentiality processing before being publicly released.The current offline processing method has a certain time period, making it difficult to ensure the real-time performance of map data.
(2) In terms of map data review, navigation electronic maps require that they must be reviewed by the natural resource management department before they can be publicly released.Currently, the existing map review mechanism in China is required.Mainly using manual offline drawing review method within a certain time period.Faced with the wave of unmanned driving technology development, there is an urgent need for high-precision real-time map updates.The existing drawing review mechanism is difficult to meet practical needs.At the technical level, breakthroughs and optimizations in rapid drawing review technology are needed.At the management level, it is necessary to effectively classify sensitive information.For incremental update data, adopt an independent audit method to accelerate the review process and compress the review time.Improve efficiency while ensuring safety.

Challenges faced by high-precision map crowdsource updates
At present, the high-precision map update technology for autonomous driving, mainly based on multisource visual data, is still in the exploratory stage.Although autonomous driving maps have made some progress, there are still many challenges in the theory and key technologies of crowdsource updating, as follows: (1) The map model of autonomous driving is not uniform.At present, there are autonomous driving map models in terms of design, expression, interface, etc., but none of them are unified.It is necessary to closely combine the actual needs of autonomous driving, especially in the face of complex road traffic scenarios, to establish a unified autonomous driving map model that supports all elements, flexible expansion, and real-time updates [35].
(2) High precision positioning at the end of the vehicle is currently a challenge.Especially in the case of satellite signal loss at the vehicle end, there are still significant challenges in achieving stable and reliable high-precision positioning in various complex traffic scenarios.
(3) High precision 3D reconstruction based on multi-source visual data has limited accuracy in mapping based on visual data.And there is a lack of effective scale information [36].How to ensure the high accuracy and reliability of map feature reconstruction based on visual data still needs to be solved and broken through.
(4) The problem of consistency fusion of multi vehicle and multi source data.The visual data obtained from different vehicle terminals has complex characteristics such as multi angles and multiple spatiotemporal scales, and the vehicle terminal data itself has various uncertainties such as false detections, missed detections, positioning errors, and matching errors.As a result, the effective fusion of multi vehicle data has become a challenge.
(5) The security of map data is involved in crowd source updates.During the process of uploading and publishing vehicle data, it is crucial to quickly encrypt the data and ensure the security of geographic information processing.At present, the traditional offline processing method of navigating electronic map data with biased encryption is difficult to ensure the timeliness of map data updates.
(6) A breakthrough in the rapid review mechanism for map incremental update data.The geographic information data involved in high-precision autonomous driving maps is closely related to national security.Require strict map review before publishing map data to ensure national geographic information security.However, the existing graph review mechanisms currently use offline processing, making it difficult to cope with real-time updates of incremental data review.
(7) Standard specifications and laws and regulations.At present, there is a lack of standard specifications related to key technologies involved in high-precision map crowdsource updates.

Conclusion
The high-precision map of autonomous driving is a necessary foundation and key support for the implementation of unmanned driving.Currently, it has become a focus of international attention.However, the difficulty of achieving real-time dynamic updates on high-precision maps seriously affects the safety and reliability of unmanned driving.It has become an important technical bottleneck that restricts the implementation of autonomous driving.At present, the intelligence level of ordinary massproduced vehicles is constantly improving, and the popularity of in-car cameras is increasing.However, mass-produced car-mounted cameras are more commonly used in advanced driving assistance systems, and their data also provides a potential data foundation for map source updates.
Although multi-source high-precision map updates based on monocular vision are still in the exploratory stage and face many challenges, rapidly developing artificial intelligence and blockchain technologies provide theoretical and technical support for breakthroughs in key technologies such as map change detection, pose estimation, and 3D reconstruction based on multi-source data.Thus, map element updates based on mass-production vehicles have good development potential.However, the critical technologies for high-precision map crowdsource updates face enormous challenges.

Figure 1 .
Figure 1.Infrastructure for Crowdsourced Updates of Traffic Signs

Figure 2 .
Figure 2. Framework for Mapping of Multi-vehicle Data Feature Layer

Figure 3 .
Figure 3. Architecture for HD Map Update