Multi-View consistency-based point cloud registration method with low overlap rate

Accurate and efficient workpiece measurement is crucial for workpiece processing and quality monitoring. Non-contact optical measurement methods have gained more attention due to their simplicity, efficiency, and flexibility compared to complicated and inefficient contact measurement methods. Multi-view registration of measurement data is a key issue in workpiece measurement, as it relies on the system’s geometric accuracy and motion stability, presenting challenges such as the insufficient overlap of multi-viewpoint cloud data and cumulative error. To address these challenges, this paper proposes a multi-view planning and registration algorithm with a low overlap rate. The multi-view planning algorithm employs a greedy method to plan the scanning viewpoints of the workpiece to obtain complete point cloud data efficiently. The multi-view registration algorithm extracts features using a multi-scale geometric feature extraction network, matches the features based on the Hungarian algorithm, builds a graph, and optimizes workpiece positions based on the G2O algorithm for multi-view registration, effectively reducing cumulative error. Measurement experiments on blade workpieces confirm the feasibility of the proposed algorithms.


Introduction
Turbine blades, screws, and other complex peripheral components are typically manufactured by robots to achieve a high level of precision and accuracy in their three-dimensional morphology, including dimensions and angles.As a result, precise 3D topography measurement during machining plays a critical role in improving the accuracy of the finished parts.
Currently, contact-based probe measurements are widely used in the design, manufacturing, and quality inspection of casting parts for their high accuracy.However, these methods have low efficiency and limited online feedback.Optical-based surface measurements, on the other hand, are gaining attention for their simplicity, flexibility, and semi-automation.Line laser measurement is known for its high accuracy and efficiency, but it has limitations in acquiring complete point cloud data in one scan due to occluded reflections and field-of-view restrictions [1] .Therefore, it is necessary to obtain a complete and highly accurate model by measuring from multiple viewpoints and aligning multiple point clouds [2] .
Aligning point cloud data from many coordinate systems into a single one is the goal of point cloud registration.Two categories of registration are two-view and multi-view, with the former aligning twopoint cloud groups with overlapping areas.Identification of the source and target point clouds and their features are extracted.Next, using the features that were extracted, corresponding points are found.Lastly, the source point cloud is converted to the target point cloud's coordinate system and the rotational translation parameter is estimated.
Multi-viewpoint cloud registration is more complex and more important than two-viewpoint cloud registration due to the larger number of parameters to be solved and cumulative errors.In practical engineering applications, a common strategy is to first perform a coarse registration of the point cloud from multiple views to pre-align the point clouds of all views, and then a fine registration of the point cloud from multiple views to eliminate accumulated errors.Coarse registration accuracy depends on the accuracy of the calibration method used to obtain the initial positional relationship of the point cloud.If the initial position relationship cannot be obtained, the two-point clouds must be aligned to obtain the initial transformation.However, multi-view coarse registration algorithms based on spanning tree or shape generation are still in essence two-point cloud registration, which generates the cumulative error.Multi-viewpoint cloud detection involves aligning multiple point clouds of the same scene from different viewing angles.Coarse registration is the first step in this process, which involves obtaining an initial set of point cloud transformation matrices to be used as input for the fine registration stage.Fine registration aims to eliminate all transformation and cumulative errors, to achieve high accuracy in multi-view point cloud registration.In recent years, more attention has been paid to this problem, which has led to the development of various solutions for the fine registration of multiple viewpoint clouds.In general, the goal of multi-point point cloud registration is to obtain an accurate alignment between multiple point clouds, and both coarse and fine registration play an important role in achieving this goal.
Chen [3] proposes a direct extension of the two-by-two ICP algorithm to consider the multi-view point cloud fine registration as two-by-two point cloud ICP fine registration; through the two point clouds continuously using the strategy of ICP fine registration and merging the point cloud data, the data of two point clouds are sequentially aligned and merged until all the data of the point cloud are merged into a single point cloud, which is simple and feasible, but with the merging of the point cloud data and the ICP fine registration algorithm.The algorithm is simple and feasible, but as the point cloud data are merged, the search time of the ICP fine-registration algorithm will increase, and there is also the problem of the accumulation of registration errors.Bergevin [4] first proposed an algorithm for fine registration of multiviewpoint clouds.The algorithm first establishes connections between all point clouds through a star network, and the center point cloud of the star network and the other point clouds search for point correspondences with each other; then the rigid transformation between point cloud data is estimated by the ICP algorithm; eventually, we use the minimum registration error as the objective function to optimize the hard transformation parameters for each point cloud.This algorithm considers all the point cloud data to optimize the registration parameters, which solves the problem of error accumulation to a certain extent.However, this algorithm is time-consuming due to the need to establish several point correspondences for the focal point cloud, and each focal point cloud must be aligned with all other point cloud data two by two.Han [5] proposed an improved RANSAC algorithm based on 3D domain covariance descriptors to improve alignment accuracy.Guo [6] proposed a practical method for coarse point cloud detection based on image feature points to reduce the number of iterations for fine-tuning.Zhang [7] utilizes an enhanced ISS key point extraction algorithm to address the issue of the point cloud's initial high position, and the low matching rate caused by the classical 3D Match's random sampling strategy that results in inconspicuous features.Toldo [8] proposes the use of a generalized Pluck analysis, which takes into account the simultaneous recording of all the point cloud data included in the ICP algorithm.Guo [9] embeds the Levenberg-Manelter (LM) algorithm into the ICP algorithm.The algorithm requires the creation of a minimum spanning tree for all point clouds along the shortest weighted path, globally aligned to the global reference coordinate system using the LM-ICP algorithm.Zhu [10] first estimates the overlap percentage of all pairs of point clouds in multi-view registration; then proposes a pruned scale iterative nearest point algorithm (TsICP) to compute the relative motions of two pairs of point clouds containing a high overlap rate, which can compute an accurate scale transformation; and finally the result of fine registration of multi-image point clouds is obtained using the MA algorithm.Gagnon [11] treats the multi-viewpoint cloud network as a whole and minimizes any view registration errors; the inter-point cloud transformations in the graph are computed by a series of matrix multiplications; if there are registration errors in the inter-point cloud transformation matrix, the matrix multiplication computation produces the cumulative errors in registration; it is crucial to obtain the network topology with minimum path lengths between the nodes; the algorithm proposes to improve a set of point cloud view registration transformations by a generic.The algorithm proposes a general method to improve the registration transformation of a set of point cloud views to minimize the cumulative error of registration among all views.Pulli [12] proposes to utilize the point cloud two-by-two registration results as constraints.Uniform elimination of two-by-two-point cloud registration errors in multi-view registration by successively estimating the stiff transformation of neighboring point clouds; spreading pairwise registration errors across multiple viewpoint clouds; combining neighboring views into a single point cloud when the registration error is small enough until all point clouds data are combined; and spreading registration errors uniformly across cyclic view pairs.Ding [13] proposes a deep mapping registration framework that uses a deep neural network to align multiple point clouds to a globally consistent coordinate system; two networks are used to accomplish global point cloud registration, one is used to estimate position, and the other is used to model scene structure by estimating the state of charge in global coordinates, transforming the registration problem into a binary occupancy classification problem, and uses gradient-based optimization to efficiently solve the fine registration of the point cloud from multiple viewpoints.
The scanning process requires the workpiece to be scanned from various angles to obtain complete point cloud data.However, to achieve higher scanning efficiency, it is practical to minimize the number of scanning angles.Additionally, multi-block point cloud registration can result in cumulative errors which can be reduced by minimizing the number of blocks to be aligned.Therefore, it is crucial to plan the scanning angle carefully.Our proposed algorithm based on the greedy idea optimizes scanning efficiency while reducing cumulative error by planning the viewpoints.The multi-view registration algorithm includes extracting features using a multi-scale geometric feature extraction network, performing feature matching based on the Hungarian algorithm, constructing graphs using workpiece positions as vertices and vertex connecting lines as edges, and optimizing workpiece positions based on the G2O algorithm for multi-view registration.This approach mitigates cumulative error caused by pairwise registration of multi-block point clouds.

Multi-Perspective Planning
To optimize the scanning efficiency and reduce the cumulative error of point cloud registration, it is important to carefully plan the scanning viewpoints.Our proposed algorithm uses a greedy approach to optimize scanning efficiency while minimizing the number of scanning angles.This is achieved through a multi-view registration process that incorporates feature extraction, feature matching, graph construction, and optimization using the G2O algorithm.As shown in Figure 1, by carefully planning the viewpoint for each cross-sectional profile curve, we can significantly reduce the number of scanning angles required for complete point cloud data acquisition.This approach helps to mitigate cumulative errors caused by pairwise registration of multi-block point clouds.

2.1Construct the set of viewable points:
As shown in Figure 2, the rotation of the workpiece for scanning is transformed into scanning with multiple cameras placed around the workpiece axis in a 360-degree range.For the workpiece crosssection contour line C, here to define the concept of a visual point: for a point P1 on the workpiece crosssection contour line C, if the point P1 and the line scanning laser camera L1 line of straight-line segment PL1 and the workpiece cross-section contour line C in addition to the point P1 no other point of intersection, then the angle of the line scanning laser camera L1 is the point P1 of a visual angle, the point P1 is the camera L1 of a visual point.The solution can be obtained by solving the set of all visible points PL1 of the camera line scanning laser camera L1 for S.

2.2Visual angle planning:
To optimize and streamline the scanning process, we propose the following steps: (1) Firstly, count and sort the number of viewable points under each camera angle.Select the camera angle  with the highest number of viewable points as the first scanning angle.
(2) Remove the viewable points under camera angle  from the point set.
(3) Count the number of viewable points under each remaining camera angle.Select the camera angle  with the highest number of viewable points.
(4) Remove the viewable points under camera angle  from the point set.
(5) Repeat Steps 3 and 4 until there are no points left in the set of viewable points.The set of camera angles obtained at this time is the scanning angle planned for the slice cross-section C1.
(6) Perform Steps 1 to 5 for each slice contour CK to obtain the camera angle set  for each slice contour.
(7) Concatenate all camera angle sets to obtain the final set of scanning angles.By following this approach, we can efficiently determine the optimal camera angles for scanning each slice cross-section, minimizing the number of scanning angles required while ensuring comprehensive coverage of the workpiece.

Multi-View Registration
In this paper, a multi-scale geometric feature extraction network is proposed for feature extraction, and at the same time the problem of registering a point cloud from multiple views is transformed into a graph optimization problem, where the vertices are defined as the initial positional attitude of each piece of point cloud output from the coarse registration of the multi-viewpoint cloud, and the vertices connecting the vertices are taken as the edges to build the graph, and the artifact position is nonlinearly optimized based on the G2O algorithm [14] to carry out the multi-view registration, and to alleviate the cumulative error caused by the two-by-two registration of the multi-viewpoint cloud.

3.1Multi-scale geometric feature extraction network
The network initially uses a multidimensional coding structure to focus on semantically rich regions at different levels, followed by a geometric coding module (GE) to consider geometric features, which are decoded by a decoder to output features at three scales.
 Multiscale coding and decoding structure The multiscale coding and decoding structure are divided into an encoder and a decoder, with the encoder being available for common use by all input point clouds.As shown in Figure 3, the encoder uses KPConvp [15] as the base unit.The neighborhood is expanded in a convolutional manner using 3stride KPConv blocks.Record the feature mappings (denoted as  ,  ,  ) before each stride KPConv block for the decoder to generate multi-scale features.This has the advantage of extending the neighborhood perception from  to  for each point feature.After the encoder has processed the input point cloud data, the output is the superimposed point  and its feature  ∈  × .
The decoder takes as input the recorded feature mappings ( ,  ,  ) as well as  and output high, medium and low-level features ( ,  ,  ) of the point cloud X.
Define the function φ *,*,* as follows: where (2) The geometric coding module and the cross-attention module are also needed to connect the encoder and decoder.
 Geometric Encoding Module (GE) As shown in Figure 4, the geometric encoding module (GE) takes as input the superposition point X' and its feature  ∈  × , and the output is a geometrically enhanced feature.

Figure 4. Geometric encoding module (GE).
The normal vector is a very important surface feature information in point cloud registration, which can help us to reduce the errors caused by noise or missing data, to more accurately describe and match the relationship between point clouds.As shown in Figure 4, to guarantee that normal vectors are represented geometrically correctly in the GE, normal vector smoothing techniques are applied.Specifically, first, the subsampled hyperpoints X ' are mapped back to the original X, and then the normal NX of X is computed using the classical API of Open 3D [16] .Finally, the normal vector of a point in X ' is averaged over the normal vectors of the points surrounding it in X, for the point   , the smooth normal vector  is computed as where  =  |  −  <  ,  ∈ ,  is the radius of the neighborhood.The geometric characteristic  of point  ∈  is calculated by the following formula [21] : where ∠( * , * ) ∈ [0, ] represents the angle between two vectors, implemented by PointNet [17] , where   is the radius of the neighborhood of  , and max( * ) denotes the maximum pooling.The feature F  inter is calculated as follows: where GE ( * , * ) is the GE module.The problem of point cloud registration can be described as follows: Given a source point cloud X = {xi R 3 } i=1,2, . .., N, and a target point cloud Y = {yj R 3 } j=1,2, . .., M, where N and M represent the number of points in X and Y respectively.The objective of point cloud registration is to determine the transformation T SE(3) that achieves the optimal alignment between X and Y.This can be formulated as follows: where σ is the correspondence set and | .| is the base.While the classical ICP [18] algorithm for finding the correct point correspondence between point clouds, after the initial coarse registration transformation, the algorithm calculates the distances from each point in the source point cloud to the target point cloud.It then identifies the point pair with the smallest distance as the corresponding point pair between the two-point clouds; ensures that there is a correspondence between the points in the source point cloud and those in the target point cloud, and at the same time constructs the residuals squared and the objective function; utilizing the least squares method, the algorithm minimizes the error function and performs iterative steps until the mean square error falls below the specified threshold.
The present work introduces an enhanced method for registering multi-view point clouds.It leverages the ICP algorithm and the feature extraction network to match corresponding features extracted from the point clouds.Additionally, as shown in Figure 5, it converts the registration problem into a problem of graph optimization, utilizing the Hungarian algorithm for feature matching.A graph is constructed with the workpiece positions as vertices and the connecting lines as edges.The position of the workpiece is then optimized using the G2O algorithm for non-linear optimization.This approach effectively addresses the cumulative error issue that arises from registering multiple blocks of point clouds in a two-by-two manner.

4.2Perspective planning experiment
The view angle planning experiment is carried out on the side of the workpiece shown in Figure 7. Firstly, the view angle planning is carried out on a single cut surface, and then the planned angles of all cut surfaces are totaled to get the final planning angle.Following the mechanical position initial transformation of the workpiece cross-section shown in Figure 8 and Table 1, the manual estimation suggests that there are 6 angles involved in scanning the point cloud, the scanning time is 198s, while after the view angle planning algorithm calculates that only 4 scanning angles can cover the entire surface of the workpiece, the scanning time is 132 s, the scanning efficiency is improved by 33.33%.

4.3Multi-view point cloud registration experiment
To enhance the registration speed, a voxel filter with a side length of 1.0f is initially employed to downsample the multi-block point cloud data after collection, and then multi-view registration is performed.The method used in the comparison experiment is to first align two and two-point clouds using the NDT [20] algorithm based on the idea of a spanning tree [19] , connect the successfully aligned point cloud pairs to build a spanning tree and then perform fine registration between two and two-point clouds according to ICP algorithm.The experimental visualization results are shown in Figure 9. Figure 9. (a) is the cad cross-section of the workpiece during machining, Figure 9.(b) is the cross-section of the workpiece between two-and two-point clouds using the NDT algorithm for registration, and the point cloud pairs that have been successfully aligned are connected to build a spanning tree, then the workpiece cross-section between the two-and two-point clouds is fine-aligned according to the ICP algorithm, and Figure 9. (c) depicts the optimized cross-section of the workpiece achieved through the proposed multi-view point cloud registration algorithm in this paper.The intermediate result image reveals that the cumulative error has led to noticeable discrepancies, after one week of registration, a cross is formed between the red and white point clouds, while from the effect image in Figure 9. (c), it can be seen that the global optimization is optimized to alleviate the cumulative error generated by the two-by-two viewpoint registration.The 3D registration result is shown in Figure 10.   2, the experimental evaluation in this paper utilizes the point cloud matching rate as the primary index.It involves the fusion of aligned point clouds into a complete point cloud, followed by down-sampling the fused point cloud to match the number of points in the CAD template.Subsequently, a match is established between the down-sampled point cloud and the CAD template by comparing the Euclidean distance between corresponding points.If the distance is below the threshold of 0.2, the point pairs are considered overlapping.The point cloud matching rate is then calculated as the ratio of the number of coincident points to the total number of points in the template point cloud.

Conclusion
This paper presents novel algorithms for multi-view planning and registration.The multi-view planning algorithm efficiently plans the scanning viewpoints for the workpiece by employing a greedy strategy.It aims to minimize the number of viewpoints while ensuring sufficient overlap between neighboring point clouds, thereby improving scanning efficiency and reducing the number of point cloud blocks for alignment.
The multi-view registration algorithm incorporates a multi-scale geometric feature extraction network to extract features from the point clouds.These features are then matched using the Hungarian algorithm.A graph is constructed, with workpiece positions as vertices and connecting lines as edges.The G2O algorithm is employed to perform non-linear optimization of the workpiece positions, enabling accurate multi-view registration and mitigating cumulative errors resulting from pairwise registration of multiplepoint cloud blocks.
Experimental measurements conducted on blade workpieces provide evidence of the viability and effectiveness of these algorithms.

Figure 1 .
Figure 1.Sectional view of a workpiece.

Figure 3 .
Figure 3. Multi-view point cloud feature extraction and registration method.

Figure 6 .
Figure 6.System diagram.Figure6depicts the system configuration, comprising an OPT line laser camera and a UR16e robot.The robot is responsible for transporting the workpiece while continuously sending an external trigger pulse signal to initiate scans by the line laser camera.This enables capturing point cloud data from the workpiece's surface.The workpiece of interest is shown in Figure7, and the measurement experiments primarily focus on its side.

Figure 7 .
Figure 7. Blade to be measured.

Figure 8 .
Figure 8.(a) cross-section in perspective plan case; (b) cross-section derived from manual experience.

Figure 9 .
Figure 9.The cross-section of (a) the template point cloud; (b) the NDT+ICP method; and (c) the multi-view registration method.

Figure 10 .
Figure 10.The final point cloud registration result.

Table 1 .
Perspective planning experimental results.

Table 2 .
Registration experimental results data.