A Monocular Vision Relative Displacement Measurement Method Based on Bundle Adjustment Optimization and Quadratic Function Correction

In order to meet the requirements of high-precision target displacement measurement, this paper proposed a measurement method based on monocular vision measurement system by which the advantages of simple structure, convenient debugging, easy to install and calibrate is recognized. This method first uses the bundle adjustment algorithm with conditional constraints to optimize the solution results of the orthogonal iterative algorithm, then calculates the relative poses of the optimization results, finally establishes an error correction model to perform secondary optimization on the relative poses. The experimental results show that the method proposed in this paper can reduce more than 54% average error of displacement measurement results, and the error between the final displacement measurement results and the actual displacement values is controlled at 0.4mm level, which proves that the method has good practical value.


Introduction
In recent years, accurately measuring the pose parameters of the target is significance in many research fields [1]. With the advantages of non-contact and high accuracy, visual measurement has been widely used. By extracting the pixel coordinates and corresponding actual coordinates of the cooperative target on the measured target, the pose parameters between the target coordinate system and the camera coordinate system can be solved.
How to measure the pose parameters more accurately has been the focus of scholars. The classic direct linear transformation (DLT) algorithm proposed by Abdel-Aziz and Karara [2] regards the nine variables of the projection matrix as independent variables that establish linear constraints. However, because the orthogonal constraints of the rotation matrix are ignored, the DLT algorithm requires a large number of control points to obtain an accurate solution. Přibyl [3] proposed the DLT-Combined-Lines algorithm based on the DLT algorithm, which uses Perspective-n-Line (PnL) linear formula to solve a large number of lines, but it is not accurate enough to use in engineering. Lepetit [4] proposed a high-efficiency and high-precision linear algorithm called EPnP, which uses 4 virtual control points to represent points in object coordinates. However, the algorithm may get bad results when the control points are coplanar and the depth changing dramatically. Chen [5] proposed a measurement method using three marker points, but the three-point method does not have enough points to optimize, resulting in large error and susceptibility to interference. In order to avoid the influence of interference, Oberkampf [6], [7], [8] proposed a POSIT algorithm, which iteratively estimates the perspective projection model by using the initial value obtained from the scaled model.  Placing the camera and adjusting the field of view, using Zhang's [12] calibration method to get the camera's internal parameters. As shown in figure 2, an object coordinate system is established on the cooperative target. The actual coordinates of each point can be calculated from the actual distance between the points.
Establishing the projection matrix (1) between the object coordinates and the corresponding pixel coordinates. The model (2) for minimizing collinear errors of objects in object space based on the collinear projection relationship between object space points and corresponding image points in object space. Formula (2) can be regarded as a quadratic function about the translation matrix T. When the optimal rotation matrix R can be given, the optimal solution of T with R can be obtained by using partial derivatives.
( ) ( ) After the new rotation matrix R and the translation matrix T are obtained, the new projection point It can be seen from the above formulas that the problem becomes the absolute orientation problem. The SVD singular value decomposition method can be used to obtain the new rotation matrix R. Until the error is less than the set value or the number of iteration reaches the set value, the obtained 1 n R + and 1 n T + are considered as the optimal rotation and translation matrix.

The Bundle Adjustment Algorithm with Conditional Constraints
In order to obtain more accurate pose parameters, this paper uses the bundle adjustment algorithm with constraints for the first optimization.
The external parameters of the camera and the object coordinates of the points are both regarded as observation values. And establishing a formula with the constraints such as the distance between the 4 control points and the object coordinates of all points should be in the same plane. The constraint formula and the bundle adjustment formula are combined to obtain formula (5).

The Establishment of Bundle Adjustment Error Formula
According to the imaging rule of the camera, the calculation formula (6) of the pixel coordinates involves the distortion correction formula and the collinear formula. Combining these formulas , using Taylor expansion and take the first order term.
The obtained formula is shown in formula (7). Formula (8) can be obtained from observation value + observation correction value = approximate value + approximate correction value. ISAIC Combining formulas (5) (6) (7) (8), the matrix form of each parameter in error formula is obtained: In the above formulas, i = 1, 2, , n refers to the number of object points, and j = 1, 2, , m refers to the number of photos.

The Establishment of Constraint Formula
Calculating the distance between two points by formula (13).
And all points should be in the same Z plane, so the plane constraint formula (14) is established.
ij ww In formulas (13) (14), i and j refer to the serial numbers of two points, and S is the actual distance between the two points. Combining (5) (13) and (14), the matrix form of the constraint formulas (15) is established:

Correcting Systematic Error of Displacement Measurements
Formula (16) represents the transformation relationship between the camera coordinate system, the starting object coordinate system S P , and the target position object coordinate system E P .
Both S Q and E Q belong to the same camera coordinate system during the measurement process, so the camera coordinate system can be used as a reference to establish the relationship between the target position object coordinate system and the starting position object coordinate system.
( ) Because of the limitations of a single camera, there will be a systematic error in measuring relative displacement. The error is caused by the magnification of the uncorrected distortion due to the large measurement field of view, and the insensitivity of the single camera to depth direction information.
As shown in figure 3, when the target is moving in non-depth of field, the errors in the X and Y directions are linear, while the errors in the Z direction is quadratic. In order to facilitate the establishment of the modified model, a quadratic function model is used.
As shown in figure 4, selecting some positions which the relative distances is known, as reference positions. Establishing an optimization model based on the pose parameters, the calculated relative displacements and the real values of these reference positions. Then inputting the pose parameters of other positions in the model, the optimized relative displacement can be obtained.  c c The calculated value is modified by the model to make it closer to the true relative displacement.

Experiment and Analysis
In this paper, the experiment used Nikon D7200 SLR camera photos, the lens model of the camera is AF-S DX NIKKOR 18-140mm f/3.5-5.6G EDVR, and the focal length f is fixed at 35mm when taking photos. After placing and calibrating the camera, orthogonal iteration algorithm (OI), orthogonal iteration + bundle adjustment algorithm with constraints (OI+BA), orthogonal iteration + bundle adjustment algorithm with constraints + systematic error correction algorithm (OI+BA+EC) are used to solve, and analyzing the results. Three experiments were conducted, in each experiment, the moving distance along the X-axis of the model was 1000mm.
As shown in figure 5, the reference positions are taken every 200mm to establish the model. The model is placed on a high-precision captive ballistic system (CTS) to ensure the accuracy of each movement.   As shown in figure 6 and table 1, compared with the original algorithm, after using the bundle adjustment optimization, the standard deviation of the X direction error is reduced from 0.349 to 0.324, while the standard deviation of the Y direction error is reduced from 0.312 to 0.287, and the standard deviation of the Z direction error is reduced from 0.852 to 0.507. After adding the error correction method, the standard deviation of the X direction error is further reduced to 0.162, the standard deviation of the Y direction error is further reduced to 0.0398, and the standard deviation of the Z direction error is further reduced to 0.0705.