Mobile robot position assessment in the room with the use of on-board vision system

The contemporary method of visual navigation of a mobile robot is presented. The visual navigation is an effective way to increase an accuracy of estimation for a current position of a mobile robot and its displacement. Using different specific features of a ceiling image along with photogrammetry theory permits to get a relative estimate for a current position of the mobile robot and its displacement in a room. Conventional approaches are ineffective because of presence of movable and immovable objects on the recognition scene. The ceiling image as the recognition scene is used in the proposed method. The recognition scene of ceiling is a stable and reliable image in the problem of finding reference points. Moreover there are no image distortions on the scene. All the mentioned above is a significant benefit of the proposed approach in terms of accuracy characteristics of the method of visual navigation of the mobile robot in the room.


Introduction
The navigation system of the mobile robot under consideration, is designed to assess the coordinates and orientation of the robot in the room [1,2]. In such works, an inertial measurement module, including accelerometers and angular velocity sensors, is usually used as the core of the navigation system. Correction of errors accumulated by the inertial system is carried out by additional, for example, ultrasonic, infrared, etc. systems [3,4].
This work is proposed to implement error correction using the robot technical vision system (STZ), using visual navigation methods [5,6]. In this case, the observed scenes are room ceiling images. The characteristic elements of the ceiling image (sensors, lines, angles, design elements, light sources) are used by the STZ as visual reference points. To obtain images, a camcorder is installed on the robot, the axis of sight of which is directed vertically upwards. The advantage of this approach in comparison with the horizontal location of the camera is the use of an observation area that is not blocked by extraneous non-stationary objects, including people.
It should be emphasized that in this work, visual navigation is used as an auxiliary correction subsystem and does not provide a continuous assessment of the measured coordinates and orientation of the mobile robot in the room.

Formulation of the problem
The ceiling navigation system allows you to determine the position and orientation of the mobile robot based on the position of the observation system assessment relatively to certain visual landmarks with known coordinates. Various unique features of the ceiling images are used as reference points. Let there be 2 images of the ceiling obtained by the mobile robot at 2 points of the trajectory of movement. In this case, some of the images capture the same section of the ceiling, the overlap of the two images must be at least 50% of the total image size. Since the optical axis of the video camera is directed vertically upwards, with a high probability it can be argued that there are geometric distortions between the obtained images in the form of linear displacement and rotation relatively to the optical center of the image. These distortions can be described using equations describing affine transformations [7]: where u 1 , v 1 are the coordinates of key points in the first image; u 2 , v 2 are coordinates of key points in the second image; c u , c v -parameters of mutual linear displacement between images; ω-rotation angle; k-scale factor. In our case, k = 1 is known, since we believe that the ceiling height within one room is constant.
The stationary coordinate system of the room OXY Z is associated with the starting point of the robot. The exact value of the coordinates and the orientation of the initial position of the robot is irrelevant. Of practical importance are the relative distances and orientation from the starting point to the current position of the robot. If the coordinates and orientation of the robot for the first image are known, then the parameters of the movement of the robot at the point where the 2nd image was obtained can be calculated using formula (1). To estimate the parameters of the relative movement of the robot, it is necessary to determine the unknown values: the parameters of the mutual linear displacement between the images c u , c v and ωrotation angle.
The key points used as visual reference points on the ceiling are those points of the image that stand out against the background, are unique in a certain neighborhood and are resistant to noise and distortion [8].
The solution of the system of equations (1) makes it possible to obtain a linear equation with respect to unknown parameters: The number of key points in the image depends on the information content of the image and contains from ten to hundreds of such points. To solve the overdetermined system of equations, the least squares method is widely used; in this case, this method allows to find some "optimal" vector of unknown parameters from the point of view of the maximum proximity of the vector of deviations to zero.
Solving equation (2) using the least squares method allows to get parameter estimates c u , c v in the pixel coordinate system of the image. For the transition from the pixel coordinate system to the three-dimensional coordinates of the observed scene, a perspective geometric model of a camcorder is used [9], which is determined by the following relations: where Z is the height of the ceiling relative to the mobile robot; f is the distance from the optical center to the image plane, which is determined at the stage of camcorder calibration.
3. Algorithm to assess a mobile robot position in a room via a computer vision system The proposed algorithm to assess a mobile robot position is based on the use of various unique features of the ceiling image, identified by the methods of identifying visual features that are used in computer vision systems. In particular, this approach can be implemented on the well-known correlation-extreme methods of visual navigation.
The algorithm to assess a mobile robot position can be divided into four stages: • stage of camcorder calibration; • stage of searching for key points on the image; • stage of comparison of key points in two images; • the stage of calculating the value of displacement and angle of rotation.

Calibrating the camcorder
The simplest from the point of view of material costs, namely the absence of the need for special equipment to install a camcorder and form the observed scene, is the method of calibrating a camcorder using a template. In this case, it is enough to take several photos of the template with known dimensions from different observation points. Usually, a ready-made template is used for these purposes, for example, in the form of a checkerboard. Key points are automatically selected on the resulting image, for which their relative position in space is known. As a result of the calibration, we obtain the matrix of the optical parameters of the video camera A, the radial distortion coefficients k 1 , k 2 , k 3 and the tangential distortion coefficients p 1 , p 2 . The matrix of the optical parameters of a camcorder is presented: where α x , α y are the focal lengths along the X and Y axes; u 0 , v 0 are coordinates of the optical center in the image plane.
Since distortion leads to distortions of the camera's field of view (field of view), this effect is not acceptable in optics designed for photogrammetric measurements.

Search for key points in the image
Today there is a large number of algorithms to find key points in the image (SURF, SIRT, BRIEF, ORB, etc.) [10][11][12]. The comparative analysis of key point detectors was carried out in terms of the quality of finding key points and the time of the detector operation. By "good" quality of finding special points we will consider such a detector, which is able to find special points both in areas with high contrast and in areas with low contrast. This need arises because on the observed scene, namely on the ceiling, there are artificial light sources that can be turned on or off at different times. When enabled, light sources become high-contrast objects. The detected key points on artificial light sources have a low positioning accuracy. Table 1  in brightness (gradient) and scale are calculated using the coefficients of the Hessian matrix. Secondly, this algorithm is optimal in terms of speed and efficient in terms of key points being detected.

Comparison of key points in two images
Since there are similar objects in the ceiling image, the matching algorithm sometimes does not correctly form a pair of key points. Figure 1 shows two sequential images, the dots show the location of key points that were found using the SURF detector. Lines connect pairs of corresponding key points. The image visually clearly shows that some pairs of key points were found incorrectly. To isolate incorrectly found pairs of key points from the sample, it is supposed to be used in the least squares method of element-wise removal of those pairs of key points that introduce the greatest error in the estimation of the model parameters (2). The idea of this approach is similar to the well-known RANSAC algorithm. The running time of the RANSAC method directly depends on the number of key points and the required accuracy to estimate the model parameters. This circumstance is essential for systems operating in real time. Before use the least squares method, the coordinates of the key points must be recalculated according to equation (5). Figures 2-4 show the result of removing incorrect pairs of key points from the initial sample.
The proposed approach allows you to correct the resulting matrix of matching key points, and, consequently, to increase the accuracy of finding unknown parameters c u , c v and ω.

Calculation of the offset and the angle of rotation
The accuracy of calculating the displacement and angle of rotation depends on many factors: the quality and characteristics of images, technological errors in the production of camcorders,  the accuracy of determining the location of key points, identification of key points, distance to the observed scene, as well as image processing methods. It is impossible to determine the impact of each source of errors separately. In addition, the nature of the action of each source varies depending on many factors. The action of an unknown source is also possible.
The formulas for transforming coordinates between the coordinate systems of the image and the object are derived provided that the image of the object in the image is obtained in an ideal central projection (3).
The algorithm was tested using a camcorder with an image size of 1920 × 1080. Figure 5 shows one of the results with the accuracy characteristics depending on the height of the room.
Thus, for an image with a size of 1920 × 1080, the error in measuring the displacement along the OX and OY axes is less than 8 cm, and the error in measuring the angle of rotation of the mobile robot is less than 0.8 degrees.

Conclusion
Theoretical and experimental studies were carried out to confirm the efficiency of the proposed approach. In the process of research, new, practically significant results were obtained. It is shown that the use of a ceiling with a tiled structure leads to the appearance of false pairs of key points, which makes it difficult to correctly assess the position of the robot. To eliminate such uncertainties, it is proposed to use a hierarchical construction of the search algorithm.
In this paper, a promising method of visual navigation of a mobile robot in rooms is considered. The uniqueness of the method lies in the use of the ceiling image as the observed scene. The use of ceiling navigation eliminates the effect of obstructing visual reference points by people and mobile objects, which increases the reliability and accuracy of the robot navigation system. The experiments carried out have confirmed the efficiency of the proposed solutions.