Image processing for automatic road inventory

Road image analysis is an important task for automatic road inventory. The determination geometric dimensions for the road and the identification road objects are subprocess of constructing a road digital image. In this article, two algorithms for solving different subtasks of automatic road image inventory are proposed. The first algorithm identifies road signs. A convolutional artificial neural network is used in this algorithm. The training set for the neural network is prepared. A computer experiment to determine the recognition effectiveness of road signs has been conducted. The second algorithm defines the edges of the pavement. The algorithm consists five stages. The edges of the road are modeled as straight lines. The result allows you to automatically determine the width of the road.


Introduction
Analyzing the parameters of roads from their images is an important task for both their inventory and the construction the GIS. Images of road sections contain many details that interfere with their analysis. The presence of cars, shadows, trees and other objects complicates the task of highlighting the road contours. Mobile mapping systems are used to replace traditional data collection methods. Obtaining geographic information from a mobile platform is one of the best methods for quickly and safely collecting data along roads and railways. Automatic road sign inventory can be used to improve road maintenance processes and reduce maintenance costs. These systems are also used for intelligent driving systems.
To date, most articles solve the problem of finding individual road elements. An integrated method of highlighting road contours based on a combination of segmentation and classification is proposed in the article [1]. Authors use a multi-resolution segmentation method. The division of objects into road and off-road is implemented using classification methods: decision trees, k-nearest neighbors and support vectors of the machine. The accuracy of the proposed method is 88%. Use linked component markings to retrieve road components. In article [2], the pixel counting method is used to determine the size of an object in an image. Errors in this method do not exceed 1.3%. The algorithm for detecting columnar objects is proposed in article [3]. The algorithm uses flat filtering and range determination. In the first step, points above the ground are highlighted. In the second step, pole objects are highlighted by plane filtering. The average efficiency of highlighting columnar objects is 87%. Automatic method for detection of hatch covers is proposed in article [4]. The cloud of mobile mapping points is taken as the basis. Convolutional neural network is used for data processing. The proposed approach ensures 95% accuracy of hatch detection. Article [5] presents a method of identifying roadside fences based on semantic segmentation of images. Images are filtered based on the appearance of objects. Article [6] proposes a hierarchical method of automatically identifying road objects from the clouds of mobile LiDAR points. Building elevations are extracted at the first stage. The clustering algorithm is used to 2 obtain a set of object candidates. Object shape recognition is based on skeletization and the principal component method. The efficiency of the proposed methods is 90%. Article [7] implements a road marking detection system based on a backbone tree. This algorithm is resistant to data noise.
Article [8] proposes an algorithm for determining and classifying vertical objects on a road image using voxels. Recognition of the form and formation of candidates for objects is carried out in the first stage. Vertical objects are detected and individualized in the second stage. The distribution of objects into four categories is carried out in the third stage. The efficiency of the method reaches 92%. Article [9] proposes an algorithm for automatically obtaining a geometric inventory of road sections (number of lanes, width of the carriageway, width of the lane) based on mobile laser scanning of the system. The authors use a methodology based on segmentation and classification. Article [10] combines automatic detection, classification of road signs and inventory of road signs. The problem of analyzing the road signs state is also discussed in this article. This problem is solved by machine vision methods. Article [11] presents the structure of the machine vision system for identifying road signs. Article [12] presents a system of automatic detection and classification of road signs based on a cloud of laser scanning points and road images. Signs are found in the point cloud. Road sign inventory is based on geometric and contextual data. RGB images are used as an addition to the point cloud. The efficiency of recognition and classification of road signs is 90%. Article [13] proposes an algorithm for extracting data on columnar objects based on the results of radar scanning. The algorithm consists of three stages: data processing, segmentation and detection. The data processing step deletes the building elevation information. The detection efficiency of columnar objects is 90%. In article [14], the algorithm for automatic detection the "zebra" transitions based on mobile LiDAR data was developed and tested. There are several processes in this algorithm. The first process segments the road image. The Haf transform and Boolean constraints are then applied to the point cloud. To optimize results, the point cloud uses binarization and median filtering. The efficiency of the proposed algorithm is 83%. In article [15], deep learning techniques are used to detect small urban elements in an image. The images are obtained in a mobile mapping system. A simple convolutional neural network with feature extraction is used.
The purpose of this article is to develop algorithms for highlighting the boundaries of the road in the image and recognizing road signs.

Road signs recognition
We solve two problems for the inventory of the road by image: determining the geometric size of objects and identifying road signs. This section describes the algorithm for identifying road signs. Convolutional neural network is used to recognize road signs. The image of the sign is translated into grayscale. Image resolution is set to 100100 pixels ( Figure 1). This image is the input of an artificial neural network. The set includes 40 different signs. For each sign, 50 images were selected. The test set includes 100 different sign images. Road sign recognition efficiency with this approach is 63%. Efficiency can be improved by pretreating road signs. Efficiency can be improved by pretreating road signs. More accurate localization increases the area of the road sign in the image.

Highlighting road boundaries
In this section, an image preprocessing algorithm is proposed to determine the width of the road. The main purpose of this algorithm is to determine the road boundaries. We solve the problem of finding lines that limit the way in this work. The algorithm processes images obtained from a camera mounted on the vehicle. We formulate restrictions on such images. 1. The road is present in the image. 2. The road is located in the center of the image and makes up at least 10% of the image area. 3. The edge of the road can be approximated by a straight line.
To determine the image of the road, we use the algorithm for segmentation and highlighting contours proposed in the article [16]. We investigate the segmentation results for the original image and the image after the transformations. The results of highlighting contours without preliminary processing are shown in Figure 2. As shown in Figure 2, segmentation allows you to select the boundaries of the road and the wire lines above the road. After that, we perform image transformations. Segmentation is performed on the transformed image.
We're doing five image transformations. 1. The image is converted to grayscale format. (Figure 3). 2. The Sobel filter is applied to a grayscale image (Figure 4). 3. Watershed algorithm applies to an image ( Figure 5), 4. Threshold filtering is applied to the obtained image ( Figure 6). 5. Binarization is applied to the resulting image ( Figure 7).
As shown in Figures 2-7, image processing does not affect road contour and wire line selection along the road. Objects along the road also do not change significantly. The Sobel filter allows you to select wire lines across the road. Further processing of the image impairs the visibility of the transverse wires.

Conclusion
We suggest two algorithms for road inventory by image. The first algorithm defines road signs. The second algorithm defines the edges of the road. This information allows you to automatically calculate the width of the pavement.