Systems of the self-driving vehicle

The article describes the basic systems of a self-driving vehicle distinguishing it from a conventional car under the control of the driver. The requirements for such systems are formulated, a number of tasks to be solved for each system is described, the interaction of these systems as a part of the design of a self-driving vehicle is described. A classification of types of self-driving control systems is proposed.


Introduction
In modern designs of self-driving vehicles, in addition to the conventional automobile systems, the following systems can be specified: 1. Perception of the environment system (sensor network); 2. Complex positioning system of the vehicle; 3. Navigation system (local and global route planning); 4. Internal control systems of the vehicle. Further, each system is overviewed separately.

Perception of the environment system (sensor network)
Perception system of the environment can be divided into external and internal sensor network.

External sensor network
The purpose of the external sensor network is to use onboard measuring devices to recognize and classify objects of the road scene ( Figure 1) and to deliver information about them to the virtual road scene assembler. The virtual road scene is a combination of external traffic conditions, which include the following objects: -  Today, measurement and sensor equipment of an external sensor network usually consists of the following components ( Figure 2):  Optical and infrared sensors (video cameras, stereo cameras);  Radar;  Light sensors (LIDAR / LEDAR -3D laser and 2D scanners);  Ultrasonic sensors ("parktronic"). It is also possible to classify the external sensor network into a passive, non-radiant (for example, video cameras) and active (radar / lidar, video cameras with infrared illumination to build a depth map).
The main functional requirements for the external sensor network of a self-driving vehicle are:  determining the distance to objects and their coordinates on the road scene;  classification of road scene objects according to the list, presented above (the list may be enlarged);  determination of the velocity vector of a moving object and predicting its movement;  work in real time. Requirements on the reaction speed of the system are dictated, usually, by the maximum speed of the vehicle in the self-driving mode;  reliability of recognition and accuracy of putting objects on the virtual road scene with an assigned probability of incorrect recognition per kilometer of path and operating time;  work in "all-weather" conditions (day, night, rain, snow, fog, sun exposure and headlights of oncoming traffic, etc.);  integration into the overall self-test system of the self-driving vehicle (system health check). When recognizing the surrounding objects and environment using data from the video cameras, video processing algorithms are widely used (video streaming pre-processing -stabilization, color and brightness correction, etc.), 3D depth map reconstruction in the direction of the stereo camera view, algorithms for classifying and recognizing objects, tracking objects and prediction of their movement by extrapolation over several video frames. A depth map in the direction of the stereo camera view is then processed into a projection plane road scene using three-dimensional geometric transformations. Scanning data from 2D/3D lidars and radars allows to build a three-dimensional cloud of points around a self-driving vehicle and use more sophisticated object recognition and tracking algorithms than using pairs of 2D images in a stereo camera (Figure 3).

Figure 2.
On the left is a stereo video camera, in the center is an infrared 3D lidar, on the right is a car radar. Today, none of the above sensor systems provide the necessary reliability of recognition of the road scene and its objects. Therefore, when assembling the road scene, the integration of recognition data from all of this systems is used: compare the coordinates and dimensions of the obstacles; object speeds; different classification and recognition algorithms results. Then objects and their recognized states are assigned with recognition confidence factors and objects are placed on the virtual road scene. Decisionmaking algorithms in the top-level control system should take into account the factors of confidence for recognized objects while calculating the route for the self-driving vehicle (slow down, choose detour trajectories, assign security zones around poorly recognized objects with an unclear prediction of their movement, etc.).

Internal sensor network
The purpose of the internal sensor network is to track the state of the systems and aggregates of the vehicle. Information about the state of vehicle systems influences the route, speed planning and all behavior strategy of the self-driving vehicle. A simple example -fuel level or battery charge sensor determines the possibility of further movement of a self-driving vehicle without refueling / recharging. This data is also very useful to other purposes: vehicle strategic motion planner, optimization tasks of the vehicle movement, scheduled maintenance and preventive maintenance, etc.
Sensors of the car internal sensor network (Figure 4) today are quite developed and usually consist of the sensors integrated into the vehicle's aggregates (wheel speed sensors, steering angle sensor, engine speed sensor, electronic brake and gas sensors, cooling system temperature sensors, pressure in wheels, etc.).
Additional requirement for internal sensor network mounted on a self-driving vehicle is to provide remote access to its measurement results for the general self-diagnosis system and the top-level control system that could be located outside the vehicle. High-developed internal sensor network, integrated into a single on-board or cloud-based database, makes it possible to predict work condition of vehicle and its components, plan maintaining based on condition, to provide the high-level control system with alternative ways to distribute the power onboard and much more.

Vehicle positioning system
Global positioning satellite systems are widely used both as part of the onboard software and hardware of modern cars, as well as applications and GPS / GLONASS receivers on driver's smartphones (popular applications are TomTom, Garmin, etc.). Using a global positioning system on a self-driving vehicle is no different from using it in a conventional car and serves the following purposes:  determine the coordinates of the location of the vehicle in real time on a global map;  determine motion parameters: velocity vector and velocity value, acceleration;  support the use of external databases of maps and online event maps;  support of data exchange with external services;  issuing operative restrictions on the route to the system of global route planning (traffic is blocked, the road is closed for this type of vehicles, etc.). The main problem of using a global positioning system based on GPS / GLONASS / GALILEO satellites receiving signal is signal instability and low accuracy in determining coordinates (up to 30 meters).
For precise coordination on the map, Differential Global Positioning Systems (DGPS) having operation range 1 -2 km and on-board inertial navigation system (limited to time of use for dozens of minutes due to accumulation of errors) are often used.
Integration of on-board inertial positioning system and global satellite system, as well as the use of previously known reference objects on the map (for example, vehicle could be coordinated with respect to the road's edge or marking lanes), allows precise positioning of the self-driving vehicle with sufficient accuracy in predefined operating conditions: with clear visibility, markup, etc. The task of such positioning is called SLAM-task (Simultaneous Localization and Mapping -definition of own coordinates on the map and the simultaneous construction of this map around a self-driving vehicle).

Navigation system (local and global route planning)
The navigation systems of a self-driving vehicle can be divided into a global system of route planning and path working operations performing and a local system of trajectory planning.

Global route planning system
The main tasks of the global route planning system are:  Receipt and launch of the travel task from the dispatcher (operator) of the self-driving vehicles;  Construction and selection of the general route of the self-driving vehicle on the map / in the area of operation (for example, Garmin Navigator route plan in the city), including taking into account the available online information on the condition on the roads;  Planning and implementation of working operations along path (loading and unloading, sowing, combat missions, cleaning the territory, patrolling, etc.) and invoking the vehicle working equipment, installed on the self-driving chassis;  Reporting actions on the completion of the travel task, events that happened during the operation. It makes sense to equip the Global Intelligent Transport System with a system of vehicle global route planning and route working operations implementation. Global route planning system requirements may include:  Support online state of maps and routes: • on-board database of maps and routes; • updating the onboard database of routes from the cloud.  Update databases of maps and states of roads according to the results of recognition (data supply): • signs (permanent, temporary), traffic lights, pedestrian crossings, bus stops, etc.; • width of the carriageway, the number of lanes; • condition and damages of the road surface; • types of intersections and rules of their passage; • temporary and permanent obstacles (repairs, accidents, open hatches).

Local route planning system
Currently, the driver is planning the local path of the vehicle by acting on its controls, based on the received information about the road scene in a visual way, by ear (car's audible warnings), in a tactile way (power response on the steering wheel, pedals, etc.). In a self-driving vehicle, this task is solved by the vehicle high-level control system based on the road virtual scene and the planned global route.
Basic requirements for local route planning:  Working in real time. Requirements on the reaction speed of the system are dictated, usually, by the maximum speed of the vehicle in the self-driving mode;  Prediction of the road scene dynamics;  Building a local trajectory  Calculation of target states of a self-driving vehicle on a trajectory (speed, direction of movement, emergency stop, detour of an obstacle, state of installed working equipment and its operation needs on a local trajectory). Systems of local movement planning can be divided into movement in a deterministic environment and a non-deterministic environment:  highway (Traffic Laws are valid);  urban environment (Traffic Laws are valid);  quarry (Special Rules of Traffic are valid);  closed territory (Special Rules of Traffic are valid). Requirements for movement in a deterministic environment may be:  Compliance with the Rules of the Traffic using typical scenarios of behavior:  Speed limit

Vehicle internal systems control
The main purpose of the internal systems control in the self-driving vehicle units is to calculate and apply control actions in order to maintain the local trajectory of movement and the target states of the vehicle, including its installed working equipment. At the same time, a control system can collect, store information about events on the route, the history of the vehicle's systems work, and calculate, together with the local movement planner, the most optimal vehicle modes of operation (minimum fuel consumption / charge, minimum travel time, minimum wear of machine parts, etc.). Requirements for the internal control system of a self-driving vehicle may be:  Real-time health-check and state assessment of the vehicle systems;  Calculation of restrictions on the possibility of a vehicle moving ability to the global and local route planning system (fuel level, systems condition, available power, prediction of available traction force, etc.);  Control signals calculation for vehicle systems and their execution;  Diagnostics of vehicle units control systems, overall health check. Today, most of the vehicle systems have their own electronic control units (electronic engine management system, electronic brake system, active steering electronic system, electronic transmission control system, see figure 5). Therefore, another requirement for the top-level control system of the selfdriving vehicle is the integration of all electronic control units into a supervisory system, ensuring the transfer and processing of sensor readings into a seamless internal vehicle sensor network. The most appropriate solution in meeting the latter requirement is building the control system of the self-driving vehicle based on a complex real-time mathematical model of the vehicle movement. This model should capture and predict vehicle behavior in motion, provide necessary signals and information for its individual on-board control systems.

Conclusions
Today development of each of the above-described systems of a self-driving vehicle is a separate research area and, usually, these developments are carried out by different scientific and engineering teams. However, these systems, closely intersecting in the design of the car, are highly dependent on each other and producing requirements to each other, often contradictory. The complex mathematical model (digital twin) of a self-driving vehicle on the virtual road, which is usually developed in the product design stage and then "works" onboard the vehicle, combining all workflows, can solve the problem of the inconsistency of these requirements, as well as identify possible functional problems at the junction of different control systems of the self-driving vehicle.