Research on effects and influence by presenting information on priority order and oncoming vehicle to operators for teleoperation of multiple autonomous vehicles

With self-driving vehicles, it is possible to manage multiple vehicles from a remote location even if one observer does not have a driver in the driver’s seat. Therefore, demonstration experiments are being conducted in various places to remotely monitor two autonomous vehicles and operate them as needed. However, when one observer manages multiple vehicles, the amount of information that can be processed is limited. If we can assist with an appropriate amount of information, we may be able to manage more vehicles. In this study, we perform an experiment in which the priority and the type of assist information are changed and presented to the observer in the overtaking scene of a parked vehicle using a simulator. Focusing on the burden on the observer during remote management of multiple units, the purpose is to identify the information required for monitoring and reduce the burden from changes depending on the type of information to be assisted.


Introduction
In the current autonomous driving, it may not be possible to complete all driving actions on the vehicle side alone, and the observer may be requested to change the driving operation. At that time, it is necessary for a human to take over the driving after the request by some means, but it is being considered to manage the vehicle from a remote location without putting the driver of the monitoring personnel on the running vehicle. In this case, unlike the actual boarding, even one observer can manage multiple vehicles from a remote location.
Therefore, demonstration experiments are being conducted in various places where one observer remotely monitors two autonomous vehicles and operates them as needed (1). If there are about two vehicles, it is unlikely that an operation request will occur at the same time, and even if they do occur, it will be possible to grasp the environment in which the vehicle is placed to some extent because only two vehicles are watching. However, when one observer in a remote location manages multiple vehicles, the amount of information that can be processed is limited. As the number of vehicles to be managed increases, it is expected that the burden will increase, monitoring will be overlooked, and the risk of mistakes in understanding the situation around the vehicle will increase (2). A system that reduces the burden is required to prevent the risk from increasing due to the impact of the burden during monitoring. However, if one observer wants to manage more vehicles, it is necessary to quickly understand the situation of the vehicle requesting operation and take appropriate measures. Therefore, it is important to appropriately inform the observer of the situation around the vehicle by presenting support.
Regarding remote management and remote control of automobiles, documents (3) and (4) report from the aspects of compensation for the effects of delays and driving support. Documents (5) and (6) propose a driving support system based on the concept of overtaking judgment by humans and a driving support system that makes judgments based on the movement prediction of others. Document (7) evaluates the effects of camera viewing angle, resolution, and the presence or absence of sensory information on the driving behavior of a remote operator. As in these documents, many research results have been reported regarding remote monitoring and remote control. However, the burden on remote observers has not been evaluated from the aspect of providing support.
In this study, we examine support by presenting information when an operation request is made from an autonomous driving vehicle to a monitor who remotely manages multiple autonomous driving vehicles. Assuming a situation where a parked vehicle is avoided, which is not good at automatic driving and is considered to be a heavy burden on the driver, we will create support that simultaneously presents judgment support based on information obtained from the surrounding environment when an operation request is generated. The purpose of this study is to conduct an experiment in which the contents of the presented information are changed in a driving simulator, confirm the influence and effect of the type of presented information, and examine the information necessary for monitoring and the burden reduction method from the results.

Information to be presented and assist methods
In this study, we consider the burden reduction effect and effect of the assist display by taking the parking vehicle avoidance scene as an example. Support by three types of elements: operation request information, oncoming vehicle information, and priority judgment of operating vehicle in order to identify the elements that place a heavy burden on the observer for information that reduces the burden when requesting operations when managing multiple units at the same time Consider the combination of displays.

Operation request information(F1)
In the current autonomous driving, it may not be possible to complete all driving actions only on the vehicle side, and if it is determined that the vehicle side cannot handle it, the observer may be requested to change the driving operation. Under the condition of presenting the assist information of this research, the display of this operation request is always presented after the parked vehicle is detected.

Oncoming vehicle information (surrounding environment)(F2)
Under the conditions of this experiment, parking vehicle avoidance is set to avoid the oncoming lane side and take the form of overtaking, and assist is presented regarding the approach information of the oncoming vehicle. When considering overtaking a stopped vehicle, it is necessary to allow a margin between the distance from the front of the stopped vehicle to the end of overtaking and the distance to the oncoming vehicle at the time of closest approach, so it is within twice+40m the distance to the parked vehicle. When there is a vehicle in, the oncoming vehicle is detected, and when there is no vehicle, it is indicated that overtaking is possible.

Priority judgment information of operating vehicle(F3)
In this experiment, in order to promote smoother traffic, it is judged that the state should be operated first, and assist is given to emphasize the notification. In this study, the priority judgment information of operating vehicle is defined as priority operating information. The distance from the oncoming vehicle required from the start of avoidance of the parked vehicle to the end of overtaking was set to 60 m from the own vehicle with some margin. On top of that, if the distance between the other vehicle and the oncoming vehicle is 120m or more, it can be judged that overtaking is possible even if the vehicle is operated after overtaking from the closer vehicle. That is, in a situation where both vehicles can pass before the oncoming vehicle passes, if the distance to the oncoming vehicle on one side is 120m or more,

Compare with type of assist
In this experiment, we compare and examine five types of assist, assist A, assist B, assist C, and assist D, which combine the presence and absence of these three elements. Nothing is presented under no assist conditions. Assist A presents information (F1), Assist B presents information (F1) (F2), Assist C presents information (F1) (F3), and Assist D presents information (F1) (F2) (F3). Summarize the condition settings in Table 1.

Display contents of assist
The contents displayed by the assist are the assist information that combines the priority operating information (F3) with the three types of information: front vehicle detection, operation request (F1), and oncoming vehicle information (F2). An example of assist is shown in Figure 1. First, "preceding vehicle detection" is displayed when a parked vehicle approaches to check the operation status of the assist. If it is determined that the vehicle will not be avoided by automatic driving, an "operation request" will be notified. In addition, regarding the assist presentation method in the overtaking state, the assist in the oncoming vehicle detection state is displayed as "oncoming vehicle detection". If there are no oncoming vehicles within this range, the assist will be displayed as "overtaking possible". The display of the assist of the vehicle that is judged to be able to be operated first from the priority operating information is expressed in red.

Experiment
In order to confirm the effect of the presented information examined in the previous chapter, we will conduct an experiment using a driving simulator (hereinafter referred to as DS) assuming an observer who manages an automatically traveling vehicle by himself from a remote location. In order to conduct an experiment in a situation where the load is high for the observer, we assumed a situation in which two autonomous vehicles traveling in different places find and avoid two parked vehicles parked on the road at the same time. Since the autonomous vehicle automatically decelerates and stops in front of the parked vehicle, the collision is automatically avoided, but the overtaking of the parked vehicle is requested by the observer. In a situation where two vehicles request operations at almost the same time  Figure 2 shows the appearance of the DS used in the experiment. On a large 42.5inch display, the front image of two autonomous vehicles and the operation request and assist are displayed respectively. The front image of each vehicle is displayed on the upper surface of the display, and the assist content is displayed on the lower right surface of the display. There is a button on the steering wheel, which allows you to select the vehicle to operate and switch between automatic and manual operation.

Autonomous driving settings and manual switching
In the experiment, a situation occurs in which a self-driving vehicle avoids a parked vehicle that appears while driving at 30km/h. The own vehicle automatically decelerates from a position 22m in front of the parked vehicle to a stop at the speed of equation (1).

Let the velocity be v[km/h] and the time be t [s].
The subject can be manually replaced on the way by switching operation. Select the vehicle to be operated with the button on the steering wheel and switch to manual driving to operate. After overtaking, return to automatic driving by pressing the button again.

Experimental courses and scenarios
It will be carried out on a straight road with one lane on each side. There are parked vehicles on the road that need to be avoided manually 100m from the starting point. In this driving environment, there are only own vehicles, oncoming vehicles, and parked vehicles that the driver is driving, and there are no pedestrians or bicycles. Figure 3 shows a summary of scenarios for avoiding parked vehicles and the timing of presentation of assist. The distance between the front end of the own vehicle and the rear end of the parked vehicle is L, and the parking vehicle detection is at the position of L=60m. The position of L=22m was set as the deceleration start point and the presentation start timing of the operation request. If you do not switch to manual, your vehicle will stop automatically at around L=5m. The assist display disappears when the vehicle runs out of the oncoming lane or when L<0. The maximum range in which oncoming vehicles can be detected is 200m, and the state in which an oncoming vehicle is 2L+40m in front of the own vehicle is defined as the oncoming vehicle detection state.  In this experiment, five types of oncoming vehicle arrangements, sc1 to sc5, were set. An example is shown in Figure 4, and all condition settings are shown in Table 2. Red is own vehicle 1, blue is own vehicle 2, green is a parked vehicle, and yellow is an oncoming vehicle. In the following text, for convenience of explanation, the upper condition of each scenario is referred to as own vehicle 1 and the lower condition is referred to as own vehicle 2, but the display position without distinguishing between own vehicle 1 and 2 is shown in the experiment. It is set separately. For sc2, sc4, and sc5, the own vehicle 2 is operated first, and for sc1 and sc3, the priority operating information (F3) is included so that it is more likely to be completed when the own vehicle 1 is operated first. It is set to be displayed in red in the assist presentation.

Experimental method
The participants in the experiment were two men in their twenties who had an ordinary driver's license. After informing us that it was necessary to avoid parked vehicles before the experiment, we conducted an experiment with a simulator. We ask the participants to perform the operations on the simulator in advance until they become accustomed to the operation, and start the experiment when they are accustomed to it. In the experiment, avoid parked vehicles in front of each vehicle. At that time, the condition without assist is implemented for the first time. After that, experiments are performed with assist A, assist B, assist C, and assist D, respectively. Considering the influence of the order, the order of the four assist except without assist is different for each experiment participant. Set the time 60m in front of the parked vehicle to 0, define the time until overtaking is completed and switch to automatic driving as the time until overtaking is completed, and compare the results. In addition, a subjective questionnaire will be conducted at the end of each assist condition. The questionnaire will be conducted on four items: difficulty of overtaking, annoyance, timing of assist, and evaluation of assist. There are 11 levels from 0 to 10 for the items of assist evaluation. Other items were evaluated on a 7-point scale from -3 to 3. Regarding the difficulty level and annoyance, the first condition without assist was set to 0, and the remaining 2 items were evaluated without setting a standard. Figure 5 shows an example of experimental participant A whose vertical axis is the time to complete overtaking for each scenario. In sc1 of assist C and D and sc5 of assist D, the time to complete overtaking is reduced compared to no assist.  Figure 6 shows an example of the results of Experiment participant B. In sc1 of assist D and sc2 of assist B, D, and sc5 of assist D, the time is reduced as compared with no assist. In addition, the time of assist C increased significantly under conditions other than sc3, and the results were significantly different for assist C.  Figure 7 shows the results of a subjective questionnaire regarding the evaluation of the assist of participants A and B conducted at the end of each experiment. Assist D, which had a large decrease in time for both experimental participants A and B, was highly evaluated. On the other hand, Experiment Participant A rated Assist C second highest, while Experiment Participant B rated Assist C the lowest.

Consideration of experimental results
Regarding experiment participant A, assist A and B did not show a decrease in overtaking time, but assist C and D showed a decrease in time. In addition, assist C and D are highly evaluated based on the results of the subjective questionnaire. From these results, the relationship between the decrease in overtaking time and the subjective evaluation was found. In the experiment participant B, there was not much change in time between assist A and B. The time decreased only with assist D, and the time increased with assist C. From these results, it is considered that assist D has the greatest effect of time reduction. In addition, a comparison of assist B and C suggests that assist C has a greater effect on time, and thus the effect of the priority operating information is greater. Experiment participant A showed a decrease in time due to assist C, and was ranked second most highly. However, experiment participant B increased the time by assist C and evaluated it the lowest, so the results differed depending on the characteristics of experiment participants.

Conclusion
In this paper, we examined a method of presenting information to a remote observer when it is necessary to manually overtake two parked vehicles at the same time when monitoring an autonomous vehicle. Information was presented based on three elements: operation request information, oncoming vehicle information, and priority operating information. As a result of conducting an experiment using four types of assist created by combining these elements, the time required to complete overtaking with assist D was the shortest compared to the case without assist, and this information was obtained under this condition. The effectiveness of the presentation method was confirmed. In addition, the results of the experiment suggested that the priority operating information had a greater influence than the factor of oncoming vehicle information. However, since the results differed depending on the characteristics of the experimental participants, we will consider detailed analysis and increasing the number of people. Also, in this experiment, the result was set to operate almost certainly as expected, but there are issues such as confirming whether the same result can be obtained under other conditions.