Assistive control of brain-control multiple robots

Brain-controlled robots have important application values in assisting persons with disabilities. However, few studies focus on brain-controlled multiple robots. This paper proposes an assistive controller based on model prediction with the leader-follower formation strategy for developing brain-controlled multiple robots. The controller is composed of a leader controller and a follower controller. The experimental results of brain-controlled multiple robots show that the proposed method can track the user’s lateral and longitudinal control intention and maintain the stability of the multi-robot formation while ensuring the safety of brain-controlled multiple robots. This work can advance the research and development of brain-controlled multiple robots.


Introduction
With the continuous advancement of science and technology and social welfare, people's quality of life has been improved. However, population aging has become increasingly severe. Furthermore, the number of patients with motor dysfunction (multiple sclerosis (MS), stroke, amyotrophic lateral sclerosis (ALS), and other diseases) increases. Because these people cannot carry out the necessary daily activities through normal neuromuscular tissues (hands and feet), they often need care from other people, which significantly burdens the family and society.
Brain-computer interfaces (BCIs) were proposed to solve the above problems. It is a system that provides a direct communication channel between the human brain and physical devices by decoding users' brain activity from neurophysiological signals into appropriate commands [1]. Brain-controlled robots based on a BCI have a good prospect in assisting patients with limb defects. Brain-controlled robots were first proposed by Millan in 2004 [2]. Since then, more and more researchers have invested in this field. At present, many researchers combine brain control with autonomous robot control and shared control strategies. Iturrate et al. [3] developed a P300 BCI to first select a target, and then the autonomous navigation system controls the robotic wheelchair to reach the selected destination. Deng et al. [4] proposed the Brain State Assessment Network (BSE-NET), which can adaptively balance the control weight between human users and robot autonomy. Bi et al. [5][6][7] designed different control strategies to ensure the safety of brain-controlled robots.
However, most of the current studies on brain-controlled robots are focused on single robots. With the development of computer technology and wireless communication technology, multi-robot collaboration has become possible, and more and more applications have been obtained. Multi-robot systems can realize collaborative work, complete complex tasks, and greatly improve work efficiency. In a dynamic environment, the multi-robot formation can better shorten the time to perform tasks than a single robot, reduce the cost of the system, and improve the efficiency of the system [8].In daily life, the collaborative cooperation of multiple robots can better assist patients with limb dysfunction. For example, multiple robots can assist patients in taking medicines and transporting materials. In addition, human-robot cooperation has become a research hotspot of human-in-the-loop cooperative control, combining the advantages of humans and robots can accomplish more complex tasks [9,10].
Therefore, given the existing problems and deficiencies of the existing brain-controlled robots, we expand the research object from the brain-controlled single robot to the brain-controlled multiple robots and study how to control the linear and angular velocity of multi-robot through the EEG signals. In this paper, we propose an assistive controller based on model prediction for multi-robot with a leader-follower formation strategy. The proposed controller is composed of a leader controller and a follower controller. The leader controller can track the control commands output by the user (including accelerating, decelerating, turning left, turning right) under safe conditions, and the follower controller can track the preset target points under the condition of ensuring safety to maintain the stability of multi-robot formation. Figure 1 is a block diagram of a brain-controlled multi-robot system. The system can assist the users to safely control multiple robots to reach the desired destination through a BCI. First, the user makes a specific decision based on the status of the robot and environmental information, and generates corresponding control commands through the BCI. Then, the interface model quantifies the human control commands and inputs them to the assistive controller. The proposed leader controller judges whether it is in a safe condition according to the environment information and the current state of the robots. If it is in a safe condition, it tracks the user's control intention; otherwise, it outputs a safe control command. The proposed follower controller can ensure that the follower robots track the preset target point (obtained from the preset relative position relationship of multiple robots in realtime) under safe conditions to achieve the stability of the multi-robot formation.

Multi-robot model
In this subsection, we introduce the establishment of a multi-robot model. The robot model used is a two-wheel differential mobile robot model. We assume that the mobile robot moves on a plane without slipping. That is, there is a pure rolling contact between the wheels and ground.
We define the state of the robot as: Therefore, in the multi-robot formation, we define: the state of the leader robot is: The state of the n-th follower robot is: , the control input is: Where, x and y is the position of the robot and is the direction of the robot in the coordinate system. v is the linear velocity of the robot, and is the angular velocity of the robot.
For the formation strategy of multiple robots, we use the leader-follower algorithm and the l - formation control method. The control goal of the method is to make the distance and relative rotation angle between the follower and leader reach the set value [11]. Therefore, for the n-th follower robot, its expected state relative to the leader robot can be described as:

MPC-based assistive controller
In this subsection, we introduce the proposed MPC-based assistive controller in detail. The assistive controller can be divided into a leader controller and a follower controller. For the leader controller, it can ensure that the leader robot tracks the user's lateral and longitudinal control commands under safe conditions. Therefore, we define the following control objectives: u is the user's brain-control commands, safe L D is the safety distance of the leader, L d is the distance between the leader and obstacles. Based on the control objectives, the cost function of the leader controller is designed as follows: where k represents the current instant, P H and C H represents the prediction horizons and control horizons, respectively. s T represents the sampling time.
 is the weight factor of penalizing control action and c 1 H L Q   is the weight matrix of control output change rate. The leader controller cost function (4) can ensure that the leader robot tracks the user's intention under safe conditions. Constraint (5) represents the prediction equation of the leader, which can predict the future state. Constraints (6) are used to ensure the safety of the leader. Constraints (7-9) are physical constraints. In addition, we set that CP HH  and assume a constant control signal for all CP H k H  to reduce the computational complexity.
For the follower controller, it can ensure that the follower robots track the preset target point Fnset P under safe conditions to achieve the stability of the multi-robot formation. Therefore, we define the following control objectives: lim 0 is the safety distance of the follower, F d is the distance between the follower and obstacles. Based on the control objectives, the cost function of the follower controller is designed as follows: ( ) where,  is the weight factor of penalizing control action and is the weight matrix of control output change rate. The follower controller cost function (12) can ensure that the follower tracks the target point to stabilize the formation under the condition of ensuring safety.

BCI and interface model
In this article, a SSVEP-based BCI was employed. For this experiment, we designed a stimulus interface containing 6 command icons and displayed them on the LCD monitor. The flashing frequency of command icons were preset to 8,9,10,11,12,13 Hz, which represented acceleration, deceleration, left turn, right turn, keep forwards and stop commands, respectively. When the user pays attention to the stimulation of one of the command icons, EEG signals with the same frequency are generated in the brain, which can reflect the user's intention. EEG signals were obtained from eight electrodes (POz, PO3, PO4, PO5, PO6, Oz, O1, and O2) with the reference electrode located on Cz. In addition, a filter bank CCA was used as a recognition model [12].
Since the output of the brain-computer interface system is the control command of the subject, and the control command is a qualitative command, it is necessary to quantify the control command of the subject through the interface model. The specific formula is as follows: Where and represent the linear velocity and angular velocity commands at time k, respectively.

Multi-robot experiment and results
In order to verify the feasibility and performance of the proposed controller, we performed a multirobot experiment. In this experiment, we constructed a simulation experiment test platform in ROS/Gazebo, as shown in figure 2. The two target points were marked A and B, respectively. The initial position of the multirobot is shown in the figure, where the leader is in the middle and the followers are on both sides. In addition, we randomly set obstacles in the scene.

Experimental procedure
One subject participated in the experiment. Before the experiment began, the subject got familiar with the experimental scenario, as shown in figure 3. The task of this experiment was to control the multirobot to navigate to the designated target point without collisions. Before the navigation task, we tested the BCI offline accuracy of the subject in advance. If the accuracy is good, we can carry out the navigation task. We carried out brain-controlled navigation tasks with the assitive controller and handcontrolled navigation tasks without the assistive controller, respectively. In each task, the subjects controlled the multi-robot to the target points A and B 5 times each. After each round of control, the subjects had a short rest. After the experiment, we computed task completion rate, collision times, and task completion time as metrics to evaluate the task performance. Experimental results showed that the performance of brain-controlled and hand-controlled multi-robot navigation tasks was basically equal in task completion rate and collision times. In terms of task completion time, the current BCI still had the problem of low command recognition accuracy, which led to the brain control task in the task completion time longer than the manual control. Figure 4 shows the trajectories of the brain-controlled and hand-controlled navigation tasks. The orange circle is the target point A (left) and B (right), the green circle represents the obstacles, and the blue curve represents the trajectories of the multi-robot navigating to the target point A., The black curve represents the trajectories of the multi-robot navigating to the target point B, and the gray dotted line represents the positional relationship between the multi-robots at the same time, which can reflect the multi-robot formation relationship. We saw that the brain-controlled navigation tasks with the