Multi-UUV Task Allocation with Limited Communication Range Based on Hoplites Framework

Multiple Unmanned Underwater Vehicles(UUVs) can be widely used in marine science expedition, ocean development and ocean engineering. And in the Unmanned Underwater Vehicles cluster, the Multi-Robot Task Allocation with communication range limitations is an important issue. In the past, many works were researched based on perfectly good communication conditions. However, in actual situations, especially underwater environments, communication range between robots is often limited. So in this paper, in order to ensure the communication range between the UUVs as much as possible, based on the Hoplites framework, we first design the communication keep revenue so that the UUVs not only perform the task but also try to ensure the communication range between the UUVs. Second, when there are individual tasks that are particularly far away from all other tasks, a communication relay node needs to be added between the UUV and other UUVs. We regard the communication relay node as a task, which is realized by using the active coordination mechanism of the Hoplites framework. Through the simulation environment of multiple UUVs, we conduct experiments and analysis of the algorithm, and the results show tangible performance gain of our methods over previous methods.


Introduction
As humans continue to explore the ocean, the Unmanned Underwater Vehicle(UUV)has become increasingly irreplaceable. The Unmanned Underwater Vehicles can photograph and investigate the geographical environment, ecological environment and hydrological information in the ocean. According to the operator's prior instructions, it can reach places where other detection tools cannot reach and avoid personal injury and waste of resources due to unknown conditions [1] [2]. At the same time, the research on multiple Unmanned Underwater Vehicles has attracted increasing attention in order to be able to perform large-scale tasks and improve the efficiency of task execution [3]. Multiple Unmanned Underwater Vehicles can provide significant advantages for a variety of applications including marine sampling, mapping, surveillance, and communications.
For the multi-UUVs, the most widely studied problem at present is the Multi-Robot Task Allocation(MRTA) [4][5] [6]. The MRTA problem refers to assigning multiple tasks to multiple robots according to certain rules. The rules may be non-repetitive allocations, the minimum total time to perform all tasks or the least resources consumed by all robots [7]. Patrolling [8], formations [9] and cooperative transportation [10] all belong to the MRTA. In this paper, we study a task set T that , where m is the number of UUVs. We hope to create a mapping between T and U, and the UUVs can work together to complete all tasks. At the same time, a very important indicator is the limited communication range rc, Throughout the task allocation process, we must ensure that the distance between each UUV and other UUV's is less than or equal to rc, through direct communication or multi-hop communication.
As we know, the underwater environment is quite complicated. It is full of areas that we don't know, and the interference of water with the communication signal is very large. So for multi-UUVs performing missions underwater, maintaining communication with each other is crucial [11]. So in this paper, we study how to use a distributed market-based approach to solve MRTA problem for multi-UUVs with limited communication range.
In the field of MRTA, people have proposed some methods for solving Multi-Robot Task Allocation problem with limited communication range. Wei Hua Sheng et al. [12] studied the exploration problem of multi-robots in unknown environments and proposed a market-based method to try to ensure that the distance of communication among multiple robots is in the limited scope. Although this can give the robots a state of mutual aggregation to a certain extent, it cannot be completely guaranteed. Nguyen et al. [13] studied whether the communication between the leader robot and the base station was interrupted by sensing the RF signal quality. If the RF signal quality is less than a certain threshold, an attached robot is required to relay the radio link. Although this method can form a complete communication chain, this is not a distributed method. If the leader robot is broken, then the entire link is meaningless. Based on the existing methods and their deficiencies, we decided to study the problem of Multi-Robot Task Allocation with communication range limitations based on the Hoplites framework.
The Hoplites framework is a distributed framework based on market-based method [14]. Hoplites framework includes two mechanisms: passive coordination and active coordination. Passive coordination mechanism is used to handle relatively simple task distribution scenarios. It can generate solutions quickly but does not consider the optimality of the global solution. If the situation is more complex, active coordination is needed to obtain better solutions through virtual economic transactions between robots. In this paper, we study the problem of multi-robot task allocation with communication range limitation based on the Hoplites framework. First, by using the market-based approach, we not only consider the revenue of the task and the cost of executing the task, but also the revenue earned by maintaining communications. In other words, when the UUV chooses a task, it will consider the distance between the task and other UUVs and perform comprehensive calculations to obtain the profit gained from the communication maintenance. Finally, the task that has the most comprehensive profit in all tasks is taken as the task that one will perform. However, the limitation of revenue alone is not enough. When we consider that there is a task that is separate from other tasks, we have designed a communication coordination mechanism based on the active coordination mechanism of the Hoplites framework. If an UUV has to perform a task that is out of communication range with all other robots, it can actively cooperate and give high rewards to the UUV that is furthest away from it, and request it to perform tasks of the communication relay node. In this way, a communication subnetwork will be maintained between the UUVs, so that the robots will not lose contact with other robots during executing tasks. At the same time, information out of sync or UUV lost due to out of communication range is also avoided.

Proposed Method
In this paper, we propose two levels of algorithms for multiple UUV communication problems with limited range based on Hoplites Framework. Now we have a detailed exposition of the proposed method. Hoplites framework is a market-based framework that includes passive coodination mechanism and active coordination mechanism. The specific algorithm of the Hoplites framework is shown in Algorithm 1. It is the Hoplites framework of the program running on each UUV. Based on the Hoplites framework, we illustrate the improved algorithm separately.

Communication keep revenue
In order to solve the Multi-Robot Task Allocation problem of multi-UUVs with communication range limitations, and also keep the UUVs in a same communication network as much as possible, we first design a revenue-based communication maintenance mechanism. In other words, when using the Hoplites framework to select the highest-profit tasks, we not only consider the revenue that comes from the completion of the task and the cost of performing the tasks, we also calculate the distance between the UUV and the other UUVs as part of the net profit calculation. That is: = + -(1) Where is the net profit obtained from the execution of task i, is the revenue of the task itself, and is the revenue gained from communication maintenance, and is the cost of executing the task i, in this paper we consider it the distance that needed to perform the task. , , are the adjustment coefficients, which are used to adjust the proportion of , , in the net profit calculation. And + + + (2) In this formula, is a constant and it is the basic profit that communication maintains. is the adjustment coefficient, , is the descending order of the distances between the task i and the tasks currently being performed by other UUVs, and is the number of UUVs, rc is the maximum communication distance that the UUV can withstand. From this formula, we can see that depends mainly on the UUV closest to this task. This can ensure that when the UUV chooses a task, it prefers to perform tasks that are closer to other UUVs. And finally: P = max (3) The UUV will choose the task that with the highest total profit in all tasks to execute.

Communication relay task
Through the method described in 2.2, we can try to ensure that the UUVs stay in a certain communication range when performing underwater tasks. However, when the degree of dispersion of tasks is particularly large, or when there is a very long distance between a single task and all other tasks, there is no way to guarantee the limited communication distance. At this time, we need to rely on the Hoplites framework, virtualize a communication relay task, and through an active coordination mechanism, request an UUV to perform a communication relay task when an UUV needs to go out of communication with all other UUVs because of the execution of a task. Here we choose to request the UUV that is currently farthest from itself, as this will not destroy the previously complete communication network. It should be noted that although this will allow the requested UUV to walk a little longer, it is worth it for the UUV not to lose communication with the entire robot team. The specific algorithm is as Algorithm 2.

Algorithm 2: Communication relay task Choose the most profitable task T Distance = get_nearest_distance(T) if Distance > rc: num = get_farthest_uuv() active_coordination(T,num) waiting for answer() else: execute(T)
Here we base on the active coordination mechanism of the Hoplites framework. The essence of the Hoplites framework is a market-based approach. Through a market-based mechanism, purely distributed methods are used to perform the task of buying and selling, to gain the UUV to perform communication relay tasks.

Application scenarios
In this paper, our experiments were simulated using UUV Simulator [15]. UUV Simulator extends the open-source robot simulator Gazebo to underwater scenes. It can simulate multiple Unmanned Underwater Vehicles. UUV Simulator is based on the ROS, an open source operating system. It implements simulation and control through a series of plugins that simulate underwater pressure effects and hydrodynamic effects. It also simulates the thrusters, sensors, and external disturbances of Unmanned Underwater Vehicles. The UUV Simulator universal robot simulation platform reuses and extends to the underwater environment, greatly facilitating the simulation of underwater environments. At the same time, we can use RVIZ to visualize data in Gazebo for research and analysis. The Figure 1 is a simulation experiment scenario. In the simulation experiment, we study the patrolling tasks of the Multi-Robot Task Allocation problem. In the underwater environment, we set 10 location points. Ten UUVs need to patrol all task points and cannot repeat. In this process, the communication range between UUVs is limited by a certain distance. Therefore, we have to meet the limitation of the communication range between UUVs through the above-mentioned method and perform all tasks reasonably.

Experimental results
For the simulation scenarios applied in the previous section, we discuss and analyse the experimental results of the communications keep revenue the communication relay task.

Communication keep revenue
In this section, we examine how the addition of communications keep revenue affects the behaviour of team UUVs. We first let the =1, =0, =1 in formula (1), and the in formula (2) is equal to 0.8, the communication range rc is 10 meters, B= =100, e is the constant. The movements of the four UUVs are shown in Figures 2, 3, 4. From the figures we can see that since no communication revenue was added, the UUV 1 (the rightmost UUV) moves away from the other three UUVs at the beginning. And due to the follow-up tasks, UUV 1 is further away from the other three UUVs.   Then we let the =1, =1, =1 in the formula (1), and the rest of the parameters remain unchanged, and the experiment results are shown in Figures 5, 6, and 7. It can be seen from the figures that due to the addition of communication keep revenue, UUV 1 chooses to perform tasks that are closer to other UUVs from the very beginning, and finally, the four robots collectively perform more distant tasks to ensure the UUVs in a relatively close distance, and the communication network is maintained.

Communication relay task
In the experiment of the communication relay task, we set a situation where one task and all other tasks are particularly far apart. If an UUV performs this task, it will certainly lose contact with other UUVs where only the communication keep revenue is not enough. Figure 16 shows the initial positions of the four UUVs before executing tasks. And the result of the movements of the UUVs in the experiment are shown in Figure 17 and 18 respectively. From the figures, we can see that the last task performed by UUV 1(the rightmost) is beyond the range of communication. Therefore, UUV 1 triggered the active coordination mechanism and given a high revenue to UUV 3 from the farthest distance to perform communication relay task, UUV 3 received the request, weighed and agreed upon the request to perform communication relay task and restore the entire communication network. It should be noted that we did not consider communication keep revenue in the experiments of communication relay tasks.

Conclusions and future work
In this paper, we studied the problem of Multi-Robot Task Allocation with communication range limitation and give two levels of solutions. First of all, we designed a communication keep revenue based on the market-based approach and Hoplites framework. The UUVs appeared to be moving closer due to the inclusion of communication keep revenue, which ensures the scope of communication to a certain extent. Secondly, we designed the communication relay task using the active coordination mechanism of the Hoplites framework. When the distance between the UUV and all other UUVs beyond the communication limit, it restored the entire communication network by actively requesting cooperation from other UUVs. Through simulation experiments, we verified the effectiveness of the algorithm. In the future, we hope to verify our algorithm in physical experiments.