Heterogeneous Underwater Swarm of Robotic Fish

Underwater robotics is prevalent today in the field of undersea surveys. The current single-agent approach struggles to cover surveys of large areas, especially when visual analyses are required. Applications of underwater multi-robots fleets need to manage the coordination of the agents, which in turn stresses the already limited capabilities of underwater localization and communication. We propose a concept for a multilayered control architecture, which leverages visual recognition and swarm behavior to negotiate the coordination requirements of the single agent while retaining a significant level of control over the fleet of vehicles as a whole. This is obtained by including multiple control layers, paired with a heterogeneous composition of vehicles.


Introduction and Motivation
Undersea exploration and survey are of great interest for numerous fields in research and commercial applications [1].As one of many examples, monitoring of the underwater ecosystem, particularly the health of the coastal shallow water areas, impacts commercial needs such as the fishing industry [2], [3], oil&gas infrastructures' development, civil structures (particularly harbors), development of new shipping lines.Sites of biological, naturalistic, and archaeological value still present open challenges concerning exploration and preservation effort [4] [5].
Due to the complexity of human underwater exploration, even when limited to shallow waters, robots are often employed in those types of surveys [6].The main limitation of studies and surveys in such locations are determined by the challenges of the underwater environment: In particular, communication [7] and localization [8] represent the main concern for Underwater Autonomous Vehicles (AUV).Water, and especially seawater, is a natural barrier to most electromagnetic frequencies.Those include most, if not all, wireless electromagnetic communication channels used above water.Thus, today's state of the art for underwater communication is focused on acoustics, which is limited to the order of a few kbits/s of bandwidth at most.This characteristic of water affects also the localization, because GNSS (Global Navigation Satellite Systems) is unreachable underwater.Coupling the absence of GNSS with underwater currents and free-floating body dynamics, the estimation of absolute position becomes extremely challenging.
One less discussed challenge of the underwater world, is represented by the limited range of visibility.This is a byproduct of the absorption of sunlight from the water and particle 1292 (2023) 012008 IOP Publishing doi:10.1088/1757-899X/1292/1/012008 2 suspensions.The first leads to progressive loss of natural light to complete darkness with the increase of depth.Particle suspension and chemical composition of water determine the absorption and scattering properties of the medium [9].Those parameters strongly affect the visibility range, even considering the use of artificial lights, and typical values of effective visibility range could vary from tens of meters to under one meter depending on the conditions.Traditional underwater robotics bypasses the issue, relying mostly on sonar and acoustic technology when performing wide areas surveys.However, the amount of information that can be extracted by a sonar survey is less detailed and of different content from what can be obtained by a close visual inspection.
The most promising approach to improve the performance of visual underwater surveys is the implementation of a multi-robot approach, specifically a swarm approach [10,11].The first obvious advantage lies in the possibility to parallelize the job among multiple agents: this not only allows for lower operating times and costs, but enables wide area studies when the time frame for the data acquisition impacts the measurements (for example: when studying fauna active only at a certain time of the day, or chemical proprieties with a 3d pattern or vertical stratification).Moreover, the swarm allows tasks' allocation among the different AUVs, having different tasks performed at the same time independently or collaboratively by the AUVs [12].This also allows more flexibility during the mission, especially when the environmental conditions are dynamically changing or external factors impose a change of the mission.Furthermore, underwater robots are susceptible to failure due to the difficult environment: a distributed approach offers significantly more robustness to a mission as the loss of functionality of single vehicles does not impede the mission itself .In addition, using a multi-robot approach opens new opportunities to challenge the issue of localization and path reconstruction.In this work, we envision the use of swarm control algorithms, where the control rules of each agent are not reliant on accurate localization of themselves or of the other agents, but only on the application of simple rules based on neighbor detection and minimal data sharing.By fusing the information of the single agents, a collective localization can be then achieved.A more traditional top-down control is then added to allow the operator to interface with the swarm.The different control layers that constitute the proposed architecture of our system are described below.
In this paper, we present a conceptual architecture of a fleet of underwater and surface robots for wide-range underwater surveys, that rely on swarm behavior to coordinate all the robots.

General Platforms description and functionalities
A full description of the platforms is presented in [13].The robotic swarm is composed of the large majority of fish-robots.Tab 1 summarizes the main characteristics of such platforms.Those fish-robots are AUVs (Autonomous Underwater Vehicles), capable of freely moving underwater.Each of those robots mounts a set of three cameras that offer up to a 270degree FOV (Field of View).The Fish-robots are then subdivided into two classes: basic and advanced.The main variation of the advanced robot is that it features an acoustic modem for communications (Succorfish V3).Differently from most AUVs, these models are not mounting any kind of sonar or DVL (Doppler Velocity Logger) or USBL (Ultra Short Base Line); those instruments are usually part of most AUV architectures and are essential in the process of localization and odometry.However, their cost, size, and complexity, are not fit to be mounted on small and expendable platforms as required for the single swarm agent.
A USV (Unmanned Surface Vehicle), named Floater is added to the swarm.The Floater features an array of sensors, along with GNSS localization, acoustic modem, downward pointing underwater camera, and wireless communication with the surface operators (see Tab 2).From the Floater, a second platform can be deployed.This second platform, called Sinker, does not have self-locomotion and it is dependent on the Floater.The two platforms are connected by a tether.A Pulley system allows for autonomous deployment and retrieval of the sinker.The Table 1.Main characteristic of the fish-robot Platform.Sinker features an array of sensors and modules as well, along which an acoustic modem and 360 FOV array of cameras (see Tab 3).Fig. 1 shows the swarm agents.

Communication
The communication among the different platforms play a crucial role.The final overall scenario considers an operator based on a control station able to be connected to the swarm underwater.This allows the operator to act as being present in the underwater environment, observe the environment and change the mission of the AUVs.The communication between the floater and the base station is based on WiFi technology, while the communication between floater and sinker in guaranteed by an ethernet cable.More interesting and open to research are the communications channel between the floater and the fish and among the fish.While most underwater communication is handled via the acoustic  channel, the proposed system requires some alternative modes of communication.The main drawbacks that the acoustic system present in our case are the limited bandwidth and the long communication delay: The acoustic channel is shared among all agents, so only one communication at a time can be transmitted at any moment for the whole swarm of robots.The acoustic modem selected for the project allows for approximately 64 bytes per second, which is close to the performance of most acoustic transceivers available.Possible solutions to overcome these limitations intrinsic to the acoustic channel rely on pre-processing of data before transmission, as for example using encoding-decoding schemes.To achieve a higher rate of communication different modes of communication have been implemented.Optical communication has been already implemented for underwater in [14] by using blue light wavelength; in this case bitrates of order of Gb/s can be achieved, with a maximum range, in optimal conditions, of 500 meters.For this purpose, blue light LEDs can be integrated; alternatively, other frequencies could be used for our case, such as high-infrared, as the communication distance required is less than 10 meters.A hybrid (multimodal) method (optical and acoustic) is the best solution [15].This way high bitrates can be achieved at short ranges, and the fish can still communicate when the members of the swarm move out of range of optical communication.Other means of communication also become available, still at a research stage, such as electric field emission and sensing, using the technology presented in [16], or low-frequency radio communication.Electric sense has been integrated on the new version of the fish platform, mainly for perception purposes.Another avenue that opens thanks to the proximity of the agents, is related to vision: the lateral line made by RGB LEDs can be used to broadcast static messages by position and color, beyond the more obvious purpose of guaranteeing visual recognition and allowing estimate of the relative position over time of neighbors.

Multi-robot control architecture
In this section, we propose a system control architecture for our heterogeneous Underwater Swarm.We approach the problem by subdividing it into layers.Each layer is described below, while Fig. 2 present a synthetic summary of each layer structure.

Single Agent Layer
The first layer is represented by the fish-robot isolated.Each robot carries a series of sensors that allow for a minimal level of control.The depth control, heading control, and approximate surge control are handled at this level.Inertial and Pressure sensors provide feedback for a closed loop control on the first parameters, while surge velocity must be controlled open loop at this layer.However, considering the large number of platforms sharing the same hydrodynamics and propulsion system, it is reasonable to invest in an in-depth characterization of the command-tovelocity model to allow for fairly reliable control.An in-depth study of such a model is being conducted for our platforms in [17].

Local Group Layer
The local group layer interests a group of robot-fish that work in close proximity.Each group is composed of at least one advanced fish-robot.In this layer, acoustic communication is not used for coordination or control.The agents are capable of seeing each other in their FOV and are close enough that the water proprieties do not affect the capability of each agent to see or recognize each other.On this layer, visual broadcast communication (LED strips) can be implemented, as well as short-range communication.Onboard software, with the application of Neural Network algorithms, is able to identify close neighbor relative positions appearing in the FOV as well as roughly estimate their relative heading.Multiple swarm-like algorithms can be implemented using relative position and internal odometry deviated from the single agent layer.This allows for different behaviors and formations to be implemented and swapped according to mission and situation.The local group uses the alternative short-range communication channels presented above to share the behavior to be applied.

Super Group Layer
The supergroup layer focuses on the implementation of acoustic communication and ranging.TDoA (Time Differential of Arrival) can be used to localize the different local group leaders (Advanced fish-robot).Acoustic communication allows sending commands to local groups, effectively using each local group as a single agent in a traditional multi-robot approach.This represents also the lower level of control accessible to the human operator.The collected TDoA is locally used for odometry refinement, but more importantly for post-mission trajectory reconstruction.

Floater and Sinker
The Floater and Sinker platforms can be considered a separated layer that interacts transversely with all other layers.The Floater and sinker have their own shared single agent layer.On top of that, they can use fast internal communication, GNSS self-localization, and vision to accurately track the 3D absolute position of the Sinker.The Sinker can be recognized by close neighbors and support local groups by sharing information with the operator, thanks to the cabled connection to the surface.Furthermore, a close neighbor can be identified visually and properly localized via triangulation, when appearing in the FOV of the Sinker and Floater downward camera.Floater and Sinker also act as separate acoustic nodes to further refine the agents' odometry and trajectory reconstruction.Finally, Floater and Sinker act as a bridge with the human operator.

User Layer
Thanks to the Floater, a human operator can send directives and receive feedback from underwater agents.It can send specific instructions to a local group or send general instructions to be executed globally.The Sinker 360 view paired with a fast connection enables continuous visual feedback that allows for a third-person view of the swarm.We envision here a Virtual Reality set to be used to enable a full virtual immersion.

Conclusions
We envision for a multilayered control approach for an underwater swarm of robots to enable several new types of mission profiles.A localized use of swarm control coupled with higher level multi-agent allows to negotiate an uncertain localization of single agents while retaining top-level control and feedback of the system.The multi-agent approach allows for a later trajectory refinement by comparing relative distances and ranging measurements collected during the mission, which in turn allows more accurate spacial localization of all the readings and measurements collected.In Fig. 3 we have a conceptual representation of the swarm agents A large number of agents enable wide-scale visual inspection as well as chemical survey mapping.Potential missions may include seabed algae and coral mapping for sea-health inspection, fish-counting surveys for fishing industries, patrolling of fish farms, archaeological sites virtual reconstruction, underwater mine clearance operations, preliminary and routine inspection for underwater constructions and facilities, including pipes and cable systems, and collaboration in Search and Rescue mission.

Figure 1 .
Figure 1.Picture of the swarm agents: on the left back the Floater platform, connected though a tether to the Sinker platform on the left.In the front 2 of the Fish-Robots are visible.Note that, since they are equipped with acoustic modem (visible on top), those agent can act as local group leaders.

Figure 2 .
Figure 2. A representation of the different layers, and the types of localization and communication strategies implemented at each level.Starting from single layer, with only basic sensing, local group dominated by visual, super-group depending acoustic, user layer interfacing from the surface, and floater sinker acting as bridge among the different layers.

Figure 3 .
Figure 3. Rendering of a potential mission scenario: Floater on the surface deploying the sinker to monitor two local grops of the agents each, busy monitoring the area.

Table 2 .
Main characteristic of the Floater Platform

Table 3 .
Main characteristic of the Sinker Platform