Structured Description of Autonomous Inland Waterway Barge Operations

Autonomous and unmanned shipping is revolutionizing the maritime industry by introducing a paradigm shift on how to design the vessels and supporting land-side infrastructure. Currently, there is a lack of formalisms on how to plan for such operations, determining the varying degrees of autonomy and human responsibility, whilst ensuring safety and security. This paper describes fundamental concepts of autonomy in the context of ships. These are then applied in a methodology used to create systematic and structured descriptions for the operation of autonomous ship systems. The examples we use are based on ongoing efforts related to a planned autonomous inland waterway (IWW) barge operation. Finally, we show how the descriptions can be used in conjunction with existing safety and security analysis techniques. Our experience with this methodology is that it allows for a smooth transition from the autonomous ship system design phase to the assessment of the same system using UML notations. We believe that the same methodology can be easily applied to the other use cases and similar systems elsewhere.


Introduction
Autonomous and unmanned ships must be at least as safe and secure as conventional ships.To ensure this, the ship's operational profile (CONOPS -Concept of Operations) must be described for each of its areas of operation and for its intended usage.Currently, few formalities exist on how to ensure that all aspects of the operations regarding autonomous, supervised, and manual operations are covered.In this paper, we describe and exemplify the methodology from the Seatonomy [1] and AEGIS [2] projects on how to formalize the description of operations of an autonomous inland waterway (IWW) barge sailing between Rotterdam and Ghent.This is an interesting case to analyze for several reasons.The barges pass through a number of locks and bridges, cross borders, and sail on canals and rivers with different degrees of traffic.Efficiency is important, as blocking or congestion may introduce serious delays both to vessels, ports, and hinterland operations.Autonomy will also create new possibilities for cyber-attacks as coordination between the human and automation via a communication link becomes a central component.Therefore, there is a need to make risk-based decisions on how the operations should be designed.
The paper is structured as follows; Section 2 explains the main concepts we use for the CONOPS and the different classes of autonomy.Section 3 gives an overview of the methodology itself, with examples from barge operations in Section 4. In Section 5, we show how the CONOPS can be used as a basis for safety and security analysis, and section 6 concludes the paper.

Autonomous Ship Systems: Humans and Automation
There are many different definitions of autonomy and levels of autonomy [3], but in this paper we lean towards ISO/TS 23860 [4]: "In the context of ships, autonomy means that one or more of a ship system's processes or equipment, under certain conditions, is designed and verified to be controlled by automation, without human assistance".This definition uses the term control which can be understood as having the full responsibility for safe operation of the processes or equipment.This in turn implies that the responsibility of control can reside either with the human or with the automation.For a ship, this may mean that the automation is fully capable of controlling the navigation in open sea and calm weather, but that human assistance is required, e.g., in port approaches and during berthing.One should keep in mind that the system, as used here, most often represents a sub-set of the full autonomous ship system capabilities, e.g., energy production, stability, fire protection or navigation.
In principle, unmanned autonomous ship systems will be designed to sail with an autonomous onboard controller (AOC) that can operate the ship without human assistance most of the time, while a manned Remote Control Centre (RCC) will be able to monitor and intervene to handle situations beyond the AOC's capabilities [5].This requires that the AOC can issue warnings in time for the operator to gain sufficient situational awareness to take safe actions.To describe the collaboration between the human and automation, we refer to the following concepts in this paper: • Operational Envelope (OEnv): This is the overall capabilities of the system, including both automation and human.This can be seen as a state space over, e.g.weather, traffic conditions, ship state, etc. [6].
• Operational Design Domain (ODD): This term is taken from SAE J3016 [7] and represents the part of the OEnv where the automation is capable of controlling the system, i.e. may be accountable.Following the ODD taxonomy of PAS 1883:2020 [8], the ODD shall be classified into the top level attributes; a) scenery (non-movable elements of the operating environment), b) environmental conditions (weather, atmospheric conditions, connectivity), and c) dynamic elements (movable elements of the ODD, e.g.traffic or subject).
• Fallback States: This is a collection of "designed states" outside the OEnv that can be entered through a fallback function when the OEnv is in some way exceeded.These states are outside the OEnv but should still represent a tolerable risk for the system and its environment.
With the above definitions, we can further define four main operational modes or classes of autonomous operation: • Fully Autonomous (FA): A system that is approved for operation completely without operators within the OEnv.Operators may still monitor or intervene in the system, but they should not be required to do so.• Constrained Autonomy (CA): The automation can safely control the system within the specified ODD.In addition, the automation must be able to detect when human control will be needed, and to warn the operator sufficiently before control is transferred to allow the operator to gain the necessary level of situational awareness to safely handle the situation.• Operator and Automation (OA): The automation can control the system with certain constraints, but it is not possible for the automation to early enough detect the need for control transfer.This may be because the ODD is not fully known or because the transition out of ODD is too fast to alert the operator in time.
• Operator Exclusive (OE): Automation may still give some assistance, e.g., through autopilot control rather than direct thruster and rudder control, but the operator needs to be continuously in control of the system.Figure 1 illustrates how the transitions between the different operational modes can take place within the OEnv during a voyage.Any mode can lead to a fallback state outside the OEnv.It can be more difficult to return to the OEnv from these states.

Methodology Overview
The methodology described in the AEGIS project [2] is used to create systematic and structured description of the operation of autonomous ship systems.Figure 2 shows the different parts of the methodology (left part) and indicates that the operational description of the autonomous ship system can be formalized by a set of UML diagrams (blue boxes to the right).

Figure 2. Summary of methodology for analysis of MASS operation (©AEGIS [2])
The methodology starts with a textual and high-level description of the mission, which may for instance be a certain voyage with port calls and cargo operations.The mission is further detailed following the flow of figure 2: The context describes the external entities and systems that the ship system cannot directly control.This includes other vessels, traffic services such as VTS and MRS (ship reporting areas), pilots, fairway information (Maritime Safety Information and aids to navigation), among others.Actors are more directly involved in the autonomous ship system.This could include the AOC and the RCC operators, automatic cranes and mooring systems, location positioning systems, among others.Ship particulars is the description of the capabilities of the autonomous ship.These capabilities are often tailored for repetitive missions, but general-purpose ships may also contain capabilities not relevant for the mission at hand.Ship processes are the different processes that the autonomous ship system must handle to implement a given mission.Examples of processes are energy production, cargo handling, and navigation.
The mission is broken down into a set of mission phases, which are characterized by a set of geographic and operational parameters, also called state variables.Example state variables can be related to traffic, communication possibilities and depth.Similar mission phases can be generalized into mission phase patterns.
System Control Tasks (SCTs) describe the safe operation in a certain mission phase and ship process.The SCTs are developed based on the context, actors, ship processes and phase patterns.One SCT can span several system processes and possibly several mission phases.Each SCT is described by an associated state space that defines the parameter ranges over which the respective component can safely function.The SCT definitions can be represented by suitable UML diagrams [9], such as use case-, activity-, collaboration-, state-and sequence-diagrams, to formalize the descriptions and allow for reuse and linking of constructs between the descriptions.

The Mission
The overall mission for our use case scenario is to enable small autonomous barges to transport wheeled cargo, also known as roll-on/roll-off (RoRo), through an inland waterway route between Rotterdam and Ghent.Information regarding this route is collected from the use cases in the AEGIS project [2], which has involved stakeholders from shipping lines, ports and service providers, and also by using the EuRIS-portal [10].The route is illustrated on the map in figure 3 as coloured lines with red circles indicating the location of the four locks along the route.The route has also two bridge passings, and even though the bridges probably have sufficient clearance for the barges to pass without requesting bridge opening, we consider bridge opening as one of the mission phase patterns to make the description more complete.Cargo loading/unloading can take place in terminals, as is done today, but possibly also at designated locations along the route to reduce the length of the hinterland transport operations.

Context
The context of inland waterway sailing can be divided into what is done today and what is foreseen with autonomous systems in the future.Parts of today's context systems are likely to be automated as well, or at least be upgraded to support the autonomous operations of the barges.The most relevant external entities and systems today are as follows: • Locks are directly manually operated or locally controlled from centres in the vicinity of the locks.While inside the locks, the barges need to do mooring while being raised/lowered.• Bridges will need to be raised when the barges do not have sufficient clearing.
• Signalling systems are used for controlling traffic, for instance related to entering/exiting locks or for waterway intersections.• Terminals contain berthing equipment and cranes for cargo operations.Also, trucks may be used for RoRo operations.• Other vessels (commercial and leisure crafts) sail in the vicinity of the barges.Traffic operations such as yielding, overtaking and turning is something that the barges must be aware of.Queuing before entry to locks is also very typical for these operations.

Foreseen situation:
• Remote Control Centres (RCC) can either directly take control of an autonomous barge or provide tactical instructions.• Locks and bridges become more automated, with little or no human presence.Remote control centres for operating these can be located far away.• The locks can provide infrastructure for automatic mooring outside and inside the locks.
• Automated guided vehicles (AGVs) will be used to unload cargo from the barges both inside and outside of the terminals.• Signalling systems and AtoNs become more virtual.
• Smart infrastructure technologies along the waterways, including sensors, CCTVs, transponders, etc. • Other vessels become more autonomous and cannot be hailed directly.
• Barges become more electrified and may need to recharge or replace batteries during the voyage (outside the terminals).

Actors
It is part of the methodology to describe the actors relevant for the system.Figure 4 shows the actors and how they relate to the various activities.The AOC is the high-level controller which supervises all relevant ship systems and gains situational awareness based on data from onboard sensors as well as external sources such as weather data and navigational aids.The RCC has functionalities such as remote control, monitoring and operation of the ship, handling exceptions, follow-up activities during the voyage such as contact with other vessels, ports, government, planning of maintenance, schedules and voyages, and cargo management such as contact with cargo providers, customers, terminal workers, among other tasks.

Ship Particulars and Ship Processes
As a reference barge, we have chosen a vessel based on typical characteristics for existing conventional vessels sailing this route.For RoRo operation, the vessel has both a bow ramp and a small side ramp, to allow for cargo operations on smaller ports/terminals.The cargo capacity can be up to 48 TEU.Navigational equipment includes video cameras (with infrared), radar/lidar, GNSS and local position system.There are different communication options such as maritime VHF (digital), 4G/5G and lone-of-sight WiFi in dedicated areas, such as ports and terminals.
When it comes to the ship processes, we have selected some of the most important one based on the definitions from the Autoship project [11]: Ship operations related to navigation including subtasks for situational awareness (observation), ship control (manoeuvring), voyage management (navigation), and nautical communication, Ship operations related to deck operation (Navigation support related to Cargo and ship supplies operations, and Mooring and anchoring).

Mission Phases and Mission Phase Patterns
In the IWW scenario from Vlaardingen (Rotterdam) to the terminal in Ghent, we identified a list of totally 28 mission phases.A mission phase typically starts and ends when there are changes in the operation of the autonomous ship system, typically when the ship leaves a port, and enter a canal, when the ship enters and leaves a lock or a bridge, or when the sailing changes from complex to simple, or from simple to complex, based on the traffic patterns and other parameters.We have grouped the mission phases into 10 distinct mission phase patterns as numbered from activity 1 to 10 in figure 4. The decision on which mission phase pattern to generalize has been based on the level attention needed by the RCC operator during the leg and operation.This depends on several parameters such as: • The traffic density, for instance the number and type of ships in the area.Using AIS density maps for the actual area, we see that the traffic is overall dense, and that the big locks often have heavy traffic with queues of barges waiting to pass in both directions.
• The traffic complexity, for instance, frequent speed change and course change that is done on this leg, the course that the vessel has relative to the traffic, crossing traffic or parallel traffic.• Fairway complexity, for instance curves, sight, how narrow the fairway is, current, tides, wind, whether it is canal or river.For this case, the whole fairway, except for the short distance when leaving the terminal in Vlaardingen and entering the main fairway, is classified as CEMT VIc [12].In some cases, specific visual signals are used by a barge to signal that the barge needs to deviate from standard navigational rules due to for instance the current (different current in the outer curves, or close to shallow areas).• Communication possibilities, including the infrastructure on the shoreside.
• The complexity of the task, for instance passing a large lock with several chambers or difficult navigation under narrow bridges with marginal clearance.All the locks on this route are manually operated, however, we also added remotely operated locks and automatically operated locks to be able to cover future cases.For instance, for the remotely operated and automatic locks, the operation will be dependent on more sensors (cameras) and communication systems between the sensors at the lock and the operator or RCC for the lock.Some of the locks are large systems with several chambers meaning that the operator or lock control system must decide which chamber to use for each ship.
As suggested by the methodology, each mission phase pattern can be described according to the Autonomous Job Analysis (AJA) schema from the Seatonomy project [13].An analysis of the mission phase pattern Passing Lock Chamber is shown in figure 5.

Mission phase pattern description 4. Passing Lock Chamber
Description: What are we trying to accomplish?What is the relationship to other sub-operations?
The barge enters the lock chamber based on signal and/or direct communication.The Lock system detects that the barge has entered the lock chamber.
The barge is automatically or manually moored and raised or lowered in the lock chamber, usually with other vessels at the same time.The barge exits when the lock gate opens, and the signal to exit is given.

System Control Tasks
Table 1 shows how the relevant ship processes (navigation, cargo operations and mooring/anchoring) are mapped to the 10 mission phase patterns, defining 14 system control tasks.Similar colour indicates same operational sub-envelope, e.g., that the system control tasks for navigation for approaching lock and bridge are similar.Figure 6 shows a UML activity diagram for Passing Lock Chamber.This activity consists of two SCTs, namely the one for navigating through the lock chamber (SCT4), possibly followed by mooring within the lock chamber (SCT13), and then repeating this.8 shows a sequence diagram for the system control task when a vessel is navigating through a lock chamber.The sequence diagram starts when the lock system prepares to open the first gate and gives green signal to the vessels to enter.This gate is subsequently closed, and a red signal is given.Then, during SCT13 (Mooring within Lock Chamber), the water is lowered or raised.The sequence in is repeated for the outgoing gate, and the vessels leave the lock chamber.

Safety and Security Analysis
As already shown by Meland et al. [14], there are many approaches to considering safety and security risks, but they need to take the dynamic nature of autonomous missions and the novelty of autonomous shipping into account.This is why we follow the practice of [14] to develop threat models for each mission phase pattern based on the SCT descriptions.The point with these models is to show the variety of attack possibilities under different conditions and which threat actors that would typically be involved.Figure 9 shows an example of a bow-tie diagram, including both natural/unintended causes (yellow rounded rectangles on the left) and malicious causes (blue rectangles on the left) for the hazardous incident of losing lock control.The consequences (red rectangles on the right) of such an event could occur independently of the type of cause.While the occurrence of natural/unintended caused events follow a somewhat predictable distribution, the likelihood of cyber-caused events needs another approach.We extend the UMLmodels from the SCT descriptions with attack scenarios, as depicted in the lock system example in figure 10.Once we have explored the attack space, we can give a qualitative likelihood estimation [15], [16] for the threats based on based on weighted averages of 1) the size of the group of potential threat actors, 2) how favorable the circumstances are for an attack, e.g., spatial opportunity (is the location of relevant equipment and/or actors to the attacker's advantage?),temporal opportunity (does the timing increase the likelihood of a successful attack?) and vulnerability exploiting opportunity (what vulnerabilities does the victim or target have that the attacker can exploit?), 3) the degree to which an attacker has the necessary means to perform an attack, and 4) motivation, which greatly depends on the attacker's expected reward in case of a successful attack.For instance, our assessment related to barges approaching and passing lock chambers revealed that there is a significant probability that cyber extortionists can use attacks involving fake requests to the lock operator, fake denials of lock passing or jamming.The consequences will typically be denial of service, causing traffic jams in these congested zones.Similarly, there is a significant probability that the same actors could jam communication or send misleading messages to stop traffic from going through.By combining these probability values with the effects of the consequences, we can measure or rank risks with a good justification and traceability.

Conclusion
Autonomous shipping is transforming the maritime industry, ushering in a new era of vessel autonomy.This paper has provided an overview of the fundamental concept of autonomy in the context of ships and it has also presented a methodology for structuring CONOPS descriptions.Our experience with this methodology is that it allows for a smooth transition from the autonomous ship system design phase to the assessment of the same system using a consistent UML notation.So far, it has been applied to different use cases as part of the AEGIS project, including inland waterways barge operations, as shown in this paper, and also for combining long-range transport on container ships with last mile delivery on smaller feeder vessels in narrow fjords.Ongoing work also includes new autonomous concepts for modern ports, including automated cranes, auto-mooring and electric charging.The adaptable nature of this methodology suggests its potential for application in other use cases and similar systems.A clear structure and libraries of actors and technologies (e.g., sensors, navigation and communication systems and infrastructure components) facilitate reusability and consistency between project descriptions.
The structured CONOPS descriptions not only aid in system design but also contribute to early identification of safety and security risks.This enables informed decision-making, even when dealing with novel technologies and new types of operations.However, it is crucial to continuously maintain risk assessment throughout the design process, technological advancements, and changing operating environments.The analysis conducted for the inland waterways mission highlighted threats in congested areas, complex sailing zones, and near bridges and locks, emphasizing the importance of comprehensive risk mitigation strategies.
We have also seen that the process of developing the CONOPS has sparked meaningful discussions among stakeholders, such as: • What should be the default fallback functions?If the general rule is just stopping the barge in all cases of discrepancies, blocking can easily take place.• How much workload/stress can RCC operators handle and how much time is needed for controlled handover?• Should the AOC in some situations disregard the direct commands of the RCC in order to prevent potential accidents?These discussions emphasize the need for case-specific considerations, acknowledging that successful adaptation of autonomous ship systems requires attention to technical safety, security, regulatory compliance, economic viability, ethical considerations, and public acceptance.
The future of autonomous shipping holds great promise for the maritime industry, with ongoing research continuing to shape its trajectory.The successful implementation of autonomous systems will require collaboration, addressing multifaceted challenges, and capitalizing on emerging opportunities.As the industry progresses, autonomous shipping has the potential to revolutionize maritime operations, enhancing safety, efficiency, and sustainability.

Figure 1 .
Figure 1.Cooperation between the automation and human

Figure 6 .Figure 7
Figure 6.UML Activity Diagram for Activity 4 Passing Lock Chamber (©AEGIS [2]) Figure 7 breaks down SCT4 into a more detailed UML activity diagram, where each of the activities are picked from a set of reusable diagram components.Figure8shows a sequence diagram for the system control task when a vessel is navigating through a lock chamber.The sequence diagram starts when the lock system prepares to open the first gate and gives green signal to the vessels to enter.This gate is subsequently closed, and a red signal is given.Then, during SCT13 (Mooring within Lock Chamber), the water is lowered or raised.The sequence in is repeated for the outgoing gate, and the vessels leave the lock chamber.

Figure
Figure 7 breaks down SCT4 into a more detailed UML activity diagram, where each of the activities are picked from a set of reusable diagram components.Figure8shows a sequence diagram for the system control task when a vessel is navigating through a lock chamber.The sequence diagram starts when the lock system prepares to open the first gate and gives green signal to the vessels to enter.This gate is subsequently closed, and a red signal is given.Then, during SCT13 (Mooring within Lock Chamber), the water is lowered or raised.The sequence in is repeated for the outgoing gate, and the vessels leave the lock chamber.

Figure 10 .
Figure 10.Attacks targeting the lock system (©AEGIS [2]) • Different kinds of inland aids to navigation (AtoN), such as buoys, beacons, fog signals, are located in the water or onshore.• In addition to VHF radio, different kinds of land-based communication systems are available, in particular 4G/5G cellular networks.• Inland AIS (iAIS) base stations on shore can track and exchange information with the barges.

: What key information needs to be communicated? What are the communication restrictions and limitations? What communication infrastructure can be used? Barge
/RCC operator -Lock operator: Ready for lock operations.Provide static information: Call sign.Communication infrastructure: Marine-band VHF, 4G/5G, Internet, (iAIS), Limitations: Need available RCC operator and Lock operator (local, remote, automatic).Perception:

Which information about the environment and the system itself must be present? Barge
: Signaling system, state of gates, other vessels.Lock system: Position of the vessels, state of gates, state of valves

Success criteria: What are the criteria for successfully executing the sub-operation? How are the criteria quantified?
Safe passing of lock, no unnecessary delays.

can go wrong: Which external and internal events should be planned for? What should the system do in case of undesirable events?
Collision (damage), Not able to enter (delay, blocking), Not able to do mooring in the lock chamber (unsafe state), No communication with remote lock operator (lock disruption), Not able to exit (delay)

Operational safe state: What should the system do in case of failure/damage?
Halt ship, mooring of ship, give alert.Lock system closes the gates if a dangerous situation is detected.Human intervention is needed to restart the lock system, after confirmation of the situation being OK.

HMI): What type of user interaction is needed? What information does the operator need? What is the role of the human?
RCC Operator should be able to monitor ship and lock state.RCC Operator should be able to request control over the ship.

Table 1 :
Mapping between ship processes and mission phase patterns