Enabling Human-Autonomy Teaming in Aviation: A Framework to Address Human Factors in Digital Assistants Design

The introduction of artificial intelligence (AI) tools in aviation necessitates more research into human-autonomy teaming in these domain settings. This paper describes the development of a design framework for supporting Human Factors novices in considering human factors, improving human-autonomy collaboration, and maintaining safety when developing AI tools for aviation settings. Combining elements of Hierarchical Task Analysis, Coactive Design, and Types and Levels of Autonomy, the design framework provides guidance in three phases: modelling and understanding the existing system and associated tasks; producing a new function allocation for optimal Human-Autonomy Teaming (HAT); and assessing HAT-related risks of the proposed design. In this framework, designers generate a comprehensive set of design considerations to support subsequent development processes. Framework limitations and future research avenues are discussed.


Introduction
The introduction of artificial intelligence (AI) tools into a system will likely have profound effects on the nature of human work in aviation.Specifically, the addition of AI tools requires that human operators in charge of different systems be able to coordinate effectively with these emerging assistance tools.An early step in systems design is deciding how work will be divided between the human operator and the system.This is known as function allocation.Fitts' [1] famous "Humans-Are-Better-At, Machines-Are-Better-At" (or "HABA-MABA") list was one of the first function allocation methods developed.Fitts' list has evolved over the years and is still a widely used method today.However, such an approach is considered by many Human Factors (HF) experts to have many limitations and is unlikely to really improve cooperation between human operators and artificial agents [2].Over time, HF design has grown to encompass other factors that influence human performance, including cognitive, social, and organizational factors.Despite significant progress in this field, current methods for HF design and analysis are either limited in their consideration of Human-Autonomy Teaming (HAT), or too complex or time-consuming for non-experts to use.This highlights a need for new lightweight design methods for HAT to provide generative, insightful, and actionable descriptions of future incarnations of aviation systems.Such coordination constraints need to be

Developing the Framework
Based on the findings of an initial literature review of HF design and function allocation methods (omitted due to page restrictions; [4]), the SafeTeam framework relies on and combines elements of evidence-based methodologies to enable individuals without HF expertise to design and/or assess their automation systems with HAT in mind.The framework focuses on the creation and exploitation of a Hierarchical Task Analysis (HTA; [5]) while considering automation task types [6] and aspects of observability, predictability, and directability to promote HAT [7].In the SafeTeam project, the framework was iteratively developed and applied at various stages to two different use cases concerning the development of digital assistants for (1) delivering air traffic control sector workload prognoses, and (2) cockpit management of unstable approaches.Semi-structured interviews and continuous discussions with use case partners offered insights into work processes and attitudes towards HF.These insights informed the development of the current design framework and its instructions.Central to developing the framework was the emphasis on the close cooperation among involved stakeholders and the importance of early evaluation and feedback of design proposals throughout the entire system development process.Frequent and continuous design concept evaluations can reduce development costs and highlight design issues that can lead to HAI issues.The integration of the criteria-from the literature review and the use cases-led to the creation of the SafeTeam framework.Similar to the processes of coactive design [7], the SafeTeam framework consists of three phases of work.Iterations within and between phases are highly encouraged as it will be difficult for anyone to capture all aspects of task allocation and its HAT effects on the first go.These three phases are detailed in the next section.

The SafeTeam Design Framework for Enabling Human-Autonomy Teaming
In this section, the three phases of the SafeTeam framework are presented.Phase I covers the modelling of system factors and interactions.Phase II provides guidelines for exploring novel HAT designs.Phase III covers the evaluation of HAT-related risks associated with a proposed design.

Phase I: Modelling key system factors and interactions
The first phase of the SafeTeam framework begins by describing the current work activities of the system considered.To identify and delimit these, consider which system factors are central to the design idea in question.Which tasks, artifacts, agents, processes, organizations, and regulations necessarily be modelled and understood to develop and evaluate the design?It is important to strike a balance between sufficient coverage and inclusion of relevant system factors that might be impacted by the implementation of a new function on the one hand, and manageable model complexity on the other.We want the designer to consider the ripple effects caused by changes to system factors in sociotechnical systems.Changes may impact the task at hand, the agent performing the task, the artifacts required to perform the task, the processes between agents of the system, the organization where the task is taking place, and eventually even the regulations considered for this system.The modelling is initiated by an idea; a design proposal or change requirement prompted by inefficient task processes, inadequate artefacts, etc.The design idea guides and focuses the work; data collection about the target domain is centered around the idea, moving outwards to include the relevant domain components (e.g., other tasks, agents, and artifacts).Appropriate data collection methods include interviews, focus groups, field studies, etc.
The information collected makes it possible to develop an HTA [5].The aim of HTA is to describe and compare the different tasks that an operator must perform to meet a predefined objective.Developing the HTA is done in several structured steps considering 1) the system factors, 2) involved agents 3) the main tasks and goals, 4) decomposed sub-tasks, 5) task interrelations (e.g., sequential or parallel), and 6) the type of tasks to be performed (information acquisition, information analysis, decision selection or action implementation; [6]).The sixth step is not part of a traditional HTA procedure, and was added to provide system understanding and structure for the work in phases II and III.The resulting HTA components are illustrated in figure 1.
Figure 1.HTA components.Goals are broken down into nested hierarchies of tasks, subtasks, and finally operations.A task must have at least two subtasks or operations.Agents are color coded (e.g., yellow for artificial agent, green for human agent).Operations are tagged with task type (information acquisition, information analysis, decision selection, or action implementation).Plans describe the order of tasks and operations (e.g., sequential, parallel, or selection criteria).Known problems are indicated (e.g., bolt icon).Horizontal lines under tasks indicate that there are additional tasks/operations in the branch that are not relevant to the analysis at hand.The resulting model of the system may then be used for comparison and evaluation of the proposed new design developed in phase II.

Phase II: Designing for safe human-autonomy teaming
The second phase focuses on proposing and structuring automation solutions in a manner that supports subsequent evaluation and detailing.The new design solution is visualized and explored by creating a new and complementary HTA.The work in phases II and III is highly iterative and any design issues that emerge later may necessitate the designer to re-assess aspects of human-automation interaction and modify the initial task allocation of the proposed design.
By modelling the new design through an HTA, tasks must be distributed among the agents to optimize HAT.When designing the task allocation (static, dynamic, etc.), it is important to keep in mind that all agents-human and non-human-must enter into agreements, exhibit mutual predictability and directability, and maintain common ground.This ensures that they can rely on each other when considering their own actions and facilitates synchronized actions and efficiency.
To support the creation of a safe autonomous system from a HAT-centered perspective we propose a set of seven design guidelines.Designers should keep these guidelines in mind when suggesting their function allocations.By doing so, they can improve safety, avoid common HAI pitfalls, and hopefully will reduce the number of design phase iterations.The design guidelines for enabling HAT are: 1. Agents should share a common goal.Ensure that any agent participating in a task understands and accepts these shared objectives [8].Individual agents should have the capability to represent, reason about, and modify their individual goals to coordinate with and maintain the common goals.Considerations include the need for clear goal definition, both for the common goal and individual goals, to prevent ambiguity and inefficiency.Additionally, agents should be able to negotiate goals, enabling adaptation in changing situations, and protocols should be developed to maintain team cohesion and resolve potential conflicts.2. Agents should be able to share their status and intentions and interpret and observe the intentions of others.This involves providing real-time updates on status, capacities, actions, decisions, and intentions to enable other parties to maintain situational awareness and make informed decisions [7,8].It is crucial to represent this information in a manner that allows teaming agents to observe and interpret it to evaluate and detect possible failures.Considerations include standardized information flow for communication, transparency in decision-making processes, sharing of intentions to enhance coordination and trust among team members, and periodic status updates to ensure awareness of changed conditions.3. Agents should be directable, allowing the human operator to guide, control, and influence the autonomous system to mitigate risks in unforeseen or complex situations [7,8].These directions can range from explicit tasks allocation to more subtle forms like providing information.Considerations include user interface design for clarity and ease of interaction, including intuitive controls and informative displays.The system should also have override capabilities, enabling the operator to intervene or adjust the system's behavior in real-time while considering potential ripple effects.Moreover, ethical and safety constraints must be implemented within the system to prevent unsafe or unethical actions, ensuring that even in autonomous mode, the system operates within predefined boundaries.4. Designers of the system should aim for Shared Situation Awareness, where agents can comprehend the dynamic environment they operate in, understand its elements, their meanings, and project their status in the near future [9].Considerations include achieving individual situation awareness and building shared situation awareness by sharing knowledge, beliefs, and assumptions.Additionally, designers should consider enabling the operator to seek additional information through visual scanning, aided by attention guidance like alerts or notifications.The integration of knowledge into smaller parts through display integration should also be considered for effective shared situation awareness.5.The system design should aim for an optimal mental workload by carefully considering how mental effort and cognitive resources are managed [10].This involves assessing the cognitive and perceptual demands on the operator.Balancing workload is crucial because high and low workloads can both harm performance.High workloads can lead to fatigue and reduced performance, while low workloads result in reduced focus.To optimize mental workload, align the workload and task complexity with the operator's cognitive abilities, making sure tasks and information engagement are mentally stimulating.Also, consider factors like task complexity, data quantity, presentation, interruptions, and multitasking during system design.6.The system should foster mutual trust within the human-autonomy team, as trust plays a pivotal role in determining how the system is utilized [11].Trustworthiness in autonomous systems facilitates acceptance and mitigates the risk of misuse and underuse.Low trust from the human operator may lead to the disregard of valuable system information, but excessive trust can result in the neglect of critical data.System reliability is fundamental, requiring consistent and accurate performance over time.Transparent system intentions and clear explanations of the information considered can also increase trust among human operators.Consider carefully how the results of algorithms are displayed to operators.This can prevent errors arising from poor design or inaccurate outcomes.Moreover, designing for operators' familiarity with the system's components can increase trust.Robust error handling and recovery mechanisms are also crucial; the type and frequency of errors can directly affect operator trust, especially in cases involving excessive false alarms and false positives.A system's ability to recover and consistently maintain performance can further solidify trust in its operation.7. Agents should act in compliance with ethical standards, emphasizing the necessity of adhering to a shared ethical framework [12].Agents should grasp and embrace these ethical principles, facilitating ethical decision-making and behavior within the human-autonomy team.Establish clear and unambiguous ethical standards to foster a mutual understanding of ethical conduct among all agents.Ethical accountability mechanisms should be in place, allowing for the monitoring and evaluation of agents' adherence to ethical standards and enabling accountability in the event of violations.Moreover, promoting transparency in ethical decisionmaking processes is essential, granting human team members the ability to question, comprehend, and trust the ethical choices made by artificial agents.Conflict resolution procedures should be implemented to address ethical conflicts within the human-autonomy team.Finally, dynamic ethics protocols should be considered, enabling agents to adapt their behavior in response to evolving shared ethical standards or emerging ethical dilemma.

Figure 2:
HTA of an unstable approach example transferred to a tabular (TTA) format.
The primary objective of phase II is to produce a task allocation that maximizes performance of the joint human-automation system while safeguarding against potential hazards.As in phase I, it is highly recommended to take a collaborative approach in phase II, where the HTA serves as a tool to facilitate discussion.Therefore-where possible-designers should include end users in the design process.Alternatively, they should collect their feedback on the design proposal at a later stage and iterate.
Once the proposed task allocation has matured and settled in HTA form, it should be transferred into a tabular format (Tabular Task Analysis: TTA), where it can be evaluated further.As for the HTA, the SafeTeam framework provides structured steps to create the TTA.However, due to page restrictions, we defer those instructions to the full version [4].In figure 2 above, an example of a TTA is provided.The resulting models of the new design-the revised HTA and TTA-may then be used for risk assessment in phase III.

Phase III: Assessing the design proposal for collaboration issues and risks
The purpose of this phase is to explore how different task allocations may lead to certain risks and generate design requirements to mitigate these risks, focusing on HAT aspects.The list that is produced in this step is the basis for further development and detailing of the design.It requires two additional columns to the TTA produced in phase II.These will contain risks that are associated with operations listed in the TTA and design considerations proposed to mitigate them, respectively (see figure 3).Designers are requested to consider each row of the TTA (each operation) and-with inspiration from a list of risk-oriented questions provided by the framework-to identify and assess HAT-related risks.If possible, designers are advised to return to end users or other stakeholders that participated in phase I and collect possible risks from their points of view as well.The questions are summarized in the instructions below.
For Information Acquisition, explore the challenges related to the completeness, accuracy, and reliability of collected data, considering the potential handling of faulty or outdated information and its consequences.Also examine situations where agents may struggle to adapt to changing environments or new information, as well as instances where system support for data collection faces challenges.Also consider factors that could hinder timely data submission and the potential consequences.Address issues such as data inconsistencies, conflicts, misinterpretation, and information overload during presentation.Investigate the impacts of inadequate feedback on collected data and the risks associated with agents' learning or failure to learn from their actions, aiming to improve data collection methods.Consider the handling of sensitive or confidential information by agents and the associated risks, including potential ethical dilemmas and privacy breaches during data collection.
Regarding Information Analysis, consider the methods and expertise agents use to identify patterns and anomalies, and explore potential consequences when such expertise is lacking, along with the contextual factors that may lead to misinterpretation or faulty error handling.Examine what factors could lead to changing data conditions and the associated risks to analysis processes.Consider the outcomes of inadequate feedback on analyzed data and the risks tied to agents' learning (or lack thereof) from their actions and interactions, aiming to enhance their data analysis methods.Focus on how agents collaborate and share insights during analysis, with an eye on situations where collaboration may hinder the process, potentially due to misunderstandings of roles and responsibilities.Explore scenarios where agents may face an overwhelming volume of data, and the risks it poses.Lastly, assess the potential impact of biases and ethical concerns on the analysis process, highlighting the implications of biased or unethical analyses.
For Decision Selection, assess the risks of agents making decisions contrary to ethical standards and the potential consequences.Explore how changing conditions can impact decision-making and the consequences if agents cannot adapt based on new information.Consider the outcomes of inadequate feedback and the risks tied to agents' learning to enhance their decision-making skills.Examine the potential for collaboration during decision-making, what could hinder it, and the consequences.Also address the need for clarification and feedback, with risks related to unclear decisions.Assess the risks of agents neglecting criteria and prioritization in complex scenarios, which can lead to decision paralysis or information overload.Also explore scenarios where agents may fail to assess and mitigate high-risk situations tied to their decision recommendations.Additionally, consider the reasons for human intervention in agent decisions and the need for procedures to handle conflicts between human and agent recommendations.Lastly, look at how agents communicate their confidence and the consequences of unclear recommendations.
For Action Implementation tasks, analyze the need for modifying agent actions in response to changing conditions, emphasizing the potential consequences that may arise if adaptation options are limited.Investigate how the system offers feedback on actions and its mechanisms for promptly alerting agents in case of anomalies or unexpected outcomes.Focus on the methods agents employ to execute actions based on decisions.Consider potential issues associated with these methods, particularly when it comes to ensuring accuracy and reliability.Assess the risks related to the speed at which artificial agents implement actions after a decision is made, examining potential sources of latency or delays and their potential consequences.Explore the potential consequences of errors, deviations, or failures in agent action implementation and inquire about mechanisms for agent error recovery.Lastly, examine issues tied to integrating agent actions within ongoing processes, conflict prevention strategies, and the potential for agents to seek approval or guidance from others.
After risk assessment, designers are asked to reconsider the design guidelines from phase II to support them in the definition of solutions without introducing new risks.Yet, if it is challenging to mitigate certain severe risks, it may be necessary to reiterate the function allocation in phase II.

Concluding discussion
We propose a methodological framework to support the consideration of HF constraints related to HAT, which is a fundamental requirement when designing systems that combine artificial and human agents.The framework provides 1) a way to systematically model tasks in sociotechnical systems, 2) tools for identifying coordination/cooperation problems generated by the introduction of an artificial agent in a sociotechnical system, and 3) design guidelines to compensate for these problems.
The proposed framework takes inspiration from multiple established methods and frameworks, primarily HTA [5], coactive design [7], and types and levels of autonomy [6].HTA's flexibility allowed us to introduce and incorporate the task type taxonomy [6] to provide some structure to the function allocation (i.e., task design) and risk assessment phases.Aspects of coactive design regarding agent (or task) interdependencies, observability, predictability, and directability were adapted and included in the current framework in the form of design guidelines and risk assessment topics/questions.We opted to include the coactive design-inspired HAT guidelines in the task allocation procedure (phase II), with the intention that function allocation would be done collaboratively, creatively, iteratively, and reflectively.Taken together, we believe that the proposed framework can help non-experts in HF to consider relevant HF and HAT aspects that were previously unaccounted for due to lack of expertise or resources.
Naturally, the proposed framework has some limitations.First, the framework is not exhaustive or comprehensive in a narrow sense, but in a broad sense it covers most of the relevant enablers of HAT, although none of them in-depth.The ambition and goal is that by using the framework, designers can achieve similar HAT qualities and system performance for less effort compared to other established (more complex and labor-intensive) methods.Additionally, by offering a broad toolset, we hope to raise the minimum level of awareness about HF and HAT in the aviation industry overall.However, whenever possible, we still recommend acquiring the assistance and expertise of HF experts when designing for HAT.
A second limitation concerns the development of the framework itself.As detailed in the method section (see section 3), the framework was iteratively developed in collaboration with and through a series of use case applications.As these use cases were ongoing project work packages, the final version of the proposed framework is still untested.Further research is needed to apply the framework in novel use cases and assess its usability and effects on the HAT qualities of the produced designs.

Figure 3 :
Figure 3: TTA with added columns for risk assessment and mitigating design considerations.