Prospect for the application of augmented and virtual reality technologies in the design, production, operation of aircraft and training of aviation personnel

Employees of the Department 703 MAI together with employees of Department 741.170 PJSC “Corporation “Irkut” carried out work on the analysis of possibilities of introduction of modern innovative technologies of augmented and virtual reality in the design, production, operation of aircraft and training of aviation personnel. In the course of work, prototypes of two virtual reality sets were developed. As part of the demonstration of the capabilities of augmented reality technology for solving the problems of aircraft operation and aviation personnel training, two mobile applications were developed. Approbation of the presented projects were carried out in MAI and PJSC “Corporation “Irkut”. The presented prototypes received the highest rating of the scientific and technical Council of PJSC “Corporation “Irkut”, and the interest in implementing such solutions in their production processes was confirmed by manufacturers of aircraft simulators (LLC NPF “SKT” represented by the chief designer), higher educational institutions (MSTU GA represented by the Vice-Rector for research and development), aircraft manufacturers (PJSC “Corporation “Irkut” represented by the Deputy head of the design and training methodology Department) and aircraft operators (PJSC “Aeroflot” represented by the Director of the aviation personnel training Department).


Introduction
The term Augmented reality was introduced in 1992 by employees of the research division of the Boeing company T. Caudell and D. Maisell [1]. They developed a system of assembly procedures for the construction of aircraft. Assemblers with wearable computers could see drawings, diagrams, and process instructions through the special semi-transparent displays of augmented reality helmets. Research on this topic is continued by Boeing until now [2].
In the 90s and 2000s, several works on augmented reality appeared in relation to helmetmounted target designation systems for the Air Force.
In 1996, R. Azuma and K. Furmansky [3] studied the issue of optimal placement of twodimensional objects when using augmented reality applications by air traffic controllers. And already in 2006, NASA integrated augmented reality with the radar system, that allowed air traffic controllers to see information about the flight number, its distance and altitude next to the aircraft in the air, which is especially important, for example, at night or in bad weather [4]. In 2008, J. Rafner and his colleagues commissioned by the US air force to construct a mock-up of an augmented reality binocular device and tested it, together with air traffic controllers, on the control tower [5].
At the beginning of the 21st century, augmented and virtual reality technologies continue to develop actively. At the end of the first decade of the century, there are several technical vision systems for aircraft pilots from leading avionics manufacturers (for example, the Smart View complex from HoneyWell), including elements of augmented reality [6]. These systems provide the pilot with a display image containing standard indicators of flight parameters, combined with a three-dimensional representation of the terrain and with a real image of this area obtained from an infrared camera on board the aircraft. In 2010, an interface was developed using augmented reality technologies for pilots based on the Targo helmet by Israeli company Elbit Systems with virtual three-dimensional objects. The presented interface traces the flight path with virtual markers projected on the pilot's helmet shield.
The relevance of the use of augmented and virtual reality technologies in aviation is confirmed by a number of scientific articles. For example, in their article "Augmented reality in aviation", Candidate of Technical Sciences A. L. Gorbunov and Doctor of Technical Sciences E. E. Nechaev [7] suggest using augmented reality technologies in the design, production, operation, management and maintenance of aircraft. Employees of the central research institute "Dinamika" in their article tell about the near future of these technologies, noting the relevance of their application in piloting aircraft, for operators of unmanned aerial vehicles and in training complexes for aviation personnel. The article "Virtual reality in science and technology" by specialists of Bauman Moscow State Technical University emphasizes the relevance of the use of virtual reality technologies in various fields, including aviation, to facilitate the piloting of aircraft and training on auxiliary technical training tools. Examples of successful application of these technologies abroad are also given here. In 1994 in Detroit General Motors Company created a virtual reality center, which cost the concern $ 5 million. The use of the virtual reality center in the design of new cars has saved $ 80 million. The use of virtual reality technology made it possible to remove from the process of developing a new model such operations as creating a preliminary, plasticine layout, blowing a life-size model in a wind tunnel and a number of crash tests. All these manipulations are now carried out by engineers and designers in a virtual space, where the effects are exposed not to the physical, but to the electronic layout of the car.
Virtual reality allows you to significantly increase the visibility of the device being developed in an electronic layout. This is especially true for large objects. Using virtual reality, designers can design, build and test their aircraft in a virtual environment, reducing the number of test and ergonomic sets and financial costs. NASA and Boeing already have examples of such design [8].
Thus, taking into account the increasing development, saturation of the market, focusing on the scientific works of domestic and foreign specialists, examples of application in design, production, operation and training, we can confidently state the relevance of virtual and augmented reality technologies in the applied areas of civil aviation.
The aim of the work was to optimize the processes of aircraft design, evaluation of the ergonomics of the aircraft cabin, training of maintenance specialists by creating technological sets using virtual reality tools. Introduction of augmented reality technologies in the processes of operation and after-sales service of promising aircraft.

Methods and Materials
To date, the department 741.170 has been working on processing 3D models (converting CAD models into polygonal ones) from the MS-21 electronic layout. The obtained models are successfully used as initial data for obtaining illustrations for the MC-21 computer based training (CBT) [9], interactive operating training sets of aircraft systems, and a maintenance training device (MTD). Existing guidelines for the preparation of multimedia content allow to use this The technology of developing a simulator of maintenance procedures using special Unity 3D and Unreal Engine software allows you to create the necessary scenes and applications using virtual and augmented reality technologies using the same software.
Thus, the principles and technologies underlying the creation of the simulator within the range of technical training tools for the MS-21 project allow to create applications and scenes for the proposed project. Moreover, one of the key principles of the workflow is implementedthe prepared material is used several times in various applied areas. This allows you to save labor costs. And given that the process is controlled and accompanied by employees of PJSC "Corporation "IRKUT", and not by contractors, significantly save time and financial costs in the event of the need for any revision or alignment with the current version of the operational and technical documentation or the content of the electronic layout.
Today, there are many programs that implement augmented and virtual reality technologies. but buying and using a ready-made solution leads to a lot of restrictions. The operator is dependent on the developer and his vision of solving a particular problem. A number of restrictions may relate to the type of content used (as in the case of Cortona3d) or binding to a specific software environment (NX, Revit). To avoid this, more and more manufacturers are choosing to develop their own software, adapting the software product to their own operating characteristics. Unity 3D and Unreal Engine real-time "engines" are gaining more and more popularity. Along with extensive features, they offer cross-platform, state-of-the-art verbosity and high-quality developer support. A wide range of plugins allows you to work with CAD models, complex animation and a wide range of tools in single or network mode, which is especially important when designing by a group of engineers. At the same time, users can work in mixed mode, when some of them use a virtual reality set for immersion, while others interact with them from their PC screens.
Along with the opportunities offered by modern graphics engines, international experience has become an important selection criterion. Boeing and Airbus have confirmed the use of this software in a number of areas of their activities [10].

Results and Discussion
To assess the ergonomics of the aircraft cabin, a test application was created using virtual reality technology, containing the MS-21 aircraft cabin (figure 1), made in accordance with an electronic layout.   A virtual reality test set was developed to visually present design data by demonstrating 3D geometry to one or more users in real time.
As part of the demonstration of the capabilities of augmented reality technology, two test mobile applications were developed. The first illustrates the possibility of quick access to the technological map of an element when the mobile device camera is pointed at the selected unit on a simulator or a real aircraft. The second shows a 3D representation of the unit when the mobile device camera is pointed at a specific technological map.
Testing of applications developed using augmented reality technology was carried out on a Samsung Galaxy S9 mobile device running Android 9.0 OS. The hardware composition of the developed virtual reality test stand is presented in table 1. To demonstrate the possibilities of training maintenance specialists, an application was created using virtual reality technologies, which implements step-by-step assembly and disassembly of the unit structure.
A possible extension of the functionality and application of virtual reality technology is presented on the example of developed test demonstration applications.
One of these applications demonstrates the ability to display various materials (leather, glass, plastic, wood) on the example of the interior of a passenger car. For example, it can be applied in marketing, advertising, and sales. The following application serves to demonstrate the possibilities for displaying particles (smoke, fire) on the example of the interior of a room. These features can be applied when developing technical training tools for stewards.
Dynamic change of 3D models and their materials is demonstrated by the application using virtual reality technology on the example of interaction with the interior of an apartment. This solution is relevant, for example, when designing the interior of an aircraft.
Approbation of the presented projects were carried out in MAI and PJSC "Corporation "Irkut". The presented prototypes received the highest rating of the scientific and technical Council of PJSC "Corporation "Irkut", and the interest in implementing such solutions in their production processes was confirmed by manufacturers of aircraft simulators (LLC NPF "SKT" represented by the chief designer), higher educational institutions (MSTU GA represented by the Vice-Rector for research and development), aircraft manufacturers (PJSC "Corporation "Irkut" represented by the Deputy head of the design and training methodology Department) and aircraft operators (PJSC "Aeroflot" represented by the Director of the aviation personnel training Department).

Summary
Based on the results of the work carried out, the main conclusion can be drawn: the introduction of virtual and augmented reality technologies in the technological processes for the design, operation of aircraft and training of aviation personnel should be started point-by-point, with small-scale pilot projects in each direction.
It is proposed to act according to the following algorithm: 1. The definition of the interest of management departments design Bureau (Department of electronic layout and configurations of aircraft, the Department of development of training systems, Department of total Department operating support, Department of methodology design and training, after-sales service of civil aircraft, etc.) in the implementation of augmented or virtual reality in technological process. 2. Defining the boundaries of the pilot project in the following areas. Development of technical assignment. The pilot project should be carried out by employees of the design bureau, cooperation with contractors is not required at this stage. 3. Preparation of the necessary multimedia content. 4. Development of the application(s). 5. Coordination with the representative of the interested division of the functional appearance of the application, the quality of the multimedia content used. 6. Implementation of the application, conducting joint tests with a representative of the division. Correction of defects in the functional appearance of the application, if necessary. 7. Assessment of the effectiveness and preparation of a report containing the results of the tests and recommendations for further development.