Design and development of intelligent unmanned construction system for open-pit mine

China’s huge mineral outputs support the national economy and people’s livelihood. However, the mining operation is rough and the safety accident happens frequently. Consequently, the demand for unmanned mining operations is urgent, which is of great significance to safe production. Facing the complex operation scenarios of open-pit mines, an unmanned construction system framework is proposed based on intelligent construction machinery. On this basis, an unmanned excavation subsystem is designed and developed based on the hydraulic excavator, and an unmanned transportation subsystem is designed and developed based on the mining truck. Equipped with advanced sensors and controllers, the unmanned excavation subsystem and the unmanned transportation subsystem possess sensing, decision-making, and control abilities by using AI algorithms. The experiment of the unmanned construction systems is successfully conducted on the open-pit mine test site. This study provides equipment and technical support for improving the intelligent level of mine construction and ensuring safe production for the open-pit mine.


Introduction
China has numerous mineral resources and the huge annual outputs make great contributions to the national economy and people's livelihood.About 80% of the country's primary energy, 80% of industrial raw materials, 70% of agricultural production materials, and 30% of drinking water are from mineral resources.However, the mining operation is rough and safety accidents are frequent.In 2022, there were 367 mining accidents and 518 deaths.On February 22, 2023, a landslide in an open-pit coal mine killed 53 people.The demand for unmanned mine construction is quite urgent, which is of great significance to safe production.
Currently, the operation is mainly carried out by the manual operation of construction machinery, and the operator completes environmental perception, task decision-making, control execution, and collaborative scheduling.For the unmanned construction system, in order to replace operators and complete the aforementioned jobs, the construction machinery should first achieve the intelligence of a single machine, which means the ability to perceive [1][2][3], decision-making [4][5][6] and control [7][8].
There have been some studies to improve the intelligence of construction machinery.Caterpillar [9] and Komatsu [10] have equipped excavator products with intelligent auxiliary functions that can improve the user experience, safety and efficiency.On the basis of intelligent assistance, the construction machinery industry has gradually carried out some exploration of unmanned operations for single machines.Sany Group has developed remote control excavators [11] and unmanned road machine equipment [12] for highway maintenance construction.These studies have improved the ability of intelligence of single machines for construction machinery products.Unmanned operations can be realized under specific scene conditions.The development of construction machinery intelligence is promoted, contributing to unmanned construction in complex scenes.However, current works are mainly based on the improvement of the existing system by upgrading the hydraulic system and control system.Intelligent functions of the single machine are added subsequently.There is still a gap to achieve full unmanned construction in the complex scenes.
In this paper, the framework of an intelligent unmanned construction system based on intelligent construction machinery is proposed.The subsystems of intelligent unmanned construction systems are designed and developed.The experiments of unmanned excavation, unmanned transportation and collaborative operation are conducted in the open-pit mine test site.

Workflow and requirements in the open-pit mine
The mining process includes four steps, which are bursting, excavation, transportation, and unloading.The bursting step is to drill the blasting hole through the drilling rig and to load the explosive into the gun hole through the explosive mixed vehicle.Then the explosives are blasted to crush and loosen the ore.In the excavation step, the excavator takes the broken ore rock from the blast pile and then loads the material into the cargo box of the mine truck.The transportation step is that the mine truck transports the ore rock to the crushing stations, ore storage sites and other places for processing.Moreover, the stripped rock and soil are transported to the dumping site for disposal.The unloading step is that the mine truck unloads the material in the crushing station, storage yard, dumping yard, etc.Then the mine truck moves to the loading point according to the task scheduling arrangement.The bursting step is the first process, which is conducted only once.The main process of mining is the cycle of the cooperative excavation, transportation and unloading operated by the excavator and the mining truck.
Intelligent unmanned systems have shown strong advantages and potential in the fields of industry, military, and service industry [13][14][15].With the improvement of the intelligence level of a single construction machine, unmanned construction in some simple scenarios can be completed.However, as the complexity of construction scenarios increases, the intelligence of a single machine will encounter limitations.On the one hand, complex unmanned construction operations will increase the computing requirements of a single machine and the cost of computing units will become a major obstacle for commercialization.On the other hand, in some complex scenarios, multiple intelligent construction machinery cluster operations are required.It is difficult to optimize resource utilization and maximize efficiency improvement by relying only on single-machine intelligence.Therefore, the framework of the unmanned construction system based on intelligent construction machinery should integrate the characteristics of construction scenarios, cloud computing and mobile communication on the basis of establishing single-machine intelligence to realize the collaboration between construction machinery and construction scenarios.

Concept and framework of the intelligent unmanned construction system
The intelligent unmanned construction system requires deep integration of digital technologies including artificial intelligence, construction scenarios and construction machinery, as shown in Figure 1.Through the application of UAV mapping and high-precision map technology, static modelling and dynamic update of construction scenarios can be realized.Moreover, VR/AR and digital twin technologies can be utilized to establish digital construction scenarios, realizing scenario reconstruction and interaction between virtuality and reality.Inspired by Komatsu's smart construction [16], Figure 2 shows the framework of the intelligent unmanned construction system.Construction machinery realizes electrical signal control by designing or refitting the basic hardware system with wire control.Then AI technologies are applied such as fusion perception, decision planning, positioning, and control to achieve single-machine intelligence.In view of complex construction requirements, it is necessary to connect intelligent construction machinery and digital construction scenarios through intelligent networked clouds.The data fusion, intelligent monitoring, safety control and cloud computing can be realized to provide a foundation for intelligent collaborative construction of multi-machine systems.

The unmanned excavation subsystem
Figure 3 shows the unmanned excavation system, which is developed based on the Sany excavator.Equipped with IMU, pressure sensor, flow meter, camera, RTK, lidar, domain controller, electric control valve and other devices, the unmanned excavation system can realize environmental perception, decision-making, planning and motion control functions.The CHCNAV IMU-RTK GNSS is applied for localization, whose accuracy can reach the centimetre level.The object detection and semantic segmentation method based on the lidar point cloud is used for target recognition of materials, mining cards, the working device and other objects in construction scenarios.The Pointspillar model is used for the perception based on the 3D lidar point cloud.To ensure perception accuracy, at least 64-line lidar should be utilized.The working device of the unmanned acquisition system is driven by the hydraulic system, which has complex characteristics of nonlinearity, strong coupling and high time delay.Hence the dynamic model is complex, making it difficult to achieve precision motion control.In order to solve this problem, the least square method is used to identify the hydraulic system parameters, and the MPC control method is used to achieve precise motion control.

The unmanned transportation subsystem
The unmanned transportation system is developed based on the Sany mining card, and is equipped with lidar, an intelligent gateway, an unmanned computing platform, an RTK antenna, a camera, and millimeter-wave radar, as shown in

Experiment
The unmanned construction test site of the mine was designed based on a mountain.The road includes uphill and downhill sections, muddy roads, water-crossing roads, rutted roads, dusty roads, and gravel bumpy roads.As shown in Figure 5, the performance, reliability and construction tests of the intelligent unmanned construction system were conducted.The experimental results are presented in Table 1. Figure 6 shows the lidar object detection results in the open-pit mine test site.The precision rate of the lidar object detection reaches 90% and the recall rate reaches 96%.The number of takeovers per 1000 km is less than 1.The speed of the unmanned transportation subsystem with heavy-duty reaches 15 km/h, while the speed is 20 km/h under unloaded conditions.Figure 7 shows the angle control error of the unmanned transportation subsystem in the experiment.The angle control accuracy is better than 1°.The performance reliability is continuously verified in the experiment sites and customer sites.As of September 2023, the cumulative mileage of unmanned transportation subsystems has exceeded 2 million kilometres and the operated earthwork has been more than 10 million cubic.The overall transportation efficiency without safety personnel has reached 90% of the labour efficiency.
Number of takeovers per 1,000 km ≤1 The speed of unmanned transportation subsystem (heavy duty) 15 km/h The speed of unmanned transportation subsystem (unloaded) 20 km/h The precision rate of the lidar object detection 90% The recall rate of the lidar object detection 96% The overall transportation efficiency 90%

Figure 1 .
Figure 1.Concepts of the intelligent unmanned construction system.

Figure 2 .
Figure 2. Framework of the intelligent unmanned construction system.

Figure 4 .
The high resolution of the lidar leads to huge computation, which is time-consuming.Compared with the unmanned excavation subsystem, the transportation subsystem requires higher computation speed.Therefore, the 32-line lidar can meet the needs.Based on the point cloud deep learning technology, the unmanned transportation system realizes vehicle identification and detection in open-pit mine scenes, and perceives obstacle location, size, heading and other information.TensorRT is utilized for accelerating the model inference process.Facing the strict requirements for reliable and stable operation in the harsh scenarios of open-pit mines, the forward design of the wire-control chassis was carried out.The functionalities of brake-by-wire, steer-by-wire, drive-by-wire, shift and body control were developed.The CAN communication capability, wire-control function, emergency stop function and reliability were validated.

Figure 5 .
Figure 5. Unmanned construction experiment in the open-pit mine test site.

Figure 6 .
Figure 6.Lidar object detection results in the open-pit mine test site.

Figure 7 .
Figure 7. Angle control error of the unmanned transportation subsystem.