Research on Robotic Flexible Peg-in-hole Assembly Technology

Peg-in-hole assembly is one of the more typical tasks in the machining industry. Although robotic peg-in-hole assembly technology has been widely used in practical production, achieving efficient and reliable flexible peg-in-hole assembly by robots still poses significant challenges. The analysis of robotic flexible peg-in-hole assembly from a control point of view is presented. Firstly, the process of robot flexible peg-in-hole assembly is introduced. Then the application of traditional model-based assembly control is described, on the basis of which learning-based intelligent assembly control is discussed. The combination of intelligent methods and traditional methods will become an important future development trend, injecting new vitality into robotic flexible peg-in-hole assembly.


INTRODUCTION
Peg-in-hole assembly is a relatively typical and common type of industrial task for automated assembly tasks.Over the past decades, automated assembly has been a challenging research area, and highperformance shaft hole assembly is even more challenging and topical.For example, in 2021, Song et al. [1] from the Beijing University of Posts and Telecommunications pointed out that the assembly of parts using robots has long been one of the research hotspots.The biggest feature of the peg-in-hole assembly task is that it needs to contact the external environment, and the simple control of the position of the robot cannot meet the task requirements.
Therefore, this paper analyses and studies the process of flexible shaft-hole assembly of robots, followed by an analysis of assembly control based on traditional models, based on which intelligent assembly control based on learning is investigated.It is found that the combination of intelligent methods and traditional methods is one of the important directions for the future development of pegin-hole assembly control, which is of great value for expanding the application of robots in the field of assembly.

THE PROCESS OF PEG-IN-HOLE ASSEMBLY
The process of single-shaft hole assembly can be divided into various states according to the variable exposure situations.The process can usually be divided into three phases, namely the approach phase, the hole search stage, and the jack phase.The insertion process tends to have a jamming problem in the initial hole entry state of the shaft.In 2022, Lu et al. [2] from Guangxi University further refined the jamming problem by dividing the hole contact stage into single-point contact and two-point contact, established geometric and mechanical models for the two different contact states, and performed the corresponding mechanical analysis.In single-axis hole assembly, if the clearance is large and the hole alignment has a high degree of accuracy, the shaft can be fitted into the hole; in multi-axis hole assembly, the contact situation is more complex than in single-axis holes and it is difficult in accomplishing assembly assignment with hole alignment alone.
Based on the motion of the component during assembly, Watson et al. [3] at the University of Colorado in 2020 divided peg-in-hole assembly into two stages.For both the hole (or shaft) search and loading phases of the assembly process, Watson et al. proposed two specification algorithms, a helixbased search algorithm, and an inclined loading algorithm.These algorithms are a good reference for the initial entry of a shaft into a hole.In 2020, Jiang et al. [4] from the Harbin Institute of Technology sorted out the robot peg-in-hole assembly method and divided the process of robot assembly into four stages: the approaching stage, the searching stage, the moving stage, and the orientating stage.The process is shown in Figure 1.The contact states of multi-axis bore assemblies are more diverse and difficult to model accurately.The assembly process requires more complex assembly processes and assembly skills, which is why artificial intelligence methods such as reinforcement learning are beginning to shine in the field of assembly.

ASSEMBLY CONTROL BASED ON CONVENTIONAL MODELS
Conventional model-based assembly control requires the creation of a model of the forces during the assembly process in the insertion phase and the use of the model to achieve precise alignment of the shaft bore and thus complete the assembly task.
Position control has a high degree of stiffness and using only position control during the insertion phase can easily damage or break parts.Active soft control can control the contact force between the parts, making the assembly flexible, which helps to protect the parts during the assembly process.
In 2022, Li et al. [5] addressed the dual-axis hole jack control problem by first analyzing the jacking process and contact state of the dual-axis hole jack, as shown in Figure 2, followed by designing an impedance controller based on impedance control to achieve a soft jacking process in the presence of contact force feedback.Model-based assembly control is strongly dependent on the modeling accuracy of the contact model during the insertion phase and is less adaptable to different tasks.For multi-axis bore parts, whose complex contact states are even more difficult to model accurately and sense online, and model-based assembly control is difficult to achieve satisfactory results.

LEARNING-BASED INTELLIGENT ASSEMBLY CONTROL
To reduce the dependence on models and improve the adaptability of assembly control, researchers have developed a range of intelligent control methods for assembly.For the last few years, with the rapid progress of artificial intelligence, learning methods such as reinforcement learning have been introduced into the field of assembly and have received widespread attention from researchers.

Reinforcement learning-based assembly control
In an unstructured environment, humans can use the experience they have learned to perform the task of borehole assembly.Inspired by this, industrial robots can learn axle hole assembly skills autonomously during the assembly process.The application of reinforcement learning algorithms in robotics is to construct an intelligent body that learns control strategies for robotics intending to obtain the maximum reward from the environment for the task at hand.Reinforcement learning can therefore be used in a robot to allow the robot to continuously explore and learn assembly skills from assembly, without the need to artificially transfer human assembly experience into the execution program of robot .
For the past few years, researchers have introduced reinforcement learning into peg-in-hole assembly to optimize assembly skills.For example, in 2019, Xu et al. [6] from Tsinghua University introduced reinforcement learning into the automatic robot assembly of multi-axis holes.Xu et al. used the shaft hole assembly task as a Markov decision process with an actor-critic reinforcement learning network and presenting a model-driven depth-determining policy gradient (MDDPG) to obtain a policy using reinforcement learning trained in a virtual environment without explicitly analyzing the contact state and to migrate it to the real environment to achieve automatic robotic assembly of a two-axis hole.Feng et al. [7] from Guangxi University proposed a deep deterministic policy gradient (DDPG) variable parameter conductance control algorithm.The test results are shown in Table 1.Combining reinforcement learning with traditional methods facilitates the advantages of traditional methods and compensates for the low efficiency of reinforcement learning.Hoppe et al. [9] from the Bosch Central Research Institute in Germany combined model-free RL with planning to generate information data for operational tasks in 2019.In addition, Khader et al. [10] introduced variable impedance control (VIC) to reinforcement learning and used the cross entropy (CET) method to optimize the sampling distribution to solve the full-time stability problem of reinforcement learning.The above-mentioned combination of traditional methods and reinforcement learning is an important reference for the automatic robot assembly of multi-axis holes.

4.2.Multi-modal combination of skills learning assembly control
A priori knowledge contributes to the quality and efficiency of assembly.How a priori knowledge can be organically integrated into learning has been one of the key areas of effort for researchers.
Combining multiple learning styles is beneficial for enhancing assembly skill learning.Currently, researchers are beginning to experiment with combining imitation learning and reinforcement learning to exploit the respective advantages of imitation and reinforcement learning at different stages of skill learning.In 2020, Cho et al. [8] from Hanyang University, South Korea, proposed a method.This motor skill is represented as a cascade of HMM and DMP.HMMs are used for action selection and DMPs for motion generation.The modeled classes are used as familiar classes and the initial motor skills are refined to refine the robot's response to the familiar classes.The unmodeled classes are treated as nonfamiliar classes and the skills are generalized to address the robot's responses to non-familiar classes.Using this framework, the robot achieves automatic assembly of multiple shaped shaft holes.
Although rough models are subject to large errors, they also contain very useful knowledge and can be effective in reducing aimless exploration in the learning process and increasing the efficiency of skill acquisition.Therefore, combining modeling with learning is also a very important avenue that has received attention from researchers.Song et al. [11] from Zhejiang University of Technology proposed a robot axle hole assembly strategy based on a combination of geometrically constrained physical models and HMM.First, by establishing the geometric and mechanical models of the shaft-hole in each contact state, the geometric constraints and force characteristics in the part assembly process were analyzed to obtain the theoretical assembly trajectory for the shaft-hole assembly strategy.Then, the HMM-GMR model is obtained by using a few-sample teaching and learning method, and the actual robot position is input into the pre-trained HMM-GMR model to obtain the expected force in the assembly process, and the theoretical trajectory is compensated by a conductance controller to achieve the tracking of the expected contact force, to ensure the exactness and robustness of the contact motion in the actual process of assemble.The figure below shows the attitude of the shaft and bore in the uncontacted phase of the assembly process.Figure 3 shows the attitude of the peg and hole during the uncontacted stage of the process.Assembly experiments were carried out for shaft bore parts with a minimum clearance of 0.16 mm, the results of which are shown in Table 2. Strategy 1 is to achieve the desired contact force tracking by the HMM-GMR model through the conductance controller, which ensures accurate and stable contact motion during the actual assembly process and reduces the repeated position adjustment process, resulting in an assembly success rate of 96% with few samples and large positioning deviations.Strategy 2, based on the physical model, guaranteed assembly success rate of 92% due to the robot's ability to make repeated positional adjustments with the help of inter-part constraints, but because of the repeated calculations and adjustments, the average completion time reaches 27 s and the average maximum contact force exceeds 29 N. Strategy 3, based on the t-GMM demonstrative learning, has an average completion time of 13.8 N. The shortest completion time was 13.8 s, but due to the small training sample relative to the deviation range, the assembly success rate was not guaranteed at 64%.This shows that the combination of the physical model and HMM can effectively improve the training efficiency of reinforcement learning and enhance the adaptability and flexibility of the assembly.Combining reinforcement learning-based neural network controllers with other intelligent controllers is therefore an important future development direction for reinforcement learning-based motion control systems for axis-hole assembly.Combining a model-based approach with an artificial intelligence-based approach helps to complement each other's strengths and is one of the important directions for further development of multi-axis hole assembly control.

CONCLUSION
With the rapid development of artificial intelligence technology, the introduction of learning mechanisms such as machine learning and reinforcement learning into robot shaft hole assembly systems has become an important development trend and hotspot for peg-in-hole assembly research in recent years.However, there is still a wide gap between robotic automatic axle hole assembly and humans.From the perspective of skill learning, on the one hand, imitation learning focuses too much on motion trajectory and less on motion intention and environmental constraints; on the other hand, reinforcement learning is only applied on local steps, far from optimizing the whole assembly process.From the perspective of knowledge use, human a priori knowledge is difficult to incorporate into the learning mechanism, resulting in low learning efficiency.The solution to the above problems will certainly accelerate the pace of assembly intelligence.
Multi-axis hole assembly is more complex compared to single-axis hole assembly and the research is still in its infancy, with less relevant studies.The complexity of the contact states and the skillfulness of the assembly actions during multi-axis hole assembly dictate that automated multi-axis hole assembly is a good target for research.Therefore, the combination of a model-based approach with a learningbased artificial intelligence approach, which incorporates a priori knowledge into the learning mechanism, will be an important future development trend for the intelligence of axial hole assembly, taking multi-axis hole assembly as the research object.

Figure 1 .
Figure 1.The process of robot peg-in-hole assembly.

Figure 2 .
Figure 2. The force on the contact process of the double axle hole.

Table 1 .
Testing result without stochastic noise.

Table 2 .
Experimental results of shaft bore assembly under different strategies