Legged locomotion over irregular terrains: state of the art of human and robot performance

Legged robotic technologies have moved out of the lab to operate in real environments, characterized by a wide variety of unpredictable irregularities and disturbances, all this in close proximity with humans. Demonstrating the ability of current robots to move robustly and reliably in these conditions is becoming essential to prove their safe operation. Here, we report an in-depth literature review aimed at verifying the existence of common or agreed protocols and metrics to test the performance of legged system in realistic environments. We primarily focused on three types of robotic technologies, i.e., hexapods, quadrupeds and bipeds. We also included a comprehensive overview on human locomotion studies, being it often considered the gold standard for performance, and one of the most important sources of bioinspiration for legged machines. We discovered that very few papers have rigorously studied robotic locomotion under irregular terrain conditions. On the contrary, numerous studies have addressed this problem on human gait, being nonetheless of highly heterogeneous nature in terms of experimental design. This lack of agreed methodology makes it challenging for the community to properly assess, compare and predict the performance of existing legged systems in real environments. On the one hand, this work provides a library of methods, metrics and experimental protocols, with a critical analysis on the limitations of the current approaches and future promising directions. On the other hand, it demonstrates the existence of an important lack of benchmarks in the literature, and the possibility of bridging different disciplines, e.g., the human and robotic, towards the definition of standardized procedures that will boost not only the scientific development of better bioinspired solutions, but also their market uptake.


Introduction
In the last decade, the robotics community has put unprecedented efforts in expanding robots' capabilities to meet the increasing needs of emerging application domains. Robots started to work in shared spaces with human users, accessing environments previously restricted to humans, like public places, collaborative industrial settings, and homes. To achieve high levels of reliability, safety and versatility in such conditions, this new generation of collaborative robots needs to demonstrate their interaction capabilities with humans and with the environment. Performance evaluation has therefore become particularly relevant in many sectors of robotics. In the field of locomotion, last years have been characterised by the advent of highly performant generations of legged robots with impressive biomimetic abilities in unstructured natural environments. However, while robotic locomotion over flat surfaces has been extensively covered in the scientific literature, few efforts have been devoted to rigorously testing locomotion abilities in non-ideal conditions [1]. Environments in which humans operate are characterized by an immense variety of irregular terrains, which pose many risks for the stability of legged systems. Exposure to these conditions can be either voluntary/predictable, as in the case of avoiding obstacles, or involuntary/unpredictable, e.g., when dealing with small surface irregularities [2][3][4].
In this paper, we performed an extensive literature review of scientific studies related to legged locomotion over irregular terrains. We reported the technical characteristics of the ecological terrains, the experimental platforms used in these studies, as well as the experimental protocols and performance indicators (PIs) used to evaluate robot abilities. We also included a revision of prior studies on human locomotion over irregular terrain, being human performance often considered the 'gold standard' for robotic legged locomotion, and a major source of inspiration for morphological, actuation and control principles [5]. With this review, we intend to provide the basic knowledge necessary to move the first steps towards a benchmarking methodology able to test and demonstrate robotic performance in out-of-thelab environments, a topic that, beside its increasing relevance in the community [6], remains still largely unexplored.

Materials and methods
This review was aimed to answer three main questions: • Which testbeds have been used to replicate ecological irregular terrain environments? • Which experimental protocols and measurements systems have been used to test robotic systems under irregular terrain conditions? • Which metrics and PIs have been used to evaluate robotic abilities? We performed various searches on Scopus scientific database between June, 2019 and June, 2022. The search strategy was determined using the AND/OR/NOT Boolean operators with different combinations of the following keywords: 'rough, uneven, unstable, soft, irregular * , terrain * , ground * , surface * , walk * , locomot * , stand * , balanc * , gait, robot * , exoskeleton * , prosth * ' The search returned 313 articles, 194 of them including robots, and 119 related to human locomotion. Ten additional articles were added from Google Scholar and PubMed databases. Five more papers were found from the reference list of relevant articles. A total of 328 articles were reviewed.
We first filtered the papers by titles and abstracts, including those with any relation with activities performed on irregular terrains. We entirely read the resulting 216 studies, and excluded those matching any of the following criteria: • Insufficiently detailed description of the irregular terrain(s) employed in the experimental setup. • Missing evaluation protocol or PIs.
• Not related to legged locomotion. This process resulted in a total of 120 papers, 20 of them related to humans and 100 to robots. The results of our review are organised in two sections, focused on human and robotic locomotion respectively.

Human locomotion over irregular terrains
Research on human walking over irregular terrains has been active during the last two decades [7], with a significant increase in the number of scientific studies in the last five years. The reviewed studies cover a wide range of objectives, ranging from more general aspects like biomechanics and energetics in healthy populations [7][8][9]   Each study was written from a different research perspective and with different aims. Protocols were slightly different to each other, but almost all shared a common structure that can be summarized in the resulting four stages: Stage 1.  figure 1).
Only half of reviewed papers reported the calculation of PIs, (see figure 2), defined as standardized metrics describing the ability of the system to perform a given task. As reported in figure 3, the different studies did not follow a standard methodology or principle to build the irregularities. However, we observed some similarities in the patterns of irregularities across the studies, such as rocky terrain replica [8, 23, 24], a

Scientific evidence
Performing gait over uneven terrain challenges the human bipedal motor control system to modify the kinematic and dynamic behaviour of the whole body to maintain balance. The most common changes observed in the different groups of subjects addressed so far (i.e., healthy, elderly, young, CP and PD patients) are an increase of thigh and lower leg muscles activation [2,9,17 Santuz et al [20] studied muscle coordination in overground, treadmill and uneven terrain during walking and running, in young and old adults. Their results showed that, both in young adults and elderlies, motor primitives are less complex in (i) running compared with walking, (ii) walking on a treadmill compared with overground walking, (iii) overground walking compared with treadmill running, and (iv) when perturbations exist compared with unperturbed locomotion.
Other evidence showed how older adults presented a decrease in balance correlated with gait adaptations [13]. Children with CP presented an impaired trunk and pelvic control and a worsening in dynamic balance when walking over uneven terrains [11]. Stroke survivors experienced an increase of ankle plantar flexion range of motion when walking through pebbled surfaces and a change in the direction of motion at the ankle joint when walking through sand [24]. Another study observed that gait parameters variance increases when walking over rough terrain with minimal shoes, but it is maintained when wearing boots [16].
Robotic prostheses. Five of the reviewed studies focused on powered prostheses. Two of them [108,109], from the same authors and focusing on ankle prostheses, proposed a 2 × 2 (inches) sections of plywood with 10 × 15 × 1 cm (length, width, height) plywood blocks stacked 0, 1, or 2 cm high in a repeating pattern. Blocks were rotated between trials to avoid repeating the same pattern. In one study [110], authors tested a transtibial prosthesis using a 3 mm thick carpet with randomly arranged triangular wooden prisms between 60 and 160 mm in length, having 26 prisms per square meter. The dimensions of the triangles in those prisms were 30 mm of base length and 15 mm of triangle height. The total surface was 8 m long and 1.5 m wide. Chiu et al [111] used a prosthesis emulator system to try a new controller whose aim was to reduce the effect of the disturbances caused by uneven terrains. The uneven terrain proposed to validate this new controller consisted of a treadmill with wooden rectangles placed at three different heights. These rectangles were 18 cm long with a width varying between 7.6 and 15 cm. Jang et al [112] also focused on developing a gait algorithm to walk through irregular terrain using impedance control as well as on designing a prosthesis that is fully prepared for this task. For this issue, a metal disk of 20 mm height was used as an obstacle to simulate uneven terrain.
Exoskeletons. No studies involving exoskeletons over irregular terrains were found.
Simulations  whereas just a few did it directly in the real world [39,52,58,63,71]. Most of these simulation approaches were aimed to improve control and/or perception abilities rather than directly quantifying locomotion performance. An exception was found in the case of papers focusing on robotic prosthesis, in which the experimental approach and metrics were quite similar to those considered in human studies.

Scientific evidence
Hexapods. When the robotic hexapods community started to address the challenge of locomotion in unstructured environments and irregular terrains [38], most results only considered two-dimensional rough terrains and were only validated in simulated environments. In 2011, Irawan et al [32] presented the first experimental tests of an hexapod robot walking on uneven terrain by using impedance control to guarantee the stability of the robot. More recently, other authors focused on improving ground force-control based navigation in these environments [33,34], while others have focused on providing these capabilities by estimating interaction forces at the robot's feet [28,30]. Some authors have also addressed the challenge of navigating in rough terrains using computer vision [37,40] and perception techniques [41,43]. Other researchers developed motion planners and foot trajectory generators to walk autonomously in unstructured environments [26,29,39,42], and developed predictive controllers to stabilize the robot while walking on uneven surfaces [31,35]. Some works also focused on allowing self-location of the hexapod robot while walking outdoors [27,40].
Most of the reviewed papers only focused on assessing the technical performance of their systems using custom-made irregular terrains, tailored to the specific characteristics of their device or algorithm, without indicating how the achieved results could extrapolate to other setups, conditions or real-life situations. The majority of the studies used steps, rocky terrains or placed blocks of different heights in levelled floors, whereas only few of them considered stairs or soft terrains.
Quadrupeds. Until a few years ago, quadrupedal robots were at a too early a stage to enable reliable locomotion over irregular terrains. For this reason, defining a benchmark, or even standard evaluation metrics, was not the main focus of the robotic community. However, in the past five years, quadrupedal systems went through huge advancements, paving the way to the deployment of robots in the real world. A crucial role in this achievement has also been played by industries. Novel reliable quadrupedal platforms have been developed by various companies such as ANYbotics AG 8 , Boston Dynamics 9 , Ghost Robotics Corporation 10 , and Unitree Robotics 11 .
Nevertheless, in extreme conditions, even these novel systems struggle. For example, extremely steep slopes are still difficult to overcome. Most of the articles test only gentle/mild slopes such as 10 • -20 • (e.g., [50,54,60,75]). A 30 • slope was tested in one paper [60], and a V-shaped walls with 50 • slope is considered in another study [49]. Time to failure was also considered during climbing stairs, with failure averagely experienced after 18 steps [66].
Experimental evidence showed that moving in severely harsh terrains often leads to falls. Therefore, recovery policies appear very important be taken into account to enable reliable locomotion over irregular terrains. Literature proposes methods that, starting from a random initial configuration, allow the robot to stand up and continue the task [72][73][74]. However, these techniques consider only flat terrain scenarios. Fall recovery from irregular terrains is still highly overlooked.
Bipeds. When robotic bipeds' performance on uneven grounds began to be evaluated, it was successfully tested with the help of a stick [85] or while touching a handrail [95]. In the following years, other challenging terrains such as staircases, slopes, ditches [3,89] or irregular rocky grounds [84] were considered, although tests were only performed in simulations, where the robot had prior knowledge of the irregularities.
Other types of irregular terrains composed of little steps such as wooden boards placed on the ground were also considered. These studies included simulations [86] and real experiments [88] applying the widely used zero moment point (ZMP) control method. In other two studies [79,91], environments with slopes up to 20 degrees were simulated using different strategies such as the CoM trajectory computation and ZMP methods. Later, walking on slopes was successfully executed through CoM adjustments and trajectory planning in real experiments without prior knowledge of the terrain characteristics [90,94]. Better results were recently achieved by planning the CoM height variations in irregular terrains [87] and stairs [92], whereas more recently, stable walking on inclined surfaces was achieved by controlling the biped's torso angular-pitch velocity using IMUs [77] and gyroscopes [78]. Real-time terrain estimation without prior terrain information was furtherly investigated, first in irregularities composed by little steps [83] and then with simplified slopes [81], joining prediction land time and expected ground reaction forces (GRF). Recently, an additional step was achieved using a GRF control scheme, which allowed fast traversal of uneven terrains without any prior knowledge of the real experimental setting [87]. Only one study [93] evaluated stability performance in a sky-type gait task. For this matter, they proposed a stability margin to choose between different step sequences. Other authors [105,106] focused on identifying and classifying ground materials and surface transitions using sensors located at the robot's feet to automatically adjust biped controllers to the specific terrain conditions.
In the revised literature, robotic systems were generally able to overcome the proposed terrain irregularities both in simulations and real tests. Most of the studies evaluated the system performance by looking at the effectiveness of their control method to overcome the considered irregularity instead of proposing performance metrics.
Different strategies were applied to determine the motion stability in the control loop such as the ZMP method that determines whether the robot CoP is inside the region of the support leg [76,79,84,88,89,101,102]. Other methods defined the motion stability with the CoM trajectory [85,87,90,92,94,100] or joint angles [83,104,107]. Finally, control stability was also addressed by the capacity of the control system to reduce GRF since high contact forces are associated with bouncing, leading to instabilities [81,86].
Only a few studies focused on describing and comparing the quality of the walk across different conditions and systems. Wang et al [93] was claiming to discriminate the best step sequence by looking at the stability performance of gait using a stability margin. A set of proposed parameters affecting stability was also presented. Among them, the foot length and width and the step length showed good potential to be applicable across robotic systems. Walking speed was also taken as a velocity stability criterion [103]. In another work [76], a stable run was defined and compared between controllers by looking at the robot angular acceleration, which was the result of reading robot vibration that tends to be stable. Concerning slopes, two studies included the number of robot steps as PI. In one paper [96] the performance was assessed through the number of steps required to overcome different slopes. In another work [97] the number of continuous steps before failure was considered as a PI of stability. Both results relied on simulations. Energy efficiency was included in two works [97,98].
Although the high success rate in overcoming irregularities, part of the results was obtained in simulation, where the robot had prior information about the terrain characteristics or by using control systems designed and evaluated to overcome some very specific tasks, raising doubts about their effectiveness in even similar but different kinds of terrain.
Robotic prosthesis. We found just a few studies characterizing gait on uneven terrain using robotic prostheses. Curtze et al [110] studied how amputees managed to control dynamic stability while walking over irregular terrain. Authors observed that temporal gait parameters when walking through irregular terrains showed no significant differences with respect to level ground walking. Besides, no change in lateral MoS was found. These facts led to the conclusion that transtibial amputees choose not to increase stability by increasing the step width but by means of lateral velocity of arm swing.
Later, Shultz et al [109] also focused on dynamic stability over irregular terrains. Authors developed a controller aimed at improving task performance, which was refined in a further study in 2018 [108]. These studies showed that ankle angles vary more than knee angles when walking on irregular terrain, while ankle moments remain quite invariant, leading to a decrease of internal quasi-stiffness.
In 2021, two studies [111,112] focused their work on studying the improvements of their controllers over uneven terrains. Chiu et al [111] observed a reduction of ankle torque variability in the sagittal and frontal plane but concluded that this was not enough to overcome the disturbances produced by the terrain irregularity. Jang et al [112] concluded that their prosthesis was able to adapt to the ground in the coronal plane, maintaining stability while walking through uneven terrain.
Others. Within this broad category there is a clear lack of homogeneity with respect to the ability of the different systems to navigate irregular terrains.
The authors of one study [114] showed the ability of a salamander-like robot to climb stairs of up to 10 cm height and 70 cm length, and holes of up to 10 cm depth. Later in 2020, Ishizono et al [113] showed a salamander robot could walk over semispheres of 8 mm and 12 mm radius lined alternately. Inagaki et al [117] developed a novel locomotion control scheme for centipede-like multilegged robots which allowed locomotion over steps of up to 0.2 m in a simulated environment, whereas more recently Ozkan-Aydin et al [116] presented a centipede robot able to climb over blocks of 10 × 10 cm, slopes up to 40 deg. and steps up to 15 cm. In another study [119], experiments showed that a modular snake robot could creep over steps of up to 7 cm, whereas Badran et al [118] showed their snake robot could climb over slopes up to 30 deg. Marvi et al [120] showed a snake robot that was able to ascend sandy slopes close to the angle of maximum slope stability. Zhu et al [115] presented a selfreconfigurable robot able to get over obstacles up to its own height, both in a simulated and physical environment, whereas Arora et al [121] showed a simulated tread robot could climb over bump-like obstacles up to 1.2 m.

Human locomotion over irregular terrains
The reviewed papers on human locomotion showed a great variety in the type and number of subjects included, as well as in the experimental design. For instance, there is a clear lack of studies that include subjects with diseases or injuries. These are needed in order to extend the knowledge on the consequences of the limitations imposed by the motor or cognitive restrictions over complex situations. Such evidence can provide useful information for robotic systems, e.g., the identification of cause-effect relationships between number of degrees of freedom, actuation typology or control strategies on the resulting performance.
Despite the huge number of papers related to human locomotion over irregular terrains [118], we only found 19 papers that were of sufficient interest for this review article, i.e., providing sufficient details on the setup or experimental protocols. Most authors focused on assessing performance under insufficiently described terrain conditions (as summarized in figure 3), showing the low relevance that the terrain setup has for the researchers. These results also highlight how the current experimental design approaches are limiting the replicability and relevance of the experiments performed under the presence of irregular terrains with humans, therefore hindering a truthful and efficient comparison across studies.

Robotic locomotion over irregular terrains
Robotic locomotion on irregular terrain has been less investigated when compared to human studies. The information on the setup configuration is often lacking or incomplete. Relaxing the importance of the terrain setup in the first phases of development of a robot may be acceptable. However, it is erroneous and misleading to state that a robot is prepared to deal with irregular grounds when it has been only tested in a set of simplified irregularities that are not properly described nor evaluated against real-case scenarios.
Considering that most robots are designed to work in close cooperation with humans, e.g., in everyday life scenarios, factories or search & rescue missions, such lack of rigor in lab testing could seriously compromise their safety and performance when used in real-world conditions. This also calls the attention to the lack of a common definition of 'irregularity' and how it should be replicated in laboratory. For instance, some studies consider that even only one step consisting of any object with long and thin rectangular shape is an irregular terrain [83,86,88,115,119] while others consider that there should be more than one step to be deemed as an irregular terrain. Despite the apparent similarities on the terrain typologies (see figure 4), all of them are quantitatively different in size, height and/or distribution over the surface, highlighting the lack of standards in this field. Another important aspect to consider is that, since robots can be different in size and weight, the testing setups should be normalized to guarantee an objective comparison among different systems. Apart from the terrain setups, we noticed a clear lack of common protocols and PIs, which impedes to determine how well the robot is able to navigate a terrain in comparison to other solutions. Most authors still use a YES/NO criterium to indicate the level of achievement of a task. This situation makes it very difficult to correctly compare the performance of the different technologies, and more importantly, to assess the readiness level of the prototypes prior to market introduction.
We also observed that most studies using robots are centred on software development and perception techniques-indeed necessary to detect and overcome the irregularities-but not on evaluating the actual resulting locomotion performance on such terrains. As such, most of these experiments are carried out in simulation environments. However, modelling contacts occurring during locomotion over irregular terrains introduces significant inaccuracies, leading to the notorious 'reality gap' [122]. Specialized techniques [123,124] are typically required to reduce this gap. Only very recently we could witness examples of legged, mostly quadruped, robots able to overcome complex ecological terrains in real world conditions, most of them resulting in commercially available solutions.
Remarkably, in the field of robotic exoskeletons, we could not find any study on complex irregular terrains. This is possibly due to the fact that so far, the great majority of lower limb exoskeletal solutions are still confined to controlled (e.g., flat) terrains [1].
In conclusion, the benchmarking of robotic performance in complex environments is currently at a very early stage, with some valuable exceptions in the quadrupedal robotic field. Now that robots are operating out of the lab, there is a clear need of a common methodology to test and compare robotic systems on high-fidelity replications of complex reallike terrains, together with methods to predict performance of these systems when used in real-world scenarios.
This review shows an increasing interest of the community in understanding how the presence of an irregular terrain affects the performance of overground legged systems, both in the case of biologic systems, such as humans, and artificial devices. However, the formal definition of irregular terrain appears as an unsolved research question so far. There is no clear standard regulating the characteristics of such types of conditions, which leads to several problems when evaluating human or robotic locomotion performance over these terrains. A first step in this direction has been taken by Torres-Pardo et al [125], who proposed a standardized test method able to reproduce a variety of irregularities, by using a modular and replicable 'Lego-like' approach. This work has led to the first formal pre-standard published by CEN CENELEC [126]. The lack of prior work on standardizable experimental methodologies, protocols and setups to assess locomotion capabilities should be urgently addressed to ensure the comparability of the experiments by different teams and systems worldwide. We identified some common procedures across the reviewed papers, mostly in the human field. However, further research on reproducible protocols, metrics, testbeds and measurement setups is needed in order to reach an agreement in the community, following the example of other international consortia, e.g., the European Project EUROBENCH [127].
It is worth mentioning the fact that the great majority of works have realized experiments in the lab to demonstrate real-world performance. Although lab-based tests are necessary to evaluate system's performance under the presence of irregular terrains in a controlled and standardized way, they could still not be representative of the conditions found in realworld scenarios, which should be the ultimate goal of this research field. In our opinion, a promising research direction is addressing the question of how, and to what extent, lab experiments are able to predict real-life performance.

Conclusions
An increasing number of legged systems have begun to operate in out-of-the-lab environments, sharing spaces with humans. In the present systematic review, we explored and analysed the methods employed in the literature to evaluate legged locomotion over irregular terrains, as well as the main scientific evidence resulting from these studies. We summarized the protocols, scenarios and PIs used by the community to characterize human and robotic gait performance. Our aim was to help those researchers interested in the development of standardized testbeds, protocols, and metrics to study, assess and compare legged locomotion in complex and realistic ground conditions. This systematic review proves a lack of agreement, details, and specifications when conducting experiments involving irregular terrains. There are poorly or non-explored areas, such as powered prostheses and exoskeletons. In addition, many researchers tried their systems via simulations instead of in real-life scenarios.
Being able to benchmark the ability and safety of these assistive devices over real-world scenarios is in our opinion a keystone in the decision-making process, not only during the technical development, e.g., testing specific bioinspired designs, but also to verify how these solutions can meet real users' needs.