Embodied Active Tactile Perception

Tactile perception plays an important role in an agent safely interacting with the environment while acquiring information about it. Bio-inspired robotics opens up possibilities for a new paradigm leveraging the morphology of the body, which filters the tactile information in physical interactions and enables investigations of new designs for embodied active tactile perception. The subjects of morphology embodied active perception and motor embodied active perception is defined and discussed in this chapter. In the scope of morphology embodied active perception, sensor optimization and sensor adaptation are further defined to describe the change of sensor morphology in the design phase and the interacting phase, respectively. More specifically, the concept of online and offline sensor adjustment is presented. Sensor optimization is solely considered in the offline process for optimization and evolution design of the sensor structure and characteristics. Sensor adaptation and motor embodied active perception are considered in the online process to actively shape the sensing process with the morphology change of the sensors themselves and the action of the body where the sensors are placed, respectively. “Design as a whole” is proposed as an inverse problem to address the sensing tasks. The design of new tactile sensors should not focus on the sensor per se but should also include design parameters for sensor optimization, sensor adaptation, and motor actions.


Introduction
Developing sensor technologies to allow an artificial agent to perceive and understand the environment has always been a frontier of research in robotics [1,2].This capability allows robots to perform in the real world and to cope with uncertainty, reacting quickly to changes in the environment.Artificial vision significantly increased the impact of robotic applications in unstructured environments owing to the advance in camera-based hardware technologies, computer vision for data processing, and learning-based solutions for recognition [3].In contrast, the development of systems to endow robots with the sense of touch is less explored due to the nature of touch elaborated on more complex sensory modalities and contact models, requiring to solve system level challenges and to implement complex control strategies to deal with physical interaction with the environment [2,4].
When we touch an object, we receive and process information from two sensory modalities: kinaesthetics and cutaneous [5].Kinaesthetics sense refers to large-scale force, motion, and configuration information, emphasizing spatial and loading realization, equivalent to what humans feel at the tendons and spindles level.Cutaneous sense reflects fine features like temperature, texture, slippage, vibration, and low-level pressure, emphasizing the resolution and sensitivity of touch.Humans rely on the mechanoreceptors underneath the skin (around 17, 000 are distributed only in the human hand) to detect and filter these various tactile features, 1292 (2023) 012007 IOP Publishing doi:10.1088/1757-899X/1292/1/012007 2 Figure 1.The schematic of a tactile-based interaction between an agent and the external environment.A conceptual classification of tactile sensors and motors is modelled in this diagram.Sensors are considered solely the sensory system itself, including receptors for detecting both cutaneous and kinaesthetic features.Motors are considered as the mechanical system where the sensors are attached.In the case of a human recognizing an object through touch, the mechanoreceptors, tissues, and skin on the hand are considered the sensory system, while the motion generated by fingers, wrist and arms control are considered motor actions.Within the framework, this chapter focuses on three subjects for embodied active tactile perception 1. sensor optimization (section.2.1), 2. sensor adaptation (section.2.2), and 3. motor embodiment (section.3). 1. sensor optimization and 2. sensor adaptation represented the active morphology change of the sensory system itself, including the modalities, materials, shape, spatial resolutions, the number of receptors, etc.In contrast, 3. motor embodiment represented the process of active perception, where the motor actions affect the perceived information of the sensory system.enabling the execution of dexterous tasks, and affective touch-based communication [6].The effective processing and interpretation of tactile information rely on the multimodal integration of both kinaesthetics and cutaneous feedback, reflecting on the challenges of developing robotic tactile sensors [7].
In addition, humans utilize not only the physical sensors but also the actions to explore and perceive via the coordination of "exteroceptive" and "proprioceptive" procedures (sensorimotor coordination process) [8,9].This adaptive process ultimately establishes various contact models between the agent and the environment, filtering the perceived touch-based information.
Touch is complex, but despite its complexity, it remains an important sensing modality which is at the base of developing robots cognitive behaviours [10,11].In the last 40+ years extensive research has been done in robotics to reproduce similar tactile function of mechanoreceptors by focusing on design of new tactile sensing technologies via capacitive [12], piezoelectric [13], piezo-resistive [14], and optical [15] sensors.However, none of these state-of-the-art solutions can reach or is even comparable to the level of tactile sensation we perceive in its biological/natural counterpart [16].
The design of these sensing technologies was aimed at the detection of different types of contact events and, in particular, focused on achieving high spatial resolutions, large static and dynamic sensing range, linearity, high sensitivity, repeatability, the durability of the sensors and, in the case of artificial skins, conformability, modularity, multimodality, networking, efficient data processing, low latency, etc. [4].
Besides the development of appropriate sensing technologies which can detect the desired contact event, a big challenge is related to the interpretation of the data coming from these sensors.As we said, the complexity of tactile perception in robotics is also related to the fact that the acquisition of tactile information is an active process which involves interaction with the environment and appropriate control strategies to deal with unexpected physical interactions.However, this interaction deeply affects the structure of the acquired data and the subsequent inference process, and it is mediated through the body.In particular, the mechanical properties of the skin, the stiffness of the joints, the morphology and degree of freedoms of the body (and its physical constraints) play an active role in shaping what is perceived by filtering the physical contact event felt by the mechanoreceptors and by allowing for particular motor actions which define the space of the interaction [17].
Active tactile perception research has investigated how humans use and select appropriate stereotyped movements (exploratory procedures taxonomy) to elicit information related to the properties of objects and has implemented the same process for improving robot's tactile perception [18], but this approach was based on a fixed taxonomy of exploration actions under the classical sense-plan-act architecture.However, as proposed in [19], the behaviour of a system comes from the dynamic coupling of the brain (control) body and environment, and if this is considered in the design process, it can lead to simplifying the inference process in the brain/controller because some control and processing aspects are distributed to the body and behaviours emerging from the interaction with the environment.
This chapter focuses on this new paradigm, i.e., the implication of embodiment in tactile perception.The schematic of a tactile-based interaction between an agent and the external environment is illustrated in Fig. 1.Three subjects of embodied active tactile perception are defined.Sensor optimization and sensor adaptation are discussed as the two subbranches of the morphology embodied active perception to describe the offline and online changes in the sensor morphology.Motor embodied active perception is discussed as an online process to exploit the role of motion in active perception.This paradigm has not yet been sufficiently explored, and this chapter wants to highlight the work that has been done so far in this domain.In particular, we will focus on the role of the morphology of the sensor, the active change of it and the mediated interaction between the body and the environment in structuring the information received by the sensory system.This not only contributes to reducing the complexity of the inference task at the brain level but can also be exploited to achieve adaptable and resilient sensors.

Morphology Embodied Active Perception
In biological systems, the variations of the sensor morphology represent the ecological performance of the organism, reflecting its unique living environment, behavioural engagement, and evolution [20].The role of sensor morphology in embodied tactile perception reflects the advantages of processing touch-related information while interacting with external stimuli.In this chapter, sensor morphology is defined as the characteristics of a sensor, including its number of receptors, shape, spatial resolutions, materials, modalities, etc.The change of these characteristics can be adopted in both sensor optimization and sensor adaptation.
The variation of the sensor morphology of an agent is selected in its evolution process as a biological agent or iterative design process as an engineering system.The 'suboptimal' choices of the sensor morphology, determined in an "offline" selecting process, ensured the survival of the agent in the suitable environment/scenario.This process is defined as sensor optimization in this chapter, reflecting recent design choices of morphology in sensor development [21].
In contrast, the term sensor adaption is defined in this chapter as the ability of an agent to actively tune its morphology to favour specific tactile signals in perception.Unlike artificial sensors, which are usually designed with a fixed characteristic during operation, the morphology of a biological sensor is not defined with a constant geometry, sensitivity, and physical properties.The morphology of the sensors is continuously adapted to the change in the environment to ensure the agent effectively detects and evaluates the stimuli regardless of the changes of the ambient conditions.This "online" process ensured the adaptation of the agent to environmental changes without the need to learn complicated motor behaviours.This process is highlighted in this chapter, reflecting recent solutions in tuning sensor morphology to adapt to the environment and adjust the perceived information [19].

Sensor Optimization
The "Offline" optimization process to determine and select sensor characteristics based on tasks or environment is essential for sensor development.The challenges of the process are exhibited with the increase in design space dimensions and complexity of interactions.The design of a single-taxel tactile sensor would only need to consider its sensory structure and its interactive cover.In contrast, an array-based tactile sensor skin would face additional challenges with spatial resolution, deformability, adaptability, and sensor coverage.
The sensor morphology design developed naturally from analysis (e.g., theoretical modelling, Finite Element Modeling) to optimization (with known design specifications).An emerging interest is also observed for sensor development based on Sim2Real approaches to iterative evaluate sensor performance with a physical simulator [22,23].Preliminary work on sensory morphology and controller has been done with a genetic algorithm to select simple sensor morphologies in the scope of a line-following robot [24].This section reviews several approaches that address the role of morphology in tactile sensors, showing how the embodiment can act as a physical filter to alter the inference process of the received tactile information.
The elastic cover of the tactile sensors is considered as a key parameter for "Offline" tuning the sensor characteristics due to its mechanical filtering effect (also known as mechanical information-coding).An early study with Shimojo [25] analyzed the low-pass spatial filtering effect of the cover, modelled the underline stress distribution as the convolution of the surface stress distribution.The finite-element method was used to calculate the filtering gain for different elastic covers, studying the effect of cover thickness, Young's Modulus, and Poisson's Ratio [25].Work in [26] studied the effects of sheltering rubber layer, applied on single-crystalline silicon 3D-force sensors.With a theoretical model to forecast the sensor response, sensor optimization can be achieved by tuning the cover to have a better receptive field size at the expense of sensitivity.The investigation carried out by Cabibihan et al. [27] shows the effects of skin thickness on discriminating the object shape (subsurface pressure profiles of flat, curved, and braille) that is in contact at the surface of the sensor.The performance of the mechanical filter ultimately depends on the environment characteristics [28].
Sensor placement and deployment are another key "Offline" parameters for altering the inference process of perceived information.This subject is particularly critical when embedding sensors with soft composite and soft robotics.The infinite degrees of freedom of the soft body act as an inherited mechanical filter when interacting with the environment.The sensor placement and deployment need to comply with the mechanical response of the soft body and utilize it for better proprioception or exteroception.Culha et al. programmed the fibre morphology of conductive thermoplastic elastomer-based strain gauge sensors to embed into soft elastic bodies [29].By analyzing the soft elastic deformations and extracting strain vectors, the programmed sensor morphology can effectively discriminate different motion patterns of the soft body [29].

Sensor Adaptation
Sensor adaptation, indicating an "Online" tuning capability of the sensor morphology, supports the process of active sensing with embodied intelligence for inference shaping.The adaptation has been well adopted in biological sensors to mitigate environmental variations.For instance, the rats control the muscle stiffness of the follicle where the whisker is connected to bias the optimal range of sensing from low to high frequencies [30].Humans change their finger stiffness and joint impedance to maximize the gained tactile information in the context of haptic explorations [31].In theory, all physical parameters for "offline" sensor optimization can be explored with "online" sensor adaptation.However, some parameters are directly associated with the construction of the sensor, resulting in a limited tuning range.He et al. proposed a tunable stiffness soft sensor that the sensing characteristics (such as sensitivity and sensing range) can be adjusted online via positive pressure [32].A limited tunable range of contact stiffness can be achieved for sensor adaptation, while the baseline sensor characteristics are provided via 3D printing elastomeric material with different stiffness for sensor optimization.A wrinkled soft tactile sensor with tunable morphology and stiffness was developed, demonstrating the potential of maximizing sensor performance by adapting morphology in a specific task, such as force sensing, shape discrimination, and texture detection [33,34].Costi et al. proposed a magneto-active elastomer filter for online stiffness tuning by placing a magnet on the endeffector, indicating an increased classification accuracy with adaptive stiffening [35].Phasechanging material such as granular jamming can also be used as an online tunable mechanical filter when placed over a tactile sensor [36].
Morphology sensor adaptation can also provide additional sensor resilience.Conventional, the reflection of sensitivity changes of polymer-based material in tactile sensing due to ageing, temperature dependence and nonlinearity is considered a critical limitation.For instance, Maiolino et al. employed temperature compensation in the tactile sensor design to address the change of capacitance of the sensor material at different temperatures [12].With an actuated adaptive sensing system, compensation can be achieved via altering the sensor morphology.An example is the soft whisker sensor system proposed by Nguyen and Nhan Huu [37], which compensates for the tactile deficiency upon the whisker being broken, torn or trimmed by changing the stiffness.Resilient sensing can also be achieved by altering the sensor spatial resolution at the software level according to the tasks.For instance, selectively reading and processing a reduced number of sensors (receptors) can mitigate sensor impairment and redundancy while reducing the computation cost.

Motor Embodied Active perception
Illustrated in the schematic shown in Fig. 1, tactile perception is an integrated process of "exteroception" and "proprioception" based on motor actions.The measured tactile information ultimately depends on the sensor's physical properties and the associated motor action when the agent interacts with the environment.The adaptation of the motor actions creates a dynamic physical reservoir that filters the tactile information for active sensing.Examples of these motor embodied perception are widely seen in nature.For instance, humans modulate finger behaviour and explore patterns better perceive touch-related information for recognition and discrimination [38].Humans also actively control the leg joint impedance and gait pattern to maintain stability while sensing in unstructured terrain [39].
Motor embodied active perception plays a significant role in the adaptation process of an agent, understanding the environment and evolving cognitive capabilities through both passive and active interaction with the environment [40].Passive interaction is considered a process for the agent to respond to active stimuli and better perceive and understand the intention of others.For instance, in the process of "being touched" with aroused "affective emotions", the agent is conditioning the motor actions based on the interpreted emotion of the agent who is performing the touch [41].The realization relies not only on the sensor spatial resolution and the type of mechanoreceptors but also on the interaction determined by the active motor actions.The subject is less explored in the field of robotics, considering the difficulties in creating a large-area tactile skin with high spatial resolution and the challenges in collecting highquality/robust training data [42].In contrast, active tactile perception is more developed in the field, demonstrating applications in environment exploration, dexterous in-hand manipulation, tactile feature recognition, and human-robot interaction [43].
Robotic whiskers are classic examples of motor embodied active perception in environment exploration [44,45].By attaching artificial whiskers to a mobile robot platform and an actuated moving mechanism, the robot can effectively increase the whisker sensing performance.For instance, each whisker module on the Shrewbot consists of a motor and shaft encoder, a 3-axis Hall effect sensor, and an embedded microprocessor [44].Adopting low latency motor control of the rhythmic motion of the whiskers in response to contact-induced stimuli, the Shrewbot constrains the sensory range while maximizing the number of whisker contacts [44].Another whisker design that utilizes material compliance instead of individual whisker motion is the TacWhisker design.By modifying a vision-based tactile sensor, TacWhisker used one motor to actuate 10 whiskers to create an open/close whisking motion for object localization [46].
In the field of object recognition with tactile information, motor embodied active perception allows information-rich data to be extracted with related actions from raw tactile signals.For instance, Kaboli et al. used a robotic hand with fingertip tactile sensors to distinguish different in-hand objects through texture properties detected via small sliding movements of the fingertips of the robot over the object surface [47].The motor action is similar to what a human does to identify an in-hand object.
The motor embodied active perception has also been demonstrated at the wrist and arm level when performing active tactile exploration with a robotic arm.Inspired by how practitioners regulate their hand and wrist motion in medical palpation to diagnose the abnormalities underneath the human skin, Scimeca et al. investigated how robot actions can augment the tactile perception in soft-body palpation [48].Validated on a robotic patient, the study shows that small changes in the motor action would significantly affect the sensor performance [49].

Designing as a whole for active perception
High-performance tactile sensing required a joint effort of both sensor morphology optimization/adaptation and strategic control of the motor actions.In the previous sections, we discussed morphology embodied active perception (including sensor optimization and sensor adaptation) and motor embodied active perception.The baseline we want to emphasize here is "designing as a whole" with sensor optimization, sensor adaptation, and motor embodiment.Considering embodied active tactile perception as the process of shaping the inference process based on the environment, both offline and online adaptation is essential for the optimization of sensing (Fig. 3).Sensor optimization is defined as the offline selection and optimization process to modify the sensor characteristics.In contrast, both sensor adaptation and motor embodiment are considered online processes, where sensor adaptation is the change of characteristics of the sensor itself, and motor embodiment is the change of motor actions of an actuator where the sensor is attached.The concept of "designing as a whole" describes sensor development as an inverse problem that goes beyond the sensor itself.Designers should understand the niche of the problem, identify the broad scope of the tasks, and then design body-sensing-action together.When considering the sensory system and the motor system as a whole, the boundary between sensor adaptation and motor embodied active perception is blurred.The difference between these two subjects is defined in this chapter for the purpose of modelling.However, in reality, the classification of sensors and motors works in a continuum domain.Both sensor adaptation with tunable sensor morphology and motor embodied active perception create an online adjustment on the mediated tactile interaction.By extending the scope and definition of the sensory system, considering the motor system as part of the active change of the sensor morphology, designers can determine the contribution of each subject to achieve the same objective of online adaptation.
It also needs to be noted that, within the framework of "designing as a whole", the designer should evaluate the performance and capability of both the sensor morphology and the motor actions.In many cases, the motor actions determined by the body of the agent provided the constraints on the level of adaptation.For instance, a single degrees of freedom (DoF) gripper can only create a single-DoF motor action to alter the embodied active sensing.With increased degrees of freedom in the motor system, the complexity of creating optimal motor embodied active perception is increased.Similar challenges are also considered in the field of sensorimotor coordination and active robot control, where the effect of learning is considered.In the case of sensing in soft robotics with infinite degrees of freedom, the role of the environment is more significant.By implementing "designing as a whole" for the sensors and motors development, the adaptation to the environment should be considered in both the online and offline phases.

Figure 2 .
Figure2.The schematic of morphology embodied active perception.Sensor morphology determines the primary contact and interaction of the agent with the environment.Both physical characteristics and sensing characteristics are considered sensor morphology in this chapter.Physical characteristics include the materials (stiffness, stretchability, etc.), shape, size, etc. Example sensing characteristics are the sensing modalities (the response of different types of receptors), the spatial resolution (distribution of the receptors), and the number of receptors.Sensor optimization refers to the iterative changes of these characteristics in the design phase where the designed morphology of the sensor remains unchanged regardless of the presence of the task-related interaction.Examples of this offline process are the iterative design-fabrication-testing engineering production or biological evolution of an animal.The design/biological structure remains unchanged for the morphology of the individual agent.In contrast, sensor adaptation refers to the online process where the sensing characteristics are tuned in association with the task-related interaction.

Figure 3 .
Figure 3.The distinction of sensor optimization, sensor adaptation, and motor embodiment in active embodied tactile perception.