Living Machines: From Biological Role Models to Soft Machines

Hippocampal reverse replay, a phenomenon in which recently active hippocampal cells reactivate in the reverse order, is thought to contribute to learning, particularly reinforcement learning (RL), in animals. Here, we present a novel computational model which exploits reverse replay to improve stability and performance on a homing task. The model takes inspiration from the hippocampal-striatal network, and learning occurs via a three-factor RL rule. To augment this model with hippocampal reverse replay, we derived a policy gradient learning rule that associates place-cell activity with responses in cells representing actions and a supervised learning rule of the same form, interpreting the replay activity as a 'target' frequency. We evaluated the model using a simulated robot spatial navigation task inspired by the Morris water maze. Results suggest that reverse replay can improve performance stability over multiple trials. Our model exploits reverse reply as an additional source for propagating information about desirable synaptic changes, reducing the requirements for long-time scales in eligibility traces combined with low learning rates. We conclude that reverse replay can positively contribute to RL, although less stable learning is possible in its absence. Analogously, we postulate that reverse replay may enhance RL in the mammalian hippocampal-striatal system rather than provide its core mechanism.

Insects are highly capable walkers, but many questions remain regarding how the insect nervous system controls locomotion. One particular question is how information is communicated between the 'lower level' ventral nerve cord (VNC) and the 'higher level' head ganglia to facilitate control. In this work, we seek to explore this question by investigating how systems traditionally described as 'positive feedback' may initiate and maintain stepping in the VNC with limited information exchanged between lower and higher level centers. We focus on the 'reflex reversal' of the stick insect femur-tibia joint between a resistance reflex (RR) and an active reaction in response to joint flexion, as well as the activation of populations of descending dorsal median unpaired (desDUM) neurons from limb strain as our primary reflex loops. We present the development of a neuromechanical model of the stick insect (Carausius morosus) femur-tibia (FTi) and coxa-trochanter joint control networks 'in-the-loop' with a physical robotic limb. The control network generates motor commands for the robotic limb, whose motion and forces generate sensory feedback for the network. We based our network architecture on the anatomy of the non-spiking interneuron joint control network that controls the FTi joint, extrapolated network connectivity based on known muscle responses, and previously developed mechanisms to produce 'sideways stepping'. Previous studies hypothesized that RR is enacted by selective inhibition of sensory afferents from the femoral chordotonal organ, but no study has tested this hypothesis with a model of an intact limb. We found that inhibiting the network's flexion position and velocity afferents generated a reflex reversal in the robot limb's FTi joint. We also explored the intact network's ability to sustain steady locomotion on our test limb. Our results suggested that the reflex reversal and limb strain reinforcement mechanisms are both necessary but individually insufficient to produce and maintain rhythmic stepping in the limb, which can be initiated or halted by brief, transient descending signals. Removing portions of this feedback loop or creating a large enough disruption can halt stepping independent of the higher-level centers. We conclude by discussing why the nervous system might control motor output in this manner, as well as how to apply these findings to generalized nervous system understanding and improved robotic control.

Control of forces is essential in both animals and walking machines. Insects measure forces as strains in their exoskeletons via campaniform sensilla (CS). Deformations of cuticular caps embedded in the exoskeleton excite afferents that project to the central nervous system. CS afferent firing frequency (i.e. 'discharge') is highly dynamic, correlating with the rate of change of the force. Discharges adapt over time to tonic forces and exhibit hysteresis during cyclic loading.

In this study we characterized a phenomenological model that predicts CS discharge, in which discharge is proportional to the instantaneous stimulus force relative to an adaptive variable. In contrast to previous studies of sensory adaptation, our model (1) is nonlinear and (2) reproduces the characteristic power-law adaptation with first order dynamics only (i.e. no 'fractional derivatives' are required to explain dynamics). We solve the response of the system analytically in multiple cases and use these solutions to derive the dynamics of the adaptive variable. We show that the model can reproduce responses of insect CS to many different force stimuli after being tuned to reproduce only one response, suggesting that the model captures the underlying dynamics of the system. We show that adaptation to tonic forces, rate-sensitivity, and hysteresis are different manifestations of the same underlying mechanism: the adaptive variable. We tune the model to replicate the dynamics of three different CS groups from two insects (cockroach and stick insect), demonstrating that it is generalizable. We also invert the model to estimate the stimulus force given the discharge recording from the animal. We discuss the adaptive neural and mechanical processes that the model may mimic and the model's use for understanding the role of load feedback in insect motor control. A preliminary model and results were previously published in the proceedings of the Conference on Biohybrid and Biomimetic Systems.

In the field of soft robotics, pneumatic elements play an important role due to their sensitive and adaptive behavior. Nevertheless, the rapid prototyping of such actuators is still challenging since conventional 3D printers are not designed to fabricate airtight objects or to specify their bending behavior by combining materials of different stiffness. In order to address this challenge, a tool changing multi-material 3D printer has been constructed, which can be equipped with various print-heads fitted to the specific application. By alternately processing filaments with varying mechanical properties, a series of pneumatic elements was produced. The actuators were printed in thermoplastic polyurethane with shore hardness A70 for flexible parts and D65 for stiff parts. A novel procedure for the feature adaptation of the flow rate allowed the fabrication of vertically printed flexible membranes with a thickness of just 500 μm. This way the bending and expanding printed structures can all be actuated with a pressure of 100 kPa or less. Furthermore, a new kind of generic actuator that is customizable to specific tasks and can perform complex motion behavior was designed. All together, these actuators demonstrate the high potential of the developed platform for further research on and production of soft robotic elements and complex pressurized systems.

Plants translate wind energy into leaf fluttering and branch motion by reversible tissue deformation. Simultaneously, the outermost structure of the plant, i.e. the dielectric cuticula, and the inner ion-conductive tissue can be used to convert mechanical vibration energy, such as that produced during fluttering in the wind, into electricity by surface contact electrification and electrostatic induction. Constraining a tailored artificial leaf to a plant leaf can enhance oscillations and transient mechanical contacts and thereby increase the electricity outcome. We have studied the effects of wind-induced mechanical interactions between the leaf of a plant (Rhododendron) and a flexible silicone elastomer-based artificial leaf fixed at the petiole on power output and whether performance can be further tuned by altering the vibrational behavior of the artificial leaf. The latter is achieved by modifying a concentrated mass at the tip of the artificial leaf and observing plant-generated current and voltage signals under air flow. In this configuration, the plant-hybrid wind-energy converters can directly power light-emitting diodes and a temperature sensor. Detailed output analysis has revealed that, under all conditions, an increase in wind speed leads to nearly linearly increased voltages and currents. Accordingly, the cumulative sum energy reaches its highest values at the highest wind speed and resulting oscillations of the plant-artificial leaf system. The mass at the tip can, in most cases, be used to increase the voltage amplitude and frequency. Nevertheless, this behavior was found to depend on the individual configuration of the system, such as the leaf morphology. Analysis of these factors under controlled conditions is crucial for optimizing systems meant to operate in unstructured outdoor scenarios. We have established, in a first approach, that the artificial leaf-plant hybrid generator is capable of autonomously generating electricity outdoors under real outdoor wind conditions, even at a low average wind speed of only 1.9 m s⁻¹.

Insect visual navigation is often assumed to depend on panoramic views of the horizon, and how these change as the animal moves. However, it is known that honey bees can visually navigate in flat, open meadows where visual information at the horizon is minimal, or would remain relatively constant across a wide range of positions. In this paper we hypothesise that these animals can navigate using view memories of the ground. We find that in natural scenes, low resolution views from an aerial perspective of ostensibly self-similar terrain (e.g. within a field of grass) provide surprisingly robust descriptors of precise spatial locations. We propose a new visual route following approach that makes use of transverse oscillations to centre a flight path along a sequence of learned views of the ground. We deploy this model on an autonomous quadcopter and demonstrate that it provides robust performance in the real world on journeys of up to 30 m. The success of our method is contingent on a robust view matching process which can evaluate the familiarity of a view with a degree of translational invariance. We show that a previously developed wavelet based bandpass orientated filter approach fits these requirements well, exhibiting double the catchment area of standard approaches. Using a realistic simulation package, we evaluate the robustness of our approach to variations in heading direction and aircraft height between inbound and outbound journeys. We also demonstrate that our approach can operate using a vision system with a biologically relevant visual acuity and viewing direction.

We present a hybrid approach to manufacturing a new class of large-scale self-shaping structures through a method of additive fabrication combining fused granular fabrication (FGF) and integrated hygroscopic wood actuators (HWAs). Wood materials naturally change shape with high forces in response to moisture stimuli. The strength and simplicity of this actuation make the material suitable for self-shaping architectural-scale components. However, the anisotropic composition of wood, which enables this inherent behavior, cannot be fully customized within existing stock. On the other hand, FGF allows for the design of large physical parts with multi-functional interior substructures as inspired by many biological materials. We propose to encode passively actuated movement into physical structures by integrating HWAs within 3D-printed meta-structures with functionally graded stiffnesses. By leveraging robotic manufacturing platforms, self-shaping biocomposite material systems can be upscaled with variable resolutions and at high volumes, resulting in large-scale structures capable of transforming from flat to curved simply through changes in relative humidity.