-
Brain-Body-Task Co-Adaptation can Improve Autonomous Learning and Speed of Bipedal Walking
Authors:
Darío Urbina-Meléndez,
Hesam Azadjou,
Francisco J. Valero-Cuevas
Abstract:
Inspired by animals that co-adapt their brain and body to interact with the environment, we present a tendon-driven and over-actuated (i.e., n joint, n+1 actuators) bipedal robot that (i) exploits its backdrivable mechanical properties to manage body-environment interactions without explicit control, and (ii) uses a simple 3-layer neural network to learn to walk after only 2 minutes of 'natural' m…
▽ More
Inspired by animals that co-adapt their brain and body to interact with the environment, we present a tendon-driven and over-actuated (i.e., n joint, n+1 actuators) bipedal robot that (i) exploits its backdrivable mechanical properties to manage body-environment interactions without explicit control, and (ii) uses a simple 3-layer neural network to learn to walk after only 2 minutes of 'natural' motor babbling (i.e., an exploration strategy that is compatible with leg and task dynamics; akin to childsplay). This brain-body collaboration first learns to produce feet cyclical movements 'in air' and, without further tuning, can produce locomotion when the biped is lowered to be in slight contact with the ground. In contrast, training with 2 minutes of 'naive' motor babbling (i.e., an exploration strategy that ignores leg task dynamics), does not produce consistent cyclical movements 'in air', and produces erratic movements and no locomotion when in slight contact with the ground. When further lowering the biped and making the desired leg trajectories reach 1cm below ground (causing the desired-vs-obtained trajectories error to be unavoidable), cyclical movements based on either natural or naive babbling presented almost equally persistent trends, and locomotion emerged with naive babbling. Therefore, we show how continual learning of walking in unforeseen circumstances can be driven by continual physical adaptation rooted in the backdrivable properties of the plant and enhanced by exploration strategies that exploit plant dynamics. Our studies also demonstrate that the bio-inspired codesign and co-adaptations of limbs and control strategies can produce locomotion without explicit control of trajectory errors.
△ Less
Submitted 4 February, 2024;
originally announced February 2024.
-
The utility of tactile force to autonomous learning of in-hand manipulation is task-dependent
Authors:
Romina Mir,
Ali Marjaninejad,
Francisco J. Valero-Cuevas
Abstract:
Tactile sensors provide information that can be used to learn and execute manipulation tasks. Different tasks, however, might require different levels of sensory information; which in turn likely affect learning rates and performance. This paper evaluates the role of tactile information on autonomous learning of manipulation with a simulated 3-finger tendon-driven hand. We compare the ability of t…
▽ More
Tactile sensors provide information that can be used to learn and execute manipulation tasks. Different tasks, however, might require different levels of sensory information; which in turn likely affect learning rates and performance. This paper evaluates the role of tactile information on autonomous learning of manipulation with a simulated 3-finger tendon-driven hand. We compare the ability of the same learning algorithm (Proximal Policy Optimization, PPO) to learn two manipulation tasks (rolling a ball about the horizontal axis with and without rotational stiffness) with three levels of tactile sensing: no sensing, 1D normal force, and 3D force vector. Surprisingly, and contrary to recent work on manipulation, adding 1D force-sensing did not always improve learning rates compared to no sensing---likely due to whether or not normal force is relevant to the task. Nonetheless, even though 3D force-sensing increases the dimensionality of the sensory input---which would in general hamper algorithm convergence---it resulted in faster learning rates and better performance. We conclude that, in general, sensory input is useful to learning only when it is relevant to the task---as is the case of 3D force-sensing for in-hand manipulation against gravity. Moreover, the utility of 3D force-sensing can even offset the added computational cost of learning with higher-dimensional sensory input.
△ Less
Submitted 5 February, 2020;
originally announced February 2020.
-
Autonomous Control of a Tendon-driven Robotic Limb with Elastic Elements Reveals that Added Elasticity can Enhance Learning
Authors:
Ali Marjaninejad,
Jie Tan,
Francisco J. Valero-Cuevas
Abstract:
Passive elastic elements can contribute to stability, energetic efficiency, and impact absorption in both biological and robotic systems. They also add dynamical complexity which makes them more challenging to model and control. The impact of this added complexity to autonomous learning has not been thoroughly explored. This is especially relevant to tendon-driven limbs whose cables and tendons ar…
▽ More
Passive elastic elements can contribute to stability, energetic efficiency, and impact absorption in both biological and robotic systems. They also add dynamical complexity which makes them more challenging to model and control. The impact of this added complexity to autonomous learning has not been thoroughly explored. This is especially relevant to tendon-driven limbs whose cables and tendons are inevitably elastic. Here, we explored the efficacy of autonomous learning and control on a simulated bio-plausible tendon-driven leg across different tendon stiffness values. We demonstrate that increasing stiffness of the simulated muscles can require more iterations for the inverse map to converge but can then perform more accurately, especially in discrete tasks. Moreover, the system is robust to subsequent changes in muscle stiffnesses and can adapt on-the-go within 5 attempts. Lastly, we test the system for the functional task of locomotion, and found similar effects of muscle stiffness to learning and performance. Given that a range of stiffness values led to improved learning and maximized performance, we conclude the robot bodies and autonomous controllers---at least for tendon-driven systems---can be co-developed to take advantage of elastic elements. Importantly, this opens also the door to development efforts that recapitulate the beneficial aspects of the co-evolution of brains and bodies in vertebrates.
△ Less
Submitted 26 September, 2019;
originally announced September 2019.
-
Simple Kinematic Feedback Enhances Autonomous Learning in Bio-Inspired Tendon-Driven Systems
Authors:
Ali Marjaninejad,
Darío Urbina-Meléndez,
Francisco J. Valero-Cuevas
Abstract:
Error feedback is known to improve performance by correcting control signals in response to perturbations. Here we show how adding simple error feedback can also accelerate and robustify autonomous learning in a tendon-driven robot. We implemented two versions of the General-to-Particular (G2P) autonomous learning algorithm to produce multiple movement tasks using a tendon-driven leg with two join…
▽ More
Error feedback is known to improve performance by correcting control signals in response to perturbations. Here we show how adding simple error feedback can also accelerate and robustify autonomous learning in a tendon-driven robot. We implemented two versions of the General-to-Particular (G2P) autonomous learning algorithm to produce multiple movement tasks using a tendon-driven leg with two joints and three tendons: one with and one without kinematic feedback. As expected, feedback improved performance in simulation and hardware. However, we see these improvements even in the presence of sensory delays of up to 100 ms and when experiencing substantial contact collisions. Importantly, feedback accelerates learning and enhances G2P's continual refinement of the initial inverse map by providing the system with more relevant data to train on. This allows the system to perform well even after only 60 seconds of initial motor babbling.
△ Less
Submitted 26 September, 2019; v1 submitted 10 July, 2019;
originally announced July 2019.
-
Autonomous Functional Locomotion in a Tendon-Driven Limb via Limited Experience
Authors:
Ali Marjaninejad,
Darío Urbina-Meléndez,
Brian A. Cohn,
Francisco J. Valero-Cuevas
Abstract:
Robots will become ubiquitously useful only when they can use few attempts to teach themselves to perform different tasks, even with complex bodies and in dynamical environments. Vertebrates, in fact, successfully use trial-and-error to learn multiple tasks in spite of their intricate tendon-driven anatomies. Roboticists find such tendon-driven systems particularly hard to control because they are…
▽ More
Robots will become ubiquitously useful only when they can use few attempts to teach themselves to perform different tasks, even with complex bodies and in dynamical environments. Vertebrates, in fact, successfully use trial-and-error to learn multiple tasks in spite of their intricate tendon-driven anatomies. Roboticists find such tendon-driven systems particularly hard to control because they are simultaneously nonlinear, under-determined (many tendon tensions combine to produce few net joint torques), and over-determined (few joint rotations define how many tendons need to be reeled-in/payed-out). We demonstrate---for the first time in simulation and in hardware---how a model-free approach allows few-shot autonomous learning to produce effective locomotion in a 3-tendon/2-joint tendon-driven leg. Initially, an artificial neural network fed by sparsely sampled data collected using motor babbling creates an inverse map from limb kinematics to motor activations, which is analogous to juvenile vertebrates playing during development. Thereafter, iterative reward-driven exploration of candidate motor activations simultaneously refines the inverse map and finds a functional locomotor limit-cycle autonomously. This biologically-inspired algorithm, which we call G2P (General to Particular), enables versatile adaptation of robots to changes in the target task, mechanics of their bodies, and environment. Moreover, this work empowers future studies of few-shot autonomous learning in biological systems, which is the foundation of their enviable functional versatility.
△ Less
Submitted 19 October, 2018;
originally announced October 2018.
-
Quantifying and attenuating pathologic tremor in virtual reality
Authors:
Brian A. Cohn,
Dilan D. Shah,
Ali Marjaninejad,
Martin Shapiro,
Serhan Ulkumen,
Christopher M. Laine,
Francisco J. Valero-Cuevas,
Kenneth H. Hayashida,
Sarah Ingersoll
Abstract:
We present a virtual reality (VR) experience that creates a research-grade benchmark in assessing patients with active upper-limb tremor, while simultaneously offering the opportunity for patients to engage with VR experiences without their pathologic tremor. Accurate and precise use of handheld motion controllers in VR gaming applications may be limited for patients with upper limb tremor. In par…
▽ More
We present a virtual reality (VR) experience that creates a research-grade benchmark in assessing patients with active upper-limb tremor, while simultaneously offering the opportunity for patients to engage with VR experiences without their pathologic tremor. Accurate and precise use of handheld motion controllers in VR gaming applications may be limited for patients with upper limb tremor. In parallel, objective tools measuring tremor are not in widespread, routine clinical use. We used a commercially available VR system and designed a challenging virtual-balloon-pop** test mimicking a common nose-to-target pointing task used by medical practitioners to subjectively evaluate tremor in the exam room. Within our VR experience, we offer a software mode which uses a low-pass filter to adjust hand position and pointing orientation over a series of past data points. This digital filter creates a smoothing function for hand movement which effectively removes the patient's tremor in the VR representation. While the patient completes trials of the reaching task, quantitative data on the pathologic tremor is digitally recorded. With speed, accuracy, and the tremor components computed across three axes of movement, patients can be evaluated for their tremor amplitudes in a quantitative, replicable, and enjoyable manner. Removal of tremor in digital space may allow patients having significant upper limb tremor to have both an objective clinical measurement of symptoms while providing patients positive feedback and interaction.
△ Less
Submitted 16 September, 2018;
originally announced September 2018.
-
On Neuromechanical Approaches for the Study of Biological Grasp and Manipulation
Authors:
Francisco J Valero-Cuevas,
Marco Santello
Abstract:
Biological and robotic grasp and manipulation are undeniably similar at the level of mechanical task performance. However, their underlying fundamental biological vs. engineering mechanisms are, by definition, dramatically different and can even be antithetical. Even our approach to each is diametrically opposite: inductive science for the study of biological systems vs. engineering synthesis for…
▽ More
Biological and robotic grasp and manipulation are undeniably similar at the level of mechanical task performance. However, their underlying fundamental biological vs. engineering mechanisms are, by definition, dramatically different and can even be antithetical. Even our approach to each is diametrically opposite: inductive science for the study of biological systems vs. engineering synthesis for the design and construction of robotic systems. The past 20 years have seen several conceptual advances in both fields and the quest to unify them. Chief among them is the reluctant recognition that their underlying fundamental mechanisms may actually share limited common ground, while exhibiting many fundamental differences. This recognition is particularly liberating because it allows us to resolve and move beyond multiple paradoxes and contradictions that arose from the initial reasonable assumption of a large common ground. Here, we begin by introducing the perspective of neuromechanics, which emphasizes that real-world behavior emerges from the intimate interactions among the physical structure of the system, the mechanical requirements of a task, the feasible neural control actions to produce it, and the ability of the neuromuscular system to adapt through interactions with the environment. This allows us to articulate a succinct overview of a few salient conceptual paradoxes and contradictions regarding under-determined vs. over-determined mechanics, under- vs. over-actuated control, prescribed vs. emergent function, learning vs. implementation vs. adaptation, prescriptive vs. descriptive synergies, and optimal vs. habitual performance. We conclude by presenting open questions and suggesting directions for future research. We hope this frank assessment of the state-of-the-art will encourage and guide these communities to continue to interact and make progress in these important areas.
△ Less
Submitted 23 October, 2017;
originally announced October 2017.
-
Shapechanger: Environments for Transfer Learning
Authors:
Sébastien M. R. Arnold,
Tsam Kiu Pun,
Théo-Tim J. Denisart,
Francisco J. Valero-Cuevas
Abstract:
We present Shapechanger, a library for transfer reinforcement learning specifically designed for robotic tasks. We consider three types of knowledge transfer---from simulation to simulation, from simulation to real, and from real to real---and a wide range of tasks with continuous states and actions. Shapechanger is under active development and open-sourced at: https://github.com/seba-1511/shapech…
▽ More
We present Shapechanger, a library for transfer reinforcement learning specifically designed for robotic tasks. We consider three types of knowledge transfer---from simulation to simulation, from simulation to real, and from real to real---and a wide range of tasks with continuous states and actions. Shapechanger is under active development and open-sourced at: https://github.com/seba-1511/shapechanger/.
△ Less
Submitted 15 September, 2017;
originally announced September 2017.
