-
The utility of tactile force to autonomous learning of in-hand manipulation is task-dependent
Authors:
Romina Mir,
Ali Marjaninejad,
Francisco J. Valero-Cuevas
Abstract:
Tactile sensors provide information that can be used to learn and execute manipulation tasks. Different tasks, however, might require different levels of sensory information; which in turn likely affect learning rates and performance. This paper evaluates the role of tactile information on autonomous learning of manipulation with a simulated 3-finger tendon-driven hand. We compare the ability of t…
▽ More
Tactile sensors provide information that can be used to learn and execute manipulation tasks. Different tasks, however, might require different levels of sensory information; which in turn likely affect learning rates and performance. This paper evaluates the role of tactile information on autonomous learning of manipulation with a simulated 3-finger tendon-driven hand. We compare the ability of the same learning algorithm (Proximal Policy Optimization, PPO) to learn two manipulation tasks (rolling a ball about the horizontal axis with and without rotational stiffness) with three levels of tactile sensing: no sensing, 1D normal force, and 3D force vector. Surprisingly, and contrary to recent work on manipulation, adding 1D force-sensing did not always improve learning rates compared to no sensing---likely due to whether or not normal force is relevant to the task. Nonetheless, even though 3D force-sensing increases the dimensionality of the sensory input---which would in general hamper algorithm convergence---it resulted in faster learning rates and better performance. We conclude that, in general, sensory input is useful to learning only when it is relevant to the task---as is the case of 3D force-sensing for in-hand manipulation against gravity. Moreover, the utility of 3D force-sensing can even offset the added computational cost of learning with higher-dimensional sensory input.
△ Less
Submitted 5 February, 2020;
originally announced February 2020.
-
Autonomous Control of a Tendon-driven Robotic Limb with Elastic Elements Reveals that Added Elasticity can Enhance Learning
Authors:
Ali Marjaninejad,
Jie Tan,
Francisco J. Valero-Cuevas
Abstract:
Passive elastic elements can contribute to stability, energetic efficiency, and impact absorption in both biological and robotic systems. They also add dynamical complexity which makes them more challenging to model and control. The impact of this added complexity to autonomous learning has not been thoroughly explored. This is especially relevant to tendon-driven limbs whose cables and tendons ar…
▽ More
Passive elastic elements can contribute to stability, energetic efficiency, and impact absorption in both biological and robotic systems. They also add dynamical complexity which makes them more challenging to model and control. The impact of this added complexity to autonomous learning has not been thoroughly explored. This is especially relevant to tendon-driven limbs whose cables and tendons are inevitably elastic. Here, we explored the efficacy of autonomous learning and control on a simulated bio-plausible tendon-driven leg across different tendon stiffness values. We demonstrate that increasing stiffness of the simulated muscles can require more iterations for the inverse map to converge but can then perform more accurately, especially in discrete tasks. Moreover, the system is robust to subsequent changes in muscle stiffnesses and can adapt on-the-go within 5 attempts. Lastly, we test the system for the functional task of locomotion, and found similar effects of muscle stiffness to learning and performance. Given that a range of stiffness values led to improved learning and maximized performance, we conclude the robot bodies and autonomous controllers---at least for tendon-driven systems---can be co-developed to take advantage of elastic elements. Importantly, this opens also the door to development efforts that recapitulate the beneficial aspects of the co-evolution of brains and bodies in vertebrates.
△ Less
Submitted 26 September, 2019;
originally announced September 2019.
-
Simple Kinematic Feedback Enhances Autonomous Learning in Bio-Inspired Tendon-Driven Systems
Authors:
Ali Marjaninejad,
Darío Urbina-Meléndez,
Francisco J. Valero-Cuevas
Abstract:
Error feedback is known to improve performance by correcting control signals in response to perturbations. Here we show how adding simple error feedback can also accelerate and robustify autonomous learning in a tendon-driven robot. We implemented two versions of the General-to-Particular (G2P) autonomous learning algorithm to produce multiple movement tasks using a tendon-driven leg with two join…
▽ More
Error feedback is known to improve performance by correcting control signals in response to perturbations. Here we show how adding simple error feedback can also accelerate and robustify autonomous learning in a tendon-driven robot. We implemented two versions of the General-to-Particular (G2P) autonomous learning algorithm to produce multiple movement tasks using a tendon-driven leg with two joints and three tendons: one with and one without kinematic feedback. As expected, feedback improved performance in simulation and hardware. However, we see these improvements even in the presence of sensory delays of up to 100 ms and when experiencing substantial contact collisions. Importantly, feedback accelerates learning and enhances G2P's continual refinement of the initial inverse map by providing the system with more relevant data to train on. This allows the system to perform well even after only 60 seconds of initial motor babbling.
△ Less
Submitted 26 September, 2019; v1 submitted 10 July, 2019;
originally announced July 2019.
-
Autonomous Functional Locomotion in a Tendon-Driven Limb via Limited Experience
Authors:
Ali Marjaninejad,
Darío Urbina-Meléndez,
Brian A. Cohn,
Francisco J. Valero-Cuevas
Abstract:
Robots will become ubiquitously useful only when they can use few attempts to teach themselves to perform different tasks, even with complex bodies and in dynamical environments. Vertebrates, in fact, successfully use trial-and-error to learn multiple tasks in spite of their intricate tendon-driven anatomies. Roboticists find such tendon-driven systems particularly hard to control because they are…
▽ More
Robots will become ubiquitously useful only when they can use few attempts to teach themselves to perform different tasks, even with complex bodies and in dynamical environments. Vertebrates, in fact, successfully use trial-and-error to learn multiple tasks in spite of their intricate tendon-driven anatomies. Roboticists find such tendon-driven systems particularly hard to control because they are simultaneously nonlinear, under-determined (many tendon tensions combine to produce few net joint torques), and over-determined (few joint rotations define how many tendons need to be reeled-in/payed-out). We demonstrate---for the first time in simulation and in hardware---how a model-free approach allows few-shot autonomous learning to produce effective locomotion in a 3-tendon/2-joint tendon-driven leg. Initially, an artificial neural network fed by sparsely sampled data collected using motor babbling creates an inverse map from limb kinematics to motor activations, which is analogous to juvenile vertebrates playing during development. Thereafter, iterative reward-driven exploration of candidate motor activations simultaneously refines the inverse map and finds a functional locomotor limit-cycle autonomously. This biologically-inspired algorithm, which we call G2P (General to Particular), enables versatile adaptation of robots to changes in the target task, mechanics of their bodies, and environment. Moreover, this work empowers future studies of few-shot autonomous learning in biological systems, which is the foundation of their enviable functional versatility.
△ Less
Submitted 19 October, 2018;
originally announced October 2018.
-
Quantifying and attenuating pathologic tremor in virtual reality
Authors:
Brian A. Cohn,
Dilan D. Shah,
Ali Marjaninejad,
Martin Shapiro,
Serhan Ulkumen,
Christopher M. Laine,
Francisco J. Valero-Cuevas,
Kenneth H. Hayashida,
Sarah Ingersoll
Abstract:
We present a virtual reality (VR) experience that creates a research-grade benchmark in assessing patients with active upper-limb tremor, while simultaneously offering the opportunity for patients to engage with VR experiences without their pathologic tremor. Accurate and precise use of handheld motion controllers in VR gaming applications may be limited for patients with upper limb tremor. In par…
▽ More
We present a virtual reality (VR) experience that creates a research-grade benchmark in assessing patients with active upper-limb tremor, while simultaneously offering the opportunity for patients to engage with VR experiences without their pathologic tremor. Accurate and precise use of handheld motion controllers in VR gaming applications may be limited for patients with upper limb tremor. In parallel, objective tools measuring tremor are not in widespread, routine clinical use. We used a commercially available VR system and designed a challenging virtual-balloon-pop** test mimicking a common nose-to-target pointing task used by medical practitioners to subjectively evaluate tremor in the exam room. Within our VR experience, we offer a software mode which uses a low-pass filter to adjust hand position and pointing orientation over a series of past data points. This digital filter creates a smoothing function for hand movement which effectively removes the patient's tremor in the VR representation. While the patient completes trials of the reaching task, quantitative data on the pathologic tremor is digitally recorded. With speed, accuracy, and the tremor components computed across three axes of movement, patients can be evaluated for their tremor amplitudes in a quantitative, replicable, and enjoyable manner. Removal of tremor in digital space may allow patients having significant upper limb tremor to have both an objective clinical measurement of symptoms while providing patients positive feedback and interaction.
△ Less
Submitted 16 September, 2018;
originally announced September 2018.
