-
Learning to Brachiate via Simplified Model Imitation
Authors:
Daniele Reda,
Hung Yu Ling,
Michiel van de Panne
Abstract:
Brachiation is the primary form of locomotion for gibbons and siamangs, in which these primates swing from tree limb to tree limb using only their arms. It is challenging to control because of the limited control authority, the required advance planning, and the precision of the required grasps. We present a novel approach to this problem using reinforcement learning, and as demonstrated on a fing…
▽ More
Brachiation is the primary form of locomotion for gibbons and siamangs, in which these primates swing from tree limb to tree limb using only their arms. It is challenging to control because of the limited control authority, the required advance planning, and the precision of the required grasps. We present a novel approach to this problem using reinforcement learning, and as demonstrated on a finger-less 14-link planar model that learns to brachiate across challenging handhold sequences. Key to our method is the use of a simplified model, a point mass with a virtual arm, for which we first learn a policy that can brachiate across handhold sequences with a prescribed order. This facilitates the learning of the policy for the full model, for which it provides guidance by providing an overall center-of-mass trajectory to imitate, as well as for the timing of the holds. Lastly, the simplified model can also readily be used for planning suitable sequences of handholds in a given environment. Our results demonstrate brachiation motions with a variety of durations for the flight and hold phases, as well as emergent extra back-and-forth swings when this proves useful. The system is evaluated with a variety of ablations. The method enables future work towards more general 3D brachiation, as well as using simplified model imitation in other settings.
△ Less
Submitted 8 May, 2022;
originally announced May 2022.
-
Character Controllers Using Motion VAEs
Authors:
Hung Yu Ling,
Fabio Zinno,
George Cheng,
Michiel van de Panne
Abstract:
A fundamental problem in computer animation is that of realizing purposeful and realistic human movement given a sufficiently-rich set of motion capture clips. We learn data-driven generative models of human movement using autoregressive conditional variational autoencoders, or Motion VAEs. The latent variables of the learned autoencoder define the action space for the movement and thereby govern…
▽ More
A fundamental problem in computer animation is that of realizing purposeful and realistic human movement given a sufficiently-rich set of motion capture clips. We learn data-driven generative models of human movement using autoregressive conditional variational autoencoders, or Motion VAEs. The latent variables of the learned autoencoder define the action space for the movement and thereby govern its evolution over time. Planning or control algorithms can then use this action space to generate desired motions. In particular, we use deep reinforcement learning to learn controllers that achieve goal-directed movements. We demonstrate the effectiveness of the approach on multiple tasks. We further evaluate system-design choices and describe the current limitations of Motion VAEs.
△ Less
Submitted 26 March, 2021;
originally announced March 2021.
-
ALLSTEPS: Curriculum-driven Learning of Step** Stone Skills
Authors:
Zhaoming Xie,
Hung Yu Ling,
Nam Hee Kim,
Michiel van de Panne
Abstract:
Humans are highly adept at walking in environments with foot placement constraints, including step**-stone scenarios where the footstep locations are fully constrained. Finding good solutions to step**-stone locomotion is a longstanding and fundamental challenge for animation and robotics. We present fully learned solutions to this difficult problem using reinforcement learning. We demonstrate…
▽ More
Humans are highly adept at walking in environments with foot placement constraints, including step**-stone scenarios where the footstep locations are fully constrained. Finding good solutions to step**-stone locomotion is a longstanding and fundamental challenge for animation and robotics. We present fully learned solutions to this difficult problem using reinforcement learning. We demonstrate the importance of a curriculum for efficient learning and evaluate four possible curriculum choices compared to a non-curriculum baseline. Results are presented for a simulated human character, a realistic bipedal robot simulation and a monster character, in each case producing robust, plausible motions for challenging step** stone sequences and terrains.
△ Less
Submitted 29 August, 2020; v1 submitted 8 May, 2020;
originally announced May 2020.
