-
Unified Path and Gait Planning for Safe Bipedal Robot Navigation
Authors:
Chengyang Peng,
Victor Paredes,
Ayonga Hereid
Abstract:
Safe path and gait planning are essential for bipedal robots to navigate complex real-world environments. The prevailing approaches often plan the path and gait separately in a hierarchical fashion, potentially resulting in unsafe movements due to neglecting the physical constraints of walking robots. A safety-critical path must not only avoid obstacles but also ensure that the robot's gaits are s…
▽ More
Safe path and gait planning are essential for bipedal robots to navigate complex real-world environments. The prevailing approaches often plan the path and gait separately in a hierarchical fashion, potentially resulting in unsafe movements due to neglecting the physical constraints of walking robots. A safety-critical path must not only avoid obstacles but also ensure that the robot's gaits are subject to its dynamic and kinematic constraints. This work presents a novel approach that unifies path planning and gait planning via a Model Predictive Control (MPC) using the Linear Inverted Pendulum (LIP) model representing bipedal locomotion. This approach considers environmental constraints, such as obstacles, and the robot's kinematics and dynamics constraints. By using discrete-time Control Barrier Functions for obstacle avoidance, our approach generates the next foot landing position, ensuring robust walking gaits and a safe navigation path within clustered environments. We validated our proposed approach in simulation using a Digit robot in 20 randomly created environments. The results demonstrate improved performance in terms of safety and robustness when compared to hierarchical path and gait planning frameworks.
△ Less
Submitted 25 March, 2024;
originally announced March 2024.
-
Adaptive Step Duration for Precise Foot Placement: Achieving Robust Bipedal Locomotion on Terrains with Restricted Footholds
Authors:
Zhaoyang Xiang,
Victor Paredes,
Ayonga Hereid
Abstract:
This paper introduces a novel multi-step preview foot placement planning algorithm designed to enhance the robustness of bipedal robotic walking across challenging terrains with restricted footholds. Traditional one-step preview planning struggles to maintain stability when step** areas are severely limited, such as with random step** stones. In this work, we developed a discrete-time Model Pr…
▽ More
This paper introduces a novel multi-step preview foot placement planning algorithm designed to enhance the robustness of bipedal robotic walking across challenging terrains with restricted footholds. Traditional one-step preview planning struggles to maintain stability when step** areas are severely limited, such as with random step** stones. In this work, we developed a discrete-time Model Predictive Control (MPC) based on the step-to-step discrete evolution of the Divergent Component of Motion (DCM) of bipedal locomotion. This approach adaptively changes the step duration for optimal foot placement under constraints, thereby ensuring the robot's operational viability over multiple future steps and significantly improving its ability to navigate through environments with tight constraints on possible footholds. The effectiveness of this planning algorithm is demonstrated through simulations that include a variety of complex step**-stone configurations and external perturbations. These tests underscore the algorithm's improved performance for navigating foothold-restricted environments, even with the presence of external disturbances.
△ Less
Submitted 25 March, 2024;
originally announced March 2024.
-
Safe Whole-Body Task Space Control for Humanoid Robots
Authors:
Victor Paredes,
Ayonga Hereid
Abstract:
Complex robotic systems require whole-body controllers to deal with contact interactions, handle closed kinematic chains, and track task-space control objectives. However, for many applications, safety-critical controllers are important to steer away from undesired robot configurations to prevent unsafe behaviors. A prime example is legged robotics, where we can have tasks such as balance control,…
▽ More
Complex robotic systems require whole-body controllers to deal with contact interactions, handle closed kinematic chains, and track task-space control objectives. However, for many applications, safety-critical controllers are important to steer away from undesired robot configurations to prevent unsafe behaviors. A prime example is legged robotics, where we can have tasks such as balance control, regulation of torso orientation, and, most importantly, walking. As the coordination of multi-body systems is non-trivial, following a combination of those tasks might lead to configurations that are deemed dangerous, such as step** on its support foot during walking, leaning the torso excessively, or producing excessive centroidal momentum, resulting in non-human-like walking. To address these challenges, we propose a formulation of an inverse dynamics control enhanced with exponential control barrier functions for robotic systems with numerous degrees of freedom. Our approach utilizes a quadratic program that respects closed kinematic chains, minimizes the control objectives, and imposes desired constraints on the Zero Moment Point, friction cone, and torque. More importantly, it also ensures the forward invariance of a general user-defined high Relative-Degree safety set. We demonstrate the effectiveness of our method by applying it to the 3D biped robot Digit, both in simulation and with hardware experiments.
△ Less
Submitted 12 October, 2023;
originally announced November 2023.
-
Data-Driven Latent Space Representation for Robust Bipedal Locomotion Learning
Authors:
Guillermo A. Castillo,
Bowen Weng,
Wei Zhang,
Ayonga Hereid
Abstract:
This paper presents a novel framework for learning robust bipedal walking by combining a data-driven state representation with a Reinforcement Learning (RL) based locomotion policy. The framework utilizes an autoencoder to learn a low-dimensional latent space that captures the complex dynamics of bipedal locomotion from existing locomotion data. This reduced dimensional state representation is the…
▽ More
This paper presents a novel framework for learning robust bipedal walking by combining a data-driven state representation with a Reinforcement Learning (RL) based locomotion policy. The framework utilizes an autoencoder to learn a low-dimensional latent space that captures the complex dynamics of bipedal locomotion from existing locomotion data. This reduced dimensional state representation is then used as states for training a robust RL-based gait policy, eliminating the need for heuristic state selections or the use of template models for gait planning. The results demonstrate that the learned latent variables are disentangled and directly correspond to different gaits or speeds, such as moving forward, backward, or walking in place. Compared to traditional template model-based approaches, our framework exhibits superior performance and robustness in simulation. The trained policy effectively tracks a wide range of walking speeds and demonstrates good generalization capabilities to unseen scenarios.
△ Less
Submitted 27 September, 2023;
originally announced September 2023.
-
Template Model Inspired Task Space Learning for Robust Bipedal Locomotion
Authors:
Guillermo A. Castillo,
Bowen Weng,
Shunpeng Yang,
Wei Zhang,
Ayonga Hereid
Abstract:
This work presents a hierarchical framework for bipedal locomotion that combines a Reinforcement Learning (RL)-based high-level (HL) planner policy for the online generation of task space commands with a model-based low-level (LL) controller to track the desired task space trajectories. Different from traditional end-to-end learning approaches, our HL policy takes insights from the angular momentu…
▽ More
This work presents a hierarchical framework for bipedal locomotion that combines a Reinforcement Learning (RL)-based high-level (HL) planner policy for the online generation of task space commands with a model-based low-level (LL) controller to track the desired task space trajectories. Different from traditional end-to-end learning approaches, our HL policy takes insights from the angular momentum-based linear inverted pendulum (ALIP) to carefully design the observation and action spaces of the Markov Decision Process (MDP). This simple yet effective design creates an insightful map** between a low-dimensional state that effectively captures the complex dynamics of bipedal locomotion and a set of task space outputs that shape the walking gait of the robot. The HL policy is agnostic to the task space LL controller, which increases the flexibility of the design and generalization of the framework to other bipedal robots. This hierarchical design results in a learning-based framework with improved performance, data efficiency, and robustness compared with the ALIP model-based approach and state-of-the-art learning-based frameworks for bipedal locomotion. The proposed hierarchical controller is tested in three different robots, Rabbit, a five-link underactuated planar biped; Walker2D, a seven-link fully-actuated planar biped; and Digit, a 3D humanoid robot with 20 actuated joints. The trained policy naturally learns human-like locomotion behaviors and is able to effectively track a wide range of walking speeds while preserving the robustness and stability of the walking gait even under adversarial conditions.
△ Less
Submitted 27 September, 2023;
originally announced September 2023.
-
Towards Standardized Disturbance Rejection Testing of Legged Robot Locomotion with Linear Impactor: A Preliminary Study, Observations, and Implications
Authors:
Bowen Weng,
Guillermo A. Castillo,
Yun-Seok Kang,
Ayonga Hereid
Abstract:
Dynamic locomotion in legged robots is close to industrial collaboration, but a lack of standardized testing obstructs commercialization. The issues are not merely political, theoretical, or algorithmic but also physical, indicating limited studies and comprehension regarding standard testing infrastructure and equipment. For decades, the approaches we have been testing legged robots were rarely s…
▽ More
Dynamic locomotion in legged robots is close to industrial collaboration, but a lack of standardized testing obstructs commercialization. The issues are not merely political, theoretical, or algorithmic but also physical, indicating limited studies and comprehension regarding standard testing infrastructure and equipment. For decades, the approaches we have been testing legged robots were rarely standardizable with hand-pushing, foot-kicking, rope-dragging, stick-poking, and ball-swinging. This paper aims to bridge the gap by proposing the use of the linear impactor, a well-established tool in other standardized testing disciplines, to serve as an adaptive, repeatable, and fair disturbance rejection testing equipment for legged robots. A pneumatic linear impactor is also adopted for the case study involving the humanoid robot Digit. Three locomotion controllers are examined, including a commercial one, using a walking-in-place task against frontal impacts. The statistically best controller was able to withstand the impact momentum (26.376 kg$\cdot$m/s) on par with a reported average effective momentum from straight punches by Olympic boxers (26.506 kg$\cdot$m/s). Moreover, the case study highlights other anti-intuitive observations, demonstrations, and implications that, to the best of the authors' knowledge, are first-of-its-kind revealed in real-world testing of legged robots.
△ Less
Submitted 29 January, 2024; v1 submitted 28 August, 2023;
originally announced August 2023.
-
Time-Varying ALIP Model and Robust Foot-Placement Control for Underactuated Bipedal Robot Walking on a Swaying Rigid Surface
Authors:
Yuan Gao,
Yukai Gong,
Victor Paredes,
Ayonga Hereid,
Yan Gu
Abstract:
Controller design for bipedal walking on dynamic rigid surfaces (DRSes), which are rigid surfaces moving in the inertial frame (e.g., ships and airplanes), remains largely uninvestigated. This paper introduces a hierarchical control approach that achieves stable underactuated bipedal robot walking on a horizontally oscillating DRS. The highest layer of our approach is a real-time motion planner th…
▽ More
Controller design for bipedal walking on dynamic rigid surfaces (DRSes), which are rigid surfaces moving in the inertial frame (e.g., ships and airplanes), remains largely uninvestigated. This paper introduces a hierarchical control approach that achieves stable underactuated bipedal robot walking on a horizontally oscillating DRS. The highest layer of our approach is a real-time motion planner that generates desired global behaviors (i.e., the center of mass trajectories and footstep locations) by stabilizing a reduced-order robot model. One key novelty of this layer is the derivation of the reduced-order model by analytically extending the angular momentum based linear inverted pendulum (ALIP) model from stationary to horizontally moving surfaces. The other novelty is the development of a discrete-time foot-placement controller that exponentially stabilizes the hybrid, linear, time-varying ALIP model. The middle layer of the proposed approach is a walking pattern generator that translates the desired global behaviors into the robot's full-body reference trajectories for all directly actuated degrees of freedom. The lowest layer is an input-output linearizing controller that exponentially tracks those full-body reference trajectories based on the full-order, hybrid, nonlinear robot dynamics. Simulations of planar underactuated bipedal walking on a swaying DRS confirm that the proposed framework ensures the walking stability under different DRS motions and gait types.
△ Less
Submitted 29 November, 2022; v1 submitted 24 October, 2022;
originally announced October 2022.
-
Safe Path Planning for Polynomial Shape Obstacles via Control Barrier Functions and Logistic Regression
Authors:
Chengyang Peng,
Octavian Donca,
Ayonga Hereid
Abstract:
Safe path planning is critical for bipedal robots to operate in safety-critical environments. Common path planning algorithms, such as RRT or RRT*, typically use geometric or kinematic collision check algorithms to ensure collision-free paths toward the target position. However, such approaches may generate non-smooth paths that do not comply with the dynamics constraints of walking robots. It has…
▽ More
Safe path planning is critical for bipedal robots to operate in safety-critical environments. Common path planning algorithms, such as RRT or RRT*, typically use geometric or kinematic collision check algorithms to ensure collision-free paths toward the target position. However, such approaches may generate non-smooth paths that do not comply with the dynamics constraints of walking robots. It has been shown that the control barrier function (CBF) can be integrated with RRT/RRT* to synthesize dynamically feasible collision-free paths. Yet, existing work has been limited to simple circular or elliptical shape obstacles due to the challenging nature of constructing appropriate barrier functions to represent irregular-shaped obstacles. In this paper, we present a CBF-based RRT* algorithm for bipedal robots to generate a collision-free path through complex space with polynomial-shaped obstacles. In particular, we used logistic regression to construct polynomial barrier functions from a grid map of the environment to represent arbitrarily shaped obstacles. Moreover, we developed a multi-step CBF steering controller to ensure the efficiency of free space exploration. The proposed approach was first validated in simulation for a differential drive model, and then experimentally evaluated with a 3D humanoid robot, Digit, in a lab setting with randomly placed obstacles.
△ Less
Submitted 7 October, 2022;
originally announced October 2022.
-
Real-Time Navigation for Bipedal Robots in Dynamic Environments
Authors:
Octavian A. Donca,
Chayapol Beokhaimook,
Ayonga Hereid
Abstract:
The popularity of mobile robots has been steadily growing, with these robots being increasingly utilized to execute tasks previously completed by human workers. For bipedal robots to see this same success, robust autonomous navigation systems need to be developed that can execute in real-time and respond to dynamic environments. These systems can be divided into three stages: perception, planning,…
▽ More
The popularity of mobile robots has been steadily growing, with these robots being increasingly utilized to execute tasks previously completed by human workers. For bipedal robots to see this same success, robust autonomous navigation systems need to be developed that can execute in real-time and respond to dynamic environments. These systems can be divided into three stages: perception, planning, and control. A holistic navigation framework for bipedal robots must successfully integrate all three components of the autonomous navigation problem to enable robust real-world navigation. In this paper, we present a real-time navigation framework for bipedal robots in dynamic environments. The proposed system addresses all components of the navigation problem: We introduce a depth-based perception system for obstacle detection, map**, and localization. A two-stage planner is developed to generate collision-free trajectories robust to unknown and dynamic environments. And execute trajectories on the Digit bipedal robot's walking gait controller. The navigation framework is validated through a series of simulation and hardware experiments that contain unknown environments and dynamic obstacles.
△ Less
Submitted 6 October, 2022;
originally announced October 2022.
-
On the Adversarial Scenario-based Safety Testing of Robots: the Comparability and Optimal Aggressiveness
Authors:
Bowen Weng,
Guillermo A. Castillo,
Wei Zhang,
Ayonga Hereid
Abstract:
This paper studies the class of scenario-based safety testing algorithms in the black-box safety testing configuration. For algorithms sharing the same state-action set coverage with different sampling distributions, it is commonly believed that prioritizing the exploration of high-risk state-actions leads to a better sampling efficiency. Our proposal disputes the above intuition by introducing an…
▽ More
This paper studies the class of scenario-based safety testing algorithms in the black-box safety testing configuration. For algorithms sharing the same state-action set coverage with different sampling distributions, it is commonly believed that prioritizing the exploration of high-risk state-actions leads to a better sampling efficiency. Our proposal disputes the above intuition by introducing an impossibility theorem that provably shows all safety testing algorithms of the aforementioned difference perform equally well with the same expected sampling efficiency. Moreover, for testing algorithms covering different sets of state-actions, the sampling efficiency criterion is no longer applicable as different algorithms do not necessarily converge to the same termination condition. We then propose a testing aggressiveness definition based on the almost safe set concept along with an unbiased and efficient algorithm that compares the aggressiveness between testing algorithms. Empirical observations from the safety testing of bipedal locomotion controllers and vehicle decision-making modules are also presented to support the proposed theoretical implications and methodologies.
△ Less
Submitted 3 April, 2023; v1 submitted 20 September, 2022;
originally announced September 2022.
-
Resolved Motion Control for 3D Underactuated Bipedal Walking using Linear Inverted Pendulum Dynamics and Neural Adaptation
Authors:
Victor Paredes,
Ayonga Hereid
Abstract:
We present a framework to generate periodic trajectory references for a 3D under-actuated bipedal robot, using a linear inverted pendulum (LIP) based controller with adaptive neural regulation. We use the LIP template model to estimate the robot's center of mass (CoM) position and velocity at the end of the current step, and formulate a discrete controller that determines the next footstep locatio…
▽ More
We present a framework to generate periodic trajectory references for a 3D under-actuated bipedal robot, using a linear inverted pendulum (LIP) based controller with adaptive neural regulation. We use the LIP template model to estimate the robot's center of mass (CoM) position and velocity at the end of the current step, and formulate a discrete controller that determines the next footstep location to achieve a desired walking profile. This controller is equipped on the frontal plane with a Neural-Network-based adaptive term that reduces the model mismatch between the template and physical robot that particularly affects the lateral motion. Then, the foot placement location computed for the LIP model is used to generate task space trajectories (CoM and swing foot trajectories) for the actual robot to realize stable walking. We use a fast, real-time QP-based inverse kinematics algorithm that produces joint references from the task space trajectories, which makes the formulation independent of the knowledge of the robot dynamics. Finally, we implemented and evaluated the proposed approach in simulation and hardware experiments with a Digit robot obtaining stable periodic locomotion for both cases.
△ Less
Submitted 2 August, 2022;
originally announced August 2022.
-
On Safety Testing, Validation, and Characterization with Scenario-Sampling: A Case Study of Legged Robots
Authors:
Bowen Weng,
Guillermo A. Castillo,
Wei Zhang,
Ayonga Hereid
Abstract:
The dynamic response of the legged robot locomotion is non-Lipschitz and can be stochastic due to environmental uncertainties. To test, validate, and characterize the safety performance of legged robots, existing solutions on observed and inferred risk can be incomplete and sampling inefficient. Some formal verification methods suffer from the model precision and other surrogate assumptions. In th…
▽ More
The dynamic response of the legged robot locomotion is non-Lipschitz and can be stochastic due to environmental uncertainties. To test, validate, and characterize the safety performance of legged robots, existing solutions on observed and inferred risk can be incomplete and sampling inefficient. Some formal verification methods suffer from the model precision and other surrogate assumptions. In this paper, we propose a scenario sampling based testing framework that characterizes the overall safety performance of a legged robot by specifying (i) where (in terms of a set of states) the robot is potentially safe, and (ii) how safe the robot is within the specified set. The framework can also help certify the commercial deployment of the legged robot in real-world environment along with human and compare safety performance among legged robots with different mechanical structures and dynamic properties. The proposed framework is further deployed to evaluate a group of state-of-the-art legged robot locomotion controllers from various model-based, deep neural network involved, and reinforcement learning based methods in the literature. Among a series of intended work domains of the studied legged robots (e.g. tracking speed on sloped surface, with abrupt changes on demanded velocity, and against adversarial push-over disturbances), we show that the method can adequately capture the overall safety characterization and the subtle performance insights. Many of the observed safety outcomes, to the best of our knowledge, have never been reported by the existing work in the legged robot literature.
△ Less
Submitted 16 April, 2022;
originally announced April 2022.
-
Linear Policies are Sufficient to Realize Robust Bipedal Walking on Challenging Terrains
Authors:
Lokesh Krishna,
Guillermo A. Castillo,
Utkarsh A. Mishra,
Ayonga Hereid,
Shishir Kolathaya
Abstract:
In this work, we demonstrate robust walking in the bipedal robot Digit on uneven terrains by just learning a single linear policy. In particular, we propose a new control pipeline, wherein the high-level trajectory modulator shapes the end-foot ellipsoidal trajectories, and the low-level gait controller regulates the torso and ankle orientation. The foot-trajectory modulator uses a linear policy a…
▽ More
In this work, we demonstrate robust walking in the bipedal robot Digit on uneven terrains by just learning a single linear policy. In particular, we propose a new control pipeline, wherein the high-level trajectory modulator shapes the end-foot ellipsoidal trajectories, and the low-level gait controller regulates the torso and ankle orientation. The foot-trajectory modulator uses a linear policy and the regulator uses a linear PD control law. As opposed to neural network-based policies, the proposed linear policy has only 13 learnable parameters, thereby not only guaranteeing sample efficient learning but also enabling simplicity and interpretability of the policy. This is achieved with no loss of performance on challenging terrains like slopes, stairs and outdoor landscapes. We first demonstrate robust walking in the custom simulation environment, MuJoCo, and then directly transfer to hardware with no modification of the control pipeline. We subject the biped to a series of pushes and terrain height changes, both indoors and outdoors, thereby validating the presented work.
△ Less
Submitted 5 October, 2021; v1 submitted 26 September, 2021;
originally announced September 2021.
-
Adaptive Feedback Regulator for Powered Lower-Limb Exoskeleton under Model Uncertainty
Authors:
Kirtankumar Thakkar,
Victor Paredes,
Ayonga Hereid
Abstract:
This paper presents a neural network (NN) based adaptive feedback regulator to ensure the lateral and longitudinal stability and regulate the desired walking velocity of a lower-limb exoskeleton under model uncertainty. The traditional model-based controllers for lower-limb exoskeletons often fail to stabilize the robot or accurately track the desired behaviors under model uncertainties or externa…
▽ More
This paper presents a neural network (NN) based adaptive feedback regulator to ensure the lateral and longitudinal stability and regulate the desired walking velocity of a lower-limb exoskeleton under model uncertainty. The traditional model-based controllers for lower-limb exoskeletons often fail to stabilize the robot or accurately track the desired behaviors under model uncertainties or external disturbances. This paper proposes a neural network (NN) based online adaptive regulator that compensates for the unknown changes in model parameters and external disturbances by modifying the nominal joint trajectory. A gradient descent-based delta rule is implemented to update the weights of a single layer NN, which can be efficiently performed online by design. We demonstrate the performance of the presented regulator on ATALANTE, a fully actuated lower limb exoskeleton designed for paraplegic patients. The simulation results show that the proposed approach noticeably improves stability and the tracking performance of the system, despite significant changes in model parameters and large adversarial pushes.
△ Less
Submitted 23 April, 2021;
originally announced April 2021.
-
Learning Linear Policies for Robust Bipedal Locomotion on Terrains with Varying Slopes
Authors:
Lokesh Krishna,
Utkarsh A. Mishra,
Guillermo A. Castillo,
Ayonga Hereid,
Shishir Kolathaya
Abstract:
In this paper, with a view toward deployment of light-weight control frameworks for bipedal walking robots, we realize end-foot trajectories that are shaped by a single linear feedback policy. We learn this policy via a model-free and a gradient-free learning algorithm, Augmented Random Search (ARS), in the two robot platforms Rabbit and Digit. Our contributions are two-fold: a) By using torso and…
▽ More
In this paper, with a view toward deployment of light-weight control frameworks for bipedal walking robots, we realize end-foot trajectories that are shaped by a single linear feedback policy. We learn this policy via a model-free and a gradient-free learning algorithm, Augmented Random Search (ARS), in the two robot platforms Rabbit and Digit. Our contributions are two-fold: a) By using torso and support plane orientation as inputs, we achieve robust walking on slopes of up to 20 degrees in simulation. b) We demonstrate additional behaviors like walking backwards, step**-in-place, and recovery from external pushes of up to 120 N. The end result is a robust and a fast feedback control law for bipedal walking on terrains with varying slopes. Towards the end, we also provide preliminary results of hardware transfer to Digit.
△ Less
Submitted 9 August, 2021; v1 submitted 4 April, 2021;
originally announced April 2021.
-
Robust Feedback Motion Policy Design Using Reinforcement Learning on a 3D Digit Bipedal Robot
Authors:
Guillermo A. Castillo,
Bowen Weng,
Wei Zhang,
Ayonga Hereid
Abstract:
In this paper, a hierarchical and robust framework for learning bipedal locomotion is presented and successfully implemented on the 3D biped robot Digit built by Agility Robotics. We propose a cascade-structure controller that combines the learning process with intuitive feedback regulations. This design allows the framework to realize robust and stable walking with a reduced-dimension state and a…
▽ More
In this paper, a hierarchical and robust framework for learning bipedal locomotion is presented and successfully implemented on the 3D biped robot Digit built by Agility Robotics. We propose a cascade-structure controller that combines the learning process with intuitive feedback regulations. This design allows the framework to realize robust and stable walking with a reduced-dimension state and action spaces of the policy, significantly simplifying the design and reducing the sampling efficiency of the learning method. The inclusion of feedback regulation into the framework improves the robustness of the learned walking gait and ensures the success of the sim-to-real transfer of the proposed controller with minimal tuning. We specifically present a learning pipeline that considers hardware-feasible initial poses of the robot within the learning process to ensure the initial state of the learning is replicated as close as possible to the initial state of the robot in hardware experiments. Finally, we demonstrate the feasibility of our method by successfully transferring the learned policy in simulation to the Digit robot hardware, realizing sustained walking gaits under external force disturbances and challenging terrains not included during the training process. To the best of our knowledge, this is the first time a learning-based policy is transferred successfully to the Digit robot in hardware experiments without using dynamic randomization or curriculum learning.
△ Less
Submitted 28 March, 2021;
originally announced March 2021.
-
Velocity Regulation of 3D Bipedal Walking Robots with Uncertain Dynamics Through Adaptive Neural Network Controller
Authors:
Guillermo A. Castillo,
Bowen Weng,
Terrence C. Stewart,
Wei Zhang,
Ayonga Hereid
Abstract:
This paper presents a neural-network based adaptive feedback control structure to regulate the velocity of 3D bipedal robots under dynamics uncertainties. Existing Hybrid Zero Dynamics (HZD)-based controllers regulate velocity through the implementation of heuristic regulators that do not consider model and environmental uncertainties, which may significantly affect the tracking performance of the…
▽ More
This paper presents a neural-network based adaptive feedback control structure to regulate the velocity of 3D bipedal robots under dynamics uncertainties. Existing Hybrid Zero Dynamics (HZD)-based controllers regulate velocity through the implementation of heuristic regulators that do not consider model and environmental uncertainties, which may significantly affect the tracking performance of the controllers. In this paper, we address the uncertainties in the robot dynamics from the perspective of the reduced dimensional representation of virtual constraints and propose the integration of an adaptive neural network-based controller to regulate the robot velocity in the presence of model parameter uncertainties. The proposed approach yields improved tracking performance under dynamics uncertainties. The shallow adaptive neural network used in this paper does not require training a priori and has the potential to be implemented on the real-time robotic controller. A comparative simulation study of a 3D Cassie robot is presented to illustrate the performance of the proposed approach under various scenarios.
△ Less
Submitted 1 August, 2020;
originally announced August 2020.
-
Hybrid Zero Dynamics Inspired Feedback Control Policy Design for 3D Bipedal Locomotion using Reinforcement Learning
Authors:
Guillermo A. Castillo,
Bowen Weng,
Wei Zhang,
Ayonga Hereid
Abstract:
This paper presents a novel model-free reinforcement learning (RL) framework to design feedback control policies for 3D bipedal walking. Existing RL algorithms are often trained in an end-to-end manner or rely on prior knowledge of some reference joint trajectories. Different from these studies, we propose a novel policy structure that appropriately incorporates physical insights gained from the h…
▽ More
This paper presents a novel model-free reinforcement learning (RL) framework to design feedback control policies for 3D bipedal walking. Existing RL algorithms are often trained in an end-to-end manner or rely on prior knowledge of some reference joint trajectories. Different from these studies, we propose a novel policy structure that appropriately incorporates physical insights gained from the hybrid nature of the walking dynamics and the well-established hybrid zero dynamics approach for 3D bipedal walking. As a result, the overall RL framework has several key advantages, including lightweight network structure, short training time, and less dependence on prior knowledge. We demonstrate the effectiveness of the proposed method on Cassie, a challenging 3D bipedal robot. The proposed solution produces stable limit walking cycles that can track various walking speed in different directions. Surprisingly, without specifically trained with disturbances to achieve robustness, it also performs robustly against various adversarial forces applied to the torso towards both the forward and the backward directions.
△ Less
Submitted 3 October, 2019;
originally announced October 2019.
-
Reinforcement Learning Meets Hybrid Zero Dynamics: A Case Study for RABBIT
Authors:
Guillermo A. Castillo,
Bowen Weng,
Ayonga Hereid,
Wei Zhang
Abstract:
The design of feedback controllers for bipedal robots is challenging due to the hybrid nature of its dynamics and the complexity imposed by high-dimensional bipedal models. In this paper, we present a novel approach for the design of feedback controllers using Reinforcement Learning (RL) and Hybrid Zero Dynamics (HZD). Existing RL approaches for bipedal walking are inefficient as they do not consi…
▽ More
The design of feedback controllers for bipedal robots is challenging due to the hybrid nature of its dynamics and the complexity imposed by high-dimensional bipedal models. In this paper, we present a novel approach for the design of feedback controllers using Reinforcement Learning (RL) and Hybrid Zero Dynamics (HZD). Existing RL approaches for bipedal walking are inefficient as they do not consider the underlying physics, often requires substantial training, and the resulting controller may not be applicable to real robots. HZD is a powerful tool for bipedal control with local stability guarantees of the walking limit cycles. In this paper, we propose a non traditional RL structure that embeds the HZD framework into the policy learning. More specifically, we propose to use RL to find a control policy that maps from the robot's reduced order states to a set of parameters that define the desired trajectories for the robot's joints through the virtual constraints. Then, these trajectories are tracked using an adaptive PD controller. The method results in a stable and robust control policy that is able to track variable speed within a continuous interval. Robustness of the policy is evaluated by applying external forces to the torso of the robot. The proposed RL framework is implemented and demonstrated in OpenAI Gym with the MuJoCo physics engine based on the well-known RABBIT robot model.
△ Less
Submitted 3 October, 2018;
originally announced October 2018.
-
Feedback Control of a Cassie Bipedal Robot: Walking, Standing, and Riding a Segway
Authors:
Yukai Gong,
Ross Hartley,
Xingye Da,
Ayonga Hereid,
Omar Harib,
Jiunn-Kai Huang,
Jessy Grizzle
Abstract:
The Cassie bipedal robot designed by Agility Robotics is providing academics a common platform for sharing and comparing algorithms for locomotion, perception, and navigation. This paper focuses on feedback control for standing and walking using the methods of virtual constraints and gait libraries. The designed controller was implemented six weeks after the robot arrived at the University of Mich…
▽ More
The Cassie bipedal robot designed by Agility Robotics is providing academics a common platform for sharing and comparing algorithms for locomotion, perception, and navigation. This paper focuses on feedback control for standing and walking using the methods of virtual constraints and gait libraries. The designed controller was implemented six weeks after the robot arrived at the University of Michigan and allowed it to stand in place as well as walk over sidewalks, grass, snow, sand, and burning brush. The controller for standing also enables the robot to ride a Segway. A model of the Cassie robot has been placed on GitHub and the controller will also be made open source if the paper is accepted.
△ Less
Submitted 19 September, 2018;
originally announced September 2018.
-
Rapid Trajectory Optimization Using C-FROST with Illustration on a Cassie-Series Dynamic Walking Biped
Authors:
Ayonga Hereid,
Omar Harib,
Ross Hartley,
Yukai Gong,
Jessy W. Grizzle
Abstract:
One of the big attractions of low-dimensional models for gait design has been the ability to compute solutions rapidly, whereas one of their drawbacks has been the difficulty in map** the solutions back to the target robot. This paper presents a set of tools for rapidly determining solutions for ``humanoids'' without removing or lum** degrees of freedom. The main tools are (1) C-FROST, an open…
▽ More
One of the big attractions of low-dimensional models for gait design has been the ability to compute solutions rapidly, whereas one of their drawbacks has been the difficulty in map** the solutions back to the target robot. This paper presents a set of tools for rapidly determining solutions for ``humanoids'' without removing or lum** degrees of freedom. The main tools are (1) C-FROST, an open-source C++ interface for FROST, a direct collocation optimization tool; and (2) multi-threading. The results will be illustrated on a 20-DoF floating-base model for a Cassie-series bipedal robot through numerical calculations and physical experiments.
△ Less
Submitted 15 March, 2019; v1 submitted 17 July, 2018;
originally announced July 2018.
-
Feedback Control of an Exoskeleton for Paraplegics: Toward Robustly Stable Hands-free Dynamic Walking
Authors:
Omar Harib,
Ayonga Hereid,
Ayush Agrawal,
Thomas Gurriet,
Sylvain Finet,
Guilhem Boeris,
Alexis Duburcq,
M. Eva Mungai,
Matthieu Masselin,
Aaron D. Ames,
Koushil Sreenath,
Jessy Grizzle
Abstract:
This manuscript presents control of a high-DOF fully actuated lower-limb exoskeleton for paraplegic individuals. The key novelty is the ability for the user to walk without the use of crutches or other external means of stabilization. We harness the power of modern optimization techniques and supervised machine learning to develop a smooth feedback control policy that provides robust velocity regu…
▽ More
This manuscript presents control of a high-DOF fully actuated lower-limb exoskeleton for paraplegic individuals. The key novelty is the ability for the user to walk without the use of crutches or other external means of stabilization. We harness the power of modern optimization techniques and supervised machine learning to develop a smooth feedback control policy that provides robust velocity regulation and perturbation rejection. Preliminary evaluation of the stability and robustness of the proposed approach is demonstrated through the Gazebo simulation environment. In addition, preliminary experimental results with (complete) paraplegic individuals are included for the previous version of the controller.
△ Less
Submitted 21 May, 2018; v1 submitted 22 February, 2018;
originally announced February 2018.
-
Enhancing the performance of a safe controller via supervised learning for truck lateral control
Authors:
Yuxiao Chen,
Ayonga Hereid,
Huei Peng,
Jessy Grizzle
Abstract:
Correct-by-construction techniques, such as control barrier functions (CBFs), can be used to guarantee closed-loop safety by acting as a supervisor of an existing or legacy controller. However, supervisory-control intervention typically compromises the performance of the closed-loop system. On the other hand, machine learning has been used to synthesize controllers that inherit good properties fro…
▽ More
Correct-by-construction techniques, such as control barrier functions (CBFs), can be used to guarantee closed-loop safety by acting as a supervisor of an existing or legacy controller. However, supervisory-control intervention typically compromises the performance of the closed-loop system. On the other hand, machine learning has been used to synthesize controllers that inherit good properties from a training dataset, though safety is typically not guaranteed due to the difficulty of analyzing the associated neural network. In this paper, supervised learning is combined with CBFs to synthesize controllers that enjoy good performance with provable safety. A training set is generated by trajectory optimization that incorporates the CBF constraint for an interesting range of initial conditions of the truck model. A control policy is obtained via supervised learning that maps a feature representing the initial conditions to a parameterized desired trajectory. The learning-based controller is used as the performance controller and a CBF-based supervisory controller guarantees safety. A case study of lane kee** for articulated trucks shows that the controller trained by supervised learning inherits the good performance of the training set and rarely requires intervention by the CBF supervisor
△ Less
Submitted 2 May, 2018; v1 submitted 14 December, 2017;
originally announced December 2017.
