Showing 1–27 of 27 results for author: Grizzle, J W

Demonstrating a Robust Walking Algorithm for Underactuated Bipedal Robots in Non-flat, Non-stationary Environments

Authors: Oluwami Dosunmu-Ogunbi, Aayushi Shrivastava, Jessy W Grizzle

Abstract: This work explores an innovative algorithm designed to enhance the mobility of underactuated bipedal robots across challenging terrains, especially when navigating through spaces with constrained opportunities for foot support, like steps or stairs. By combining ankle torque with a refined angular momentum-based linear inverted pendulum model (ALIP), our method allows variability in the robot's ce… ▽ More This work explores an innovative algorithm designed to enhance the mobility of underactuated bipedal robots across challenging terrains, especially when navigating through spaces with constrained opportunities for foot support, like steps or stairs. By combining ankle torque with a refined angular momentum-based linear inverted pendulum model (ALIP), our method allows variability in the robot's center of mass height. We employ a dual-strategy controller that merges virtual constraints for precise motion regulation across essential degrees of freedom with an ALIP-centric model predictive control (MPC) framework, aimed at enforcing gait stability. The effectiveness of our feedback design is demonstrated through its application on the Cassie bipedal robot, which features 20 degrees of freedom. Key to our implementation is the development of tailored nominal trajectories and an optimized MPC that reduces the execution time to under 500 microseconds--and, hence, is compatible with Cassie's controller update frequency. This paper not only showcases the successful hardware deployment but also demonstrates a new capability, a bipedal robot using a moving walkway. △ Less

Submitted 4 March, 2024; originally announced March 2024.

Fall Prediction for Bipedal Robots: The Standing Phase

Authors: M. Eva Mungai, Gokul Prabhakaran, Jessy W. Grizzle

Abstract: This paper presents a novel approach to fall prediction for bipedal robots, specifically targeting the detection of potential falls while standing caused by abrupt, incipient, and intermittent faults. Leveraging a 1D convolutional neural network (CNN), our method aims to maximize lead time for fall prediction while minimizing false positive rates. The proposed algorithm uniquely integrates the det… ▽ More This paper presents a novel approach to fall prediction for bipedal robots, specifically targeting the detection of potential falls while standing caused by abrupt, incipient, and intermittent faults. Leveraging a 1D convolutional neural network (CNN), our method aims to maximize lead time for fall prediction while minimizing false positive rates. The proposed algorithm uniquely integrates the detection of various fault types and estimates the lead time for potential falls. Our contributions include the development of an algorithm capable of detecting abrupt, incipient, and intermittent faults in full-sized robots, its implementation using both simulation and hardware data for a humanoid robot, and a method for estimating lead time. Evaluation metrics, including false positive rate, lead time, and response time, demonstrate the efficacy of our approach. Particularly, our model achieves impressive lead times and response times across different fault scenarios with a false positive rate of 0. The findings of this study hold significant implications for enhancing the safety and reliability of bipedal robotic systems. △ Less

Submitted 25 September, 2023; originally announced September 2023.

Comments: Submitted to ICRA 2024. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Stair Climbing using the Angular Momentum Linear Inverted Pendulum Model and Model Predictive Control

Authors: Oluwami Dosunmu-Ogunbi, Aayushi Shrivastava, Grant Gibson, Jessy W Grizzle

Abstract: A new control paradigm using angular momentum and foot placement as state variables in the linear inverted pendulum model has expanded the realm of possibilities for the control of bipedal robots. This new paradigm, known as the ALIP model, has shown effectiveness in cases where a robot's center of mass height can be assumed to be constant or near constant as well as in cases where there are no no… ▽ More A new control paradigm using angular momentum and foot placement as state variables in the linear inverted pendulum model has expanded the realm of possibilities for the control of bipedal robots. This new paradigm, known as the ALIP model, has shown effectiveness in cases where a robot's center of mass height can be assumed to be constant or near constant as well as in cases where there are no non-kinematic restrictions on foot placement. Walking up and down stairs violates both of these assumptions, where center of mass height varies significantly within a step and the geometry of the stairs restrict the effectiveness of foot placement. In this paper, we explore a variation of the ALIP model that allows the length of the virtual pendulum formed by the robot's stance foot and center of mass to follow smooth trajectories during a step. We couple this model with a control strategy constructed from a novel combination of virtual constraint-based control and a model predictive control algorithm to stabilize a stair climbing gait that does not soley rely on foot placement. Simulations on a 20-degree of freedom model of the Cassie biped in the SimMechanics simulation environment show that the controller is able to achieve periodic gait. △ Less

Submitted 10 July, 2023; v1 submitted 5 July, 2023; originally announced July 2023.

Exploring Kinodynamic Fabrics for Reactive Whole-Body Control of Underactuated Humanoid Robots

Authors: Alphonsus Adu-Bredu, Grant Gibson, Jessy W. Grizzle

Abstract: For bipedal humanoid robots to successfully operate in the real world, they must be competent at simultaneously executing multiple motion tasks while reacting to unforeseen external disturbances in real-time. We propose Kinodynamic Fabrics as an approach for the specification, solution and simultaneous execution of multiple motion tasks in real-time while being reactive to dynamism in the environm… ▽ More For bipedal humanoid robots to successfully operate in the real world, they must be competent at simultaneously executing multiple motion tasks while reacting to unforeseen external disturbances in real-time. We propose Kinodynamic Fabrics as an approach for the specification, solution and simultaneous execution of multiple motion tasks in real-time while being reactive to dynamism in the environment. Kinodynamic Fabrics allows for the specification of prioritized motion tasks as forced spectral semi-sprays and solves for desired robot joint accelerations at real-time frequencies. We evaluate the capabilities of Kinodynamic fabrics on diverse physically challenging whole-body control tasks with a bipedal humanoid robot both in simulation and in the real-world. Kinodynamic Fabrics outperforms the state-of-the-art Quadratic Program based whole-body controller on a variety of whole-body control tasks on run-time and reactivity metrics in our experiments. Our open-source implementation of Kinodynamic Fabrics as well as robot demonstration videos can be found at this url: https://adubredu.github.io/kinofabs. △ Less

Submitted 23 August, 2023; v1 submitted 7 March, 2023; originally announced March 2023.

Realtime Safety Control for Bipedal Robots to Avoid Multiple Obstacles via CLF-CBF Constraints

Authors: **ze Liu, Minzhe Li, Jiunn-Kai Huang, Jessy W. Grizzle

Abstract: This paper presents a reactive planning system that allows a Cassie-series bipedal robot to avoid multiple non-overlap** obstacles via a single, continuously differentiable control barrier function (CBF). The overall system detects an individual obstacle via a height map derived from a LiDAR point cloud and computes an elliptical outer approximation, which is then turned into a CBF. The QP-CLF-C… ▽ More This paper presents a reactive planning system that allows a Cassie-series bipedal robot to avoid multiple non-overlap** obstacles via a single, continuously differentiable control barrier function (CBF). The overall system detects an individual obstacle via a height map derived from a LiDAR point cloud and computes an elliptical outer approximation, which is then turned into a CBF. The QP-CLF-CBF formalism developed by Ames et al. is applied to ensure that safe trajectories are generated. Liveness is ensured by an analysis of induced equilibrium points that are distinct from the goal state. Safe planning in environments with multiple obstacles is demonstrated both in simulation and experimentally on the Cassie biped. △ Less

Submitted 23 June, 2023; v1 submitted 4 January, 2023; originally announced January 2023.

Energy-based Legged Robots Terrain Traversability Modeling via Deep Inverse Reinforcement Learning

Authors: Lu Gan, Jessy W. Grizzle, Ryan M. Eustice, Maani Ghaffari

Abstract: This work reports on develo** a deep inverse reinforcement learning method for legged robots terrain traversability modeling that incorporates both exteroceptive and proprioceptive sensory data. Existing works use robot-agnostic exteroceptive environmental features or handcrafted kinematic features; instead, we propose to also learn robot-specific inertial features from proprioceptive sensory da… ▽ More This work reports on develo** a deep inverse reinforcement learning method for legged robots terrain traversability modeling that incorporates both exteroceptive and proprioceptive sensory data. Existing works use robot-agnostic exteroceptive environmental features or handcrafted kinematic features; instead, we propose to also learn robot-specific inertial features from proprioceptive sensory data for reward approximation in a single deep neural network. Incorporating the inertial features can improve the model fidelity and provide a reward that depends on the robot's state during deployment. We train the reward network using the Maximum Entropy Deep Inverse Reinforcement Learning (MEDIRL) algorithm and propose simultaneously minimizing a trajectory ranking loss to deal with the suboptimality of legged robot demonstrations. The demonstrated trajectories are ranked by locomotion energy consumption, in order to learn an energy-aware reward function and a more energy-efficient policy than demonstration. We evaluate our method using a dataset collected by an MIT Mini-Cheetah robot and a Mini-Cheetah simulator. The code is publicly available at https://github.com/ganlumomo/minicheetah-traversability-irl. △ Less

Submitted 6 July, 2022; originally announced July 2022.

Informable Multi-Objective and Multi-Directional RRT* System for Robot Path Planning

Authors: Jiunn-Kai Huang, Yingwen Tan, Dongmyeong Lee, Vishnu R. Desaraju, Jessy W. Grizzle

Abstract: Multi-objective or multi-destination path planning is crucial for mobile robotics applications such as mobility as a service, robotics inspection, and electric vehicle charging for long trips. This work proposes an anytime iterative system to concurrently solve the multi-objective path planning problem and determine the visiting order of destinations. The system is comprised of an anytime informab… ▽ More Multi-objective or multi-destination path planning is crucial for mobile robotics applications such as mobility as a service, robotics inspection, and electric vehicle charging for long trips. This work proposes an anytime iterative system to concurrently solve the multi-objective path planning problem and determine the visiting order of destinations. The system is comprised of an anytime informable multi-objective and multi-directional RRT* algorithm to form a simple connected graph, and a proposed solver that consists of an enhanced cheapest insertion algorithm and a genetic algorithm to solve the relaxed traveling salesman problem in polynomial time. Moreover, a list of waypoints is often provided for robotics inspection and vehicle routing so that the robot can preferentially visit certain equipment or areas of interest. We show that the proposed system can inherently incorporate such knowledge, and can navigate through challenging topology. The proposed anytime system is evaluated on large and complex graphs built for real-world driving applications. All implementations are coded in multi-threaded C++ and are available at: https://github.com/UMich-BipedLab/IMOMD-RRTStar. △ Less

Submitted 30 May, 2022; originally announced May 2022.

Optimal Target Shape for LiDAR Pose Estimation

Authors: Jiunn-Kai Huang, William Clark, Jessy W. Grizzle

Abstract: Targets are essential in problems such as object tracking in cluttered or textureless environments, camera (and multi-sensor) calibration tasks, and simultaneous localization and map** (SLAM). Target shapes for these tasks typically are symmetric (square, rectangular, or circular) and work well for structured, dense sensor data such as pixel arrays (i.e., image). However, symmetric shapes lead t… ▽ More Targets are essential in problems such as object tracking in cluttered or textureless environments, camera (and multi-sensor) calibration tasks, and simultaneous localization and map** (SLAM). Target shapes for these tasks typically are symmetric (square, rectangular, or circular) and work well for structured, dense sensor data such as pixel arrays (i.e., image). However, symmetric shapes lead to pose ambiguity when using sparse sensor data such as LiDAR point clouds and suffer from the quantization uncertainty of the LiDAR. This paper introduces the concept of optimizing target shape to remove pose ambiguity for LiDAR point clouds. A target is designed to induce large gradients at edge points under rotation and translation relative to the LiDAR to ameliorate the quantization uncertainty associated with point cloud sparseness. Moreover, given a target shape, we present a means that leverages the target's geometry to estimate the target's vertices while globally estimating the pose. Both the simulation and the experimental results (verified by a motion capture system) confirm that by using the optimal shape and the global solver, we achieve centimeter error in translation and a few degrees in rotation even when a partially illuminated target is placed 30 meters away. All the implementations and datasets are available at https://github.com/UMich-BipedLab/optimal_shape_global_pose_estimation. △ Less

Submitted 21 December, 2021; v1 submitted 2 September, 2021; originally announced September 2021.

Efficient Anytime CLF Reactive Planning System for a Bipedal Robot on Undulating Terrain

Authors: Jiunn-Kai Huang, Jessy W. Grizzle

Abstract: We propose and experimentally demonstrate a reactive planning system for bipedal robots on unexplored, challenging terrains. The system consists of a low-frequency planning thread (5 Hz) to find an asymptotically optimal path and a high-frequency reactive thread (300 Hz) to accommodate robot deviation. The planning thread includes: a multi-layer local map to compute traversability for the robot on… ▽ More We propose and experimentally demonstrate a reactive planning system for bipedal robots on unexplored, challenging terrains. The system consists of a low-frequency planning thread (5 Hz) to find an asymptotically optimal path and a high-frequency reactive thread (300 Hz) to accommodate robot deviation. The planning thread includes: a multi-layer local map to compute traversability for the robot on the terrain; an anytime omnidirectional Control Lyapunov Function (CLF) for use with a Rapidly Exploring Random Tree Star (RRT*) that generates a vector field for specifying motion between nodes; a sub-goal finder when the final goal is outside of the current map; and a finite-state machine to handle high-level mission decisions. The system also includes a reactive thread to obviate the non-smooth motions that arise with traditional RRT* algorithms when performing path following. The reactive thread copes with robot deviation while eliminating non-smooth motions via a vector field (defined by a closed-loop feedback policy) that provides real-time control commands to the robot's gait controller as a function of instantaneous robot pose. The system is evaluated on various challenging outdoor terrains and cluttered indoor scenes in both simulation and experiment on Cassie Blue, a bipedal robot with 20 degrees of freedom. All implementations are coded in C++ with the Robot Operating System (ROS) and are available at https://github.com/UMich-BipedLab/CLF_reactive_planning_system. △ Less

Submitted 7 December, 2021; v1 submitted 15 August, 2021; originally announced August 2021.

Multitask Learning for Scalable and Dense Multilayer Bayesian Map Inference

Authors: Lu Gan, Youngji Kim, Jessy W. Grizzle, Jeffrey M. Walls, Ayoung Kim, Ryan M. Eustice, Maani Ghaffari

Abstract: This article presents a novel and flexible multitask multilayer Bayesian map** framework with readily extendable attribute layers. The proposed framework goes beyond modern metric-semantic maps to provide even richer environmental information for robots in a single map** formalism while exploiting intralayer and interlayer correlations. It removes the need for a robot to access and process inf… ▽ More This article presents a novel and flexible multitask multilayer Bayesian map** framework with readily extendable attribute layers. The proposed framework goes beyond modern metric-semantic maps to provide even richer environmental information for robots in a single map** formalism while exploiting intralayer and interlayer correlations. It removes the need for a robot to access and process information from many separate maps when performing a complex task, advancing the way robots interact with their environments. To this end, we design a multitask deep neural network with attention mechanisms as our front-end to provide heterogeneous observations for multiple map layers simultaneously. Our back-end runs a scalable closed-form Bayesian inference with only logarithmic time complexity. We apply the framework to build a dense robotic map including metric-semantic occupancy and traversability layers. Traversability ground truth labels are automatically generated from exteroceptive sensory data in a self-supervised manner. We present extensive experimental results on publicly available datasets and data collected by a 3D bipedal robot platform and show reliable map** performance in different environments. Finally, we also discuss how the current framework can be extended to incorporate more information such as friction, signal strength, temperature, and physical quantity concentration using Gaussian map layers. The software for reproducing the presented results or running on customized data is made publicly available. △ Less

Submitted 9 October, 2022; v1 submitted 28 June, 2021; originally announced June 2021.

Toward Safety-Aware Informative Motion Planning for Legged Robots

Authors: Sangli Teng, Yukai Gong, Jessy W. Grizzle, Maani Ghaffari

Abstract: This paper reports on develo** an integrated framework for safety-aware informative motion planning suitable for legged robots. The information-gathering planner takes a dense stochastic map of the environment into account, while safety constraints are enforced via Control Barrier Functions (CBFs). The planner is based on the Incrementally-exploring Information Gathering (IIG) algorithm and allo… ▽ More This paper reports on develo** an integrated framework for safety-aware informative motion planning suitable for legged robots. The information-gathering planner takes a dense stochastic map of the environment into account, while safety constraints are enforced via Control Barrier Functions (CBFs). The planner is based on the Incrementally-exploring Information Gathering (IIG) algorithm and allows closed-loop kinodynamic node expansion using a Model Predictive Control (MPC) formalism. Robotic exploration and information gathering problems are inherently path-dependent problems. That is, the information collected along a path depends on the state and observation history. As such, motion planning solely based on a modular cost does not lead to suitable plans for exploration. We propose SAFE-IIG, an integrated informative motion planning algorithm that takes into account: 1) a robot's perceptual field of view via a submodular information function computed over a stochastic map of the environment, 2) a robot's dynamics and safety constraints via discrete-time CBFs and MPC for closed-loop multi-horizon node expansions, and 3) an automatic stop** criterion via setting an information-theoretic planning horizon. The simulation results show that SAFE-IIG can plan a safe and dynamically feasible path while exploring a dense map. △ Less

Submitted 26 March, 2021; originally announced March 2021.

Comments: 11 pages

A New Framework for Registration of Semantic Point Clouds from Stereo and RGB-D Cameras

Authors: Ray Zhang, Tzu-Yuan Lin, Chien Erh Lin, Steven A. Parkison, William Clark, Jessy W. Grizzle, Ryan M. Eustice, Maani Ghaffari

Abstract: This paper reports on a novel nonparametric rigid point cloud registration framework that jointly integrates geometric and semantic measurements such as color or semantic labels into the alignment process and does not require explicit data association. The point clouds are represented as nonparametric functions in a reproducible kernel Hilbert space. The alignment problem is formulated as maximizi… ▽ More This paper reports on a novel nonparametric rigid point cloud registration framework that jointly integrates geometric and semantic measurements such as color or semantic labels into the alignment process and does not require explicit data association. The point clouds are represented as nonparametric functions in a reproducible kernel Hilbert space. The alignment problem is formulated as maximizing the inner product between two functions, essentially a sum of weighted kernels, each of which exploits the local geometric and semantic features. As a result of the continuous models, analytical gradients can be computed, and a local solution can be obtained by optimization over the rigid body transformation group. Besides, we present a new point cloud alignment metric that is intrinsic to the proposed framework and takes into account geometric and semantic information. The evaluations using publicly available stereo and RGB-D datasets show that the proposed method outperforms state-of-the-art outdoor and indoor frame-to-frame registration methods. An open-source GPU implementation is also provided. △ Less

Submitted 10 November, 2020; originally announced December 2020.

Comments: 8 pages, 5 figures

Global Unifying Intrinsic Calibration for Spinning and Solid-State LiDARs

Authors: Jiunn-Kai Huang, Chenxi Feng, Madhav Achar, Maani Ghaffari, Jessy W. Grizzle

Abstract: Sensor calibration, which can be intrinsic or extrinsic, is an essential step to achieve the measurement accuracy required for modern perception and navigation systems deployed on autonomous robots. To date, intrinsic calibration models for spinning LiDARs have been based on hypothesized based on their physical mechanisms, resulting in anywhere from three to ten parameters to be estimated from dat… ▽ More Sensor calibration, which can be intrinsic or extrinsic, is an essential step to achieve the measurement accuracy required for modern perception and navigation systems deployed on autonomous robots. To date, intrinsic calibration models for spinning LiDARs have been based on hypothesized based on their physical mechanisms, resulting in anywhere from three to ten parameters to be estimated from data, while no phenomenological models have yet been proposed for solid-state LiDARs. Instead of going down that road, we propose to abstract away from the physics of a LiDAR type (spinning vs solid-state, for example), and focus on the spatial geometry of the point cloud generated by the sensor. By modeling the calibration parameters as an element of a special matrix Lie Group, we achieve a unifying view of calibration for different types of LiDARs. We further prove mathematically that the proposed model is well-constrained (has a unique answer) given four appropriately orientated targets. The proof provides a guideline for target positioning in the form of a tetrahedron. Moreover, an existing Semidefinite programming global solver for SE(3) can be modified to compute efficiently the optimal calibration parameters. For solid state LiDARs, we illustrate how the method works in simulation. For spinning LiDARs, we show with experimental data that the proposed matrix Lie Group model performs equally well as physics-based models in terms of reducing the P2P distance, while being more robust to noise. △ Less

Submitted 27 July, 2021; v1 submitted 6 December, 2020; originally announced December 2020.

Improvements to Target-Based 3D LiDAR to Camera Calibration

Authors: Jiunn-Kai Huang, Jessy W. Grizzle

Abstract: The homogeneous transformation between a LiDAR and monocular camera is required for sensor fusion tasks, such as SLAM. While determining such a transformation is not considered glamorous in any sense of the word, it is nonetheless crucial for many modern autonomous systems. Indeed, an error of a few degrees in rotation or a few percent in translation can lead to 20 cm translation errors at a dista… ▽ More The homogeneous transformation between a LiDAR and monocular camera is required for sensor fusion tasks, such as SLAM. While determining such a transformation is not considered glamorous in any sense of the word, it is nonetheless crucial for many modern autonomous systems. Indeed, an error of a few degrees in rotation or a few percent in translation can lead to 20 cm translation errors at a distance of 5 m when overlaying a LiDAR image on a camera image. The biggest impediments to determining the transformation accurately are the relative sparsity of LiDAR point clouds and systematic errors in their distance measurements. This paper proposes (1) the use of targets of known dimension and geometry to ameliorate target pose estimation in face of the quantization and systematic errors inherent in a LiDAR image of a target, and (2) a fitting method for the LiDAR to monocular camera transformation that fundamentally assumes the camera image data is the most accurate information in one's possession. △ Less

Submitted 18 July, 2020; v1 submitted 7 October, 2019; originally announced October 2019.

Journal ref: IEEE Access, vol. 8, 2020, pp. 134101-134110

Adaptive Continuous Visual Odometry from RGB-D Images

Authors: Tzu-Yuan Lin, William Clark, Ryan M. Eustice, Jessy W. Grizzle, Anthony Bloch, Maani Ghaffari

Abstract: In this paper, we extend the recently developed continuous visual odometry framework for RGB-D cameras to an adaptive framework via online hyperparameter learning. We focus on the case of isotropic kernels with a scalar as the length-scale. In practice and as expected, the length-scale has remarkable impacts on the performance of the original framework. Previously it was handled using a fixed set… ▽ More In this paper, we extend the recently developed continuous visual odometry framework for RGB-D cameras to an adaptive framework via online hyperparameter learning. We focus on the case of isotropic kernels with a scalar as the length-scale. In practice and as expected, the length-scale has remarkable impacts on the performance of the original framework. Previously it was handled using a fixed set of conditions within the solver to reduce the length-scale as the algorithm reaches a local minimum. We automate this process by a greedy gradient descent step at each iteration to find the next-best length-scale. Furthermore, to handle failure cases in the gradient descent step where the gradient is not well-behaved, such as the absence of structure or texture in the scene, we use a search interval for the length-scale and guide it gradually toward the smaller values. This latter strategy reverts the adaptive framework to the original setup. The experimental evaluations using publicly available RGB-D benchmarks show the proposed adaptive continuous visual odometry outperforms the original framework and the current state-of-the-art. We also make the software for the developed algorithm publicly available. △ Less

Submitted 1 October, 2019; originally announced October 2019.

Comments: 7 pages

MSC Class: 53B21; 46C05; 46C07; 68T40; 68T45; 93C85 ACM Class: I.2.9; I.2.10; I.4.10; G.1.6; G.4

Bayesian Spatial Kernel Smoothing for Scalable Dense Semantic Map**

Authors: Lu Gan, Ray Zhang, Jessy W. Grizzle, Ryan M. Eustice, Maani Ghaffari

Abstract: This paper develops a Bayesian continuous 3D semantic occupancy map from noisy point clouds by generalizing the Bayesian kernel inference model for building occupancy maps, a binary problem, to semantic maps, a multi-class problem. The proposed method provides a unified probabilistic model for both occupancy and semantic probabilities and nicely reverts to the original occupancy map** framework… ▽ More This paper develops a Bayesian continuous 3D semantic occupancy map from noisy point clouds by generalizing the Bayesian kernel inference model for building occupancy maps, a binary problem, to semantic maps, a multi-class problem. The proposed method provides a unified probabilistic model for both occupancy and semantic probabilities and nicely reverts to the original occupancy map** framework when only one occupied class exists in obtained measurements. The Bayesian spatial kernel inference relaxes the independent grid assumption and brings smoothness and continuity to the map inference, enabling to exploit local correlations present in the environment and increasing the performance. The accompanying software uses multi-threading and vectorization, and runs at about 2 Hz on a laptop CPU. Evaluations using multiple sequences of stereo camera and LiDAR datasets show that the proposed method consistently outperforms current baselines. We also present a qualitative evaluation using data collected with a bipedal robot platform on the University of Michigan - North Campus. △ Less

Submitted 26 January, 2020; v1 submitted 10 September, 2019; originally announced September 2019.

LiDARTag: A Real-Time Fiducial Tag System for Point Clouds

Authors: Jiunn-Kai Huang, Shoutian Wang, Maani Ghaffari, Jessy W. Grizzle

Abstract: Image-based fiducial markers are useful in problems such as object tracking in cluttered or textureless environments, camera (and multi-sensor) calibration tasks, and vision-based simultaneous localization and map** (SLAM). The state-of-the-art fiducial marker detection algorithms rely on the consistency of the ambient lighting. This paper introduces LiDARTag, a novel fiducial tag design and det… ▽ More Image-based fiducial markers are useful in problems such as object tracking in cluttered or textureless environments, camera (and multi-sensor) calibration tasks, and vision-based simultaneous localization and map** (SLAM). The state-of-the-art fiducial marker detection algorithms rely on the consistency of the ambient lighting. This paper introduces LiDARTag, a novel fiducial tag design and detection algorithm suitable for light detection and ranging (LiDAR) point clouds. The proposed method runs in real-time and can process data at 100 Hz, which is faster than the currently available LiDAR sensor frequencies. Because of the LiDAR sensors' nature, rapidly changing ambient lighting will not affect the detection of a LiDARTag; hence, the proposed fiducial marker can operate in a completely dark environment. In addition, the LiDARTag nicely complements and is compatible with existing visual fiducial markers, such as AprilTags, allowing for efficient multi-sensor fusion and calibration tasks. We further propose a concept of minimizing a fitting error between a point cloud and the marker's template to estimate the marker's pose. The proposed method achieves millimeter error in translation and a few degrees in rotation. Due to LiDAR returns' sparsity, the point cloud is lifted to a continuous function in a reproducing kernel Hilbert space where the inner product can be used to determine a marker's ID. The experimental results, verified by a motion capture system, confirm that the proposed method can reliably provide a tag's pose and unique ID code. The rejection of false positives is validated on the Google Cartographer indoor dataset and the Honda H3D outdoor dataset. All implementations are coded in C++ and are available at: https://github.com/UMich-BipedLab/LiDARTag. △ Less

Submitted 13 February, 2021; v1 submitted 23 August, 2019; originally announced August 2019.

Journal ref: IEEE Robotics and Automation Letters, 31 March 2021

Contact-Aided Invariant Extended Kalman Filtering for Robot State Estimation

Authors: Ross Hartley, Maani Ghaffari, Ryan M. Eustice, Jessy W. Grizzle

Abstract: Legged robots require knowledge of pose and velocity in order to maintain stability and execute walking paths. Current solutions either rely on vision data, which is susceptible to environmental and lighting conditions, or fusion of kinematic and contact data with measurements from an inertial measurement unit (IMU). In this work, we develop a contact-aided invariant extended Kalman filter (InEKF)… ▽ More Legged robots require knowledge of pose and velocity in order to maintain stability and execute walking paths. Current solutions either rely on vision data, which is susceptible to environmental and lighting conditions, or fusion of kinematic and contact data with measurements from an inertial measurement unit (IMU). In this work, we develop a contact-aided invariant extended Kalman filter (InEKF) using the theory of Lie groups and invariant observer design. This filter combines contact-inertial dynamics with forward kinematic corrections to estimate pose and velocity along with all current contact points. We show that the error dynamics follows a log-linear autonomous differential equation with several important consequences: (a) the observable state variables can be rendered convergent with a domain of attraction that is independent of the system's trajectory; (b) unlike the standard EKF, neither the linearized error dynamics nor the linearized observation model depend on the current state estimate, which (c) leads to improved convergence properties and (d) a local observability matrix that is consistent with the underlying nonlinear system. Furthermore, we demonstrate how to include IMU biases, add/remove contacts, and formulate both world-centric and robo-centric versions. We compare the convergence of the proposed InEKF with the commonly used quaternion-based EKF though both simulations and experiments on a Cassie-series bipedal robot. Filter accuracy is analyzed using motion capture, while a LiDAR map** experiment provides a practical use case. Overall, the developed contact-aided InEKF provides better performance in comparison with the quaternion-based EKF as a result of exploiting symmetries present in system. △ Less

Submitted 10 November, 2019; v1 submitted 19 April, 2019; originally announced April 2019.

Comments: arXiv admin note: text overlap with arXiv:1805.10410

Continuous Direct Sparse Visual Odometry from RGB-D Images

Authors: Maani Ghaffari, William Clark, Anthony Bloch, Ryan M. Eustice, Jessy W. Grizzle

Abstract: This paper reports on a novel formulation and evaluation of visual odometry from RGB-D images. Assuming a static scene, the developed theoretical framework generalizes the widely used direct energy formulation (photometric error minimization) technique for obtaining a rigid body transformation that aligns two overlap** RGB-D images to a continuous formulation. The continuity is achieved through… ▽ More This paper reports on a novel formulation and evaluation of visual odometry from RGB-D images. Assuming a static scene, the developed theoretical framework generalizes the widely used direct energy formulation (photometric error minimization) technique for obtaining a rigid body transformation that aligns two overlap** RGB-D images to a continuous formulation. The continuity is achieved through functional treatment of the problem and representing the process models over RGB-D images in a reproducing kernel Hilbert space; consequently, the registration is not limited to the specific image resolution and the framework is fully analytical with a closed-form derivation of the gradient. We solve the problem by maximizing the inner product between two functions defined over RGB-D images, while the continuous action of the rigid body motion Lie group is captured through the integration of the flow in the corresponding Lie algebra. Energy-based approaches have been extremely successful and the developed framework in this paper shares many of their desired properties such as the parallel structure on both CPUs and GPUs, sparsity, semi-dense tracking, avoiding explicit data association which is computationally expensive, and possible extensions to the simultaneous localization and map** frameworks. The evaluations on experimental data and comparison with the equivalent energy-based formulation of the problem confirm the effectiveness of the proposed technique, especially, when the lack of structure and texture in the environment is evident. △ Less

Submitted 23 August, 2019; v1 submitted 3 April, 2019; originally announced April 2019.

Comments: 9 pages

MSC Class: 53B21; 46C05; 46C07; 68T40; 68T45; 93C85 ACM Class: I.2.9; I.2.10; I.4.10; G.1.6; G.4

Rapid Trajectory Optimization Using C-FROST with Illustration on a Cassie-Series Dynamic Walking Biped

Authors: Ayonga Hereid, Omar Harib, Ross Hartley, Yukai Gong, Jessy W. Grizzle

Abstract: One of the big attractions of low-dimensional models for gait design has been the ability to compute solutions rapidly, whereas one of their drawbacks has been the difficulty in map** the solutions back to the target robot. This paper presents a set of tools for rapidly determining solutions for ``humanoids'' without removing or lum** degrees of freedom. The main tools are (1) C-FROST, an open… ▽ More One of the big attractions of low-dimensional models for gait design has been the ability to compute solutions rapidly, whereas one of their drawbacks has been the difficulty in map** the solutions back to the target robot. This paper presents a set of tools for rapidly determining solutions for ``humanoids'' without removing or lum** degrees of freedom. The main tools are (1) C-FROST, an open-source C++ interface for FROST, a direct collocation optimization tool; and (2) multi-threading. The results will be illustrated on a 20-DoF floating-base model for a Cassie-series bipedal robot through numerical calculations and physical experiments. △ Less

Submitted 15 March, 2019; v1 submitted 17 July, 2018; originally announced July 2018.

Contact-Aided Invariant Extended Kalman Filtering for Legged Robot State Estimation

Authors: Ross Hartley, Maani Ghaffari Jadidi, Jessy W. Grizzle, Ryan M. Eustice

Abstract: This paper derives a contact-aided inertial navigation observer for a 3D bipedal robot using the theory of invariant observer design. Aided inertial navigation is fundamentally a nonlinear observer design problem; thus, current solutions are based on approximations of the system dynamics, such as an Extended Kalman Filter (EKF), which uses a system's Jacobian linearization along the current best e… ▽ More This paper derives a contact-aided inertial navigation observer for a 3D bipedal robot using the theory of invariant observer design. Aided inertial navigation is fundamentally a nonlinear observer design problem; thus, current solutions are based on approximations of the system dynamics, such as an Extended Kalman Filter (EKF), which uses a system's Jacobian linearization along the current best estimate of its trajectory. On the basis of the theory of invariant observer design by Barrau and Bonnabel, and in particular, the Invariant EKF (InEKF), we show that the error dynamics of the point contact-inertial system follows a log-linear autonomous differential equation; hence, the observable state variables can be rendered convergent with a domain of attraction that is independent of the system's trajectory. Due to the log-linear form of the error dynamics, it is not necessary to perform a nonlinear observability analysis to show that when using an Inertial Measurement Unit (IMU) and contact sensors, the absolute position of the robot and a rotation about the gravity vector (yaw) are unobservable. We further augment the state of the developed InEKF with IMU biases, as the online estimation of these parameters has a crucial impact on system performance. We evaluate the convergence of the proposed system with the commonly used quaternion-based EKF observer using a Monte-Carlo simulation. In addition, our experimental evaluation using a Cassie-series bipedal robot shows that the contact-aided InEKF provides better performance in comparison with the quaternion-based EKF as a result of exploiting symmetries present in the system dynamics. △ Less

Submitted 25 May, 2018; originally announced May 2018.

Comments: Published in the proceedings of Robotics: Science and Systems 2018

Hybrid Contact Preintegration for Visual-Inertial-Contact State Estimation Using Factor Graphs

Authors: Ross Hartley, Maani Ghaffari Jadidi, Lu Gan, Jiunn-Kai Huang, Jessy W. Grizzle, Ryan M. Eustice

Abstract: The factor graph framework is a convenient modeling technique for robotic state estimation where states are represented as nodes, and measurements are modeled as factors. When designing a sensor fusion framework for legged robots, one often has access to visual, inertial, joint encoder, and contact sensors. While visual-inertial odometry has been studied extensively in this framework, the addition… ▽ More The factor graph framework is a convenient modeling technique for robotic state estimation where states are represented as nodes, and measurements are modeled as factors. When designing a sensor fusion framework for legged robots, one often has access to visual, inertial, joint encoder, and contact sensors. While visual-inertial odometry has been studied extensively in this framework, the addition of a preintegrated contact factor for legged robots has been only recently proposed. This allowed for integration of encoder and contact measurements into existing factor graphs, however, new nodes had to be added to the graph every time contact was made or broken. In this work, to cope with the problem of switching contact frames, we propose a hybrid contact preintegration theory that allows contact information to be integrated through an arbitrary number of contact switches. The proposed hybrid modeling approach reduces the number of required variables in the nonlinear optimization problem by only requiring new states to be added alongside camera or selected keyframes. This method is evaluated using real experimental data collected from a Cassie-series robot where the trajectory of the robot produced by a motion capture system is used as a proxy for ground truth. The evaluation shows that inclusion of the proposed preintegrated hybrid contact factor alongside visual-inertial navigation systems improves estimation accuracy as well as robustness to vision failure, while its generalization makes it more accessible for legged platforms. △ Less

Submitted 2 October, 2018; v1 submitted 20 March, 2018; originally announced March 2018.

Comments: Detailed derivations are provided in the supplementary material document listed under "Ancillary files"

Legged Robot State-Estimation Through Combined Forward Kinematic and Preintegrated Contact Factors

Authors: Ross Hartley, Josh Mangelson, Lu Gan, Maani Ghaffari Jadidi, Jeffrey M. Walls, Ryan M. Eustice, Jessy W. Grizzle

Abstract: State-of-the-art robotic perception systems have achieved sufficiently good performance using Inertial Measurement Units (IMUs), cameras, and nonlinear optimization techniques, that they are now being deployed as technologies. However, many of these methods rely significantly on vision and often fail when visual tracking is lost due to lighting or scarcity of features. This paper presents a state-… ▽ More State-of-the-art robotic perception systems have achieved sufficiently good performance using Inertial Measurement Units (IMUs), cameras, and nonlinear optimization techniques, that they are now being deployed as technologies. However, many of these methods rely significantly on vision and often fail when visual tracking is lost due to lighting or scarcity of features. This paper presents a state-estimation technique for legged robots that takes into account the robot's kinematic model as well as its contact with the environment. We introduce forward kinematic factors and preintegrated contact factors into a factor graph framework that can be incrementally solved in real-time. The forward kinematic factor relates the robot's base pose to a contact frame through noisy encoder measurements. The preintegrated contact factor provides odometry measurements of this contact frame while accounting for possible foot slippage. Together, the two developed factors constrain the graph optimization problem allowing the robot's trajectory to be estimated. The paper evaluates the method using simulated and real sensory IMU and kinematic data from experiments with a Cassie-series robot designed by Agility Robotics. These preliminary experiments show that using the proposed method in addition to IMU decreases drift and improves localization accuracy, suggesting that its use can enable successful recovery from a loss of visual tracking. △ Less

Submitted 24 February, 2018; v1 submitted 15 December, 2017; originally announced December 2017.

Comments: Detailed derivations are provided in the supplementary material document listed under "Ancillary files"

Virtual Constraints and Hybrid Zero Dynamics for Realizing Underactuated Bipedal Locomotion

Authors: Jessy W Grizzle, Christine Chevallereau

Abstract: Underactuation is ubiquitous in human locomotion and should be ubiquitous in bipedal robotic locomotion as well. This chapter presents a coherent theory for the design of feedback controllers that achieve stable walking gaits in underactuated bipedal robots. Two fundamental tools are introduced, virtual constraints and hybrid zero dynamics. Virtual constraints are relations on the state variables… ▽ More Underactuation is ubiquitous in human locomotion and should be ubiquitous in bipedal robotic locomotion as well. This chapter presents a coherent theory for the design of feedback controllers that achieve stable walking gaits in underactuated bipedal robots. Two fundamental tools are introduced, virtual constraints and hybrid zero dynamics. Virtual constraints are relations on the state variables of a mechanical model that are imposed through a time-invariant feedback controller. One of their roles is to synchronize the robot's joints to an internal gait phasing variable. A second role is to induce a low dimensional system, the zero dynamics, that captures the underactuated aspects of a robot's model, without any approximations. To enhance intuition, the relation between physical constraints and virtual constraints is first established. From here, the hybrid zero dynamics of an underactuated bipedal model is developed, and its fundamental role in the design of asymptotically stable walking motions is established. The chapter includes numerous references to robots on which the highlighted techniques have been implemented. △ Less

Submitted 4 June, 2017; originally announced June 2017.

Comments: 17 pages, 4 figures, bookchapter

Restricted Discrete Invariance and Self-Synchronization For Stable Walking of Bipedal Robots

Authors: Hamed Razavi, Anthony M. Bloch, Christine Chevallereau, J. W. Grizzle

Abstract: Models of bipedal locomotion are hybrid, with a continuous component often generated by a Lagrangian plus actuators, and a discrete component where leg transfer takes place. The discrete component typically consists of a locally embedded co-dimension one submanifold in the continuous state space of the robot, called the switching surface, and a reset map that provides a new initial condition when… ▽ More Models of bipedal locomotion are hybrid, with a continuous component often generated by a Lagrangian plus actuators, and a discrete component where leg transfer takes place. The discrete component typically consists of a locally embedded co-dimension one submanifold in the continuous state space of the robot, called the switching surface, and a reset map that provides a new initial condition when a solution of the continuous component intersects the switching surface. The aim of this paper is to identify a low-dimensional submanifold of the switching surface, which, when it can be rendered invariant by the closed-loop dynamics, leads to asymptotically stable periodic gaits. The paper begins this process by studying the well-known 3D Linear Inverted Pendulum (LIP) model, where analytical results are much easier to obtain. A key contribution here is the notion of \textit{self-synchronization}, which refers to the periods of the pendular motions in the sagittal and frontal planes tending to a common period. The notion of invariance resulting from the study of the 3D LIP model is then extended to a 9-DOF 3D biped. A numerical study is performed to illustrate that asymptotically stable walking may be obtained. △ Less

Submitted 9 June, 2016; v1 submitted 1 November, 2014; originally announced November 2014.

Comments: Conference

Torque Saturation in Bipedal Robotic Walking through Control Lyapunov Function Based Quadratic Programs

Authors: Kevin Galloway, Koushil Sreenath, Aaron D. Ames, J. W. Grizzle

Abstract: This paper presents a novel method for directly incorporating user-defined control input saturations into the calculation of a control Lyapunov function (CLF)-based walking controller for a biped robot. Previous work by the authors has demonstrated the effectiveness of CLF controllers for stabilizing periodic gaits for biped walkers, and the current work expands on those results by providing a mor… ▽ More This paper presents a novel method for directly incorporating user-defined control input saturations into the calculation of a control Lyapunov function (CLF)-based walking controller for a biped robot. Previous work by the authors has demonstrated the effectiveness of CLF controllers for stabilizing periodic gaits for biped walkers, and the current work expands on those results by providing a more effective means for handling control saturations. The new approach, based on a convex optimization routine running at a 1 kHz control update rate, is useful not only for handling torque saturations but also for incorporating a whole family of user-defined constraints into the online computation of a CLF controller. The paper concludes with an experimental implementation of the main results on the bipedal robot MABEL. △ Less

Submitted 28 February, 2013; originally announced February 2013.

Asymptotically Stable Walking of a Five-Link Underactuated 3D Bipedal Robot

Authors: Christine Chevallereau, Jessy W. Grizzle, Ching-Long Shih

Abstract: This paper presents three feedback controllers that achieve an asymptotically stable, periodic, and fast walking gait for a 3D (spatial) bipedal robot consisting of a torso, two legs, and passive (unactuated) point feet. The contact between the robot and the walking surface is assumed to inhibit yaw rotation. The studied robot has 8 DOF in the single support phase and 6 actuators. The interest o… ▽ More This paper presents three feedback controllers that achieve an asymptotically stable, periodic, and fast walking gait for a 3D (spatial) bipedal robot consisting of a torso, two legs, and passive (unactuated) point feet. The contact between the robot and the walking surface is assumed to inhibit yaw rotation. The studied robot has 8 DOF in the single support phase and 6 actuators. The interest of studying robots with point feet is that the robot's natural dynamics must be explicitly taken into account to achieve balance while walking. We use an extension of the method of virtual constraints and hybrid zero dynamics, in order to simultaneously compute a periodic orbit and an autonomous feedback controller that realizes the orbit. This method allows the computations to be carried out on a 2-DOF subsystem of the 8-DOF robot model. The stability of the walking gait under closed-loop control is evaluated with the linearization of the restricted Poincaré map of the hybrid zero dynamics. Three strategies are explored. The first strategy consists of imposing a stability condition during the search of a periodic gait by optimization. The second strategy uses an event-based controller. In the third approach, the effect of output selection is discussed and a pertinent choice of outputs is proposed, leading to stabilization without the use of a supplemental event-based controller. △ Less

Submitted 17 February, 2010; originally announced February 2010.

Journal ref: IEEE Transactions on Robotics 25, 1 (2009) 37-50