-
The Kinetics Observer: A Tightly Coupled Estimator for Legged Robots
Authors:
Arnaud Demont,
Mehdi Benallegue,
Abdelaziz Benallegue,
Pierre Gergondet,
Antonin Dallard,
Rafael Cisneros,
Masaki Murooka,
Fumio Kanehiro
Abstract:
In this paper, we propose the "Kinetics Observer", a novel estimator addressing the challenge of state estimation for legged robots using proprioceptive sensors (encoders, IMU and force/torque sensors). Based on a Multiplicative Extended Kalman Filter, the Kinetics Observer allows the real-time simultaneous estimation of contact and perturbation forces, and of the robot's kinematics, which are acc…
▽ More
In this paper, we propose the "Kinetics Observer", a novel estimator addressing the challenge of state estimation for legged robots using proprioceptive sensors (encoders, IMU and force/torque sensors). Based on a Multiplicative Extended Kalman Filter, the Kinetics Observer allows the real-time simultaneous estimation of contact and perturbation forces, and of the robot's kinematics, which are accurate enough to perform proprioceptive odometry. Thanks to a visco-elastic model of the contacts linking their kinematics to the ones of the centroid of the robot, the Kinetics Observer ensures a tight coupling between the whole-body kinematics and dynamics of the robot. This coupling entails a redundancy of the measurements that enhances the robustness and the accuracy of the estimation. This estimator was tested on two humanoid robots performing long distance walking on even terrain and non-coplanar multi-contact locomotion.
△ Less
Submitted 19 June, 2024;
originally announced June 2024.
-
Enhanced Visual Feedback with Decoupled Viewpoint Control in Immersive Humanoid Robot Teleoperation using SLAM
Authors:
Yang Chen,
Leyuan Sun,
Mehdi Benallegue,
Rafael Cisneros,
Rohan P. Singh,
Kenji Kaneko,
Arnaud Tanguy,
Guillaume Caron,
Kenji Suzuki,
Abderrahmane Kheddar,
Fumio Kanehiro
Abstract:
In immersive humanoid robot teleoperation, there are three main shortcomings that can alter the transparency of the visual feedback: the lag between the motion of the operator's and robot's head due to network communication delays or slow robot joint motion. This latency could cause a noticeable delay in the visual feedback, which jeopardizes the embodiment quality, can cause dizziness, and affect…
▽ More
In immersive humanoid robot teleoperation, there are three main shortcomings that can alter the transparency of the visual feedback: the lag between the motion of the operator's and robot's head due to network communication delays or slow robot joint motion. This latency could cause a noticeable delay in the visual feedback, which jeopardizes the embodiment quality, can cause dizziness, and affects the interactivity resulting in operator frequent motion pauses for the visual feedback to settle; (ii) the mismatch between the camera's and the headset's field-of-views (FOV), the former having generally a lower FOV; and (iii) a mismatch between human's and robot's range of motions of the neck, the latter being also generally lower. In order to leverage these drawbacks, we developed a decoupled viewpoint control solution for a humanoid platform which allows visual feedback with low-latency and artificially increases the camera's FOV range to match that of the operator's headset. Our novel solution uses SLAM technology to enhance the visual feedback from a reconstructed mesh, complementing the areas that are not covered by the visual feedback from the robot. The visual feedback is presented as a point cloud in real-time to the operator. As a result, the operator is fed with real-time vision from the robot's head orientation by observing the pose of the point cloud. Balancing this kind of awareness and immersion is important in virtual reality based teleoperation, considering the safety and robustness of the control system. An experiment shows the effectiveness of our solution.
△ Less
Submitted 3 November, 2022;
originally announced November 2022.
-
Instance-specific 6-DoF Object Pose Estimation from Minimal Annotations
Authors:
Rohan Pratap Singh,
Iori Kumagai,
Antonio Gabas,
Mehdi Benallegue,
Yusuke Yoshiyasu,
Fumio Kanehiro
Abstract:
In many robotic applications, the environment setting in which the 6-DoF pose estimation of a known, rigid object and its subsequent gras** is to be performed, remains nearly unchanging and might even be known to the robot in advance. In this paper, we refer to this problem as instance-specific pose estimation: the robot is expected to estimate the pose with a high degree of accuracy in only a l…
▽ More
In many robotic applications, the environment setting in which the 6-DoF pose estimation of a known, rigid object and its subsequent gras** is to be performed, remains nearly unchanging and might even be known to the robot in advance. In this paper, we refer to this problem as instance-specific pose estimation: the robot is expected to estimate the pose with a high degree of accuracy in only a limited set of familiar scenarios. Minor changes in the scene, including variations in lighting conditions and background appearance, are acceptable but drastic alterations are not anticipated. To this end, we present a method to rapidly train and deploy a pipeline for estimating the continuous 6-DoF pose of an object from a single RGB image. The key idea is to leverage known camera poses and rigid body geometry to partially automate the generation of a large labeled dataset. The dataset, along with sufficient domain randomization, is then used to supervise the training of deep neural networks for predicting semantic keypoints. Experimentally, we demonstrate the convenience and effectiveness of our proposed method to accurately estimate object pose requiring only a very small amount of manual annotation for training.
△ Less
Submitted 26 July, 2022;
originally announced July 2022.
-
Learning Bipedal Walking On Planned Footsteps For Humanoid Robots
Authors:
Rohan Pratap Singh,
Mehdi Benallegue,
Mitsuharu Morisawa,
Rafael Cisneros,
Fumio Kanehiro
Abstract:
Deep reinforcement learning (RL) based controllers for legged robots have demonstrated impressive robustness for walking in different environments for several robot platforms. To enable the application of RL policies for humanoid robots in real-world settings, it is crucial to build a system that can achieve robust walking in any direction, on 2D and 3D terrains, and be controllable by a user-comm…
▽ More
Deep reinforcement learning (RL) based controllers for legged robots have demonstrated impressive robustness for walking in different environments for several robot platforms. To enable the application of RL policies for humanoid robots in real-world settings, it is crucial to build a system that can achieve robust walking in any direction, on 2D and 3D terrains, and be controllable by a user-command. In this paper, we tackle this problem by learning a policy to follow a given step sequence. The policy is trained with the help of a set of procedurally generated step sequences (also called footstep plans). We show that simply feeding the upcoming 2 steps to the policy is sufficient to achieve omnidirectional walking, turning in place, standing, and climbing stairs. Our method employs curriculum learning on the complexity of terrains, and circumvents the need for reference motions or pre-trained weights. We demonstrate the application of our proposed method to learn RL policies for 2 new robot platforms - HRP5P and JVRC-1 - in the MuJoCo simulation environment. The code for training and evaluation is available online.
△ Less
Submitted 31 October, 2022; v1 submitted 26 July, 2022;
originally announced July 2022.
-
Rapid Pose Label Generation through Sparse Representation of Unknown Objects
Authors:
Rohan Pratap Singh,
Mehdi Benallegue,
Yusuke Yoshiyasu,
Fumio Kanehiro
Abstract:
Deep Convolutional Neural Networks (CNNs) have been successfully deployed on robots for 6-DoF object pose estimation through visual perception. However, obtaining labeled data on a scale required for the supervised training of CNNs is a difficult task - exacerbated if the object is novel and a 3D model is unavailable. To this end, this work presents an approach for rapidly generating real-world, p…
▽ More
Deep Convolutional Neural Networks (CNNs) have been successfully deployed on robots for 6-DoF object pose estimation through visual perception. However, obtaining labeled data on a scale required for the supervised training of CNNs is a difficult task - exacerbated if the object is novel and a 3D model is unavailable. To this end, this work presents an approach for rapidly generating real-world, pose-annotated RGB-D data for unknown objects. Our method not only circumvents the need for a prior 3D object model (textured or otherwise) but also bypasses complicated setups of fiducial markers, turntables, and sensors. With the help of a human user, we first source minimalistic labelings of an ordered set of arbitrarily chosen keypoints over a set of RGB-D videos. Then, by solving an optimization problem, we combine these labels under a world frame to recover a sparse, keypoint-based representation of the object. The sparse representation leads to the development of a dense model and the pose labels for each image frame in the set of scenes. We show that the sparse model can also be efficiently used for scaling to a large number of new scenes. We demonstrate the practicality of the generated labeled dataset by training a pipeline for 6-DoF object pose estimation and a pixel-wise segmentation network.
△ Less
Submitted 7 November, 2020;
originally announced November 2020.
-
On the mechanical contribution of head stabilization to passive dynamics of anthropometric walkers
Authors:
Mehdi Benallegue,
Jean-Paul Laumond,
Alain Berthoz
Abstract:
During the steady gait, humans stabilize their head around the vertical orientation. While there are sensori-cognitive explanations for this phenomenon, its mechanical e fect on the body dynamics remains un-explored. In this study, we take profit from the similarities that human steady gait share with the locomotion of passive dynamics robots. We introduce a simplified anthropometric D model to re…
▽ More
During the steady gait, humans stabilize their head around the vertical orientation. While there are sensori-cognitive explanations for this phenomenon, its mechanical e fect on the body dynamics remains un-explored. In this study, we take profit from the similarities that human steady gait share with the locomotion of passive dynamics robots. We introduce a simplified anthropometric D model to reproduce a broad walking dynamics. In a previous study, we showed heuristically that the presence of a stabilized head-neck system significantly influences the dynamics of walking. This paper gives new insights that lead to understanding this mechanical e fect. In particular, we introduce an original cart upper-body model that allows to better understand the mechanical interest of head stabilization when walking, and we study how this e fect is sensitive to the choice of control parameters.
△ Less
Submitted 29 October, 2020; v1 submitted 23 October, 2020;
originally announced October 2020.
-
Lyapunov-Stable Orientation Estimator for Humanoid Robots
Authors:
Mehdi Benallegue,
Rafael Cisneros,
Abdelaziz Benallegue,
Yacine Chitour,
Mitsuharu Morisawa,
Fumio Kanehiro
Abstract:
In this paper, we present an observation scheme, with proven Lyapunov stability, for estimating a humanoid's floating base orientation. The idea is to use velocity aided attitude estimation, which requires to know the velocity of the system. This velocity can be obtained by taking into account the kinematic data provided by contact information with the environment and using the IMU and joint encod…
▽ More
In this paper, we present an observation scheme, with proven Lyapunov stability, for estimating a humanoid's floating base orientation. The idea is to use velocity aided attitude estimation, which requires to know the velocity of the system. This velocity can be obtained by taking into account the kinematic data provided by contact information with the environment and using the IMU and joint encoders. We demonstrate how this operation can be used in the case of a fixed or a moving contact, allowing it to be employed for locomotion. We show how to use this velocity estimation within a selected two-stage state tilt estimator: (i) the first which has a global and quick convergence (ii) and the second which has smooth and robust dynamics. We provide new specific proofs of almost global Lyapunov asymptotic stability and local exponential convergence for this observer. Finally, we assess its performance by employing a comparative simulation and by using it within a closed-loop stabilization scheme for HRP-5P and HRP-2KAI robots performing whole-body kinematic tasks and locomotion.
△ Less
Submitted 9 October, 2020;
originally announced October 2020.
-
Tilt estimator for 3D non-rigid pendulum based on a tri-axial accelerometer and gyrometer
Authors:
Mehdi Benallegue,
Abdelaziz Benallegue,
Yacine Chitour
Abstract:
The paper presents a new observer for tilt estimation of a 3-D non-rigid pendulum. The system can be seen as a multibody robot attached to the environment with a ball joint. There is no sensor for the joint position of the sensor. The estimation of tilt, i.e. roll and pitch angles, is mandatory for balance control for a humanoid robot and all tasks requiring verticality. Our method obtains tilt es…
▽ More
The paper presents a new observer for tilt estimation of a 3-D non-rigid pendulum. The system can be seen as a multibody robot attached to the environment with a ball joint. There is no sensor for the joint position of the sensor. The estimation of tilt, i.e. roll and pitch angles, is mandatory for balance control for a humanoid robot and all tasks requiring verticality. Our method obtains tilt estimations using encoders on other joints and inertial measurements given by an IMU equipped with tri-axial accelerometer and gyrometer mounted in any body of the robot. The estimator takes profit from the kinematic coupling resulting from the pivot constraint and uses the entire signal of accelerometer including linear accelerations. Almost Global Asymptotic convergence of the estimation errors is proven together with local exponential stability. The performance of the proposed observer is illustrated by simulations.
△ Less
Submitted 31 October, 2018;
originally announced October 2018.
