Showing 1–13 of 13 results for author: Kanehiro, F

The Kinetics Observer: A Tightly Coupled Estimator for Legged Robots

Authors: Arnaud Demont, Mehdi Benallegue, Abdelaziz Benallegue, Pierre Gergondet, Antonin Dallard, Rafael Cisneros, Masaki Murooka, Fumio Kanehiro

Abstract: In this paper, we propose the "Kinetics Observer", a novel estimator addressing the challenge of state estimation for legged robots using proprioceptive sensors (encoders, IMU and force/torque sensors). Based on a Multiplicative Extended Kalman Filter, the Kinetics Observer allows the real-time simultaneous estimation of contact and perturbation forces, and of the robot's kinematics, which are acc… ▽ More In this paper, we propose the "Kinetics Observer", a novel estimator addressing the challenge of state estimation for legged robots using proprioceptive sensors (encoders, IMU and force/torque sensors). Based on a Multiplicative Extended Kalman Filter, the Kinetics Observer allows the real-time simultaneous estimation of contact and perturbation forces, and of the robot's kinematics, which are accurate enough to perform proprioceptive odometry. Thanks to a visco-elastic model of the contacts linking their kinematics to the ones of the centroid of the robot, the Kinetics Observer ensures a tight coupling between the whole-body kinematics and dynamics of the robot. This coupling entails a redundancy of the measurements that enhances the robustness and the accuracy of the estimation. This estimator was tested on two humanoid robots performing long distance walking on even terrain and non-coplanar multi-contact locomotion. △ Less

Submitted 19 June, 2024; originally announced June 2024.

Analysis and Perspectives on the ANA Avatar XPRIZE Competition

Authors: Kris Hauser, Eleanor Watson, Joonbum Bae, Josh Bankston, Sven Behnke, Bill Borgia, Manuel G. Catalano, Stefano Dafarra, Jan B. F. van Erp, Thomas Ferris, Jeremy Fishel, Guy Hoffman, Serena Ivaldi, Fumio Kanehiro, Abderrahmane Kheddar, Gaelle Lannuzel, Jacqueline Ford Morie, Patrick Naughton, Steve NGuyen, Paul Oh, Taskin Padir, Jim Pippine, Jaeheung Park, Daniele Pucci, Jean Vaz , et al. (3 additional authors not shown)

Abstract: The ANA Avatar XPRIZE was a four-year competition to develop a robotic "avatar" system to allow a human operator to sense, communicate, and act in a remote environment as though physically present. The competition featured a unique requirement that judges would operate the avatars after less than one hour of training on the human-machine interfaces, and avatar systems were judged on both objective… ▽ More The ANA Avatar XPRIZE was a four-year competition to develop a robotic "avatar" system to allow a human operator to sense, communicate, and act in a remote environment as though physically present. The competition featured a unique requirement that judges would operate the avatars after less than one hour of training on the human-machine interfaces, and avatar systems were judged on both objective and subjective scoring metrics. This paper presents a unified summary and analysis of the competition from technical, judging, and organizational perspectives. We study the use of telerobotics technologies and innovations pursued by the competing teams in their avatar systems, and correlate the use of these technologies with judges' task performance and subjective survey ratings. It also summarizes perspectives from team leads, judges, and organizers about the competition's execution and impact to inform the future development of telerobotics and telepresence. △ Less

Submitted 10 January, 2024; originally announced January 2024.

Comments: 26 pages, preprint of article appearing in International Journal of Social Robotics

TransFusionOdom: Interpretable Transformer-based LiDAR-Inertial Fusion Odometry Estimation

Authors: Leyuan Sun, Guanqun Ding, Yue Qiu, Yusuke Yoshiyasu, Fumio Kanehiro

Abstract: Multi-modal fusion of sensors is a commonly used approach to enhance the performance of odometry estimation, which is also a fundamental module for mobile robots. However, the question of \textit{how to perform fusion among different modalities in a supervised sensor fusion odometry estimation task?} is still one of challenging issues remains. Some simple operations, such as element-wise summation… ▽ More Multi-modal fusion of sensors is a commonly used approach to enhance the performance of odometry estimation, which is also a fundamental module for mobile robots. However, the question of \textit{how to perform fusion among different modalities in a supervised sensor fusion odometry estimation task?} is still one of challenging issues remains. Some simple operations, such as element-wise summation and concatenation, are not capable of assigning adaptive attentional weights to incorporate different modalities efficiently, which make it difficult to achieve competitive odometry results. Recently, the Transformer architecture has shown potential for multi-modal fusion tasks, particularly in the domains of vision with language. In this work, we propose an end-to-end supervised Transformer-based LiDAR-Inertial fusion framework (namely TransFusionOdom) for odometry estimation. The multi-attention fusion module demonstrates different fusion approaches for homogeneous and heterogeneous modalities to address the overfitting problem that can arise from blindly increasing the complexity of the model. Additionally, to interpret the learning process of the Transformer-based multi-modal interactions, a general visualization approach is introduced to illustrate the interactions between modalities. Moreover, exhaustive ablation studies evaluate different multi-modal fusion strategies to verify the performance of the proposed fusion strategy. A synthetic multi-modal dataset is made public to validate the generalization ability of the proposed fusion strategy, which also works for other combinations of different modalities. The quantitative and qualitative odometry evaluations on the KITTI dataset verify the proposed TransFusionOdom could achieve superior performance compared with other related works. △ Less

Submitted 25 April, 2023; v1 submitted 16 April, 2023; originally announced April 2023.

Comments: Submitted to IEEE Sensors Journal with some modifications. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Learning Bipedal Walking for Humanoids with Current Feedback

Authors: Rohan Pratap Singh, Zhaoming Xie, Pierre Gergondet, Fumio Kanehiro

Abstract: Recent advances in deep reinforcement learning (RL) based techniques combined with training in simulation have offered a new approach to develo** robust controllers for legged robots. However, the application of such approaches to real hardware has largely been limited to quadrupedal robots with direct-drive actuators and light-weight bipedal robots with low gear-ratio transmission systems. Appl… ▽ More Recent advances in deep reinforcement learning (RL) based techniques combined with training in simulation have offered a new approach to develo** robust controllers for legged robots. However, the application of such approaches to real hardware has largely been limited to quadrupedal robots with direct-drive actuators and light-weight bipedal robots with low gear-ratio transmission systems. Application to real, life-sized humanoid robots has been less common arguably due to a large sim2real gap. In this paper, we present an approach for effectively overcoming the sim2real gap issue for humanoid robots arising from inaccurate torque-tracking at the actuator level. Our key idea is to utilize the current feedback from the actuators on the real robot, after training the policy in a simulation environment artificially degraded with poor torque-tracking. Our approach successfully trains a unified, end-to-end policy in simulation that can be deployed on a real HRP-5P humanoid robot to achieve bipedal locomotion. Through ablations, we also show that a feedforward policy architecture combined with targeted dynamics randomization is sufficient for zero-shot sim2real success, thus eliminating the need for computationally expensive, memory-based network architectures. Finally, we validate the robustness of the proposed RL policy by comparing its performance against a conventional model-based controller for walking on uneven terrain with the real robot. △ Less

Submitted 7 August, 2023; v1 submitted 7 March, 2023; originally announced March 2023.

Enhanced Visual Feedback with Decoupled Viewpoint Control in Immersive Humanoid Robot Teleoperation using SLAM

Authors: Yang Chen, Leyuan Sun, Mehdi Benallegue, Rafael Cisneros, Rohan P. Singh, Kenji Kaneko, Arnaud Tanguy, Guillaume Caron, Kenji Suzuki, Abderrahmane Kheddar, Fumio Kanehiro

Abstract: In immersive humanoid robot teleoperation, there are three main shortcomings that can alter the transparency of the visual feedback: the lag between the motion of the operator's and robot's head due to network communication delays or slow robot joint motion. This latency could cause a noticeable delay in the visual feedback, which jeopardizes the embodiment quality, can cause dizziness, and affect… ▽ More In immersive humanoid robot teleoperation, there are three main shortcomings that can alter the transparency of the visual feedback: the lag between the motion of the operator's and robot's head due to network communication delays or slow robot joint motion. This latency could cause a noticeable delay in the visual feedback, which jeopardizes the embodiment quality, can cause dizziness, and affects the interactivity resulting in operator frequent motion pauses for the visual feedback to settle; (ii) the mismatch between the camera's and the headset's field-of-views (FOV), the former having generally a lower FOV; and (iii) a mismatch between human's and robot's range of motions of the neck, the latter being also generally lower. In order to leverage these drawbacks, we developed a decoupled viewpoint control solution for a humanoid platform which allows visual feedback with low-latency and artificially increases the camera's FOV range to match that of the operator's headset. Our novel solution uses SLAM technology to enhance the visual feedback from a reconstructed mesh, complementing the areas that are not covered by the visual feedback from the robot. The visual feedback is presented as a point cloud in real-time to the operator. As a result, the operator is fed with real-time vision from the robot's head orientation by observing the pose of the point cloud. Balancing this kind of awareness and immersion is important in virtual reality based teleoperation, considering the safety and robustness of the control system. An experiment shows the effectiveness of our solution. △ Less

Submitted 3 November, 2022; originally announced November 2022.

Comments: IEEE-RAS International Conference on Humanoid Robots (Humanoids 2022)

mc-mujoco: Simulating Articulated Robots with FSM Controllers in MuJoCo

Authors: Rohan P. Singh, Pierre Gergondet, Fumio Kanehiro

Abstract: For safe and reliable deployment of any robot controller on the real hardware platform, it is generally a necessary practice to comprehensively assess the performance of the controller with the specific robot in a realistic simulation environment beforehand. While there exist several software solutions that can provide the core physics engine for this purpose, it is often a cumbersome and error-pr… ▽ More For safe and reliable deployment of any robot controller on the real hardware platform, it is generally a necessary practice to comprehensively assess the performance of the controller with the specific robot in a realistic simulation environment beforehand. While there exist several software solutions that can provide the core physics engine for this purpose, it is often a cumbersome and error-prone effort to interface the simulation environment with the robot controller being evaluated. The controller may have a complex structure consisting of multiple states and transitions within a finite-state machine (FSM), and may even require input through a GUI. In this work, we present mc-mujoco -- an open-source software framework that forms an interface between the MuJoCo physics simulator and the mc-rtc robot control framework. We provide implementation details and describe the process for adding support for essentially any new robot. We also demonstrate and publish a sample FSM controller for bipedal locomotion and stable gras** of a rigid object by the HRP-5P humanoid robot in MuJoCo. The code and usage instructions for mc-mujoco, the developed robot modules, and the FSM controller are available online. △ Less

Submitted 1 September, 2022; originally announced September 2022.

Comments: GitHub code: https://github.com/rohanpsingh/mc_mujoco

Instance-specific 6-DoF Object Pose Estimation from Minimal Annotations

Authors: Rohan Pratap Singh, Iori Kumagai, Antonio Gabas, Mehdi Benallegue, Yusuke Yoshiyasu, Fumio Kanehiro

Abstract: In many robotic applications, the environment setting in which the 6-DoF pose estimation of a known, rigid object and its subsequent gras** is to be performed, remains nearly unchanging and might even be known to the robot in advance. In this paper, we refer to this problem as instance-specific pose estimation: the robot is expected to estimate the pose with a high degree of accuracy in only a l… ▽ More In many robotic applications, the environment setting in which the 6-DoF pose estimation of a known, rigid object and its subsequent gras** is to be performed, remains nearly unchanging and might even be known to the robot in advance. In this paper, we refer to this problem as instance-specific pose estimation: the robot is expected to estimate the pose with a high degree of accuracy in only a limited set of familiar scenarios. Minor changes in the scene, including variations in lighting conditions and background appearance, are acceptable but drastic alterations are not anticipated. To this end, we present a method to rapidly train and deploy a pipeline for estimating the continuous 6-DoF pose of an object from a single RGB image. The key idea is to leverage known camera poses and rigid body geometry to partially automate the generation of a large labeled dataset. The dataset, along with sufficient domain randomization, is then used to supervise the training of deep neural networks for predicting semantic keypoints. Experimentally, we demonstrate the convenience and effectiveness of our proposed method to accurately estimate object pose requiring only a very small amount of manual annotation for training. △ Less

Submitted 26 July, 2022; originally announced July 2022.

Comments: GitHub code: https://github.com/rohanpsingh/ObjectKeypointTrainer

Journal ref: 2020 IEEE/SICE International Symposium on System Integration (SII)

Learning Bipedal Walking On Planned Footsteps For Humanoid Robots

Authors: Rohan Pratap Singh, Mehdi Benallegue, Mitsuharu Morisawa, Rafael Cisneros, Fumio Kanehiro

Abstract: Deep reinforcement learning (RL) based controllers for legged robots have demonstrated impressive robustness for walking in different environments for several robot platforms. To enable the application of RL policies for humanoid robots in real-world settings, it is crucial to build a system that can achieve robust walking in any direction, on 2D and 3D terrains, and be controllable by a user-comm… ▽ More Deep reinforcement learning (RL) based controllers for legged robots have demonstrated impressive robustness for walking in different environments for several robot platforms. To enable the application of RL policies for humanoid robots in real-world settings, it is crucial to build a system that can achieve robust walking in any direction, on 2D and 3D terrains, and be controllable by a user-command. In this paper, we tackle this problem by learning a policy to follow a given step sequence. The policy is trained with the help of a set of procedurally generated step sequences (also called footstep plans). We show that simply feeding the upcoming 2 steps to the policy is sufficient to achieve omnidirectional walking, turning in place, standing, and climbing stairs. Our method employs curriculum learning on the complexity of terrains, and circumvents the need for reference motions or pre-trained weights. We demonstrate the application of our proposed method to learn RL policies for 2 new robot platforms - HRP5P and JVRC-1 - in the MuJoCo simulation environment. The code for training and evaluation is available online. △ Less

Submitted 31 October, 2022; v1 submitted 26 July, 2022; originally announced July 2022.

Comments: GitHub code: https://github.com/rohanpsingh/LearningHumanoidWalking

Rapid Pose Label Generation through Sparse Representation of Unknown Objects

Authors: Rohan Pratap Singh, Mehdi Benallegue, Yusuke Yoshiyasu, Fumio Kanehiro

Abstract: Deep Convolutional Neural Networks (CNNs) have been successfully deployed on robots for 6-DoF object pose estimation through visual perception. However, obtaining labeled data on a scale required for the supervised training of CNNs is a difficult task - exacerbated if the object is novel and a 3D model is unavailable. To this end, this work presents an approach for rapidly generating real-world, p… ▽ More Deep Convolutional Neural Networks (CNNs) have been successfully deployed on robots for 6-DoF object pose estimation through visual perception. However, obtaining labeled data on a scale required for the supervised training of CNNs is a difficult task - exacerbated if the object is novel and a 3D model is unavailable. To this end, this work presents an approach for rapidly generating real-world, pose-annotated RGB-D data for unknown objects. Our method not only circumvents the need for a prior 3D object model (textured or otherwise) but also bypasses complicated setups of fiducial markers, turntables, and sensors. With the help of a human user, we first source minimalistic labelings of an ordered set of arbitrarily chosen keypoints over a set of RGB-D videos. Then, by solving an optimization problem, we combine these labels under a world frame to recover a sparse, keypoint-based representation of the object. The sparse representation leads to the development of a dense model and the pose labels for each image frame in the set of scenes. We show that the sparse model can also be efficiently used for scaling to a large number of new scenes. We demonstrate the practicality of the generated labeled dataset by training a pipeline for 6-DoF object pose estimation and a pixel-wise segmentation network. △ Less

Submitted 7 November, 2020; originally announced November 2020.

Journal ref: 2021 IEEE International Conference on Robotics and Automation (ICRA)

Lyapunov-Stable Orientation Estimator for Humanoid Robots

Authors: Mehdi Benallegue, Rafael Cisneros, Abdelaziz Benallegue, Yacine Chitour, Mitsuharu Morisawa, Fumio Kanehiro

Abstract: In this paper, we present an observation scheme, with proven Lyapunov stability, for estimating a humanoid's floating base orientation. The idea is to use velocity aided attitude estimation, which requires to know the velocity of the system. This velocity can be obtained by taking into account the kinematic data provided by contact information with the environment and using the IMU and joint encod… ▽ More In this paper, we present an observation scheme, with proven Lyapunov stability, for estimating a humanoid's floating base orientation. The idea is to use velocity aided attitude estimation, which requires to know the velocity of the system. This velocity can be obtained by taking into account the kinematic data provided by contact information with the environment and using the IMU and joint encoders. We demonstrate how this operation can be used in the case of a fixed or a moving contact, allowing it to be employed for locomotion. We show how to use this velocity estimation within a selected two-stage state tilt estimator: (i) the first which has a global and quick convergence (ii) and the second which has smooth and robust dynamics. We provide new specific proofs of almost global Lyapunov asymptotic stability and local exponential convergence for this observer. Finally, we assess its performance by employing a comparative simulation and by using it within a closed-loop stabilization scheme for HRP-5P and HRP-2KAI robots performing whole-body kinematic tasks and locomotion. △ Less

Submitted 9 October, 2020; originally announced October 2020.

Journal ref: IEEE Robotics and Automation Letters, IEEE 2020, 5 (4), pp.6371-6378

Planning Grasps for Assembly Tasks

Authors: Weiwei Wan, Kensuke Harada, Fumio Kanehiro

Abstract: This paper develops model-based grasp planning algorithms for assembly tasks. It focuses on industrial end-effectors like grippers and suction cups, and plans grasp configurations considering CAD models of target objects. The developed algorithms are able to stably plan a large number of high-quality grasps, with high precision and little dependency on the quality of CAD models. The undergoing cor… ▽ More This paper develops model-based grasp planning algorithms for assembly tasks. It focuses on industrial end-effectors like grippers and suction cups, and plans grasp configurations considering CAD models of target objects. The developed algorithms are able to stably plan a large number of high-quality grasps, with high precision and little dependency on the quality of CAD models. The undergoing core technique is superimposed segmentation, which pre-processes a mesh model by peeling it into facets. The algorithms use superimposed segments to locate contact points and parallel facets, and synthesize grasp poses for popular industrial end-effectors. Several tunable parameters were prepared to adapt the algorithms to meet various requirements. The experimental section demonstrates the advantages of the algorithms by analyzing the cost and stability of the algorithms, the precision of the planned grasps, and the tunable parameters with both simulations and real-world experiments. Also, some examples of robotic assembly systems using the proposed algorithms are presented to demonstrate the efficacy. △ Less

Submitted 4 March, 2019; originally announced March 2019.

Preparatory Manipulation Planning using Automatically Determined Single and Dual Arms

Authors: Weiwei Wan, Kensuke Harada, Fumio Kanehiro

Abstract: This paper presents a manipulation planning algorithm for robots to reorient objects. It automatically finds a sequence of robot motion that manipulates and prepares an object for specific tasks. Examples of the preparatory manipulation planning problems include reorienting an electric drill to cut holes, reorienting workpieces for assembly, and reorienting cargo for packing, etc. The proposed alg… ▽ More This paper presents a manipulation planning algorithm for robots to reorient objects. It automatically finds a sequence of robot motion that manipulates and prepares an object for specific tasks. Examples of the preparatory manipulation planning problems include reorienting an electric drill to cut holes, reorienting workpieces for assembly, and reorienting cargo for packing, etc. The proposed algorithm could plan single and dual arm manipulation sequences to solve the problems. The mechanism under the planner is a regrasp graph which encodes grasp configurations and object poses. The algorithms search the graph to find a sequence of robot motion to reorient objects. The planner is able to plan both single and dual arm manipulation. It could also automatically determine whether to use a single arm, dual arms, or their combinations to finish given tasks. The planner is examined by various humanoid robots like Nextage, HRP2Kai, HRP5P, etc., using both simulation and real-world experiments. △ Less

Submitted 8 December, 2018; originally announced December 2018.

Regrasp Planning Considering Bipedal Stability Constraints

Authors: Daniel Sanchez, Weiwei Wan, Kensuke Harada, Fumio Kanehiro

Abstract: This paper presents a Center of Mass (CoM) based manipulation and regrasp planner that implements stability constraints to preserve the robot balance. The planner provides a graph of IK-feasible, collision-free and stable motion sequences, constructed using an energy based motion planning algorithm. It assures that the assembly motions are stable and prevent the robot from falling while performing… ▽ More This paper presents a Center of Mass (CoM) based manipulation and regrasp planner that implements stability constraints to preserve the robot balance. The planner provides a graph of IK-feasible, collision-free and stable motion sequences, constructed using an energy based motion planning algorithm. It assures that the assembly motions are stable and prevent the robot from falling while performing dexterous tasks in different situations. Furthermore, the constraints are also used to perform an RRT-inspired task-related stability estimation in several simulations. The estimation can be used to select between single-arm and dual-arm regras** configurations to achieve more stability and robustness for a given manipulation task. To validate the planner and the task-related stability estimations, several tests are performed in simulations and real-world experiments involving the HRP5P humanoid robot, the 5th generation of the HRP robot family. The experiment results suggest that the planner and the task-related stability estimation provide robust behavior for the humanoid robot while performing regrasp tasks. △ Less

Submitted 14 October, 2018; originally announced October 2018.