-
Interactive Identification of Granular Materials using Force Measurements
Authors:
Samuli Hynninen,
Tran Nguyen Le,
Ville Kyrki
Abstract:
The ability to identify granular materials facilitates the emergence of various new applications in robotics, ranging from cooking at home to truck loading at mining sites. However, granular material identification remains a challenging and underexplored area. In this work, we present a novel interactive material identification framework that enables robots to identify a wide range of granular mat…
▽ More
The ability to identify granular materials facilitates the emergence of various new applications in robotics, ranging from cooking at home to truck loading at mining sites. However, granular material identification remains a challenging and underexplored area. In this work, we present a novel interactive material identification framework that enables robots to identify a wide range of granular materials using only a force-torque sensor for perception. Our framework, comprising interactive exploration, feature extraction, and classification stages, prioritizes simplicity and transparency for seamless integration into various manipulation pipelines. We evaluate the proposed approach through extensive experiments with a real-world dataset comprising 11 granular materials, which we also make publicly available. Additionally, we conducted a comprehensive qualitative analysis of the dataset to offer deeper insights into its nature, aiding future development. Our results show that the proposed method is capable of accurately identifying a wide range of granular materials solely relying on force measurements obtained from direct interaction with the materials. Code and dataset are available at: https://irobotics.aalto.fi/indentify_granular/.
△ Less
Submitted 26 March, 2024;
originally announced March 2024.
-
Dynamic Manipulation of Deformable Objects using Imitation Learning with Adaptation to Hardware Constraints
Authors:
Eric Hannus,
Tran Nguyen Le,
David Blanco-Mulero,
Ville Kyrki
Abstract:
Imitation Learning (IL) is a promising paradigm for learning dynamic manipulation of deformable objects since it does not depend on difficult-to-create accurate simulations of such objects. However, the translation of motions demonstrated by a human to a robot is a challenge for IL, due to differences in the embodiments and the robot's physical limits. These limits are especially relevant in dynam…
▽ More
Imitation Learning (IL) is a promising paradigm for learning dynamic manipulation of deformable objects since it does not depend on difficult-to-create accurate simulations of such objects. However, the translation of motions demonstrated by a human to a robot is a challenge for IL, due to differences in the embodiments and the robot's physical limits. These limits are especially relevant in dynamic manipulation where high velocities and accelerations are typical. To address this problem, we propose a framework that first maps a dynamic demonstration into a motion that respects the robot's constraints using a constrained Dynamic Movement Primitive. Second, the resulting object state is further optimized by quasi-static refinement motions to optimize task performance metrics. This allows both efficiently altering the object state by dynamic motions and stable small-scale refinements. We evaluate the framework in the challenging task of bag opening, designing the system BILBO: Bimanual dynamic manipulation using Imitation Learning for Bag Opening. Our results show that BILBO can successfully open a wide range of crumpled bags, using a demonstration with a single bag. See supplementary material at https://sites.google.com/view/bilbo-bag.
△ Less
Submitted 19 March, 2024;
originally announced March 2024.
-
Online Learning of Human Constraints from Feedback in Shared Autonomy
Authors:
Shibei Zhu,
Tran Nguyen Le,
Samuel Kaski,
Ville Kyrki
Abstract:
Real-time collaboration with humans poses challenges due to the different behavior patterns of humans resulting from diverse physical constraints. Existing works typically focus on learning safety constraints for collaboration, or how to divide and distribute the subtasks between the participating agents to carry out the main task. In contrast, we propose to learn a human constraints model that, i…
▽ More
Real-time collaboration with humans poses challenges due to the different behavior patterns of humans resulting from diverse physical constraints. Existing works typically focus on learning safety constraints for collaboration, or how to divide and distribute the subtasks between the participating agents to carry out the main task. In contrast, we propose to learn a human constraints model that, in addition, considers the diverse behaviors of different human operators. We consider a type of collaboration in a shared-autonomy fashion, where both a human operator and an assistive robot act simultaneously in the same task space that affects each other's actions. The task of the assistive agent is to augment the skill of humans to perform a shared task by supporting humans as much as possible, both in terms of reducing the workload and minimizing the discomfort for the human operator. Therefore, we propose an augmentative assistant agent capable of learning and adapting to human physical constraints, aligning its actions with the ergonomic preferences and limitations of the human operator.
△ Less
Submitted 5 March, 2024;
originally announced March 2024.
-
SPONGE: Sequence Planning with Deformable-ON-Rigid Contact Prediction from Geometric Features
Authors:
Tran Nguyen Le,
Fares J. Abu-Dakka,
Ville Kyrki
Abstract:
Planning robotic manipulation tasks, especially those that involve interaction between deformable and rigid objects, is challenging due to the complexity in predicting such interactions. We introduce SPONGE, a sequence planning pipeline powered by a deep learning-based contact prediction model for contacts between deformable and rigid bodies under interactions. The contact prediction model is trai…
▽ More
Planning robotic manipulation tasks, especially those that involve interaction between deformable and rigid objects, is challenging due to the complexity in predicting such interactions. We introduce SPONGE, a sequence planning pipeline powered by a deep learning-based contact prediction model for contacts between deformable and rigid bodies under interactions. The contact prediction model is trained on synthetic data generated by a developed simulation environment to learn the map** from point-cloud observation of a rigid target object and the pose of a deformable tool, to 3D representation of the contact points between the two bodies. We experimentally evaluated the proposed approach for a dish cleaning task both in simulation and on a real \panda with real-world objects. The experimental results demonstrate that in both scenarios the proposed planning pipeline is capable of generating high-quality trajectories that can accomplish the task by achieving more than 90\% area coverage on different objects of varying sizes and curvatures while minimizing travel distance. Code and video are available at: \url{https://irobotics.aalto.fi/sponge/}.
△ Less
Submitted 24 March, 2023;
originally announced March 2023.
-
Constrained Generative Sampling of 6-DoF Grasps
Authors:
Jens Lundell,
Francesco Verdoja,
Tran Nguyen Le,
Arsalan Mousavian,
Dieter Fox,
Ville Kyrki
Abstract:
Most state-of-the-art data-driven grasp sampling methods propose stable and collision-free grasps uniformly on the target object. For bin-picking, executing any of those reachable grasps is sufficient. However, for completing specific tasks, such as squeezing out liquid from a bottle, we want the grasp to be on a specific part of the object's body while avoiding other locations, such as the cap. T…
▽ More
Most state-of-the-art data-driven grasp sampling methods propose stable and collision-free grasps uniformly on the target object. For bin-picking, executing any of those reachable grasps is sufficient. However, for completing specific tasks, such as squeezing out liquid from a bottle, we want the grasp to be on a specific part of the object's body while avoiding other locations, such as the cap. This work presents a generative grasp sampling network, VCGS, capable of constrained 6 Degrees of Freedom (DoF) grasp sampling. In addition, we also curate a new dataset designed to train and evaluate methods for constrained gras**. The new dataset, called CONG, consists of over 14 million training samples of synthetically rendered point clouds and grasps at random target areas on 2889 objects. VCGS is benchmarked against GraspNet, a state-of-the-art unconstrained grasp sampler, in simulation and on a real robot. The results demonstrate that VCGS achieves a 10-15% higher grasp success rate than the baseline while being 2-3 times as sample efficient. Supplementary material is available on our project website.
△ Less
Submitted 17 August, 2023; v1 submitted 21 February, 2023;
originally announced February 2023.
-
A Novel Simulation-Based Quality Metric for Evaluating Grasps on 3D Deformable Objects
Authors:
Tran Nguyen Le,
Jens Lundell,
Fares J. Abu-Dakka,
Ville Kyrki
Abstract:
Evaluation of grasps on deformable 3D objects is a little-studied problem, even if the applicability of rigid object grasp quality measures for deformable ones is an open question. A central issue with most quality measures is their dependence on contact points which for deformable objects depend on the deformations. This paper proposes a grasp quality measure for deformable objects that uses info…
▽ More
Evaluation of grasps on deformable 3D objects is a little-studied problem, even if the applicability of rigid object grasp quality measures for deformable ones is an open question. A central issue with most quality measures is their dependence on contact points which for deformable objects depend on the deformations. This paper proposes a grasp quality measure for deformable objects that uses information about object deformation to calculate the grasp quality. Grasps are evaluated by simulating the deformations during gras** and predicting the contacts between the gripper and the grasped object. The contact information is then used as input for a new grasp quality metric to quantify the grasp quality. The approach is benchmarked against two classical rigid-body quality metrics on over 600 grasps in the Isaac gym simulation and over 50 real-world grasps. Experimental results show an average improvement of 18\% in the grasp success rate for deformable objects compared to the classical rigid-body quality metrics.
△ Less
Submitted 23 March, 2022;
originally announced March 2022.
-
Deformation-Aware Data-Driven Grasp Synthesis
Authors:
Tran Nguyen Le,
Jens Lundell,
Fares J. Abu-Dakka,
Ville Kyrki
Abstract:
Grasp synthesis for 3D deformable objects remains a little-explored topic, most works aiming to minimize deformations. However, deformations are not necessarily harmful -- humans are, for example, able to exploit deformations to generate new potential grasps. How to achieve that on a robot is though an open question. This paper proposes an approach that uses object stiffness information in additio…
▽ More
Grasp synthesis for 3D deformable objects remains a little-explored topic, most works aiming to minimize deformations. However, deformations are not necessarily harmful -- humans are, for example, able to exploit deformations to generate new potential grasps. How to achieve that on a robot is though an open question. This paper proposes an approach that uses object stiffness information in addition to depth images for synthesizing high-quality grasps. We achieve this by incorporating object stiffness as an additional input to a state-of-the-art deep grasp planning network. We also curate a new synthetic dataset of grasps on objects of varying stiffness using the Isaac Gym simulator for training the network. We experimentally validate and compare our proposed approach against the case where we do not incorporate object stiffness on a total of 2800 grasps in simulation and 420 grasps on a real Franka Emika Panda. The experimental results show significant improvement in grasp success rate using the proposed approach on a wide range of objects with varying shapes, sizes, and stiffness. Furthermore, we demonstrate that the approach can generate different gras** strategies for different stiffness values, such as pinching for soft objects and caging for hard objects. Together, the results clearly show the value of incorporating stiffness information when gras** objects of varying stiffness.
△ Less
Submitted 11 September, 2021;
originally announced September 2021.
-
Towards synthesizing grasps for 3D deformable objects with physics-based simulation
Authors:
Tran Nguyen Le,
Jens Lundell,
Fares J. Abu-Dakka,
Ville Kyrki
Abstract:
Gras** deformable objects is not well researched due to the complexity in modelling and simulating the dynamic behavior of such objects. However, with the rapid development of physics-based simulators that support soft bodies, the research gap between rigid and deformable objects is getting smaller. To leverage the capability of such simulators and to challenge the assumption that has guided rob…
▽ More
Gras** deformable objects is not well researched due to the complexity in modelling and simulating the dynamic behavior of such objects. However, with the rapid development of physics-based simulators that support soft bodies, the research gap between rigid and deformable objects is getting smaller. To leverage the capability of such simulators and to challenge the assumption that has guided robotic gras** research so far, i.e., object rigidity, we proposed a deep-learning based approach that generates stiffness-dependent grasps. Our network is trained on purely synthetic data generated from a physics-based simulator. The same simulator is also used to evaluate the trained network. The results show improvement in terms of grasp ranking and grasp success rate. Furthermore, our network can adapt the grasps based on the stiffness. We are currently validating the proposed approach on a larger test dataset in simulation and on a physical robot.
△ Less
Submitted 19 July, 2021;
originally announced July 2021.
-
Multi-FinGAN: Generative Coarse-To-Fine Sampling of Multi-Finger Grasps
Authors:
Jens Lundell,
Enric Corona,
Tran Nguyen Le,
Francesco Verdoja,
Philippe Weinzaepfel,
Gregory Rogez,
Francesc Moreno-Noguer,
Ville Kyrki
Abstract:
While there exists many methods for manipulating rigid objects with parallel-jaw grippers, gras** with multi-finger robotic hands remains a quite unexplored research topic. Reasoning and planning collision-free trajectories on the additional degrees of freedom of several fingers represents an important challenge that, so far, involves computationally costly and slow processes. In this work, we p…
▽ More
While there exists many methods for manipulating rigid objects with parallel-jaw grippers, gras** with multi-finger robotic hands remains a quite unexplored research topic. Reasoning and planning collision-free trajectories on the additional degrees of freedom of several fingers represents an important challenge that, so far, involves computationally costly and slow processes. In this work, we present Multi-FinGAN, a fast generative multi-finger grasp sampling method that synthesizes high quality grasps directly from RGB-D images in about a second. We achieve this by training in an end-to-end fashion a coarse-to-fine model composed of a classification network that distinguishes grasp types according to a specific taxonomy and a refinement network that produces refined grasp poses and joint angles. We experimentally validate and benchmark our method against a standard grasp-sampling method on 790 grasps in simulation and 20 grasps on a real Franka Emika Panda. All experimental results using our method show consistent improvements both in terms of grasp quality metrics and grasp success rate. Remarkably, our approach is up to 20-30 times faster than the baseline, a significant improvement that opens the door to feedback-based grasp re-planning and task informative gras**. Code is available at https://irobotics.aalto.fi/multi-fingan/.
△ Less
Submitted 15 March, 2021; v1 submitted 17 December, 2020;
originally announced December 2020.
-
Probabilistic Surface Friction Estimation Based on Visual and Haptic Measurements
Authors:
Tran Nguyen Le,
Francesco Verdoja,
Fares J. Abu-Dakka,
Ville Kyrki
Abstract:
Accurately modeling local surface properties of objects is crucial to many robotic applications, from gras** to material recognition. Surface properties like friction are however difficult to estimate, as visual observation of the object does not convey enough information over these properties. In contrast, haptic exploration is time consuming as it only provides information relevant to the expl…
▽ More
Accurately modeling local surface properties of objects is crucial to many robotic applications, from gras** to material recognition. Surface properties like friction are however difficult to estimate, as visual observation of the object does not convey enough information over these properties. In contrast, haptic exploration is time consuming as it only provides information relevant to the explored parts of the object. In this work, we propose a joint visuo-haptic object model that enables the estimation of surface friction coefficient over an entire object by exploiting the correlation of visual and haptic information, together with a limited haptic exploration by a robotic arm. We demonstrate the validity of the proposed method by showing its ability to estimate varying friction coefficients on a range of real multi-material objects. Furthermore, we illustrate how the estimated friction coefficients can improve gras** success rate by guiding a grasp planner toward high friction areas.
△ Less
Submitted 12 March, 2021; v1 submitted 16 October, 2020;
originally announced October 2020.
-
A Novel Design of Soft Robotic Hand with a Human-inspired Soft Palm for Dexterous Gras**
Authors:
Haihang Wang,
Fares J. Abu-Dakka,
Tran Nguyen Le,
Ville Kyrki,
He Xu
Abstract:
Soft robotic hands and grippers are increasingly attracting attention as a robotic end-effector. Compared with rigid counterparts, they are safer for human-robot and environment-robot interactions, easier to control, lower cost and weight, and more compliant. Current soft robotic hands have mostly focused on the soft fingers and bending actuators. However, the palm is also essential part for grasp…
▽ More
Soft robotic hands and grippers are increasingly attracting attention as a robotic end-effector. Compared with rigid counterparts, they are safer for human-robot and environment-robot interactions, easier to control, lower cost and weight, and more compliant. Current soft robotic hands have mostly focused on the soft fingers and bending actuators. However, the palm is also essential part for gras**. In this work, we propose a novel design of soft humanoid hand with pneumatic soft fingers and soft palm. The hand is inexpensive to fabricate. The configuration of the soft palm is based on modular design which can be easily applied into actuating all kinds of soft fingers before. The splaying of the fingers, bending of the whole palm, abduction and adduction of the thumb are implemented by the soft palm. Moreover, we present a new design of soft finger, called hybrid bending soft finger (HBSF). It can both bend in the gras** axis and deflect in the side-to-side axis as human-like motion. The functions of the HBSF and soft palm were simulated by SOFA framework. And their performance was tested in experiments. The 6 fingers with 1 to 11 segments were tested and analyzed. The versatility of the soft hand is evaluated and testified by the gras** experiments in real scenario according to Feix taxonomy. And the results present the diversity of grasps and show promise for gras** a variety of objects with different shapes and weights.
△ Less
Submitted 2 September, 2020;
originally announced September 2020.
-
Flex: Closing the Gaps between Usage and Allocation
Authors:
Tan N. Le,
Zhenhua Liu
Abstract:
Data centers are giant factories of Internet data and services. Worldwide data centers consume energy and emit emissions more than airline industry. Unfortunately, most of data centers are significantly underutilized. One of the major reasons is the big gaps between the real usage and the provisioned resources because users tend to over-estimate their demand and data center operators often rely on…
▽ More
Data centers are giant factories of Internet data and services. Worldwide data centers consume energy and emit emissions more than airline industry. Unfortunately, most of data centers are significantly underutilized. One of the major reasons is the big gaps between the real usage and the provisioned resources because users tend to over-estimate their demand and data center operators often rely on users' requests for resource allocation. In this paper, we first conduct an in-depth analysis of a Google cluster trace to unveil the root causes for low utilization and highlight the great potential to improve it. We then developed an online resource manager Flex to maximize the cluster utilization while satisfying the Quality of Service (QoS). Large-scale evaluations based on real-world traces show that Flex admits up to 1.74x more requests and 1.6x higher utilization compared to tradition schedulers while maintaining the QoS.
△ Less
Submitted 1 June, 2020;
originally announced June 2020.
-
BoPF: Mitigating the Burstiness-Fairness Tradeoff in Multi-Resource Clusters
Authors:
Tan N. Le,
Xiao Sun,
Mosharaf Chowdhury,
Zhenhua Liu
Abstract:
Simultaneously supporting latency- and throughout-sensitive workloads in a shared environment is an increasingly more common challenge in big data clusters. Despite many advances, existing cluster schedulers force the same performance goal - fairness in most cases - on all jobs. Latency-sensitive jobs suffer, while throughput-sensitive ones thrive. Using prioritization does the opposite: it opens…
▽ More
Simultaneously supporting latency- and throughout-sensitive workloads in a shared environment is an increasingly more common challenge in big data clusters. Despite many advances, existing cluster schedulers force the same performance goal - fairness in most cases - on all jobs. Latency-sensitive jobs suffer, while throughput-sensitive ones thrive. Using prioritization does the opposite: it opens up a path for latency-sensitive jobs to dominate. In this paper, we tackle the challenges in supporting both short-term performance and long-term fairness simultaneously with high resource utilization by proposing Bounded Priority Fairness (BoPF). BoPF provides short-term resource guarantees to latency-sensitive jobs and maintains long-term fairness for throughput-sensitive jobs. BoPF is the first scheduler that can provide long-term fairness, burst guarantee, and Pareto efficiency in a strategyproof manner for multi-resource scheduling. Deployments and large-scale simulations show that BoPF closely approximates the performance of Strict Priority as well as the fairness characteristics of DRF. In deployments, BoPF speeds up latency-sensitive jobs by 5.38 times compared to DRF, while still maintaining long-term fairness. In the meantime, BoPF improves the average completion times of throughput-sensitive jobs by up to 3.05 times compared to Strict Priority.
△ Less
Submitted 7 December, 2019;
originally announced December 2019.
-
Safe Gras** with a Force Controlled Soft Robotic Hand
Authors:
Tran Nguyen Le,
Jens Lundell,
Ville Kyrki
Abstract:
Safe yet stable gras** requires a robotic hand to apply sufficient force on the object to immobilize it while kee** it from getting damaged. Soft robotic hands have been proposed for safe gras** due to their passive compliance, but even such a hand can crush objects if the applied force is too high. Thus for safe gras**, regulating the gras** force is of uttermost importance even with so…
▽ More
Safe yet stable gras** requires a robotic hand to apply sufficient force on the object to immobilize it while kee** it from getting damaged. Soft robotic hands have been proposed for safe gras** due to their passive compliance, but even such a hand can crush objects if the applied force is too high. Thus for safe gras**, regulating the gras** force is of uttermost importance even with soft hands. In this work, we present a force controlled soft hand and use it to achieve safe gras**. To this end, resistive force and bend sensors are integrated in a soft hand, and a data-driven calibration method is proposed to estimate contact interaction forces. Given the force readings, the pneumatic pressures are regulated using a proportional-integral controller to achieve desired force. The controller is experimentally evaluated and benchmarked by gras** easily deformable objects such as plastic and paper cups without neither drop** nor deforming them. Together, the results demonstrate that our force controlled soft hand can grasp deformable objects in a safe yet stable manner.
△ Less
Submitted 31 August, 2020; v1 submitted 15 September, 2019;
originally announced September 2019.
