-
Unfolding the Literature: A Review of Robotic Cloth Manipulation
Authors:
Alberta Longhini,
Yufei Wang,
Irene Garcia-Camacho,
David Blanco-Mulero,
Marco Moletta,
Michael Welle,
Guillem Alenyà,
Hang Yin,
Zackory Erickson,
David Held,
Júlia Borràs,
Danica Kragic
Abstract:
The realm of textiles spans clothing, households, healthcare, sports, and industrial applications. The deformable nature of these objects poses unique challenges that prior work on rigid objects cannot fully address. The increasing interest within the community in textile perception and manipulation has led to new methods that aim to address challenges in modeling, perception, and control, resulti…
▽ More
The realm of textiles spans clothing, households, healthcare, sports, and industrial applications. The deformable nature of these objects poses unique challenges that prior work on rigid objects cannot fully address. The increasing interest within the community in textile perception and manipulation has led to new methods that aim to address challenges in modeling, perception, and control, resulting in significant progress. However, this progress is often tailored to one specific textile or a subcategory of these textiles. To understand what restricts these methods and hinders current approaches from generalizing to a broader range of real-world textiles, this review provides an overview of the field, focusing specifically on how and to what extent textile variations are addressed in modeling, perception, benchmarking, and manipulation of textiles. We finally conclude by identifying key open problems and outlining grand challenges that will drive future advancements in the field.
△ Less
Submitted 1 July, 2024;
originally announced July 2024.
-
Standardization of Cloth Objects and its Relevance in Robotic Manipulation
Authors:
Irene Garcia-Camacho,
Alberta Longhini,
Michael Welle,
Guillem Alenyà,
Danica Kragic,
Júlia Borràs
Abstract:
The field of robotics faces inherent challenges in manipulating deformable objects, particularly in understanding and standardising fabric properties like elasticity, stiffness, and friction. While the significance of these properties is evident in the realm of cloth manipulation, accurately categorising and comprehending them in real-world applications remains elusive. This study sets out to addr…
▽ More
The field of robotics faces inherent challenges in manipulating deformable objects, particularly in understanding and standardising fabric properties like elasticity, stiffness, and friction. While the significance of these properties is evident in the realm of cloth manipulation, accurately categorising and comprehending them in real-world applications remains elusive. This study sets out to address two primary objectives: (1) to provide a framework suitable for robotics applications to characterise cloth objects, and (2) to study how these properties influence robotic manipulation tasks. Our preliminary results validate the framework's ability to characterise cloth properties and compare cloth sets, and reveal the influence that different properties have on the outcome of five manipulation primitives. We believe that, in general, results on the manipulation of clothes should be reported along with a better description of the garments used in the evaluation. This paper proposes a set of these measures.
△ Less
Submitted 7 March, 2024;
originally announced March 2024.
-
Towards Transferring Tactile-based Continuous Force Control Policies from Simulation to Robot
Authors:
Luca Lach,
Robert Haschke,
Davide Tateo,
Jan Peters,
Helge Ritter,
Júlia Borràs,
Carme Torras
Abstract:
The advent of tactile sensors in robotics has sparked many ideas on how robots can leverage direct contact measurements of their environment interactions to improve manipulation tasks. An important line of research in this regard is that of grasp force control, which aims to manipulate objects safely by limiting the amount of force exerted on the object. While prior works have either hand-modeled…
▽ More
The advent of tactile sensors in robotics has sparked many ideas on how robots can leverage direct contact measurements of their environment interactions to improve manipulation tasks. An important line of research in this regard is that of grasp force control, which aims to manipulate objects safely by limiting the amount of force exerted on the object. While prior works have either hand-modeled their force controllers, employed model-based approaches, or have not shown sim-to-real transfer, we propose a model-free deep reinforcement learning approach trained in simulation and then transferred to the robot without further fine-tuning. We therefore present a simulation environment that produces realistic normal forces, which we use to train continuous force control policies. An evaluation in which we compare against a baseline and perform an ablation study shows that our approach outperforms the hand-modeled baseline and that our proposed inductive bias and domain randomization facilitate sim-to-real transfer. Code, models, and supplementary videos are available on https://sites.google.com/view/rl-force-ctrl
△ Less
Submitted 13 November, 2023;
originally announced November 2023.
-
A quantum sensing metrology for magnetic memories
Authors:
Vicent J Borràs,
Robert Carpenter,
Liza Žaper,
Siddharth Rao,
Sébastien Couet,
Mathieu Munsch,
Patrick Maletinsky,
Peter Rickhaus
Abstract:
Magnetic random access memory (MRAM) is a leading emergent memory technology that is poised to replace current non-volatile memory technologies such as eFlash. However, the scaling of MRAM technologies is heavily affected by device-to-device variability rooted in the stochastic nature of the MRAM writing process into nanoscale magnetic layers. Here, we introduce a non-contact metrology technique d…
▽ More
Magnetic random access memory (MRAM) is a leading emergent memory technology that is poised to replace current non-volatile memory technologies such as eFlash. However, the scaling of MRAM technologies is heavily affected by device-to-device variability rooted in the stochastic nature of the MRAM writing process into nanoscale magnetic layers. Here, we introduce a non-contact metrology technique deploying Scanning NV Magnetometry (SNVM) to investigate MRAM performance at the individual bit level. We demonstrate magnetic reversal characterization in individual, < 60 nm sized bits, to extract key magnetic properties, thermal stability, and switching statistics, and thereby gauge bit-to-bit uniformity. We showcase the performance of our method by benchmarking two distinct bit etching processes immediately after pattern formation. Unlike previous methods, our approach unveils marked differences in switching behaviour of fully contacted MRAM devices stemming from these processes. Our findings highlight the potential of nanoscale quantum sensing of MRAM devices for early-stage screening in the processing line, paving the way for future incorporation of this nanoscale characterization tool in the semiconductor industry.
△ Less
Submitted 27 June, 2023;
originally announced June 2023.
-
The dGLI Cloth Coordinates: A Topological Representation for Semantic Classification of Cloth States
Authors:
Franco Coltraro,
Josep Fontana,
Jaume Amorós,
Maria Alberich-Carramiñana,
Júlia Borràs,
Carme Torras
Abstract:
Robotic manipulation of cloth is a highly complex task because of its infinite-dimensional shape-state space that makes cloth state estimation very difficult. In this paper we introduce the dGLI Cloth Coordinates, a low-dimensional representation of the state of a rectangular piece of cloth that allows to efficiently distinguish key topological changes in a folding sequence, opening the door to ef…
▽ More
Robotic manipulation of cloth is a highly complex task because of its infinite-dimensional shape-state space that makes cloth state estimation very difficult. In this paper we introduce the dGLI Cloth Coordinates, a low-dimensional representation of the state of a rectangular piece of cloth that allows to efficiently distinguish key topological changes in a folding sequence, opening the door to efficient learning methods for cloth manipulation planning and control. Our representation is based on a directional derivative of the Gauss Linking Integral and allows us to represent both planar and spatial configurations in a consistent unified way. The proposed dGLI Cloth Coordinates are shown to be more accurate in the classification of cloth states and significantly more sensitive to changes in gras** affordances than other classic shape distance methods. Finally, we apply our representation to real images of a cloth, showing we can identify the different states using a simple distance-based classifier.
△ Less
Submitted 14 September, 2022;
originally announced September 2022.
-
Semantic State Estimation in Cloth Manipulation Tasks
Authors:
Georgies Tzelepis,
Eren Erdal Aksoy,
Júlia Borràs,
Guillem Alenyà
Abstract:
Understanding of deformable object manipulations such as textiles is a challenge due to the complexity and high dimensionality of the problem. Particularly, the lack of a generic representation of semantic states (e.g., \textit{crumpled}, \textit{diagonally folded}) during a continuous manipulation process introduces an obstacle to identify the manipulation type. In this paper, we aim to solve the…
▽ More
Understanding of deformable object manipulations such as textiles is a challenge due to the complexity and high dimensionality of the problem. Particularly, the lack of a generic representation of semantic states (e.g., \textit{crumpled}, \textit{diagonally folded}) during a continuous manipulation process introduces an obstacle to identify the manipulation type. In this paper, we aim to solve the problem of semantic state estimation in cloth manipulation tasks. For this purpose, we introduce a new large-scale fully-annotated RGB image dataset showing various human demonstrations of different complicated cloth manipulations. We provide a set of baseline deep networks and benchmark them on the problem of semantic state estimation using our proposed dataset. Furthermore, we investigate the scalability of our semantic state estimation framework in robot monitoring tasks of long and complex cloth manipulations.
△ Less
Submitted 22 March, 2022;
originally announced March 2022.
-
Household Cloth Object Set: Fostering Benchmarking in Deformable Object Manipulation
Authors:
Irene Garcia-Camacho,
Júlia Borràs,
Berk Calli,
Adam Norton,
Guillem Alenyà
Abstract:
Benchmarking of robotic manipulations is one of the open issues in robotic research. An important factor that has enabled progress in this area in the last decade is the existence of common object sets that have been shared among different research groups. However, the existing object sets are very limited when it comes to cloth-like objects that have unique particularities and challenges. This pa…
▽ More
Benchmarking of robotic manipulations is one of the open issues in robotic research. An important factor that has enabled progress in this area in the last decade is the existence of common object sets that have been shared among different research groups. However, the existing object sets are very limited when it comes to cloth-like objects that have unique particularities and challenges. This paper is a first step towards the design of a cloth object set to be distributed among research groups from the robotics cloth manipulation community. We present a set of household cloth objects and related tasks that serve to expose the challenges related to gathering such an object set and propose a roadmap to the design of common benchmarks in cloth manipulation tasks, with the intention to set the grounds for a future debate in the community that will be necessary to foster benchmarking for the manipulation of cloth-like objects. Some RGB-D and object scans are also collected as examples for the objects in relevant configurations. More details about the cloth set are shared in http://www.iri.upc.edu/groups/perception/ClothObjectSet/HouseholdClothSet.html.
△ Less
Submitted 2 November, 2021;
originally announced November 2021.
-
Task-Adaptive Robot Learning from Demonstration with Gaussian Process Models under Replication
Authors:
Miguel Arduengo,
Adrià Colomé,
Júlia Borràs,
Luis Sentis,
Carme Torras
Abstract:
Learning from Demonstration (LfD) is a paradigm that allows robots to learn complex manipulation tasks that can not be easily scripted, but can be demonstrated by a human teacher. One of the challenges of LfD is to enable robots to acquire skills that can be adapted to different scenarios. In this paper, we propose to achieve this by exploiting the variations in the demonstrations to retrieve an a…
▽ More
Learning from Demonstration (LfD) is a paradigm that allows robots to learn complex manipulation tasks that can not be easily scripted, but can be demonstrated by a human teacher. One of the challenges of LfD is to enable robots to acquire skills that can be adapted to different scenarios. In this paper, we propose to achieve this by exploiting the variations in the demonstrations to retrieve an adaptive and robust policy, using Gaussian Process (GP) models. Adaptability is enhanced by incorporating task parameters into the model, which encode different specifications within the same task. With our formulation, these parameters can be either real, integer, or categorical. Furthermore, we propose a GP design that exploits the structure of replications, i.e., repeated demonstrations with identical conditions within data. Our method significantly reduces the computational cost of model fitting in complex tasks, where replications are essential to obtain a robust model. We illustrate our approach through several experiments on a handwritten letter demonstration dataset.
△ Less
Submitted 5 February, 2021; v1 submitted 15 October, 2020;
originally announced October 2020.
-
Encoding cloth manipulations using a graph of states and transitions
Authors:
Júlia Borràs,
Guillem Alenyà,
Carme Torras
Abstract:
Cloth manipulation is very relevant for domestic robotic tasks, but it presents many challenges due to the complexity of representing, recognizing and predicting the behaviour of cloth under manipulation. In this work, we propose a generic, compact and simplified representation of the states of cloth manipulation that allows for representing tasks as sequences of states and transitions. We also de…
▽ More
Cloth manipulation is very relevant for domestic robotic tasks, but it presents many challenges due to the complexity of representing, recognizing and predicting the behaviour of cloth under manipulation. In this work, we propose a generic, compact and simplified representation of the states of cloth manipulation that allows for representing tasks as sequences of states and transitions. We also define a Cloth Manipulation Graph that encodes all the strategies to accomplish a task. Our novel representation is used to encode two different cloth manipulation tasks, learned from an experiment with human subjects with video and motion data. We show how our simplified representation allows to obtain a map of meaningful motion primitives.
△ Less
Submitted 3 March, 2022; v1 submitted 30 September, 2020;
originally announced September 2020.
-
Context-Aware Decentralized Invariant Signaling for Opportunistic Communications
Authors:
Jordi Borras,
Gregori Vazquez
Abstract:
A novel scenario-adapted distributed signaling technique in the context of opportunistic communications is presented in this work. Each opportunistic user acquires locally sampled observations from the wireless environment to determine the occupied and available degrees-of-freedom (DoF). Due to sensing errors and locality of observations, a performance loss and inter-system interference arise from…
▽ More
A novel scenario-adapted distributed signaling technique in the context of opportunistic communications is presented in this work. Each opportunistic user acquires locally sampled observations from the wireless environment to determine the occupied and available degrees-of-freedom (DoF). Due to sensing errors and locality of observations, a performance loss and inter-system interference arise from subspace uncertainties. Yet, we show that addressing the problem as a total least-squares (TLS) optimization, signaling patterns robust to subspace uncertainties can be designed. Furthermore, given the equivalence of minimum norm and TLS, the latter exhibits the interesting properties of linear predictors. Specifically, the rotationally invariance property is of paramount importance to guarantee the detectability by neighboring nodes. Albeit these advantages, end-to-end subspace uncertainties yield a performance loss that compromises both detectability and wireless environment's performance. To combat the latter, we tackle the distributed identification of the active subspace with and without side information about neighboring nodes' subspaces. An extensive simulation analysis highlights the performance of distributed concurrency schemes to achieve subspace agreement.
△ Less
Submitted 29 October, 2021; v1 submitted 25 September, 2019;
originally announced September 2019.
-
A Gras**-centered Analysis for Cloth Manipulation
Authors:
Júlia Borràs,
Guillem Alenya,
Carme Torras
Abstract:
Compliant and soft hands have gained a lot of attention in the past decade because of their ability to adapt to the shape of the objects, increasing their effectiveness for gras**. However, when it comes to gras** highly flexible objects such as textiles, we face the dual problem: it is the object that will adapt to the shape of the hand or gripper. In this context, the classic grasp analysis…
▽ More
Compliant and soft hands have gained a lot of attention in the past decade because of their ability to adapt to the shape of the objects, increasing their effectiveness for gras**. However, when it comes to gras** highly flexible objects such as textiles, we face the dual problem: it is the object that will adapt to the shape of the hand or gripper. In this context, the classic grasp analysis or gras** taxonomies are not suitable for describing textile objects grasps. This work proposes a novel definition of textile object grasps that abstracts from the robotic embodiment or hand shape and recovers concepts from the early neuroscience literature on hand prehension skills. This framework enables us to identify what grasps have been used in literature until now to perform robotic cloth manipulation, and allows for a precise definition of all the tasks that have been tackled in terms of manipulation primitives based on regrasps. In addition, we also review what grippers have been used. Our analysis shows how the vast majority of cloth manipulations have relied only on one type of grasp, and at the same time we identify several tasks that need more variety of grasp types to be executed successfully. Our framework is generic, provides a classification of cloth manipulation primitives and can inspire gripper design and benchmark construction for cloth manipulation.
△ Less
Submitted 9 April, 2020; v1 submitted 19 June, 2019;
originally announced June 2019.
-
Analyzing Whole-Body Pose Transitions in Multi-Contact Motions
Authors:
Christian Mandery,
Júlia Borràs,
Mirjam Jöchner,
Tamim Asfour
Abstract:
When executing whole-body motions, humans are able to use a large variety of support poses which not only utilize the feet, but also hands, knees and elbows to enhance stability. While there are many works analyzing the transitions involved in walking, very few works analyze human motion where more complex supports occur.
In this work, we analyze complex support pose transitions in human motion…
▽ More
When executing whole-body motions, humans are able to use a large variety of support poses which not only utilize the feet, but also hands, knees and elbows to enhance stability. While there are many works analyzing the transitions involved in walking, very few works analyze human motion where more complex supports occur.
In this work, we analyze complex support pose transitions in human motion involving locomotion and manipulation tasks (loco-manipulation). We have applied a method for the detection of human support contacts from motion capture data to a large-scale dataset of loco-manipulation motions involving multi-contact supports, providing a semantic representation of them. Our results provide a statistical analysis of the used support poses, their transitions and the time spent in each of them. In addition, our data partially validates our taxonomy of whole-body support poses presented in our previous work.
We believe that this work extends our understanding of human motion for humanoids, with a long-term objective of develo** methods for autonomous multi-contact motion planning.
△ Less
Submitted 30 September, 2015; v1 submitted 31 July, 2015;
originally announced July 2015.
-
A Whole-Body Pose Taxonomy for Loco-Manipulation Tasks
Authors:
Júlia Borràs,
Tamim Asfour
Abstract:
Exploiting interaction with the environment is a promising and powerful way to enhance stability of humanoid robots and robustness while executing locomotion and manipulation tasks. Recently some works have started to show advances in this direction considering humanoid locomotion with multi-contacts, but to be able to fully develop such abilities in a more autonomous way, we need to first underst…
▽ More
Exploiting interaction with the environment is a promising and powerful way to enhance stability of humanoid robots and robustness while executing locomotion and manipulation tasks. Recently some works have started to show advances in this direction considering humanoid locomotion with multi-contacts, but to be able to fully develop such abilities in a more autonomous way, we need to first understand and classify the variety of possible poses a humanoid robot can achieve to balance. To this end, we propose the adaptation of a successful idea widely used in the field of robot gras** to the field of humanoid balance with multi-contacts: a whole-body pose taxonomy classifying the set of whole-body robot configurations that use the environment to enhance stability. We have revised criteria of classification used to develop gras** taxonomies, focusing on structuring and simplifying the large number of possible poses the human body can adopt. We propose a taxonomy with 46 poses, containing three main categories, considering number and type of supports as well as possible transitions between poses. The taxonomy induces a classification of motion primitives based on the pose used for support, and a set of rules to store and generate new motions. We present preliminary results that apply known segmentation techniques to motion data from the KIT whole-body motion database. Using motion capture data with multi-contacts, we can identify support poses providing a segmentation that can distinguish between locomotion and manipulation parts of an action.
△ Less
Submitted 22 September, 2015; v1 submitted 23 March, 2015;
originally announced March 2015.
