-
Fusing Visuo-Tactile Perception into Kernelized Synergies for Robust Gras** and Fine Manipulation of Non-rigid Objects
Authors:
Sunny Katyara,
Nikhil Deshpande,
Fanny Ficuciello,
Fei Chen,
Bruno Siciliano,
Darwin G. Caldwell
Abstract:
Handling non-rigid objects using robot hands necessities a framework that does not only incorporate human-level dexterity and cognition but also the multi-sensory information and system dynamics for robust and fine interactions. In this research, our previously developed kernelized synergies framework, inspired from human behaviour on reusing same subspace for gras** and manipulation, is augment…
▽ More
Handling non-rigid objects using robot hands necessities a framework that does not only incorporate human-level dexterity and cognition but also the multi-sensory information and system dynamics for robust and fine interactions. In this research, our previously developed kernelized synergies framework, inspired from human behaviour on reusing same subspace for gras** and manipulation, is augmented with visuo-tactile perception for autonomous and flexible adaptation to unknown objects. To detect objects and estimate their poses, a simplified visual pipeline using RANSAC algorithm with Euclidean clustering and SVM classifier is exploited. To modulate interaction efforts while gras** and manipulating non-rigid objects, the tactile feedback using T40S shokac chip sensor, generating 3D force information, is incorporated. Moreover, different kernel functions are examined in the kernelized synergies framework, to evaluate its performance and potential against task reproducibility, execution, generalization and synergistic re-usability. Experiments performed with robot arm-hand system validates the capability and usability of upgraded framework on stably gras** and dexterously manipulating the non-rigid objects.
△ Less
Submitted 15 September, 2021;
originally announced September 2021.
-
Challenges and Outlook in Robotic Manipulation of Deformable Objects
Authors:
Jihong Zhu,
Andrea Cherubini,
Claire Dune,
David Navarro-Alarcon,
Farshid Alambeigi,
Dmitry Berenson,
Fanny Ficuciello,
Kensuke Harada,
Jens Kober,
Xiang Li,
Jia Pan,
Wenzhen Yuan,
Michael Gienger
Abstract:
Deformable object manipulation (DOM) is an emerging research problem in robotics. The ability to manipulate deformable objects endows robots with higher autonomy and promises new applications in the industrial, services, and healthcare sectors. However, compared to rigid object manipulation, the manipulation of deformable objects is considerably more complex, and is still an open research problem.…
▽ More
Deformable object manipulation (DOM) is an emerging research problem in robotics. The ability to manipulate deformable objects endows robots with higher autonomy and promises new applications in the industrial, services, and healthcare sectors. However, compared to rigid object manipulation, the manipulation of deformable objects is considerably more complex, and is still an open research problem. Addressing DOM challenges demand breakthroughs in almost all aspects of robotics, namely hardware design, sensing, (deformation) modeling, planning, and control. In this article, we review recent advances and highlight the main challenges when considering deformation in each sub-field. A particular focus of our paper lies in the discussions of these challenges and proposing future directions of research.
△ Less
Submitted 14 December, 2021; v1 submitted 4 May, 2021;
originally announced May 2021.
-
Formulating Intuitive Stack-of-Tasks using Visuo-Tactile Perception for Collaborative Human-Robot Fine Manipulation
Authors:
Sunny Katyara,
Nikhil Deshpande,
Fanny Ficuciello,
Tao Teng,
Bruno Siciliano,
Darwin G. Caldwell,
Fei Chen
Abstract:
Enabling robots to work in close proximity to humans necessitates a control framework that does not only incorporate multi-sensory information for autonomous and coordinated interactions but also has perceptive task planning to ensure an adaptable and flexible collaborative behaviour. In this research, an intuitive stack-of-tasks (iSoT) formulation is proposed, that defines the robot's actions by…
▽ More
Enabling robots to work in close proximity to humans necessitates a control framework that does not only incorporate multi-sensory information for autonomous and coordinated interactions but also has perceptive task planning to ensure an adaptable and flexible collaborative behaviour. In this research, an intuitive stack-of-tasks (iSoT) formulation is proposed, that defines the robot's actions by considering the human-arm postures and the task progression. The framework is augmented with visuo-tactile information to effectively perceive the collaborative environment and intuitively switch between the planned sub-tasks. The visual feedback from depth cameras monitors and estimates the objects' poses and human-arm postures, while the tactile data provides the exploration skills to detect and maintain the desired contacts to avoid object slippage. To evaluate the performance, effectiveness and usability of the proposed framework, assembly and disassembly tasks, performed by the human-human and human-robot partners, are considered and analyzed using distinct evaluation metrics i.e, approach adaptation, grasp correction, task coordination latency, cumulative posture deviation, and task repeatability.
△ Less
Submitted 4 January, 2022; v1 submitted 9 March, 2021;
originally announced March 2021.
-
Vision Based Adaptation to Kernelized Synergies for Human Inspired Robotic Manipulation
Authors:
Sunny Katyara,
Fanny Ficuciello,
Fei Chen,
Bruno Siciliano,
Darwin G. Caldwell
Abstract:
Humans in contrast to robots are excellent in performing fine manipulation tasks owing to their remarkable dexterity and sensorimotor organization. Enabling robots to acquire such capabilities, necessitates a framework that not only replicates the human behaviour but also integrates the multi-sensory information for autonomous object interaction. To address such limitations, this research proposes…
▽ More
Humans in contrast to robots are excellent in performing fine manipulation tasks owing to their remarkable dexterity and sensorimotor organization. Enabling robots to acquire such capabilities, necessitates a framework that not only replicates the human behaviour but also integrates the multi-sensory information for autonomous object interaction. To address such limitations, this research proposes to augment the previously developed kernelized synergies framework with visual perception to automatically adapt to the unknown objects. The kernelized synergies, inspired from humans, retain the same reduced subspace for object gras** and manipulation. To detect object in the scene, a simplified perception pipeline is used that leverages the RANSAC algorithm with Euclidean clustering and SVM for object segmentation and recognition respectively. Further, the comparative analysis of kernelized synergies with other state of art approaches is made to confirm their flexibility and effectiveness on the robotic manipulation tasks. The experiments conducted on the robot hand confirm the robustness of modified kernelized synergies framework against the uncertainties related to the perception of environment.
△ Less
Submitted 13 December, 2020;
originally announced December 2020.
-
Force and state-feedback control for robots with non-collocated environmental and actuator forces
Authors:
Alejandro Donaire,
Luigi Villani,
Fanny Ficuciello,
Juan Tomassini,
Bruno Siciliano
Abstract:
In this paper, we present an impedance control design for multi-variable linear and nonlinear robotic systems. The control design considers force and state feedback to improve the performance of the closed loop. Simultaneous feedback of forces and states allows the controller for an extra degree of freedom to approximate the desired impedance port behaviour. A numerical analysis is used to demonst…
▽ More
In this paper, we present an impedance control design for multi-variable linear and nonlinear robotic systems. The control design considers force and state feedback to improve the performance of the closed loop. Simultaneous feedback of forces and states allows the controller for an extra degree of freedom to approximate the desired impedance port behaviour. A numerical analysis is used to demonstrate the desired impedance closed-loop behaviour.
△ Less
Submitted 24 October, 2020;
originally announced October 2020.
-
Reproducible Pruning System on Dynamic Natural Plants for Field Agricultural Robots
Authors:
Sunny Katyara,
Fanny Ficuciello,
Darwin G. Caldwell,
Fei Chen,
Bruno Siciliano
Abstract:
Pruning is the art of cutting unwanted and unhealthy plant branches and is one of the difficult tasks in the field robotics. It becomes even more complex when the plant branches are moving. Moreover, the reproducibility of robot pruning skills is another challenge to deal with due to the heterogeneous nature of vines in the vineyard. This research proposes a multi-modal framework to deal with the…
▽ More
Pruning is the art of cutting unwanted and unhealthy plant branches and is one of the difficult tasks in the field robotics. It becomes even more complex when the plant branches are moving. Moreover, the reproducibility of robot pruning skills is another challenge to deal with due to the heterogeneous nature of vines in the vineyard. This research proposes a multi-modal framework to deal with the dynamic vines with the aim of sim2real skill transfer. The 3D models of vines are constructed in blender engine and rendered in simulated environment as a need for training the robot. The Natural Admittance Controller (NAC) is applied to deal with the dynamics of vines. It uses force feedback and compensates the friction effects while maintaining the passivity of system. The faster R-CNN is used to detect the spurs on the vines and then statistical pattern recognition algorithm using K-means clustering is applied to find the effective pruning points. The proposed framework is tested in simulated and real environments.
△ Less
Submitted 13 December, 2020; v1 submitted 26 August, 2020;
originally announced August 2020.
-
Leveraging Kernelized Synergies on Shared Subspace for Precision Grasp and Dexterous Manipulation
Authors:
Sunny Katyara,
Fanny Ficuciello,
Darwin Caldwell,
Bruno Siciliano,
Fei Chen
Abstract:
Manipulation in contrast to gras** is a trajectorial task that needs to use dexterous hands. Improving the dexterity of robot hands, increases the controller complexity and thus requires to use the concept of postural synergies. Inspired from postural synergies, this research proposes a new framework called kernelized synergies that focuses on the re-usability of the same subspace for precision…
▽ More
Manipulation in contrast to gras** is a trajectorial task that needs to use dexterous hands. Improving the dexterity of robot hands, increases the controller complexity and thus requires to use the concept of postural synergies. Inspired from postural synergies, this research proposes a new framework called kernelized synergies that focuses on the re-usability of the same subspace for precision gras** and dexterous manipulation. In this work, the computed subspace of postural synergies; parameterized by probabilistic movement primitives, is treated with kernel to preserve its gras** and manipulation characteristics and allows its reuse for new objects. The grasp stability of the proposed framework is assessed with a force closure quality index. For performance evaluation, the proposed framework is tested on two different simulated robot hand models using the Syngrasp toolbox and experimentally, four complex gras** and manipulation tasks are performed and reported. The results confirm the hand agnostic approach of the proposed framework and its generalization to distinct objects irrespective of their shape and size.
△ Less
Submitted 3 September, 2020; v1 submitted 26 August, 2020;
originally announced August 2020.
