-
DexSkills: Skill Segmentation Using Haptic Data for Learning Autonomous Long-Horizon Robotic Manipulation Tasks
Authors:
Xiaofeng Mao,
Gabriele Giudici,
Claudio Coppola,
Kaspar Althoefer,
Ildar Farkhatdinov,
Zhibin Li,
Lorenzo Jamone
Abstract:
Effective execution of long-horizon tasks with dexterous robotic hands remains a significant challenge in real-world problems. While learning from human demonstrations have shown encouraging results, they require extensive data collection for training. Hence, decomposing long-horizon tasks into reusable primitive skills is a more efficient approach. To achieve so, we developed DexSkills, a novel s…
▽ More
Effective execution of long-horizon tasks with dexterous robotic hands remains a significant challenge in real-world problems. While learning from human demonstrations have shown encouraging results, they require extensive data collection for training. Hence, decomposing long-horizon tasks into reusable primitive skills is a more efficient approach. To achieve so, we developed DexSkills, a novel supervised learning framework that addresses long-horizon dexterous manipulation tasks using primitive skills. DexSkills is trained to recognize and replicate a select set of skills using human demonstration data, which can then segment a demonstrated long-horizon dexterous manipulation task into a sequence of primitive skills to achieve one-shot execution by the robot directly. Significantly, DexSkills operates solely on proprioceptive and tactile data, i.e., haptic data. Our real-world robotic experiments show that DexSkills can accurately segment skills, thereby enabling autonomous robot execution of a diverse range of tasks.
△ Less
Submitted 6 May, 2024;
originally announced May 2024.
-
Robotic surface exploration with vision and tactile sensing for cracks detection and characterisation
Authors:
Francesca Palermo,
Bukeikhan Omarali,
Changae Oh,
Kaspar Althoefer,
Ildar Farkhatdinov
Abstract:
This paper presents a novel algorithm for crack localisation and detection based on visual and tactile analysis via fibre-optics. A finger-shaped sensor based on fibre-optics is employed for the data acquisition to collect data for the analysis and the experiments. To detect the possible locations of cracks a camera is used to scan an environment while running an object detection algorithm. Once t…
▽ More
This paper presents a novel algorithm for crack localisation and detection based on visual and tactile analysis via fibre-optics. A finger-shaped sensor based on fibre-optics is employed for the data acquisition to collect data for the analysis and the experiments. To detect the possible locations of cracks a camera is used to scan an environment while running an object detection algorithm. Once the crack is detected, a fully-connected graph is created from a skeletonised version of the crack. A minimum spanning tree is then employed for calculating the shortest path to explore the crack which is then used to develop the motion planner for the robotic manipulator. The motion planner divides the crack into multiple nodes which are then explored individually. Then, the manipulator starts the exploration and performs the tactile data classification to confirm if there is indeed a crack in that location or just a false positive from the vision algorithm. If a crack is detected, also the length, width, orientation and number of branches are calculated. This is repeated until all the nodes of the crack are explored.
In order to validate the complete algorithm, various experiments are performed: comparison of exploration of cracks through full scan and motion planning algorithm, implementation of frequency-based features for crack classification and geometry analysis using a combination of vision and tactile data. From the results of the experiments, it is shown that the proposed algorithm is able to detect cracks and improve the results obtained from vision to correctly classify cracks and their geometry with minimal cost thanks to the motion planning algorithm.
△ Less
Submitted 13 July, 2023;
originally announced July 2023.
-
Extended Reality (XR) Remote Research: a Survey of Drawbacks and Opportunities
Authors:
Jack Ratcliffe,
Francesco Soave,
Nick Bryan-Kinns,
Laurissa Tokarchuk,
Ildar Farkhatdinov
Abstract:
Extended Reality (XR) technology - such as virtual and augmented reality - is now widely used in Human Computer Interaction (HCI), social science and psychology experimentation. However, these experiments are predominantly deployed in-lab with a co-present researcher. Remote experiments, without co-present researchers, have not flourished, despite the success of remote approaches for non-XR invest…
▽ More
Extended Reality (XR) technology - such as virtual and augmented reality - is now widely used in Human Computer Interaction (HCI), social science and psychology experimentation. However, these experiments are predominantly deployed in-lab with a co-present researcher. Remote experiments, without co-present researchers, have not flourished, despite the success of remote approaches for non-XR investigations. This paper summarises findings from a 30-item survey of 46 XR researchers to understand perceived limitations and benefits of remote XR experimentation. Our thematic analysis identifies concerns common with non-XR remote research, such as participant recruitment, as well as XR-specific issues, including safety and hardware variability. We identify potential positive affordances of XR technology, including leveraging data collection functionalities builtin to HMDs (e.g. hand, gaze tracking) and the portability and reproducibility of an experimental setting. We suggest that XR technology could be conceptualised as an interactive technology and a capable data-collection device suited for remote experimentation.
△ Less
Submitted 20 January, 2021;
originally announced January 2021.
-
The influence of posture, applied force and perturbation direction on hip joint viscoelasticity
Authors:
Hsien-Yung Huang,
Arash Arami,
Ildar Farkhatdinov,
Domenico Formica,
Etienne Burdet
Abstract:
Limb viscoelasticity is a critical factor used to regulate the interaction with the environment. It plays a key role in modelling human sensorimotor control, and can be used to assess the condition of healthy and neurologically affected individuals. This paper reports the estimation of hip joint viscoelasticity during voluntary force control using a novel device that applies a leg displacement wit…
▽ More
Limb viscoelasticity is a critical factor used to regulate the interaction with the environment. It plays a key role in modelling human sensorimotor control, and can be used to assess the condition of healthy and neurologically affected individuals. This paper reports the estimation of hip joint viscoelasticity during voluntary force control using a novel device that applies a leg displacement without constraining the hip joint. The influence of hip angle, applied limb force and perturbation direction on the stiffness values was studied in ten subjects. No difference was detected in the hip joint stiffness between the dominant and non-dominant legs, but a small dependency was observed on the perturbation direction. Both hip stiffness and viscosity increased monotonically with the applied force magnitude, with posture to being observed to have a slight influence. These results are in line with previous measurements carried out on upper limbs, and can be used as a baseline for lower limb movement simulation and further neuromechanical investigations.
△ Less
Submitted 23 March, 2020; v1 submitted 21 August, 2019;
originally announced August 2019.
-
Cable-driven robotic interface for lower limb neuromechanics identification
Authors:
Hsien-Yung Huang,
Ildar Farkhatdinov,
Arash Arami,
Mohamed Bouri,
Etienne Burdet
Abstract:
This paper presents a versatile cable-driven robotic interface to investigate the single-joint joint neuromechanics of the hip, knee and ankle. This endpoint-based interface offers highly dynamic interaction and accurate position control, as is typically required for neuromechanics identification. It can be used with the subject upright, corresponding to natural posture during walking or standing,…
▽ More
This paper presents a versatile cable-driven robotic interface to investigate the single-joint joint neuromechanics of the hip, knee and ankle. This endpoint-based interface offers highly dynamic interaction and accurate position control, as is typically required for neuromechanics identification. It can be used with the subject upright, corresponding to natural posture during walking or standing, and does not impose kinematic constraints on a joint, in contrast to existing interfaces. Mechanical evaluations demonstrated that the interface yields a rigidity above 500N/m with low viscosity. Tests with a rigid dummy leg and linear springs show that it can identify the mechanical impedance of a limb accurately. A smooth perturbation is developed and tested with a human subject, which can be used to estimate the hip neuromechanics.
△ Less
Submitted 21 June, 2020; v1 submitted 7 August, 2019;
originally announced August 2019.
