-
SVan: A Mobile Hub as a Field Robotics Development and Deployment Platform
Authors:
Alexander Moortgat-Pick,
Anna Adamczyk,
Daniel A Duecker,
Sami Haddadin
Abstract:
As robotics becomes increasingly vital for environmental protection, there is a growing need for effective deployment methods that match the pace of robotics innovation. Current strategies often fall short, leaving a gap between the potential of robotics and their practical application in the field. Addressing this challenge, we introduce a mobile hub concept designed to provide the necessary infr…
▽ More
As robotics becomes increasingly vital for environmental protection, there is a growing need for effective deployment methods that match the pace of robotics innovation. Current strategies often fall short, leaving a gap between the potential of robotics and their practical application in the field. Addressing this challenge, we introduce a mobile hub concept designed to provide the necessary infrastructure and support for deploying a diverse, multi-domain robot team effectively. This paper presents the development and insights into `SVAN' (Synchronous Team-Robot Van), a prototype of our mobile hub concept. We delve into the mechanical construction and software setup of SVAN, offering a comprehensive overview of its capabilities and design considerations. Further, we discuss the hardware specifications and share valuable lessons learned during the prototype's development and deployment. In addition to this paper, an accepted video complements our exploration by depicting SVAN in its envisioned role as an environmental guardian, highlighting its potential in ecological monitoring and preservation. Furthermore, our discussion is enriched by referencing a previously accepted paper detailing a novel methodology for continuous UAV mission cycling enabled by a mobile hub like SVAN. These accompanying works underscore our contribution towards addressing the existing gaps in robot deployment strategies, presenting a scalable and efficient framework to overcome operational challenges in environmental robotics.
△ Less
Submitted 6 May, 2024;
originally announced May 2024.
-
Towards Safe Robot Use with Edged or Pointed Objects: A Surrogate Study Assembling a Human Hand Injury Protection Database
Authors:
Robin Jeanne Kirschner,
Carina M. Micheler,
Yangcan Zhou,
Sebastian Siegner,
Mazin Hamad,
Claudio Glowalla,
Jan Neumann,
Nader Rajaei,
Rainer Burgkart,
Sami Haddadin
Abstract:
The use of pointed or edged tools or objects is one of the most challenging aspects of today's application of physical human-robot interaction (pHRI). One reason for this is that the severity of harm caused by such edged or pointed impactors is less well studied than for blunt impactors. Consequently, the standards specify well-reasoned force and pressure thresholds for blunt impactors and advise…
▽ More
The use of pointed or edged tools or objects is one of the most challenging aspects of today's application of physical human-robot interaction (pHRI). One reason for this is that the severity of harm caused by such edged or pointed impactors is less well studied than for blunt impactors. Consequently, the standards specify well-reasoned force and pressure thresholds for blunt impactors and advise avoiding any edges and corners in contacts. Nevertheless, pointed or edged impactor geometries cannot be completely ruled out in real pHRI applications. For example, to allow edged or pointed tools such as screwdrivers near human operators, the knowledge of injury severity needs to be extended so that robot integrators can perform well-reasoned, time-efficient risk assessments. In this paper, we provide the initial datasets on injury prevention for the human hand based on drop tests with surrogates for the human hand, namely pig claws and chicken drumsticks. We then demonstrate the ease and efficiency of robot use using the dataset for contact on two examples. Finally, our experiments provide a set of injuries that may also be expected for human subjects under certain robot mass-velocity constellations in collisions. To extend this work, testing on human samples and a collaborative effort from research institutes worldwide is needed to create a comprehensive human injury avoidance database for any pHRI scenario and thus for safe pHRI applications including edged and pointed geometries.
△ Less
Submitted 5 April, 2024;
originally announced April 2024.
-
Learning Barrier-Certified Polynomial Dynamical Systems for Obstacle Avoidance with Robots
Authors:
Martin Schonger,
Hugo T. M. Kussaba,
Lingyun Chen,
Luis Figueredo,
Abdalla Swikir,
Aude Billard,
Sami Haddadin
Abstract:
Established techniques that enable robots to learn from demonstrations are based on learning a stable dynamical system (DS). To increase the robots' resilience to perturbations during tasks that involve static obstacle avoidance, we propose incorporating barrier certificates into an optimization problem to learn a stable and barrier-certified DS. Such optimization problem can be very complex or ex…
▽ More
Established techniques that enable robots to learn from demonstrations are based on learning a stable dynamical system (DS). To increase the robots' resilience to perturbations during tasks that involve static obstacle avoidance, we propose incorporating barrier certificates into an optimization problem to learn a stable and barrier-certified DS. Such optimization problem can be very complex or extremely conservative when the traditional linear parameter-varying formulation is used. Thus, different from previous approaches in the literature, we propose to use polynomial representations for DSs, which yields an optimization problem that can be tackled by sum-of-squares techniques. Finally, our approach can handle obstacle shapes that fall outside the scope of assumptions typically found in the literature concerning obstacle avoidance within the DS learning framework. Supplementary material can be found at the project webpage: https://martinschonger.github.io/abc-ds
△ Less
Submitted 12 March, 2024;
originally announced March 2024.
-
Safe Execution of Learned Orientation Skills with Conic Control Barrier Functions
Authors:
Zheng Shen,
Matteo Saveriano,
Fares J. Abu-Dakka,
Sami Haddadin
Abstract:
In the field of Learning from Demonstration (LfD), Dynamical Systems (DSs) have gained significant attention due to their ability to generate real-time motions and reach predefined targets. However, the conventional convergence-centric behavior exhibited by DSs may fall short in safety-critical tasks, specifically, those requiring precise replication of demonstrated trajectories or strict adherenc…
▽ More
In the field of Learning from Demonstration (LfD), Dynamical Systems (DSs) have gained significant attention due to their ability to generate real-time motions and reach predefined targets. However, the conventional convergence-centric behavior exhibited by DSs may fall short in safety-critical tasks, specifically, those requiring precise replication of demonstrated trajectories or strict adherence to constrained regions even in the presence of perturbations or human intervention. Moreover, existing DS research often assumes demonstrations solely in Euclidean space, overlooking the crucial aspect of orientation in various applications. To alleviate these shortcomings, we present an innovative approach geared toward ensuring the safe execution of learned orientation skills within constrained regions surrounding a reference trajectory. This involves learning a stable DS on SO(3), extracting time-varying conic constraints from the variability observed in expert demonstrations, and bounding the evolution of the DS with Conic Control Barrier Function (CCBF) to fulfill the constraints. We validated our approach through extensive evaluation in simulation and showcased its effectiveness for a cutting skill in the context of assisted teleoperation.
△ Less
Submitted 8 March, 2024;
originally announced March 2024.
-
Geometric Slosh-Free Tracking for Robotic Manipulators
Authors:
Jon Arrizabalaga,
Lukas Pries,
Riddhiman Laha,
Runkang Li,
Sami Haddadin,
Markus Ryll
Abstract:
This work focuses on the agile transportation of liquids with robotic manipulators. In contrast to existing methods that are either computationally heavy, system/container specific or dependant on a singularity-prone pendulum model, we present a real-time slosh-free tracking technique. This method solely requires the reference trajectory and the robot's kinematic constraints to output kinematicall…
▽ More
This work focuses on the agile transportation of liquids with robotic manipulators. In contrast to existing methods that are either computationally heavy, system/container specific or dependant on a singularity-prone pendulum model, we present a real-time slosh-free tracking technique. This method solely requires the reference trajectory and the robot's kinematic constraints to output kinematically feasible joint space commands. The crucial element underlying this approach consists on mimicking the end-effector's motion through a virtual quadrotor, which is inherently slosh-free and differentially flat, thereby allowing us to calculate a slosh-free reference orientation. Through the utilization of a cascaded proportional-derivative (PD) controller, this slosh-free reference is transformed into task space acceleration commands, which, following the resolution of a Quadratic Program (QP) based on Resolved Acceleration Control (RAC), are translated into a feasible joint configuration. The validity of the proposed approach is demonstrated by simulated and real-world experiments on a 7 DoF Franka Emika Panda robot.
Code: https://github.com/jonarriza96/gsft Video: https://youtu.be/4kitqYVS9n8
△ Less
Submitted 7 February, 2024;
originally announced February 2024.
-
Smooth real-time motion planning based on a cascade dual-quaternion screw-geometry MPC
Authors:
Ainoor Teimoorzadeh,
Frederico Fernandes Afonso Silva,
Luis F. C. Figueredo,
Sami Haddadin
Abstract:
This paper investigates the tracking problem of a smooth coordinate-invariant trajectory using dual quaternion algebra. The proposed architecture consists of a cascade structure in which the outer-loop MPC performs real-time smoothing of the manipulator's end-effector twist while an inner-loop kinematic controller ensures tracking of the instantaneous desired end-effector pose. Experiments on a…
▽ More
This paper investigates the tracking problem of a smooth coordinate-invariant trajectory using dual quaternion algebra. The proposed architecture consists of a cascade structure in which the outer-loop MPC performs real-time smoothing of the manipulator's end-effector twist while an inner-loop kinematic controller ensures tracking of the instantaneous desired end-effector pose. Experiments on a $7$-DoF Franka Emika Panda robotic manipulator validate the proposed method demonstrating its application to constraint the robot twists, accelerations and jerks within prescribed bounds.
△ Less
Submitted 7 February, 2024;
originally announced February 2024.
-
Real-time Contact State Estimation in Shape Control of Deformable Linear Objects under Small Environmental Constraints
Authors:
Kejia Chen,
Zhenshan Bing,
Yansong Wu,
Fan Wu,
Liding Zhang,
Sami Haddadin,
Alois Knoll
Abstract:
Controlling the shape of deformable linear objects using robots and constraints provided by environmental fixtures has diverse industrial applications. In order to establish robust contacts with these fixtures, accurate estimation of the contact state is essential for preventing and rectifying potential anomalies. However, this task is challenging due to the small sizes of fixtures, the requiremen…
▽ More
Controlling the shape of deformable linear objects using robots and constraints provided by environmental fixtures has diverse industrial applications. In order to establish robust contacts with these fixtures, accurate estimation of the contact state is essential for preventing and rectifying potential anomalies. However, this task is challenging due to the small sizes of fixtures, the requirement for real-time performances, and the infinite degrees of freedom of the deformable linear objects. In this paper, we propose a real-time approach for estimating both contact establishment and subsequent changes by leveraging the dependency between the applied and detected contact force on the deformable linear objects. We seamlessly integrate this method into the robot control loop and achieve an adaptive shape control framework which avoids, detects and corrects anomalies automatically. Real-world experiments validate the robustness and effectiveness of our contact estimation approach across various scenarios, significantly increasing the success rate of shape control processes.
△ Less
Submitted 30 January, 2024;
originally announced January 2024.
-
ULT-model: Towards a one-legged unified locomotion template model for forward hop** with an upright trunk
Authors:
Dennis Ossadnik,
Elisabeth Jensen,
Sami Haddadin
Abstract:
While many advancements have been made in the development of template models for describing upright-trunk locomotion, the majority of the effort has been focused on the stance phase. In this paper, we develop a new compact dynamic model as a first step toward a fully unified locomotion template model (ULT-model) of an upright-trunk forward hop** system, which will also require a unified control…
▽ More
While many advancements have been made in the development of template models for describing upright-trunk locomotion, the majority of the effort has been focused on the stance phase. In this paper, we develop a new compact dynamic model as a first step toward a fully unified locomotion template model (ULT-model) of an upright-trunk forward hop** system, which will also require a unified control law in the next step. We demonstrate that all locomotion subfunctions are enabled by adding just a point foot mass and a parallel leg actuator to the well-known trunk SLIP model and that a stable limit cycle can be achieved. This brings us closer toward the ultimate goal of enabling closed-loop dynamics for anchor matching and thus achieving simple, efficient, robust and stable upright-trunk gait control, as observed in biological systems.
△ Less
Submitted 16 January, 2024;
originally announced January 2024.
-
Nonlinear stiffness allows passive dynamic hop** for one-legged robots with an upright trunk
Authors:
Dennis Ossadnik,
Elisabeth Jensen,
Sami Haddadin
Abstract:
Template models are frequently used to simplify the control dynamics for robot hop** or running. Passive limit cycles can emerge for such systems and be exploited for energy-efficient control. A grand challenge in locomotion is trunk stabilization when the hip is offset from the center of mass (CoM). The swing phase plays a major role in this process due to the moment of inertia of the leg; howe…
▽ More
Template models are frequently used to simplify the control dynamics for robot hop** or running. Passive limit cycles can emerge for such systems and be exploited for energy-efficient control. A grand challenge in locomotion is trunk stabilization when the hip is offset from the center of mass (CoM). The swing phase plays a major role in this process due to the moment of inertia of the leg; however, many template models ignore the leg mass. In this work, the authors consider a robot hopper model (RHM) with a rigid trunk and leg plus a hip that is displaced from the CoM. It has been previously shown that no passive limit cycle exists for such a model given a linear hip spring. In this work, we show that passive limit cycles can be found when a nonlinear hip spring is used instead. To the authors' knowledge, this is the first time that a passive limit cycle has been found for this type of system.
△ Less
Submitted 16 January, 2024;
originally announced January 2024.
-
Anthropomorphic Gras** with Neural Object Shape Completion
Authors:
Diego Hidalgo-Carvajal,
Hanzhi Chen,
Gemma C. Bettelani,
Jaesug Jung,
Melissa Zavaglia,
Laura Busse,
Abdeldjallil Naceri,
Stefan Leutenegger,
Sami Haddadin
Abstract:
The progressive prevalence of robots in human-suited environments has given rise to a myriad of object manipulation techniques, in which dexterity plays a paramount role. It is well-established that humans exhibit extraordinary dexterity when handling objects. Such dexterity seems to derive from a robust understanding of object properties (such as weight, size, and shape), as well as a remarkable…
▽ More
The progressive prevalence of robots in human-suited environments has given rise to a myriad of object manipulation techniques, in which dexterity plays a paramount role. It is well-established that humans exhibit extraordinary dexterity when handling objects. Such dexterity seems to derive from a robust understanding of object properties (such as weight, size, and shape), as well as a remarkable capacity to interact with them. Hand postures commonly demonstrate the influence of specific regions on objects that need to be grasped, especially when objects are partially visible. In this work, we leverage human-like object understanding by reconstructing and completing their full geometry from partial observations, and manipulating them using a 7-DoF anthropomorphic robot hand. Our approach has significantly improved the gras** success rates of baselines with only partial reconstruction by nearly 30% and achieved over 150 successful grasps with three different object categories. This demonstrates our approach's consistent ability to predict and execute gras** postures based on the completed object shapes from various directions and positions in real-world scenarios. Our work opens up new possibilities for enhancing robotic applications that require precise gras** and manipulation skills of real-world reconstructed objects.
△ Less
Submitted 9 November, 2023; v1 submitted 4 November, 2023;
originally announced November 2023.
-
Flexible Informed Trees (FIT*): Adaptive Batch-Size Approach in Informed Sampling-Based Path Planning
Authors:
Liding Zhang,
Zhenshan Bing,
Kejia Chen,
Lingyun Chen,
Kuanqi Cai,
Yu Zhang,
Fan Wu,
Peter Krumbholz,
Zhilin Yuan,
Sami Haddadin,
Alois Knoll
Abstract:
In path planning, anytime almost-surely asymptotically optimal planners dominate the benchmark of sampling-based planners. A notable example is Batch Informed Trees (BIT*), where planners iteratively determine paths to batches of vertices within the exploration area. However, utilizing a consistent batch size is inefficient for initial pathfinding and optimal performance, it relies on effective ta…
▽ More
In path planning, anytime almost-surely asymptotically optimal planners dominate the benchmark of sampling-based planners. A notable example is Batch Informed Trees (BIT*), where planners iteratively determine paths to batches of vertices within the exploration area. However, utilizing a consistent batch size is inefficient for initial pathfinding and optimal performance, it relies on effective task allocation. This paper introduces Flexible Informed Trees (FIT*), a sampling-based planner that integrates an adaptive batch-size method to enhance the initial path convergence rate. FIT* employs a flexible approach in adjusting batch sizes dynamically based on the inherent dimension of the configuration spaces and the hypervolume of the n-dimensional hyperellipsoid. By applying dense and sparse sampling strategy, FIT* improves convergence rate while finding successful solutions faster with lower initial solution cost. This method enhances the planner's ability to handle confined, narrow spaces in the initial finding phase and increases batch vertices sampling frequency in the optimization phase. FIT* outperforms existing single-query, sampling-based planners on the tested problems in R^2 to R^8, and was demonstrated on a real-world mobile manipulation task.
△ Less
Submitted 2 March, 2024; v1 submitted 19 October, 2023;
originally announced October 2023.
-
LoHoRavens: A Long-Horizon Language-Conditioned Benchmark for Robotic Tabletop Manipulation
Authors:
Shengqiang Zhang,
Philipp Wicke,
Lütfi Kerem Şenel,
Luis Figueredo,
Abdeldjallil Naceri,
Sami Haddadin,
Barbara Plank,
Hinrich Schütze
Abstract:
The convergence of embodied agents and large language models (LLMs) has brought significant advancements to embodied instruction following. Particularly, the strong reasoning capabilities of LLMs make it possible for robots to perform long-horizon tasks without expensive annotated demonstrations. However, public benchmarks for testing the long-horizon reasoning capabilities of language-conditioned…
▽ More
The convergence of embodied agents and large language models (LLMs) has brought significant advancements to embodied instruction following. Particularly, the strong reasoning capabilities of LLMs make it possible for robots to perform long-horizon tasks without expensive annotated demonstrations. However, public benchmarks for testing the long-horizon reasoning capabilities of language-conditioned robots in various scenarios are still missing. To fill this gap, this work focuses on the tabletop manipulation task and releases a simulation benchmark, \textit{LoHoRavens}, which covers various long-horizon reasoning aspects spanning color, size, space, arithmetics and reference. Furthermore, there is a key modality bridging problem for long-horizon manipulation tasks with LLMs: how to incorporate the observation feedback during robot execution for the LLM's closed-loop planning, which is however less studied by prior work. We investigate two methods of bridging the modality gap: caption generation and learnable interface for incorporating explicit and implicit observation feedback to the LLM, respectively. These methods serve as the two baselines for our proposed benchmark. Experiments show that both methods struggle to solve some tasks, indicating long-horizon manipulation tasks are still challenging for current popular models. We expect the proposed public benchmark and baselines can help the community develop better models for long-horizon tabletop manipulation tasks.
△ Less
Submitted 23 October, 2023; v1 submitted 18 October, 2023;
originally announced October 2023.
-
Care3D: An Active 3D Object Detection Dataset of Real Robotic-Care Environments
Authors:
Michael G. Adam,
Sebastian Eger,
Martin Piccolrovazzi,
Maged Iskandar,
Joern Vogel,
Alexander Dietrich,
Seongjien Bien,
Jon Skerlj,
Abdeldjallil Naceri,
Eckehard Steinbach,
Alin Albu-Schaeffer,
Sami Haddadin,
Wolfram Burgard
Abstract:
As labor shortage increases in the health sector, the demand for assistive robotics grows. However, the needed test data to develop those robots is scarce, especially for the application of active 3D object detection, where no real data exists at all. This short paper counters this by introducing such an annotated dataset of real environments. The captured environments represent areas which are al…
▽ More
As labor shortage increases in the health sector, the demand for assistive robotics grows. However, the needed test data to develop those robots is scarce, especially for the application of active 3D object detection, where no real data exists at all. This short paper counters this by introducing such an annotated dataset of real environments. The captured environments represent areas which are already in use in the field of robotic health care research. We further provide ground truth data within one room, for assessing SLAM algorithms running directly on a health care robot.
△ Less
Submitted 9 October, 2023;
originally announced October 2023.
-
A Concise Overview of Safety Aspects in Human-Robot Interaction
Authors:
Mazin Hamad,
Simone Nertinger,
Robin J. Kirschner,
Luis Figueredo,
Abdeldjallil Naceri,
Sami Haddadin
Abstract:
As of today, robots exhibit impressive agility but also pose potential hazards to humans using/collaborating with them. Consequently, safety is considered the most paramount factor in human-robot interaction (HRI). This paper presents a multi-layered safety architecture, integrating both physical and cognitive aspects for effective HRI. We outline critical requirements for physical safety layers a…
▽ More
As of today, robots exhibit impressive agility but also pose potential hazards to humans using/collaborating with them. Consequently, safety is considered the most paramount factor in human-robot interaction (HRI). This paper presents a multi-layered safety architecture, integrating both physical and cognitive aspects for effective HRI. We outline critical requirements for physical safety layers as service modules that can be arbitrarily queried. Further, we showcase an HRI scheme that addresses human factors and perceived safety as high-level constraints on a validated impact safety paradigm. The aim is to enable safety certification of human-friendly robots across various HRI scenarios.
△ Less
Submitted 18 September, 2023;
originally announced September 2023.
-
Optimally Controlling the Timing of Energy Transfer in Elastic Joints: Experimental Validation of the Bi-Stiffness Actuation Concept
Authors:
Edmundo Pozo Fortunić,
Mehmet C. Yildirim,
Dennis Ossadnik,
Abdalla Swikir,
Saeed Abdolshah,
Sami Haddadin
Abstract:
Elastic actuation taps into elastic elements' energy storage for dynamic motions beyond rigid actuation. While Series Elastic Actuators (SEA) and Variable Stiffness Actuators (VSA) are highly sophisticated, they do not fully provide control over energy transfer timing. To overcome this problem on the basic system level, the Bi-Stiffness Actuation (BSA) concept was recently proposed. Theoretically,…
▽ More
Elastic actuation taps into elastic elements' energy storage for dynamic motions beyond rigid actuation. While Series Elastic Actuators (SEA) and Variable Stiffness Actuators (VSA) are highly sophisticated, they do not fully provide control over energy transfer timing. To overcome this problem on the basic system level, the Bi-Stiffness Actuation (BSA) concept was recently proposed. Theoretically, it allows for full link decoupling, while simultaneously being able to lock the spring in the drive train via a switch-and-hold mechanism. Thus, the user would be in full control of the potential energy storage and release timing. In this work, we introduce an initial proof-of-concept of Bi-Stiffness-Actuation in the form of a 1-DoF physical prototype, which is implemented using a modular testbed. We present a hybrid system model, as well as the mechatronic implementation of the actuator. We corroborate the feasibility of the concept by conducting a series of hardware experiments using an open-loop control signal obtained by trajectory optimization. Here, we compare the performance of the prototype with a comparable SEA implementation. We show that BSA outperforms SEA 1) in terms of maximum velocity at low final times and 2) in terms of the movement strategy itself: The clutch mechanism allows the BSA to generate consistent launch sequences while the SEA has to rely on lengthy and possibly dangerous oscillatory swing-up motions. Furthermore, we demonstrate that providing full control authority over the energy transfer timing and link decoupling allows the user to synchronously release both elastic joint and gravitational energy. This facilitates the optimal exploitation of elastic and gravitational potentials in a synergistic manner.
△ Less
Submitted 14 September, 2023;
originally announced September 2023.
-
Towards Connecting Control to Perception: High-Performance Whole-Body Collision Avoidance Using Control-Compatible Obstacles
Authors:
Moritz Eckhoff,
Dennis Knobbe,
Henning Zwirnmann,
Abdalla Swikir,
Sami Haddadin
Abstract:
One of the most important aspects of autonomous systems is safety. This includes ensuring safe human-robot and safe robot-environment interaction when autonomously performing complex tasks or in collaborative scenarios. Although several methods have been introduced to tackle this, most are unsuitable for real-time applications and require carefully hand-crafted obstacle descriptions. In this work,…
▽ More
One of the most important aspects of autonomous systems is safety. This includes ensuring safe human-robot and safe robot-environment interaction when autonomously performing complex tasks or in collaborative scenarios. Although several methods have been introduced to tackle this, most are unsuitable for real-time applications and require carefully hand-crafted obstacle descriptions. In this work, we propose a method combining high-frequency and real-time self and environment collision avoidance of a robotic manipulator with low-frequency, multimodal, and high-resolution environmental perceptions accumulated in a digital twin system. Our method is based on geometric primitives, so-called primitive skeletons. These, in turn, are information-compressed and real-time compatible digital representations of the robot's body and environment, automatically generated from ultra-realistic virtual replicas of the real world provided by the digital twin. Our approach is a key enabler for closing the loop between environment perception and robot control by providing the millisecond real-time control stage with a current and accurate world description, empowering it to react to environmental changes. We evaluate our whole-body collision avoidance on a 9-DOFs robot system through five experiments, demonstrating the functionality and efficiency of our framework.
△ Less
Submitted 15 December, 2023; v1 submitted 13 September, 2023;
originally announced September 2023.
-
Contact-aware Sha** and Maintenance of Deformable Linear Objects With Fixtures
Authors:
Kejia Chen,
Zhenshan Bing,
Fan Wu,
Yuan Meng,
Andre Kraft,
Sami Haddadin,
Alois Knoll
Abstract:
Studying the manipulation of deformable linear objects has significant practical applications in industry, including car manufacturing, textile production, and electronics automation. However, deformable linear object manipulation poses a significant challenge in develo** planning and control algorithms, due to the precise and continuous control required to effectively manipulate the deformable…
▽ More
Studying the manipulation of deformable linear objects has significant practical applications in industry, including car manufacturing, textile production, and electronics automation. However, deformable linear object manipulation poses a significant challenge in develo** planning and control algorithms, due to the precise and continuous control required to effectively manipulate the deformable nature of these objects. In this paper, we propose a new framework to control and maintain the shape of deformable linear objects with two robot manipulators utilizing environmental contacts. The framework is composed of a shape planning algorithm which automatically generates appropriate positions to place fixtures, and an object-centered skill engine which includes task and motion planning to control the motion and force of both robots based on the object status. The status of the deformable linear object is estimated online utilizing visual as well as force information. The framework manages to handle a cable routing task in real-world experiments with two Panda robots and especially achieves contact-aware and flexible clip fixing with challenging fixtures.
△ Less
Submitted 19 July, 2023;
originally announced July 2023.
-
Towards Language-Based Modulation of Assistive Robots through Multimodal Models
Authors:
Philipp Wicke,
Lüfti Kerem Şenel,
Shengqiang Zhang,
Luis Figueredo,
Abdeldjallil Naceri,
Sami Haddadin,
Hinrich Schütze
Abstract:
In the field of Geriatronics, enabling effective and transparent communication between humans and robots is crucial for enhancing the acceptance and performance of assistive robots. Our early-stage research project investigates the potential of language-based modulation as a means to improve human-robot interaction. We propose to explore real-time modulation during task execution, leveraging langu…
▽ More
In the field of Geriatronics, enabling effective and transparent communication between humans and robots is crucial for enhancing the acceptance and performance of assistive robots. Our early-stage research project investigates the potential of language-based modulation as a means to improve human-robot interaction. We propose to explore real-time modulation during task execution, leveraging language cues, visual references, and multimodal inputs. By develo** transparent and interpretable methods, we aim to enable robots to adapt and respond to language commands, enhancing their usability and flexibility. Through the exchange of insights and knowledge at the workshop, we seek to gather valuable feedback to advance our research and contribute to the development of interactive robotic systems for Geriatronics and beyond.
△ Less
Submitted 27 June, 2023; v1 submitted 26 June, 2023;
originally announced June 2023.
-
Fast yet predictable braking manoeuvers for real-time robot control
Authors:
Mazin Hamad,
Jesus Gutierrez-Moreno,
Hugo T. M. Kussaba,
Nico Mansfeld,
Saeed Abdolshah,
Abdalla Swikir,
Wolfram Burgard,
Sami Haddadin
Abstract:
This paper proposes a framework for generating fast, smooth and predictable braking manoeuvers for a controlled robot. The proposed framework integrates two approaches to obtain feasible modal limits for designing braking trajectories. The first approach is real-time capable but conservative considering the usage of the available feasible actuator control region, resulting in longer braking times.…
▽ More
This paper proposes a framework for generating fast, smooth and predictable braking manoeuvers for a controlled robot. The proposed framework integrates two approaches to obtain feasible modal limits for designing braking trajectories. The first approach is real-time capable but conservative considering the usage of the available feasible actuator control region, resulting in longer braking times. In contrast, the second approach maximizes the used braking control inputs at the cost of requiring more time to evaluate larger, feasible modal limits via optimization. Both approaches allow for predicting the robot's stop** trajectory online. In addition, we also formulated and solved a constrained, nonlinear final-time minimization problem to find optimal torque inputs. The optimal solutions were used as a benchmark to evaluate the performance of the proposed predictable braking framework. A comparative study was compiled in simulation versus a classical optimal controller on a 7-DoF robot arm with only three moving joints. The results verified the effectiveness of our proposed framework and its integrated approaches in achieving fast robot braking manoeuvers with accurate online predictions of the stop** trajectories and distances under various braking settings.
△ Less
Submitted 10 June, 2023;
originally announced June 2023.
-
Learning optimal controllers: a dynamical motion primitive approach
Authors:
Hugo T. M. Kussaba,
Abdalla Swikir,
Fan Wu,
Anastasija Demerdjieva,
Gitta Kutyniok,
Sami Haddadin
Abstract:
Real-time computation of optimal control is a challenging problem and, to solve this difficulty, many frameworks proposed to use learning techniques to learn (possibly sub-optimal) controllers and enable their usage in an online fashion. Among these techniques, the optimal motion framework is a simple, yet powerful technique, that obtained success in many complex real-world applications. The main…
▽ More
Real-time computation of optimal control is a challenging problem and, to solve this difficulty, many frameworks proposed to use learning techniques to learn (possibly sub-optimal) controllers and enable their usage in an online fashion. Among these techniques, the optimal motion framework is a simple, yet powerful technique, that obtained success in many complex real-world applications. The main idea of this approach is to take advantage of dynamic motion primitives, a widely used tool in robotics to learn trajectories from demonstrations. While usually these demonstrations come from humans, the optimal motion framework is based on demonstrations coming from optimal solutions, such as the ones obtained by numeric solvers. As usual in many learning techniques, a drawback of this approach is that it is hard to estimate the suboptimality of learned solutions, since finding easily computable and non-trivial upper bounds to the error between an optimal solution and a learned solution is, in general, unfeasible. However, we show in this paper that it is possible to estimate this error for a broad class of problems. Furthermore, we apply this estimation technique to achieve a novel and more efficient sampling scheme to be used within the optimal motion framework, enabling the usage of this framework in some scenarios where the computational resources are limited.
△ Less
Submitted 10 June, 2023;
originally announced June 2023.
-
Towards Flexible Biolaboratory Automation: Container Taxonomy-Based, 3D-Printed Gripper Fingers
Authors:
Henning Zwirnmann,
Dennis Knobbe,
Utku Culha,
Sami Haddadin
Abstract:
Automation in the life science research laboratory is a paradigm that has gained increasing relevance in recent years. Current robotic solutions often have a limited scope, which reduces their acceptance and prevents the realization of complex workflows. The transport and manipulation of laboratory supplies with a robot is a particular case where this limitation manifests. In this paper, we deduce…
▽ More
Automation in the life science research laboratory is a paradigm that has gained increasing relevance in recent years. Current robotic solutions often have a limited scope, which reduces their acceptance and prevents the realization of complex workflows. The transport and manipulation of laboratory supplies with a robot is a particular case where this limitation manifests. In this paper, we deduce a taxonomy of biolaboratory liquid containers that clarifies the need for a flexible gras** solution. Using the taxonomy as a guideline, we design fingers for a parallel robotic gripper which are developed with a monolithic dual-extrusion 3D print that integrates rigid and soft materials to optimize grip** properties. We design fine-tuned fingertips that provide stable grasps of the containers in question. A simple actuation system and a low weight are maintained by adopting a passive compliant mechanism. The ability to resist chemicals and high temperatures and the integration with a tool exchange system render the fingers usable for daily laboratory use and complex workflows. We present the task suitability of the fingers in experiments that show the wide range of vessels that can be handled as well as their tolerance against displacements and their grasp stability.
△ Less
Submitted 15 December, 2023; v1 submitted 7 February, 2023;
originally announced February 2023.
-
BSA -- Bi-Stiffness Actuation for optimally exploiting intrinsic compliance and inertial coupling effects in elastic joint robots
Authors:
Dennis Ossadnik,
Mehmet C. Yildirim,
Fan Wu,
Abdalla Swikir,
Hugo T. M. Kussaba,
Saeed Abdolshah,
Sami Haddadin
Abstract:
Compliance in actuation has been exploited to generate highly dynamic maneuvers such as throwing that take advantage of the potential energy stored in joint springs. However, the energy storage and release could not be well-timed yet. On the contrary, for multi-link systems, the natural system dynamics might even work against the actual goal. With the introduction of variable stiffness actuators,…
▽ More
Compliance in actuation has been exploited to generate highly dynamic maneuvers such as throwing that take advantage of the potential energy stored in joint springs. However, the energy storage and release could not be well-timed yet. On the contrary, for multi-link systems, the natural system dynamics might even work against the actual goal. With the introduction of variable stiffness actuators, this problem has been partially addressed. With a suitable optimal control strategy, the approximate decoupling of the motor from the link can be achieved to maximize the energy transfer into the distal link prior to launch. However, such continuous stiffness variation is complex and typically leads to oscillatory swing-up motions instead of clear launch sequences. To circumvent this issue, we investigate decoupling for speed maximization with a dedicated novel actuator concept denoted Bi-Stiffness Actuation. With this, it is possible to fully decouple the link from the joint mechanism by a switch-and-hold clutch and simultaneously keep the elastic energy stored. We show that with this novel paradigm, it is not only possible to reach the same optimal performance as with power-equivalent variable stiffness actuation, but even directly control the energy transfer timing. This is a major step forward compared to previous optimal control approaches, which rely on optimizing the full time-series control input.
△ Less
Submitted 30 December, 2022;
originally announced December 2022.
-
Informed Circular Fields for Global Reactive Obstacle Avoidance of Robotic Manipulators
Authors:
Marvin Becker,
Philipp Caspers,
Tom Hattendorf,
Torsten Lilge,
Sami Haddadin,
Matthias A. Müller
Abstract:
In this paper a global reactive motion planning framework for robotic manipulators in complex dynamic environments is presented. In particular, the circular field predictions (CFP) planner from Becker et al. (2021) is extended to ensure obstacle avoidance of the whole structure of a robotic manipulator. Towards this end, a motion planning framework is developed that leverages global information ab…
▽ More
In this paper a global reactive motion planning framework for robotic manipulators in complex dynamic environments is presented. In particular, the circular field predictions (CFP) planner from Becker et al. (2021) is extended to ensure obstacle avoidance of the whole structure of a robotic manipulator. Towards this end, a motion planning framework is developed that leverages global information about promising avoidance directions from arbitrary configuration space motion planners, resulting in improved global trajectories while reactively avoiding dynamic obstacles and decreasing the required computational power. The resulting motion planning framework is tested in multiple simulations with complex and dynamic obstacles and demonstrates great potential compared to existing motion planning approaches.
△ Less
Submitted 4 August, 2023; v1 submitted 12 December, 2022;
originally announced December 2022.
-
Motion Planning using Reactive Circular Fields: A 2D Analysis of Collision Avoidance and Goal Convergence
Authors:
Marvin Becker,
Johannes Köhler,
Sami Haddadin,
Matthias A. Müller
Abstract:
Recently, many reactive trajectory planning approaches were suggested in the literature because of their inherent immediate adaption in the ever more demanding cluttered and unpredictable environments of robotic systems. However, typically those approaches are only locally reactive without considering global path planning and no guarantees for simultaneous collision avoidance and goal convergence…
▽ More
Recently, many reactive trajectory planning approaches were suggested in the literature because of their inherent immediate adaption in the ever more demanding cluttered and unpredictable environments of robotic systems. However, typically those approaches are only locally reactive without considering global path planning and no guarantees for simultaneous collision avoidance and goal convergence can be given. In this paper, we study a recently developed circular field (CF)-based motion planner that combines local reactive control with global trajectory generation by adapting an artificial magnetic field such that multiple trajectories around obstacles can be evaluated. In particular, we provide a mathematically rigorous analysis of this planner in a planar environment to ensure safe motion of the controlled robot. Contrary to existing results, the derived collision avoidance analysis covers the entire CF motion planning algorithm including attractive forces for goal convergence and is not limited to a specific choice of the rotation field, i.e., our guarantees are not limited to a specific potentially suboptimal trajectory. Our Lyapunov-type collision avoidance analysis is based on the definition of an (equivalent) two-dimensional auxiliary system, which enables us to provide tight, if and only if conditions for the case of a collision with point obstacles. Furthermore, we show how this analysis naturally extends to multiple obstacles and we specify sufficient conditions for goal convergence. Finally, we provide a challenging simulation scenario with multiple non-convex point cloud obstacles and demonstrate collision avoidance and goal convergence.
△ Less
Submitted 3 November, 2023; v1 submitted 28 October, 2022;
originally announced October 2022.
-
A Solution to Slosh-free Robot Trajectory Optimization
Authors:
Rafael I. Cabral Muchacho,
Riddhiman Laha,
Luis F. C. Figueredo,
Sami Haddadin
Abstract:
This paper is about fast slosh free fluid transportation. Existing approaches are either computationally heavy or only suitable for specific robots and container shapes. We model the end effector as a point mass suspended by a spherical pendulum and study the requirements for slosh free motion and the validity of the point mass model. In this approach, slosh free trajectories are generated by cont…
▽ More
This paper is about fast slosh free fluid transportation. Existing approaches are either computationally heavy or only suitable for specific robots and container shapes. We model the end effector as a point mass suspended by a spherical pendulum and study the requirements for slosh free motion and the validity of the point mass model. In this approach, slosh free trajectories are generated by controlling the pendulum's pivot and simulating the motion of the point mass. We cast the trajectory optimization problem as a quadratic program; this strategy can be used to obtain valid control inputs. Through simulations and experiments on a 7 DoF Franka Emika Panda robot we validate the effectiveness of the proposed approach.
△ Less
Submitted 23 October, 2022;
originally announced October 2022.
-
Towards Task-Specific Modular Gripper Fingers: Automatic Production of Fingertip Mechanics
Authors:
Johannes Ringwald,
Samuel Schneider,
Lingyun Chen,
Dennis Knobbe,
Lars Johannsmeier,
Abdalla Swikir,
Sami Haddadin
Abstract:
The number of sequential tasks a single gripper can perform is significantly limited by its design. In many cases, changing the gripper fingers is required to successfully conduct multiple consecutive tasks. For this reason, several robotic tool change systems have been introduced that allow an automatic changing of the entire end-effector. However, many situations require only the modification or…
▽ More
The number of sequential tasks a single gripper can perform is significantly limited by its design. In many cases, changing the gripper fingers is required to successfully conduct multiple consecutive tasks. For this reason, several robotic tool change systems have been introduced that allow an automatic changing of the entire end-effector. However, many situations require only the modification or the change of the fingertip, making the exchange of the entire gripper uneconomic. In this paper, we introduce a paradigm for automatic task-specific fingertip production. The setup used in the proposed framework consists of a production and task execution unit, containing a robotic manipulator, and two 3D printers - autonomously producing the gripper fingers. It also consists of a second manipulator that uses a quick-exchange mechanism to pick up the printed fingertips and evaluates grip** performance. The setup is experimentally validated by conducting automatic production of three different fingertips and executing graspstability tests as well as multiple pick- and insertion tasks, with and without position offsets - using these fingertips. The proposed paradigm, indeed, goes beyond fingertip production and serves as a foundation for a fully automatic fingertip design, production and application pipeline - potentially improving manufacturing flexibility and representing a new production paradigm: tactile 3D manufacturing.
△ Less
Submitted 18 October, 2022;
originally announced October 2022.
-
Human-to-Robot Manipulability Domain Adaptation with Parallel Transport and Manifold-Aware ICP
Authors:
Anna Reithmeir,
Luis Figueredo,
Sami Haddadin
Abstract:
Manipulability ellipsoids efficiently capture the human pose and reveal information about the task at hand. Their use in task-dependent robot teaching - particularly their transfer from a teacher to a learner - can advance emulation of human-like motion. Although in recent literature focus is shifted towards manipulability transfer between two robots, the adaptation to the capabilities of the othe…
▽ More
Manipulability ellipsoids efficiently capture the human pose and reveal information about the task at hand. Their use in task-dependent robot teaching - particularly their transfer from a teacher to a learner - can advance emulation of human-like motion. Although in recent literature focus is shifted towards manipulability transfer between two robots, the adaptation to the capabilities of the other kinematic system is to date not addressed and research in transfer from human to robot is still in its infancy. This work presents a novel manipulability domain adaptation method for the transfer of manipulability information to the domain of another kinematic system. As manipulability matrices/ellipsoids are symmetric positive-definite (SPD) they can be viewed as points on the Riemannian manifold of SPD matrices. We are the first to address the problem of manipulability transfer from the perspective of point cloud registration. We propose a manifold-aware Iterative Closest Point algorithm (ICP) with parallel transport initialization. Furthermore, we introduce a correspondence matching heuristic for manipulability ellipsoids based on inherent geometric features. We confirm our method in simulation experiments with 2-DoF manipulators as well as 7-DoF models representing the human-arm kinematics.
△ Less
Submitted 16 August, 2022;
originally announced August 2022.
-
LATTE: LAnguage Trajectory TransformEr
Authors:
Arthur Bucker,
Luis Figueredo,
Sami Haddadin,
Ashish Kapoor,
Shuang Ma,
Sai Vemprala,
Rogerio Bonatti
Abstract:
Natural language is one of the most intuitive ways to express human intent. However, translating instructions and commands towards robotic motion generation and deployment in the real world is far from being an easy task. The challenge of combining a robot's inherent low-level geometric and kinodynamic constraints with a human's high-level semantic instructions traditionally is solved using task-s…
▽ More
Natural language is one of the most intuitive ways to express human intent. However, translating instructions and commands towards robotic motion generation and deployment in the real world is far from being an easy task. The challenge of combining a robot's inherent low-level geometric and kinodynamic constraints with a human's high-level semantic instructions traditionally is solved using task-specific solutions with little generalizability between hardware platforms, often with the use of static sets of target actions and commands. This work instead proposes a flexible language-based framework that allows a user to modify generic robotic trajectories. Our method leverages pre-trained language models (BERT and CLIP) to encode the user's intent and target objects directly from a free-form text input and scene images, fuses geometrical features generated by a transformer encoder network, and finally outputs trajectories using a transformer decoder, without the need of priors related to the task or robot information. We significantly extend our own previous work presented in Bucker et al. by expanding the trajectory parametrization space to 3D and velocity as opposed to just XY movements. In addition, we now train the model to use actual images of the objects in the scene for context (as opposed to textual descriptions), and we evaluate the system in a diverse set of scenarios beyond manipulation, such as aerial and legged robots. Our simulated and real-life experiments demonstrate that our transformer model can successfully follow human intent, modifying the shape and speed of trajectories within multiple environments. Codebase available at: https://github.com/arthurfenderbucker/LaTTe-Language-Trajectory-TransformEr.git
△ Less
Submitted 16 September, 2022; v1 submitted 4 August, 2022;
originally announced August 2022.
-
Resha** Robot Trajectories Using Natural Language Commands: A Study of Multi-Modal Data Alignment Using Transformers
Authors:
Arthur Bucker,
Luis Figueredo,
Sami Haddadin,
Ashish Kapoor,
Shuang Ma,
Rogerio Bonatti
Abstract:
Natural language is the most intuitive medium for us to interact with other people when expressing commands and instructions. However, using language is seldom an easy task when humans need to express their intent towards robots, since most of the current language interfaces require rigid templates with a static set of action targets and commands. In this work, we provide a flexible language-based…
▽ More
Natural language is the most intuitive medium for us to interact with other people when expressing commands and instructions. However, using language is seldom an easy task when humans need to express their intent towards robots, since most of the current language interfaces require rigid templates with a static set of action targets and commands. In this work, we provide a flexible language-based interface for human-robot collaboration, which allows a user to reshape existing trajectories for an autonomous agent. We take advantage of recent advancements in the field of large language models (BERT and CLIP) to encode the user command, and then combine these features with trajectory information using multi-modal attention transformers. We train the model using imitation learning over a dataset containing robot trajectories modified by language commands, and treat the trajectory generation process as a sequence prediction problem, analogously to how language generation architectures operate. We evaluate the system in multiple simulated trajectory scenarios, and show a significant performance increase of our model over baseline approaches. In addition, our real-world experiments with a robot arm show that users significantly prefer our natural language interface over traditional methods such as kinesthetic teaching or cost-function programming. Our study shows how the field of robotics can take advantage of large pre-trained language models towards creating more intuitive interfaces between robots and machines. Project webpage: https://arthurfenderbucker.github.io/NL_trajectory_reshaper/
△ Less
Submitted 24 March, 2022;
originally announced March 2022.
-
Coordinate Invariant User-Guided Constrained Path Planning with Reactive Rapidly Expanding Plane-Oriented Esca** Trees
Authors:
Riddhiman Laha,
Ruiai Sun,
Wenxi Wu,
Dasharadhan Mahalingam,
Nilanjan Chakraborty,
Luis F. C. Figueredo,
Sami Haddadin
Abstract:
As collaborative robots move closer to human environments, motion generation and reactive planning strategies that allow for elaborate task execution with minimal easy-to-implement guidance whilst co** with changes in the environment is of paramount importance. In this paper, we present a novel approach for generating real-time motion plans for point-to-point tasks using a single successful huma…
▽ More
As collaborative robots move closer to human environments, motion generation and reactive planning strategies that allow for elaborate task execution with minimal easy-to-implement guidance whilst co** with changes in the environment is of paramount importance. In this paper, we present a novel approach for generating real-time motion plans for point-to-point tasks using a single successful human demonstration. Our approach is based on screw linear interpolation,which allows us to respect the underlying geometric constraints that characterize the task and are implicitly present in the demonstration. We also integrate an original reactive collision avoidance approach with our planner. We present extensive experimental results to demonstrate that with our approach,by using a single demonstration of moving one block, we can generate motion plans for complex tasks like stacking multiple blocks (in a dynamic environment). Analogous generalization abilities are also shown for tasks like pouring and loading shelves. For the pouring task, we also show that a demonstration given for one-armed pouring can be used for planning pouring with a dual-armed manipulator of different kinematic structure.
△ Less
Submitted 20 March, 2022;
originally announced March 2022.
-
ISO/TS 15066: How Different Interpretations Affect Risk Assessment
Authors:
Robin Jeanne Kirschner,
Nico Mansfeld,
Saeed Abdolshah,
Sami Haddadin
Abstract:
The current technical specification ISO/TS15066:2016(E) for safe human-robot interaction contains logically conflicting definitions for the contact between human and robot. This may result in different interpretations for the contact classification and thus no unique outcome can be expected, which may even cause a risk to the human. In previous work, we showed a first set of implications. This pap…
▽ More
The current technical specification ISO/TS15066:2016(E) for safe human-robot interaction contains logically conflicting definitions for the contact between human and robot. This may result in different interpretations for the contact classification and thus no unique outcome can be expected, which may even cause a risk to the human. In previous work, we showed a first set of implications. This paper addresses the possible interpretations of a collision scenario as a result of the varying interpretations for a risk assessment. With an experiment including four commercially available robot systems we demonstrate the procedure of the risk assessment following the different interpretations of the TS. The results indicate possible incorrect use of the technical specification, which we believe needs to be resolved in future revisions. For this, we suggest tools in form of a decision tree and constrained collision force maps, which enable a simple, unambiguous risk assessment for HRI.
△ Less
Submitted 5 March, 2022;
originally announced March 2022.
-
Towards Remote Robotic Competitions: An Internet-Connected Task Board and Dashboard
Authors:
Peter So,
Jonas Wittmann,
Patrick Ruhkamp,
Andriy Sarabakha,
Sami Haddadin
Abstract:
In this work we present a platform to assess robot platform skills using an internet-of-things (IoT) task board device to aggregate performances across remote sites. We demonstrate a concept for a modular, scale-able device and web dashboard enabling remote competitions as an alternative to in-person robot competitions. We share data from nine robot platforms located across four continents in thre…
▽ More
In this work we present a platform to assess robot platform skills using an internet-of-things (IoT) task board device to aggregate performances across remote sites. We demonstrate a concept for a modular, scale-able device and web dashboard enabling remote competitions as an alternative to in-person robot competitions. We share data from nine robot platforms located across four continents in three manipulation task categories of object localization, object insertion, and component disassembly through an organized international robot competition - the Robothon Grand Challenge. This paper discusses the design of an electronic task board, the strategies implemented by the top-performing teams and compares their results with a benchmark solution to the presented task board. Through this platform, we demonstrate fully remote, online competitions can generate innovative robotic solutions and tested a tool for measuring remote performances. Using the open-sourced task board code and design files, the reader can reproduce the benchmark solution or configure the platform for their own use case and share their results transparently without transporting their robot platform.
△ Less
Submitted 24 January, 2022;
originally announced January 2022.
-
Learning Robotic Manipulation Skills Using an Adaptive Force-Impedance Action Space
Authors:
Maximilian Ulmer,
Elie Aljalbout,
Sascha Schwarz,
Sami Haddadin
Abstract:
Intelligent agents must be able to think fast and slow to perform elaborate manipulation tasks. Reinforcement Learning (RL) has led to many promising results on a range of challenging decision-making tasks. However, in real-world robotics, these methods still struggle, as they require large amounts of expensive interactions and have slow feedback loops. On the other hand, fast human-like adaptive…
▽ More
Intelligent agents must be able to think fast and slow to perform elaborate manipulation tasks. Reinforcement Learning (RL) has led to many promising results on a range of challenging decision-making tasks. However, in real-world robotics, these methods still struggle, as they require large amounts of expensive interactions and have slow feedback loops. On the other hand, fast human-like adaptive control methods can optimize complex robotic interactions, yet fail to integrate multimodal feedback needed for unstructured tasks. In this work, we propose to factor the learning problem in a hierarchical learning and adaption architecture to get the best of both worlds. The framework consists of two components, a slow reinforcement learning policy optimizing the task strategy given multimodal observations, and a fast, real-time adaptive control policy continuously optimizing the motion, stability, and effort of the manipulator. We combine these components through a bio-inspired action space that we call AFORCE. We demonstrate the new action space on a contact-rich manipulation task on real hardware and evaluate its performance on three simulated manipulation tasks. Our experiments show that AFORCE drastically improves sample efficiency while reducing energy consumption and improving safety.
△ Less
Submitted 20 October, 2021; v1 submitted 19 October, 2021;
originally announced October 2021.
-
Expectable Motion Unit: Avoiding Hazards From Human Involuntary Motions in Human-Robot Interaction
Authors:
Robin Jeanne Kirschner,
Henning Mayer,
Lisa Burr,
Nico Mansfeld,
Saeed Abdolshah,
Sami Haddadin
Abstract:
In robotics, many control and planning schemes have been developed to ensure human physical safety in human-robot interaction. The human psychological state and the expectation towards the robot, however, are typically neglected. Even if the robot behaviour is regarded as biomechanically safe, humans may still react with a rapid involuntary motion (IM) caused by a startle or surprise. Such sudden,…
▽ More
In robotics, many control and planning schemes have been developed to ensure human physical safety in human-robot interaction. The human psychological state and the expectation towards the robot, however, are typically neglected. Even if the robot behaviour is regarded as biomechanically safe, humans may still react with a rapid involuntary motion (IM) caused by a startle or surprise. Such sudden, uncontrolled motions can jeopardize safety and should be prevented by any means. In this letter, we propose the Expectable Motion Unit (EMU), which ensures that a certain probability of IM occurrence is not exceeded in a typical HRI setting. Based on a model of IM occurrence generated through an experiment with 29 participants, we establish the map** between robot velocity, robot-human distance, and the relative frequency of IM occurrence. This map** is processed towards a real-time capable robot motion generator that limits the robot velocity during task execution if necessary. The EMU is combined in a holistic safety framework that integrates both the physical and psychological safety knowledge. A validation experiment showed that the EMU successfully avoids human IM in five out of six cases.
△ Less
Submitted 4 April, 2024; v1 submitted 15 September, 2021;
originally announced September 2021.
-
SyReNets: Symbolic Residual Neural Networks
Authors:
Carlos Magno C. O. Valle,
Sami Haddadin
Abstract:
Despite successful seminal works on passive systems in the literature, learning free-form physical laws for controlled dynamical systems given experimental data is still an open problem. For decades, symbolic mathematical equations and system identification were the golden standards. Unfortunately, a set of assumptions about the properties of the underlying system is required, which makes the mode…
▽ More
Despite successful seminal works on passive systems in the literature, learning free-form physical laws for controlled dynamical systems given experimental data is still an open problem. For decades, symbolic mathematical equations and system identification were the golden standards. Unfortunately, a set of assumptions about the properties of the underlying system is required, which makes the model very rigid and unable to adapt to unforeseen changes in the physical system. Neural networks, on the other hand, are known universal function approximators but are prone to over-fit, limited accuracy, and bias problems, which makes them alone unreliable candidates for such tasks. In this paper, we propose SyReNets, an approach that leverages neural networks for learning symbolic relations to accurately describe dynamic physical systems from data. It explores a sequence of symbolic layers that build, in a residual manner, mathematical relations that describes a given desired output from input variables. We apply it to learn the symbolic equation that describes the Lagrangian of a given physical system. We do this by only observing random samples of position, velocity, and acceleration as input and torque as output. Therefore, using the Lagrangian as a latent representation from which we derive torque using the Euler-Lagrange equations. The approach is evaluated using a simulated controlled double pendulum and compared with neural networks, genetic programming, and traditional system identification. The results demonstrate that, compared to neural networks and genetic programming, SyReNets converges to representations that are more accurate and precise throughout the state space. Despite having slower convergence than traditional system identification, similar to neural networks, the approach remains flexible enough to adapt to an unforeseen change in the physical system structure.
△ Less
Submitted 29 May, 2021;
originally announced May 2021.
-
Learning Vision-based Reactive Policies for Obstacle Avoidance
Authors:
Elie Aljalbout,
Ji Chen,
Konstantin Ritt,
Maximilian Ulmer,
Sami Haddadin
Abstract:
In this paper, we address the problem of vision-based obstacle avoidance for robotic manipulators. This topic poses challenges for both perception and motion generation. While most work in the field aims at improving one of those aspects, we provide a unified framework for approaching this problem. The main goal of this framework is to connect perception and motion by identifying the relationship…
▽ More
In this paper, we address the problem of vision-based obstacle avoidance for robotic manipulators. This topic poses challenges for both perception and motion generation. While most work in the field aims at improving one of those aspects, we provide a unified framework for approaching this problem. The main goal of this framework is to connect perception and motion by identifying the relationship between the visual input and the corresponding motion representation. To this end, we propose a method for learning reactive obstacle avoidance policies. We evaluate our method on goal-reaching tasks for single and multiple obstacles scenarios. We show the ability of the proposed method to efficiently learn stable obstacle avoidance strategies at a high success rate, while maintaining closed-loop responsiveness required for critical applications like human-robot interaction.
△ Less
Submitted 30 October, 2020;
originally announced October 2020.
-
How to Make Deep RL Work in Practice
Authors:
Nirnai Rao,
Elie Aljalbout,
Axel Sauer,
Sami Haddadin
Abstract:
In recent years, challenging control problems became solvable with deep reinforcement learning (RL). To be able to use RL for large-scale real-world applications, a certain degree of reliability in their performance is necessary. Reported results of state-of-the-art algorithms are often difficult to reproduce. One reason for this is that certain implementation details influence the performance sig…
▽ More
In recent years, challenging control problems became solvable with deep reinforcement learning (RL). To be able to use RL for large-scale real-world applications, a certain degree of reliability in their performance is necessary. Reported results of state-of-the-art algorithms are often difficult to reproduce. One reason for this is that certain implementation details influence the performance significantly. Commonly, these details are not highlighted as important techniques to achieve state-of-the-art performance. Additionally, techniques from supervised learning are often used by default but influence the algorithms in a reinforcement learning setting in different and not well-understood ways. In this paper, we investigate the influence of certain initialization, input normalization, and adaptive learning techniques on the performance of state-of-the-art RL algorithms. We make suggestions which of those techniques to use by default and highlight areas that could benefit from a solution specifically tailored to RL.
△ Less
Submitted 10 November, 2020; v1 submitted 25 October, 2020;
originally announced October 2020.
-
Tracking Holistic Object Representations
Authors:
Axel Sauer,
Elie Aljalbout,
Sami Haddadin
Abstract:
Recent advances in visual tracking are based on siamese feature extractors and template matching. For this category of trackers, latest research focuses on better feature embeddings and similarity measures. In this work, we focus on building holistic object representations for tracking. We propose a framework that is designed to be used on top of previous trackers without any need for further trai…
▽ More
Recent advances in visual tracking are based on siamese feature extractors and template matching. For this category of trackers, latest research focuses on better feature embeddings and similarity measures. In this work, we focus on building holistic object representations for tracking. We propose a framework that is designed to be used on top of previous trackers without any need for further training of the siamese network. The framework leverages the idea of obtaining additional object templates during the tracking process. Since the number of stored templates is limited, our method only keeps the most diverse ones. We achieve this by providing a new diversity measure in the space of siamese features. The obtained representation contains information beyond the ground truth object location provided to the system. It is then useful for tracking itself but also for further tasks which require a visual understanding of objects. Strong empirical results on tracking benchmarks indicate that our method can improve the performance and robustness of the underlying trackers while barely reducing their speed. In addition, our method is able to match current state-of-the-art results, while using a simpler and older network architecture and running three times faster.
△ Less
Submitted 6 August, 2019; v1 submitted 21 July, 2019;
originally announced July 2019.
-
Simultaneous Contact and Aerodynamic Force Estimation (s-CAFE) for Aerial Robots
Authors:
Teodor Tomić,
Philipp Lutz,
Korbinian Schmid,
Andrew Mathers,
Sami Haddadin
Abstract:
In this paper, we consider the problem of multirotor flying robots physically interacting with the environment under wind influence. The result are the first algorithms for simultaneous online estimation of contact and aerodynamic wrenches acting on the robot based on real-world data, without the need for dedicated sensors. For this purpose, we investigate two model-based techniques for discrimina…
▽ More
In this paper, we consider the problem of multirotor flying robots physically interacting with the environment under wind influence. The result are the first algorithms for simultaneous online estimation of contact and aerodynamic wrenches acting on the robot based on real-world data, without the need for dedicated sensors. For this purpose, we investigate two model-based techniques for discriminating between aerodynamic and interaction forces. The first technique is based on aerodynamic and contact torque models, and uses the external force to estimate wind speed. Contacts are then detected based on the residual between estimated external torque and expected (modeled) aerodynamic torque. Upon detecting contact, wind speed is assumed to change very slowly. From the estimated interaction wrench, we are also able to determine the contact location. This is embedded into a particle filter framework to further improve contact location estimation. The second algorithm uses the propeller aerodynamic power and angular speed as measured by the speed controllers to obtain an estimate of the airspeed. An aerodynamics model is then used to determine the aerodynamic wrench. Both methods rely on accurate aerodynamics models. Therefore, we evaluate data-driven and physics based models as well as offline system identification for flying robots. For obtaining ground truth data we performed autonomous flights in a 3D wind tunnel. Using this data, aerodynamic model selection, parameter identification, and discrimination between aerodynamic and contact forces could be done. Finally, the developed methods could serve as useful estimators for interaction control schemes with simultaneous compensation of wind disturbances.
△ Less
Submitted 30 October, 2018;
originally announced October 2018.
-
A Framework for Robot Manipulation: Skill Formalism, Meta Learning and Adaptive Control
Authors:
Lars Johannsmeier,
Malkin Gerchow,
Sami Haddadin
Abstract:
In this paper we introduce a novel framework for expressing and learning force-sensitive robot manipulation skills. It is based on a formalism that extends our previous work on adaptive impedance control with meta parameter learning and compatible skill specifications. This way the system is also able to make use of abstract expert knowledge by incorporating process descriptions and quality evalua…
▽ More
In this paper we introduce a novel framework for expressing and learning force-sensitive robot manipulation skills. It is based on a formalism that extends our previous work on adaptive impedance control with meta parameter learning and compatible skill specifications. This way the system is also able to make use of abstract expert knowledge by incorporating process descriptions and quality evaluation metrics. We evaluate various state-of-the-art schemes for the meta parameter learning and experimentally compare selected ones. Our results clearly indicate that the combination of our adaptive impedance controller with a carefully defined skill formalism significantly reduces the complexity of manipulation tasks even for learning peg-in-hole with submillimeter industrial tolerances. Overall, the considered system is able to learn variations of this skill in under 20 minutes. In fact, experimentally the system was able to perform the learned tasks faster than humans, leading to the first learning-based solution of complex assembly at such real-world performance.
△ Less
Submitted 22 May, 2018;
originally announced May 2018.
