Showing 1–50 of 65 results for author: Mastrogiovanni, F

Robotic in-hand manipulation with relaxed optimization

Authors: Ali Hammoud, Valerio Belcamino, Quentin Huet, Alessandro Carfì, Mahdi Khoramshahi, Veronique Perdereau, Fulvio Mastrogiovanni

Abstract: Dexterous in-hand manipulation is a unique and valuable human skill requiring sophisticated sensorimotor interaction with the environment while respecting stability constraints. Satisfying these constraints with generated motions is essential for a robotic platform to achieve reliable in-hand manipulation skills. Explicitly modelling these constraints can be challenging, but they can be implicitly… ▽ More Dexterous in-hand manipulation is a unique and valuable human skill requiring sophisticated sensorimotor interaction with the environment while respecting stability constraints. Satisfying these constraints with generated motions is essential for a robotic platform to achieve reliable in-hand manipulation skills. Explicitly modelling these constraints can be challenging, but they can be implicitly modelled and learned through experience or human demonstrations. We propose a learning and control approach based on dictionaries of motion primitives generated from human demonstrations. To achieve this, we defined an optimization process that combines motion primitives to generate robot fingertip trajectories for moving an object from an initial to a desired final pose. Based on our experiments, our approach allows a robotic hand to handle objects like humans, adhering to stability constraints without requiring explicit formalization. In other words, the proposed motion primitive dictionaries learn and implicitly embed the constraints crucial to the in-hand manipulation task. △ Less

Submitted 7 June, 2024; originally announced June 2024.

Comments: 9 pages, 6 pictures, ROMAN 2024

A Modular Framework for Flexible Planning in Human-Robot Collaboration

Authors: Valerio Belcamino, Mariya Kilina, Linda Lastrico, Alessandro Carfì, Fulvio Mastrogiovanni

Abstract: This paper presents a comprehensive framework to enhance Human-Robot Collaboration (HRC) in real-world scenarios. It introduces a formalism to model articulated tasks, requiring cooperation between two agents, through a smaller set of primitives. Our implementation leverages Hierarchical Task Networks (HTN) planning and a modular multisensory perception pipeline, which includes vision, human activ… ▽ More This paper presents a comprehensive framework to enhance Human-Robot Collaboration (HRC) in real-world scenarios. It introduces a formalism to model articulated tasks, requiring cooperation between two agents, through a smaller set of primitives. Our implementation leverages Hierarchical Task Networks (HTN) planning and a modular multisensory perception pipeline, which includes vision, human activity recognition, and tactile sensing. To showcase the system's scalability, we present an experimental scenario where two humans alternate in collaborating with a Baxter robot to assemble four pieces of furniture with variable components. This integration highlights promising advancements in HRC, suggesting a scalable approach for complex, cooperative tasks across diverse applications. △ Less

Submitted 7 June, 2024; originally announced June 2024.

Comments: 9 pages, 5 figures, ROMAN 2024

A Systematic Review on Custom Data Gloves

Authors: Valerio Belcamino, Alessandro Carfì, Fulvio Mastrogiovanni

Abstract: Hands are a fundamental tool humans use to interact with the environment and objects. Through hand motions, we can obtain information about the shape and materials of the surfaces we touch, modify our surroundings by interacting with objects, manipulate objects and tools, or communicate with other people by leveraging the power of gestures. For these reasons, sensorized gloves, which can collect i… ▽ More Hands are a fundamental tool humans use to interact with the environment and objects. Through hand motions, we can obtain information about the shape and materials of the surfaces we touch, modify our surroundings by interacting with objects, manipulate objects and tools, or communicate with other people by leveraging the power of gestures. For these reasons, sensorized gloves, which can collect information about hand motions and interactions, have been of interest since the 1980s in various fields, such as Human-Machine Interaction (HMI) and the analysis and control of human motions. Over the last 40 years, research in this field explored different technological approaches and contributed to the popularity of wearable custom and commercial products targeting hand sensorization. Despite a positive research trend, these instruments are not widespread yet outside research environments and devices aimed at research are often ad hoc solutions with a low chance of being reused. This paper aims to provide a systematic literature review for custom gloves to analyze their main characteristics and critical issues, from the type and number of sensors to the limitations due to device encumbrance. The collection of this information lays the foundation for a standardization process necessary for future breakthroughs in this research field. △ Less

Submitted 24 May, 2024; originally announced May 2024.

Comments: 17 pages, 12 images

Gaze-Based Intention Recognition for Human-Robot Collaboration

Authors: Valerio Belcamino, Miwa Takase, Mariya Kilina, Alessandro Carfì, Akira Shimada, Sota Shimizu, Fulvio Mastrogiovanni

Abstract: This work aims to tackle the intent recognition problem in Human-Robot Collaborative assembly scenarios. Precisely, we consider an interactive assembly of a wooden stool where the robot fetches the pieces in the correct order and the human builds the parts following the instruction manual. The intent recognition is limited to the idle state estimation and it is needed to ensure a better synchroniz… ▽ More This work aims to tackle the intent recognition problem in Human-Robot Collaborative assembly scenarios. Precisely, we consider an interactive assembly of a wooden stool where the robot fetches the pieces in the correct order and the human builds the parts following the instruction manual. The intent recognition is limited to the idle state estimation and it is needed to ensure a better synchronization between the two agents. We carried out a comparison between two distinct solutions involving wearable sensors and eye tracking integrated into the perception pipeline of a flexible planning architecture based on Hierarchical Task Networks. At runtime, the wearable sensing module exploits the raw measurements from four 9-axis Inertial Measurement Units positioned on the wrists and hands of the user as an input for a Long Short-Term Memory Network. On the other hand, the eye tracking relies on a Head Mounted Display and Unreal Engine. We tested the effectiveness of the two approaches with 10 participants, each of whom explored both options in alternate order. We collected explicit metrics about the attractiveness and efficiency of the two techniques through User Experience Questionnaires as well as implicit criteria regarding the classification time and the overall assembly time. The results of our work show that the two methods can reach comparable performances both in terms of effectiveness and user preference. Future development could aim at joining the two approaches two allow the recognition of more complex activities and to anticipate the user actions. △ Less

Submitted 13 May, 2024; originally announced May 2024.

Comments: 5 pages, 4 figures, AVI2024 conference

Incremental Bootstrap** and Classification of Structured Scenes in a Fuzzy Ontology

Authors: Luca Buoncompagni, Fulvio Mastrogiovanni

Abstract: We foresee robots that bootstrap knowledge representations and use them for classifying relevant situations and making decisions based on future observations. Particularly for assistive robots, the bootstrap** mechanism might be supervised by humans who should not repeat a training phase several times and should be able to refine the taught representation. We consider robots that bootstrap struc… ▽ More We foresee robots that bootstrap knowledge representations and use them for classifying relevant situations and making decisions based on future observations. Particularly for assistive robots, the bootstrap** mechanism might be supervised by humans who should not repeat a training phase several times and should be able to refine the taught representation. We consider robots that bootstrap structured representations to classify some intelligible categories. Such a structure should be incrementally bootstrapped, i.e., without invalidating the identified category models when a new additional category is considered. To tackle this scenario, we presented the Scene Identification and Tagging (SIT) algorithm, which bootstraps structured knowledge representation in a crisp OWL-DL ontology. Over time, SIT bootstraps a graph representing scenes, sub-scenes and similar scenes. Then, SIT can classify new scenes within the bootstrapped graph through logic-based reasoning. However, SIT has issues with sensory data because its crisp implementation is not robust to perception noises. This paper presents a reformulation of SIT within the fuzzy domain, which exploits a fuzzy DL ontology to overcome the robustness issues. By comparing the performances of fuzzy and crisp implementations of SIT, we show that fuzzy SIT is robust, preserves the properties of its crisp formulation, and enhances the bootstrapped representations. On the contrary, the fuzzy implementation of SIT leads to less intelligible knowledge representations than the one bootstrapped in the crisp domain. △ Less

Submitted 17 April, 2024; originally announced April 2024.

MSC Class: 68T40 (Primary) 68T30; 68T27; 68T37; 03B52 (Secondary) ACM Class: I.2.4; I.2.6; I.2.3; I.2.9; I.2.10

Learning Symbolic Task Representation from a Human-Led Demonstration: A Memory to Store, Retrieve, Consolidate, and Forget Experiences

Authors: Luca Buoncompagni, Fulvio Mastrogiovanni

Abstract: We present a symbolic learning framework inspired by cognitive-like memory functionalities (i.e., storing, retrieving, consolidating and forgetting) to generate task representations to support high-level task planning and knowledge bootstrap**. We address a scenario involving a non-expert human, who performs a single task demonstration, and a robot, which online learns structured knowledge to re… ▽ More We present a symbolic learning framework inspired by cognitive-like memory functionalities (i.e., storing, retrieving, consolidating and forgetting) to generate task representations to support high-level task planning and knowledge bootstrap**. We address a scenario involving a non-expert human, who performs a single task demonstration, and a robot, which online learns structured knowledge to re-execute the task based on experiences, i.e., observations. We consider a one-shot learning process based on non-annotated data to store an intelligible representation of the task, which can be refined through interaction, e.g., via verbal or visual communication. Our general-purpose framework relies on fuzzy Description Logic, which has been used to extend the previously developed Scene Identification and Tagging algorithm. In this paper, we exploit such an algorithm to implement cognitive-like memory functionalities employing scores that rank memorised observations over time based on simple heuristics. Our main contribution is the formalisation of a framework that can be used to systematically investigate different heuristics for bootstrap** hierarchical knowledge representations based on robot observations. Through an illustrative assembly task scenario, the paper presents the performance of our framework to discuss its benefits and limitations. △ Less

Submitted 19 April, 2024; v1 submitted 16 April, 2024; originally announced April 2024.

MSC Class: 68T40 (Primary) 68T20; 68T27; 68T30; 68T37; 05C72; 68Q32 (Secondary) ACM Class: I.2.4; I.2.6; E.1

OWLOOP: Interfaces for Map** OWL Axioms into OOP Hierarchies

Authors: Luca Buoncompagni, Fulvio Mastrogiovanni

Abstract: The paper tackles the issue of map** logic axioms formalised in the Ontology Web Language (OWL) within the Object-Oriented Programming (OOP) paradigm. The issues of map** OWL axioms hierarchies and OOP objects hierarchies are due to OWL-based reasoning algorithms, which might change an OWL hierarchy at runtime; instead, OOP hierarchies are usually defined as static structures. Although program… ▽ More The paper tackles the issue of map** logic axioms formalised in the Ontology Web Language (OWL) within the Object-Oriented Programming (OOP) paradigm. The issues of map** OWL axioms hierarchies and OOP objects hierarchies are due to OWL-based reasoning algorithms, which might change an OWL hierarchy at runtime; instead, OOP hierarchies are usually defined as static structures. Although programming paradigms based on reflection allow changing the OOP hierarchies at runtime and map** OWL axioms dynamically, there are no currently available mechanisms that do not limit the reasoning algorithms. Thus, the factory-based paradigm is typically used since it decouples the OWL and OOP hierarchies. However, the factory inhibits OOP polymorphism and introduces a paradigm shift with respect to widely accepted OOP paradigms. We present the OWLOOP API, which exploits the factory to not limit reasoning algorithms, and it provides novel OOP interfaces concerning the axioms in an ontology. OWLOOP is designed to limit the paradigm shift required for using ontologies while improving, through OOP-like polymorphism, the modularity of software architectures that exploit logic reasoning. The paper details our OWL to OOP map** mechanism, and it shows the benefits and limitations of OWLOOP through examples concerning a robot in a smart environment. △ Less

Submitted 19 April, 2024; v1 submitted 14 April, 2024; originally announced April 2024.

Comments: This manuscript details the implementation of the OWLOOP API. A simplified (and "citable") presentation of our API has been published in the SoftwareX Elsevier journal with the title "OWLOOP: A modular API to describe OWL axioms in OOP objects hierarchies" ( https://doi.org/10.1016/j.softx.2021.100952). The OWLOOP API repository is available at https://github.com/buoncubi/owloop

MSC Class: 68T27 (Primary) 68T30; 68N19; 68T40 (Secondary) ACM Class: D.2.11; D.1.5; D.1.6; E.2; I.2.4

A novel method to compute the contact surface area between an organ and cancer tissue

Authors: Alessandra Bulanti, Alessandro Carfì, Paolo Traverso, Carlo Terrone, Fulvio Mastrogiovanni

Abstract: With "contact surface area" (CSA) we refers to the area of contact between a tumor and an organ. This indicator has been identified as a predictive factor for surgical peri-operative parameters, particularly in the context of kidney cancer. However, state-of-the-art algorithms for computing the CSA rely on assumptions about the tumor shape and require manual human annotation. In this study, we int… ▽ More With "contact surface area" (CSA) we refers to the area of contact between a tumor and an organ. This indicator has been identified as a predictive factor for surgical peri-operative parameters, particularly in the context of kidney cancer. However, state-of-the-art algorithms for computing the CSA rely on assumptions about the tumor shape and require manual human annotation. In this study, we introduce an innovative method that relies on 3D reconstructions of tumors and organs to provide an accurate and objective estimate of the CSA. Our approach consists of a segmentation protocol for reconstructing organs and tumors from Computed Tomography (CT) images and an algorithm leveraging the reconstructed meshes to compute the CSA. With the aim to contributing to the literature with replicable results, we provide an open-source implementation of our algorithm, along with an easy-to-use graphical user interface to support its adoption and widespread use. We evaluated the accuracy of our method using both a synthetic dataset and reconstructions of 87 real tumor-organ pairs. △ Less

Submitted 19 January, 2024; originally announced February 2024.

A modular architecture for IMU-based data gloves

Authors: Alessandro Carfì, Mohamad Alameh, Valerio Belcamino, Fulvio Mastrogiovanni

Abstract: The flexibility and range of motion in human hands play a crucial role in human interaction with the environment and have been studied across different fields. Researchers explored various technological solutions for gathering information from the hands. These solutions include tracking hand motion through cameras or wearable sensors and using wearable sensors to measure the position and pressure… ▽ More The flexibility and range of motion in human hands play a crucial role in human interaction with the environment and have been studied across different fields. Researchers explored various technological solutions for gathering information from the hands. These solutions include tracking hand motion through cameras or wearable sensors and using wearable sensors to measure the position and pressure of contact points. Data gloves can collect both types of information by utilizing inertial measurement units, flex sensors, magnetic trackers for motion tracking, and force resistors or touch sensors for contact measurement. Although there are commercially available data gloves, researchers often create custom data gloves to achieve the desired flexibility and control over the hardware. However, the existing literature lacks standardization and the reuse of previously designed data gloves. As a result, many gloves with unclear characteristics exist, which makes replication challenging and negatively impacts the reproducibility of studies. This work proposes a modular, open hardware and software architecture for creating customized data gloves based on IMU technology. We also provide an architecture implementation along with an experimental protocol to evaluate device performance. △ Less

Submitted 24 January, 2024; originally announced January 2024.

Comments: Mechatronics Topic Group workshop at European Robotics Forum 2024

Digital Twins for Human-Robot Collaboration: A Future Perspective

Authors: Mohamad Shaaban, Alessandro Carfì, Fulvio Mastrogiovanni

Abstract: As collaborative robot (Cobot) adoption in many sectors grows, so does the interest in integrating digital twins in human-robot collaboration (HRC). Virtual representations of physical systems (PT) and assets, known as digital twins, can revolutionize human-robot collaboration by enabling real-time simulation, monitoring, and control. In this article, we present a review of the state-of-the-art an… ▽ More As collaborative robot (Cobot) adoption in many sectors grows, so does the interest in integrating digital twins in human-robot collaboration (HRC). Virtual representations of physical systems (PT) and assets, known as digital twins, can revolutionize human-robot collaboration by enabling real-time simulation, monitoring, and control. In this article, we present a review of the state-of-the-art and our perspective on the future of digital twins (DT) in human-robot collaboration. We argue that DT will be crucial in increasing the efficiency and effectiveness of these systems by presenting compelling evidence and a concise vision of the future of DT in human-robot collaboration, as well as insights into the possible advantages and challenges associated with their integration. △ Less

Submitted 4 November, 2023; originally announced November 2023.

Expressing and Inferring Action Carefulness in Human-to-Robot Handovers

Authors: Linda Lastrico, Nuno Ferreira Duarte, Alessandro Carfì, Francesco Rea, Alessandra Sciutti, Fulvio Mastrogiovanni, José Santos-Victor

Abstract: Implicit communication plays such a crucial role during social exchanges that it must be considered for a good experience in human-robot interaction. This work addresses implicit communication associated with the detection of physical properties, transport, and manipulation of objects. We propose an ecological approach to infer object characteristics from subtle modulations of the natural kinemati… ▽ More Implicit communication plays such a crucial role during social exchanges that it must be considered for a good experience in human-robot interaction. This work addresses implicit communication associated with the detection of physical properties, transport, and manipulation of objects. We propose an ecological approach to infer object characteristics from subtle modulations of the natural kinematics occurring during human object manipulation. Similarly, we take inspiration from human strategies to shape robot movements to be communicative of the object properties while pursuing the action goals. In a realistic HRI scenario, participants handed over cups - filled with water or empty - to a robotic manipulator that sorted them. We implemented an online classifier to differentiate careful/not careful human movements, associated with the cups' content. We compared our proposed "expressive" controller, which modulates the movements according to the cup filling, against a neutral motion controller. Results show that human kinematics is adjusted during the task, as a function of the cup content, even in reach-to-grasp motion. Moreover, the carefulness during the handover of full cups can be reliably inferred online, well before action completion. Finally, although questionnaires did not reveal explicit preferences from participants, the expressive robot condition improved task efficiency. △ Less

Submitted 30 September, 2023; originally announced October 2023.

Comments: PREPRINT VERSION Accepted for publication at the 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS2023)

Journal ref: 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

RICO-MR: An Open-Source Architecture for Robot Intent Communication through Mixed Reality

Authors: Simone Macciò, Mohamad Shaaban, Alessandro Carfì, Renato Zaccaria, Fulvio Mastrogiovanni

Abstract: This article presents an open-source architecture for conveying robots' intentions to human teammates using Mixed Reality and Head-Mounted Displays. The architecture has been developed focusing on its modularity and re-usability aspects. Both binaries and source code are available, enabling researchers and companies to adopt the proposed architecture as a standalone solution or to integrate it in… ▽ More This article presents an open-source architecture for conveying robots' intentions to human teammates using Mixed Reality and Head-Mounted Displays. The architecture has been developed focusing on its modularity and re-usability aspects. Both binaries and source code are available, enabling researchers and companies to adopt the proposed architecture as a standalone solution or to integrate it in more comprehensive implementations. Due to its scalability, the proposed architecture can be easily employed to develop shared Mixed Reality experiences involving multiple robots and human teammates in complex collaborative scenarios. △ Less

Submitted 9 September, 2023; originally announced September 2023.

Comments: 6 pages, 3 figures, accepted for publication in the proceedings of the 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)

In-hand manipulation planning using human motion dictionary

Authors: Ali Hammoud, Valerio Belcamino, Alessandro Carfi, Veronique Perdereau, Fulvio Mastrogiovanni

Abstract: Dexterous in-hand manipulation is a peculiar and useful human skill. This ability requires the coordination of many senses and hand motion to adhere to many constraints. These constraints vary and can be influenced by the object characteristics or the specific application. One of the key elements for a robotic platform to implement reliable inhand manipulation skills is to be able to integrate tho… ▽ More Dexterous in-hand manipulation is a peculiar and useful human skill. This ability requires the coordination of many senses and hand motion to adhere to many constraints. These constraints vary and can be influenced by the object characteristics or the specific application. One of the key elements for a robotic platform to implement reliable inhand manipulation skills is to be able to integrate those constraints in their motion generations. These constraints can be implicitly modelled, learned through experience or human demonstrations. We propose a method based on motion primitives dictionaries to learn and reproduce in-hand manipulation skills. In particular, we focused on fingertip motions during the manipulation, and we defined an optimization process to combine motion primitives to reach specific fingertip configurations. The results of this work show that the proposed approach can generate manipulation motion coherent with the human one and that manipulation constraints are inherited even without an explicit formalization. △ Less

Submitted 29 August, 2023; originally announced August 2023.

Journal ref: 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 927-933

Safe motion planning with environment uncertainty

Authors: Antony Thomas, Fulvio Mastrogiovanni, Marco Baglietto

Abstract: We present an approach for safe motion planning under robot state and environment (obstacle and landmark location) uncertainties. To this end, we first develop an approach that accounts for the landmark uncertainties during robot localization. Existing planning approaches assume that the landmark locations are well known or are known with little uncertainty. However, this might not be true in prac… ▽ More We present an approach for safe motion planning under robot state and environment (obstacle and landmark location) uncertainties. To this end, we first develop an approach that accounts for the landmark uncertainties during robot localization. Existing planning approaches assume that the landmark locations are well known or are known with little uncertainty. However, this might not be true in practice. Noisy sensors and imperfect motions compound to the errors originating from the estimate of environment features. Moreover, possible occlusions and dynamic objects in the environment render imperfect landmark estimation. Consequently, not considering this uncertainty can wrongly localize the robot, leading to inefficient plans. Our approach thus incorporates the landmark uncertainty within the Bayes filter estimation framework. We also analyze the effect of considering this uncertainty and delineate the conditions under which it can be ignored. Second, we extend the state-of-the-art by computing an exact expression for the collision probability under Gaussian distributed robot motion, perception and obstacle location uncertainties. We formulate the collision probability process as a quadratic form in random variables. Under Gaussian distribution assumptions, an exact expression for collision probability is thus obtained which is computable in real-time. In contrast, existing approaches approximate the collision probability using upper-bounds that can lead to overly conservative estimate and thereby suboptimal plans. We demonstrate and evaluate our approach using a theoretical example and simulations. We also present a comparison of our approach to different state-of-the-art methods. △ Less

Submitted 10 May, 2023; originally announced May 2023.

Comments: arXiv admin note: text overlap with arXiv:2101.11566

Journal ref: Robotics and Autonomous Systems, Volume 156, October 2022, 104203

Revisiting the Minimum Constraint Removal Problem in Mobile Robotics

Authors: Antony Thomas, Fulvio Mastrogiovanni, Marco Baglietto

Abstract: The minimum constraint removal problem seeks to find the minimum number of constraints, i.e., obstacles, that need to be removed to connect a start to a goal location with a collision-free path. This problem is NP-hard and has been studied in robotics, wireless sensing, and computational geometry. This work contributes to the existing literature by presenting and discussing two results. The first… ▽ More The minimum constraint removal problem seeks to find the minimum number of constraints, i.e., obstacles, that need to be removed to connect a start to a goal location with a collision-free path. This problem is NP-hard and has been studied in robotics, wireless sensing, and computational geometry. This work contributes to the existing literature by presenting and discussing two results. The first result shows that the minimum constraint removal is NP-hard for simply connected obstacles where each obstacle intersects a constant number of other obstacles. The second result demonstrates that for $n$ simply connected obstacles in the plane, instances of the minimum constraint removal problem with minimum removable obstacles lower than $(n+1)/3$ can be solved in polynomial time. This result is also empirically validated using several instances of randomly sampled axis-parallel rectangles. △ Less

Submitted 2 May, 2023; originally announced May 2023.

Comments: Accepted for presentation at the 18th international conference on Intelligent Autonomous System 2023

Embodiment perception of a smart home assistant

Authors: Mariya Kilina, Tommaso Elia, Syed Yusha Kareem, Alessandro Carfi, Fulvio Mastrogiovanni

Abstract: Demographic growth and rise in the average age of the population is increasing the demand for the elderly assistance. Health care oriented ambient intelligence technologies are fundamental to support elderly peoples' autonomy. In this paper, we present a smart home system that is able to recognize human activities and is integrated with a proactive vocal assistant. We chose one of possible user sc… ▽ More Demographic growth and rise in the average age of the population is increasing the demand for the elderly assistance. Health care oriented ambient intelligence technologies are fundamental to support elderly peoples' autonomy. In this paper, we present a smart home system that is able to recognize human activities and is integrated with a proactive vocal assistant. We chose one of possible user scenarios to show the performance of this smart home system and to perform a preliminary comparison between users' experience while watching videos of a volunteer interacting with an embodied versus a not-embodied assistant. The scenario is recorded from the user's point of view, while the user interacts with a robot assistant or a simple vocal assistant. The results of the User Experience Questionnaire show that participants found the robot assistant considerably more attractive, innovative and stimulating in comparison to the vocal assistant. △ Less

Submitted 28 April, 2023; originally announced April 2023.

Comments: Published at International Conference on Social Robotics 2022

A Framework for Neurosymbolic Robot Action Planning using Large Language Models

Authors: Alessio Capitanelli, Fulvio Mastrogiovanni

Abstract: Symbolic task planning is a widely used approach to enforce robot autonomy due to its ease of understanding and deployment in robot architectures. However, techniques for symbolic task planning are difficult to scale in real-world, human-robot collaboration scenarios because of the poor performance in complex planning domains or when frequent re-planning is needed. We present a framework, Teriyaki… ▽ More Symbolic task planning is a widely used approach to enforce robot autonomy due to its ease of understanding and deployment in robot architectures. However, techniques for symbolic task planning are difficult to scale in real-world, human-robot collaboration scenarios because of the poor performance in complex planning domains or when frequent re-planning is needed. We present a framework, Teriyaki, specifically aimed at bridging the gap between symbolic task planning and machine learning approaches. The rationale is training Large Language Models (LLMs), namely GPT-3, into a neurosymbolic task planner compatible with the Planning Domain Definition Language (PDDL), and then leveraging its generative capabilities to overcome a number of limitations inherent to symbolic task planners. Potential benefits include (i) a better scalability in so far as the planning domain complexity increases, since LLMs' response time linearly scales with the combined length of the input and the output, and (ii) the ability to synthesize a plan action-by-action instead of end-to-end, making each action available for execution as soon as it is generated instead of waiting for the whole plan to be available, which in turn enables concurrent planning and execution. Recently, significant efforts have been devoted by the research community to evaluate the cognitive capabilities of LLMs, with alternate successes. Instead, with Teriyaki we aim to provide an overall planning performance comparable to traditional planners in specific planning domains, while leveraging LLMs capabilities to build a look-ahead predictive planning model. Preliminary results in selected domains show that our method can: (i) solve 95.5% of problems in a test data set of 1,000 samples; (ii) produce plans up to 13.5% shorter than a traditional symbolic planner; (iii) reduce average overall waiting times for a plan availability by up to 61.4% △ Less

Submitted 4 June, 2024; v1 submitted 1 March, 2023; originally announced March 2023.

Comments: 36 pages, 7 figures, 2 tables. Updated according to reviewers' comments. Previous title: A Framework to Generate Neurosymbolic PDDL-compliant Planners

ACM Class: I.2.6; I.2.8; I.2.9

Journal ref: Front. Neurorobot. 18:1342786, 04 June 2024

Computational Tradeoff in Minimum Obstacle Displacement Planning for Robot Navigation

Authors: Antony Thomas, Giulio Ferro, Fulvio Mastrogiovanni, Michela Robba

Abstract: In this paper, we look into the minimum obstacle displacement (MOD) planning problem from a mobile robot motion planning perspective. This problem finds an optimal path to goal by displacing movable obstacles when no path exists due to collision with obstacles. However this problem is computationally expensive and grows exponentially in the size of number of movable obstacles. This work looks into… ▽ More In this paper, we look into the minimum obstacle displacement (MOD) planning problem from a mobile robot motion planning perspective. This problem finds an optimal path to goal by displacing movable obstacles when no path exists due to collision with obstacles. However this problem is computationally expensive and grows exponentially in the size of number of movable obstacles. This work looks into approximate solutions that are computationally less intensive and differ from the optimal solution by a factor of the optimal cost. △ Less

Submitted 14 February, 2023; originally announced February 2023.

Comments: Accepted for presentation at the 2023 IEEE International Conference on Robotics and Automation (ICRA)

An Ergonomic Role Allocation Framework for Dynamic Human-Robot Collaborative Tasks

Authors: Elena Merlo, Edoardo Lamon, Fabio Fusaro, Marta Lorenzini, Alessandro Carfì, Fulvio Mastrogiovanni, Arash Ajoudani

Abstract: By incorporating ergonomics principles into the task allocation processes, human-robot collaboration (HRC) frameworks can favour the prevention of work-related musculoskeletal disorders (WMSDs). In this context, existing offline methodologies do not account for the variability of human actions and states; therefore, planning and dynamically assigning roles in human-robot teams remains an unaddress… ▽ More By incorporating ergonomics principles into the task allocation processes, human-robot collaboration (HRC) frameworks can favour the prevention of work-related musculoskeletal disorders (WMSDs). In this context, existing offline methodologies do not account for the variability of human actions and states; therefore, planning and dynamically assigning roles in human-robot teams remains an unaddressed challenge.This study aims to create an ergonomic role allocation framework that optimises the HRC, taking into account task features and human state measurements. The presented framework consists of two main modules: the first provides the HRC task model, exploiting AND/OR Graphs (AOG)s, which we adapted to solve the allocation problem; the second module describes the ergonomic risk assessment during task execution through a risk indicator and updates the AOG-related variables to influence future task allocation. The proposed framework can be combined with any time-varying ergonomic risk indicator that evaluates human cognitive and physical burden. In this work, we tested our framework in an assembly scenario, introducing a risk index named Kinematic Wear.The overall framework has been tested with a multi-subject experiment. The task allocation results and subjective evaluations, measured with questionnaires, show that high-risk actions are correctly recognised and not assigned to humans, reducing fatigue and frustration in collaborative tasks. △ Less

Submitted 19 January, 2023; originally announced January 2023.

Comments: 14 pages, 22 figures, accepted to the special issue on Human-centric Smart Manufacturing: Trends, Issues and Challenges of the Journal of Manufacturing Systems

If You Are Careful, So Am I! How Robot Communicative Motions Can Influence Human Approach in a Joint Task

Authors: Linda Lastrico, Nuno Ferreira Duarte, Alessandro Carfì, Francesco Rea, Fulvio Mastrogiovanni, Alessandra Sciutti, José Santos-Victor

Abstract: As humans, we have a remarkable capacity for reading the characteristics of objects only by observing how another person carries them. Indeed, how we perform our actions naturally embeds information on the item features. Collaborative robots can achieve the same ability by modulating the strategy used to transport objects with their end-effector. A contribution in this sense would promote spontane… ▽ More As humans, we have a remarkable capacity for reading the characteristics of objects only by observing how another person carries them. Indeed, how we perform our actions naturally embeds information on the item features. Collaborative robots can achieve the same ability by modulating the strategy used to transport objects with their end-effector. A contribution in this sense would promote spontaneous interactions by making an implicit yet effective communication channel available. This work investigates if humans correctly perceive the implicit information shared by a robotic manipulator through its movements during a dyadic collaboration task. Exploiting a generative approach, we designed robot actions to convey virtual properties of the transported objects, particularly to inform the partner if any caution is required to handle the carried item. We found that carefulness is correctly interpreted when observed through the robot movements. In the experiment, we used identical empty plastic cups; nevertheless, participants approached them differently depending on the attitude shown by the robot: humans change how they reach for the object, being more careful whenever the robot does the same. This emerging form of motor contagion is entirely spontaneous and happens even if the task does not require it. △ Less

Submitted 24 October, 2022; originally announced October 2022.

Comments: Accepted to ICSR 2022, 14th International Conference on Social Robotics, preprint version

Robots with Different Embodiments Can Express and Influence Carefulness in Object Manipulation

Authors: Linda Lastrico, Luca Garello, Francesco Rea, Nicoletta Noceti, Fulvio Mastrogiovanni, Alessandra Sciutti, Alessandro Carfi

Abstract: Humans have an extraordinary ability to communicate and read the properties of objects by simply watching them being carried by someone else. This level of communicative skills and interpretation, available to humans, is essential for collaborative robots if they are to interact naturally and effectively. For example, suppose a robot is handing over a fragile object. In that case, the human who re… ▽ More Humans have an extraordinary ability to communicate and read the properties of objects by simply watching them being carried by someone else. This level of communicative skills and interpretation, available to humans, is essential for collaborative robots if they are to interact naturally and effectively. For example, suppose a robot is handing over a fragile object. In that case, the human who receives it should be informed of its fragility in advance, through an immediate and implicit message, i.e., by the direct modulation of the robot's action. This work investigates the perception of object manipulations performed with a communicative intent by two robots with different embodiments (an iCub humanoid robot and a Baxter robot). We designed the robots' movements to communicate carefulness or not during the transportation of objects. We found that not only this feature is correctly perceived by human observers, but it can elicit as well a form of motor adaptation in subsequent human object manipulations. In addition, we get an insight into which motion features may induce to manipulate an object more or less carefully. △ Less

Submitted 23 December, 2022; v1 submitted 3 August, 2022; originally announced August 2022.

Comments: Proceedings of the IEEE International Conference on Development and Learning (ICDL 2022) - preprint version, published version available at https://ieeexplore.ieee.org/document/9962196

Journal ref: 2022 IEEE International Conference on Development and Learning (ICDL)

Gestural and Touchscreen Interaction for Human-Robot Collaboration: a Comparative Study

Authors: Antonino Bongiovanni, Alessio De Luca, Luna Gava, Lucrezia Grassi, Marta Lagomarsino, Marco Lapolla, Antonio Marino, Patrick Roncagliolo, Simone Macciò, Alessandro Carfì, Fulvio Mastrogiovanni

Abstract: Close human-robot interaction (HRI), especially in industrial scenarios, has been vastly investigated for the advantages of combining human and robot skills. For an effective HRI, the validity of currently available human-machine communication media or tools should be questioned, and new communication modalities should be explored. This article proposes a modular architecture allowing human operat… ▽ More Close human-robot interaction (HRI), especially in industrial scenarios, has been vastly investigated for the advantages of combining human and robot skills. For an effective HRI, the validity of currently available human-machine communication media or tools should be questioned, and new communication modalities should be explored. This article proposes a modular architecture allowing human operators to interact with robots through different modalities. In particular, we implemented the architecture to handle gestural and touchscreen input, respectively, using a smartwatch and a tablet. Finally, we performed a comparative user experience study between these two modalities. △ Less

Submitted 8 July, 2022; originally announced July 2022.

Comments: Accepted for publication at the 17th International Conference on Intelligent Autonomous Systems (IAS-17)

Mixed Reality as Communication Medium for Human-Robot Collaboration

Authors: Simone Macciò, Alessandro Carfì, Fulvio Mastrogiovanni

Abstract: Humans engaged in collaborative activities are naturally able to convey their intentions to teammates through multi-modal communication, which is made up of explicit and implicit cues. Similarly, a more natural form of human-robot collaboration may be achieved by enabling robots to convey their intentions to human teammates via multiple communication channels. In this paper, we postulate that a be… ▽ More Humans engaged in collaborative activities are naturally able to convey their intentions to teammates through multi-modal communication, which is made up of explicit and implicit cues. Similarly, a more natural form of human-robot collaboration may be achieved by enabling robots to convey their intentions to human teammates via multiple communication channels. In this paper, we postulate that a better communication may take place should collaborative robots be able to anticipate their movements to human teammates in an intuitive way. In order to support such a claim, we propose a robot system's architecture through which robots can communicate planned motions to human teammates leveraging a Mixed Reality interface powered by modern head-mounted displays. Specifically, the robot's hologram, which is superimposed to the real robot in the human teammate's point of view, shows the robot's future movements, allowing the human to understand them in advance, and possibly react to them in an appropriate way. We conduct a preliminary user study to evaluate the effectiveness of the proposed anticipatory visualization during a complex collaborative task. The experimental results suggest that an improved and more natural collaboration can be achieved by employing this anticipatory communication mode. △ Less

Submitted 30 June, 2022; originally announced June 2022.

Minimum Displacement Motion Planning for Movable Obstacles

Authors: Antony Thomas, Fulvio Mastrogiovanni

Abstract: This paper presents a minimum displacement motion planning problem wherein obstacles are displaced by a minimum amount to find a feasible path. We define a metric for robot-obstacle intersection that measures the extent of the intersection and use this to penalize robot-obstacle overlaps. Employing the actual robot dynamics, the planner first finds a path through the obstacles that minimizes the r… ▽ More This paper presents a minimum displacement motion planning problem wherein obstacles are displaced by a minimum amount to find a feasible path. We define a metric for robot-obstacle intersection that measures the extent of the intersection and use this to penalize robot-obstacle overlaps. Employing the actual robot dynamics, the planner first finds a path through the obstacles that minimizes the robot-obstacle intersections. The metric is then used to iteratively displace the obstacles to achieve a feasible path. Several examples are provided that successfully demonstrates the proposed problem. △ Less

Submitted 27 April, 2022; originally announced April 2022.

Comments: Accepted for presentation at the 17th International Conference on Intelligent Autonomous Systems (IAS-17)

Synthesis and Execution of Communicative Robotic Movements with Generative Adversarial Networks

Authors: Luca Garello, Linda Lastrico, Alessandra Sciutti, Nicoletta Noceti, Fulvio Mastrogiovanni, Francesco Rea

Abstract: Object manipulation is a natural activity we perform every day. How humans handle objects can communicate not only the willfulness of the acting, or key aspects of the context where we operate, but also the properties of the objects involved, without any need for explicit verbal description. Since human intelligence comprises the ability to read the context, allowing robots to perform actions that… ▽ More Object manipulation is a natural activity we perform every day. How humans handle objects can communicate not only the willfulness of the acting, or key aspects of the context where we operate, but also the properties of the objects involved, without any need for explicit verbal description. Since human intelligence comprises the ability to read the context, allowing robots to perform actions that intuitively convey this kind of information would greatly facilitate collaboration. In this work, we focus on how to transfer on two different robotic platforms the same kinematics modulation that humans adopt when manipulating delicate objects, aiming to endow robots with the capability to show carefulness in their movements. We choose to modulate the velocity profile adopted by the robots' end-effector, inspired by what humans do when transporting objects with different characteristics. We exploit a novel Generative Adversarial Network architecture, trained with human kinematics examples, to generalize over them and generate new and meaningful velocity profiles, either associated with careful or not careful attitudes. This approach would allow next generation robots to select the most appropriate style of movement, depending on the perceived context, and autonomously generate their motor action execution. △ Less

Submitted 31 March, 2022; v1 submitted 29 March, 2022; originally announced March 2022.

Comments: Submitted to the Special Issue on Emerging Topics on Development and Learning, IEEE TCDS. Unpublished, review process ongoing. Luca Garello and Linda Lastrico contributed equally to this work, hence they share the first name

Gesture-based Human-Machine Interaction: Taxonomy, Problem Definition, and Analysis

Authors: Alessandro Carfì, Fulvio Mastrogiovanni

Abstract: The possibility for humans to interact with physical or virtual systems using gestures has been vastly explored by researchers and designers in the last twenty years to provide new and intuitive interaction modalities. Unfortunately, the literature about gestural interaction is not homogeneous, and it is characterised by a lack of shared terminology. This leads to fragmented results and makes it d… ▽ More The possibility for humans to interact with physical or virtual systems using gestures has been vastly explored by researchers and designers in the last twenty years to provide new and intuitive interaction modalities. Unfortunately, the literature about gestural interaction is not homogeneous, and it is characterised by a lack of shared terminology. This leads to fragmented results and makes it difficult for research activities to build on top of state-of-the-art results and approaches. The analysis in this paper aims at creating a common conceptual design framework to enforce development efforts in gesture-based human-machine interaction. The main contributions of the paper can be summarised as follows: (i) we provide a broad definition for the notion of functional gesture in human-machine interaction, (ii) we design a flexible and expandable gesture taxonomy, and (iii) we put forward a detailed problem statement for gesture-based human-machine interaction. Finally, to support our main contribution, the paper presents, and analyses 83 most pertinent articles classified on the basis of our taxonomy and problem statement. △ Less

Submitted 25 January, 2022; originally announced January 2022.

OWLOOP: A Modular API to Describe OWL Axioms in OOP Objects Hierarchies

Authors: Luca Buoncompagni, Syed Yusha Kareem, Fulvio Mastrogiovanni

Abstract: OWLOOP is an Application Programming Interface (API) for using the Ontology Web Language (OWL) by the means of Object-Oriented Programming (OOP). It is common to design software architectures using the OOP paradigm for increasing their modularity. If the components of an architecture also exploit OWL ontologies for knowledge representation and reasoning, they would require to be interfaced with OW… ▽ More OWLOOP is an Application Programming Interface (API) for using the Ontology Web Language (OWL) by the means of Object-Oriented Programming (OOP). It is common to design software architectures using the OOP paradigm for increasing their modularity. If the components of an architecture also exploit OWL ontologies for knowledge representation and reasoning, they would require to be interfaced with OWL axioms. Since OWL does not adhere to the OOP paradigm, such an interface often leads to boilerplate code affecting modularity, and OWLOOP is designed to address this issue as well as the associated computational aspects. We present an extension of the OWL-API to provide a general-purpose interface between OWL axioms subject to reasoning and modular OOP objects hierarchies. △ Less

Submitted 31 December, 2021; originally announced December 2021.

Comments: This version of the manuscript has been published on the SoftwareX Elsevier journal in January 2022. The manuscript is made of 21 pages, which include 3 tables, 6 figures, and 4 listings

MSC Class: 68T01 ACM Class: D.2.0; D.1.0

Journal ref: SoftwareX, January 2022, 100952, Vol. 17, Elsevier

Dynamic Human-Robot Role Allocation based on Human Ergonomics Risk Prediction and Robot Actions Adaptation

Authors: Elena Merlo, Edoardo Lamon, Fabio Fusaro, Marta Lorenzini, Alessandro Carfì, Fulvio Mastrogiovanni, Arash Ajoudani, .

Abstract: Despite cobots have high potential in bringing several benefits in the manufacturing and logistic processes, but their rapid (re-)deployment in changing environments is still limited. To enable fast adaptation to new product demands and to boost the fitness of the human workers to the allocated tasks, we propose a novel method that optimizes assembly strategies and distributes the effort among the… ▽ More Despite cobots have high potential in bringing several benefits in the manufacturing and logistic processes, but their rapid (re-)deployment in changing environments is still limited. To enable fast adaptation to new product demands and to boost the fitness of the human workers to the allocated tasks, we propose a novel method that optimizes assembly strategies and distributes the effort among the workers in human-robot cooperative tasks. The cooperation model exploits AND/OR Graphs that we adapted to solve also the role allocation problem. The allocation algorithm considers quantitative measurements that are computed online to describe human operator's ergonomic status and task properties. We conducted preliminary experiments to demonstrate that the proposed approach succeeds in controlling the task allocation process to ensure safe and ergonomic conditions for the human worker. △ Less

Submitted 19 January, 2023; v1 submitted 5 November, 2021; originally announced November 2021.

Comments: 7 pages, 11 figures, presented at the 2022 IEEE International Conference on Robotics and Automation (ICRA)

Exact and Bounded Collision Probability for Motion Planning under Gaussian Uncertainty

Authors: Antony Thomas, Fulvio Mastrogiovanni, Marco Baglietto

Abstract: Computing collision-free trajectories is of prime importance for safe navigation. We present an approach for computing the collision probability under Gaussian distributed motion and sensing uncertainty with the robot and static obstacle shapes approximated as ellipsoids. The collision condition is formulated as the distance between ellipsoids and unlike previous approaches we provide a method for… ▽ More Computing collision-free trajectories is of prime importance for safe navigation. We present an approach for computing the collision probability under Gaussian distributed motion and sensing uncertainty with the robot and static obstacle shapes approximated as ellipsoids. The collision condition is formulated as the distance between ellipsoids and unlike previous approaches we provide a method for computing the exact collision probability. Furthermore, we provide a tight upper bound that can be computed much faster during online planning. Comparison to other state-of-the-art methods is also provided. The proposed method is evaluated in simulation under varying configuration and number of obstacles. △ Less

Submitted 12 October, 2021; originally announced October 2021.

Comments: Accepted for publication in the IEEE Robotics and Automation Letters (RA-L)

Task Allocation for Multi-Robot Task and Motion Planning: a case for Object Picking in Cluttered Workspaces

Authors: Hossein Karami, Antony Thomas, Fulvio Mastrogiovanni

Abstract: We present an AND/OR graph-based, integrated multi-robot task and motion planning approach which (i) performs task allocation coordinating the activity of a given number of robots, and (ii) is capable of handling tasks which involve an a priori unknown number of object re-arrangements, such as those involved in retrieving objects from cluttered workspaces. Such situations may arise, for example, i… ▽ More We present an AND/OR graph-based, integrated multi-robot task and motion planning approach which (i) performs task allocation coordinating the activity of a given number of robots, and (ii) is capable of handling tasks which involve an a priori unknown number of object re-arrangements, such as those involved in retrieving objects from cluttered workspaces. Such situations may arise, for example, in search and rescue scenarios, while locating/picking a cluttered object of interest. The corresponding problem falls under the category of planning in clutter. One of the challenges while planning in clutter is that the number of object re-arrangements required to pick the target object is not known beforehand, in general. Moreover, such tasks can be decomposed in a variety of ways, since different cluttering object re-arrangements are possible to reach the target object. In our approach, task allocation and decomposition is achieved by maximizing a combined utility function. The allocated tasks are performed by an integrated task and motion planner, which is robust to the requirement of an unknown number of re-arrangement tasks. We demonstrate our results with experiments in simulation on two Franka Emika manipulators. △ Less

Submitted 8 October, 2021; originally announced October 2021.

Comments: Accepted for presentation at 20th International Conference of the Italian Association for Artificial Intelligence, Milano (IT), December 1st-3rd, 2021

From Movement Kinematics to Object Properties: Online Recognition of Human Carefulness

Authors: Linda Lastrico, Alessandro Carfì, Francesco Rea, Alessandra Sciutti, Fulvio Mastrogiovanni

Abstract: When manipulating objects, humans finely adapt their motions to the characteristics of what they are handling. Thus, an attentive observer can foresee hidden properties of the manipulated object, such as its weight, temperature, and even whether it requires special care in manipulation. This study is a step towards endowing a humanoid robot with this last capability. Specifically, we study how a r… ▽ More When manipulating objects, humans finely adapt their motions to the characteristics of what they are handling. Thus, an attentive observer can foresee hidden properties of the manipulated object, such as its weight, temperature, and even whether it requires special care in manipulation. This study is a step towards endowing a humanoid robot with this last capability. Specifically, we study how a robot can infer online, from vision alone, whether or not the human partner is careful when moving an object. We demonstrated that a humanoid robot could perform this inference with high accuracy (up to 81.3%) even with a low-resolution camera. Only for short movements without obstacles, carefulness recognition was insufficient. The prompt recognition of movement carefulness from observing the partner's action will allow robots to adapt their actions on the object to show the same degree of care as their human partners. △ Less

Submitted 1 September, 2021; originally announced September 2021.

Comments: Accepted for full paper publication in the Proceedings of the Thirteenth International Conference on Social Robotics (ICSR2021) 10 pages, 7 figures

Journal ref: Lecture Notes in Computer Science, 13086 (2021) 61-72

Property-Aware Robot Object Manipulation: a Generative Approach

Authors: Luca Garello, Linda Lastrico, Francesco Rea, Fulvio Mastrogiovanni, Nicoletta Noceti, Alessandra Sciutti

Abstract: When transporting an object, we unconsciously adapt our movement to its properties, for instance by slowing down when the item is fragile. The most relevant features of an object are immediately revealed to a human observer by the way the handling occurs, without any need for verbal description. It would greatly facilitate collaboration to enable humanoid robots to perform movements that convey si… ▽ More When transporting an object, we unconsciously adapt our movement to its properties, for instance by slowing down when the item is fragile. The most relevant features of an object are immediately revealed to a human observer by the way the handling occurs, without any need for verbal description. It would greatly facilitate collaboration to enable humanoid robots to perform movements that convey similar intuitive cues to the observers. In this work, we focus on how to generate robot motion adapted to the hidden properties of the manipulated objects, such as their weight and fragility. We explore the possibility of leveraging Generative Adversarial Networks to synthesize new actions coherent with the properties of the object. The use of a generative approach allows us to create new and consistent motion patterns, without the need of collecting a large number of recorded human-led demonstrations. Besides, the informative content of the actions is preserved. Our results show that Generative Adversarial Nets can be a powerful tool for the generation of novel and meaningful transportation actions, which result effectively modulated as a function of the object weight and the carefulness required in its handling. △ Less

Submitted 8 June, 2021; originally announced June 2021.

Comments: Accepted for publication in the Proceedings of the IEEE International Conference on Development and Learning (ICDL) 2021 - 11th ICDL-EPIROB 7 pages, 5 figures

Journal ref: 2021 IEEE International Conference on Development and Learning (ICDL), 2021, pp. 1-7

Human Activity Recognition Models in Ontology Networks

Authors: Luca Buoncompagni, Syed Yusha Kareem, Fulvio Mastrogiovanni

Abstract: We present Arianna+, a framework to design networks of ontologies for representing knowledge enabling smart homes to perform human activity recognition online. In the network, nodes are ontologies allowing for various data contextualisation, while edges are general-purpose computational procedures elaborating data. Arianna+ provides a flexible interface between the inputs and outputs of procedures… ▽ More We present Arianna+, a framework to design networks of ontologies for representing knowledge enabling smart homes to perform human activity recognition online. In the network, nodes are ontologies allowing for various data contextualisation, while edges are general-purpose computational procedures elaborating data. Arianna+ provides a flexible interface between the inputs and outputs of procedures and statements, which are atomic representations of ontological knowledge. Arianna+ schedules procedures on the basis of events by employing logic-based reasoning, i.e., by checking the classification of certain statements in the ontologies. Each procedure involves input and output statements that are differently contextualised in the ontologies based on specific prior knowledge. Arianna+ allows to design networks that encode data within multiple contexts and, as a reference scenario, we present a modular network based on a spatial context shared among all activities and a temporal context specialised for each activity to be recognised. In the paper, we argue that a network of small ontologies is more intelligible and has a reduced computational load than a single ontology encoding the same knowledge. Arianna+ integrates in the same architecture heterogeneous data processing techniques, which may be better suited to different contexts. Thus, we do not propose a new algorithmic approach to activity recognition, instead, we focus on the architectural aspects for accommodating logic-based and data-driven activity models in a context-oriented way. Also, we discuss how to leverage data contextualisation and reasoning for activity recognition, and to support an iterative development process driven by domain experts. △ Less

Submitted 5 May, 2021; originally announced May 2021.

Comments: The paper has been accepted for publication in the IEEE Transactions on Cybernetics journal on April 2021 and with DOI 10.1109/TCYB.2021.3073539. It is an extension of arXiv:1707.03988v1 and it is related to the arXiv:1809.08208v1 article. It contains 20 pages, 6 figures, 4 tables and 2 Appendices

MSC Class: 68T27; 68T35 ACM Class: I.2.4; I.2.1; I.1.4; F.3.1; F.4.1; I.5.1; I.5.2; D.2.2; D.2.11

Journal ref: IEEE Transactions on Cybernetics, 2021

MPTP: Motion-Planning-aware Task Planning for Navigation in Belief Space

Authors: Antony Thomas, Fulvio Mastrogiovanni, Marco Baglietto

Abstract: We present an integrated Task-Motion Planning (TMP) framework for navigation in large-scale environments. Of late, TMP for manipulation has attracted significant interest resulting in a proliferation of different approaches. In contrast, TMP for navigation has received considerably less attention. Autonomous robots operating in real-world complex scenarios require planning in the discrete (task) s… ▽ More We present an integrated Task-Motion Planning (TMP) framework for navigation in large-scale environments. Of late, TMP for manipulation has attracted significant interest resulting in a proliferation of different approaches. In contrast, TMP for navigation has received considerably less attention. Autonomous robots operating in real-world complex scenarios require planning in the discrete (task) space and the continuous (motion) space. In knowledge-intensive domains, on the one hand, a robot has to reason at the highest-level, for example, the objects to procure, the regions to navigate to in order to acquire them; on the other hand, the feasibility of the respective navigation tasks have to be checked at the execution level. This presents a need for motion-planning-aware task planners. In this paper, we discuss a probabilistically complete approach that leverages this task-motion interaction for navigating in large knowledge-intensive domains, returning a plan that is optimal at the task-level. The framework is intended for motion planning under motion and sensing uncertainty, which is formally known as belief space planning. The underlying methodology is validated in simulation, in an office environment and its scalability is tested in the larger Willow Garage world. A reasonable comparison with a work that is closest to our approach is also provided. We also demonstrate the adaptability of our approach by considering a building floor navigation domain. Finally, we also discuss the limitations of our approach and put forward suggestions for improvements and future work. △ Less

Submitted 10 April, 2021; originally announced April 2021.

Comments: Accepted for publication in Robotics and Autonomous Systems. arXiv admin note: text overlap with arXiv:1910.11683, arXiv:2010.00780

Probabilistic Collision Constraint for Motion Planning in Dynamic Environments

Authors: Antony Thomas, Fulvio Mastrogiovanni, Marco Baglietto

Abstract: Online generation of collision free trajectories is of prime importance for autonomous navigation. Dynamic environments, robot motion and sensing uncertainties adds further challenges to collision avoidance systems. This paper presents an approach for collision avoidance in dynamic environments, incorporating robot and obstacle state uncertainties. We derive a tight upper bound for collision proba… ▽ More Online generation of collision free trajectories is of prime importance for autonomous navigation. Dynamic environments, robot motion and sensing uncertainties adds further challenges to collision avoidance systems. This paper presents an approach for collision avoidance in dynamic environments, incorporating robot and obstacle state uncertainties. We derive a tight upper bound for collision probability between robot and obstacle and formulate it as a motion planning constraint which is solvable in real time. The proposed approach is tested in simulation considering mobile robots as well as quadrotors to demonstrate that successful collision avoidance is achieved in real time application. We also provide a comparison of our approach with several state-of-the-art methods. △ Less

Submitted 4 April, 2021; originally announced April 2021.

Comments: Accepted for presentation at the 16th International Conference on Intelligent Autonomous Systems (IAS-16)

A Task-Motion Planning Framework Using Iteratively Deepened AND/OR Graph Networks

Authors: Hossein Karami, Antony Thomas, Fulvio Mastrogiovanni

Abstract: We present an approach for Task-Motion Planning (TMP) using Iterative Deepened AND/OR Graph Networks (TMP-IDAN) that uses an AND/OR graph network based novel abstraction for compactly representing the task-level states and actions. While retrieving a target object from clutter, the number of object re-arrangements required to grasp the target is not known ahead of time. To address this challenge,… ▽ More We present an approach for Task-Motion Planning (TMP) using Iterative Deepened AND/OR Graph Networks (TMP-IDAN) that uses an AND/OR graph network based novel abstraction for compactly representing the task-level states and actions. While retrieving a target object from clutter, the number of object re-arrangements required to grasp the target is not known ahead of time. To address this challenge, in contrast to traditional AND/OR graph-based planners, we grow the AND/OR graph online until the target grasp is feasible and thereby obtain a network of AND/OR graphs. The AND/OR graph network allows faster computations than traditional task planners. We validate our approach and evaluate its capabilities using a Baxter robot and a state-of-the-art robotics simulator in several challenging non-trivial cluttered table-top scenarios. The experiments show that our approach is readily scalable to increasing number of objects and different degrees of clutter. △ Less

Submitted 4 April, 2021; originally announced April 2021.

Comments: Submitted to IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2021

Careful with That! Observation of Human Movements to Estimate Objects Properties

Authors: Linda Lastrico, Alessandro Carfì, Alessia Vignolo, Alessandra Sciutti, Fulvio Mastrogiovanni, Francesco Rea

Abstract: Humans are very effective at interpreting subtle properties of the partner's movement and use this skill to promote smooth interactions. Therefore, robotic platforms that support human partners in daily activities should acquire similar abilities. In this work we focused on the features of human motor actions that communicate insights on the weight of an object and the carefulness required in its… ▽ More Humans are very effective at interpreting subtle properties of the partner's movement and use this skill to promote smooth interactions. Therefore, robotic platforms that support human partners in daily activities should acquire similar abilities. In this work we focused on the features of human motor actions that communicate insights on the weight of an object and the carefulness required in its manipulation. Our final goal is to enable a robot to autonomously infer the degree of care required in object handling and to discriminate whether the item is light or heavy, just by observing a human manipulation. This preliminary study represents a promising step towards the implementation of those abilities on a robot observing the scene with its camera. Indeed, we succeeded in demonstrating that it is possible to reliably deduct if the human operator is careful when handling an object, through machine learning algorithms relying on the stream of visual acquisition from either a robot camera or from a motion capture system. On the other hand, we observed that the same approach is inadequate to discriminate between light and heavy objects. △ Less

Submitted 10 March, 2021; v1 submitted 2 March, 2021; originally announced March 2021.

Comments: Preprint version - 13th International Workshop of Human-Friendly Robotics (HFR 2020)

Journal ref: Human-Friendly Robotics, SPAR 18 (2020) 127-141

An Integrated Localisation, Motion Planning and Obstacle Avoidance Algorithm in Belief Space

Authors: Antony Thomas, Fulvio Mastrogiovanni, Marco Baglietto

Abstract: As robots are being increasingly used in close proximity to humans and objects, it is imperative that robots operate safely and efficiently under real-world conditions. Yet, the environment is seldom known perfectly. Noisy sensors and actuation errors compound to the errors introduced while estimating features of the environment. We present a novel approach (1) to incorporate these uncertainties f… ▽ More As robots are being increasingly used in close proximity to humans and objects, it is imperative that robots operate safely and efficiently under real-world conditions. Yet, the environment is seldom known perfectly. Noisy sensors and actuation errors compound to the errors introduced while estimating features of the environment. We present a novel approach (1) to incorporate these uncertainties for robot state estimation and (2) to compute the probability of collision pertaining to the estimated robot configurations. The expression for collision probability is obtained as an infinite series and we prove its convergence. An upper bound for the truncation error is also derived and the number of terms required is demonstrated by analyzing the convergence for different robot and obstacle configurations. We evaluate our approach using two simulation domains which use a roadmap-based strategy to synthesize trajectories that satisfy collision probability bounds. △ Less

Submitted 27 January, 2021; originally announced January 2021.

Comments: Accepted for publication in Intelligent Service Robotics

Collaborative Robotic Manipulation: A Use Case of Articulated Objects in Three-dimensions with Gravity

Authors: Riccardo Bertolucci, Alessio Capitanelli, Marco Maratea, Fulvio Mastrogiovanni, Mauro Vallati

Abstract: This paper addresses two intertwined needs for collaborative robots operating in shop-floor environments. The first is the ability to perform complex manipulation operations, such as those on articulated or even flexible objects, in a way robust to a high degree of variability in the actions possibly carried out by human operators during collaborative tasks. The second is encoding in such operatio… ▽ More This paper addresses two intertwined needs for collaborative robots operating in shop-floor environments. The first is the ability to perform complex manipulation operations, such as those on articulated or even flexible objects, in a way robust to a high degree of variability in the actions possibly carried out by human operators during collaborative tasks. The second is encoding in such operations a basic knowledge about physical laws (e.g., gravity), and their effects on the models used by the robot to plan its actions, to generate more robust plans. We adopt the manipulation in three-dimensional space of articulated objects as an effective use case to ground both needs, and we use a variant of the Planning Domain Definition Language to integrate the planning process with a notion of gravity. Different complexity levels in modelling gravity are evaluated, which trade-off model faithfulness and performance. A thorough validation of the framework is done in simulation using a dual-arm Baxter manipulator. △ Less

Submitted 13 November, 2020; originally announced November 2020.

Comments: This paper has been accepted for IEEE ICTAI 2020: https://ictai2020.org/

Manipulation of Articulated Objects using Dual-arm Robots via Answer Set Programming

Authors: Riccardo Bertolucci, Alessio Capitanelli, Carmine Dodaro, Nicola Leone, Marco Maratea, Fulvio Mastrogiovanni, Mauro Vallati

Abstract: The manipulation of articulated objects is of primary importance in Robotics, and can be considered as one of the most complex manipulation tasks. Traditionally, this problem has been tackled by develo** ad-hoc approaches, which lack flexibility and portability. In this paper we present a framework based on Answer Set Programming (ASP) for the automated manipulation of articulated objects in a… ▽ More The manipulation of articulated objects is of primary importance in Robotics, and can be considered as one of the most complex manipulation tasks. Traditionally, this problem has been tackled by develo** ad-hoc approaches, which lack flexibility and portability. In this paper we present a framework based on Answer Set Programming (ASP) for the automated manipulation of articulated objects in a robot control architecture. In particular, ASP is employed for representing the configuration of the articulated object, for checking the consistency of such representation in the knowledge base, and for generating the sequence of manipulation actions. The framework is exemplified and validated on the Baxter dual-arm manipulator in a first, simple scenario. Then, we extend such scenario to improve the overall setup accuracy, and to introduce a few constraints in robot actions execution to enforce their feasibility. The extended scenario entails a high number of possible actions that can be fruitfully combined together. Therefore, we exploit macro actions from automated planning in order to provide more effective plans. We validate the overall framework in the extended scenario, thereby confirming the applicability of ASP also in more realistic Robotics settings, and showing the usefulness of macro actions for the robot-based manipulation of articulated objects. Under consideration in Theory and Practice of Logic Programming (TPLP). △ Less

Submitted 2 October, 2020; originally announced October 2020.

Comments: Under consideration in Theory and Practice of Logic Programming (TPLP)

Towards Multi-Robot Task-Motion Planning for Navigation in Belief Space

Authors: Antony Thomas, Fulvio Mastrogiovanni, Marco Baglietto

Abstract: Autonomous robots operating in large knowledgeintensive domains require planning in the discrete (task) space and the continuous (motion) space. In knowledge-intensive domains, on the one hand, robots have to reason at the highestlevel, for example the regions to navigate to or objects to be picked up and their properties; on the other hand, the feasibility of the respective navigation tasks have… ▽ More Autonomous robots operating in large knowledgeintensive domains require planning in the discrete (task) space and the continuous (motion) space. In knowledge-intensive domains, on the one hand, robots have to reason at the highestlevel, for example the regions to navigate to or objects to be picked up and their properties; on the other hand, the feasibility of the respective navigation tasks have to be checked at the controller execution level. Moreover, employing multiple robots offer enhanced performance capabilities over a single robot performing the same task. To this end, we present an integrated multi-robot task-motion planning framework for navigation in knowledge-intensive domains. In particular, we consider a distributed multi-robot setting incorporating mutual observations between the robots. The framework is intended for motion planning under motion and sensing uncertainty, which is formally known as belief space planning. The underlying methodology and its limitations are discussed, providing suggestions for improvements and future work. We validate key aspects of our approach in simulation. △ Less

Submitted 1 October, 2020; originally announced October 2020.

Comments: Full version of the ECAI-STAIRS 2020 paper. arXiv admin note: text overlap with arXiv:1910.11683

A Task Allocation Approach for Human-Robot Collaboration in Product Defects Inspection Scenarios

Authors: Hossein Karami, Kourosh Darvish, Fulvio Mastrogiovanni

Abstract: The presence and coexistence of human operators and collaborative robots in shop-floor environments raises the need for assigning tasks to either operators or robots, or both. Depending on task characteristics, operator capabilities and the involved robot functionalities, it is of the utmost importance to design strategies allowing for the concurrent and/or sequential allocation of tasks related t… ▽ More The presence and coexistence of human operators and collaborative robots in shop-floor environments raises the need for assigning tasks to either operators or robots, or both. Depending on task characteristics, operator capabilities and the involved robot functionalities, it is of the utmost importance to design strategies allowing for the concurrent and/or sequential allocation of tasks related to object manipulation and assembly. In this paper, we extend the \textsc{FlexHRC} framework presented in \cite{darvish2018flexible} to allow a human operator to interact with multiple, heterogeneous robots at the same time in order to jointly carry out a given task. The extended \textsc{FlexHRC} framework leverages a concurrent and sequential task representation framework to allocate tasks to either operators or robots as part of a dynamic collaboration process. In particular, we focus on a use case related to the inspection of product defects, which involves a human operator, a dual-arm Baxter manipulator from Rethink Robotics and a Kuka youBot mobile manipulator. △ Less

Submitted 14 September, 2020; originally announced September 2020.

Comments: 8 pages, 5 figures,The 29th IEEE International Conference on Robot & Human Interactive Communication

A Hierarchical Architecture for Human-Robot Cooperation Processes

Authors: Kourosh Darvish, Enrico Simetti, Fulvio Mastrogiovanni, Giuseppe Casalino

Abstract: In this paper we propose FlexHRC+, a hierarchical human-robot cooperation architecture designed to provide collaborative robots with an extended degree of autonomy when supporting human operators in high-variability shop-floor tasks. The architecture encompasses three levels, namely for perception, representation, and action. Building up on previous work, here we focus on (i) an in-the-loop decisi… ▽ More In this paper we propose FlexHRC+, a hierarchical human-robot cooperation architecture designed to provide collaborative robots with an extended degree of autonomy when supporting human operators in high-variability shop-floor tasks. The architecture encompasses three levels, namely for perception, representation, and action. Building up on previous work, here we focus on (i) an in-the-loop decision making process for the operations of collaborative robots co** with the variability of actions carried out by human operators, and (ii) the representation level, integrating a hierarchical AND/OR graph whose online behaviour is formally specified using First Order Logic. The architecture is accompanied by experiments including collaborative furniture assembly and object positioning tasks. △ Less

Submitted 6 September, 2020; originally announced September 2020.

Deployment and Evaluation of a Flexible Human-Robot Collaboration Model Based on AND/OR Graphs in a Manufacturing Environment

Authors: Prajval Kumar Murali, Kourosh Darvish, Fulvio Mastrogiovanni

Abstract: The Industry 4.0 paradigm promises shorter development times, increased ergonomy, higher flexibility, and resource efficiency in manufacturing environments. Collaborative robots are an important tangible technology for implementing such a paradigm. A major bottleneck to effectively deploy collaborative robots to manufacturing industries is develo** task planning algorithms that enable them to re… ▽ More The Industry 4.0 paradigm promises shorter development times, increased ergonomy, higher flexibility, and resource efficiency in manufacturing environments. Collaborative robots are an important tangible technology for implementing such a paradigm. A major bottleneck to effectively deploy collaborative robots to manufacturing industries is develo** task planning algorithms that enable them to recognize and naturally adapt to varying and even unpredictable human actions while simultaneously ensuring an overall efficiency in terms of production cycle time. In this context, an architecture encompassing task representation, task planning, sensing, and robot control has been designed, developed and evaluated in a real industrial environment. A pick-and-place palletization task, which requires the collaboration between humans and robots, is investigated. The architecture uses AND/OR graphs for representing and reasoning upon human-robot collaboration models online. Furthermore, objective measures of the overall computational performance and subjective measures of naturalness in human-robot collaboration have been evaluated by performing experiments with production-line operators. The results of this user study demonstrate how human-robot collaboration models like the one we propose can leverage the flexibility and the comfort of operators in the workplace. In this regard, an extensive comparison study among recent models has been carried out. △ Less

Submitted 13 July, 2020; originally announced July 2020.

Multi-sensory Integration in a Quantum-Like Robot Perception Model

Authors: Davide Lanza, Paolo Solinas, Fulvio Mastrogiovanni

Abstract: Formalisms inspired by Quantum theory have been used in Cognitive Science for decades. Indeed, Quantum-Like (QL) approaches provide descriptive features that are inherently suitable for perception, cognition, and decision processing. A preliminary study on the feasibility of a QL robot perception model has been carried out for a robot with limited sensing capabilities. In this paper, we generalize… ▽ More Formalisms inspired by Quantum theory have been used in Cognitive Science for decades. Indeed, Quantum-Like (QL) approaches provide descriptive features that are inherently suitable for perception, cognition, and decision processing. A preliminary study on the feasibility of a QL robot perception model has been carried out for a robot with limited sensing capabilities. In this paper, we generalize such a model for multi-sensory inputs, creating a multidimensional world representation directly based on sensor readings. Given a 3-dimensional case study, we highlight how this model provides a compact and elegant representation, embodying features that are extremely useful for modeling uncertainty and decision. Moreover, the model enables to naturally define query operators to inspect any world state, which answers quantifies the robot's degree of belief on that state. △ Less

Submitted 29 June, 2020; originally announced June 2020.

Comments: Paper submitted to the 17th International Symposium on Experimental Robotics, Malta, Nov. 9-12, 2020

A Preliminary Study for a Quantum-like Robot Perception Model

Authors: Davide Lanza, Paolo Solinas, Fulvio Mastrogiovanni

Abstract: Formalisms based on quantum theory have been used in Cognitive Science for decades due to their descriptive features. A quantum-like (QL) approach provides descriptive features such as state superposition and probabilistic interference behavior. Moreover, quantum systems dynamics have been found isomorphic to cognitive or biological systems dynamics. The objective of this paper is to study the fea… ▽ More Formalisms based on quantum theory have been used in Cognitive Science for decades due to their descriptive features. A quantum-like (QL) approach provides descriptive features such as state superposition and probabilistic interference behavior. Moreover, quantum systems dynamics have been found isomorphic to cognitive or biological systems dynamics. The objective of this paper is to study the feasibility of a QL perception model for a robot with limited sensing capabilities. We introduce a case study, we highlight its limitations, and we investigate and analyze actual robot behaviors through simulations, while actual implementations based on quantum devices encounter errors for unbalanced situations. In order to investigate QL models for robot behavior, and to study the advantages leveraged by QL approaches for robot knowledge representation and processing, we argue that it is preferable to proceed with simulation-oriented techniques rather than actual realizations on quantum backends. △ Less

Submitted 4 June, 2020; originally announced June 2020.

Comments: Paper submitted to 29th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN), 2020, Naples, Italy

Task-Motion Planning for Navigation in Belief Space

Authors: Antony Thomas, Fulvio Mastrogiovanni, Marco Baglietto

Abstract: We present an integrated Task-Motion Planning (TMP) framework for navigation in large-scale environment. Autonomous robots operating in real world complex scenarios require planning in the discrete (task) space and the continuous (motion) space. In knowledge intensive domains, on the one hand, a robot has to reason at the highest-level, for example the regions to navigate to; on the other hand, th… ▽ More We present an integrated Task-Motion Planning (TMP) framework for navigation in large-scale environment. Autonomous robots operating in real world complex scenarios require planning in the discrete (task) space and the continuous (motion) space. In knowledge intensive domains, on the one hand, a robot has to reason at the highest-level, for example the regions to navigate to; on the other hand, the feasibility of the respective navigation tasks have to be checked at the execution level. This presents a need for motion-planning-aware task planners. We discuss a probabilistically complete approach that leverages this task-motion interaction for navigating in indoor domains, returning a plan that is optimal at the task-level. Furthermore, our framework is intended for motion planning under motion and sensing uncertainty, which is formally known as belief space planning. The underlying methodology is validated with a simulated office environment in Gazebo. In addition, we discuss the limitations and provide suggestions for improvements and future work. △ Less

Submitted 24 October, 2019; originally announced October 2019.

Comments: Accepted for publication in the proceedings of the International Symposium on Robotics Research (ISRR) 2019. arXiv admin note: text overlap with arXiv:1908.10227

Task-assisted Motion Planning in Partially Observable Domains

Authors: Antony Thomas, Sunny Amatya, Fulvio Mastrogiovanni, Marco Baglietto

Abstract: We present an integrated Task-Motion Planning framework for robot navigation in belief space. Autonomous robots operating in real world complex scenarios require planning in the discrete (task) space and the continuous (motion) space. To this end, we propose a framework for integrating belief space reasoning within a hybrid task planner. The expressive power of PDDL+ combined with heuristic-driven… ▽ More We present an integrated Task-Motion Planning framework for robot navigation in belief space. Autonomous robots operating in real world complex scenarios require planning in the discrete (task) space and the continuous (motion) space. To this end, we propose a framework for integrating belief space reasoning within a hybrid task planner. The expressive power of PDDL+ combined with heuristic-driven semantic attachments performs the propagated and posterior belief estimates while planning. The underlying methodology for the development of the combined hybrid planner is discussed, providing suggestions for improvements and future work. Furthermore we validate key aspects of our approach using a realistic scenario in simulation. △ Less

Submitted 27 August, 2019; originally announced August 2019.

Long-term area coverage and radio relay positioning using swarms of UAVs

Authors: Floriana Benedetti, Alessio Capitanelli, Fulvio Mastrogiovanni, Gianni Vercelli

Abstract: Unmanned Aerial Vehicles (UAVs) are becoming increasingly useful for tasks which require the acquisition of data over large areas. The coverage problem, i.e., the problem of periodically visiting all subregions of an area at a desired frequency, is especially interesting because of its practical applications, both in industry and long-term monitoring of areas hit by a natural disaster. We focus he… ▽ More Unmanned Aerial Vehicles (UAVs) are becoming increasingly useful for tasks which require the acquisition of data over large areas. The coverage problem, i.e., the problem of periodically visiting all subregions of an area at a desired frequency, is especially interesting because of its practical applications, both in industry and long-term monitoring of areas hit by a natural disaster. We focus here on the latter scenario, and take into consideration its peculiar characteristic, i.e. the a coverage system should be resilient to a changing environment and not be dependent on pre-existing infrastructures for communication. To this purpose, we designed a novel algorithm for online area coverage and simultaneous signal relay that allows a UAV to cover an area freely, while a variable number of other UAVs provide a stable communication with the base and support in the coverage process at the same time. Finally, a test architecture based on the algorithm has been developed and tests have been performed. By comparison with a simple relay chain system, our approach employs up to 64% less time to reach a certain goal of coverage iterations over the map with only 17% worse average communication cost and no impact on the worst case communication cost. △ Less

Submitted 29 October, 2018; originally announced October 2018.

Comments: Book chapter, 23 pages, 11 figures

Online Human Gesture Recognition using Recurrent Neural Networks and Wearable Sensors

Authors: Alessandro Carfi, Carola Motolese, Barbara Bruno, Fulvio Mastrogiovanni

Abstract: Gestures are a natural communication modality for humans. The ability to interpret gestures is fundamental for robots aiming to naturally interact with humans. Wearable sensors are promising to monitor human activity, in particular the usage of triaxial accelerometers for gesture recognition have been explored. Despite this, the state of the art presents lack of systems for reliable online gesture… ▽ More Gestures are a natural communication modality for humans. The ability to interpret gestures is fundamental for robots aiming to naturally interact with humans. Wearable sensors are promising to monitor human activity, in particular the usage of triaxial accelerometers for gesture recognition have been explored. Despite this, the state of the art presents lack of systems for reliable online gesture recognition using accelerometer data. The article proposes SLOTH, an architecture for online gesture recognition, based on a wearable triaxial accelerometer, a Recurrent Neural Network (RNN) probabilistic classifier and a procedure for continuous gesture detection, relying on modelling gesture probabilities, that guarantees (i) good recognition results in terms of precision and recall, (ii) immediate system reactivity. △ Less

Submitted 26 October, 2018; originally announced October 2018.