-
Labeling Sentences with Symbolic and Deictic Gestures via Semantic Similarity
Authors:
Ariel Gjaci,
Carmine Tommaso Recchiuto,
Antonio Sgorbissa
Abstract:
Co-speech gesture generation on artificial agents has gained attention recently, mainly when it is based on data-driven models. However, end-to-end methods often fail to generate co-speech gestures related to semantics with specific forms, i.e., Symbolic and Deictic gestures. In this work, we identify which words in a sentence are contextually related to Symbolic and Deictic gestures. Firstly, we…
▽ More
Co-speech gesture generation on artificial agents has gained attention recently, mainly when it is based on data-driven models. However, end-to-end methods often fail to generate co-speech gestures related to semantics with specific forms, i.e., Symbolic and Deictic gestures. In this work, we identify which words in a sentence are contextually related to Symbolic and Deictic gestures. Firstly, we appropriately chose 12 gestures recognized by people from the Italian culture, which different humanoid robots can reproduce. Then, we implemented two rule-based algorithms to label sentences with Symbolic and Deictic gestures. The rules depend on the semantic similarity scores computed with the RoBerta model between sentences that heuristically represent gestures and sub-sentences inside an objective sentence that artificial agents have to pronounce. We also implemented a baseline algorithm that assigns gestures without computing similarity scores. Finally, to validate the results, we asked 30 persons to label a set of sentences with Deictic and Symbolic gestures through a Graphical User Interface (GUI), and we compared the labels with the ones produced by our algorithms. For this scope, we computed Average Precision (AP) and Intersection Over Union (IOU) scores, and we evaluated the Average Computational Time (ACT). Our results show that semantic similarity scores are useful for finding Symbolic and Deictic gestures in utterances.
△ Less
Submitted 3 July, 2024; v1 submitted 2 July, 2024;
originally announced July 2024.
-
Enhancing LLM-Based Human-Robot Interaction with Nuances for Diversity Awareness
Authors:
Lucrezia Grassi,
Carmine Tommaso Recchiuto,
Antonio Sgorbissa
Abstract:
This paper presents a system for diversity-aware autonomous conversation leveraging the capabilities of large language models (LLMs). The system adapts to diverse populations and individuals, considering factors like background, personality, age, gender, and culture. The conversation flow is guided by the structure of the system's pre-established knowledge base, while LLMs are tasked with various…
▽ More
This paper presents a system for diversity-aware autonomous conversation leveraging the capabilities of large language models (LLMs). The system adapts to diverse populations and individuals, considering factors like background, personality, age, gender, and culture. The conversation flow is guided by the structure of the system's pre-established knowledge base, while LLMs are tasked with various functions, including generating diversity-aware sentences. Achieving diversity-awareness involves providing carefully crafted prompts to the models, incorporating comprehensive information about users, conversation history, contextual details, and specific guidelines. To assess the system's performance, we conducted both controlled and real-world experiments, measuring a wide range of performance indicators.
△ Less
Submitted 25 June, 2024;
originally announced June 2024.
-
Exploring the ethical sensitivity of Ph.D. students in robotics
Authors:
Linda Battistuzzi,
Lucrezia Grassi,
Antonio Sgorbissa
Abstract:
Ethical sensitivity, generally defined as a person's ability to recognize ethical issues and attribute importance to them, is considered to be a crucial competency in the life of professionals and academics and an essential prerequisite to successfully meeting ethical challenges. A concept that first emerged in moral psychology almost 40 years ago, ethical sensitivity has been widely studied in he…
▽ More
Ethical sensitivity, generally defined as a person's ability to recognize ethical issues and attribute importance to them, is considered to be a crucial competency in the life of professionals and academics and an essential prerequisite to successfully meeting ethical challenges. A concept that first emerged in moral psychology almost 40 years ago, ethical sensitivity has been widely studied in healthcare, business, and other domains. Conversely, it appears to have received little to no attention within the robotics community, even though choices in the design and deployment of robots are likely to have wide-ranging, profound ethical impacts on society. Due to the negative repercussions that a lack of ethical sensitivity can have in these contexts, promoting the development of ethical sensitivity among roboticists is imperative, and endeavoring to train this competency becomes a critical undertaking. Therefore, as a first step in this direction and within the context of a broader effort aimed at develo** an online interactive ethics training module for roboticists, we conducted a qualitative exploration of the ethical sensitivity of a sample of Ph.D. students in robotics using case vignettes that exemplified ethical tensions in disaster robotics.
△ Less
Submitted 5 May, 2024;
originally announced May 2024.
-
Efficient Frontier Management for Collaborative Active SLAM
Authors:
Muhammad Farhan Ahmed,
Matteo Maragliano,
Vincent FremontCarmine,
Tommaso Recchiuto,
Antonio Sgorbissa
Abstract:
In autonomous robotics, a critical challenge lies in develo** robust solutions for Active Collaborative SLAM, wherein multiple robots collaboratively explore and map an unknown environment while intelligently coordinating their movements and sensor data acquisitions. In this article, we present an efficient centralized frontier sharing approach that maximizes exploration by taking into account i…
▽ More
In autonomous robotics, a critical challenge lies in develo** robust solutions for Active Collaborative SLAM, wherein multiple robots collaboratively explore and map an unknown environment while intelligently coordinating their movements and sensor data acquisitions. In this article, we present an efficient centralized frontier sharing approach that maximizes exploration by taking into account information gain in the merged map, distance, and reward computation among frontier candidates and encourages the spread of agents into the environment. Eventually, our method efficiently spreads the robots for maximum exploration while kee** SLAM uncertainty low. Additionally, we also present two coordination approaches, synchronous and asynchronous to prioritize robot goal assignments by the central server. The proposed method is implemented in ROS and evaluated through simulation and experiments on publicly available datasets and similar methods, rendering promising results.
△ Less
Submitted 15 May, 2024; v1 submitted 3 October, 2023;
originally announced October 2023.
-
Diversity-aware social robots meet people: beyond context-aware embodied AI
Authors:
Carmine Recchiuto,
Antonio Sgorbissa
Abstract:
The article introduces the concept of "diversity-aware" robotics and discusses the need to develop computational models to embed robots with diversity-awareness: that is, robots capable of adapting and re-configuring their behavior to recognize, respect, and value the uniqueness of the person they interact with to promote inclusion regardless of their age, race, gender, cognitive or physical capab…
▽ More
The article introduces the concept of "diversity-aware" robotics and discusses the need to develop computational models to embed robots with diversity-awareness: that is, robots capable of adapting and re-configuring their behavior to recognize, respect, and value the uniqueness of the person they interact with to promote inclusion regardless of their age, race, gender, cognitive or physical capabilities, etc. Finally, the article discusses possible technical solutions based on Ontologies and Bayesian Networks, starting from previous experience with culturally competent robots.
△ Less
Submitted 10 October, 2022; v1 submitted 12 July, 2022;
originally announced July 2022.
-
Sustainable Cloud Services for Verbal Interaction with Embodied Agents
Authors:
Lucrezia Grassi,
Carmine Tommaso Recchiuto,
Antonio Sgorbissa
Abstract:
This article presents the design and the implementation of a cloud system for knowledge-based autonomous interaction devised for Social Robots and other conversational agents. The system is particularly convenient for low-cost robots and devices: it can be used as a stand-alone dialogue system or as an integration to provide "background" dialogue capabilities to any preexisting Natural Language Pr…
▽ More
This article presents the design and the implementation of a cloud system for knowledge-based autonomous interaction devised for Social Robots and other conversational agents. The system is particularly convenient for low-cost robots and devices: it can be used as a stand-alone dialogue system or as an integration to provide "background" dialogue capabilities to any preexisting Natural Language Processing ability that the robot may already have as part of its basic skills. By connecting to the cloud, developers are provided with a sustainable solution to manage verbal interaction through a network connection, with about 3,000 topics of conversation ready for "chit-chatting" and a library of pre-cooked plans that only needs to be grounded into the robot's physical capabilities. The system is structured as a set of REST API endpoints so that it can be easily expanded by adding new APIs to improve the capabilities of the clients connected to the cloud. Another key feature of the system is that it has been designed to make the development of its clients straightforward: in this way, multiple robots and devices can be easily endowed with the capability of autonomously interacting with the user, understanding when to perform specific actions, and exploiting all the information provided by cloud services. The article outlines and discusses the results of the experiments performed to assess the system's performance in terms of response time, paving the way for its use both for research and market solutions. Links to repositories with clients for ROS and popular robots such as Pepper and NAO are available on request.
△ Less
Submitted 15 March, 2023; v1 submitted 4 March, 2022;
originally announced March 2022.
-
Thermal and Visual Tracking of Photovoltaic Plants for Autonomous UAV inspection
Authors:
Luca Morando,
Carmine Tommaso Recchiuto,
Jacopo Callà,
Paolo Scuteri,
Antonio Sgorbissa
Abstract:
Since photovoltaic (PV) plants require periodic maintenance, using Unmanned Aerial Vehicles (UAV) for inspections can help reduce costs. The thermal and visual inspection of PV installations is currently based on UAV photogrammetry. A UAV equipped with a Global Positioning System (GPS) receiver is assigned a flight zone: the UAV will cover it back and forth to collect images to be later composed i…
▽ More
Since photovoltaic (PV) plants require periodic maintenance, using Unmanned Aerial Vehicles (UAV) for inspections can help reduce costs. The thermal and visual inspection of PV installations is currently based on UAV photogrammetry. A UAV equipped with a Global Positioning System (GPS) receiver is assigned a flight zone: the UAV will cover it back and forth to collect images to be later composed in an orthomosaic. The UAV typically flies at a height above the ground that is appropriate to ensure that images overlap even in the presence of GPS positioning errors. However, this approach has two limitations. Firstly, it requires to cover the whole flight zone, including "empty" areas between PV module rows. Secondly, flying high above the ground limits the resolution of the images to be later inspected. The article proposes a novel approach using an autonomous UAV equipped with an RGB and a thermal camera for PV module tracking. The UAV moves along PV module rows at a lower height than usual and inspects them back and forth in a boustrophedon way by ignoring "empty" areas with no PV modules. Experimental tests performed in simulation and an actual PV plant are reported.
△ Less
Submitted 11 October, 2022; v1 submitted 2 February, 2022;
originally announced February 2022.
-
Culture-to-Culture Image Translation and User Evaluation
Authors:
Giulia Zaino,
Carmine Tommaso Recchiuto,
Antonio Sgorbissa
Abstract:
The article introduces the concept of image "culturization," which we define as the process of altering the ``brushstroke of cultural features" that make objects perceived as belonging to a given culture while preserving their functionalities. First, we defined a pipeline for translating objects' images from a source to a target cultural domain based on state-of-the-art Generative Adversarial Netw…
▽ More
The article introduces the concept of image "culturization," which we define as the process of altering the ``brushstroke of cultural features" that make objects perceived as belonging to a given culture while preserving their functionalities. First, we defined a pipeline for translating objects' images from a source to a target cultural domain based on state-of-the-art Generative Adversarial Networks. Then, we gathered data through an online questionnaire to test four hypotheses concerning the impact of images belonging to different cultural domains on Italian participants. As expected, results depend on individual tastes and preferences: however, they align with our conjecture that some people, during the interaction with an intelligent system, will prefer to be shown images modified to match their cultural background. The study has two main limitations. First, we focussed on the culturization of individual objects instead of complete scenes. However, objects play a crucial role in conveying cultural meanings and can strongly influence how an image is perceived within a specific cultural context. Understanding and addressing object-level translation is a vital step toward achieving more comprehensive scene-level translation in future research. Second, we performed experiments with Italian participants only. We think that there are unique aspects of Italian culture that make it an interesting and relevant case study for exploring the impact of image culturization. Italy is a very culturally conservative society, and Italians have specific sensitivities and expectations regarding the accurate representation of their cultural identity and traditions, which can shape individuals' preferences and inclinations toward certain visual styles, aesthetics, and design choices. As a consequence, we think they are an ideal candidate for a preliminary investigation of image culturization.
△ Less
Submitted 12 May, 2023; v1 submitted 5 January, 2022;
originally announced January 2022.
-
Knowledge-Grounded Dialogue Flow Management for Social Robots and Conversational Agents
Authors:
Lucrezia Grassi,
Carmine Tommaso Recchiuto,
Antonio Sgorbissa
Abstract:
The article proposes a system for knowledge-based conversation designed for Social Robots and other conversational agents. The proposed system relies on an Ontology for the description of all concepts that may be relevant conversation topics, as well as their mutual relationships. The article focuses on the algorithm for Dialogue Management that selects the most appropriate conversation topic depe…
▽ More
The article proposes a system for knowledge-based conversation designed for Social Robots and other conversational agents. The proposed system relies on an Ontology for the description of all concepts that may be relevant conversation topics, as well as their mutual relationships. The article focuses on the algorithm for Dialogue Management that selects the most appropriate conversation topic depending on the user's input. Moreover, it discusses strategies to ensure a conversation flow that captures, as more coherently as possible, the user's intention to drive the conversation in specific directions while avoiding purely reactive responses to what the user says. To measure the quality of the conversation, the article reports the tests performed with 100 recruited participants, comparing five conversational agents: (i) an agent addressing dialogue flow management based only on the detection of keywords in the speech, (ii) an agent based both on the detection of keywords and the Content Classification feature of Google Cloud Natural Language, (iii) an agent that picks conversation topics randomly, (iv) a human pretending to be a chatbot, and (v) one of the most famous chatbots worldwide: Replika. The subjective perception of the participants is measured both with the SASSI (Subjective Assessment of Speech System Interfaces) tool, as well as with a custom survey for measuring the subjective perception of coherence.
△ Less
Submitted 4 August, 2021;
originally announced August 2021.
-
Knowledge Triggering, Extraction and Storage via Human-Robot Verbal Interaction
Authors:
Lucrezia Grassi,
Carmine Tommaso Recchiuto,
Antonio Sgorbissa
Abstract:
This article describes a novel approach to expand in run-time the knowledge base of an Artificial Conversational Agent. A technique for automatic knowledge extraction from the user's sentence and four methods to insert the new acquired concepts in the knowledge base have been developed and integrated into a system that has already been tested for knowledge-based conversation between a social human…
▽ More
This article describes a novel approach to expand in run-time the knowledge base of an Artificial Conversational Agent. A technique for automatic knowledge extraction from the user's sentence and four methods to insert the new acquired concepts in the knowledge base have been developed and integrated into a system that has already been tested for knowledge-based conversation between a social humanoid robot and residents of care homes. The run-time addition of new knowledge allows overcoming some limitations that affect most robots and chatbots: the incapability of engaging the user for a long time due to the restricted number of conversation topics. The insertion in the knowledge base of new concepts recognized in the user's sentence is expected to result in a wider range of topics that can be covered during an interaction, making the conversation less repetitive. Two experiments are presented to assess the performance of the knowledge extraction technique, and the efficiency of the developed insertion methods when adding several concepts in the Ontology.
△ Less
Submitted 22 April, 2021;
originally announced April 2021.
-
Caring robots are here to help
Authors:
Irena Papadopoulos,
Antonio Sgorbissa,
Christina Koulouglioti
Abstract:
Robots, along with sensors and telemedicine, have been identified as technologies that can assist and prolong independent living for older people, with robots especially being used to help prevent social isolation and depression.
Robots, along with sensors and telemedicine, have been identified as technologies that can assist and prolong independent living for older people, with robots especially being used to help prevent social isolation and depression.
△ Less
Submitted 22 March, 2018;
originally announced March 2018.
-
Paving the Way for Culturally Competent Robots: a Position Paper
Authors:
Barbara Bruno,
Nak Young Chong,
Hiroko Kamide,
Sanjeev Kanoria,
Jaeryoung Lee,
Yuto Lim,
Amit Kumar Pandey,
Chris Papadopoulos,
Irena Papadopoulos,
Federico Pecora,
Alessandro Saffiotti,
Antonio Sgorbissa
Abstract:
Cultural competence is a well known requirement for an effective healthcare, widely investigated in the nursing literature. We claim that personal assistive robots should likewise be culturally competent, aware of general cultural characteristics and of the different forms they take in different individuals, and sensitive to cultural differences while perceiving, reasoning, and acting. Drawing ins…
▽ More
Cultural competence is a well known requirement for an effective healthcare, widely investigated in the nursing literature. We claim that personal assistive robots should likewise be culturally competent, aware of general cultural characteristics and of the different forms they take in different individuals, and sensitive to cultural differences while perceiving, reasoning, and acting. Drawing inspiration from existing guidelines for culturally competent healthcare and the state-of-the-art in culturally competent robotics, we identify the key robot capabilities which enable culturally competent behaviours and discuss methodologies for their development and evaluation.
△ Less
Submitted 22 March, 2018;
originally announced March 2018.
-
A framework for Culture-aware Robots based on Fuzzy Logic
Authors:
Barbara Bruno,
Fulvio Mastrogiovanni,
Federico Pecora,
Antonio Sgorbissa,
Alessandro Saffiotti
Abstract:
Cultural adaptation, i.e., the matching of a robot's behaviours to the cultural norms and preferences of its user, is a well known key requirement for the success of any assistive application. However, culture-dependent robot behaviours are often implicitly set by designers, thus not allowing for an easy and automatic adaptation to different cultures. This paper presents a method for the design of…
▽ More
Cultural adaptation, i.e., the matching of a robot's behaviours to the cultural norms and preferences of its user, is a well known key requirement for the success of any assistive application. However, culture-dependent robot behaviours are often implicitly set by designers, thus not allowing for an easy and automatic adaptation to different cultures. This paper presents a method for the design of culture-aware robots, that can automatically adapt their behaviour to conform to a given culture. We propose a map** from cultural factors to related parameters of robot behaviours which relies on linguistic variables to encode heterogeneous cultural factors in a uniform formalism, and on fuzzy rules to encode qualitative relations among multiple variables. We illustrate the approach in two practical case studies.
△ Less
Submitted 22 March, 2018;
originally announced March 2018.
-
Modelling the Influence of Cultural Information on Vision-Based Human Home Activity Recognition
Authors:
Roberto Menicatti,
Barbara Bruno,
Antonio Sgorbissa
Abstract:
Daily life activities, such as eating and slee**, are deeply influenced by a person's culture, hence generating differences in the way a same activity is performed by individuals belonging to different cultures. We argue that taking cultural information into account can improve the performance of systems for the automated recognition of human activities. We propose four different solutions to th…
▽ More
Daily life activities, such as eating and slee**, are deeply influenced by a person's culture, hence generating differences in the way a same activity is performed by individuals belonging to different cultures. We argue that taking cultural information into account can improve the performance of systems for the automated recognition of human activities. We propose four different solutions to the problem and present a system which uses a Naive Bayes model to associate cultural information with semantic information extracted from still images. Preliminary experiments with a dataset of images of individuals lying on the floor, slee** on a futon and slee** on a bed suggest that: i) solutions explicitly taking cultural information into account are more accurate than culture-unaware solutions; and ii) the proposed system is a promising starting point for the development of culture-aware Human Activity Recognition methods.
△ Less
Submitted 21 March, 2018;
originally announced March 2018.
-
The CARESSES EU-Japan project: making assistive robots culturally competent
Authors:
Barbara Bruno,
Nak Young Chong,
Hiroko Kamide,
Sanjeev Kanoria,
Jaeryoung Lee,
Yuto Lim,
Amit Kumar Pandey,
Chris Papadopoulos,
Irena Papadopoulos,
Federico Pecora,
Alessandro Saffiotti,
Antonio Sgorbissa
Abstract:
The nursing literature shows that cultural competence is an important requirement for effective healthcare. We claim that personal assistive robots should likewise be culturally competent, that is, they should be aware of general cultural characteristics and of the different forms they take in different individuals, and take these into account while perceiving, reasoning, and acting. The CARESSES…
▽ More
The nursing literature shows that cultural competence is an important requirement for effective healthcare. We claim that personal assistive robots should likewise be culturally competent, that is, they should be aware of general cultural characteristics and of the different forms they take in different individuals, and take these into account while perceiving, reasoning, and acting. The CARESSES project is an Europe-Japan collaborative effort that aims at designing, develo** and evaluating culturally competent assistive robots. These robots will be able to adapt the way they behave, speak and interact to the cultural identity of the person they assist. This paper describes the approach taken in the CARESSES project, its initial steps, and its future plans.
△ Less
Submitted 21 August, 2017;
originally announced August 2017.
