Showing 1–6 of 6 results for author: Mosca, E

Simpler becomes Harder: Do LLMs Exhibit a Coherent Behavior on Simplified Corpora?

Authors: Miriam Anschütz, Edoardo Mosca, Georg Groh

Abstract: Text simplification seeks to improve readability while retaining the original content and meaning. Our study investigates whether pre-trained classifiers also maintain such coherence by comparing their predictions on both original and simplified inputs. We conduct experiments using 11 pre-trained models, including BERT and OpenAI's GPT 3.5, across six datasets spanning three languages. Additionall… ▽ More Text simplification seeks to improve readability while retaining the original content and meaning. Our study investigates whether pre-trained classifiers also maintain such coherence by comparing their predictions on both original and simplified inputs. We conduct experiments using 11 pre-trained models, including BERT and OpenAI's GPT 3.5, across six datasets spanning three languages. Additionally, we conduct a detailed analysis of the correlation between prediction change rates and simplification types/strengths. Our findings reveal alarming inconsistencies across all languages and models. If not promptly addressed, simplified inputs can be easily exploited to craft zero-iteration model-agnostic adversarial attacks with success rates of up to 50% △ Less

Submitted 10 April, 2024; originally announced April 2024.

Comments: Published at DeTermIt! Workshop at LREC-COLING 2024

IFAN: An Explainability-Focused Interaction Framework for Humans and NLP Models

Authors: Edoardo Mosca, Daryna Dementieva, Tohid Ebrahim Ajdari, Maximilian Kummeth, Kirill Gringauz, Yutong Zhou, Georg Groh

Abstract: Interpretability and human oversight are fundamental pillars of deploying complex NLP models into real-world applications. However, applying explainability and human-in-the-loop methods requires technical proficiency. Despite existing toolkits for model understanding and analysis, options to integrate human feedback are still limited. We propose IFAN, a framework for real-time explanation-based in… ▽ More Interpretability and human oversight are fundamental pillars of deploying complex NLP models into real-world applications. However, applying explainability and human-in-the-loop methods requires technical proficiency. Despite existing toolkits for model understanding and analysis, options to integrate human feedback are still limited. We propose IFAN, a framework for real-time explanation-based interaction with NLP models. Through IFAN's interface, users can provide feedback to selected model explanations, which is then integrated through adapter layers to align the model with human rationale. We show the system to be effective in debiasing a hate speech classifier with minimal impact on performance. IFAN also offers a visual admin system and API to manage models (and datasets) as well as control access rights. A demo is live at https://ifan.ml. △ Less

Submitted 2 October, 2023; v1 submitted 6 March, 2023; originally announced March 2023.

Comments: Accepted to AACL 2023 Demonstration systems Track

An affective and adaptive educational robot

Authors: Cristina Gena, Alberto Lillo, Claudio Mattutino, Enrico Mosca

Abstract: In this paper we present an educational robot called Wolly, designed to engage children in an affective and social interaction. Indeed, we are now focusing on its role as an educational and affective robot capable of being controlled by coding instructions and at the same time interacting verbally and affectively with children by recognizing their emotions and remembering their interests, and adap… ▽ More In this paper we present an educational robot called Wolly, designed to engage children in an affective and social interaction. Indeed, we are now focusing on its role as an educational and affective robot capable of being controlled by coding instructions and at the same time interacting verbally and affectively with children by recognizing their emotions and remembering their interests, and adapting its behavior accordingly. △ Less

Submitted 20 May, 2022; originally announced May 2022.

Comments: extened version of the paper Wolly: An affective and adaptive educational robot, submitted to CAESAR 2022. arXiv admin note: text overlap with arXiv:2203.06439

"That Is a Suspicious Reaction!": Interpreting Logits Variation to Detect NLP Adversarial Attacks

Authors: Edoardo Mosca, Shreyash Agarwal, Javier Rando, Georg Groh

Abstract: Adversarial attacks are a major challenge faced by current machine learning research. These purposely crafted inputs fool even the most advanced models, precluding their deployment in safety-critical applications. Extensive research in computer vision has been carried to develop reliable defense strategies. However, the same issue remains less explored in natural language processing. Our work pres… ▽ More Adversarial attacks are a major challenge faced by current machine learning research. These purposely crafted inputs fool even the most advanced models, precluding their deployment in safety-critical applications. Extensive research in computer vision has been carried to develop reliable defense strategies. However, the same issue remains less explored in natural language processing. Our work presents a model-agnostic detector of adversarial text examples. The approach identifies patterns in the logits of the target classifier when perturbing the input text. The proposed detector improves the current state-of-the-art performance in recognizing adversarial inputs and exhibits strong generalization capabilities across different NLP models, datasets, and word-level attacks. △ Less

Submitted 29 June, 2023; v1 submitted 10 April, 2022; originally announced April 2022.

Comments: ACL 2022

An end-user coding-based environment for programming an educational affective robot

Authors: Cristina Gena, Claudio Mattutino, Enrico Mosca, Alberto Lillo

Abstract: In this paper we present an open source educational robot, designed both to engage children in an affective and social interaction, and to be programmable also in its social and affective behaviour. Indeed the robot, in addition to classic programming tasks, can also be programmed as a social robot. In addition to movements, the user can make the robot express emotions and make it say things. The… ▽ More In this paper we present an open source educational robot, designed both to engage children in an affective and social interaction, and to be programmable also in its social and affective behaviour. Indeed the robot, in addition to classic programming tasks, can also be programmed as a social robot. In addition to movements, the user can make the robot express emotions and make it say things. The robot can also be left in autonomous mode, in which it is able to carry out both biometric user's features and emotion recognition, and greeting the user. △ Less

Submitted 12 March, 2022; originally announced March 2022.

Educational robotics for children and their teachers

Authors: Cristina Gena, Claudio Mattutino, Davide Cellie, Enrico Mosca

Abstract: This paper describes a Google Educator funded project devoted to the training of teachers (primary and secondary school) through an e-learning platform that will introduce them to educational robotics using Wolly, a social, educational and affective robot. This paper describes a Google Educator funded project devoted to the training of teachers (primary and secondary school) through an e-learning platform that will introduce them to educational robotics using Wolly, a social, educational and affective robot. △ Less

Submitted 1 May, 2022; v1 submitted 16 November, 2020; originally announced November 2020.

Comments: 3 pages extracted from an unpublished work presented at Child-Robot Interaction workshop: Present and Future Relationships