-
Explanations in Everyday Software Systems: Towards a Taxonomy for Explainability Needs
Authors:
Jakob Droste,
Hannah Deters,
Martin Obaidi,
Kurt Schneider
Abstract:
Modern software systems are becoming increasingly complex and opaque. The integration of explanations within software has shown the potential to address this opacity and can make the system more understandable to end-users. As a result, explainability has gained much traction as a non-functional requirement of complex systems. Understanding what type of system requires what types of explanations i…
▽ More
Modern software systems are becoming increasingly complex and opaque. The integration of explanations within software has shown the potential to address this opacity and can make the system more understandable to end-users. As a result, explainability has gained much traction as a non-functional requirement of complex systems. Understanding what type of system requires what types of explanations is necessary to facilitate the inclusion of explainability in early software design processes. In order to specify explainability requirements, an explainability taxonomy that applies to a variety of different software types is needed. In this paper, we present the results of an online survey with 84 participants. We asked the participants to state their questions and confusions concerning their three most recently used software systems and elicited both explicit and implicit explainability needs from their statements. These needs were coded by three researchers. In total, we identified and classified 315 explainability needs from the survey answers. Drawing from a large pool of explainability needs and our coding procedure, we present two major contributions of this work: 1) a taxonomy for explainability needs in everyday software systems and 2) an overview of how the need for explanations differs between different types of software systems.
△ Less
Submitted 25 April, 2024;
originally announced April 2024.
-
Explanation Needs in App Reviews: Taxonomy and Automated Detection
Authors:
Max Unterbusch,
Mersedeh Sadeghi,
Jannik Fischbach,
Martin Obaidi,
Andreas Vogelsang
Abstract:
Explainability, i.e. the ability of a system to explain its behavior to users, has become an important quality of software-intensive systems. Recent work has focused on methods for generating explanations for various algorithmic paradigms (e.g., machine learning, self-adaptive systems). There is relatively little work on what situations and types of behavior should be explained. There is also a la…
▽ More
Explainability, i.e. the ability of a system to explain its behavior to users, has become an important quality of software-intensive systems. Recent work has focused on methods for generating explanations for various algorithmic paradigms (e.g., machine learning, self-adaptive systems). There is relatively little work on what situations and types of behavior should be explained. There is also a lack of support for eliciting explainability requirements. In this work, we explore the need for explanation expressed by users in app reviews. We manually coded a set of 1,730 app reviews from 8 apps and derived a taxonomy of Explanation Needs. We also explore several approaches to automatically identify Explanation Needs in app reviews. Our best classifier identifies Explanation Needs in 486 unseen reviews of 4 different apps with a weighted F-score of 86%. Our work contributes to a better understanding of users' Explanation Needs. Automated tools can help engineers focus on these needs and ultimately elicit valid Explanation Needs.
△ Less
Submitted 10 July, 2023;
originally announced July 2023.
-
Faked Speech Detection with Zero Prior Knowledge
Authors:
Sahar Al Ajmi,
Khizar Hayat,
Alaa M. Al Obaidi,
Naresh Kumar,
Munaf Najmuldeen,
Baptiste Magnier
Abstract:
Audio is one of the most used ways of human communication, but at the same time it can be easily misused to trick people. With the revolution of AI, the related technologies are now accessible to almost everyone, thus making it simple for the criminals to commit crimes and forgeries. In this work, we introduce a neural network method to develop a classifier that will blindly classify an input audi…
▽ More
Audio is one of the most used ways of human communication, but at the same time it can be easily misused to trick people. With the revolution of AI, the related technologies are now accessible to almost everyone, thus making it simple for the criminals to commit crimes and forgeries. In this work, we introduce a neural network method to develop a classifier that will blindly classify an input audio as real or mimicked; the word 'blindly' refers to the ability to detect mimicked audio without references or real sources. We propose a deep neural network following a sequential model that comprises three hidden layers, with alternating dense and drop out layers. The proposed model was trained on a set of 26 important features extracted from a large dataset of audios to get a classifier that was tested on the same set of features from different audios. The data was extracted from two raw datasets, especially composed for this work; an all English dataset and a mixed dataset (Arabic plus English) (The dataset can be provided, in raw form, by writing an email to the first author). For the purpose of comparison, the audios were also classified through human inspection with the subjects being the native speakers. The ensued results were interesting and exhibited formidable accuracy, as we were able to get at least 94% correct classification of the test cases, as against the 85% accuracy in the case of human observers.
△ Less
Submitted 2 April, 2024; v1 submitted 26 September, 2022;
originally announced September 2022.
-
On the Subjectivity of Emotions in Software Projects: How Reliable are Pre-Labeled Data Sets for Sentiment Analysis?
Authors:
Marc Herrmann,
Martin Obaidi,
Larissa Chazette,
Jil Klünder
Abstract:
Social aspects of software projects become increasingly important for research and practice. Different approaches analyze the sentiment of a development team, ranging from simply asking the team to so-called sentiment analysis on text-based communication. These sentiment analysis tools are trained using pre-labeled data sets from different sources, including GitHub and Stack Overflow. In this pape…
▽ More
Social aspects of software projects become increasingly important for research and practice. Different approaches analyze the sentiment of a development team, ranging from simply asking the team to so-called sentiment analysis on text-based communication. These sentiment analysis tools are trained using pre-labeled data sets from different sources, including GitHub and Stack Overflow. In this paper, we investigate if the labels of the statements in the data sets coincide with the perception of potential members of a software project team. Based on an international survey, we compare the median perception of 94 participants with the pre-labeled data sets as well as every single participant's agreement with the predefined labels. Our results point to three remarkable findings: (1) Although the median values coincide with the predefined labels of the data sets in 62.5% of the cases, we observe a huge difference between the single participant's ratings and the labels; (2) there is not a single participant who totally agrees with the predefined labels; and (3) the data set whose labels are based on guidelines performs better than the ad hoc labeled data set.
△ Less
Submitted 16 July, 2022;
originally announced July 2022.
-
SEnti-Analyzer: Joint Sentiment Analysis For Text-Based and Verbal Communication in Software Projects
Authors:
Marc Herrmann,
Martin Obaidi,
Jil Klünder
Abstract:
Social aspects in software development teams are of particular importance for a successful project closure. To analyze sentiments in software projects, there are several tools and approaches available. These tools analyze text-based communication based on the used words to predict whether they appear to be positive, negative, or neutral for the receiver of the message. In the research project ComC…
▽ More
Social aspects in software development teams are of particular importance for a successful project closure. To analyze sentiments in software projects, there are several tools and approaches available. These tools analyze text-based communication based on the used words to predict whether they appear to be positive, negative, or neutral for the receiver of the message. In the research project ComContA, we investigate so-called sentiment analysis striving to analyze the content of text-based communication in development teams with regard to the statement's polarity. That is, we analyze whether the communication appears to be adequate (i.e., positive or neutral) or negative. In a workshop paper, we presented a tool called SEnti-Analyzer that allows to apply sentiment analysis to verbal communication in meetings of software projects. In this technical report, we present the extended functionalities of the SEnti-Analyzer by also allowing the analysis of text-based communication, we improve the prediction of the tool by including established sentiment analysis tools, and we evaluate the tool with respect to its accuracy. We evaluate the tool by comparing the prediction of the SEnti-Analyzer to pre-labeled established data sets used for sentiment analysis in software engineering and to perceptions of computer scientists. Our results indicate that in almost all cases at least two of the three votes coincide, but in only about half of the cases all three votes coincide. Our results raise the question of the "ultimate truth" of sentiment analysis outcomes: What do we want to predict with sentiment analysis tools? The pre-defined labels of established data sets? The perception of computer scientists? Or the perception of single computer scientists which appears to be the most meaningful objective?
△ Less
Submitted 22 June, 2022;
originally announced June 2022.
-
Development and Application of Sentiment Analysis Tools in Software Engineering: A Systematic Literature Review
Authors:
Martin Obaidi,
Jil Klünder
Abstract:
Software development is a collaborative task and, hence, involves different persons. Research has shown the relevance of social aspects in the development team for a successful and satisfying project closure. Especially the mood of a team has been proven to be of particular importance. Thus, project managers or project leaders want to be aware of situations in which negative mood is present to all…
▽ More
Software development is a collaborative task and, hence, involves different persons. Research has shown the relevance of social aspects in the development team for a successful and satisfying project closure. Especially the mood of a team has been proven to be of particular importance. Thus, project managers or project leaders want to be aware of situations in which negative mood is present to allow for interventions. So-called sentiment analysis tools offer a way to determine the mood based on text-based communication. In this paper, we present the results of a systematic literature review of sentiment analysis tools developed for or applied in the context of software engineering. Our results summarize insights from 80 papers with respect to (1) the application domain, (2) the purpose, (3) the used data sets, (4) the approaches for develo** sentiment analysis tools and (5) the difficulties researchers face when applying sentiment analysis in the context of software projects. According to our results, sentiment analysis is frequently applied to open-source software projects, and most tools are based on support-vector machines. Despite the frequent use of sentiment analysis in software engineering, there are open issues, e.g., regarding the identification of irony or sarcasm, pointing to future research directions.
△ Less
Submitted 6 May, 2021;
originally announced May 2021.
-
Monitoring Targeted Hate in Online Environments
Authors:
Tim Isbister,
Magnus Sahlgren,
Lisa Kaati,
Milan Obaidi,
Nazar Akrami
Abstract:
Hateful comments, swearwords and sometimes even death threats are becoming a reality for many people today in online environments. This is especially true for journalists, politicians, artists, and other public figures. This paper describes how hate directed towards individuals can be measured in online environments using a simple dictionary-based approach. We present a case study on Swedish polit…
▽ More
Hateful comments, swearwords and sometimes even death threats are becoming a reality for many people today in online environments. This is especially true for journalists, politicians, artists, and other public figures. This paper describes how hate directed towards individuals can be measured in online environments using a simple dictionary-based approach. We present a case study on Swedish politicians, and use examples from this study to discuss shortcomings of the proposed dictionary-based approach. We also outline possibilities for potential refinements of the proposed approach.
△ Less
Submitted 13 March, 2018;
originally announced March 2018.
