-
Unsupervised Topic Discovery in User Comments
Authors:
Christoph Stanik,
Tim Pietz,
Walid Maalej
Abstract:
On social media platforms like Twitter, users regularly share their opinions and comments with software vendors and service providers. Popular software products might get thousands of user comments per day. Research has shown that such comments contain valuable information for stakeholders, such as feature ideas, problem reports, or support inquiries. However, it is hard to manually manage and gra…
▽ More
On social media platforms like Twitter, users regularly share their opinions and comments with software vendors and service providers. Popular software products might get thousands of user comments per day. Research has shown that such comments contain valuable information for stakeholders, such as feature ideas, problem reports, or support inquiries. However, it is hard to manually manage and grasp a large amount of user comments, which can be redundant and of a different quality. Consequently, researchers suggested automated approaches to extract valuable comments, e.g., through problem report classifiers. However, these approaches do not aggregate semantically similar comments into specific aspects to provide insights like how often users reported a certain problem.
We introduce an approach for automatically discovering topics composed of semantically similar user comments based on deep bidirectional natural language processing algorithms. Stakeholders can use our approach without the need to configure critical parameters like the number of clusters. We present our approach and report on a rigorous multiple-step empirical evaluation to assess how cohesive and meaningful the resulting clusters are. Each evaluation step was peer-coded and resulted in inter-coder agreements of up to 98%, giving us high confidence in the approach. We also report a thematic analysis on the topics discovered from tweets in the telecommunication domain.
△ Less
Submitted 19 August, 2021;
originally announced August 2021.
-
Lessons Learned from Customizing and Applying ACTA to Design a Novel Device for Emergency Medical Care
Authors:
Christoph Stanik,
Tim Puhlfürß,
Anne Mahler,
Phillip Brenya Sasu,
Wikhart Reip,
Walid Maalej
Abstract:
Preclinical patient care is both mentally and physically challenging and exhausting for emergency teams. The teams intensively use medical technology to help the patient on site. However, they must carry and handle multiple heavy medical devices such as a monitor for the patient's vital signs, a ventilator to support an unconscious patient, and a resuscitation device. In an industry project, we ai…
▽ More
Preclinical patient care is both mentally and physically challenging and exhausting for emergency teams. The teams intensively use medical technology to help the patient on site. However, they must carry and handle multiple heavy medical devices such as a monitor for the patient's vital signs, a ventilator to support an unconscious patient, and a resuscitation device. In an industry project, we aim at develo** a combined device that lowers the emergency teams' mental and physical load caused by multiple screens, devices, and their high weight. The focus of this paper is to describe our ideation and requirements elicitation process regarding the user interface design of the combined device. For one year, we applied a fully digital customized version of the Applied Cognitive Task Analysis (ACTA) method to systematically elicit the requirements. Domain and requirements engineering experts created a detailed hierarchical task diagram of an extensive emergency scenario, conducted eleven interviews with subject matter experts (SMEs), and executed two design workshops, which led to 34 sketches and three mockups of the combined device's user interface. Cross-functional teams accompanied the entire process and brought together expertise in preclinical patient care, requirements engineering, and medical product development. We report on the lessons learned for each of the four consecutive stages of our customized ACTA process.
△ Less
Submitted 4 August, 2022; v1 submitted 12 August, 2021;
originally announced August 2021.
-
Automatically Matching Bug Reports With Related App Reviews
Authors:
Marlo Häring,
Christoph Stanik,
Walid Maalej
Abstract:
App stores allow users to give valuable feedback on apps, and developers to find this feedback and use it for the software evolution. However, finding user feedback that matches existing bug reports in issue trackers is challenging as users and developers often use a different language. In this work, we introduce DeepMatcher, an automatic approach using state-of-the-art deep learning methods to ma…
▽ More
App stores allow users to give valuable feedback on apps, and developers to find this feedback and use it for the software evolution. However, finding user feedback that matches existing bug reports in issue trackers is challenging as users and developers often use a different language. In this work, we introduce DeepMatcher, an automatic approach using state-of-the-art deep learning methods to match problem reports in app reviews to bug reports in issue trackers. We evaluated DeepMatcher with four open-source apps quantitatively and qualitatively. On average, DeepMatcher achieved a hit ratio of 0.71 and a Mean Average Precision of 0.55. For 91 problem reports, DeepMatcher did not find any matching bug report. When manually analyzing these 91 problem reports and the issue trackers of the studied apps, we found that in 47 cases, users actually described a problem before developers discovered and documented it in the issue tracker. We discuss our findings and different use cases for DeepMatcher.
△ Less
Submitted 14 February, 2021;
originally announced February 2021.
-
Requirements Intelligence with OpenReq Analytics
Authors:
Christoph Stanik,
Walid Maalej
Abstract:
With the rise of social media like Twitter and distribution platforms like app stores, users have various ways to express their opinions about software products. Popular software vendors get user feedback thousandfold per day. Research has shown that such feedback contains valuable information for software development teams. However, a manual analysis of user feedback is cumbersome and hard to man…
▽ More
With the rise of social media like Twitter and distribution platforms like app stores, users have various ways to express their opinions about software products. Popular software vendors get user feedback thousandfold per day. Research has shown that such feedback contains valuable information for software development teams. However, a manual analysis of user feedback is cumbersome and hard to manage. We present OpenReq Analytics, a software requirements intelligence service, that collects, processes, analyzes, and visualizes user feedback.
△ Less
Submitted 12 September, 2019;
originally announced September 2019.
-
Classifying Multilingual User Feedback using Traditional Machine Learning and Deep Learning
Authors:
Christoph Stanik,
Marlo Haering,
Walid Maalej
Abstract:
With the rise of social media like Twitter and of software distribution platforms like app stores, users got various ways to express their opinion about software products. Popular software vendors get user feedback thousandfold per day. Research has shown that such feedback contains valuable information for software development teams such as problem reports or feature and support inquires. Since t…
▽ More
With the rise of social media like Twitter and of software distribution platforms like app stores, users got various ways to express their opinion about software products. Popular software vendors get user feedback thousandfold per day. Research has shown that such feedback contains valuable information for software development teams such as problem reports or feature and support inquires. Since the manual analysis of user feedback is cumbersome and hard to manage many researchers and tool vendors suggested to use automated analyses based on traditional supervised machine learning approaches. In this work, we compare the results of traditional machine learning and deep learning in classifying user feedback in English and Italian into problem reports, inquiries, and irrelevant. Our results show that using traditional machine learning, we can still achieve comparable results to deep learning, although we collected thousands of labels.
△ Less
Submitted 12 September, 2019;
originally announced September 2019.
-
A Simple NLP-based Approach to Support Onboarding and Retention in Open Source Communities
Authors:
Christoph Stanik,
Lloyd Montgomery,
Daniel Martens,
Davide Fucci,
Walid Maalej
Abstract:
Successful open source communities are constantly looking for new members and hel** them become active developers. A common approach for developer onboarding in open source projects is to let newcomers focus on relevant yet easy-to-solve issues to familiarize themselves with the code and the community. The goal of this research is twofold. First, we aim at automatically identifying issues that n…
▽ More
Successful open source communities are constantly looking for new members and hel** them become active developers. A common approach for developer onboarding in open source projects is to let newcomers focus on relevant yet easy-to-solve issues to familiarize themselves with the code and the community. The goal of this research is twofold. First, we aim at automatically identifying issues that newcomers can resolve by analyzing the history of resolved issues by simply using the title and description of issues. Second, we aim at automatically identifying issues, that can be resolved by newcomers who later become active developers. We mined the issue trackers of three large open source projects and extracted natural language features from the title and description of resolved issues. In a series of experiments, we optimized and compared the accuracy of four supervised classifiers to address our research goals. Random Forest, achieved up to 91% precision (F1-score 72%) towards the first goal while for the second goal, Decision Tree achieved a precision of 92% (F1-score 91%). A qualitative evaluation gave insights on what information in the issue description is helpful for newcomers. Our approach can be used to automatically identify, label, and recommend issues for newcomers in open source software projects based only on the text of the issues.
△ Less
Submitted 16 August, 2018; v1 submitted 7 June, 2018;
originally announced June 2018.
