-
GPT-Powered Elicitation Interview Script Generator for Requirements Engineering Training
Authors:
Binnur Görer,
Fatma Başak Aydemir
Abstract:
Elicitation interviews are the most common requirements elicitation technique, and proficiency in conducting these interviews is crucial for requirements elicitation. Traditional training methods, typically limited to textbook learning, may not sufficiently address the practical complexities of interviewing techniques. Practical training with various interview scenarios is important for understand…
▽ More
Elicitation interviews are the most common requirements elicitation technique, and proficiency in conducting these interviews is crucial for requirements elicitation. Traditional training methods, typically limited to textbook learning, may not sufficiently address the practical complexities of interviewing techniques. Practical training with various interview scenarios is important for understanding how to apply theoretical knowledge in real-world contexts. However, there is a shortage of educational interview material, as creating interview scripts requires both technical expertise and creativity. To address this issue, we develop a specialized GPT agent for auto-generating interview scripts. The GPT agent is equipped with a dedicated knowledge base tailored to the guidelines and best practices of requirements elicitation interview procedures. We employ a prompt chaining approach to mitigate the output length constraint of GPT to be able to generate thorough and detailed interview scripts. This involves dividing the interview into sections and crafting distinct prompts for each, allowing for the generation of complete content for each section. The generated scripts are assessed through standard natural language generation evaluation metrics and an expert judgment study, confirming their applicability in requirements engineering training.
△ Less
Submitted 17 June, 2024;
originally announced June 2024.
-
Comparison of Machine Learning Methods for Assigning Software Issues to Team Members
Authors:
Büşra Tabak,
Fatma Başak Aydemir
Abstract:
Software issues contain units of work to fix, improve, or create new threads during the development and facilitate communication among the team members. Assigning an issue to the most relevant team member and determining a category of an issue is a tedious and challenging task. Wrong classifications cause delays and rework in the project and trouble among the team members. This paper proposes a se…
▽ More
Software issues contain units of work to fix, improve, or create new threads during the development and facilitate communication among the team members. Assigning an issue to the most relevant team member and determining a category of an issue is a tedious and challenging task. Wrong classifications cause delays and rework in the project and trouble among the team members. This paper proposes a set of carefully curated linguistic features for shallow machine learning methods and compares the performance of shallow and ensemble methods with deep language models. Unlike the state-of-the-art, we assign issues to four roles (designer, developer, tester, and leader) rather than to specific individuals or teams to contribute to the generality of our solution. We also consider the level of experience of the developers to reflect the industrial practices in our solution formulation. We collect and annotate five industrial data sets from one of the top three global television producers to evaluate our proposal and compare it with deep language models. Our data sets contain 5324 issues in total. We show that an ensemble classifier of shallow techniques achieves 0.92 for issue assignment in accuracy which is statistically comparable to the state-of-the-art deep language models. The contributions include the public sharing of five annotated industrial issue data sets, the development of a clear and comprehensive feature set, the introduction of a novel label set, and the validation of the efficacy of an ensemble classifier of shallow machine learning techniques.
△ Less
Submitted 23 August, 2023; v1 submitted 18 June, 2023;
originally announced July 2023.
-
Exploring Emerging Technologies for Requirements Elicitation Interview Training: Empirical Assessment of Robotic and Virtual Tutors
Authors:
Binnur Görer,
Fatma Başak Aydemir
Abstract:
Requirements elicitation interviews are a widely adopted technique, where the interview success heavily depends on the interviewer's preparedness and communication skills. Students can enhance these skills through practice interviews. However, organizing practice interviews for many students presents scalability challenges, given the time and effort required to involve stakeholders in each session…
▽ More
Requirements elicitation interviews are a widely adopted technique, where the interview success heavily depends on the interviewer's preparedness and communication skills. Students can enhance these skills through practice interviews. However, organizing practice interviews for many students presents scalability challenges, given the time and effort required to involve stakeholders in each session. To address this, we propose REIT, an extensible architecture for Requirements Elicitation Interview Training system based on emerging educational technologies. REIT has components to support both the interview phase, wherein students act as interviewers while the system assumes the role of an interviewee, and the feedback phase, during which the system assesses students' performance and offers contextual and behavioral feedback to enhance their interviewing skills. We demonstrate the applicability of REIT through two implementations: RoREIT with a physical robotic agent and VoREIT with a virtual voice-only agent. We empirically evaluated both instances with a group of graduate students. The participants appreciated both systems. They demonstrated higher learning gain when trained with RoREIT, but they found VoREIT more engaging and easier to use. These findings indicate that each system has distinct benefits and drawbacks, suggesting that REIT can be realized for various educational settings based on preferences and available resources.
△ Less
Submitted 30 August, 2023; v1 submitted 28 April, 2023;
originally announced May 2023.
-
Replication in Requirements Engineering: the NLP for RE Case
Authors:
Sallam Abualhaija,
F. BaŞAk Aydemir,
Fabiano Dalpiaz,
Davide Dell'Anna,
Alessio Ferrari,
Xavier Franch,
Davide Fucci
Abstract:
[Context]} Natural language processing (NLP) techniques have been widely applied in the requirements engineering (RE) field to support tasks such as classification and ambiguity detection. Despite its empirical vocation, RE research has given limited attention to replication of NLP for RE studies. Replication is hampered by several factors, including the context specificity of the studies, the het…
▽ More
[Context]} Natural language processing (NLP) techniques have been widely applied in the requirements engineering (RE) field to support tasks such as classification and ambiguity detection. Despite its empirical vocation, RE research has given limited attention to replication of NLP for RE studies. Replication is hampered by several factors, including the context specificity of the studies, the heterogeneity of the tasks involving NLP, the tasks' inherent hairiness, and, in turn, the heterogeneous reporting structure. [Solution] To address these issues, we propose a new artifact, referred to as ID-Card, whose goal is to provide a structured summary of research papers emphasizing replication-relevant information. We construct the ID-Card through a structured, iterative process based on design science. [Results] In this paper: (i) we report on hands-on experiences of replication, (ii) we review the state-of-the-art and extract replication-relevant information, (iii) we identify, through focus groups, challenges across two typical dimensions of replication: data annotation and tool reconstruction, and (iv) we present the concept and structure of the ID-Card to mitigate the identified challenges. [Contribution] This study aims to create awareness of replication in NLP for RE. We propose an ID-Card that is intended to foster study replication, but can also be used in other contexts, e.g., for educational purposes.
△ Less
Submitted 18 April, 2024; v1 submitted 20 April, 2023;
originally announced April 2023.
-
RoboREIT: an Interactive Robotic Tutor with Instructive Feedback Component for Requirements Elicitation Interview Training
Authors:
Binnur Görer,
Fatma Başak Aydemir
Abstract:
[Context] Interviewing stakeholders is the most popular requirements elicitation technique among multiple methods. The success of an interview depends on the collaboration of the interviewee which can be fostered through the interviewer's preparedness and communication skills. Mastering these skills requires experience and practicing interviews. [Problem] Practical training is resource-heavy as it…
▽ More
[Context] Interviewing stakeholders is the most popular requirements elicitation technique among multiple methods. The success of an interview depends on the collaboration of the interviewee which can be fostered through the interviewer's preparedness and communication skills. Mastering these skills requires experience and practicing interviews. [Problem] Practical training is resource-heavy as it calls for the time and effort of a stakeholder for each student which may not be feasible for a large number of students. [Method] To address this scalability problem, this paper proposes RoboREIT, an interactive Robotic tutor for Requirements Elicitation Interview Training. The humanoid robotic component of RoboREIT responds to the questions of the interviewer, which the interviewer chooses from a set of predefined alternatives for a particular scenario. After the interview session, RoboREIT provides contextual feedback to the interviewer on their performance and allows the student to inspect their mistakes. RoboREIT is extensible with various scenarios. [Results] We performed an exploratory user study to evaluate RoboREIT and demonstrate its applicability in requirements elicitation interview training. The quantitative and qualitative analyses of the users' responses reveal the appreciation of RoboREIT and provide further suggestions about how to improve it. [Contribution] Our study is the first in the literature that utilizes a social robot in requirements elicitation interview education. RoboREIT's innovative design incorporates replaying faulty interview stages and allows the student to learn from mistakes by a second time practicing. All participants praised the feedback component, which is not present in the state-of-the-art, for being helpful in identifying the mistakes. A favorable response rate of 81% for the system's usefulness indicates the positive perception of the participants.
△ Less
Submitted 15 April, 2023;
originally announced April 2023.
-
A Framework To Improve User Story Sets Through Collaboration
Authors:
Salih Göktuğ Köse,
Fatma Başak Aydemir
Abstract:
Agile methodologies have become increasingly popular in recent years. Due to its inherent nature, agile methodologies involve stakeholders with a wide range of expertise and require interaction between them, relying on collaboration and customer involvement. Hence, agile methodologies encourage collaboration between all team members so that more efficient and effective processes are maintained. Ge…
▽ More
Agile methodologies have become increasingly popular in recent years. Due to its inherent nature, agile methodologies involve stakeholders with a wide range of expertise and require interaction between them, relying on collaboration and customer involvement. Hence, agile methodologies encourage collaboration between all team members so that more efficient and effective processes are maintained. Generating requirements can be challenging, as it requires the participation of multiple stakeholders who describe various aspects of the project and possess a shared understanding of essential concepts. One simple method for capturing requirements using natural language is through user stories, which document the agreed-upon properties of a project. Stakeholders try to strive for completeness while generating user stories, but the final user story set may still be flawed. To address this issue, we propose SCOUT: Supporting Completeness of User Story Sets, which employs a natural language processing pipeline to extract key concepts from user stories and construct a knowledge graph by connecting related terms. The knowledge graph and different heuristics are then utilized to enhance the quality and completeness of the user story sets by generating suggestions for the stakeholders. We perform a user study to evaluate SCOUT and demonstrate its performance in constructing user stories. The quantitative and qualitative results indicate that SCOUT significantly enhance the quality and completeness of the user story sets. Our contribution is threefold. First, we develop heuristics to suggest new concepts to include in user stories by considering both the individuals' and other team members' contributions. Second, we implement an open-source collaborative tool to support writing user stories and ensuring their quality. Third, we share the experimental setup and materials.
△ Less
Submitted 24 January, 2023;
originally announced January 2023.
-
TempSAL -- Uncovering Temporal Information for Deep Saliency Prediction
Authors:
Bahar Aydemir,
Ludo Hoffstetter,
Tong Zhang,
Mathieu Salzmann,
Sabine Süsstrunk
Abstract:
Deep saliency prediction algorithms complement the object recognition features, they typically rely on additional information, such as scene context, semantic relationships, gaze direction, and object dissimilarity. However, none of these models consider the temporal nature of gaze shifts during image observation. We introduce a novel saliency prediction model that learns to output saliency maps i…
▽ More
Deep saliency prediction algorithms complement the object recognition features, they typically rely on additional information, such as scene context, semantic relationships, gaze direction, and object dissimilarity. However, none of these models consider the temporal nature of gaze shifts during image observation. We introduce a novel saliency prediction model that learns to output saliency maps in sequential time intervals by exploiting human temporal attention patterns. Our approach locally modulates the saliency predictions by combining the learned temporal maps. Our experiments show that our method outperforms the state-of-the-art models, including a multi-duration saliency model, on the SALICON benchmark. Our code will be publicly available on GitHub.
△ Less
Submitted 5 January, 2023;
originally announced January 2023.
-
PulseImpute: A Novel Benchmark Task for Pulsative Physiological Signal Imputation
Authors:
Maxwell A. Xu,
Alexander Moreno,
Supriya Nagesh,
V. Burak Aydemir,
David W. Wetter,
Santosh Kumar,
James M. Rehg
Abstract:
The promise of Mobile Health (mHealth) is the ability to use wearable sensors to monitor participant physiology at high frequencies during daily life to enable temporally-precise health interventions. However, a major challenge is frequent missing data. Despite a rich imputation literature, existing techniques are ineffective for the pulsative signals which comprise many mHealth applications, and…
▽ More
The promise of Mobile Health (mHealth) is the ability to use wearable sensors to monitor participant physiology at high frequencies during daily life to enable temporally-precise health interventions. However, a major challenge is frequent missing data. Despite a rich imputation literature, existing techniques are ineffective for the pulsative signals which comprise many mHealth applications, and a lack of available datasets has stymied progress. We address this gap with PulseImpute, the first large-scale pulsative signal imputation challenge which includes realistic mHealth missingness models, an extensive set of baselines, and clinically-relevant downstream tasks. Our baseline models include a novel transformer-based architecture designed to exploit the structure of pulsative signals. We hope that PulseImpute will enable the ML community to tackle this significant and challenging task.
△ Less
Submitted 15 December, 2023; v1 submitted 14 December, 2022;
originally announced December 2022.
-
Modeling Object Dissimilarity for Deep Saliency Prediction
Authors:
Bahar Aydemir,
Deblina Bhattacharjee,
Tong Zhang,
Seungryong Kim,
Mathieu Salzmann,
Sabine Süsstrunk
Abstract:
Saliency prediction has made great strides over the past two decades, with current techniques modeling low-level information, such as color, intensity and size contrasts, and high-level ones, such as attention and gaze direction for entire objects. Despite this, these methods fail to account for the dissimilarity between objects, which affects human visual attention. In this paper, we introduce a…
▽ More
Saliency prediction has made great strides over the past two decades, with current techniques modeling low-level information, such as color, intensity and size contrasts, and high-level ones, such as attention and gaze direction for entire objects. Despite this, these methods fail to account for the dissimilarity between objects, which affects human visual attention. In this paper, we introduce a detection-guided saliency prediction network that explicitly models the differences between multiple objects, such as their appearance and size dissimilarities. Our approach allows us to fuse our object dissimilarities with features extracted by any deep saliency prediction network. As evidenced by our experiments, this consistently boosts the accuracy of the baseline networks, enabling us to outperform the state-of-the-art models on three saliency benchmarks, namely SALICON, MIT300 and CAT2000. Our project page is at https://github.com/IVRL/DisSal.
△ Less
Submitted 24 November, 2022; v1 submitted 8 April, 2021;
originally announced April 2021.
-
Accelerating Translational Image Registration for HDR Images on GPU
Authors:
Kadir Cenk Alpay,
Kadir Berkay Aydemir,
Alptekin Temizel
Abstract:
High Dynamic Range (HDR) images are generated using multiple exposures of a scene. When a hand-held camera is used to capture a static scene, these images need to be aligned by globally shifting each image in both dimensions. For a fast and robust alignment, the shift amount is commonly calculated using Median Threshold Bitmaps (MTB) and creating an image pyramid. In this study, we optimize these…
▽ More
High Dynamic Range (HDR) images are generated using multiple exposures of a scene. When a hand-held camera is used to capture a static scene, these images need to be aligned by globally shifting each image in both dimensions. For a fast and robust alignment, the shift amount is commonly calculated using Median Threshold Bitmaps (MTB) and creating an image pyramid. In this study, we optimize these computations using a parallel processing approach utilizing GPU. Experimental evaluation shows that the proposed implementation achieves a speed-up of up to 6.24 times over the baseline multi-threaded CPU implementation on the alignment of one image pair. The source code is available at https://github.com/kadircenk/WardMTBCuda
△ Less
Submitted 13 July, 2020;
originally announced July 2020.
