-
Some things never change: how far generative AI can really change software engineering practice
Authors:
Aline de Campos,
Jorge Melegati,
Nicolas Nascimento,
Rafael Chanin,
Afonso Sales,
Igor Wiese
Abstract:
Generative Artificial Intelligence (GenAI) has become an emerging technology with the availability of several tools that could impact Software Engineering (SE) activities. As any other disruptive technology, GenAI led to the speculation that its full potential can deeply change SE. However, an overfocus on improving activities for which GenAI is more suitable could negligent other relevant areas o…
▽ More
Generative Artificial Intelligence (GenAI) has become an emerging technology with the availability of several tools that could impact Software Engineering (SE) activities. As any other disruptive technology, GenAI led to the speculation that its full potential can deeply change SE. However, an overfocus on improving activities for which GenAI is more suitable could negligent other relevant areas of the process. In this paper, we aim to explore which SE activities are not expected to be profoundly changed by GenAI. To achieve this goal, we performed a survey with SE practitioners to identify their expectations regarding GenAI in SE, including impacts, challenges, ethical issues, and aspects they do not expect to change. We compared our results with previous roadmaps proposed in SE literature. Our results show that although practitioners expect an increase in productivity, coding, and process quality, they envision that some aspects will not change, such as the need for human expertise, creativity, and project management. Our results point to SE areas for which GenAI is probably not so useful, and future research could tackle them to improve SE practice.
△ Less
Submitted 14 June, 2024;
originally announced June 2024.
-
Generative AI for Test Driven Development: Preliminary Results
Authors:
Moritz Mock,
Jorge Melegati,
Barbara Russo
Abstract:
Test Driven Development (TDD) is one of the major practices of Extreme Programming for which incremental testing and refactoring trigger the code development. TDD has limited adoption in the industry, as it requires more code to be developed and experienced developers. Generative AI (GenAI) may reduce the extra effort imposed by TDD. In this work, we introduce an approach to automatize TDD by embr…
▽ More
Test Driven Development (TDD) is one of the major practices of Extreme Programming for which incremental testing and refactoring trigger the code development. TDD has limited adoption in the industry, as it requires more code to be developed and experienced developers. Generative AI (GenAI) may reduce the extra effort imposed by TDD. In this work, we introduce an approach to automatize TDD by embracing GenAI either in a collaborative interaction pattern in which developers create tests and supervise the AI generation during each iteration or a fully-automated pattern in which developers only supervise the AI generation at the end of the iterations. We run an exploratory experiment with ChatGPT in which the interaction patterns are compared with the non-AI TDD regarding test and code quality and development speed. Overall, we found that, for our experiment and settings, GenAI can be efficiently used in TDD, but it requires supervision of the quality of the produced code. In some cases, it can even mislead non-expert developers and propose solutions just for the sake of the query.
△ Less
Submitted 17 May, 2024;
originally announced May 2024.
-
Experimentation in Early-Stage Video Game Startups: Practices and Challenges
Authors:
Henry Edison,
Jorge Melegati,
Elizabeth Bjarnason
Abstract:
Experimentation has been considered critical for successful software product and business development, including in video game startups. Video game startups need "wow" qualities that distinguish them from the competition. Thus, they need to continuously experiment to find these qualities before running out of time and resources. In this study, we aimed to explore how these companies perform experi…
▽ More
Experimentation has been considered critical for successful software product and business development, including in video game startups. Video game startups need "wow" qualities that distinguish them from the competition. Thus, they need to continuously experiment to find these qualities before running out of time and resources. In this study, we aimed to explore how these companies perform experimentation. We interviewed four co-founders of video game startups. Our findings identify six practices, or scenarios, through which video game startups conduct experiments and challenges associated with these. The initial results could inform these startups about the possibilities and challenges and guide future research.
△ Less
Submitted 22 November, 2023;
originally announced November 2023.
-
Generative Artificial Intelligence for Software Engineering -- A Research Agenda
Authors:
Anh Nguyen-Duc,
Beatriz Cabrero-Daniel,
Adam Przybylek,
Chetan Arora,
Dron Khanna,
Tomas Herda,
Usman Rafiq,
Jorge Melegati,
Eduardo Guerra,
Kai-Kristian Kemell,
Mika Saari,
Zheying Zhang,
Huy Le,
Tho Quan,
Pekka Abrahamsson
Abstract:
Generative Artificial Intelligence (GenAI) tools have become increasingly prevalent in software development, offering assistance to various managerial and technical project activities. Notable examples of these tools include OpenAIs ChatGPT, GitHub Copilot, and Amazon CodeWhisperer. Although many recent publications have explored and evaluated the application of GenAI, a comprehensive understandin…
▽ More
Generative Artificial Intelligence (GenAI) tools have become increasingly prevalent in software development, offering assistance to various managerial and technical project activities. Notable examples of these tools include OpenAIs ChatGPT, GitHub Copilot, and Amazon CodeWhisperer. Although many recent publications have explored and evaluated the application of GenAI, a comprehensive understanding of the current development, applications, limitations, and open challenges remains unclear to many. Particularly, we do not have an overall picture of the current state of GenAI technology in practical software engineering usage scenarios. We conducted a literature review and focus groups for a duration of five months to develop a research agenda on GenAI for Software Engineering. We identified 78 open Research Questions (RQs) in 11 areas of Software Engineering. Our results show that it is possible to explore the adoption of GenAI in partial automation and support decision-making in all software development activities. While the current literature is skewed toward software implementation, quality assurance and software maintenance, other areas, such as requirements engineering, software design, and software engineering education, would need further research attention. Common considerations when implementing GenAI include industry-level assessment, dependability and accuracy, data accessibility, transparency, and sustainability aspects associated with the technology. GenAI is bringing significant changes to the field of software engineering. Nevertheless, the state of research on the topic still remains immature. We believe that this research agenda holds significance and practical value for informing both researchers and practitioners about current applications and guiding future research.
△ Less
Submitted 28 October, 2023;
originally announced October 2023.
-
DAnTE: a taxonomy for the automation degree of software engineering tasks
Authors:
Jorge Melegati,
Eduardo Guerra
Abstract:
Software engineering researchers and practitioners have pursued manners to reduce the amount of time and effort required to develop code and increase productivity since the emergence of the discipline. Generative language models are just another step in this journey, but it will probably not be the last one. In this chapter, we propose DAnTE, a Degree of Automation Taxonomy for software Engineerin…
▽ More
Software engineering researchers and practitioners have pursued manners to reduce the amount of time and effort required to develop code and increase productivity since the emergence of the discipline. Generative language models are just another step in this journey, but it will probably not be the last one. In this chapter, we propose DAnTE, a Degree of Automation Taxonomy for software Engineering, describing several levels of automation based on the idiosyncrasies of the field. Based on the taxonomy, we evaluated several tools used in the past and in the present for software engineering practices. Then, we give particular attention to AI-based tools, including generative language models, discussing how they are located within the proposed taxonomy, and reasoning about possible limitations they currently have. Based on this analysis, we discuss what novel tools could emerge in the middle and long term.
△ Less
Submitted 26 September, 2023;
originally announced September 2023.
-
Digital Mentor: towards a conversational bot to identify hypotheses for software startups
Authors:
Jorge Melegati,
Xiaofeng Wang
Abstract:
Software startups develop innovative, software-intensive product and services. This context leads to uncertainty regarding the software they are building. Experimentation, a process of testing hypotheses about the product, helps these companies to reduce uncertainty through different evidence-based approaches. The first step in experimentation is to identify the hypotheses to be tested. HyMap is a…
▽ More
Software startups develop innovative, software-intensive product and services. This context leads to uncertainty regarding the software they are building. Experimentation, a process of testing hypotheses about the product, helps these companies to reduce uncertainty through different evidence-based approaches. The first step in experimentation is to identify the hypotheses to be tested. HyMap is a technique where a facilitator helps a software startup founder to draw a cognitive map representing her understanding of the context and, based on that, create hypotheses about the software to be built. In this paper, we present the Digital Mentor, an working-in-progress conversational bot to help creating a HyMap without the need of a human facilitator. We report the proposed solution consisting of a web application with the backend of a natural language understanding system, the current state of development, the challenges we faced so far and the next steps we plan to move forward.
△ Less
Submitted 1 April, 2022;
originally announced April 2022.
-
Work-from-home and its implication for project management, resilience and innovation -- a global survey on software companies
Authors:
Anh Nguyen-Duc,
Dron Khanna,
Des Greer,
Xiaofeng Wang,
Luciana Martinez Zaina,
Gerardo Matturro,
Jorge Melegati,
Eduardo Guerra,
Giang Huong Le,
Petri Kettunen,
Sami Hyrynsalmi,
Henry Edison,
Afonso Sales,
Didzis Rutitis,
Kai-Kristian Kemell,
Abdullah Aldaeej,
Tommi Mikkonen,
Juan Garbajosa,
Pekka Abrahamsson
Abstract:
[Context] The COVID-19 pandemic has had a disruptive impact on how people work and collaborate across all global economic sectors, including the software business. While remote working is not new for software engineers, forced Work-from-home situations to come with both constraints, limitations, and opportunities for individuals, software teams and software companies. As the "new normal" for worki…
▽ More
[Context] The COVID-19 pandemic has had a disruptive impact on how people work and collaborate across all global economic sectors, including the software business. While remote working is not new for software engineers, forced Work-from-home situations to come with both constraints, limitations, and opportunities for individuals, software teams and software companies. As the "new normal" for working might be based on the current state of Work From Home (WFH), it is useful to understand what has happened and learn from that. [Objective] The goal of this study is to gain insights on how their WFH environment impacts software projects and software companies. We are also interested in understanding if the impact differs between software startups and established companies. [Method] We conducted a global-scale, cross-sectional survey during spring and summer 2021. Our results are based on quantitative and qualitative analysis of 297 valid responses. [Results] We observed a mixed perception of the impact of WFH on software project management, resilience, and innovation. Certain patterns on WFH, control and coordination mechanisms and collaborative tools are observed globally. We find that team, agility and leadership are the three most important factors for achieving resilience during the pandemic. Although startups do not perceive the impact of WFH differently, there is a difference between engineers who work in a small team context and those who work in a large team context. [Conclusion] The result suggests a contingency approach in studying and improving WFH practices and environment in the future software industry.
△ Less
Submitted 10 February, 2022;
originally announced February 2022.
-
CADV: A software visualization approach for code annotations distribution
Authors:
Phyllipe Lima,
Jorge Melegati,
Everaldo Gomes,
Nathalya Stefhany Pereira,
Eduardo Guerra,
Paulo Meirelles
Abstract:
Code annotations is a widely used feature in Java systems to configure custom metadata on programming elements. Their increasing presence creates the need for approaches to assess and comprehend their usage and distribution. In this context, software visualization has been studied and researched to improve program comprehension in different aspects. This study aimed at designing a software visuali…
▽ More
Code annotations is a widely used feature in Java systems to configure custom metadata on programming elements. Their increasing presence creates the need for approaches to assess and comprehend their usage and distribution. In this context, software visualization has been studied and researched to improve program comprehension in different aspects. This study aimed at designing a software visualization approach that graphically displays how code annotations are distributed and organized in a software system and develo** a tool, as a reference implementation of the approach, to generate views and interact with users. We conducted an empirical evaluation through questionnaires and interviews to evaluate our visualization approach considering four aspects: effectiveness for program comprehension, perceived usefulness, perceived ease of use, and suitability for the intended audience. The resulting data was used to perform a qualitative and quantitative analysis. The tool identifies package responsibilities providing visual information about their annotations at different levels. Using the developed tool, the participants achieved a high correctness rate in the program comprehension tasks and performed very well in questions about the overview of the system under analysis. Finally, participants perceived that the tool outperforms existing approaches for code inspection when searching for information related to code annotations. The results show that the visualization approach using the developed tool is effective in program comprehension tasks related to code annotations, which can also be used to identify responsibilities in the application packages. Moreover, it was evaluated as suitable for newcomers to overview the usage of annotations in the system and for architects to perform a deep analysis that can potentially detect misplaced annotations and abnormal growths on their usage.
△ Less
Submitted 27 June, 2022; v1 submitted 20 December, 2021;
originally announced December 2021.
-
HyMap: eliciting hypotheses in early-stage software startups using cognitive map**
Authors:
Jorge Melegati,
Eduardo Guerra,
Xiaofeng Wang
Abstract:
Context: Software startups develop innovative, software-intensive products. Given the uncertainty associated with such an innovative context, experimentation is a valuable approach for these companies, especially in the early stages of the development, when implementing unnecessary features represents a higher risk for companies' survival. Nevertheless, researchers have argued that the lack of cle…
▽ More
Context: Software startups develop innovative, software-intensive products. Given the uncertainty associated with such an innovative context, experimentation is a valuable approach for these companies, especially in the early stages of the development, when implementing unnecessary features represents a higher risk for companies' survival. Nevertheless, researchers have argued that the lack of clearly defined practices led to limited adoption of experimentation. In this regard, the first step is to define the hypotheses based on which teams will create experiments. Objective: We aim to develop a systematic technique to identify hypotheses for early-stage software startups. Methods: We followed a Design Science approach consisted of three cycles in the construction phase, that involved seven startups in total, and an evaluation of the final artifact within three startups. Results: We developed the HyMap, a hypotheses elicitation technique based on cognitive map**. It consists of a visual language to depict a cognitive map representing the founder's understanding of the product, and a process to elicit this map consisted of a series of questions the founder must answer. Our evaluation showed that the artifacts are clear, easy to use, and useful leading to hypotheses and facilitating founders to visualize their idea. Conclusion: Our study contributes to both descriptive and prescriptive bodies of knowledge. Regarding the first, it provides a better understanding of the guidance founders use to develop their startups and, for the latter, a technique to identify hypotheses in early-stage software startups.
△ Less
Submitted 9 January, 2022; v1 submitted 18 February, 2021;
originally announced February 2021.
-
Business Model Canvas Should Pay More Attention to the Software Startup Team
Authors:
Kai-Kristian Kemell,
Atte Elonen,
Mari Suoranta,
Anh Nguyen-Duc,
Juan Garbajosa,
Rafael Chanin,
Jorge Melegati,
Usman Rafiq,
Abdullah Aldaeej,
Nana Assyne,
Afonso Sales,
Sami Hyrynsalmi,
Juhani Riski,
Henry Edison,
Pekka Abrahamsson
Abstract:
Business Model Canvas (BMC) is a tool widely used to describe startup business models. Despite the various business aspects described, BMC pays a little emphasis on team-related factors. The importance of team-related factors in software development has been acknowledged widely in literature. While not as extensively studied, the importance of teams in software startups is also known in both liter…
▽ More
Business Model Canvas (BMC) is a tool widely used to describe startup business models. Despite the various business aspects described, BMC pays a little emphasis on team-related factors. The importance of team-related factors in software development has been acknowledged widely in literature. While not as extensively studied, the importance of teams in software startups is also known in both literature and among practitioners. In this paper, we propose potential changes to BMC to have the tool better reflect the importance of the team, especially in a software startup environment. Based on a literature review, we identify various components related to the team, which we then further support with empirical data. We do so by means of a qualitative case study of five startups.
△ Less
Submitted 11 February, 2021;
originally announced February 2021.
-
Empirical Standards for Software Engineering Research
Authors:
Paul Ralph,
Nauman bin Ali,
Sebastian Baltes,
Domenico Bianculli,
Jessica Diaz,
Yvonne Dittrich,
Neil Ernst,
Michael Felderer,
Robert Feldt,
Antonio Filieri,
Breno Bernard Nicolau de França,
Carlo Alberto Furia,
Greg Gay,
Nicolas Gold,
Daniel Graziotin,
Pinjia He,
Rashina Hoda,
Natalia Juristo,
Barbara Kitchenham,
Valentina Lenarduzzi,
Jorge Martínez,
Jorge Melegati,
Daniel Mendez,
Tim Menzies,
Jefferson Molleri
, et al. (18 additional authors not shown)
Abstract:
Empirical Standards are natural-language models of a scientific community's expectations for a specific kind of study (e.g. a questionnaire survey). The ACM SIGSOFT Paper and Peer Review Quality Initiative generated empirical standards for research methods commonly used in software engineering. These living documents, which should be continuously revised to reflect evolving consensus around resear…
▽ More
Empirical Standards are natural-language models of a scientific community's expectations for a specific kind of study (e.g. a questionnaire survey). The ACM SIGSOFT Paper and Peer Review Quality Initiative generated empirical standards for research methods commonly used in software engineering. These living documents, which should be continuously revised to reflect evolving consensus around research best practices, will improve research quality and make peer review more effective, reliable, transparent and fair.
△ Less
Submitted 4 March, 2021; v1 submitted 7 October, 2020;
originally announced October 2020.
-
Case Survey Studies in Software Engineering Research
Authors:
Jorge Melegati,
Xiaofeng Wang
Abstract:
Background: Given the social aspects of Software Engineering (SE), in the last twenty years, researchers from the field started using research methods common in social sciences such as case study, ethnography, and grounded theory. More recently, case survey, another imported research method, has seen its increasing use in SE studies. It is based on existing case studies reported in the literature…
▽ More
Background: Given the social aspects of Software Engineering (SE), in the last twenty years, researchers from the field started using research methods common in social sciences such as case study, ethnography, and grounded theory. More recently, case survey, another imported research method, has seen its increasing use in SE studies. It is based on existing case studies reported in the literature and intends to harness the generalizability of survey and the depth of case study. However, little is known on how case survey has been applied in SE research, let alone guidelines on how to employ it properly. Aims: This article aims to provide a better understanding of how case survey has been applied in Software Engineering research. Method: To address this knowledge gap, we performed a systematic map** study and analyzed 12 Software Engineering studies that used the case survey method. Results: Our findings show that these studies presented a heterogeneous understanding of the approach ranging from secondary studies to primary inquiries focused on a large number of instances of a research phenomenon. They have not applied the case survey method consistently as defined in the seminal methodological papers. Conclusions: We conclude that a set of clearly defined guidelines are needed on how to use case survey in SE research, to ensure the quality of the studies employing this approach and to provide a set of clearly defined criteria to evaluate such work.
△ Less
Submitted 27 July, 2020;
originally announced July 2020.
