Showing 1–36 of 36 results for author: Gorschek, T

Naming the Pain in Machine Learning-Enabled Systems Engineering

Authors: Marcos Kalinowski, Daniel Mendez, Görkem Giray, Antonio Pedro Santos Alves, Kelly Azevedo, Tatiana Escovedo, Hugo Villamizar, Helio Lopes, Teresa Baldassarre, Stefan Wagner, Stefan Biffl, Jürgen Musil, Michael Felderer, Niklas Lavesson, Tony Gorschek

Abstract: Context: Machine learning (ML)-enabled systems are being increasingly adopted by companies aiming to enhance their products and operational processes. Objective: This paper aims to deliver a comprehensive overview of the current status quo of engineering ML-enabled systems and lay the foundation to steer practically relevant and problem-driven academic research. Method: We conducted an internation… ▽ More Context: Machine learning (ML)-enabled systems are being increasingly adopted by companies aiming to enhance their products and operational processes. Objective: This paper aims to deliver a comprehensive overview of the current status quo of engineering ML-enabled systems and lay the foundation to steer practically relevant and problem-driven academic research. Method: We conducted an international survey to collect insights from practitioners on the current practices and problems in engineering ML-enabled systems. We received 188 complete responses from 25 countries. We conducted quantitative statistical analyses on contemporary practices using bootstrap** with confidence intervals and qualitative analyses on the reported problems using open and axial coding procedures. Results: Our survey results reinforce and extend existing empirical evidence on engineering ML-enabled systems, providing additional insights into typical ML-enabled systems project contexts, the perceived relevance and complexity of ML life cycle phases, and current practices related to problem understanding, model deployment, and model monitoring. Furthermore, the qualitative analysis provides a detailed map of the problems practitioners face within each ML life cycle phase and the problems causing overall project failure. Conclusions: The results contribute to a better understanding of the status quo and problems in practical environments. We advocate for the further adaptation and dissemination of software engineering practices to enhance the engineering of ML-enabled systems. △ Less

Submitted 20 May, 2024; originally announced June 2024.

Comments: arXiv admin note: text overlap with arXiv:2310.06726

On Develo** an Artifact-based Approach to Regulatory Requirements Engineering

Authors: Oleksandr Kosenkov, Michael Unterkalmsteiner, Jannik Fischbach, Daniel Mendez, Davide Fucci, Tony Gorschek

Abstract: Context: Regulatory acts are a challenging source when eliciting, interpreting, and analyzing requirements. Requirements engineers often need to involve legal experts who, however, may often not be available. This raises the need for approaches to regulatory Requirements Engineering (RE) covering and integrating both legal and engineering perspectives. Problem: Regulatory RE approaches need to c… ▽ More Context: Regulatory acts are a challenging source when eliciting, interpreting, and analyzing requirements. Requirements engineers often need to involve legal experts who, however, may often not be available. This raises the need for approaches to regulatory Requirements Engineering (RE) covering and integrating both legal and engineering perspectives. Problem: Regulatory RE approaches need to capture and reflect both the elementary concepts and relationships from a legal perspective and their seamless transition to concepts used to specify software requirements. No existing approach considers explicating and managing legal domain knowledge and engineering-legal coordination. Method: We conducted focus group sessions with legal researchers to identify the core challenges to establishing a regulatory RE approach. Based on our findings, we developed a candidate solution and conducted a first conceptual validation to assess its feasibility. Results: We introduce the first version of our Artifact Model for Regulatory Requirements Engineering (AM4RRE) and its conceptual foundation. It provides a blueprint for applying legal (modelling) concepts and well-established RE concepts. Our initial results suggest that artifact-centric RE can be applied to managing legal domain knowledge and engineering-legal coordination. Conclusions: The focus groups that served as a basis for building our model and the results from the expert validation both strengthen our confidence that we already provide a valuable basis for systematically integrating legal concepts into RE. This overcomes contemporary challenges to regulatory RE and serves as a basis for exposure to critical discussions in the community before continuing with the development of tool-supported extensions and large-scale empirical evaluations in practice. △ Less

Submitted 1 May, 2024; originally announced May 2024.

Comments: The paper was accepted to the 14th International Model-Driven Requirements Engineering (MoDRE) workshop co-located with the 32nd IEEE International Requirements Engineering Conference (RE 2024) in Reykjavik, Iceland

Use of Agile Practices in Start-ups

Authors: Eriks Klotins, Michael Unterkalmsteiner, Panagiota Chatzipetrou, Tony Gorschek, Rafael Prikladnicki, Nirnaya Tripathi, Leandro Bento Pompermaier

Abstract: Context Software start-ups have shown their ability to develop and launch innovative software products and services. Small, motivated teams and uncertain project scope makes start-ups good candidates for adopting Agile practices. Objective We explore how start-ups use Agile practices and what effects can be associated with the use of those practices. Method We use a case survey to analyze 84 start… ▽ More Context Software start-ups have shown their ability to develop and launch innovative software products and services. Small, motivated teams and uncertain project scope makes start-ups good candidates for adopting Agile practices. Objective We explore how start-ups use Agile practices and what effects can be associated with the use of those practices. Method We use a case survey to analyze 84 start-up cases and 56 Agile practices. We apply statistical methods to test for statistically significant associations between the use of Agile practices, team, and product factors. Results Our results suggest that development of the backlog, use of version control, code refactoring, and development of user stories are the most frequently reported practices. We identify 22 associations between the use of Agile practices, team, and product factors. The use of Agile practices is associated with effects on source code and overall product quality. A team's positive or negative attitude towards best engineering practices is a significant indicator for either adoption or rejection of certain Agile practices. To explore the relationships in our findings, we set forth a number of propositions that can be investigated in future research. Conclusions We conclude that start-ups use Agile practices, however without following any specific methodology. We identify the opportunity for more fine-grained studies into the adoption and effects of individual Agile practices. Start-up practitioners could benefit from Agile practices in terms of better overall quality, tighter control over team performance, and resource utilization. △ Less

Submitted 14 February, 2024; originally announced February 2024.

Comments: arXiv admin note: text overlap with arXiv:2309.12434

Journal ref: e-Informatica Software Engineering Journal, 2021, 15.1

ML-Enabled Systems Model Deployment and Monitoring: Status Quo and Problems

Authors: Eduardo Zimelewicz, Marcos Kalinowski, Daniel Mendez, Görkem Giray, Antonio Pedro Santos Alves, Niklas Lavesson, Kelly Azevedo, Hugo Villamizar, Tatiana Escovedo, Helio Lopes, Stefan Biffl, Juergen Musil, Michael Felderer, Stefan Wagner, Teresa Baldassarre, Tony Gorschek

Abstract: [Context] Systems incorporating Machine Learning (ML) models, often called ML-enabled systems, have become commonplace. However, empirical evidence on how ML-enabled systems are engineered in practice is still limited, especially for activities surrounding ML model dissemination. [Goal] We investigate contemporary industrial practices and problems related to ML model dissemination, focusing on the… ▽ More [Context] Systems incorporating Machine Learning (ML) models, often called ML-enabled systems, have become commonplace. However, empirical evidence on how ML-enabled systems are engineered in practice is still limited, especially for activities surrounding ML model dissemination. [Goal] We investigate contemporary industrial practices and problems related to ML model dissemination, focusing on the model deployment and the monitoring of ML life cycle phases. [Method] We conducted an international survey to gather practitioner insights on how ML-enabled systems are engineered. We gathered a total of 188 complete responses from 25 countries. We analyze the status quo and problems reported for the model deployment and monitoring phases. We analyzed contemporary practices using bootstrap** with confidence intervals and conducted qualitative analyses on the reported problems applying open and axial coding procedures. [Results] Practitioners perceive the model deployment and monitoring phases as relevant and difficult. With respect to model deployment, models are typically deployed as separate services, with limited adoption of MLOps principles. Reported problems include difficulties in designing the architecture of the infrastructure for production deployment and legacy application integration. Concerning model monitoring, many models in production are not monitored. The main monitored aspects are inputs, outputs, and decisions. Reported problems involve the absence of monitoring practices, the need to create custom monitoring tools, and the selection of suitable metrics. [Conclusion] Our results help provide a better understanding of the adopted practices and problems in practice and support guiding ML deployment and monitoring research in a problem-driven manner. △ Less

Submitted 7 February, 2024; originally announced February 2024.

Comments: arXiv admin note: text overlap with arXiv:2310.06726

A Progression Model of Software Engineering Goals, Challenges, and Practices in Start-Ups

Authors: Eriks Klotins, Michael Unterkalmsteiner, Panagiota Chatzipetrou, Tony Gorschek, Rafael Prikladnicki, Nirnaya Tripathi, Leandro Bento Pompermaier

Abstract: Context: Software start-ups are emerging as suppliers of innovation and software-intensive products. However, traditional software engineering practices are not evaluated in the context, nor adopted to goals and challenges of start-ups. As a result, there is insufficient support for software engineering in the start-up context. Objective: We aim to collect data related to engineering goals, challe… ▽ More Context: Software start-ups are emerging as suppliers of innovation and software-intensive products. However, traditional software engineering practices are not evaluated in the context, nor adopted to goals and challenges of start-ups. As a result, there is insufficient support for software engineering in the start-up context. Objective: We aim to collect data related to engineering goals, challenges, and practices in start-up companies to ascertain trends and patterns characterizing engineering work in start-ups. Such data allows researchers to understand better how goals and challenges are related to practices. This understanding can then inform future studies aimed at designing solutions addressing those goals and challenges. Besides, these trends and patterns can be useful for practitioners to make more informed decisions in their engineering practice. Method: We use a case survey method to gather first-hand, in-depth experiences from a large sample of software start-ups. We use open coding and cross-case analysis to describe and identify patterns, and corroborate the findings with statistical analysis. Results: We analyze 84 start-up cases and identify 16 goals, 9 challenges, and 16 engineering practices that are common among start-ups. We have mapped these goals, challenges, and practices to start-up life-cycle stages (inception, stabilization, growth, and maturity). Thus, creating the progression model guiding software engineering efforts in start-ups. Conclusions: We conclude that start-ups to a large extent face the same challenges and use the same practices as established companies. However, the primary software engineering challenge in start-ups is to evolve multiple process areas at once, with a little margin for serious errors. △ Less

Submitted 12 December, 2023; originally announced December 2023.

Journal ref: IEEE Trans. Software Eng. 47(3): 498-521 (2021)

Software engineering in start-up companies: An analysis of 88 experience reports

Authors: Eriks Klotins, Michael Unterkalmsteiner, Tony Gorschek

Abstract: Context: Start-up companies have become an important supplier of innovation and software-intensive products. The flexibility and reactiveness of start-ups enables fast development and launch of innovative products. However, a majority of software start-up companies fail before achieving any success. Among other factors, poor software engineering could be a significant contributor to the challenges… ▽ More Context: Start-up companies have become an important supplier of innovation and software-intensive products. The flexibility and reactiveness of start-ups enables fast development and launch of innovative products. However, a majority of software start-up companies fail before achieving any success. Among other factors, poor software engineering could be a significant contributor to the challenges experienced by start-ups. However, the state-of-practice of software engineering in start-ups, as well as the utilization of state-of-the-art is largely an unexplored area. Objective: In this study we investigate how software engineering is applied in start-up context with a focus to identify key knowledge areas and opportunities for further research. Method: We perform a multi-vocal exploratory study of 88 start-up experience reports. We develop a custom taxonomy to categorize the reported software engineering practices and their interrelation with business aspects, and apply qualitative data analysis to explore influences and dependencies between the knowledge areas. Results: We identify the most frequently reported software engineering (requirements engineering, software design and quality) and business aspect (vision and strategy development) knowledge areas, and illustrate their relationships. We also present a summary of how relevant software engineering knowledge areas are implemented in start-ups and identify potentially useful practices for adoption in start-ups. Conclusions: The results enable a more focused research on engineering practices in start-ups. We conclude that most engineering challenges in start-ups stem from inadequacies in requirements engineering. Many promising practices to address specific engineering challenges exists, however more research on adaptation of established practices, and validation of new start-up specific practices is needed. △ Less

Submitted 20 November, 2023; originally announced November 2023.

Journal ref: Empir. Softw. Eng. 24(1): 68-102 (2019)

Software Engineering Antipatterns in Start-Ups

Authors: Eriks Klotins, Michael Unterkalmsteiner, Tony Gorschek

Abstract: Software start-up failures are often explained with poor business model, market issues, insufficient funding, or simply a bad product idea. However, inadequacies in software product engineering are relatively little explored and could be a significant contributing factor to high start-up failure rate. In this paper we present analysis of 88 start-up experience reports. The analysis is presented in… ▽ More Software start-up failures are often explained with poor business model, market issues, insufficient funding, or simply a bad product idea. However, inadequacies in software product engineering are relatively little explored and could be a significant contributing factor to high start-up failure rate. In this paper we present analysis of 88 start-up experience reports. The analysis is presented in a form of three anti-patterns illustrating common symptoms, actual causes, and potential countermeasures of engineering inadequacies. The three anti-patterns are: product uncertainty comprising of issues in requirements engineering, poor product quality comprising of inadequacies in product quality, and team breakup comprising of team issues. The anti-patterns show that challenges and failure scenarios that appear to be business or market-related can actually originate from inadequacies in product engineering. △ Less

Submitted 20 November, 2023; originally announced November 2023.

Journal ref: IEEE Softw. 36(2): 118-126 (2019)

Status Quo and Problems of Requirements Engineering for Machine Learning: Results from an International Survey

Authors: Antonio Pedro Santos Alves, Marcos Kalinowski, Görkem Giray, Daniel Mendez, Niklas Lavesson, Kelly Azevedo, Hugo Villamizar, Tatiana Escovedo, Helio Lopes, Stefan Biffl, Jürgen Musil, Michael Felderer, Stefan Wagner, Teresa Baldassarre, Tony Gorschek

Abstract: Systems that use Machine Learning (ML) have become commonplace for companies that want to improve their products and processes. Literature suggests that Requirements Engineering (RE) can help address many problems when engineering ML-enabled systems. However, the state of empirical evidence on how RE is applied in practice in the context of ML-enabled systems is mainly dominated by isolated case s… ▽ More Systems that use Machine Learning (ML) have become commonplace for companies that want to improve their products and processes. Literature suggests that Requirements Engineering (RE) can help address many problems when engineering ML-enabled systems. However, the state of empirical evidence on how RE is applied in practice in the context of ML-enabled systems is mainly dominated by isolated case studies with limited generalizability. We conducted an international survey to gather practitioner insights into the status quo and problems of RE in ML-enabled systems. We gathered 188 complete responses from 25 countries. We conducted quantitative statistical analyses on contemporary practices using bootstrap** with confidence intervals and qualitative analyses on the reported problems involving open and axial coding procedures. We found significant differences in RE practices within ML projects. For instance, (i) RE-related activities are mostly conducted by project leaders and data scientists, (ii) the prevalent requirements documentation format concerns interactive Notebooks, (iii) the main focus of non-functional requirements includes data quality, model reliability, and model explainability, and (iv) main challenges include managing customer expectations and aligning requirements with data. The qualitative analyses revealed that practitioners face problems related to lack of business domain understanding, unclear goals and requirements, low customer engagement, and communication issues. These results help to provide a better understanding of the adopted practices and of which problems exist in practical environments. We put forward the need to adapt further and disseminate RE-related practices for engineering ML-enabled systems. △ Less

Submitted 10 October, 2023; originally announced October 2023.

Comments: Accepted for Publication at PROFES 2023

Requirements' Characteristics: How do they Impact on Project Budget in a Systems Engineering Context?

Authors: Panagiota Chatzipetrou, Michael Unterkalmsteiner, Tony Gorschek

Abstract: Background: Requirements engineering is of a principal importance when starting a new project. However, the number of the requirements involved in a single project can reach up to thousands. Controlling and assuring the quality of natural language requirements (NLRs), in these quantities, is challenging. Aims: In a field study, we investigated with the Swedish Transportation Agency (STA) to what e… ▽ More Background: Requirements engineering is of a principal importance when starting a new project. However, the number of the requirements involved in a single project can reach up to thousands. Controlling and assuring the quality of natural language requirements (NLRs), in these quantities, is challenging. Aims: In a field study, we investigated with the Swedish Transportation Agency (STA) to what extent the characteristics of requirements had an influence on change requests and budget changes in the project. Method: We choose the following models to characterize system requirements formulated in natural language: Concern-based Model of Requirements (CMR), Requirements Abstractions Model (RAM) and Software-Hardware model (SHM). The classification of the NLRs was conducted by the three authors. The robust statistical measure Fleiss' Kappa was used to verify the reliability of the results. We used descriptive statistics, contingency tables, results from the Chi-Square test of association along with post hoc tests. Finally, a multivariate statistical technique, Correspondence analysis was used in order to provide a means of displaying a set of requirements in two-dimensional graphical form. Results: The results showed that software requirements are associated with less budget cost than hardware requirements. Moreover, software requirements tend to stay open for a longer period indicating that they are "harder" to handle. Finally, the more discussion or interaction on a change request can lower the actual estimated change request cost. Conclusions: The results lead us to a need to further investigate the reasons why the software requirements are treated differently from the hardware requirements, interview the project managers, understand better the way those requirements are formulated and propose effective ways of Software management. △ Less

Submitted 2 October, 2023; originally announced October 2023.

Comments: SEAA 2019: 260-267

A Method for Identification and Ranking of Requirements Sources

Authors: Eriks Klotins, Veselka Boeva, Krzysztof Wnuk, Michael Unterkalmsteiner, Tony Gorschek, Slinger Jansen

Abstract: Requirements engineering (RE) literature acknowledges the importance of early stakeholder identification. The sources of requirements are many and also constantly changing as the market and business constantly change. Identifying and consulting all stakeholders on the market is impractical; thus many companies utilize indirect data sources, e.g. documents and representatives of larger groups of… ▽ More Requirements engineering (RE) literature acknowledges the importance of early stakeholder identification. The sources of requirements are many and also constantly changing as the market and business constantly change. Identifying and consulting all stakeholders on the market is impractical; thus many companies utilize indirect data sources, e.g. documents and representatives of larger groups of stakeholders. However, companies often collect irrelevant data or develop their products based on the sub-optimal information sources that may lead to missing market opportunities. We propose a collaborative method for identification and selection of data sources. The method consists of four steps and aims to build consensus between different perspectives in an organization. We demonstrate the use of the method with three industrial case studies. We have presented and statically validated the method to support prioritization of stakeholders for MDRE. Our results show that the method can support the identification and selection of data sources in three ways: (1) by providing systematic steps to identify and prioritize data sources for RE, (2) by highlighting and resolving discrepancies between different perspectives in an organization, and (3) by analyzing the underlying rationale for using certain data sources. △ Less

Submitted 29 September, 2023; originally announced September 2023.

Software-Intensive Product Engineering in Start-Ups: A Taxonomy

Authors: Eriks Klotins, Michael Unterkalmsteiner, Tony Gorschek

Abstract: Software start-ups are new companies aiming to launch an innovative product to mass markets fast with minimal resources. However, most start-ups fail before realizing their potential. Poor software engineering, among other factors, could be a significant contributor to the challenges that start-ups experience. Little is known about the engineering context in start-up companies. On the surface, sta… ▽ More Software start-ups are new companies aiming to launch an innovative product to mass markets fast with minimal resources. However, most start-ups fail before realizing their potential. Poor software engineering, among other factors, could be a significant contributor to the challenges that start-ups experience. Little is known about the engineering context in start-up companies. On the surface, start-ups are characterized by uncertainty, high risk, and minimal resources. However, such a characterization isn't granular enough to support identification of specific engineering challenges and to devise start-up-specific engineering practices. The first step toward an understanding of software engineering in start-ups is the definition of a Start-Up Context Map - a taxonomy of engineering practices, environment factors, and goals influencing the engineering process. This map aims to support further research on the field and serve as an engineering decision support tool for start-ups. This article is part of a theme issue on Process Improvement. △ Less

Submitted 28 September, 2023; originally announced September 2023.

Journal ref: IEEE Softw. 35(4): 44-52 (2018)

Process Improvement Archaeology: What Led Us Here, and What's Next?

Authors: Michael Unterkalmsteiner, Tony Gorschek

Abstract: While in every organization corporate culture and history change over time, intentional efforts to identify performance problems are of particular interest when trying to understand the current state of an organization. The results of past improvement initiatives can shed light on the evolution of an organization and represent, with the advantage of perfect hindsight, a learning opportunity for fu… ▽ More While in every organization corporate culture and history change over time, intentional efforts to identify performance problems are of particular interest when trying to understand the current state of an organization. The results of past improvement initiatives can shed light on the evolution of an organization and represent, with the advantage of perfect hindsight, a learning opportunity for future process improvements. The opportunity to test this premise occurred in an applied research collaboration with the Swedish Transport Administration, the government agency responsible for the planning, implementation, and maintenance of long-term rail, road, ship**, and aviation infrastructure in Sweden. This article is part of a theme issue on Process Improvement. △ Less

Submitted 21 September, 2023; originally announced September 2023.

Journal ref: IEEE Softw. 35(4): 53-61 (2018)

Exploration of technical debt in start-ups

Authors: Eriks Klotins, Michael Unterkalmsteiner, Panagiota Chatzipetrou, Tony Gorschek, Rafael Prikladnicki, Nirnaya Tripathi, Leandro Bento Pompermaier

Abstract: Context: Software start-ups are young companies aiming to build and market software-intensive products fast with little resources. Aiming to accelerate time-to-market, start-ups often opt for ad-hoc engineering practices, make shortcuts in product engineering, and accumulate technical debt. Objective: In this paper we explore to what extent precedents, dimensions and outcomes associated with techn… ▽ More Context: Software start-ups are young companies aiming to build and market software-intensive products fast with little resources. Aiming to accelerate time-to-market, start-ups often opt for ad-hoc engineering practices, make shortcuts in product engineering, and accumulate technical debt. Objective: In this paper we explore to what extent precedents, dimensions and outcomes associated with technical debt are prevalent in start-ups. Method: We apply a case survey method to identify aspects of technical debt and contextual information characterizing the engineering context in start-ups. Results: By analyzing responses from 86 start-up cases we found that start-ups accumulate most technical debt in the testing dimension, despite attempts to automate testing. Furthermore, we found that start-up team size and experience is a leading precedent for accumulating technical debt: larger teams face more challenges in kee** the debt under control. Conclusions: This study highlights the necessity to monitor levels of technical debt and to preemptively introduce practices to keep the debt under control. Adding more people to an already difficult to maintain product could amplify other precedents, such as resource shortages, communication issues and negatively affect decisions pertaining to the use of good engineering practices. △ Less

Submitted 21 September, 2023; originally announced September 2023.

Comments: ICSE-SEIP '18: Proceedings of the 40th International Conference on Software Engineering: Software Engineering in Practice

Which Requirements Artifact Quality Defects are Automatically Detectable? A Case Study

Authors: Henning Femmer, Michael Unterkalmsteiner, Tony Gorschek

Abstract: [Context] The quality of requirements engineering artifacts, e.g. requirements specifications, is acknowledged to be an important success factor for projects. Therefore, many companies spend significant amounts of money to control the quality of their RE artifacts. To reduce spending and improve the RE artifact quality, methods were proposed that combine manual quality control, i.e. reviews, with… ▽ More [Context] The quality of requirements engineering artifacts, e.g. requirements specifications, is acknowledged to be an important success factor for projects. Therefore, many companies spend significant amounts of money to control the quality of their RE artifacts. To reduce spending and improve the RE artifact quality, methods were proposed that combine manual quality control, i.e. reviews, with automated approaches. [Problem] So far, we have seen various approaches to automatically detect certain aspects in RE artifacts. However, we still lack an overview what can and cannot be automatically detected. [Approach] Starting from an industry guideline for RE artifacts, we classify 166 existing rules for RE artifacts along various categories to discuss the share and the characteristics of those rules that can be automated. For those rules, that cannot be automated, we discuss the main reasons. [Contribution] We estimate that 53% of the 166 rules can be checked automatically either perfectly or with a good heuristic. Most rules need only simple techniques for checking. The main reason why some rules resist automation is due to imprecise definition. [Impact] By giving first estimates and analyses of automatically detectable and not automatically detectable rule violations, we aim to provide an overview of the potential of automated methods in requirements quality control. △ Less

Submitted 29 August, 2023; originally announced August 2023.

Comments: 2017 25th International Requirements Engineering Conference Workshops (REW) (pp. 400-406)

Requirements Quality Assurance in Industry: Why, What and How?

Authors: Michael Unterkalmsteiner, Tony Gorschek

Abstract: Context and Motivation: Natural language is the most common form to specify requirements in industry. The quality of the specification depends on the capability of the writer to formulate requirements aimed at different stakeholders: they are an expression of the customer's needs that are used by analysts, designers and testers. Given this central role of requirements as a mean to communicate inte… ▽ More Context and Motivation: Natural language is the most common form to specify requirements in industry. The quality of the specification depends on the capability of the writer to formulate requirements aimed at different stakeholders: they are an expression of the customer's needs that are used by analysts, designers and testers. Given this central role of requirements as a mean to communicate intention, assuring their quality is essential to reduce misunderstandings that lead to potential waste. Problem: Quality assurance of requirement specifications is largely a manual effort that requires expertise and domain knowledge. However, this demanding cognitive process is also congested by trivial quality issues that should not occur in the first place. Principal ideas: We propose a taxonomy of requirements quality assurance complexity that characterizes cognitive load of verifying a quality aspect from the human perspective, and automation complexity and accuracy from the machine perspective. Contribution: Once this taxonomy is realized and validated, it can serve as the basis for a decision framework of automated requirements quality assurance support. △ Less

Submitted 24 August, 2023; originally announced August 2023.

Comments: Conference proceedings of Requirements Engineering: Foundation for Software Quality (REFSQ) 2017: 77-84

Software Startups -- A Research Agenda

Authors: Michael Unterkalmsteiner, Pekka Abrahamsson, Xiaofeng Wang, Anh Nguyen-Duc, Syed M. Ali Shah, Sohaib Shahid Bajwa, Guido H. Baltes, Kieran Conboy, Eoin Cullina, Denis Dennehy, Henry Edison, Carlos Fernández-Sánchez, Juan Garbajosa, Tony Gorschek, Eriks Klotins, Laura Hokkanen, Fabio Kon, Ilaria Lunesu, Michele Marchesi, Lorraine Morgan, Markku Oivo, Christoph Selig, Pertti Seppänen, Roger Sweetman, Pasi Tyrväinen , et al. (2 additional authors not shown)

Abstract: Software startup companies develop innovative, software-intensive products within limited time frames and with few resources, searching for sustainable and scalable business models. Software startups are quite distinct from traditional mature software companies, but also from micro-, small-, and medium-sized enterprises, introducing new challenges relevant for software engineering research. This p… ▽ More Software startup companies develop innovative, software-intensive products within limited time frames and with few resources, searching for sustainable and scalable business models. Software startups are quite distinct from traditional mature software companies, but also from micro-, small-, and medium-sized enterprises, introducing new challenges relevant for software engineering research. This paper's research agenda focuses on software engineering in startups, identifying, in particular, 70+ research questions in the areas of supporting startup engineering activities, startup evolution models and patterns, ecosystems and innovation hubs, human aspects in software startups, applying startup concepts in non-startup environments, and methodologies and theories for startup research. We connect and motivate this research agenda with past studies in software startup research, while pointing out possible future directions. While all authors of this research agenda have their main background in Software Engineering or Computer Science, their interest in software startups broadens the perspective to the challenges, but also to the opportunities that emerge from multi-disciplinary research. Our audience is therefore primarily software engineering researchers, even though we aim at stimulating collaborations and research that crosses disciplinary boundaries. We believe that with this research agenda we cover a wide spectrum of the software startup industry current needs. △ Less

Submitted 24 August, 2023; originally announced August 2023.

Journal ref: e-Informatica Softw. Eng. J. 10(1): 89-124 (2016)

Large-scale information retrieval in software engineering -- an experience report from industrial application

Authors: Michael Unterkalmsteiner, Tony Gorschek, Robert Feldt, Niklas Lavesson

Abstract: Software Engineering activities are information intensive. Research proposes Information Retrieval (IR) techniques to support engineers in their daily tasks, such as establishing and maintaining traceability links, fault identification, and software maintenance. We describe an engineering task, test case selection, and illustrate our problem analysis and solution discovery process. The objective o… ▽ More Software Engineering activities are information intensive. Research proposes Information Retrieval (IR) techniques to support engineers in their daily tasks, such as establishing and maintaining traceability links, fault identification, and software maintenance. We describe an engineering task, test case selection, and illustrate our problem analysis and solution discovery process. The objective of the study is to gain an understanding of to what extent IR techniques (one potential solution) can be applied to test case selection and provide decision support in a large-scale, industrial setting. We analyze, in the context of the studied company, how test case selection is performed and design a series of experiments evaluating the performance of different IR techniques. Each experiment provides lessons learned from implementation, execution, and results, feeding to its successor. The three experiments led to the following observations: 1) there is a lack of research on scalable parameter optimization of IR techniques for software engineering problems; 2) scaling IR techniques to industry data is challenging, in particular for latent semantic analysis; 3) the IR context poses constraints on the empirical evaluation of IR techniques, requiring more research on develo** valid statistical approaches. We believe that our experiences in conducting a series of IR experiments with industry grade data are valuable for peer researchers so that they can avoid the pitfalls that we have encountered. Furthermore, we identified challenges that need to be addressed in order to bridge the gap between laboratory IR experiments and real applications of IR in the industry. △ Less

Submitted 22 August, 2023; originally announced August 2023.

Journal ref: Empir. Softw. Eng. 21(6): 2324-2365 (2016)

Software Development in Startup Companies: The Greenfield Startup Model

Authors: Carmine Giardino, Nicolò Paternoster, Michael Unterkalmsteiner, Tony Gorschek, Pekka Abrahamsson

Abstract: Software startups are newly created companies with no operating history and oriented towards producing cutting-edge products. However, despite the increasing importance of startups in the economy, few scientific studies attempt to address software engineering issues, especially for early-stage startups. If anything, startups need engineering practices of the same level or better than those of larg… ▽ More Software startups are newly created companies with no operating history and oriented towards producing cutting-edge products. However, despite the increasing importance of startups in the economy, few scientific studies attempt to address software engineering issues, especially for early-stage startups. If anything, startups need engineering practices of the same level or better than those of larger companies, as their time and resources are more scarce, and one failed project can put them out of business. In this study we aim to improve understanding of the software development strategies employed by startups. We performed this state-of-practice investigation using a grounded theory approach. We packaged the results in the Greenfield Startup Model (GSM), which explains the priority of startups to release the product as quickly as possible. This strategy allows startups to verify product and market fit, and to adjust the product trajectory according to early collected user feedback. The need to shorten time-to-market, by speeding up the development through low-precision engineering activities, is counterbalanced by the need to restructure the product before targeting further growth. The resulting implications of the GSM outline challenges and gaps, pointing out opportunities for future research to develop and validate engineering practices in the startup context. △ Less

Submitted 18 August, 2023; originally announced August 2023.

Journal ref: IEEE Trans. Software Eng. 42(6): 585-604 (2016)

Assessing requirements engineering and software test alignment -- Five case studies

Authors: Michael Unterkalmsteiner, Tony Gorschek, Robert Feldt, Eriks Klotins

Abstract: The development of large, software-intensive systems is a complex undertaking that we generally tackle by a divide and conquer strategy. Companies thereby face the challenge of coordinating individual aspects of software development, in particular between requirements engineering (RE) and software testing (ST). A lack of REST alignment can not only lead to wasted effort but also to defective softw… ▽ More The development of large, software-intensive systems is a complex undertaking that we generally tackle by a divide and conquer strategy. Companies thereby face the challenge of coordinating individual aspects of software development, in particular between requirements engineering (RE) and software testing (ST). A lack of REST alignment can not only lead to wasted effort but also to defective software. However, before a company can improve the mechanisms of coordination they need to be understood first. With REST-bench we aim at providing an assessment tool that illustrates the coordination in software development projects and identify concrete improvement opportunities. We have developed REST-bench on the sound fundamentals of a taxonomy on REST alignment methods and validated the method in five case studies. Following the principles of technical action research, we collaborated with five companies, applying REST-bench and iteratively improving the method based on the lessons we learned. We applied REST-bench both in Agile and plan-driven environments, in projects lasting from weeks to years, and staffed as large as 1000 employees. The improvement opportunities we identified and the feedback we received indicate that the assessment was effective and efficient. Furthermore, participants confirmed that their understanding on the coordination between RE and ST improved. △ Less

Submitted 15 August, 2023; originally announced August 2023.

Journal ref: J. Syst. Softw. 109: 62-77 (2015)

Software Engineering Knowledge Areas in Startup Companies: A Map** Study

Authors: Eriks Klotins, Michael Unterkalmsteiner, Tony Gorschek

Abstract: Background - Startup companies are becoming important suppliers of innovative and software intensive products. The failure rate among startups is high due to lack of resources, immaturity, multiple influences and dynamic technologies. However, software product engineering is the core activity in startups, therefore inadequacies in applied engineering practices might be a significant contributing f… ▽ More Background - Startup companies are becoming important suppliers of innovative and software intensive products. The failure rate among startups is high due to lack of resources, immaturity, multiple influences and dynamic technologies. However, software product engineering is the core activity in startups, therefore inadequacies in applied engineering practices might be a significant contributing factor for high failure rates. Aim - This study identifies and categorizes software engineering knowledge areas utilized in startups to map out the state-of-art, identifying gaps for further research. Method - We perform a systematic literature map** study, applying snowball sampling to identify relevant primary studies. Results - We have identified 54 practices from 14 studies. Although 11 of 15 main knowledge areas from SWEBOK are covered, a large part of categories is not. Conclusions - Existing research does not provide reliable support for software engineering in any phase of a startup life cycle. Transfer of results to other startups is difficult due to low rigor in current studies. △ Less

Submitted 15 August, 2023; originally announced August 2023.

Comments: Proceedings 6th International Conference on Software Business (ICSOB 2015), Braga, Portugal, 245-257

What do we know about software development in startups?

Authors: Carmine Giardino, Michael Unterkalmsteiner, Nicolò Paternoster, Tony Gorschek, Pekka Abrahamsson

Abstract: An impressive number of new startups are launched every day as a result of growing new markets, accessible technologies, and venture capital. New ventures such as Facebook, Supercell, Linkedin, Spotify, {WhatsApp}, and Dropbox, to name a few, are good examples of startups that evolved into successful businesses. However, despite many successful stories, the great majority of them fail prematurely.… ▽ More An impressive number of new startups are launched every day as a result of growing new markets, accessible technologies, and venture capital. New ventures such as Facebook, Supercell, Linkedin, Spotify, {WhatsApp}, and Dropbox, to name a few, are good examples of startups that evolved into successful businesses. However, despite many successful stories, the great majority of them fail prematurely. Operating in a chaotic and rapidly evolving domain conveys new uncharted challenges for startuppers. In this study, the authors characterize their context and identify common software development startup practices. △ Less

Submitted 24 July, 2023; originally announced July 2023.

Comments: arXiv admin note: substantial text overlap with arXiv:2307.13104

Journal ref: IEEE Softw. 31(5): 28-32 (2014)

Evaluation and Measurement of Software Process Improvement -- A Systematic Literature Review

Authors: Michael Unterkalmsteiner, Tony Gorschek, A. K. M. Moinul Islam, Chow Kian Cheng, Rahadian Bayu Permadi, Robert Feldt

Abstract: BACKGROUND: Software Process Improvement (SPI) is a systematic approach to increase the efficiency and effectiveness of a software development organization and to enhance software products. OBJECTIVE: This paper aims to identify and characterize evaluation strategies and measurements used to assess the impact of different SPI initiatives. METHOD: The systematic literature review includes 148 paper… ▽ More BACKGROUND: Software Process Improvement (SPI) is a systematic approach to increase the efficiency and effectiveness of a software development organization and to enhance software products. OBJECTIVE: This paper aims to identify and characterize evaluation strategies and measurements used to assess the impact of different SPI initiatives. METHOD: The systematic literature review includes 148 papers published between 1991 and 2008. The selected papers were classified according to SPI initiative, applied evaluation strategies, and measurement perspectives. Potential confounding factors interfering with the evaluation of the improvement effort were assessed. RESULTS: Seven distinct evaluation strategies were identified, wherein the most common one, "Pre-Post Comparison" was applied in 49 percent of the inspected papers. Quality was the most measured attribute (62 percent), followed by Cost (41 percent), and Schedule (18 percent). Looking at measurement perspectives, "Project" represents the majority with 66 percent. CONCLUSION: The evaluation validity of SPI initiatives is challenged by the scarce consideration of potential confounding factors, particularly given that "Pre-Post Comparison" was identified as the most common evaluation strategy, and the inaccurate descriptions of the evaluation context. Measurements to assess the short and mid-term impact of SPI initiatives prevail, whereas long-term measurements in terms of customer satisfaction and return on investment tend to be less used. △ Less

Submitted 24 July, 2023; originally announced July 2023.

Journal ref: IEEE Trans. Software Eng. 38(2): 398-424 (2012)

Software development in startup companies: A systematic map** study

Authors: Nicolò Paternoster, Carmine Giardino, Michael Unterkalmsteiner, Tony Gorschek, Pekka Abrahamsson

Abstract: Context: Software startups are newly created companies with no operating history and fast in producing cutting-edge technologies. These companies develop software under highly uncertain conditions, tackling fast-growing markets under severe lack of resources. Therefore, software startups present an unique combination of characteristics which pose several challenges to software development activiti… ▽ More Context: Software startups are newly created companies with no operating history and fast in producing cutting-edge technologies. These companies develop software under highly uncertain conditions, tackling fast-growing markets under severe lack of resources. Therefore, software startups present an unique combination of characteristics which pose several challenges to software development activities. Objective: This study aims to structure and analyze the literature on software development in startup companies, determining thereby the potential for technology transfer and identifying software development work practices reported by practitioners and researchers. Method: We conducted a systematic map** study, develo** a classification schema, ranking the selected primary studies according their rigor and relevance, and analyzing reported software development work practices in startups. Results: A total of 43 primary studies were identified and mapped, synthesizing the available evidence on software development in startups. Only 16 studies are entirely dedicated to software development in startups, of which 10 result in a weak contribution (advice and implications (6); lesson learned (3); tool (1)). Nineteen studies focus on managerial and organizational factors. Moreover, only 9 studies exhibit high scientific rigor and relevance. From the reviewed primary studies, 213 software engineering work practices were extracted, categorized and analyzed. Conclusion: This map** study provides the first systematic exploration of the state-of-art on software startup research. The existing body of knowledge is limited to a few high quality studies. Furthermore, the results indicate that software engineering work practices are chosen opportunistically, adapted and configured to provide value under the constrains imposed by the startup context. △ Less

Submitted 24 July, 2023; originally announced July 2023.

Journal ref: Inf. Softw. Technol. 56(10): 1200-1218 (2014)

A conceptual framework for SPI evaluation

Authors: Michael Unterkalmsteiner, Tony Gorschek, A. K. M. Moinul Islam, Chow Kian Cheng, Rahadian Bayu Permadi, Robert Feldt

Abstract: Software Process Improvement (SPI) encompasses the analysis and modification of the processes within software development, aimed at improving key areas that contribute to the organizations' goals. The task of evaluating whether the selected improvement path meets these goals is challenging. On the basis of the results of a systematic literature review on SPI measurement and evaluation practices, w… ▽ More Software Process Improvement (SPI) encompasses the analysis and modification of the processes within software development, aimed at improving key areas that contribute to the organizations' goals. The task of evaluating whether the selected improvement path meets these goals is challenging. On the basis of the results of a systematic literature review on SPI measurement and evaluation practices, we developed a framework (SPI Measurement and Evaluation Framework (SPI-MEF)) that supports the planning and implementation of SPI evaluations. SPI-MEF guides the practitioner in sco** the evaluation, determining measures, and performing the assessment. SPI-MEF does not assume a specific approach to process improvement and can be integrated in existing measurement programs, refocusing the assessment on evaluating the improvement initiative's outcome. Sixteen industry and academic experts evaluated the framework's usability and capability to support practitioners, providing additional insights that were integrated in the application guidelines of the framework. △ Less

Submitted 24 July, 2023; originally announced July 2023.

Journal ref: J. Softw. Evol. Process. 26(2): 251-279 (2014)

Challenges and Practices in Aligning Requirements with Verification and Validation: A Case Study of Six Companies

Authors: Elizabeth Bjarnason, Per Runeson, Markus Borg, Michael Unterkalmsteiner, Emelie Engström, Björn Regnell, Giedre Sabaliauskaite, Annabella Loconsole, Tony Gorschek, Robert Feldt

Abstract: Weak alignment of requirements engineering (RE) with verification and validation (VV) may lead to problems in delivering the required products in time with the right quality. For example, weak communication of requirements changes to testers may result in lack of verification of new requirements and incorrect verification of old invalid requirements, leading to software quality problems, wasted ef… ▽ More Weak alignment of requirements engineering (RE) with verification and validation (VV) may lead to problems in delivering the required products in time with the right quality. For example, weak communication of requirements changes to testers may result in lack of verification of new requirements and incorrect verification of old invalid requirements, leading to software quality problems, wasted effort and delays. However, despite the serious implications of weak alignment research and practice both tend to focus on one or the other of RE or VV rather than on the alignment of the two. We have performed a multi-unit case study to gain insight into issues around aligning RE and VV by interviewing 30 practitioners from 6 software develo** companies, involving 10 researchers in a flexible research process for case studies. The results describe current industry challenges and practices in aligning RE with VV, ranging from quality of the individual RE and VV activities, through tracing and tools, to change control and sharing a common understanding at strategy, goal and design level. The study identified that human aspects are central, i.e. cooperation and communication, and that requirements engineering practices are a critical basis for alignment. Further, the size of an organisation and its motivation for applying alignment practices, e.g. external enforcement of traceability, are variation factors that play a key role in achieving alignment. Our results provide a strategic roadmap for practitioners improvement work to address alignment challenges. Furthermore, the study provides a foundation for continued research to improve the alignment of RE with VV. △ Less

Submitted 23 July, 2023; originally announced July 2023.

Journal ref: Empir. Softw. Eng. 19(6): 1809-1855 (2014)

A Taxonomy for Requirements Engineering and Software Test Alignment

Authors: Michael Unterkalmsteiner, Robert Feldt, Tony Gorschek

Abstract: Requirements Engineering and Software Testing are mature areas and have seen a lot of research. Nevertheless, their interactions have been sparsely explored beyond the concept of traceability. To fill this gap, we propose a definition of requirements engineering and software test (REST) alignment, a taxonomy that characterizes the methods linking the respective areas, and a process to assess align… ▽ More Requirements Engineering and Software Testing are mature areas and have seen a lot of research. Nevertheless, their interactions have been sparsely explored beyond the concept of traceability. To fill this gap, we propose a definition of requirements engineering and software test (REST) alignment, a taxonomy that characterizes the methods linking the respective areas, and a process to assess alignment. The taxonomy can support researchers to identify new opportunities for investigation, as well as practitioners to compare alignment methods and evaluate alignment, or lack thereof. We constructed the REST taxonomy by analyzing alignment methods published in literature, iteratively validating the emerging dimensions. The resulting concept of an information dyad characterizes the exchange of information required for any alignment to take place. We demonstrate use of the taxonomy by applying it on five in-depth cases and illustrate angles of analysis on a set of thirteen alignment methods. In addition, we developed an assessment framework (REST-bench), applied it in an industrial assessment, and showed that it, with a low effort, can identify opportunities to improve REST alignment. Although we expect that the taxonomy can be further refined, we believe that the information dyad is a valid and useful construct to understand alignment. △ Less

Submitted 23 July, 2023; originally announced July 2023.

Journal ref: ACM Trans. Softw. Eng. Methodol. 23(2): 16:1-16:38 (2014)

Challenges in aligning requirements engineering and verification in a large-scale industrial context

Authors: Giedre Sabaliauskaite, Annabella Loconsole, Emelie Engström, Michael Unterkalmsteiner, Björn Regnell, Per Runeson, Tony Gorschek, Robert Feldt

Abstract: [Context and motivation] When develo** software, coordination between different organizational units is essential in order to develop a good quality product, on time and within budget. Particularly, the synchronization between requirements and verification processes is crucial in order to assure that the developed software product satisfies customer requirements. [Question/problem] Our research… ▽ More [Context and motivation] When develo** software, coordination between different organizational units is essential in order to develop a good quality product, on time and within budget. Particularly, the synchronization between requirements and verification processes is crucial in order to assure that the developed software product satisfies customer requirements. [Question/problem] Our research question is: what are the current challenges in aligning the requirements and verification processes? [Principal ideas/results] We conducted an interview study at a large software development company. This paper presents preliminary findings of these interviews that identify key challenges in aligning requirements and verification processes. [Contribution] The result of this study includes a range of challenges faced by the studied organization grouped into the categories: organization and processes, people, tools, requirements process, testing process, change management, traceability, and measurement. The findings of this study can be used by practitioners as a basis for investigating alignment in their organizations, and by scientists in develo** approaches for more efficient and effective management of the alignment between requirements and verification. △ Less

Submitted 23 July, 2023; originally announced July 2023.

Comments: Requirements Engineering: Foundation for Software Quality: 16th International Working Conference, REFSQ 2010, Essen, Germany, June 30-July 2, 2010. Proceedings 16 (pp. 128-142). Springer Berlin Heidelberg

Preliminary Guideline for Creating Boundary Artefacts in Software Engineering

Authors: Raquel Ouriques, Fabian Fagerholm, Daniel Mendez, Tony Gorschek, Baldvin Gislason Bern

Abstract: Context: Software development benefits from having Boundary Artefacts (BAs), as a single artefact can supply stakeholders with different boundaries, facilitating collaboration among social worlds. When those artefacts display inconsistencies, such as incorrect information, the practitioners have decreased trust in the BA. As trust is an essential factor guiding the utilisation of BAs in software p… ▽ More Context: Software development benefits from having Boundary Artefacts (BAs), as a single artefact can supply stakeholders with different boundaries, facilitating collaboration among social worlds. When those artefacts display inconsistencies, such as incorrect information, the practitioners have decreased trust in the BA. As trust is an essential factor guiding the utilisation of BAs in software projects, it is necessary to understand which principles should be observed when creating them. Objective: This study aimed at develop and validate a preliminary guideline support the creation of trustworthy BAs. Method: We followed a multi-step approach. We developed our guideline through a literature review and previous results from our case study. Second, we submitted the guideline for an expert evaluation via two workshops and a survey. At last, we adjusted our guideline by incorporating the feedback obtained during the workshops. Results: We grouped the principles collected from a literature review into three categories. The first category (Scope) focuses on the scope, displaying principles referring to defining each boundary's target audience, needs, and terminology. The second category (Structure) relates to how the artefact's content is structured to meet stakeholders' needs. The third (Management) refers to principles that can guide the establishment of practices to manage the artefact throughout time. The expert validation revealed that the principles contribute to creating trustworthy BAs at different levels. Also, the relevance of the guideline and its usefulness. Conclusions: The guideline strengthen BA traits such as shared understanding, plasticity and ability to transfer. Practitioners can utilise the guideline to guide the creation or even evaluate current practices for existing BAs. △ Less

Submitted 9 June, 2023; originally announced June 2023.

Comments: 25 pages

Connecting the Dots of Knowledge in Agile Software Development

Authors: Raquel Ouriques, Tony Gorschek, Daniel Mendez, Fabian Fagerholm

Abstract: This article discusses the importance of managing knowledge as a resource due to its great potential to create economic value. We detail the types of knowledge resources, the challenges associated with their management, and potential solutions to maximise their utility. Our contribution is based on empirical studies performed in an industry context. This article discusses the importance of managing knowledge as a resource due to its great potential to create economic value. We detail the types of knowledge resources, the challenges associated with their management, and potential solutions to maximise their utility. Our contribution is based on empirical studies performed in an industry context. △ Less

Submitted 9 June, 2023; originally announced June 2023.

Comments: 8 pages

Continuous Software Engineering in the Wild

Authors: Eriks Klotins, Tony Gorschek

Abstract: Software is becoming a critical component of most products and organizational functions. The ability to continuously improve software determines how well the organization can respond to market opportunities. Continuous software engineering promises numerous advantages over sprint-based or plan-driven development. However, implementing a continuous software engineering pipeline in an existing organ… ▽ More Software is becoming a critical component of most products and organizational functions. The ability to continuously improve software determines how well the organization can respond to market opportunities. Continuous software engineering promises numerous advantages over sprint-based or plan-driven development. However, implementing a continuous software engineering pipeline in an existing organization is challenging. In this invited position paper, we discuss the adoption challenges and argue for a more systematic methodology to drive the adoption of continuous engineering. Our discussion is based on ongoing work with several industrial partners as well as experience reported in both state-of-practice and state-of-the-art. We conclude that the adoption of continuous software engineering primarily requires analysis of the organization, its goals, and constraints. One size does not fit all purposes, meaning that many of the principles behind continuous engineering are relevant for most organizations, but the level of realization and the benefits may still vary. The main hindrances to continuous flow of software arise from sub-optimal organizational structures and the lack of alignment. Once those are removed, the organization can implement automation to further improve the software delivery. △ Less

Submitted 24 March, 2022; v1 submitted 23 March, 2022; originally announced March 2022.

On Understanding the Relation of Knowledge and Confidence to Requirements Quality

Authors: Razieh Dehghani, Krzysztof Wnuk, Daniel Mendez, Tony Gorschek, Raman Ramsin

Abstract: Context and Motivation: Software requirements are affected by the knowledge and confidence of software engineers. Analyzing the interrelated impact of these factors is difficult because of the challenges of assessing knowledge and confidence. Question/Problem: This research aims to draw attention to the need for considering the interrelated effects of confidence and knowledge on requirements qua… ▽ More Context and Motivation: Software requirements are affected by the knowledge and confidence of software engineers. Analyzing the interrelated impact of these factors is difficult because of the challenges of assessing knowledge and confidence. Question/Problem: This research aims to draw attention to the need for considering the interrelated effects of confidence and knowledge on requirements quality, which has not been addressed by previous publications. Principal ideas/results: For this purpose, the following steps have been taken: 1) requirements quality was defined based on the instructions provided by the ISO29148:2011 standard, 2) we selected the symptoms of low qualified requirements based on ISO29148:2011, 3) we analyzed five Software Requirements Specification (SRS) documents to find these symptoms, 3) people who have prepared the documents were categorized in four classes to specify the more/less knowledge and confidence they have regarding the symptoms, and 4) finally, the relation of lack of enough knowledge and confidence to symptoms of low quality was investigated. The results revealed that the simultaneous deficiency of confidence and knowledge has more negative effects in comparison with a deficiency of knowledge or confidence. Contribution: In brief, this study has achieved these results: 1) the realization that a combined lack of knowledge and confidence has a larger effect on requirements quality than only one of the two factors, 2) the relation between low qualified requirements and requirements engineers' needs for knowledge and confidence, and 3) variety of requirements engineers' needs for knowledge based on their abilities to make discriminative and consistent decisions. △ Less

Submitted 3 March, 2021; originally announced March 2021.

Comments: Preprint accepted for publication at the 27th International Working Conference on Requirement Engineering: Foundation for Software Quality

Journal ref: Proc. of the 27th International Working Conference on Requirement Engineering: Foundation for Software Quality 2021

The State-of-Practice in Requirements Elicitation: An Extended Interview Study at 12 Companies

Authors: Cristina Palomares, Xavier Franch, Carme Quer, Panagiota Chatzipetrou, Lidia López, Tony Gorschek

Abstract: Context. Requirements engineering remains a discipline that is faced with a large number of challenges, including the implementation of a requirements elicitation process in industry. Although several proposals have been suggested by researchers and academics, little is known of the practices that are actually followed in industry. Objective. We investigate the SoTA with respect to requirements el… ▽ More Context. Requirements engineering remains a discipline that is faced with a large number of challenges, including the implementation of a requirements elicitation process in industry. Although several proposals have been suggested by researchers and academics, little is known of the practices that are actually followed in industry. Objective. We investigate the SoTA with respect to requirements elicitation, examining practitioners' practices. We focus on the techniques, the roles involved, and the challenges associated to the process. Method. We conducted an interview-based survey study involving 24 practitioners from 12 different Swedish IT companies. Results. We found that group interaction techniques, including meetings and workshops, are the most popular type of elicitation techniques that are employed by the practitioners, except in the case of small projects. We noted that customers are frequently involved in the elicitation process, except in the case of market-driven organizations. Technical staff (for example, developers and architects) are more frequently involved in the elicitation process compared to the involvement of business- or strategic staff. Finally, we identified a number of challenges with respect to stakeholders. These challenges include difficulties in understanding and prioritizing their needs. Further, it was noted that requirements instability (i.e., caused by changing needs or priorities) was a predominant challenge. These observations need to be interpreted in the context of the study. Conclusion. The relevant observations regarding the survey participants' experiences should be of interest to the industry; experiences that should be analyzed in the practitioners' context. Researchers may find evidence for the use of academic results in practice, thereby inspiring future theoretical work, as well as further empirical studies in the same area. △ Less

Submitted 23 February, 2021; originally announced February 2021.

Comments: 32 pages, Accepted for publication in the Requirements Engineering Journal, 2021. Cite as Palomares, C., Franch, X., Quer, C. et al. The state-of-practice in requirements elicitation: an extended interview study at 12 companies. Requirements Eng (2021)

A Taxonomy of Assets for the Development of Software-Intensive Products and Services

Authors: Ehsan Zabardast, Javier Gonzalez-Huerta, Tony Gorschek, Darja Šmite, Emil Alégroth, Fabian Fagerholm

Abstract: Context: Develo** software-intensive products or services usually involves a plethora of software artefacts. Assets are artefacts intended to be used more than once and have value for organisations; examples include test cases, code, requirements, and documentation. During the development process, assets might degrade, affecting the effectiveness and efficiency of the development process. Theref… ▽ More Context: Develo** software-intensive products or services usually involves a plethora of software artefacts. Assets are artefacts intended to be used more than once and have value for organisations; examples include test cases, code, requirements, and documentation. During the development process, assets might degrade, affecting the effectiveness and efficiency of the development process. Therefore, assets are an investment that requires continuous management. Identifying assets is the first step for their effective management. However, there is a lack of awareness of what assets and types of assets are common in software-develo** organisations. Most types of assets are understudied, and their state of quality and how they degrade over time have not been well-understood. Method: We perform a systematic literature review and a field study at five companies to study and identify assets to fill the gap in research. The results were analysed qualitatively and summarised in a taxonomy. Results: We create the first comprehensive, structured, yet extendable taxonomy of assets, containing 57 types of assets. Conclusions: The taxonomy serves as a foundation for identifying assets that are relevant for an organisation and enables the study of asset management and asset degradation concepts. △ Less

Submitted 21 October, 2022; v1 submitted 19 February, 2021; originally announced February 2021.

Comments: Submitted to the Journal of Systems and Software (JSS)

Assets in Software Engineering: What are they after all?

Authors: Ehsan Zabardast, Julian Frattini, Javier Gonzalez-Huerta, Daniel Mendez, Tony Gorschek, Krzysztof Wnuk

Abstract: During the development and maintenance of software-intensive products or services, we depend on various artefacts. Some of those artefacts, we deem central to the feasibility of a project and the product's final quality. Typically, these central artefacts are referred to as assets. However, despite their central role in the software development process, little thought is yet invested into what eve… ▽ More During the development and maintenance of software-intensive products or services, we depend on various artefacts. Some of those artefacts, we deem central to the feasibility of a project and the product's final quality. Typically, these central artefacts are referred to as assets. However, despite their central role in the software development process, little thought is yet invested into what eventually characterises as an asset, often resulting in many terms and underlying concepts being mixed and used inconsistently. A precise terminology of assets and related concepts, such as asset degradation, are crucial for setting up a new generation of cost-effective software engineering practices. In this position paper, we critically reflect upon the notion of assets in software engineering. As a starting point, we define the terminology and concepts of assets and extend the reasoning behind them. We explore assets' characteristics and discuss what asset degradation is as well as its various types and the implications that asset degradation might bring for the planning, realisation, and evolution of software-intensive products and services over time. We aspire to contribute to a more standardised definition of assets in software engineering and foster research endeavours and their practical dissemination in a common, more unified direction. △ Less

Submitted 11 July, 2022; v1 submitted 19 January, 2021; originally announced January 2021.

Comments: Manuscript submitted to the Journal of Systems and Software

An Empirical Study on Decision making for Quality Requirements

Authors: Thomas Olsson, Krzystof Wnuk, Tony Gorschek

Abstract: [Context] Quality requirements are important for product success yet often handled poorly. The problems with scope decision lead to delayed handling and an unbalanced scope. [Objective] This study characterizes the scope decision process to understand influencing factors and properties affecting the scope decision of quality requirements. [Method] We studied one company's scope decision process ov… ▽ More [Context] Quality requirements are important for product success yet often handled poorly. The problems with scope decision lead to delayed handling and an unbalanced scope. [Objective] This study characterizes the scope decision process to understand influencing factors and properties affecting the scope decision of quality requirements. [Method] We studied one company's scope decision process over a period of five years. We analyzed the decisions artifacts and interviewed experienced engineers involved in the scope decision process. [Results] Features addressing quality aspects explicitly are a minor part (4.41%) of all features handled. The phase of the product line seems to influence the prevalence and acceptance rate of quality features. Lastly, relying on external stakeholders and upfront analysis seems to lead to long lead-times and an insufficient quality requirements scope. [Conclusions] There is a need to make quality mode explicit in the scope decision process. We propose a scope decision process at a strategic level and a tactical level. The former to address long-term planning and the latter to cater for a speedy process. Furthermore, we believe it is key to balance the stakeholder input with feedback from usage and market in a more direct way than through a long plan-driven process. △ Less

Submitted 12 December, 2018; originally announced December 2018.

Journal ref: The Journal of Systems & Software 149C (2019) pp. 217-233

Knowledge Management Strategies and Processes in Agile Software Development: A Systematic Literature Review

Authors: Raquel Andrade Barros Ouriques, Krzysztof Wnuk, Tony Gorschek, Richard Berntsson Svensson

Abstract: Knowledge-intensive companies that adopt Agile Software Development (ASD) relay on efficient implementation of Knowledge Management (KM) strategies to promotes different Knowledge Processes (KPs) to gain competitive advantage. This study aims to explore how companies that adopt ASD implement KM strategies utilizing practices that promote the KPs in the different organizational layers. Through a sy… ▽ More Knowledge-intensive companies that adopt Agile Software Development (ASD) relay on efficient implementation of Knowledge Management (KM) strategies to promotes different Knowledge Processes (KPs) to gain competitive advantage. This study aims to explore how companies that adopt ASD implement KM strategies utilizing practices that promote the KPs in the different organizational layers. Through a systematic literature review, we analyzed 32 primary studies, selected by automated search and snowballing in the extant literature. To analyze the data, we applied narrative synthesis. Most of the identified KM practices implement personalization strategies (81 %), supported by codification (19 %). Our review shows that the primary studies do not report KM practices in the strategic layer and two of them in the product portfolio layer; on the other hand, in the project layer, the studies report 33 practices that implement personalization strategy, and seven practices that implement codification. KM strategies in ASD promote mainly the knowledge transfer process with practices that stimulate social interaction to share tacit knowledge in the project layer. As a result of using informal communication, a significant amount of knowledge can be lost or not properly transferred to other individuals and, instead of propagating the knowledge, it remains inside a few individuals minds. △ Less

Submitted 13 July, 2018; originally announced July 2018.

Comments: 39 pages