-
How to Integrate Security Compliance Requirements with Agile Software Engineering at Scale?
Authors:
Fabiola Moyón,
Daniel Méndez Fernández,
Kristian Beckers,
Sebastian Klepper
Abstract:
Integrating security into agile software development is an open issue for research and practice. Especially in strongly regulated industries, complexity increases not only when scaling agile practices but also when aiming for compliance with security standards. To achieve security compliance in a large-scale agile context, we developed S2C-SAFe: An extension of the Scaled Agile Framework that is c…
▽ More
Integrating security into agile software development is an open issue for research and practice. Especially in strongly regulated industries, complexity increases not only when scaling agile practices but also when aiming for compliance with security standards. To achieve security compliance in a large-scale agile context, we developed S2C-SAFe: An extension of the Scaled Agile Framework that is compliant to the security standard IEC~62443-4-1 for secure product development.
In this paper, we present the framework and its evaluation by agile and security experts within Siemens' large-scale project ecosystem. We discuss benefits and limitations as well as challenges from a practitioners' perspective. Our results indicate that \ssafe contributes to successfully integrating security compliance with lean and agile development in regulated environments. We also hope to raise awareness for the importance and challenges of integrating security in the scope of Continuous Software Engineering.
△ Less
Submitted 27 May, 2021;
originally announced May 2021.
-
Awareness of Secure Coding Guidelines in the Industry -- A first data analysis
Authors:
Tiago Espinha Gasiba,
Ulrike Lechner,
Maria Pinto-Albuquerque,
Daniel Mendez Fernandez
Abstract:
Software needs to be secure, in particular, when deployed to critical infrastructures. Secure coding guidelines capture practices in industrial software engineering to ensure the security of code. This study aims to assess the level of awareness of secure coding in industrial software engineering, the skills of software developers to spot weaknesses in software code, avoid them, and the organizati…
▽ More
Software needs to be secure, in particular, when deployed to critical infrastructures. Secure coding guidelines capture practices in industrial software engineering to ensure the security of code. This study aims to assess the level of awareness of secure coding in industrial software engineering, the skills of software developers to spot weaknesses in software code, avoid them, and the organizational support to adhere to coding guidelines. The approach draws on well-established theories of policy compliance, neutralization theory, and security-related stress and the authors' many years of experience in industrial software engineering and on lessons identified from training secure coding in the industry. The paper presents the questionnaire design for the online survey and the first analysis of data from the pilot study.
△ Less
Submitted 6 January, 2021;
originally announced January 2021.
-
An Efficient Approach for Reviewing Security-Related Aspects in Agile Requirements Specifications of Web Applications
Authors:
Hugo Villamizar,
Marcos Kalinowski,
Alessandro Garcia,
Daniel Mendez Fernández
Abstract:
Defects in requirements specifications can have severe consequences during the software development lifecycle. Some of them may result in poor product quality and/or time and budget overruns due to incorrect or missing quality characteristics, such as security. This characteristic requires special attention in web applications because they have become a target for manipulating sensible data. Sever…
▽ More
Defects in requirements specifications can have severe consequences during the software development lifecycle. Some of them may result in poor product quality and/or time and budget overruns due to incorrect or missing quality characteristics, such as security. This characteristic requires special attention in web applications because they have become a target for manipulating sensible data. Several concerns make security difficult to deal with. For instance, security requirements are often misunderstood and improperly specified due to lack of security expertise and emphasis on security during early stages of software development. This often leads to unspecified or ill-defined security-related aspects. These concerns become even more challenging in agile contexts, where lightweight documentation is typically produced. To tackle this problem, we designed an approach for reviewing security-related aspects in agile requirements specifications of web applications. Our proposal considers user stories and security specifications as inputs and relates those user stories to security properties via Natural Language Processing. Based on the related security properties, our approach identifies high-level security requirements from the Open Web Application Security Project (OWASP) to be verified, and generates a reading technique to support reviewers in detecting defects. We evaluate our approach via three experiment trials conducted with 56 novice software engineers, measuring effectiveness, efficiency, usefulness, and ease of use. We compare our approach against using: (1) the OWASP high-level security requirements, and (2) a perspective-based approach as proposed in contemporary state of the art. The results strengthen our confidence that using our approach has a positive impact (with large effect size) on the performance of inspectors in terms of effectiveness and efficiency.
△ Less
Submitted 6 September, 2020;
originally announced September 2020.
-
Deep Metric Structured Learning For Facial Expression Recognition
Authors:
Pedro D. Marrero Fernandez,
Tsang Ing Ren,
Tsang Ing Jyh,
Fidel A. Guerrero Peña,
Alexandre Cunha
Abstract:
We propose a deep metric learning model to create embedded sub-spaces with a well defined structure. A new loss function that imposes Gaussian structures on the output space is introduced to create these sub-spaces thus sha** the distribution of the data. Having a mixture of Gaussians solution space is advantageous given its simplified and well established structure. It allows fast discovering o…
▽ More
We propose a deep metric learning model to create embedded sub-spaces with a well defined structure. A new loss function that imposes Gaussian structures on the output space is introduced to create these sub-spaces thus sha** the distribution of the data. Having a mixture of Gaussians solution space is advantageous given its simplified and well established structure. It allows fast discovering of classes within classes and the identification of mean representatives at the centroids of individual classes. We also propose a new semi-supervised method to create sub-classes. We illustrate our methods on the facial expression recognition problem and validate results on the FER+, AffectNet, Extended Cohn-Kanade (CK+), BU-3DFE, and JAFFE datasets. We experimentally demonstrate that the learned embedding can be successfully used for various applications including expression retrieval and emotion recognition.
△ Less
Submitted 5 January, 2022; v1 submitted 18 January, 2020;
originally announced January 2020.
-
J Regularization Improves Imbalanced Multiclass Segmentation
Authors:
Fidel A. Guerrero Peña,
Pedro D. Marrero Fernandez,
Paul T. Tarr,
Tsang Ing Ren,
Elliot M. Meyerowitz,
Alexandre Cunha
Abstract:
We propose a new loss formulation to further advance the multiclass segmentation of cluttered cells under weakly supervised conditions.
We improve the separation of touching and immediate cells, obtaining sharp segmentation boundaries with high adequacy, when we add Youden's $J$ statistic regularization term to the cross entropy loss. This regularization intrinsically supports class imbalance th…
▽ More
We propose a new loss formulation to further advance the multiclass segmentation of cluttered cells under weakly supervised conditions.
We improve the separation of touching and immediate cells, obtaining sharp segmentation boundaries with high adequacy, when we add Youden's $J$ statistic regularization term to the cross entropy loss. This regularization intrinsically supports class imbalance thus eliminating the necessity of explicitly using weights to balance training. Simulations demonstrate this capability and show how the regularization leads to better results by hel** advancing the optimization when cross entropy stalls.
We build upon our previous work on multiclass segmentation by adding yet another training class representing gaps between adjacent cells.
This addition helps the classifier identify narrow gaps as background and no longer as touching regions.
We present results of our methods for 2D and 3D images, from bright field to confocal stacks containing different types of cells, and we show that they accurately segment individual cells after training with a limited number of annotated images, some of which are poorly annotated.
△ Less
Submitted 22 October, 2019;
originally announced October 2019.
-
Generating Requirements Out of Thin Air: Towards Automated Feature Identification for New Apps
Authors:
Tahira Iqbal,
Norbert Seyff,
Daniel Mendez Fernández
Abstract:
App store mining has proven to be a promising technique for requirements elicitation as companies can gain valuable knowledge to maintain and evolve existing apps. However, despite first advancements in using mining techniques for requirements elicitation, little is yet known how to distill requirements for new apps based on existing (similar) solutions and how exactly practitioners would benefit…
▽ More
App store mining has proven to be a promising technique for requirements elicitation as companies can gain valuable knowledge to maintain and evolve existing apps. However, despite first advancements in using mining techniques for requirements elicitation, little is yet known how to distill requirements for new apps based on existing (similar) solutions and how exactly practitioners would benefit from such a technique. In the proposed work, we focus on exploring information (e.g. app store data) provided by the crowd about existing solutions to identify key features of applications in a particular domain. We argue that these discovered features and other related influential aspects (e.g. ratings) can help practitioners(e.g. software developer) to identify potential key features for new applications. To support this argument, we first conducted an interview study with practitioners to understand the extent to which such an approach would find champions in practice. In this paper, we present the first results of our ongoing research in the context of a larger road-map. Our interview study confirms that practitioners see the need for our envisioned approach. Furthermore, we present an early conceptual solution to discuss the feasibility of our approach. However, this manuscript is also intended to foster discussions on the extent to which machine learning can and should be applied to elicit automated requirements on crowd generated data on different forums and to identify further collaborations in this endeavor.
△ Less
Submitted 25 September, 2019;
originally announced September 2019.
-
A Multiple Source Hourglass Deep Network for Multi-Focus Image Fusion
Authors:
Fidel Alejandro Guerrero Peña,
Pedro Diamel Marrero Fernández,
Tsang Ing Ren,
Germano Crispim Vasconcelos,
Alexandre Cunha
Abstract:
Multi-Focus Image Fusion seeks to improve the quality of an acquired burst of images with different focus planes. For solving the task, an activity level measurement and a fusion rule are typically established to select and fuse the most relevant information from the sources. However, the design of this kind of method by hand is really hard and sometimes restricted to solution spaces where the opt…
▽ More
Multi-Focus Image Fusion seeks to improve the quality of an acquired burst of images with different focus planes. For solving the task, an activity level measurement and a fusion rule are typically established to select and fuse the most relevant information from the sources. However, the design of this kind of method by hand is really hard and sometimes restricted to solution spaces where the optimal all-in-focus images are not contained. Then, we propose here two fast and straightforward approaches for image fusion based on deep neural networks. Our solution uses a multiple source Hourglass architecture trained in an end-to-end fashion. Models are data-driven and can be easily generalized for other kinds of fusion problems. A segmentation approach is used for recognition of the focus map, while the weighted average rule is used for fusion. We designed a training loss function for our regression-based fusion function, which allows the network to learn both the activity level measurement and the fusion rule. Experimental results show our approach has comparable results to the state-of-the-art methods with a 60X increase of computational efficiency for 520X520 resolution images.
△ Less
Submitted 28 August, 2019;
originally announced August 2019.
-
A Weakly Supervised Method for Instance Segmentation of Biological Cells
Authors:
Fidel A. Guerrero-Peña,
Pedro D. Marrero Fernandez,
Tsang Ing Ren,
Alexandre Cunha
Abstract:
We present a weakly supervised deep learning method to perform instance segmentation of cells present in microscopy images. Annotation of biomedical images in the lab can be scarce, incomplete, and inaccurate. This is of concern when supervised learning is used for image analysis as the discriminative power of a learning model might be compromised in these situations. To overcome the curse of poor…
▽ More
We present a weakly supervised deep learning method to perform instance segmentation of cells present in microscopy images. Annotation of biomedical images in the lab can be scarce, incomplete, and inaccurate. This is of concern when supervised learning is used for image analysis as the discriminative power of a learning model might be compromised in these situations. To overcome the curse of poor labeling, our method focuses on three aspects to improve learning: i) we propose a loss function operating in three classes to facilitate separating adjacent cells and to drive the optimizer to properly classify underrepresented regions; ii) a contour-aware weight map model is introduced to strengthen contour detection while improving the network generalization capacity; and iii) we augment data by carefully modulating local intensities on edges shared by adjoining regions and to account for possibly weak signals on these edges. Generated probability maps are segmented using different methods, with the watershed based one generally offering the best solutions, specially in those regions where the prevalence of a single class is not clear. The combination of these contributions allows segmenting individual cells on challenging images. We demonstrate our methods in sparse and crowded cell images, showing improvements in the learning process for a fixed network architecture.
△ Less
Submitted 26 August, 2019;
originally announced August 2019.
-
Challenges in Survey Research
Authors:
Stefan Wagner,
Daniel Méndez Fernández,
Michael Felderer,
Daniel Graziotin,
Marcos Kalinowski
Abstract:
While being an important and often used research method, survey research has been less often discussed on a methodological level in empirical software engineering than other types of research. This chapter compiles a set of important and challenging issues in survey research based on experiences with several large-scale international surveys. The chapter covers theory building, sampling, invitatio…
▽ More
While being an important and often used research method, survey research has been less often discussed on a methodological level in empirical software engineering than other types of research. This chapter compiles a set of important and challenging issues in survey research based on experiences with several large-scale international surveys. The chapter covers theory building, sampling, invitation and follow-up, statistical as well as qualitative analysis of survey data and the usage of psychometrics in software engineering surveys.
△ Less
Submitted 27 April, 2020; v1 submitted 16 August, 2019;
originally announced August 2019.
-
An Approach for Reviewing Security-Related Aspects in Agile Requirements Specifications of Web Applications
Authors:
H. Villamizar,
A. A. Neto,
M. Kalinowski,
A. Garcia,
D. Mendez Fernández
Abstract:
Defects in requirements specifications can have severe consequences during the software development lifecycle. Some of them result in overall project failure due to incorrect or missing quality characteristics such as security. There are several concerns that make security difficult to deal with; for instance, (1) when stakeholders discuss general requirements in (review) meetings, they are often…
▽ More
Defects in requirements specifications can have severe consequences during the software development lifecycle. Some of them result in overall project failure due to incorrect or missing quality characteristics such as security. There are several concerns that make security difficult to deal with; for instance, (1) when stakeholders discuss general requirements in (review) meetings, they are often not aware that they should also discuss security-related topics, and (2) they typically do not have enough security expertise. These concerns become even more challenging in agile development contexts, where lightweight documentation is typically involved. The goal of this paper is to design and evaluate an approach to support reviewing security-related aspects in agile requirements specifications of web applications. The designed approach considers user stories and security specifications as input and relates those user stories to security properties via Natural Language Processing (NLP) techniques. Based on the related security properties, our approach then identifies high-level security requirements from the Open Web Application Security Project (OWASP) to be verified and generates a focused reading techniques to support reviewers in detecting detects. We evaluate our approach via two controlled experiment trials, comparing the effectiveness and efficiency of novice inspectors verifying security aspects in agile requirements using our reading technique against using the complete list of OWASP high-level security requirements. The (statistically significant) results indicate that using the reading technique has a positive impact (with very large effect size) on the performance of inspectors in terms of effectiveness and efficiency.
△ Less
Submitted 27 June, 2019;
originally announced June 2019.
-
Open Science in Software Engineering
Authors:
Daniel Méndez Fernández,
Daniel Graziotin,
Stefan Wagner,
Heidi Seibold
Abstract:
Open science describes the movement of making any research artefact available to the public and includes, but is not limited to, open access, open data, and open source. While open science is becoming generally accepted as a norm in other scientific disciplines, in software engineering, we are still struggling in adapting open science to the particularities of our discipline, rendering progress in…
▽ More
Open science describes the movement of making any research artefact available to the public and includes, but is not limited to, open access, open data, and open source. While open science is becoming generally accepted as a norm in other scientific disciplines, in software engineering, we are still struggling in adapting open science to the particularities of our discipline, rendering progress in our scientific community cumbersome. In this chapter, we reflect upon the essentials in open science for software engineering including what open science is, why we should engage in it, and how we should do it. We particularly draw from our experiences made as conference chairs implementing open science initiatives and as researchers actively engaging in open science to critically discuss challenges and pitfalls, and to address more advanced topics such as how and under which conditions to share preprints, what infrastructure and licence model to cover, or how do it within the limitations of different reviewing models, such as double-blind reviewing. Our hope is to help establishing a common ground and to contribute to make open science a norm also in software engineering.
△ Less
Submitted 13 August, 2019; v1 submitted 13 April, 2019;
originally announced April 2019.
-
FERAtt: Facial Expression Recognition with Attention Net
Authors:
Pedro D. Marrero Fernandez,
Fidel A. Guerrero Peña,
Tsang Ing Ren,
Alexandre Cunha
Abstract:
We present a new end-to-end network architecture for facial expression recognition with an attention model. It focuses attention in the human face and uses a Gaussian space representation for expression recognition. We devise this architecture based on two fundamental complementary components: (1) facial image correction and attention and (2) facial expression representation and classification. Th…
▽ More
We present a new end-to-end network architecture for facial expression recognition with an attention model. It focuses attention in the human face and uses a Gaussian space representation for expression recognition. We devise this architecture based on two fundamental complementary components: (1) facial image correction and attention and (2) facial expression representation and classification. The first component uses an encoder-decoder style network and a convolutional feature extractor that are pixel-wise multiplied to obtain a feature attention map. The second component is responsible for obtaining an embedded representation and classification of the facial expression. We propose a loss function that creates a Gaussian structure on the representation space. To demonstrate the proposed method, we create two larger and more comprehensive synthetic datasets using the traditional BU3DFE and CK+ facial datasets. We compared results with the PreActResNet18 baseline. Our experiments on these datasets have shown the superiority of our approach in recognizing facial expressions.
△ Less
Submitted 8 February, 2019;
originally announced February 2019.
-
Do We Preach What We Practice? Investigating the Practical Relevance of Requirements Engineering Syllabi - The IREB Case
Authors:
Daniel Méndez Fernández,
Xavier Franch,
Norbert Seyff,
Michael Felderer,
Martin Glinz,
Marcos Kalinowski,
Andreas Volgelsang,
Stefan Wagner,
Stan Bühne,
Kim Lauenroth
Abstract:
Nowadays, there exist a plethora of different educational syllabi for Requirements Engineering (RE), all aiming at incorporating practically relevant educational units (EUs). Many of these syllabi are based, in one way or the other, on the syllabi provided by the International Requirements Engineering Board (IREB), a non-profit organisation devoted to standardised certification programs for RE. IR…
▽ More
Nowadays, there exist a plethora of different educational syllabi for Requirements Engineering (RE), all aiming at incorporating practically relevant educational units (EUs). Many of these syllabi are based, in one way or the other, on the syllabi provided by the International Requirements Engineering Board (IREB), a non-profit organisation devoted to standardised certification programs for RE. IREB syllabi are developed by RE experts and are, thus, based on the assumption that they address topics of practical relevance. However, little is known about to what extent practitioners actually perceive those contents as useful. We have started a study to investigate the relevance of the EUs included in the IREB Foundation Level certification programme. In a first phase reported in this paper, we have surveyed practitioners mainly from DACH countries (Germany, Austria and Switzerland) participating in the IREB certification. Later phases will widen the scope both by including other countries and by not requiring IREB-certified participants. The results shall foster a critical reflection on the practical relevance of EUs built upon the de-facto standard syllabus of IREB.
△ Less
Submitted 10 June, 2019; v1 submitted 1 February, 2019;
originally announced February 2019.
-
Burst ranking for blind multi-image deblurring
Authors:
Fidel A. Guerrero Peña,
Pedro D. Marrero Fernández,
Tsang Ing Ren,
Jorge J. G. Leandro,
Ricardo Nishihara
Abstract:
We propose a new incremental aggregation algorithm for multi-image deblurring with automatic image selection. The primary motivation is that current bursts deblurring methods do not handle well situations in which misalignment or out-of-context frames are present in the burst. These real-life situations result in poor reconstructions or manual selection of the images that will be used to deblur. A…
▽ More
We propose a new incremental aggregation algorithm for multi-image deblurring with automatic image selection. The primary motivation is that current bursts deblurring methods do not handle well situations in which misalignment or out-of-context frames are present in the burst. These real-life situations result in poor reconstructions or manual selection of the images that will be used to deblur. Automatically selecting best frames within the burst to improve the base reconstruction is challenging because the amount of possible images fusions is equal to the power set cardinal. Here, we approach the multi-image deblurring problem as a two steps process. First, we successfully learn a comparison function to rank a burst of images using a deep convolutional neural network. Then, an incremental Fourier burst accumulation with a reconstruction degradation mechanism is applied fusing only less blurred images that are sufficient to maximize the reconstruction quality. Experiments with the proposed algorithm have shown superior results when compared to other similar approaches, outperforming other methods described in the literature in previously described situations. We validate our findings on several synthetic and real datasets.
△ Less
Submitted 30 October, 2018; v1 submitted 29 October, 2018;
originally announced October 2018.
-
Fast and Robust Multiple ColorChecker Detection using Deep Convolutional Neural Networks
Authors:
Pedro D. Marrero Fernandez,
Fidel A. Guerrero-Peña,
Tsang Ing Ren,
Jorge J. G. Leandro
Abstract:
ColorCheckers are reference standards that professional photographers and filmmakers use to ensure predictable results under every lighting condition. The objective of this work is to propose a new fast and robust method for automatic ColorChecker detection. The process is divided into two steps: (1) ColorCheckers localization and (2) ColorChecker patches recognition. For the ColorChecker localiza…
▽ More
ColorCheckers are reference standards that professional photographers and filmmakers use to ensure predictable results under every lighting condition. The objective of this work is to propose a new fast and robust method for automatic ColorChecker detection. The process is divided into two steps: (1) ColorCheckers localization and (2) ColorChecker patches recognition. For the ColorChecker localization, we trained a detection convolutional neural network using synthetic images. The synthetic images are created with the 3D models of the ColorChecker and different background images. The output of the neural networks are the bounding box of each possible ColorChecker candidates in the input image. Each bounding box defines a cropped image which is evaluated by a recognition system, and each image is canonized with regards to color and dimensions. Subsequently, all possible color patches are extracted and grouped with respect to the center's distance. Each group is evaluated as a candidate for a ColorChecker part, and its position in the scene is estimated. Finally, a cost function is applied to evaluate the accuracy of the estimation. The method is tested using real and synthetic images. The proposed method is fast, robust to overlaps and invariant to affine projections. The algorithm also performs well in case of multiple ColorCheckers detection.
△ Less
Submitted 19 October, 2018;
originally announced October 2018.
-
A Systematic Map** Study on Security in Agile Requirements Engineering
Authors:
H. Villamizar,
M. Kalinowski,
M. Viana,
D. Méndez Fernández
Abstract:
[Background] The rapidly changing business environments in which many companies operate is challenging traditional Requirements Engineering (RE) approaches. This gave rise to agile approaches for RE. Security, at the same time, is an essential non-functional requirement that still tends to be difficult to address in agile development contexts. Given the fuzzy notion of "agile" in context of RE and…
▽ More
[Background] The rapidly changing business environments in which many companies operate is challenging traditional Requirements Engineering (RE) approaches. This gave rise to agile approaches for RE. Security, at the same time, is an essential non-functional requirement that still tends to be difficult to address in agile development contexts. Given the fuzzy notion of "agile" in context of RE and the difficulties of appropriately handling security requirements, the overall understanding of how to handle security requirements in agile RE is still vague. [Objective] Our goal is to characterize the publication landscape of approaches that handle security requirements in agile software projects. [Method] We conducted a systematic map** to outline relevant work and contemporary gaps for future research. [Results] In total, we identified 21 studies that met our inclusion criteria, dated from 2005 to 2017. We found that the approaches typically involve modifying agile methods, introducing new artifacts (e.g., extending the concept of user story to abuser story), or introducing guidelines to handle security issues. We also identified limitations of using these approaches related to environment, people, effort and resources. [Conclusion] Our analysis suggests that more effort needs to be invested into empirically evaluating the existing approaches and that there is an avenue for future research in the direction of mitigating the identified limitations.
△ Less
Submitted 4 June, 2018;
originally announced June 2018.
-
Artefacts in Software Engineering: A Fundamental Positioning
Authors:
D. Méndez Fernández,
W. Böhm,
A. Vogelsang,
J. Mund,
M. Broy,
M. Kuhrmann,
T. Weyer
Abstract:
Artefacts play a vital role in software and systems development processes. Other terms like documents, deliverables, or work products are widely used in software development communities instead of the term artefact. In the following, we use the term `artefact' including all these other terms. Despite its relevance, the exact denotation of the term `artefact' is still not clear due to a variety of…
▽ More
Artefacts play a vital role in software and systems development processes. Other terms like documents, deliverables, or work products are widely used in software development communities instead of the term artefact. In the following, we use the term `artefact' including all these other terms. Despite its relevance, the exact denotation of the term `artefact' is still not clear due to a variety of different understandings of the term and to a careless negligent usage. This often leads to approaches being grounded in a fuzzy, unclear understanding of the essential concepts involved. In fact, there does not exist a common terminology. Therefore, it is our goal that the term artefact be standardised so that researchers and practitioners have a common understanding for discussions and contributions. In this position paper, we provide a positioning and critical reflection upon the notion of artefact in software engineering at different levels of perception and how these relate to each other. We further contribute a meta model that provides a description of an artefact that is independent from any underlying process model. This meta model defines artefacts at three levels. Abstraction and refinement relations between these levels allow correlating artefacts to each other and defining the notion of related, refined, and equivalent artefacts. Our contribution shall foster the long overdue and too often underestimated terminological discussion on what artefacts are to provide a common ground with clearer concepts and principles for future software engineering contributions, such as the design of artefact-oriented development processes and tools.
△ Less
Submitted 26 November, 2018; v1 submitted 31 May, 2018;
originally announced June 2018.
-
Empirical Software Engineering: From Discipline to Interdiscipline
Authors:
Daniel Méndez Fernández,
Jan-Hendrik Passoth
Abstract:
Empirical software engineering has received much attention in recent years and coined the shift from a more design-science-driven engineering discipline to an insight-oriented, and theory-centric one. Yet, we still face many challenges, among which some increase the need for interdisciplinary research. This is especially true for the investigation of human-centric aspects of software engineering.…
▽ More
Empirical software engineering has received much attention in recent years and coined the shift from a more design-science-driven engineering discipline to an insight-oriented, and theory-centric one. Yet, we still face many challenges, among which some increase the need for interdisciplinary research. This is especially true for the investigation of human-centric aspects of software engineering. Although we can already observe an increased recognition of the need for more interdisciplinary research in (empirical) software engineering, such research configurations come with challenges barely discussed from a scientific point of view. In this position paper, we critically reflect upon the epistemological setting of empirical software engineering and elaborate its configuration as an Interdiscipline. In particular, we (1) elaborate a pragmatic view on empirical research for software engineering reflecting a cyclic process for knowledge creation, (2) motivate a path towards symmetrical interdisciplinary research, and (3) adopt five rules of thumb from other interdisciplinary collaborations in our field before concluding with new emerging challenges. This shall support stop** to treating empirical software engineering as a develo** discipline moving towards a paradigmatic stage of normal science, but as a configuration of symmetric interdisciplinary teams and research methods.
△ Less
Submitted 17 November, 2018; v1 submitted 21 May, 2018;
originally announced May 2018.
-
Status Quo in Requirements Engineering: A Theory and a Global Family of Surveys
Authors:
Stefan Wagner,
Daniel Méndez Fernández,
Michael Felderer,
Antonio Vetró,
Marcos Kalinowski,
Roel Wieringa,
Dietmar Pfahl,
Tayana Conte,
Marie-Therese Christiansson,
Desmond Greer,
Casper Lassenius,
Tomi Männistö,
Maleknaz Nayebi,
Markku Oivo,
Birgit Penzenstadler,
Rafael Prikladnicki,
Guenther Ruhe,
André Schekelmann,
Sagar Sen,
Rodrigo Spínola,
Ahmed Tuzcu,
Jose Luis de la Vara,
Dietmar Winkler
Abstract:
Requirements Engineering (RE) has established itself as a software engineering discipline during the past decades. While researchers have been investigating the RE discipline with a plethora of empirical studies, attempts to systematically derive an empirically-based theory in context of the RE discipline have just recently been started. However, such a theory is needed if we are to define and mot…
▽ More
Requirements Engineering (RE) has established itself as a software engineering discipline during the past decades. While researchers have been investigating the RE discipline with a plethora of empirical studies, attempts to systematically derive an empirically-based theory in context of the RE discipline have just recently been started. However, such a theory is needed if we are to define and motivate guidance in performing high quality RE research and practice. We aim at providing an empirical and valid foundation for a theory of RE, which helps software engineers establish effective and efficient RE processes. We designed a survey instrument and theory that has now been replicated in 10 countries world-wide. We evaluate the propositions of the theory with bootstrapped confidence intervals and derive potential explanations for the propositions. We report on the underlying theory and the full results obtained from the replication studies with participants from 228 organisations. Our results represent a substantial step forward towards develo** an empirically-based theory of RE giving insights into current practices with RE processes. The results reveal, for example, that there are no strong differences between organisations in different countries and regions, that interviews, facilitated meetings and prototy** are the most used elicitation techniques, that requirements are often documented textually, that traces between requirements and code or design documents is common, requirements specifications themselves are rarely changed and that requirements engineering (process) improvement endeavours are mostly intrinsically motivated. Our study establishes a theory that can be used as starting point for many further studies for more detailed investigations. Practitioners can use the results as theory-supported guidance on selecting suitable RE methods and techniques.
△ Less
Submitted 17 December, 2018; v1 submitted 21 May, 2018;
originally announced May 2018.
-
Multiclass Weighted Loss for Instance Segmentation of Cluttered Cells
Authors:
Fidel A. Guerrero-Pena,
Pedro D. Marrero Fernandez,
Tsang Ing Ren,
Mary Yui,
Ellen Rothenberg,
Alexandre Cunha
Abstract:
We propose a new multiclass weighted loss function for instance segmentation of cluttered cells. We are primarily motivated by the need of developmental biologists to quantify and model the behavior of blood T-cells which might help us in understanding their regulation mechanisms and ultimately help researchers in their quest for develo** an effective immuno-therapy cancer treatment. Segmenting…
▽ More
We propose a new multiclass weighted loss function for instance segmentation of cluttered cells. We are primarily motivated by the need of developmental biologists to quantify and model the behavior of blood T-cells which might help us in understanding their regulation mechanisms and ultimately help researchers in their quest for develo** an effective immuno-therapy cancer treatment. Segmenting individual touching cells in cluttered regions is challenging as the feature distribution on shared borders and cell foreground are similar thus difficulting discriminating pixels into proper classes. We present two novel weight maps applied to the weighted cross entropy loss function which take into account both class imbalance and cell geometry. Binary ground truth training data is augmented so the learning model can handle not only foreground and background but also a third touching class. This framework allows training using U-Net. Experiments with our formulations have shown superior results when compared to other similar schemes, outperforming binary class models with significant improvement of boundary adequacy and instance detection. We validate our results on manually annotated microscope images of T-cells.
△ Less
Submitted 21 February, 2018;
originally announced February 2018.
-
Supporting Requirements Engineering Research that Industry Needs: The Naming the Pain in Requirements Engineering Initiative
Authors:
Daniel Méndez Fernández
Abstract:
In light of the 40th jubilee of Requirements Engineering (RE), roughly 40 experts met in Switzerland to discuss where our discipline stands today. As of today, the common view is, indisputably, that RE as a discipline is stable and respected, as pointed out by Sarah Gregory when covering the seminar in her column to which articles like this one are invited to present ongoing research. However, it…
▽ More
In light of the 40th jubilee of Requirements Engineering (RE), roughly 40 experts met in Switzerland to discuss where our discipline stands today. As of today, the common view is, indisputably, that RE as a discipline is stable and respected, as pointed out by Sarah Gregory when covering the seminar in her column to which articles like this one are invited to present ongoing research. However, it is also evident that after 40 years of promising research, conducting research that industry needs is still an ongoing challenge. Research that industry needs means research that solves industrial problems practitioners face; but do we really understand those problems? Here, I want to recapitulate on this research challenge and outline an initiative, the Naming the Pain in Requirements Engineering Initiative, that aims at tackling this problem.
△ Less
Submitted 12 October, 2017;
originally announced October 2017.
-
On Evidence-based Risk Management in Requirements Engineering
Authors:
Daniel Méndez Fernández,
Michaela Tießler,
Marcos Kalinowski,
Michael Felderer,
Marco Kuhrmann
Abstract:
Background: The sensitivity of Requirements Engineering (RE) to the context makes it difficult to efficiently control problems therein, thus, hampering an effective risk management devoted to allow for early corrective or even preventive measures. Problem: There is still little empirical knowledge about context-specific RE phenomena which would be necessary for an effective context- sensitive risk…
▽ More
Background: The sensitivity of Requirements Engineering (RE) to the context makes it difficult to efficiently control problems therein, thus, hampering an effective risk management devoted to allow for early corrective or even preventive measures. Problem: There is still little empirical knowledge about context-specific RE phenomena which would be necessary for an effective context- sensitive risk management in RE. Goal: We propose and validate an evidence-based approach to assess risks in RE using cross-company data about problems, causes and effects. Research Method: We use survey data from 228 companies and build a probabilistic network that supports the forecast of context-specific RE phenomena. We implement this approach using spreadsheets to support a light-weight risk assessment. Results: Our results from an initial validation in 6 companies strengthen our confidence that the approach increases the awareness for individual risk factors in RE, and the feedback further allows for disseminating our approach into practice.
△ Less
Submitted 31 July, 2017; v1 submitted 1 July, 2017;
originally announced July 2017.
-
A Community's Perspective on the Status and Future of Peer Review in Software Engineering
Authors:
Lutz Prechelt,
Daniel Graziotin,
Daniel Méndez Fernández
Abstract:
Context: Pre-publication peer review of scientific articles is considered a key element of the research process in software engineering, yet it is often perceived as not to work fully well.
Objective: We aim at understanding the perceptions of and attitudes towards peer review of authors and reviewers at one of software engineering's most prestigious venues, the International Conference on Softw…
▽ More
Context: Pre-publication peer review of scientific articles is considered a key element of the research process in software engineering, yet it is often perceived as not to work fully well.
Objective: We aim at understanding the perceptions of and attitudes towards peer review of authors and reviewers at one of software engineering's most prestigious venues, the International Conference on Software Engineering (ICSE).
Method: We invited 932 ICSE 2014/15/16 authors and reviewers to participate in a survey with 10 closed and 9 open questions.
Results: We present a multitude of results, such as: Respondents perceive only one third of all reviews to be good, yet one third as useless or misleading; they propose double-blind or zero-blind reviewing regimes for improvement; they would like to see showable proofs of (good) reviewing work be introduced; attitude change trends are weak.
Conclusion: The perception of the current state of software engineering peer review is fairly negative. Also, we found hardly any trend that suggests reviewing will improve by itself over time; the community will have to make explicit efforts. Fortunately, our (mostly senior) respondents appear more open for trying different peer reviewing regimes than we had expected.
△ Less
Submitted 30 October, 2017; v1 submitted 22 June, 2017;
originally announced June 2017.
-
How do Practitioners Perceive the Relevance of Requirements Engineering Research? An Ongoing Study
Authors:
X. Franch,
D. Méndez Fernández,
M. Oriol,
A. Vogelsang,
R. Heldal,
E. Knauss,
G. Horta Travassos,
J. C. Carver,
O. Dieste,
T. Zimmermann
Abstract:
The relevance of Requirements Engineering (RE) research to practitioners is a prerequisite for problem-driven research in the area and key for a long-term dissemination of research results to everyday practice. To better understand how industry practitioners perceive the practical relevance of RE research, we have initiated the RE-Pract project, an international collaboration conducting an empiric…
▽ More
The relevance of Requirements Engineering (RE) research to practitioners is a prerequisite for problem-driven research in the area and key for a long-term dissemination of research results to everyday practice. To better understand how industry practitioners perceive the practical relevance of RE research, we have initiated the RE-Pract project, an international collaboration conducting an empirical study. This project opts for a replication of previous work done in two different domains and relies on survey research. To this end, we have designed a survey to be sent to several hundred industry practitioners at various companies around the world and ask them to rate their perceived practical relevance of the research described in a sample of 418 RE papers published between 2010 and 2015 at the RE, ICSE, FSE, ESEC/FSE, ESEM and REFSQ conferences. In this paper, we summarise our research protocol and present the current status of our study and the planned future steps.
△ Less
Submitted 14 June, 2017; v1 submitted 17 May, 2017;
originally announced May 2017.
-
Requirements Engineering Practice and Problems in Agile Projects: Results from an International Survey
Authors:
Stefan Wagner,
Daniel Méndez Fernández,
Michael Felderer,
Marcos Kalinowski
Abstract:
Requirements engineering (RE) is considerably different in agile development than in more traditional development processes. Yet, there is little empirical knowledge on the state of the practice and contemporary problems in agile RE. As part of a bigger survey initiative (Naming the Pain in Requirements Engineering), we build an empirical basis on such aspects of agile RE. Based on the responses o…
▽ More
Requirements engineering (RE) is considerably different in agile development than in more traditional development processes. Yet, there is little empirical knowledge on the state of the practice and contemporary problems in agile RE. As part of a bigger survey initiative (Naming the Pain in Requirements Engineering), we build an empirical basis on such aspects of agile RE. Based on the responses of representatives from 92 different organisations, we found that agile RE concentrates on free-text documentation of requirements elicited with a variety of techniques. Often, traces between requirements and code are explicitly managed and also software testing and RE are aligned. Furthermore, continuous improvement of RE is performed due to intrinsic motivation. Important experienced problems include unclear requirements and communication flaws. Overall, we found that most organisations conduct RE in a way we would expect and that agile RE is in several aspects not so different from RE in other development processes.
△ Less
Submitted 24 March, 2017;
originally announced March 2017.
-
Does Quality of Requirements Specifications matter? Combined Results of Two Empirical Studies
Authors:
Jakob Mund,
Henning Femmer,
Daniel Méndez Fernández,
Jonas Eckhardt
Abstract:
Background: Requirements Engineering is crucial for project success, and to this end, many measures for quality assurance of the software requirements specification (SRS) have been proposed. Goal: However, we still need an empirical understanding on the extent to which SRS are created and used in practice, as well as the degree to which the quality of an SRS matters to subsequent development activ…
▽ More
Background: Requirements Engineering is crucial for project success, and to this end, many measures for quality assurance of the software requirements specification (SRS) have been proposed. Goal: However, we still need an empirical understanding on the extent to which SRS are created and used in practice, as well as the degree to which the quality of an SRS matters to subsequent development activities. Method: We studied the relevance of SRS by relying on survey research and explored the impact of quality defects in SRS by relying on a controlled experiment. Results: Our results suggest that the relevance of SRS quality depends both on particular project characteristics and what is considered as a quality defect; for instance, the domain of safety critical systems seems to motivate for an intense usage of SRS as a means for communication whereas defects hampering the pragmatic quality do not seem to be as relevant as initially thought. Conclusion: Efficient and effective quality assurance measures must be specific for carefully characterized contexts and carefully select defect classes.
△ Less
Submitted 24 February, 2017;
originally announced February 2017.
-
How to specify Non-functional Requirements to support seamless modeling? A Study Design and Preliminary Results
Authors:
Jonas Eckhardt,
Daniel Méndez Fernández,
Andreas Vogelsang
Abstract:
Context: Seamless model-based development provides integrated chains of models, covering all software engineering phases. Non-functional requirements (NFRs), like reusability, further play a vital role in software and systems engineering, but are often neglected in research and practice. It is still unclear how to integrate NFRs in a seamless model-based development. Goal: Our long-term goal is to…
▽ More
Context: Seamless model-based development provides integrated chains of models, covering all software engineering phases. Non-functional requirements (NFRs), like reusability, further play a vital role in software and systems engineering, but are often neglected in research and practice. It is still unclear how to integrate NFRs in a seamless model-based development. Goal: Our long-term goal is to develop a theory on the specification of NFRs such that they can be integrated in seamless model-based development. Method: Our overall study design includes a multi-staged procedure to infer an empirically founded theory on specifying NFRs to support seamless modeling. In this short paper, we present the study design and provide a discussion of (i) preliminary results obtained from a sample, and (ii) current issues related to the design. Results: Our study already shows significant fields of improvement, e.g., the low agreement during the classification. However, the results indicate to interesting points; for example, many of commonly used NFR classes concern system modeling concepts in a way that shows how blurry the borders between functional and NFRs are. Conclusions: We conclude so far that our overall study design seems suitable to obtain the envisioned theory in the long run, but we could also show current issues that are worth discussing within the empirical software engineering community. The main goal of this contribution is not to present and discuss current results only, but to foster discussions on the issues related to the integration of NFRs in seamless modeling in general and, in particular, discussions on open methodological issues.
△ Less
Submitted 24 February, 2017;
originally announced February 2017.
-
Lessons Learnt in Conducting Survey Research
Authors:
Marco Torchiano,
Daniel Méndez Fernández,
Guilherme Horta Travassos,
Rafael Maiani de Mello
Abstract:
Context: Surveys constitute an valuable tool to capture a large-scale snapshot of the state of the practice. Apparently trivial to adopt, surveys hide, however, several pitfalls that might hinder rendering the result valid and, thus, useful. Goal: We aim at providing an overview of main pitfalls in software engineering surveys and report on practical ways to deal with them. Method: We build on the…
▽ More
Context: Surveys constitute an valuable tool to capture a large-scale snapshot of the state of the practice. Apparently trivial to adopt, surveys hide, however, several pitfalls that might hinder rendering the result valid and, thus, useful. Goal: We aim at providing an overview of main pitfalls in software engineering surveys and report on practical ways to deal with them. Method: We build on the experiences we collected in conducting many studies and distill the main lessons learnt. Results: The eight lessons learnt we report cover different aspects of the survey process ranging from the design of initial research objectives to the design of a questionnaire. Conclusions: Our hope is that by sharing our lessons learnt, combined with a disciplined application of the general survey theory, we contribute to improving the quality of the research results achievable by employing software engineering surveys.
△ Less
Submitted 24 February, 2017; v1 submitted 19 February, 2017;
originally announced February 2017.
-
On the Use of Variability Operations in the V-Modell XT Software Process Line
Authors:
Marco Kuhrmann,
Daniel Méndez Fernández,
Thomas Ternité
Abstract:
Software process lines provide a systematic approach to develop and manage software processes. It defines a reference process containing general process assets, whereas a well-defined customization approach allows process engineers to create new process variants, e.g., by extending or modifying process assets. Variability operations are an instrument to realize flexibility by explicitly declaring…
▽ More
Software process lines provide a systematic approach to develop and manage software processes. It defines a reference process containing general process assets, whereas a well-defined customization approach allows process engineers to create new process variants, e.g., by extending or modifying process assets. Variability operations are an instrument to realize flexibility by explicitly declaring required modifications, which are applied to create a procedurally generated company-specific process. However, little is known about which variability operations are suitable in practice. In this article, we present a study on the feasibility of variability operations to support the development of software process lines in the context of the V-Modell XT. We analyze which variability operations are defined and practically used. We provide an initial catalog of variability operations as an improvement proposal for other process models. Our findings show that 69 variability operation types are defined across several metamodel versions of which, however, 25 remain unused. The found variability operations allow for systematically modifying the content of process model elements and the process documentation, and they allow for altering the structure of a process model and its description. Furthermore, we also find that variability operations can help process engineers to compensate process metamodel evolution.
△ Less
Submitted 19 February, 2017;
originally announced February 2017.
-
From Pragmatic to Systematic Software Process Improvement: An Evaluated Approach
Authors:
Marco Kuhrmann,
Daniel Méndez Fernández
Abstract:
Software processes improvement (SPI) is a challenging task, as many different stakeholders, project settings, and contexts and goals need to be considered. SPI projects are often operated in a complex and volatile environment and, thus, require a sound management that is resource-intensive requiring many stakeholders to contribute to the process assessment, analysis, design, realisation, and deplo…
▽ More
Software processes improvement (SPI) is a challenging task, as many different stakeholders, project settings, and contexts and goals need to be considered. SPI projects are often operated in a complex and volatile environment and, thus, require a sound management that is resource-intensive requiring many stakeholders to contribute to the process assessment, analysis, design, realisation, and deployment. Although there exist many valuable SPI approaches, none address the needs of both process engineers and project managers. This article presents an Artefact-based Software Process Improvement & Management approach (ArSPI) that closes this gap. ArSPI was developed and tested across several SPI projects in large organisations in Germany and Eastern Europe. The approach further encompasses a template for initiating, performing, and managing SPI projects by defining a set of 5 key artefacts and 24 support artefacts. We present ArSPI and discus results of its validation indicating ArSPI to be a helpful instrument to set up and steer SPI projects.
△ Less
Submitted 19 February, 2017;
originally announced February 2017.
-
Supporting Defect Causal Analysis in Practice with Cross-Company Data on Causes of Requirements Engineering Problems
Authors:
Marcos Kalinowski,
Pablo Curty,
Aline Paes,
Alexandre Ferreira,
Rodrigo Spínola,
Daniel Méndez Fernández,
Michael Felderer,
Stefan Wagner
Abstract:
[Context] Defect Causal Analysis (DCA) represents an efficient practice to improve software processes. While knowledge on cause-effect relations is helpful to support DCA, collecting cause-effect data may require significant effort and time. [Goal] We propose and evaluate a new DCA approach that uses cross-company data to support the practical application of DCA. [Method] We collected cross-compan…
▽ More
[Context] Defect Causal Analysis (DCA) represents an efficient practice to improve software processes. While knowledge on cause-effect relations is helpful to support DCA, collecting cause-effect data may require significant effort and time. [Goal] We propose and evaluate a new DCA approach that uses cross-company data to support the practical application of DCA. [Method] We collected cross-company data on causes of requirements engineering problems from 74 Brazilian organizations and built a Bayesian network. Our DCA approach uses the diagnostic inference of the Bayesian network to support DCA sessions. We evaluated our approach by applying a model for technology transfer to industry and conducted three consecutive evaluations: (i) in academia, (ii) with industry representatives of the Fraunhofer Project Center at UFBA, and (iii) in an industrial case study at the Brazilian National Development Bank (BNDES). [Results] We received positive feedback in all three evaluations and the cross-company data was considered helpful for determining main causes. [Conclusions] Our results strengthen our confidence in that supporting DCA with cross-company data is promising and should be further investigated.
△ Less
Submitted 13 February, 2017;
originally announced February 2017.
-
Where do we stand in requirements engineering improvement today? First results from a map** study
Authors:
Daniel Méndez Fernández,
Saahil Ognawala,
Stefan Wagner,
Maya Daneva
Abstract:
Context: Requirements engineering process improvement (REPI) approaches have gained much attention in research and practice. Goal: So far, there is no comprehensive view on the research in REPI in terms of solutions and current state of reported evidence. We aims to provide an overview on the existing solutions, their underlying principles and their research type facets, i.e. their state of empiri…
▽ More
Context: Requirements engineering process improvement (REPI) approaches have gained much attention in research and practice. Goal: So far, there is no comprehensive view on the research in REPI in terms of solutions and current state of reported evidence. We aims to provide an overview on the existing solutions, their underlying principles and their research type facets, i.e. their state of empirical evidence. Method: To this end, we conducted a systematic map** study of the REPI publication space. Results: This paper reports on the first findings regarding research type facets of the contributions as well as selected methodological principles. We found a strong focus in the existing research on solution proposals for REPI approaches that concentrate on normative assessments and benchmarks of the RE activities rather than on holistic RE improvements according to individual goals of companies. Conclusions: We conclude, so far, that there is a need to broaden the work and to investigate more problem-driven REPI which also targets the improvement of the quality of the underlying RE artefacts, which currently seem out of scope.
△ Less
Submitted 19 January, 2017;
originally announced January 2017.
-
On the Pragmatic Design of Literature Studies in Software Engineering: An Experience-based Guideline
Authors:
M. Kuhrmann,
D. Méndez Fernández,
M. Daneva
Abstract:
Systematic literature studies have received much attention in empirical software engineering in recent years. They have become a powerful tool to collect and structure reported knowledge in a systematic and reproducible way. We distinguish systematic literature reviews to systematically analyze reported evidence in depth, and systematic map** studies to structure a field of interest in a broader…
▽ More
Systematic literature studies have received much attention in empirical software engineering in recent years. They have become a powerful tool to collect and structure reported knowledge in a systematic and reproducible way. We distinguish systematic literature reviews to systematically analyze reported evidence in depth, and systematic map** studies to structure a field of interest in a broader, usually quantified manner. Due to the rapidly increasing body of knowledge in software engineering, researchers who want to capture the published work in a domain often face an extensive amount of publications, which need to be screened, rated for relevance, classified, and eventually analyzed. Although there are several guidelines to conduct literature studies, they do not yet help researchers co** with the specific difficulties encountered in the practical application of these guidelines. In this article, we present an experience-based guideline to aid researchers in designing systematic literature studies with special emphasis on the data collection and selection procedures. Our guideline aims at providing a blueprint for a practical and pragmatic path through the plethora of currently available practices and deliverables capturing the dependencies among the single steps. The guideline emerges from various map** studies and literature reviews conducted by the authors and provides recommendations for the general study design, data collection, and study selection procedures. Finally, we share our experiences and lessons learned in applying the different practices of the proposed guideline.
△ Less
Submitted 12 December, 2016;
originally announced December 2016.
-
In Quest for Proper Mediums for Technology Transfer in Software Engineering
Authors:
F. Grigoleit,
A. Vetrò,
D. Méndez Fernández,
W. Böhm,
P. Diebold
Abstract:
Successful transfer of the results of research projects into practice is of great interest to all project participants. It can be assumed that different transfer mediums fulfill technology transfer (TT) with different levels of success and that they are impaired by different kinds of barriers. The goal of this study is to gain a better understanding about the different mediums used for TT in softw…
▽ More
Successful transfer of the results of research projects into practice is of great interest to all project participants. It can be assumed that different transfer mediums fulfill technology transfer (TT) with different levels of success and that they are impaired by different kinds of barriers. The goal of this study is to gain a better understanding about the different mediums used for TT in software engineering, and to identify barriers weakening the success of the application of such mediums. We conducted an exploratory study implemented by a survey in the context of a German research project with a broad range of used mediums. The main reported barriers were low expectations of usefulness, no awareness of existence, lack of resources, or inadequateness in terms of outdated material or being in an immature state. We interpreted our results as symptoms of a lack of a dissemination plan in the project. Further work will be needed to explore the implications for the transfer of research results (knowledge and techniques) to practice.
△ Less
Submitted 5 December, 2016;
originally announced December 2016.
-
Analysing Text in Software Projects
Authors:
S. Wagner,
D. Méndez Fernández
Abstract:
Most of the data produced in software projects is of textual nature: source code, specifications, or documentations. The advances in quantitative analysis methods drove a lot of data analytics in software engineering. This has overshadowed to some degree the importance of texts and their qualitative analysis. Such analysis has, however, merits for researchers and practitioners as well.
In this c…
▽ More
Most of the data produced in software projects is of textual nature: source code, specifications, or documentations. The advances in quantitative analysis methods drove a lot of data analytics in software engineering. This has overshadowed to some degree the importance of texts and their qualitative analysis. Such analysis has, however, merits for researchers and practitioners as well.
In this chapter, we describe the basics of analysing text in software projects. We first describe how to manually analyse and code textual data. Next, we give an overview of mixed methods to automatic text analysis including N-Grams and clone detection as well as more sophisticated natural language processing identifying syntax and contexts of words. Those methods and tools are of critical importance to aid in the challenges in today's huge amounts of textual data.
We illustrate the introduced methods via a running example and conclude by presenting two industrial studies.
△ Less
Submitted 1 December, 2016;
originally announced December 2016.
-
Preventing Incomplete/Hidden Requirements: Reflections on Survey Data from Austria and Brazil
Authors:
M. Kalinowski,
M. Felderer,
T. Conte,
R. Spínola,
R. Prikladnicki,
D. Winkler,
D. Méndez Fernández,
S. Wagner
Abstract:
Many software projects fail due to problems in requirements engineering (RE). The goal of this paper is analyzing a specific and relevant RE problem in detail: incomplete/hidden requirements. We replicated a global family of RE surveys with representatives of software organizations in Austria and Brazil. We used the data to (a) characterize the criticality of the selected RE problem, and to (b) an…
▽ More
Many software projects fail due to problems in requirements engineering (RE). The goal of this paper is analyzing a specific and relevant RE problem in detail: incomplete/hidden requirements. We replicated a global family of RE surveys with representatives of software organizations in Austria and Brazil. We used the data to (a) characterize the criticality of the selected RE problem, and to (b) analyze the reported main causes and mitigation actions. Based on the analysis, we discuss how to prevent the problem. The survey includes 14 different organizations in Austria and 74 in Brazil, including small, medium and large sized companies, conducting both, plan-driven and agile development processes. Respondents from both countries cited the incomplete/hidden requirements problem as one of the most critical RE problems. We identified and graphically represented the main causes and documented solution options to address these causes. Further, we compiled a list of reported mitigation actions. From a practical point of view, this paper provides further insights into common causes of incomplete/hidden requirements and on how to prevent this problem.
△ Less
Submitted 1 December, 2016;
originally announced December 2016.
-
Naming the Pain in Requirements Engineering: Contemporary Problems, Causes, and Effects in Practice
Authors:
D. Méndez Fernández,
S. Wagner,
M. Kalinowski,
M. Felderer,
P. Mafra,
A. Vetrò,
T. Conte,
M. -T. Christiansson,
D. Greer,
C. Lassenius,
T. Männistö,
M. Nayabi,
M. Oivo,
B. Penzenstadler,
D. Pfahl,
R. Prikladnicki,
G. Ruhe,
A. Schekelmann,
S. Sen,
R. Spinola,
A. Tuzcu,
J. L. de la Vara,
R. Wieringa
Abstract:
Requirements Engineering (RE) has received much attention in research and practice due to its importance to software project success. Its interdisciplinary nature, the dependency to the customer, and its inherent uncertainty still render the discipline difficult to investigate. This results in a lack of empirical data. These are necessary, however, to demonstrate which practically relevant RE prob…
▽ More
Requirements Engineering (RE) has received much attention in research and practice due to its importance to software project success. Its interdisciplinary nature, the dependency to the customer, and its inherent uncertainty still render the discipline difficult to investigate. This results in a lack of empirical data. These are necessary, however, to demonstrate which practically relevant RE problems exist and to what extent they matter. Motivated by this situation, we initiated the Naming the Pain in Requirements Engineering (NaPiRE) initiative which constitutes a globally distributed, bi-yearly replicated family of surveys on the status quo and problems in practical RE. In this article, we report on the qualitative analysis of data obtained from 228 companies working in 10 countries in various domains and we reveal which contemporary problems practitioners encounter. To this end, we analyse 21 problems derived from the literature with respect to their relevance and criticality in dependency to their context, and we complement this picture with a cause-effect analysis showing the causes and effects surrounding the most critical problems. Our results give us a better understanding of which problems exist and how they manifest themselves in practical environments. Thus, we provide a first step to ground contributions to RE on empirical observations which, until now, were dominated by conventional wisdom only.
△ Less
Submitted 27 November, 2016;
originally announced November 2016.
-
Artefact-based Requirements Engineering: The AMDiRE Approach
Authors:
D. Méndez Fernández,
B. Penzenstadler
Abstract:
The various influences in the processes and application domains make Requirements Engineering (RE) inherently complex and difficult to implement. In general, we have two options for establishing an RE approach: we can either establish an activity-based RE approach or we can establish an artefact-based one where project participants concentrate on the RE artefacts rather than on the way of creating…
▽ More
The various influences in the processes and application domains make Requirements Engineering (RE) inherently complex and difficult to implement. In general, we have two options for establishing an RE approach: we can either establish an activity-based RE approach or we can establish an artefact-based one where project participants concentrate on the RE artefacts rather than on the way of creating them. While a number of activity-based RE approaches have been proposed in recent years, we have gained much empirical evidence and experiences about the advantages of the artefact-based paradigm for RE. However, artefact orientation is still a young paradigm with various interpretations and practical manifestations whereby we need a clear understanding of its basic concepts and a consolidated and evaluated view on the paradigm.
In this article, we contribute an artefact-based approach to RE (AMDiRE) that emerges from six years of experiences in fundamental and evidence-based research. To this end, we first discuss the basic notion of artefact orientation and its evolution in recent years. We briefly introduce a set of artefact-based RE models we developed in industrial research cooperations for different application domains, show their empirical evaluations, and their dissemination into academia and practice, eventually leading to the AMDiRE approach. We conclude with a discussion of experiences we made during the development and different industrial evaluations, and lessons learnt.
△ Less
Submitted 30 November, 2016;
originally announced November 2016.
-
A Field Study on the Elicitation and Classification of Defects for Defect Models
Authors:
D. Holling,
D. Méndez Fernández,
A. Pretschner
Abstract:
Defect models capture faults and methods to provoke failures. To integrate such defect models into existing quality assurance processes, we developed a defect model lifecycle framework, in which the elicitation and classification of context-specific defects forms a crucial step. Although we could gather first insights from its practical application, we still have little knowledge about its benefit…
▽ More
Defect models capture faults and methods to provoke failures. To integrate such defect models into existing quality assurance processes, we developed a defect model lifecycle framework, in which the elicitation and classification of context-specific defects forms a crucial step. Although we could gather first insights from its practical application, we still have little knowledge about its benefits and limitations. We aim at qualitatively analyzing the context-specific elicitation and classification of defects to explore the suitability of our approach for practical application. We apply case study research in multiple contexts and analyze (1) what kind of defects we can elicit and the degree to which the defects matter to a context only, (2) the extent to which it leads to results useful enough for describing and operationalizing defect models, and (3) if there is a perceived additional immediate benefit from a practitioner's perspective. Our results strengthen our confidence on the suitability of our approach to elicit defects that are context-specific as well as context-independent. We conclude so far that our approach is suitable to provide a blueprint on how to elicit and classify defects for specific contexts to be used for the improvement of quality assurance techniques.
△ Less
Submitted 30 November, 2016;
originally announced November 2016.
-
Field study on requirements engineering: Investigation of artefacts, project parameters, and execution strategies
Authors:
D. Méndez Fernández,
S. Wagner,
K. Lochmann,
A. Baumann,
H. de Carne
Abstract:
Requirements Engineering (RE) is a critical discipline mostly driven by uncertainty, since it is influenced by the customer domain or by the development process model used. We aim to investigate RE processes in successful project environments to discover characteristics and strategies that allow us to elaborate RE tailoring approaches in the future. We perform a field study on a set of projects at…
▽ More
Requirements Engineering (RE) is a critical discipline mostly driven by uncertainty, since it is influenced by the customer domain or by the development process model used. We aim to investigate RE processes in successful project environments to discover characteristics and strategies that allow us to elaborate RE tailoring approaches in the future. We perform a field study on a set of projects at one company. First, we investigate by content analysis which RE artefacts were produced in each project and to what extent they were produced. Second, we perform qualitative analysis of semi-structured interviews to discover project parameters that relate to the produced artefacts. Third, we use cluster analysis to infer artefact patterns and probable RE execution strategies, which are the responses to specific project parameters. Fourth, we investigate by statistical tests the effort spent in each strategy in relation to the effort spent in change requests to evaluate the efficiency of execution strategies. Our results show no statistically significant difference between the efficiency of the strategies. In addition, it turned out that many parameters considered as the main causes for project failures can be successfully handled. Hence, practitioners can apply the artefact patterns and related project parameters to tailor the RE process according to individual project characteristics.
△ Less
Submitted 30 November, 2016;
originally announced November 2016.
-
Naming the Pain in Requirements Engineering: A Design for a Global Family of Surveys and First Results from Germany
Authors:
D. Méndez Fernández,
S. Wagner
Abstract:
For many years, we have observed industry struggling in defining a high quality requirements engineering (RE) and researchers trying to understand industrial expectations and problems. Although we are investigating the discipline with a plethora of empirical studies, they still do not allow for empirical generalisations. To lay an empirical and externally valid foundation about the state of the pr…
▽ More
For many years, we have observed industry struggling in defining a high quality requirements engineering (RE) and researchers trying to understand industrial expectations and problems. Although we are investigating the discipline with a plethora of empirical studies, they still do not allow for empirical generalisations. To lay an empirical and externally valid foundation about the state of the practice in RE, we aim at a series of open and reproducible surveys that allow us to steer future research in a problem-driven manner. We designed a globally distributed family of surveys in joint collaborations with different researchers and completed the first run in Germany. The instrument is based on a theory in the form of a set of hypotheses inferred from our experiences and available studies. We test each hypothesis in our theory and identify further candidates to extend the theory by correlation and Grounded Theory analysis. In this article, we report on the design of the family of surveys, its underlying theory, and the full results obtained from Germany with participants from 58 companies. The results reveal, for example, a tendency to improve RE via internally defined qualitative methods rather than relying on normative approaches like CMMI. We also discovered various RE problems that are statistically significant in practice. For instance, we could corroborate communication flaws or moving targets as problems in practice. Our results are not yet fully representative but already give first insights into current practices and problems in RE, and they allow us to draw lessons learnt for future replications. Our results obtained from this first run in Germany make us confident that the survey design and instrument are well-suited to be replicated and, thereby, to create a generalisable empirical basis of RE in practice.
△ Less
Submitted 29 November, 2016;
originally announced November 2016.
-
Naming the Pain in Requirements Engineering: Comparing Practices in Brazil and Germany
Authors:
Daniel Méndez Fernández,
Stefan Wagner,
Marcos Kalinowski,
André Schekelmann,
Ahmet Tuzcu,
Tayana Conte,
Rodrigo Spinola,
Rafael Prikladnicki
Abstract:
As part of the Naming the Pain in Requirements Engineering (NaPiRE) initiative, researchers compared problems that companies in Brazil and Germany encountered during requirements engineering (RE). The key takeaway was that in RE, human interaction is necessary for eliciting and specifying high-quality requirements, regardless of country, project type, or company size.
As part of the Naming the Pain in Requirements Engineering (NaPiRE) initiative, researchers compared problems that companies in Brazil and Germany encountered during requirements engineering (RE). The key takeaway was that in RE, human interaction is necessary for eliciting and specifying high-quality requirements, regardless of country, project type, or company size.
△ Less
Submitted 28 November, 2016;
originally announced November 2016.
-
Are "Non-functional" Requirements really Non-functional?
Authors:
J. Eckhardt,
A. Vogelsang,
D. Méndez Fernández
Abstract:
Non-functional requirements (NFRs) are commonly distinguished from functional requirements by differentiating how the system shall do something in contrast to what the system shall do. This distinction is not only prevalent in research, but also influences how requirements are handled in practice. NFRs are usually documented separately from functional requirements, without quantitative measures, a…
▽ More
Non-functional requirements (NFRs) are commonly distinguished from functional requirements by differentiating how the system shall do something in contrast to what the system shall do. This distinction is not only prevalent in research, but also influences how requirements are handled in practice. NFRs are usually documented separately from functional requirements, without quantitative measures, and with relatively vague descriptions. As a result, they remain difficult to analyze and test. Several authors argue, however, that many so-called NFRs actually describe behavioral properties and may be treated the same way as functional requirements. In this paper, we empirically investigate this point of view and aim to increase our understanding on the nature of NFRs addressing system properties. We report on the classification of 530 NFRs extracted from 11 industrial requirements specifications and analyze to which extent these NFRs describe system behavior. Our results suggest that most "non-functional" requirements are not non-functional as they describe behavior of a system. Consequently, we argue that many so-called NFRs can be handled similarly to functional requirements.
△ Less
Submitted 27 November, 2016;
originally announced November 2016.
-
Rapid quality assurance with Requirements Smells
Authors:
H. Femmer,
D. Méndez Fernández,
S. Wagner,
S. Eder
Abstract:
Bad requirements quality can cause expensive consequences during the software development lifecycle, especially if iterations are long and feedback comes late. %-- the faster a problem is found, the cheaper it is to fix. This makes explicit the need of a lightweight detection mechanism of requirements quality violations. We aim at a light-weight static requirements analysis approach that allows fo…
▽ More
Bad requirements quality can cause expensive consequences during the software development lifecycle, especially if iterations are long and feedback comes late. %-- the faster a problem is found, the cheaper it is to fix. This makes explicit the need of a lightweight detection mechanism of requirements quality violations. We aim at a light-weight static requirements analysis approach that allows for rapid checks immediately when requirements are written down. We transfer the concept of code smells to Requirements Engineering as Requirements Smells. To evaluate the benefits and limitations, we define Requirements Smells, realize our concepts for a smell detection in a prototype called Smella and apply Smella in a series of cases provided by three industrial and a university context. The automatic detection yields an average precision of 59% at an average recall of 82% with high variation. The evaluation in practical environments indicates benefits such as an increase of the awareness of quality defects. Yet, some smells were not clearly distinguishable. Lightweight smell detection can uncover many practically relevant requirements defects in a reasonably precise way. Although some smells need to be defined more clearly, smell detection provides a helpful means to support quality assurance in Requirements Engineering, for instance, as a supplement to reviews.
△ Less
Submitted 27 November, 2016;
originally announced November 2016.
-
Case Studies in Industry: What We Have Learnt
Authors:
D. Méndez Fernández,
S. Wagner
Abstract:
Case study research has become an important research methodology for exploring phenomena in their natural contexts. Case studies have earned a distinct role in the empirical analysis of software engineering phenomena which are difficult to capture in isolation. Such phenomena often appear in the context of methods and development processes for which it is difficult to run large, controlled experim…
▽ More
Case study research has become an important research methodology for exploring phenomena in their natural contexts. Case studies have earned a distinct role in the empirical analysis of software engineering phenomena which are difficult to capture in isolation. Such phenomena often appear in the context of methods and development processes for which it is difficult to run large, controlled experiments as they usually have to reduce the scale in several respects and, hence, are detached from the reality of industrial software development. The other side of the medal is that the realistic socio-economic environments where we conduct case studies -- with real-life cases and realistic conditions -- also pose a plethora of practical challenges to planning and conducting case studies. In this experience report, we discuss such practical challenges and the lessons we learnt in conducting case studies in industry. Our goal is to help especially inexperienced researchers facing their first case studies in industry by increasing their awareness for typical obstacles they might face and practical ways to deal with those obstacles.
△ Less
Submitted 27 November, 2016;
originally announced November 2016.
-
Towards Guidelines for Preventing Critical Requirements Engineering Problems
Authors:
P. Mafra,
M. Kalinowski,
D. Méndez Fernández,
M. Felderer,
S. Wagner
Abstract:
Context] Problems in Requirements Engineering (RE) can lead to serious consequences during the software development lifecycle. [Goal] The goal of this paper is to propose empirically-based guidelines that can be used by different types of organisations according to their size (small, medium or large) and process model (agile or plan-driven) to help them in preventing such problems. [Method] We ana…
▽ More
Context] Problems in Requirements Engineering (RE) can lead to serious consequences during the software development lifecycle. [Goal] The goal of this paper is to propose empirically-based guidelines that can be used by different types of organisations according to their size (small, medium or large) and process model (agile or plan-driven) to help them in preventing such problems. [Method] We analysed data from a survey on RE problems answered by 228 organisations in 10 different countries. [Results] We identified the most critical RE problems, their causes and mitigation actions, organizing this information by clusters of size and process model. Finally, we analysed the causes and mitigation actions of the critical problems of each cluster to get further insights into how to prevent them. [Conclusions] Based on our results, we suggest preliminary guidelines for preventing critical RE problems in response to context characteristics of the companies.
△ Less
Submitted 27 November, 2016;
originally announced November 2016.
-
On the Distinction of Functional and Quality Requirements in Practice
Authors:
J. Eckhardt,
A. Vogelsang,
D. Méndez Fernández
Abstract:
Requirements are often divided into functional requirements (FRs) and quality requirements (QRs). However, we still have little knowledge about to which extent this distinction makes sense from a practical perspective. In this paper, we report on a survey we conducted with 103 practitioners to explore whether and, if so, why they handle requirements labeled as FRs differently from those labeled as…
▽ More
Requirements are often divided into functional requirements (FRs) and quality requirements (QRs). However, we still have little knowledge about to which extent this distinction makes sense from a practical perspective. In this paper, we report on a survey we conducted with 103 practitioners to explore whether and, if so, why they handle requirements labeled as FRs differently from those labeled as QRs. We additionally asked for consequences of this distinction w.r.t. the development process. Our results indicate that the development process for requirements of the two classes strongly differs (e.g., in testing). We identified a number of reasons why practitioners do (or do not) distinguish between QRs and FRs in their documentation and we analyzed both problems and benefits that arise from that. We found, for instance, that many reasons are based on expectations rather than on evidence. Those expectations are, in fact, not reflected in specific negative or positive consequences per se. It therefore seems more important that the decision whether to make an explicit distinction or not should be made consciously such that people are also aware of the risks that this distinction bears so that they may take appropriate countermeasures.
△ Less
Submitted 27 November, 2016;
originally announced November 2016.
-
Naming the Pain in Requirements Engineering: Design of a Global Family of Surveys and First Results from Germany
Authors:
Daniel Méndez Fernández,
Stefan Wagner
Abstract:
Context: For many years, we have observed industry struggling in defining a high quality requirements engineering (RE) and researchers trying to understand industrial expectations and problems. Although we are investigating the discipline with a plethora of empirical studies, those studies either concentrate on validating specific methods or on single companies or countries. Therefore, they allow…
▽ More
Context: For many years, we have observed industry struggling in defining a high quality requirements engineering (RE) and researchers trying to understand industrial expectations and problems. Although we are investigating the discipline with a plethora of empirical studies, those studies either concentrate on validating specific methods or on single companies or countries. Therefore, they allow only for limited empirical generalisations. Objective: To lay an empirical and generalisable foundation about the state of the practice in RE, we aim at a series of open and reproducible surveys that allow us to steer future research in a problem-driven manner. Method: We designed a globally distributed family of surveys in joint collaborations with different researchers from different countries. The instrument is based on an initial theory inferred from available studies. As a long-term goal, the survey will be regularly replicated to manifest a clear understanding on the status quo and practical needs in RE. In this paper, we present the design of the family of surveys and first results of its start in Germany. Results: Our first results contain responses from 30 German companies. The results are not yet generalisable, but already indicate several trends and problems. For instance, a commonly stated problem respondents see in their company standards are artefacts being underrepresented, and important problems they experience in their projects are incomplete and inconsistent requirements. Conclusion: The results suggest that the survey design and instrument are well-suited to be replicated and, thereby, to create a generalisable empirical basis of RE in practice.
△ Less
Submitted 15 November, 2016;
originally announced November 2016.
-
Teaching Software Process Modeling
Authors:
Marco Kuhrmann,
Daniel Méndez Fernández,
Jürgen Münch
Abstract:
Most university curricula consider software processes to be on the fringes of software engineering (SE). Students are told there exists a plethora of software processes ranging from RUP over V-shaped processes to agile methods. Furthermore, the usual students' programming tasks are of a size that either one student or a small group of students can manage the work. Comprehensive processes being ess…
▽ More
Most university curricula consider software processes to be on the fringes of software engineering (SE). Students are told there exists a plethora of software processes ranging from RUP over V-shaped processes to agile methods. Furthermore, the usual students' programming tasks are of a size that either one student or a small group of students can manage the work. Comprehensive processes being essential for large companies in terms of reflecting the organization structure, coordinating teams, or interfaces to business processes such as contracting or sales, are complex and hard to teach in a lecture, and, therefore, often out of scope. We experienced tutorials on using Java or C#, or on develo** applications for the iPhone to gather more attention by students, simply speaking, as these are more fun for them. So, why should students spend their time in software processes? From our experiences and the discussions with a variety of industrial partners, we learned that students often face trouble when taking their first "real" jobs, even if the company is organized in a lean or agile shape. Therefore, we propose to include software processes more explicitly into the SE curricula. We designed and implemented a course at Master's level in which students learn why software processes are necessary, and how they can be analyzed, designed, implemented, and continuously improved. In this paper, we present our course's structure, its goals, and corresponding teaching methods. We evaluate the course and further discuss our experiences so that lecturers and researchers can directly use our lessons learned in their own curricula.
△ Less
Submitted 18 December, 2013;
originally announced December 2013.