-
Teaching Scrum with a focus on compliance assessment
Authors:
Marco Torchiano,
Antonio Vetrò,
Riccardo Coppola
Abstract:
The Scrum framework has gained widespread adoption in the industry for its emphasis on collaboration and continuous improvement. However, it has not reached a similar relevance in Software Engineering (SE) curricula. This work reports the experience of five editions of a SE course within an MSc. Degree in Computer Engineering. The course primary educational objective is to provide students with th…
▽ More
The Scrum framework has gained widespread adoption in the industry for its emphasis on collaboration and continuous improvement. However, it has not reached a similar relevance in Software Engineering (SE) curricula. This work reports the experience of five editions of a SE course within an MSc. Degree in Computer Engineering. The course primary educational objective is to provide students with the skills to manage software development projects with Scrum. The course is based on the execution of a team project and on the definition of qualitative and quantitative means of assessment of the application of Scrum. The conduction of five editions of the course allowed us to identify several lessons learned about time budgeting and team compositions in agile student projects and its evidence of the applicability of the framework to software development courses.
△ Less
Submitted 23 April, 2024; v1 submitted 22 April, 2024;
originally announced April 2024.
-
IntelliGame in Action: An Experience Report on Gamifying JavaScript Unit Tests
Authors:
Philipp Straubinger,
Tommaso Fulcini,
Gordon Fraser,
Marco Torchiano
Abstract:
This paper investigates the integration and assessment of IntelliGame, a gamification plugin initially designed for Java development, within the realm of JavaScript unit testing. We aim to verify the generalizability of IntelliGame to JavaScript development and to provide valuable insights into the experiment's design. For this, we first customize IntelliGame for JavaScript, and then conduct a con…
▽ More
This paper investigates the integration and assessment of IntelliGame, a gamification plugin initially designed for Java development, within the realm of JavaScript unit testing. We aim to verify the generalizability of IntelliGame to JavaScript development and to provide valuable insights into the experiment's design. For this, we first customize IntelliGame for JavaScript, and then conduct a controlled experiment involving 152 participants utilizing the Jest testing framework, and finally examine its influence on testing behavior and the overall developer experience. The findings from this study provide valuable insights for improving JavaScript testing methodologies through the incorporation of gamification.
△ Less
Submitted 6 March, 2024;
originally announced March 2024.
-
Empirical Standards for Software Engineering Research
Authors:
Paul Ralph,
Nauman bin Ali,
Sebastian Baltes,
Domenico Bianculli,
Jessica Diaz,
Yvonne Dittrich,
Neil Ernst,
Michael Felderer,
Robert Feldt,
Antonio Filieri,
Breno Bernard Nicolau de França,
Carlo Alberto Furia,
Greg Gay,
Nicolas Gold,
Daniel Graziotin,
Pinjia He,
Rashina Hoda,
Natalia Juristo,
Barbara Kitchenham,
Valentina Lenarduzzi,
Jorge Martínez,
Jorge Melegati,
Daniel Mendez,
Tim Menzies,
Jefferson Molleri
, et al. (18 additional authors not shown)
Abstract:
Empirical Standards are natural-language models of a scientific community's expectations for a specific kind of study (e.g. a questionnaire survey). The ACM SIGSOFT Paper and Peer Review Quality Initiative generated empirical standards for research methods commonly used in software engineering. These living documents, which should be continuously revised to reflect evolving consensus around resear…
▽ More
Empirical Standards are natural-language models of a scientific community's expectations for a specific kind of study (e.g. a questionnaire survey). The ACM SIGSOFT Paper and Peer Review Quality Initiative generated empirical standards for research methods commonly used in software engineering. These living documents, which should be continuously revised to reflect evolving consensus around research best practices, will improve research quality and make peer review more effective, reliable, transparent and fair.
△ Less
Submitted 4 March, 2021; v1 submitted 7 October, 2020;
originally announced October 2020.
-
Understanding Peer Review of Software Engineering Papers
Authors:
Neil A. Ernst,
Jeffrey C. Carver,
Daniel Mendez,
Marco Torchiano
Abstract:
Peer review is a key activity intended to preserve the quality and integrity of scientific publications. However, in practice it is far from perfect.
We aim at understanding how reviewers, including those who have won awards for reviewing, perform their reviews of software engineering papers to identify both what makes a good reviewing approach and what makes a good paper.
We first conducted a…
▽ More
Peer review is a key activity intended to preserve the quality and integrity of scientific publications. However, in practice it is far from perfect.
We aim at understanding how reviewers, including those who have won awards for reviewing, perform their reviews of software engineering papers to identify both what makes a good reviewing approach and what makes a good paper.
We first conducted a series of in-person interviews with well-respected reviewers in the software engineering field. Then, we used the results of those interviews to develop a questionnaire used in an online survey and sent out to reviewers from well-respected venues covering a number of software engineering disciplines, some of whom had won awards for their reviewing efforts.
We analyzed the responses from the interviews and from 175 reviewers who completed the online survey (including both reviewers who had won awards and those who had not). We report on several descriptive results, including: 45% of award-winners are reviewing 20+ conference papers a year, while 28% of non-award winners conduct that many. 88% of reviewers are taking more than two hours on journal reviews. We also report on qualitative results. To write a good review, the important criteria were it should be factual and helpful, ranked above others such as being detailed or kind. The most important features of papers that result in positive reviews are clear and supported validation, an interesting problem, and novelty. Conversely, negative reviews tend to result from papers that have a mismatch between the method and the claims and from those with overly grandiose claims.
The main recommendation for authors is to make the contribution of the work very clear in their paper. In addition, reviewers viewed data availability and its consistency as being important.
△ Less
Submitted 17 June, 2021; v1 submitted 2 September, 2020;
originally announced September 2020.
-
Characterizing the transition to Kotlin of Android apps: a study on F-Droid, Play Store and GitHub
Authors:
Riccardo Coppola,
Luca Ardito,
Marco Torchiano
Abstract:
Kotlin is a novel language that represents an alternative to Java, and has been recently adopted as a first-class programming language for Android applications. Kotlin is achieving a significant diffusion among developers, and several studies have highlighted various advantages of the language when compared to Java.
The objective of this paper is to analyze a set of open-source Android apps, to…
▽ More
Kotlin is a novel language that represents an alternative to Java, and has been recently adopted as a first-class programming language for Android applications. Kotlin is achieving a significant diffusion among developers, and several studies have highlighted various advantages of the language when compared to Java.
The objective of this paper is to analyze a set of open-source Android apps, to evaluate their transition to the Kotlin programming language throughout their lifespan and understand whether the adoption of Kotlin has impacts on the success of Android apps.
We mined all the projects from the F-Droid repository of Android open-source applications, and we found the corresponding projects on the official Google Play Store and on the GitHub platform. We defined a set of eight metrics to quantify the relevance of Kotlin code in the latest update and through all releases of an application. Then, we statistically analyzed the correlation between the presence of Kotlin code in a project and popularity metrics mined from the platforms where the apps were released.
Of a set of 1232 projects that were updated after October 2017, near 20% adopted Kotlin and about 12% had more Kotlin code than Java; most of the projects that adopted Kotlin quickly transitioned from Java to the new language. The projects featuring Kotlin had on average higher popularity metrics; a statistically significant correlation has been found between the presence of Kotlin and the number of stars on the GitHub repository.
The Kotlin language seems able to guarantee a seamless migration from Java for Android developers. With an inspection on a large set of open-source Android apps, we observed that the adoption of the Kotlin language is rapid (when compared to the average lifespan of an Android project) and seems to come at no cost in terms of popularity among the users and other developers.
△ Less
Submitted 18 August, 2019;
originally announced August 2019.
-
Fragility of Layout-Based and Visual GUI Test Scripts: An Assessment Study on a Hybrid Mobile Application
Authors:
Riccardo Coppola,
Luca Ardito,
Marco Torchiano
Abstract:
Context: Albeit different approaches exist for automated GUI testing of hybrid mobile applications, the practice appears to be not so commonly adopted by developers. A possible reason for such a low diffusion can be the fragility of the techniques, i.e. the frequent need for maintaining test cases when the GUI of the app is changed.
Goal: In this paper, we perform an assessment of the maintenanc…
▽ More
Context: Albeit different approaches exist for automated GUI testing of hybrid mobile applications, the practice appears to be not so commonly adopted by developers. A possible reason for such a low diffusion can be the fragility of the techniques, i.e. the frequent need for maintaining test cases when the GUI of the app is changed.
Goal: In this paper, we perform an assessment of the maintenance needed by test cases for a hybrid mobile app, and the related fragility causes.
Methods: We evaluated a small test suite with a Layout-based testing tool (Appium) and a Visual one (EyeAutomate) and observed the changes needed by tests during the co-evolution with the GUI of the app.
Results: We found that 20% Layout-based test methods and 30% Visual test methods had to be modified at least once, and that each release induced fragilities in 3-4% of the test methods.
Conclusion: Fragility of GUI tests can induce relevant maintenance efforts in test suites of large applications. Several principal causes for fragilities have been identified for the tested hybrid application, and guidelines for developers are deduced from them.
△ Less
Submitted 27 July, 2019; v1 submitted 18 July, 2019;
originally announced July 2019.
-
A Software Ecosystem Reshaped by a Paradigm Shift: the CSI-Piemonte Case
Authors:
Federico Tomassetti,
Marco Torchiano,
Mauro Antonaci,
Paolo Arvati,
Maurizio Morisio
Abstract:
Context: Changes in the software development paradigm, when operated by entities with a pivotal role, have the power to affect a number of groups and entities in their sphere of influence, changing both their working habits and relations.
Objective: In this paper we present the organizational changes occurred in a software ecosystem as consequence of a technological change. In particular we exam…
▽ More
Context: Changes in the software development paradigm, when operated by entities with a pivotal role, have the power to affect a number of groups and entities in their sphere of influence, changing both their working habits and relations.
Objective: In this paper we present the organizational changes occurred in a software ecosystem as consequence of a technological change. In particular we examine the evolution of an MDD solution and the changing roles of the company promoting it, the public administrations and the sub-contractors.
Method: The paper focuses on a single case study that encompasses the six years long evolution of a Model-driven development solution, starting from its conception until is recent open-source release, across five distinct phases. The history was analyzed jointly by software engineering academics and industrial managers directly involved in the case study.
Results: A report of the ecosystem evolution from an idiographic perspective is reported. An analysis of the history allowed an abstraction that led to the identification of several distinct ecosystem evolution motifs.
Conclusion: The motifs represent a set of key process areas for the evolution of a software ecosystem. They are potentially generalizable to other similar ecosystems. As such, they can be used by researchers to evaluate existing in-progress case studies, and by practitioners as a set of guidelines.
△ Less
Submitted 18 December, 2018;
originally announced December 2018.
-
Completeness and Consistency Analysis for Evolving Knowledge Bases
Authors:
Mohammad Rifat Ahmmad Rashid,
Giuseppe Rizzo,
Marco Torchiano,
Nandana Mihindukulasooriya,
Oscar Corcho,
Raúl García-Castro
Abstract:
Assessing the quality of an evolving knowledge base is a challenging task as it often requires to identify correct quality assessment procedures.
Since data is often derived from autonomous, and increasingly large data sources, it is impractical to manually curate the data, and challenging to continuously and automatically assess their quality.
In this paper, we explore two main areas of quali…
▽ More
Assessing the quality of an evolving knowledge base is a challenging task as it often requires to identify correct quality assessment procedures.
Since data is often derived from autonomous, and increasingly large data sources, it is impractical to manually curate the data, and challenging to continuously and automatically assess their quality.
In this paper, we explore two main areas of quality assessment related to evolving knowledge bases: (i) identification of completeness issues using knowledge base evolution analysis, and (ii) identification of consistency issues based on integrity constraints, such as minimum and maximum cardinality, and range constraints.
For completeness analysis, we use data profiling information from consecutive knowledge base releases to estimate completeness measures that allow predicting quality issues. Then, we perform consistency checks to validate the results of the completeness analysis using integrity constraints and learning models.
The approach has been tested both quantitatively and qualitatively by using a subset of datasets from both DBpedia and 3cixty knowledge bases. The performance of the approach is evaluated using precision, recall, and F1 score. From completeness analysis, we observe a 94% precision for the English DBpedia KB and 95% precision for the 3cixty Nice KB. We also assessed the performance of our consistency analysis by using five learning models over three sub-tasks, namely minimum cardinality, maximum cardinality, and range constraint. We observed that the best performing model in our experimental setup is the Random Forest, reaching an F1 score greater than 90% for minimum and maximum cardinality and 84% for range constraints.
△ Less
Submitted 30 November, 2018;
originally announced November 2018.
-
Integrating Software Engineering Key Practices into an OOP Massive In-Classroom Course: an Experience Report
Authors:
Marco Torchiano,
Giorgio Bruno
Abstract:
Programming and software engineering courses in computer science curricula typically focus on both providing theoretical knowledge of programming languages and best-practices, and develo** practical development skills. In a massive course - several hundred students - the teachers are not able to adequately attend to the practical part, therefore process automation and incentives to students must…
▽ More
Programming and software engineering courses in computer science curricula typically focus on both providing theoretical knowledge of programming languages and best-practices, and develo** practical development skills. In a massive course - several hundred students - the teachers are not able to adequately attend to the practical part, therefore process automation and incentives to students must be used to drive the students in the right direction. Our goals was to design an automated programming assignment infrastructure capable of supporting massive courses. The infrastructure should encourage students to apply the key software engineering (SE) practices - automated testing, con guration management, and Integrated Development Environment (IDE) - and acquire the basic skills for using the corresponding tools. We selected a few widely adopted development tools used to support the key software engineering practices and mapped them to the basic activities in our exam assignment management process. This experience report describes the results from the past academic year. The infrastructure we built has been used for a full academic year and supported four exam sessions for a total of over a thousand students. The satisfaction level reported by the students is generally high.
△ Less
Submitted 5 April, 2018;
originally announced April 2018.
-
Scripted GUI Testing of Android Apps: A Study on Diffusion, Evolution and Fragility
Authors:
Riccardo Coppola,
Maurizio Morisio,
Marco Torchiano
Abstract:
Background. Evidence suggests that mobile applications are not thoroughly tested as their desktop counterparts. In particular GUI testing is generally limited. Like web-based applications, mobile apps suffer from GUI test fragility, i.e. GUI test classes failing due to minor modifications in the GUI, without the application functionalities being altered.
Aims. The objective of our study is to ex…
▽ More
Background. Evidence suggests that mobile applications are not thoroughly tested as their desktop counterparts. In particular GUI testing is generally limited. Like web-based applications, mobile apps suffer from GUI test fragility, i.e. GUI test classes failing due to minor modifications in the GUI, without the application functionalities being altered.
Aims. The objective of our study is to examine the diffusion of GUI testing on Android, and the amount of changes required to keep test classes up to date, and in particular the changes due to GUI test fragility. We define metrics to characterize the modifications and evolution of test classes and test methods, and proxies to estimate fragility-induced changes.
Method. To perform our experiments, we selected six widely used open-source tools for scripted GUI testing of mobile applications previously described in the literature. We have mined the repositories on GitHub that used those tools, and computed our set of metrics.
Results. We found that none of the considered GUI testing frameworks achieved a major diffusion among the open-source Android projects available on GitHub. For projects with GUI tests, we found that test suites have to be modified often, specifically 5\%-10\% of developers' modified LOCs belong to tests, and that a relevant portion (60\% on average) of such modifications are induced by fragility.
Conclusions. Fragility of GUI test classes constitute a relevant concern, possibly being an obstacle for developers to adopt automated scripted GUI tests. This first evaluation and measure of fragility of Android scripted GUI testing can constitute a benchmark for developers, and the basis for the definition of a taxonomy of fragility causes, and actionable guidelines to mitigate the issue.
△ Less
Submitted 9 November, 2017;
originally announced November 2017.
-
How Professional Hackers Understand Protected Code while Performing Attack Tasks
Authors:
Mariano Ceccato,
Paolo Tonella,
Cataldo Basile,
Bart Coppens,
Bjorn De Sutter,
Paolo Falcarin,
Marco Torchiano
Abstract:
Code protections aim at blocking (or at least delaying) reverse engineering and tampering attacks to critical assets within programs. Knowing the way hackers understand protected code and perform attacks is important to achieve a stronger protection of the software assets, based on realistic assumptions about the hackers' behaviour. However, building such knowledge is difficult because hackers can…
▽ More
Code protections aim at blocking (or at least delaying) reverse engineering and tampering attacks to critical assets within programs. Knowing the way hackers understand protected code and perform attacks is important to achieve a stronger protection of the software assets, based on realistic assumptions about the hackers' behaviour. However, building such knowledge is difficult because hackers can hardly be involved in controlled experiments and empirical studies. The FP7 European project Aspire has given the authors of this paper the unique opportunity to have access to the professional penetration testers employed by the three industrial partners. In particular, we have been able to perform a qualitative analysis of three reports of professional penetration test performed on protected industrial code. Our qualitative analysis of the reports consists of open coding, carried out by 7 annotators and resulting in 459 annotations, followed by concept extraction and model inference. We identified the main activities: understanding, building attack, choosing and customizing tools, and working around or defeating protections. We built a model of how such activities take place. We used such models to identify a set of research directions for the creation of stronger code protections.
△ Less
Submitted 26 May, 2017; v1 submitted 10 April, 2017;
originally announced April 2017.
-
Assessment of Source Code Obfuscation Techniques
Authors:
Alessio Viticchié,
Leonardo Regano,
Marco Torchiano,
Cataldo Basile,
Mariano Ceccato,
Paolo Tonella,
Roberto Tiella
Abstract:
Obfuscation techniques are a general category of software protections widely adopted to prevent malicious tampering of the code by making applications more difficult to understand and thus harder to modify. Obfuscation techniques are divided in code and data obfuscation, depending on the protected asset. While preliminary empirical studies have been conducted to determine the impact of code obfusc…
▽ More
Obfuscation techniques are a general category of software protections widely adopted to prevent malicious tampering of the code by making applications more difficult to understand and thus harder to modify. Obfuscation techniques are divided in code and data obfuscation, depending on the protected asset. While preliminary empirical studies have been conducted to determine the impact of code obfuscation, our work aims at assessing the effectiveness and efficiency in preventing attacks of a specific data obfuscation technique - VarMerge. We conducted an experiment with student participants performing two attack tasks on clear and obfuscated versions of two applications written in C. The experiment showed a significant effect of data obfuscation on both the time required to complete and the successful attack efficiency. An application with VarMerge reduces by six times the number of successful attacks per unit of time. This outcome provides a practical clue that can be used when applying software protections based on data obfuscation.
△ Less
Submitted 7 April, 2017;
originally announced April 2017.
-
Lessons Learnt in Conducting Survey Research
Authors:
Marco Torchiano,
Daniel Méndez Fernández,
Guilherme Horta Travassos,
Rafael Maiani de Mello
Abstract:
Context: Surveys constitute an valuable tool to capture a large-scale snapshot of the state of the practice. Apparently trivial to adopt, surveys hide, however, several pitfalls that might hinder rendering the result valid and, thus, useful. Goal: We aim at providing an overview of main pitfalls in software engineering surveys and report on practical ways to deal with them. Method: We build on the…
▽ More
Context: Surveys constitute an valuable tool to capture a large-scale snapshot of the state of the practice. Apparently trivial to adopt, surveys hide, however, several pitfalls that might hinder rendering the result valid and, thus, useful. Goal: We aim at providing an overview of main pitfalls in software engineering surveys and report on practical ways to deal with them. Method: We build on the experiences we collected in conducting many studies and distill the main lessons learnt. Results: The eight lessons learnt we report cover different aspects of the survey process ranging from the design of initial research objectives to the design of a questionnaire. Conclusions: Our hope is that by sharing our lessons learnt, combined with a disciplined application of the general survey theory, we contribute to improving the quality of the research results achievable by employing software engineering surveys.
△ Less
Submitted 24 February, 2017; v1 submitted 19 February, 2017;
originally announced February 2017.
