Showing 1–11 of 11 results for author: Straubinger, P

Acknowledging Good Java Code with Code Perfumes

Authors: Philipp Straubinger, Florian Obermüller, Gordon Fraser

Abstract: Java remains one of the most popular programming languages in education. Although Java programming education is well supported by study materials, learners also need more immediate support on the problems they face in their own code. When this support cannot be offered by educators personally, learners can resort to automated program analysis tools such as linters, which provide feedback on potent… ▽ More Java remains one of the most popular programming languages in education. Although Java programming education is well supported by study materials, learners also need more immediate support on the problems they face in their own code. When this support cannot be offered by educators personally, learners can resort to automated program analysis tools such as linters, which provide feedback on potential bugs or code issues. This is constructive feedback, but it may nevertheless feel like criticism. This paper introduces code perfumes for Java, a simple program analysis technique similar to linting, but commending the correct application of good programming practices. We present a catalogue of 20 Java code perfumes related to common Java language constructs for beginner to immediate learners. Our evaluation shows that these code perfumes occur frequently in learners' code, and programs with more code perfume instances tend to have better functionality and readability. Moreover, students who incorporate more code perfumes tend to achieve higher grades. Thus, code perfumes serve as a valuable tool to acknowledge learners' successes, and as a means to inform instructors about their learners' progress. △ Less

Submitted 24 June, 2024; originally announced June 2024.

Engaging Young Learners with Testing Using the Code Critters Mutation Game

Authors: Philipp Straubinger, Lena Bloch, Gordon Fraser

Abstract: Everyone learns to code nowadays. Writing code, however, does not go without testing, which unfortunately rarely seems to be taught explicitly. Testing is often not deemed important enough or is just not perceived as sufficiently exciting. Testing can be exciting: In this paper, we introduce Code Critters, a serious game designed to teach testing concepts engagingly. In the style of popular tower… ▽ More Everyone learns to code nowadays. Writing code, however, does not go without testing, which unfortunately rarely seems to be taught explicitly. Testing is often not deemed important enough or is just not perceived as sufficiently exciting. Testing can be exciting: In this paper, we introduce Code Critters, a serious game designed to teach testing concepts engagingly. In the style of popular tower defense games, players strategically position magical portals that need to distinguish between creatures exhibiting the behavior described by correct code from those that are mutated, and thus faulty. When placing portals, players are implicitly testing: They choose test inputs (i.e., where to place portals), as well as test oracles (i.e., what behavior to expect), and they observe test executions as the creatures wander across the landscape passing the players' portals. An empirical study involving 40 children demonstrates that they actively engage with Code Critters. Their positive feedback provides evidence that they enjoyed playing the game, and some of the children even continued to play Code Critters at home, outside the educational setting of our study. △ Less

Submitted 14 April, 2024; originally announced April 2024.

An Empirical Evaluation of Manually Created Equivalent Mutants

Authors: Philipp Straubinger, Alexander Degenhart, Gordon Fraser

Abstract: Mutation testing consists of evaluating how effective test suites are at detecting artificially seeded defects in the source code, and guiding the improvement of the test suites. Although mutation testing tools are increasingly adopted in practice, equivalent mutants, i.e., mutants that differ only in syntax but not semantics, hamper this process. While prior research investigated how frequently e… ▽ More Mutation testing consists of evaluating how effective test suites are at detecting artificially seeded defects in the source code, and guiding the improvement of the test suites. Although mutation testing tools are increasingly adopted in practice, equivalent mutants, i.e., mutants that differ only in syntax but not semantics, hamper this process. While prior research investigated how frequently equivalent mutants are produced by mutation testing tools and how effective existing methods of detecting these equivalent mutants are, it remains unclear to what degree humans also create equivalent mutants, and how well they perform at identifying these. We therefore study these questions in the context of Code Defenders, a mutation testing game, in which players competitively produce mutants and tests. Using manual inspection as well as automated identification methods we establish that less than 10 % of manually created mutants are equivalent. Surprisingly, our findings indicate that a significant portion of developers struggle to accurately identify equivalent mutants, emphasizing the need for improved detection mechanisms and developer training in mutation testing. △ Less

Submitted 14 April, 2024; originally announced April 2024.

IntelliGame in Action: An Experience Report on Gamifying JavaScript Unit Tests

Authors: Philipp Straubinger, Tommaso Fulcini, Gordon Fraser, Marco Torchiano

Abstract: This paper investigates the integration and assessment of IntelliGame, a gamification plugin initially designed for Java development, within the realm of JavaScript unit testing. We aim to verify the generalizability of IntelliGame to JavaScript development and to provide valuable insights into the experiment's design. For this, we first customize IntelliGame for JavaScript, and then conduct a con… ▽ More This paper investigates the integration and assessment of IntelliGame, a gamification plugin initially designed for Java development, within the realm of JavaScript unit testing. We aim to verify the generalizability of IntelliGame to JavaScript development and to provide valuable insights into the experiment's design. For this, we first customize IntelliGame for JavaScript, and then conduct a controlled experiment involving 152 participants utilizing the Jest testing framework, and finally examine its influence on testing behavior and the overall developer experience. The findings from this study provide valuable insights for improving JavaScript testing methodologies through the incorporation of gamification. △ Less

Submitted 6 March, 2024; originally announced March 2024.

An IDE Plugin for Gamified Continuous Integration

Authors: Philipp Straubinger, Gordon Fraser

Abstract: Interruptions and context switches resulting from meetings, urgent tasks, emails, and queries from colleagues contribute to productivity losses in developers' daily routines. This is particularly challenging for tasks like software testing, which are already perceived as less enjoyable, prompting developers to seek distractions. To mitigate this, applying gamification to testing activities can enh… ▽ More Interruptions and context switches resulting from meetings, urgent tasks, emails, and queries from colleagues contribute to productivity losses in developers' daily routines. This is particularly challenging for tasks like software testing, which are already perceived as less enjoyable, prompting developers to seek distractions. To mitigate this, applying gamification to testing activities can enhance motivation for test writing. One such gamification tool is Gamekins, which integrates challenges, quests, achievements, and leaderboards into the Jenkins CI (continuous integration) platform. However, as Gamekins is typically accessed through a browser, it introduces a context switch. This paper presents an IntelliJ plugin designed to seamlessly integrate Gamekins' gamification elements into the IDE, aiming to minimize context switches and boost developer motivation for test writing. △ Less

Submitted 6 March, 2024; originally announced March 2024.

Gamifying a Software Testing Course with Continuous Integration

Authors: Philipp Straubinger, Gordon Fraser

Abstract: Testing plays a crucial role in software development, and it is essential for software engineering students to receive proper testing education. However, motivating students to write tests and use automated testing during software development can be challenging. To address this issue and enhance student engagement in testing when they write code, we propose to incentivize students to test more by… ▽ More Testing plays a crucial role in software development, and it is essential for software engineering students to receive proper testing education. However, motivating students to write tests and use automated testing during software development can be challenging. To address this issue and enhance student engagement in testing when they write code, we propose to incentivize students to test more by gamifying continuous integration. For this we use Gamekins, a tool that is seamlessly integrated into the Jenkins continuous integration platform and uses game elements based on commits to the source code repository: Developers can earn points by completing test challenges and quests generated by Gamekins, compete with other developers or teams on a leaderboard, and receive achievements for their test-related accomplishments. In this paper, we present our integration of Gamekins into an undergraduate-level course on software testing. We observe a correlation between how students test their code and their use of Gamekins, as well as a significant improvement in the accuracy of their results compared to a previous iteration of the course without gamification. As a further indicator of how this approach improves testing behavior, the students reported enjoyment in writing tests with Gamekins. △ Less

Submitted 31 January, 2024; originally announced January 2024.

PlayTest: A Gamified Test Generator for Games

Authors: Patric Feldmeier, Philipp Straubinger, Gordon Fraser

Abstract: Games are usually created incrementally, requiring repeated testing of the same scenarios, which is a tedious and error-prone task for game developers. Therefore, we aim to alleviate this game testing process by encapsulating it into a game called Playtest, which transforms the tiring testing process into a competitive game with a purpose. Playtest automates the generation of valuable test cases b… ▽ More Games are usually created incrementally, requiring repeated testing of the same scenarios, which is a tedious and error-prone task for game developers. Therefore, we aim to alleviate this game testing process by encapsulating it into a game called Playtest, which transforms the tiring testing process into a competitive game with a purpose. Playtest automates the generation of valuable test cases based on player actions, without the players even realising it. We envision the use of Playtest to crowdsource the task of testing games by giving players access to the respective games through our tool in the playtesting phases during the development process. △ Less

Submitted 30 October, 2023; originally announced October 2023.

Comments: 4 pages with 4 figures, to be published in Proceedings of the 2nd International Workshop on Gamification in Software Development, Verification, and Validation 2023

Improving Testing Behavior by Gamifying IntelliJ

Authors: Philipp Straubinger, Gordon Fraser

Abstract: Testing is an important aspect of software development, but unfortunately, it is often neglected. While test quality analyses such as code coverage or mutation analysis inform developers about the quality of their tests, such reports are viewed only sporadically during continuous integration or code review, if they are considered at all, and their impact on the developers' testing behavior therefo… ▽ More Testing is an important aspect of software development, but unfortunately, it is often neglected. While test quality analyses such as code coverage or mutation analysis inform developers about the quality of their tests, such reports are viewed only sporadically during continuous integration or code review, if they are considered at all, and their impact on the developers' testing behavior therefore tends to be negligible. To actually influence developer behavior, it may rather be necessary to motivate developers directly within their programming environment, while they are coding. We introduce IntelliGame, a gamified plugin for the popular IntelliJ Java Integrated Development Environment, which rewards developers for positive testing behavior using a multi-level achievement system: A total of 27 different achievements, each with incremental levels, provide affirming feedback when developers exhibit commendable testing behavior, and provide an incentive to further continue and improve this behavior. A controlled experiment with 49 participants given a Java programming task reveals substantial differences in the testing behavior triggered by IntelliGame: Incentivized developers write more tests, achieve higher coverage and mutation scores, run their tests more often, and achieve functionality earlier. △ Less

Submitted 17 October, 2023; originally announced October 2023.

A Survey on What Developers Think About Testing

Authors: Philipp Straubinger, Gordon Fraser

Abstract: Software is infamous for its poor quality and frequent occurrence of bugs. While there is no doubt that thorough testing is an appropriate answer to ensure sufficient quality, the poor state of software generally suggests that developers may not always engage as thoroughly with testing as they should. This observation aligns with the prevailing belief that developers simply do not like writing tes… ▽ More Software is infamous for its poor quality and frequent occurrence of bugs. While there is no doubt that thorough testing is an appropriate answer to ensure sufficient quality, the poor state of software generally suggests that developers may not always engage as thoroughly with testing as they should. This observation aligns with the prevailing belief that developers simply do not like writing tests. In order to determine the truth of this belief, we conducted a comprehensive survey with 21 questions aimed at (1) assessing developers' current engagement with testing and (2) identifying factors influencing their inclination toward testing; that is, whether they would actually like to test more but are inhibited by their work environment, or whether they would really prefer to test even less if given the choice. Drawing on 284 responses from professional software developers, we uncover reasons that positively and negatively impact developers' motivation to test. Notably, reasons for motivation to write more tests encompass not only a general pursuit of software quality but also personal satisfaction. However, developers nevertheless perceive testing as mundane and tend to prioritize other tasks. One approach emerging from the responses to mitigate these negative factors is by providing better recognition for developers' testing efforts. △ Less

Submitted 3 September, 2023; originally announced September 2023.

Code Critters: A Block-Based Testing Game

Authors: Philipp Straubinger, Laura Caspari, Gordon Fraser

Abstract: Learning to program has become common in schools, higher education and individual learning. Although testing is an important aspect of programming, it is often neglected in education due to a perceived lack of time and knowledge, or simply because testing is considered less important or fun. To make testing more engaging, we therefore introduce Code Critters, a Tower Defense game based on testing… ▽ More Learning to program has become common in schools, higher education and individual learning. Although testing is an important aspect of programming, it is often neglected in education due to a perceived lack of time and knowledge, or simply because testing is considered less important or fun. To make testing more engaging, we therefore introduce Code Critters, a Tower Defense game based on testing concepts: The aim of the game is to place magic mines along the route taken by small "critters" from their home to a tower, such that the mines distinguish between critters executing correct code from those executing buggy code. Code is shown and edited using a block-based language to make the game accessible for younger learners. The mines encode test inputs as well as test oracles, thus making testing an integral and fun component of the game. △ Less

Submitted 5 April, 2023; originally announced April 2023.

Gamekins: Gamifying Software Testing in Jenkins

Authors: Philipp Straubinger, Gordon Fraser

Abstract: Developers have to write thorough tests for their software in order to find bugs and to prevent regressions. Writing tests, however, is not every developer's favourite occupation, and if a lack of motivation leads to a lack of tests, then this may have dire consequences, such as programs with poor quality or even project failures. This paper introduces Gamekins, a tool that uses gamification to mo… ▽ More Developers have to write thorough tests for their software in order to find bugs and to prevent regressions. Writing tests, however, is not every developer's favourite occupation, and if a lack of motivation leads to a lack of tests, then this may have dire consequences, such as programs with poor quality or even project failures. This paper introduces Gamekins, a tool that uses gamification to motivate developers to write more and better tests. Gamekins is integrated into the Jenkins continuous integration platform where game elements are based on commits to the source code repository: Developers can earn points for completing test challenges and quests posed by Gamekins, compete with other developers or developer teams on a leaderboard, and are rewarded for their test-related achievements. △ Less

Submitted 14 February, 2022; originally announced February 2022.