-
Testing and Debugging Quantum Programs: The Road to 2030
Authors:
Neilson Carlos Leite Ramalho,
Higor Amario de Souza,
Marcos Lordello Chaim
Abstract:
Quantum Computing has existed in the theoretical realm for several decades. Recently, given the latest developments in hardware, quantum computing has re-emerged as a promising technology with the potential to solve problems that a classical computer could take hundreds of years to solve. With the rising interest in the field, there are challenges and opportunities for academics and practitioners…
▽ More
Quantum Computing has existed in the theoretical realm for several decades. Recently, given the latest developments in hardware, quantum computing has re-emerged as a promising technology with the potential to solve problems that a classical computer could take hundreds of years to solve. With the rising interest in the field, there are challenges and opportunities for academics and practitioners in terms of software engineering practices, particularly in testing and debugging quantum programs. This paper presents a roadmap for addressing these challenges, pointing out the existing gaps in the literature and suggesting research directions. We present the current state-of-the-art testing and debugging strategies, including classical techniques applied to quantum programs, the development and implementation of quantum-specific assertions, and the identification and classification of bug patterns unique to quantum computing. Additionally, we introduce a conceptual model to illustrate the main concepts regarding the testing and debugging of quantum programs as well as the relationship between them. Those concepts are then used to identify and discuss the main research challenges to cope with quantum programs through 2030, focusing on the interfaces between classical and quantum computing and on creating testing and debugging techniques that take advantage of the unique quantum computing characteristics.
△ Less
Submitted 15 May, 2024;
originally announced May 2024.
-
Evaluating data-flow coverage in spectrum-based fault localization
Authors:
Henrique Lemos Ribeiro,
Higor Amario de Souza,
Roberto Paulo de Andrioli Araujo,
Marcos Lordello Chaim,
Fabio Kon
Abstract:
Background: Debugging is a key task during the software development cycle. Spectrum-based Fault Localization (SFL) is a promising technique to improve and automate debugging. SFL techniques use control-flow spectra to pinpoint the most suspicious program elements. However, data-flow spectra provide more detailed information about the program execution, which may be useful for fault localization. A…
▽ More
Background: Debugging is a key task during the software development cycle. Spectrum-based Fault Localization (SFL) is a promising technique to improve and automate debugging. SFL techniques use control-flow spectra to pinpoint the most suspicious program elements. However, data-flow spectra provide more detailed information about the program execution, which may be useful for fault localization. Aims: We evaluate the effectiveness and efficiency of ten SFL ranking metrics using data-flow spectra. Method: We compare the performance of data- and control-flow spectra for SFL using 163 faults from 5 real-world open source programs, which contain from 468 to 4130 test cases. The data- and control-flow spectra types used in our evaluation are definition-use associations (DUAs) and lines, respectively. Results: Using data-flow spectra, up to 50% more faults are ranked in the top-15 positions compared to control-flow spectra. Also, most SFL ranking metrics present better effectiveness using data-flow to inspect up to the top-40 positions. The execution cost of data-flow spectra is higher than control-flow, taking from 22 seconds to less than 9 minutes. Data-flow has an average overhead of 353% for all programs, while the average overhead for control-flow is of 102%. Conclusions: The results suggest that SFL techniques can benefit from using data-flow spectra to classify faults in better positions, which may lead developers to inspect less code to find bugs. The execution cost to gather data-flow is higher compared to control-flow, but it is not prohibitive. Moreover, data-flow spectra also provide information about suspicious variables for fault localization, which may improve the developers' performance using SFL.
△ Less
Submitted 27 June, 2019;
originally announced June 2019.
-
Spectrum-based Software Fault Localization: A Survey of Techniques, Advances, and Challenges
Authors:
Higor A. de Souza,
Marcos L. Chaim,
Fabio Kon
Abstract:
Despite being one of the most basic tasks in software development, debugging is still performed in a mostly manual way, leading to high cost and low performance. To address this problem, researchers have studied promising approaches, such as Spectrum-based Fault Localization (SFL) techniques, which pinpoint program elements more likely to contain faults. This survey discusses the state-of-the-art…
▽ More
Despite being one of the most basic tasks in software development, debugging is still performed in a mostly manual way, leading to high cost and low performance. To address this problem, researchers have studied promising approaches, such as Spectrum-based Fault Localization (SFL) techniques, which pinpoint program elements more likely to contain faults. This survey discusses the state-of-the-art of SFL, including the different techniques that have been proposed, the type and number of faults they address, the types of spectra they use, the programs they utilize in their validation, the testing data that support them, and their use at industrial settings. Notwithstanding the advances, there are still challenges for the industry to adopt these techniques, which we analyze in this paper. SFL techniques should propose new ways to generate reduced sets of suspicious entities, combine different spectra to fine-tune the fault localization ability, use strategies to collect fine-grained coverage levels from suspicious coarser levels for balancing execution costs and output precision, and propose new techniques to cope with multiple-fault programs. Moreover, additional user studies are needed to understand better how SFL techniques can be used in practice. We conclude by presenting a concept map about topics and challenges for future research in SFL.
△ Less
Submitted 26 November, 2017; v1 submitted 14 July, 2016;
originally announced July 2016.
