-
Towards Better Graph Neural Network-based Fault Localization Through Enhanced Code Representation
Authors:
Md Nakhla Rafi,
Dong Jae Kim,
An Ran Chen,
Tse-Hsun Chen,
Shaowei Wang
Abstract:
Automatic software fault localization plays an important role in software quality assurance by pinpointing faulty locations for easier debugging. Coverage-based fault localization, a widely used technique, employs statistics on coverage spectra to rank code based on suspiciousness scores. However, the rigidity of statistical approaches calls for learning-based techniques. Amongst all, Grace, a gra…
▽ More
Automatic software fault localization plays an important role in software quality assurance by pinpointing faulty locations for easier debugging. Coverage-based fault localization, a widely used technique, employs statistics on coverage spectra to rank code based on suspiciousness scores. However, the rigidity of statistical approaches calls for learning-based techniques. Amongst all, Grace, a graph-neural network (GNN) based technique has achieved state-of-the-art due to its capacity to preserve coverage spectra, i.e., test-to-source coverage relationships, as precise abstract syntax-enhanced graph representation, mitigating the limitation of other learning-based technique which compresses the feature representation. However, such representation struggles with scalability due to the increasing complexity of software and associated coverage spectra and AST graphs. In this work, we proposed a new graph representation, DepGraph, that reduces the complexity of the graph representation by 70% in nodes and edges by integrating interprocedural call graph in the graph representation of the code. Moreover, we integrate additional features such as code change information in the graph as attributes so the model can leverage rich historical project data. We evaluate DepGraph using Defects4j 2.0.0, and it outperforms Grace by locating 20% more faults in Top-1 and improving the Mean First Rank (MFR) and the Mean Average Rank (MAR) by over 50% while decreasing GPU memory usage by 44% and training/inference time by 85%. Additionally, in cross-project settings, DepGraph surpasses the state-of-the-art baseline with a 42% higher Top-1 accuracy, and 68% and 65% improvement in MFR and MAR, respectively. Our study demonstrates DepGraph's robustness, achieving state-of-the-art accuracy and scalability for future extension and adoption.
△ Less
Submitted 30 April, 2024; v1 submitted 6 April, 2024;
originally announced April 2024.
-
Back to the Future! Studying Data Cleanness in Defects4J and its Impact on Fault Localization
Authors:
An Ran Chen,
Md Nakhla Rafi,
Tse-Hsun Chen,
Shaohua Wang
Abstract:
For software testing research, Defects4J stands out as the primary benchmark dataset, offering a controlled environment to study real bugs from prominent open-source systems. However, prior research indicates that Defects4J might include tests added post-bug report, embedding developer knowledge and affecting fault localization efficacy. In this paper, we examine Defects4J's fault-triggering tests…
▽ More
For software testing research, Defects4J stands out as the primary benchmark dataset, offering a controlled environment to study real bugs from prominent open-source systems. However, prior research indicates that Defects4J might include tests added post-bug report, embedding developer knowledge and affecting fault localization efficacy. In this paper, we examine Defects4J's fault-triggering tests, emphasizing the implications of developer knowledge of SBFL techniques. We study the timelines of changes made to these tests concerning bug report creation. Then, we study the effectiveness of SBFL techniques without developer knowledge in the tests. We found that 1) 55% of the fault-triggering tests were newly added to replicate the bug or to test for regression; 2) 22% of the fault-triggering tests were modified after the bug reports were created, containing developer knowledge of the bug; 3) developers often modify the tests to include new assertions or change the test code to reflect the changes in the source code; and 4) the performance of SBFL techniques degrades significantly (up to --415% for Mean First Rank) when evaluated on the bugs without developer knowledge. We provide a dataset of bugs without developer insights, aiding future SBFL evaluations in Defects4J and informing considerations for future bug benchmarks.
△ Less
Submitted 9 November, 2023; v1 submitted 29 October, 2023;
originally announced October 2023.
-
LMFLOSS: A Hybrid Loss For Imbalanced Medical Image Classification
Authors:
Abu Adnan Sadi,
Labib Chowdhury,
Nursrat Jahan,
Mohammad Newaz Sharif Rafi,
Radeya Chowdhury,
Faisal Ahamed Khan,
Nabeel Mohammed
Abstract:
Automatic medical image classification is a very important field where the use of AI has the potential to have a real social impact. However, there are still many challenges that act as obstacles to making practically effective solutions. One of those is the fact that most of the medical imaging datasets have a class imbalance problem. This leads to the fact that existing AI techniques, particular…
▽ More
Automatic medical image classification is a very important field where the use of AI has the potential to have a real social impact. However, there are still many challenges that act as obstacles to making practically effective solutions. One of those is the fact that most of the medical imaging datasets have a class imbalance problem. This leads to the fact that existing AI techniques, particularly neural network-based deep-learning methodologies, often perform poorly in such scenarios. Thus this makes this area an interesting and active research focus for researchers. In this study, we propose a novel loss function to train neural network models to mitigate this critical issue in this important field. Through rigorous experiments on three independently collected datasets of three different medical imaging domains, we empirically show that our proposed loss function consistently performs well with an improvement between 2%-10% macro f1 when compared to the baseline models. We hope that our work will precipitate new research toward a more generalized approach to medical image classification.
△ Less
Submitted 24 December, 2022;
originally announced December 2022.
-
Performance Evaluation of the LoRa Protocol in the context of Smart Meter
Authors:
Muhammad Nouman Rafi,
Muhammad Muaaz
Abstract:
In recent years, the use of Low Power Wide Area Network (LPWAN) is increasing for the Internet of Things (IoT) applications. In order to demonstrate the application of LPWAN technologies for a realistic smart metering scenario, we set-up and implement a widely used LPWAN protocol which is called LoRaWAN. In this study, the LoRaWAN is implemented by using Multitech devices (end-node and gateway) an…
▽ More
In recent years, the use of Low Power Wide Area Network (LPWAN) is increasing for the Internet of Things (IoT) applications. In order to demonstrate the application of LPWAN technologies for a realistic smart metering scenario, we set-up and implement a widely used LPWAN protocol which is called LoRaWAN. In this study, the LoRaWAN is implemented by using Multitech devices (end-node and gateway) and the performance of the network is evaluated for different physical (e.g. location, distance etc.) and link parameters (e.g. data rate, transmission power etc.), under the European regulations. To evaluate the performance of the networks, we collected uplink packets in different indoor and outdoor scenarios at various locations. Our results show that LoRaWAN is easy to setup, configurable, scalable, and it performs well for real-time smart metering applications. Moreover, it is necessary to evaluate the physical conditions for the selection of the available system parameters for deploying a robust LoRaWAN network.
△ Less
Submitted 4 July, 2019;
originally announced July 2019.
