-
Execution-free Program Repair
Authors:
Li Huang,
Bertrand Meyer,
Ilgiz Mustafin,
Manuel Oriol
Abstract:
Automatic program repair usually relies heavily on test cases for both bug identification and fix validation. The issue is that writing test cases is tedious, running them takes much time, and validating a fix through tests does not guarantee its correctness. The novel idea in the Proof2Fix methodology and tool presented here is to rely instead on a program prover, without the need to run tests or…
▽ More
Automatic program repair usually relies heavily on test cases for both bug identification and fix validation. The issue is that writing test cases is tedious, running them takes much time, and validating a fix through tests does not guarantee its correctness. The novel idea in the Proof2Fix methodology and tool presented here is to rely instead on a program prover, without the need to run tests or to run the program at all. Results show that Proof2Fix finds and fixes significant historical bugs.
△ Less
Submitted 9 May, 2024; v1 submitted 2 May, 2024;
originally announced May 2024.
-
C4Q: A Chatbot for Quantum
Authors:
Yaiza Aragonés-Soria,
Manuel Oriol
Abstract:
Quantum computing is a growing field that promises many real-world applications such as quantum cryptography or quantum finance. The number of people able to use quantum computing is however still very small. This limitation comes from the difficulty to understand the concepts and to know how to start coding. Therefore, there is a need for tools that can assist non-expert in overcoming this comple…
▽ More
Quantum computing is a growing field that promises many real-world applications such as quantum cryptography or quantum finance. The number of people able to use quantum computing is however still very small. This limitation comes from the difficulty to understand the concepts and to know how to start coding. Therefore, there is a need for tools that can assist non-expert in overcoming this complexity. One possibility would be to use existing conversational agents. Unfortunately ChatGPT and other Large-Language Models produce inaccurate results. This article presents C4Q, a chatbot that answers accurately basic questions and guides users when trying to code quantum programs. Contrary to other approaches C4Q uses a pre-trained large language model only to discover and classify user requests. It then generates an accurate answer using an own engine. Thanks to this architectural design, C4Q's answers are always correct, and thus C4Q can become a support tool that makes quantum computing more available to non-experts.
△ Less
Submitted 29 January, 2024;
originally announced February 2024.
-
Seeding Contradiction: a fast method for generating full-coverage test suites
Authors:
Li Huang,
Bertrand Meyer,
Manuel Oriol
Abstract:
The regression test suite, a key resource for managing program evolution, needs to achieve 100% coverage, or very close, to be useful. Devising a test suite manually is unacceptably tedious, but existing automated methods are often inefficient. The method described in this article, ``Seeding Contradiction'', inserts incorrect instructions into every basic block of the program, enabling an SMT-base…
▽ More
The regression test suite, a key resource for managing program evolution, needs to achieve 100% coverage, or very close, to be useful. Devising a test suite manually is unacceptably tedious, but existing automated methods are often inefficient. The method described in this article, ``Seeding Contradiction'', inserts incorrect instructions into every basic block of the program, enabling an SMT-based Hoare-style prover to generate a counterexample for every branch of the program and, from the collection of all such counterexamples, a test suite. The method is static, works fast, and achieves excellent coverage.
△ Less
Submitted 8 September, 2023;
originally announced September 2023.
-
Improving Counterexample Quality from Failed Program Verification
Authors:
Li Huang,
Bertrand Meyer,
Manuel Oriol
Abstract:
In software verification, a successful automated program proof is the ultimate triumph. The road to such success is, however, paved with many failed proof attempts. The message produced by the prover when a proof fails is often obscure, making it very hard to know how to proceed further. The work reported here attempts to help in such cases by providing immediately understandable counterexamples.…
▽ More
In software verification, a successful automated program proof is the ultimate triumph. The road to such success is, however, paved with many failed proof attempts. The message produced by the prover when a proof fails is often obscure, making it very hard to know how to proceed further. The work reported here attempts to help in such cases by providing immediately understandable counterexamples.
To this end, it introduces an approach called Counterexample Extraction and Minimization (CEAM). When a proof fails, CEAM turns the counterexample model generated by the prover into a a clearly understandable version; it can in addition simplify the counterexamples further by minimizing the integer values they contain. We have implemented the CEAM approach as an extension to the AutoProof verifier and demonstrate its application to a collection of examples.
△ Less
Submitted 26 August, 2022; v1 submitted 21 August, 2022;
originally announced August 2022.
-
Software Engineering for AI-Based Systems: A Survey
Authors:
Silverio Martínez-Fernández,
Justus Bogner,
Xavier Franch,
Marc Oriol,
Julien Siebert,
Adam Trendowicz,
Anna Maria Vollmer,
Stefan Wagner
Abstract:
AI-based systems are software systems with functionalities enabled by at least one AI component (e.g., for image- and speech-recognition, and autonomous driving). AI-based systems are becoming pervasive in society due to advances in AI. However, there is limited synthesized knowledge on Software Engineering (SE) approaches for building, operating, and maintaining AI-based systems. To collect and a…
▽ More
AI-based systems are software systems with functionalities enabled by at least one AI component (e.g., for image- and speech-recognition, and autonomous driving). AI-based systems are becoming pervasive in society due to advances in AI. However, there is limited synthesized knowledge on Software Engineering (SE) approaches for building, operating, and maintaining AI-based systems. To collect and analyze state-of-the-art knowledge about SE for AI-based systems, we conducted a systematic map** study. We considered 248 studies published between January 2010 and March 2020. SE for AI-based systems is an emerging research area, where more than 2/3 of the studies have been published since 2018. The most studied properties of AI-based systems are dependability and safety. We identified multiple SE approaches for AI-based systems, which we classified according to the SWEBOK areas. Studies related to software testing and software quality are very prevalent, while areas like software maintenance seem neglected. Data-related issues are the most recurrent challenges. Our results are valuable for: researchers, to quickly understand the state of the art and learn which topics need more research; practitioners, to learn about the approaches and challenges that SE entails for AI-based systems; and, educators, to bridge the gap among SE and AI in their curricula.
△ Less
Submitted 2 September, 2021; v1 submitted 5 May, 2021;
originally announced May 2021.
-
How do Practitioners Perceive the Relevance of Requirements Engineering Research?
Authors:
Xavier Franch,
Daniel Mendez,
Andreas Vogelsang,
Rogardt Heldal,
Eric Knauss,
Marc Oriol,
Guilherme H. Travassos,
Jeffrey C. Carver,
Thomas Zimmermann
Abstract:
The relevance of Requirements Engineering (RE) research to practitioners is vital for a long-term dissemination of research results to everyday practice. Some authors have speculated about a mismatch between research and practice in the RE discipline. However, there is not much evidence to support or refute this perception. This paper presents the results of a study aimed at gathering evidence fro…
▽ More
The relevance of Requirements Engineering (RE) research to practitioners is vital for a long-term dissemination of research results to everyday practice. Some authors have speculated about a mismatch between research and practice in the RE discipline. However, there is not much evidence to support or refute this perception. This paper presents the results of a study aimed at gathering evidence from practitioners about their perception of the relevance of RE research and at understanding the factors that influence that perception. We conducted a questionnaire-based survey of industry practitioners with expertise in RE. The participants rated the perceived relevance of 435 scientific papers presented at five top RE-related conferences. The 153 participants provided a total of 2,164 ratings. The practitioners rated RE research as essential or worthwhile in a majority of cases. However, the percentage of non-positive ratings is still higher than we would like. Among the factors that affect the perception of relevance are the research's links to industry, the research method used, and respondents' roles. The reasons for positive perceptions were primarily related to the relevance of the problem and the soundness of the solution, while the causes for negative perceptions were more varied. The respondents also provided suggestions for future research, including topics researchers have studied for decades, like elicitation or requirement quality criteria.
△ Less
Submitted 3 December, 2020; v1 submitted 10 November, 2020;
originally announced November 2020.
-
Develo** and Operating Artificial Intelligence Models in Trustworthy Autonomous Systems
Authors:
Silverio Martínez-Fernández,
Xavier Franch,
Andreas Jedlitschka,
Marc Oriol,
Adam Trendowicz
Abstract:
Companies dealing with Artificial Intelligence (AI) models in Autonomous Systems (AS) face several problems, such as users' lack of trust in adverse or unknown conditions, gaps between software engineering and AI model development, and operation in a continuously changing operational environment. This work-in-progress paper aims to close the gap between the development and operation of trustworthy…
▽ More
Companies dealing with Artificial Intelligence (AI) models in Autonomous Systems (AS) face several problems, such as users' lack of trust in adverse or unknown conditions, gaps between software engineering and AI model development, and operation in a continuously changing operational environment. This work-in-progress paper aims to close the gap between the development and operation of trustworthy AI-based AS by defining an approach that coordinates both activities. We synthesize the main challenges of AI-based AS in industrial settings. We reflect on the research efforts required to overcome these challenges and propose a novel, holistic DevOps approach to put it into practice. We elaborate on four research directions: (a) increased users' trust by monitoring operational AI-based AS and identifying self-adaptation needs in critical situations; (b) integrated agile process for the development and evolution of AI models and AS; (c) continuous deployment of different context-specific instances of AI models in a distributed setting of AS; and (d) holistic DevOps-based lifecycle for AI-based AS.
△ Less
Submitted 23 April, 2021; v1 submitted 11 March, 2020;
originally announced March 2020.
-
How do Practitioners Perceive the Relevance of Requirements Engineering Research? An Ongoing Study
Authors:
X. Franch,
D. Méndez Fernández,
M. Oriol,
A. Vogelsang,
R. Heldal,
E. Knauss,
G. Horta Travassos,
J. C. Carver,
O. Dieste,
T. Zimmermann
Abstract:
The relevance of Requirements Engineering (RE) research to practitioners is a prerequisite for problem-driven research in the area and key for a long-term dissemination of research results to everyday practice. To better understand how industry practitioners perceive the practical relevance of RE research, we have initiated the RE-Pract project, an international collaboration conducting an empiric…
▽ More
The relevance of Requirements Engineering (RE) research to practitioners is a prerequisite for problem-driven research in the area and key for a long-term dissemination of research results to everyday practice. To better understand how industry practitioners perceive the practical relevance of RE research, we have initiated the RE-Pract project, an international collaboration conducting an empirical study. This project opts for a replication of previous work done in two different domains and relies on survey research. To this end, we have designed a survey to be sent to several hundred industry practitioners at various companies around the world and ask them to rate their perceived practical relevance of the research described in a sample of 418 RE papers published between 2010 and 2015 at the RE, ICSE, FSE, ESEC/FSE, ESEM and REFSQ conferences. In this paper, we summarise our research protocol and present the current status of our study and the planned future steps.
△ Less
Submitted 14 June, 2017; v1 submitted 17 May, 2017;
originally announced May 2017.
-
Stochastic Contracts for Runtime Checking of Component-based Real-time Systems
Authors:
Chandrakana Nandi,
Aurelien Monot,
Manuel Oriol
Abstract:
This paper introduces a new technique for dynamic verification of component-based real-time systems based on statistical inference. Verifying such systems requires checking two types of properties: functional and real-time. For functional properties, a standard approach for ensuring correctness is Design by Contract: annotating programs with executable pre- and postconditions. We extend contracts…
▽ More
This paper introduces a new technique for dynamic verification of component-based real-time systems based on statistical inference. Verifying such systems requires checking two types of properties: functional and real-time. For functional properties, a standard approach for ensuring correctness is Design by Contract: annotating programs with executable pre- and postconditions. We extend contracts for specifying real-time properties. In the industry, components are often bought from vendors and meant to be used off-the-shelf which makes it very difficult to determine their execution times and express related properties. We present a solution to this problem by using statistical inference for estimating the properties. The contract framework allows application developers to express contracts like "the execution time of component $X$ lies within $γ$ standard deviations from the mean execution time". Experiments based on industrial case studies show that this framework can be smoothly integrated into existing control applications, thereby increasing their reliability while having an acceptable execution time overhead (less than 10%).
△ Less
Submitted 10 January, 2015;
originally announced January 2015.
-
Execution Time Analysis for Industrial Control Applications
Authors:
Stefan Stattelmann,
Manuel Oriol,
Thomas Gamer
Abstract:
Estimating the execution time of software components is often mandatory when evaluating the non-functional properties of software-intensive systems. This particularly holds for real-time embedded systems, e.g., in the context of industrial automation. In practice it is however very hard to obtain reliable execution time estimates which are accurate, but not overly pessimistic with respect to the…
▽ More
Estimating the execution time of software components is often mandatory when evaluating the non-functional properties of software-intensive systems. This particularly holds for real-time embedded systems, e.g., in the context of industrial automation. In practice it is however very hard to obtain reliable execution time estimates which are accurate, but not overly pessimistic with respect to the typical behavior of the software.
This article proposes two new concepts to ease the use of execution time analysis for industrial control applications: (1) a method based on recurring occurrences of code sequences for automatically creating a timing model of a given processor and (2) an interactive way to integrate execution time analysis into the development environment, thus making timing analysis results easily accessible for software developers. The proposed methods are validated by an industrial case study, which shows that a significant amount of code reuse is present in a set of representative industrial control applications.
△ Less
Submitted 3 April, 2014;
originally announced April 2014.
-
The Search for the Laws of Automatic Random Testing
Authors:
Carlo A. Furia,
Bertrand Meyer,
Manuel Oriol,
Andrey Tikhomirov,
Yi Wei
Abstract:
Can one estimate the number of remaining faults in a software system? A credible estimation technique would be immensely useful to project managers as well as customers. It would also be of theoretical interest, as a general law of software engineering. We investigate possible answers in the context of automated random testing, a method that is increasingly accepted as an effective way to discover…
▽ More
Can one estimate the number of remaining faults in a software system? A credible estimation technique would be immensely useful to project managers as well as customers. It would also be of theoretical interest, as a general law of software engineering. We investigate possible answers in the context of automated random testing, a method that is increasingly accepted as an effective way to discover faults. Our experimental results, derived from best-fit analysis of a variety of mathematical functions, based on a large number of automated tests of library code equipped with automated oracles in the form of contracts, suggest a poly-logarithmic law. Although further confirmation remains necessary on different code bases and testing techniques, we argue that understanding the laws of testing may bring significant benefits for estimating the number of detectable faults and comparing different projects and practices.
△ Less
Submitted 14 November, 2012;
originally announced November 2012.
-
C to O-O Translation: Beyond the Easy Stuff
Authors:
Marco Trudel,
Carlo A. Furia,
Martin Nordio,
Bertrand Meyer,
Manuel Oriol
Abstract:
Can we reuse some of the huge code-base developed in C to take advantage of modern programming language features such as type safety, object-orientation, and contracts? This paper presents a source-to-source translation of C code into Eiffel, a modern object-oriented programming language, and the supporting tool C2Eif. The translation is completely automatic and supports the entire C language (ANS…
▽ More
Can we reuse some of the huge code-base developed in C to take advantage of modern programming language features such as type safety, object-orientation, and contracts? This paper presents a source-to-source translation of C code into Eiffel, a modern object-oriented programming language, and the supporting tool C2Eif. The translation is completely automatic and supports the entire C language (ANSI, as well as many GNU C Compiler extensions, through CIL) as used in practice, including its usage of native system libraries and inlined assembly code. Our experiments show that C2Eif can handle C applications and libraries of significant size (such as vim and libgsl), as well as challenging benchmarks such as the GCC torture tests. The produced Eiffel code is functionally equivalent to the original C code, and takes advantage of some of Eiffel's object-oriented features to produce safe and easy-to-debug translations.
△ Less
Submitted 19 September, 2013; v1 submitted 25 June, 2012;
originally announced June 2012.
-
WeSSQoS: A Configurable SOA System for Quality-aware Web Service Selection
Authors:
Oscar Cabrera,
Marc Oriol,
Xavier Franch,
Lidia López,
Jordi Marco,
Olivia Fragoso,
René Santaolaya
Abstract:
Web Services (WS) have become one the most used technologies nowadays in software systems. Among the challenges when integrating WS in a given system, requirements-driven selection occupies a prominent place. A comprehensive selection process needs to check compliance of Non-Functional Requirements (NFR), which can be assessed by analysing WS Quality of Service (QoS). In this paper, we describe th…
▽ More
Web Services (WS) have become one the most used technologies nowadays in software systems. Among the challenges when integrating WS in a given system, requirements-driven selection occupies a prominent place. A comprehensive selection process needs to check compliance of Non-Functional Requirements (NFR), which can be assessed by analysing WS Quality of Service (QoS). In this paper, we describe the WeSSQoS system that aims at ranking available WS based on the comparison of their QoS and the stated NFRs. WeSSQoS is designed as an open service-oriented architecture that hosts a configurable portfolio of normalization and ranking algorithms that can be selected by the engineer when starting a selection process. WS' QoS can be obtained either from a static, WSDL-like description, or computed dynamically through monitoring techniques. WeSSQoS is designed to work over multiple WS repositories and QoS sources. The impact of having a portfolio of different normalization and ranking algorithms is illustrated with an example.
△ Less
Submitted 25 October, 2011;
originally announced October 2011.
-
Class Schema Evolution for Persistent Object-Oriented Software: Model, Empirical Study, and Automated Support
Authors:
Marco Piccioni,
Manuel Oriol,
Bertrand Meyer
Abstract:
With the wide support for object serialization in object-oriented programming languages, persistent objects have become common place and most large object-oriented software systems rely on extensive amounts of persistent data. Such systems also evolve over time. Retrieving previously persisted objects from classes whose schema has changed is however difficult, and may lead to invalidating the cons…
▽ More
With the wide support for object serialization in object-oriented programming languages, persistent objects have become common place and most large object-oriented software systems rely on extensive amounts of persistent data. Such systems also evolve over time. Retrieving previously persisted objects from classes whose schema has changed is however difficult, and may lead to invalidating the consistency of the application. The ESCHER framework addresses these issues through an IDE-integrated approach that handles class schema evolution by managing versions of the code and generating transformation functions automatically. The infrastructure also enforces class invariants to prevent the introduction of potentially corrupt objects. This article describes a model for class attribute changes, a measure for class evolution robustness, four empirical studies, and the design and implementation of the ESCHER system.
△ Less
Submitted 21 June, 2012; v1 submitted 3 March, 2011;
originally announced March 2011.
-
The Business of Selling Electronic Documents
Authors:
Manuel Oriol
Abstract:
The music industry has huge troubles adapting to the new technologies. As many pointed out, when copying music is essentially free and socially accepted it becomes increasingly tempting for users to infringe copyrights and copy music from one person to another. The answer of the music industry is to outlaw a majority of citizens. This article describes how the music industry should reinvent itse…
▽ More
The music industry has huge troubles adapting to the new technologies. As many pointed out, when copying music is essentially free and socially accepted it becomes increasingly tempting for users to infringe copyrights and copy music from one person to another. The answer of the music industry is to outlaw a majority of citizens. This article describes how the music industry should reinvent itself and adapt to a world where the network is ubiquitous and exchanging information is essentially free. It relies on adapting prices to the demand and lower costs of electronic documents in a dramatic way.
△ Less
Submitted 21 April, 2009;
originally announced April 2009.