-
Improving Autoformalization using Type Checking
Authors:
Auguste Poiroux,
Gail Weiss,
Viktor Kunčak,
Antoine Bosselut
Abstract:
Large language models show promise for autoformalization, the task of automatically translating natural language into formal languages. However, current autoformalization methods remain limited. The last reported state-of-the-art performance on the ProofNet formalization benchmark for the Lean proof assistant, achieved using Codex for Lean 3, only showed successful formalization of 16.1% of inform…
▽ More
Large language models show promise for autoformalization, the task of automatically translating natural language into formal languages. However, current autoformalization methods remain limited. The last reported state-of-the-art performance on the ProofNet formalization benchmark for the Lean proof assistant, achieved using Codex for Lean 3, only showed successful formalization of 16.1% of informal statements. Similarly, our evaluation of GPT-4o for Lean 4 only produces successful translations 34.9% of the time. Our analysis shows that the performance of these models is largely limited by their inability to generate formal statements that successfully type-check (i.e., are syntactically correct and consistent with types) - with a whop** 86.6% of GPT-4o errors starting from a type-check failure. In this work, we propose a method to fix this issue through decoding with type-check filtering, where we initially sample a diverse set of candidate formalizations for an informal statement, then use the Lean proof assistant to filter out candidates that do not type-check. Using GPT-4o as a base model, and combining our method with self-consistency, we obtain a +18.3% absolute increase in formalization accuracy, and achieve a new state-of-the-art of 53.2% on ProofNet with Lean 4.
△ Less
Submitted 11 June, 2024;
originally announced June 2024.
-
Mechanized HOL Reasoning in Set Theory
Authors:
Simon Guilloud,
Sankalp Gambhir,
Andrea Gilot,
Viktor Kunčak
Abstract:
We present a mechanized embedding of higher-order logic (HOL) and algebraic data types (ADT) into first-order logic with ZFC axioms. We implement this in the Lisa proof assistant for schematic first-order logic and its library based on axiomatic set theory. HOL proof steps are implemented as proof producing tactics in Lisa, and the types are interpreted as sets, with function (or arrow) types coin…
▽ More
We present a mechanized embedding of higher-order logic (HOL) and algebraic data types (ADT) into first-order logic with ZFC axioms. We implement this in the Lisa proof assistant for schematic first-order logic and its library based on axiomatic set theory. HOL proof steps are implemented as proof producing tactics in Lisa, and the types are interpreted as sets, with function (or arrow) types coinciding with set-theoretic function spaces. The embedded HOL proofs, as opposed to being a layer over the existing proofs, are interoperable with the existing library. This yields a form of soft type system supporting top-level polymorphism and ADTs over set theory, and offer tools to reason about functions in set theory.
△ Less
Submitted 20 March, 2024;
originally announced March 2024.
-
Orthologic with Axioms
Authors:
Simon Guilloud,
Viktor Kuncak
Abstract:
We study the proof theory and algorithms for orthologic, a logical system based on ortholattices, which have shown practical relevance in simplification and normalization of verification conditions. Ortholattices weaken Boolean algebras while having polynomial-time equivalence checking that is sound with respect to Boolean algebra semantics. We generalize ortholattice reasoning and obtain an algor…
▽ More
We study the proof theory and algorithms for orthologic, a logical system based on ortholattices, which have shown practical relevance in simplification and normalization of verification conditions. Ortholattices weaken Boolean algebras while having polynomial-time equivalence checking that is sound with respect to Boolean algebra semantics. We generalize ortholattice reasoning and obtain an algorithm for proving a larger class of classically valid formulas.
As the key result, we analyze a proof system for orthologic augmented with axioms. An important feature of the system is that it limits the number of formulas in a sequent to at most two, which makes the extension with axioms non-trivial. We show a generalized form of cut elimination for this system, which implies a sub-formula property. From there we derive a cubic-time algorithm for provability from axioms, or equivalently, for validity in finitely presented ortholattices. We further show that propositional resolution of width 5 proves all formulas provable in orthologic with axioms. We show that orthologic system subsumes resolution of width 2 and arbitrarily wide unit resolution and is complete for reasoning about generalizations of propositional Horn clauses.
Moving beyond ground axioms, we introduce effectively propositional orthologic, presenting its semantics as well as a sound and complete proof system. Our proof system implies the decidability of effectively propositional orthologic, as well as its fixed-parameter tractability for a bounded maximal number of variables in each axiom. As a special case, we obtain a generalization of Datalog with negation and disjunction.
△ Less
Submitted 7 December, 2023; v1 submitted 14 July, 2023;
originally announced July 2023.
-
NP Decision Procedure for Monomial and Linear Integer Constraints
Authors:
Rodrigo Raya,
Jad Hamza,
Viktor Kunčak
Abstract:
Motivated by satisfiability of constraints with function symbols, we consider numerical inequalities on non-negative integers. The constraints we consider are a conjunction of a linear system Ax = b and a conjunction of (non-)convex constraints of the form x_i >= x_j^n (x_i <= x_j^n). We show that the satisfiability of these constraints is NP-complete even if the solution to the linear part is giv…
▽ More
Motivated by satisfiability of constraints with function symbols, we consider numerical inequalities on non-negative integers. The constraints we consider are a conjunction of a linear system Ax = b and a conjunction of (non-)convex constraints of the form x_i >= x_j^n (x_i <= x_j^n). We show that the satisfiability of these constraints is NP-complete even if the solution to the linear part is given explicitly. As a consequence, we obtain NP completeness for an extension of certain quantifier-free constraints on sets with cardinalities (QFBAPA) with function images S = f[P^n].
△ Less
Submitted 20 October, 2022; v1 submitted 4 August, 2022;
originally announced August 2022.
-
On Equivalence Checking for Orthocomplemented Bisemilattices in Log-Linear Time
Authors:
Simon Guilloud,
Viktor Kunčak
Abstract:
We present a quasilinear time algorithm to decide the word problem on a natural algebraic structures we call orthocomplemented bisemilattices, a subtheory of boolean algebra. We use as a base a variation of Hopcroft, Ullman and Aho algorithm for tree isomorphism which we combine with a term rewriting system to decide equivalence of two terms. We prove that the rewriting system is terminating and c…
▽ More
We present a quasilinear time algorithm to decide the word problem on a natural algebraic structures we call orthocomplemented bisemilattices, a subtheory of boolean algebra. We use as a base a variation of Hopcroft, Ullman and Aho algorithm for tree isomorphism which we combine with a term rewriting system to decide equivalence of two terms. We prove that the rewriting system is terminating and confluent and hence the existence of a normal form, and that our algorithm is computing it. We also discuss applications and present an effective implementation in Scala.
△ Less
Submitted 7 March, 2022; v1 submitted 7 October, 2021;
originally announced October 2021.
-
NP Satisfiability for Arrays as Powers
Authors:
Rodrigo Raya,
Viktor Kunčak
Abstract:
We show that the satisfiability problem for the quantifier-free theory of product structures with the equicardinality relation is in NP. As an application, we extend the combinatory array logic fragment to handle cardinality constraints. The resulting fragment is independent of the base element and index set theories.
We show that the satisfiability problem for the quantifier-free theory of product structures with the equicardinality relation is in NP. As an application, we extend the combinatory array logic fragment to handle cardinality constraints. The resulting fragment is independent of the base element and index set theories.
△ Less
Submitted 11 September, 2021;
originally announced September 2021.
-
Verifying a Realistic Mutable Hash Table
Authors:
Samuel Chassot,
Viktor Kunčak
Abstract:
In this work, we verify the mutable LongMap from the Scala standard library, a hash table using open addressing within a single array, using the Stainless program verifier. As a reference implementation, we write an immutable map based on a list of tuples. We then show that LongMap's operations correspond to operations of this association list. To express the resizing of the hash table array, we i…
▽ More
In this work, we verify the mutable LongMap from the Scala standard library, a hash table using open addressing within a single array, using the Stainless program verifier. As a reference implementation, we write an immutable map based on a list of tuples. We then show that LongMap's operations correspond to operations of this association list. To express the resizing of the hash table array, we introduce a new reference swap** construct in Stainless. This allows us to apply the decorator pattern without introducing aliasing. Our verification effort led us to find and fix a bug in the original implementation that manifests for large hash tables. Our performance analysis shows the verified version to be within a 1.5 factor of the original data structure.
△ Less
Submitted 30 January, 2024; v1 submitted 16 July, 2021;
originally announced July 2021.
-
Proving and Disproving Programs with Shared Mutable Data
Authors:
Georg Schmid,
Viktor Kunčak
Abstract:
We present a tool for verification of deterministic programs with shared mutable references against specifications such as assertions, preconditions, postconditions, and read/write effects. We implement our tool by encoding programs with mutable references into annotated purely functional recursive programs. We then rely on function unfolding and the SMT solver Z3 to prove or disprove safety and t…
▽ More
We present a tool for verification of deterministic programs with shared mutable references against specifications such as assertions, preconditions, postconditions, and read/write effects. We implement our tool by encoding programs with mutable references into annotated purely functional recursive programs. We then rely on function unfolding and the SMT solver Z3 to prove or disprove safety and to establish program termination. Our tool uses a new translation of programs where frame conditions are encoded using quantifier-free formulas in first-order logic (instead of relying on quantifiers or separation logic). This quantifier-free encoding enables SMT solvers to prove safety or report counterexamples relative to the semantics of procedure specifications. Our encoding is possible thanks to the expressive power of the extended array theory of the Z3 SMT solver. In addition to the ability to report counterexamples, our tool retains efficiency of reasoning about purely functional layers of data structures, providing expressiveness for mutable data but also a significant level of automation for purely functional aspects of software. We illustrate our tool through examples manipulating mutable linked structures and arrays.
△ Less
Submitted 13 March, 2021;
originally announced March 2021.
-
Coming to Terms with Your Choices: An Existential Take on Dependent Types
Authors:
Georg Stefan Schmid,
Olivier Blanvillain,
Jad Hamza,
Viktor Kunčak
Abstract:
Type-level programming is an increasingly popular way to obtain additional type safety. Unfortunately, it remains a second-class citizen in the majority of industrially-used programming languages. We propose a new dependently-typed system with subty** and singleton types whose goal is to enable type-level programming in an accessible style. At the heart of our system lies a non-deterministic cho…
▽ More
Type-level programming is an increasingly popular way to obtain additional type safety. Unfortunately, it remains a second-class citizen in the majority of industrially-used programming languages. We propose a new dependently-typed system with subty** and singleton types whose goal is to enable type-level programming in an accessible style. At the heart of our system lies a non-deterministic choice operator. We argue that embracing non-determinism is crucial for bringing dependent types to a broader audience of programmers, since real-world programs will inevitably interact with imprecisely-typed, or even impure code. Furthermore, we show that singleton types combined with the choice operator can serve as a replacement for many type functions of interest in practice. We establish the soundness of our approach using the Coq proof assistant. Our soundness approach models non-determinism using additional function arguments to represent choices. We represent type-level computation using singleton types and existential types that quantify over choice arguments. To demonstrate the practicality of our type system, we present an implementation as a modification of the Scala compiler. We provide a case study in which we develop a strongly-typed wrapper for Spark datasets.
△ Less
Submitted 15 November, 2020;
originally announced November 2020.
-
LL(1) Parsing with Derivatives and Zippers
Authors:
Romain Edelmann,
Jad Hamza,
Viktor Kunčak
Abstract:
In this paper, we present an efficient, functional, and formally verified parsing algorithm for LL(1) context-free expressions based on the concept of derivatives of formal languages. Parsing with derivatives is an elegant parsing technique, which, in the general case, suffers from cubic worst-case time complexity and slow performance in practice. We specialise the parsing with derivatives algorit…
▽ More
In this paper, we present an efficient, functional, and formally verified parsing algorithm for LL(1) context-free expressions based on the concept of derivatives of formal languages. Parsing with derivatives is an elegant parsing technique, which, in the general case, suffers from cubic worst-case time complexity and slow performance in practice. We specialise the parsing with derivatives algorithm to LL(1) context-free expressions, where alternatives can be chosen given a single token of lookahead. We formalise the notion of LL(1) expressions and show how to efficiently check the LL(1) property. Next, we present a novel linear-time parsing with derivatives algorithm for LL(1) expressions operating on a zipper-inspired data structure. We prove the algorithm correct in Coq and present an implementation as a parser combinators framework in Scala, with enumeration and pretty printing capabilities.
△ Less
Submitted 21 January, 2021; v1 submitted 28 November, 2019;
originally announced November 2019.
-
System FR as Foundations for Stainless
Authors:
Jad Hamza,
Nicolas Voirol,
Viktor Kunčak
Abstract:
We present the design, implementation, and foundation of a verifier for higher-order functional programs with generics and recursive data types. Our system supports proving safety and termination using preconditions, postconditions and assertions. It supports writing proof hints using assertions and recursive calls. To formalize the soundness of the system we introduce System FR, a calculus suppor…
▽ More
We present the design, implementation, and foundation of a verifier for higher-order functional programs with generics and recursive data types. Our system supports proving safety and termination using preconditions, postconditions and assertions. It supports writing proof hints using assertions and recursive calls. To formalize the soundness of the system we introduce System FR, a calculus supporting System F polymorphism, dependent refinement types, and recursive types (including recursion through contravariant positions of function types). Through the use of sized types, System FR supports reasoning about termination of lazy data structures such as streams. We formalize a reducibility argument using the Coq proof assistant and prove the soundness of a type-checker with respect to call-by-value semantics, ensuring type safety and normalization for typeable programs. Our program verifier is implemented as an alternative verification-condition generator for the Stainless tool, which relies on existing SMT-based solver backend for automation. We demonstrate the efficiency of our approach by verifying a collection of higher-order functional programs comprising around 14000 lines of polymorphic higher-order Scala code, including graph search algorithms, basic number theory, monad laws, functional data structures, and assignments from popular Functional Programming MOOCs.
△ Less
Submitted 24 March, 2020; v1 submitted 6 April, 2019;
originally announced April 2019.
-
Identifying Maximal Non-Redundant Integer Cone Generators
Authors:
Slobodan Mitrović,
Ruzica Piskac,
Viktor Kunčak
Abstract:
A non-redundant integer cone generator (NICG) of dimension $d$ is a set $S$ of vectors from $\{0,1\}^d$ whose vector sum cannot be generated as a positive integer linear combination of a proper subset of $S$. The largest possible cardinality of NICG of a dimension $d$, denoted by $N(d)$, provides an upper bound on the sparsity of systems of integer equations with a large number of integer variable…
▽ More
A non-redundant integer cone generator (NICG) of dimension $d$ is a set $S$ of vectors from $\{0,1\}^d$ whose vector sum cannot be generated as a positive integer linear combination of a proper subset of $S$. The largest possible cardinality of NICG of a dimension $d$, denoted by $N(d)$, provides an upper bound on the sparsity of systems of integer equations with a large number of integer variables. A better estimate of $N(d)$ means that we can consider smaller sub-systems of integer equations when solving systems with many integer variables. Furthermore, given that we can reduce constraints on set algebra expressions to constraints on cardinalities of Venn regions, tighter upper bound on $N(d)$ yields a more efficient decision procedure for a logic of sets with cardinality constraints (BAPA), which has been applied in software verification. Previous attempts to compute $N(d)$ using SAT solvers have not succeeded even for $d=3$. The only known values were computed manually: $N(d)=d$ for $d < 4$ and $N(4) > 4$. We provide the first exact values for $d > 3$, namely, $N(4)=5$, $N(5)=7$, and $N(6)=9$, which is a significant improvement of the known asymptotic bound (which would give only e.g. $N(6) \le 29$, making a decision procedure impractical for $d=6$). We also give lower bounds for $N(7)$, $N(8)$, $N(9)$, and $N(10)$, which are: $11$, $13$, $14$, and $16$, respectively. We describe increasingly sophisticated specialized search algorithms that we used to explore the space of non-redundant generators and obtain these results.
△ Less
Submitted 20 March, 2019;
originally announced March 2019.
-
Neural-Network Guided Expression Transformation
Authors:
Romain Edelmann,
Viktor Kunčak
Abstract:
Optimizing compilers, as well as other translator systems, often work by rewriting expressions according to equivalence preserving rules. Given an input expression and its optimized form, finding the sequence of rules that were applied is a non-trivial task. Most of the time, the tools provide no proof, of any kind, of the equivalence between the original expression and its optimized form. In this…
▽ More
Optimizing compilers, as well as other translator systems, often work by rewriting expressions according to equivalence preserving rules. Given an input expression and its optimized form, finding the sequence of rules that were applied is a non-trivial task. Most of the time, the tools provide no proof, of any kind, of the equivalence between the original expression and its optimized form. In this work, we propose to reconstruct proofs of equivalence of simple mathematical expressions, after the fact, by finding paths of equivalence preserving transformations between expressions. We propose to find those sequences of transformations using a search algorithm, guided by a neural network heuristic. Using a Tree-LSTM recursive neural network, we learn a distributed representation of expressions where the Manhattan distance between vectors approximately corresponds to the rewrite distance between expressions. We then show how the neural network can be efficiently used to search for transformation paths, leading to substantial gain in speed compared to an uninformed exhaustive search. In one of our experiments, our neural-network guided search algorithm is able to solve more instances with a 2 seconds timeout per instance than breadth-first search does with a 5 minutes timeout per instance.
△ Less
Submitted 6 February, 2019;
originally announced February 2019.
-
Bidirectional Evaluation with Direct Manipulation
Authors:
Mikaël Mayer,
Viktor Kunčak,
Ravi Chugh
Abstract:
We present an evaluation update (or simply, update) algorithm for a full-featured functional programming language, which synthesizes program changes based on output changes. Intuitively, the update algorithm retraces the steps of the original evaluation, rewriting the program as needed to reconcile differences between the original and updated output values. Our approach, furthermore, allows expert…
▽ More
We present an evaluation update (or simply, update) algorithm for a full-featured functional programming language, which synthesizes program changes based on output changes. Intuitively, the update algorithm retraces the steps of the original evaluation, rewriting the program as needed to reconcile differences between the original and updated output values. Our approach, furthermore, allows expert users to define custom lenses that augment the update algorithm with more advanced or domain-specific program updates.
To demonstrate the utility of evaluation update, we implement the algorithm in Sketch-n-Sketch, a novel direct manipulation programming system for generating HTML documents. In Sketch-n-Sketch, the user writes an ML-style functional program to generate HTML output. When the user directly manipulates the output using a graphical user interface, the update algorithm reconciles the changes. We evaluate bidirectional evaluation in Sketch-n-Sketch by authoring ten examples comprising approximately 1400 lines of code in total. These examples demonstrate how a variety of HTML documents and applications can be developed and edited interactively in Sketch-n-Sketch, mitigating the tedious edit-run-view cycle in traditional programming environments.
△ Less
Submitted 18 October, 2018; v1 submitted 11 September, 2018;
originally announced September 2018.
-
Minimal Synthesis of String To String Functions From Examples
Authors:
Jad Hamza,
Viktor Kunčak
Abstract:
We study the problem of synthesizing string to string transformations from a set of input/output examples. The transformations we consider are expressed using deterministic finite automata (DFA) that read pairs of letters, one letter from the input and one from the output. The DFA corresponding to these transformations have additional constraints, ensuring that each input string is mapped to exact…
▽ More
We study the problem of synthesizing string to string transformations from a set of input/output examples. The transformations we consider are expressed using deterministic finite automata (DFA) that read pairs of letters, one letter from the input and one from the output. The DFA corresponding to these transformations have additional constraints, ensuring that each input string is mapped to exactly one output string.
We suggest that, given a set of input/output examples, the smallest DFA consistent with the examples is a good candidate for the transformation the user was expecting. We therefore study the problem of, given a set of examples, finding a minimal DFA consistent with the examples and satisfying the functionality and totality constraints mentioned above.
We prove that, in general, this problem (the corresponding decision problem) is NP-complete. This is unlike the standard DFA minimization problem which can be solved in polynomial time. We provide several NP-hardness proofs that show the hardness of multiple (independent) variants of the problem.
Finally, we propose an algorithm for finding the minimal DFA consistent with input/output examples, that uses a reduction to SMT solvers. We implemented the algorithm, and used it to evaluate the likelihood that the minimal DFA indeed corresponds to the DFA expected by the user.
△ Less
Submitted 4 June, 2018; v1 submitted 25 October, 2017;
originally announced October 2017.
-
On Repair with Probabilistic Attribute Grammars
Authors:
Manos Koukoutos,
Mukund Raghothaman,
Etienne Kneuss,
Viktor Kuncak
Abstract:
Program synthesis and repair have emerged as an exciting area of research, driven by the potential for revolutionary advances in programmer productivity. Among most promising ideas emerging for synthesis are syntax-driven search, probabilistic models of code, and the use of input-output examples. Our paper shows how to combine these techniques and use them for program repair, which is among the mo…
▽ More
Program synthesis and repair have emerged as an exciting area of research, driven by the potential for revolutionary advances in programmer productivity. Among most promising ideas emerging for synthesis are syntax-driven search, probabilistic models of code, and the use of input-output examples. Our paper shows how to combine these techniques and use them for program repair, which is among the most relevant applications of synthesis to general-purpose code. Our approach combines semantic specifications, in the form of pre- and post-conditions and input-output examples with syntactic specifications in the form of term grammars and AST-level statistics extracted from code corpora. We show that synthesis in this framework can be viewed as an instance of graph search, permitting the use of well-understood families of techniques such as A*. We implement our algorithm in a framework for verification, synthesis and repair of functional programs, demonstrating that our approach can repair programs that are beyond the reach of previous tools.
△ Less
Submitted 13 July, 2017;
originally announced July 2017.
-
Polynomial-Time Proactive Synthesis of Tree-to-String Functions from Examples
Authors:
Mikaël Mayer,
Jad Hamza,
Viktor Kuncak
Abstract:
Synthesis from examples enables non-expert users to generate programs by specifying examples of their behavior. A domain-specific form of such synthesis has been recently deployed in a widely used spreadsheet software product. In this paper we contribute to foundations of such techniques and present a complete algorithm for synthesis of a class of recursive functions defined by structural recursio…
▽ More
Synthesis from examples enables non-expert users to generate programs by specifying examples of their behavior. A domain-specific form of such synthesis has been recently deployed in a widely used spreadsheet software product. In this paper we contribute to foundations of such techniques and present a complete algorithm for synthesis of a class of recursive functions defined by structural recursion over a given algebraic data type definition. The functions we consider map an algebraic data type to a string; they are useful for, e.g., pretty printing and serialization of programs and data. We formalize our problem as learning deterministic sequential top-down tree-to-string transducers with a single state.
The first problem we consider is learning a tree-to-string transducer from any set of input/output examples provided by the user. We show that this problem is NP-complete in general, but can be solved in polynomial time under a (practically useful) closure condition that each subtree of a tree in the input/output example set is also part of the input/output examples.
Because coming up with relevant input/output examples may be difficult for the user while creating hard constraint problems for the synthesizer, we also study a more automated active learning scenario in which the algorithm chooses the inputs for which the user provides the outputs. Our algorithm asks a worst-case linear number of queries as a function of the size of the algebraic data type definition to determine a unique transducer.
△ Less
Submitted 24 May, 2017; v1 submitted 16 January, 2017;
originally announced January 2017.
-
An Update on Deductive Synthesis and Repair in the Leon Tool
Authors:
Manos Koukoutos,
Etienne Kneuss,
Viktor Kuncak
Abstract:
We report our progress in scaling deductive synthesis and repair of recursive functional Scala programs in the Leon tool. We describe new techniques, including a more precise mechanism for encoding the space of meaningful candidate programs. Our techniques increase the scope of synthesis by expanding the space of programs we can synthesize and by reducing the synthesis time in many cases. As a new…
▽ More
We report our progress in scaling deductive synthesis and repair of recursive functional Scala programs in the Leon tool. We describe new techniques, including a more precise mechanism for encoding the space of meaningful candidate programs. Our techniques increase the scope of synthesis by expanding the space of programs we can synthesize and by reducing the synthesis time in many cases. As a new example, we present a run-length encoding function for a list of values, which Leon can now automatically synthesize from specification consisting of the decoding function and the local minimality property of the encoded value.
△ Less
Submitted 22 November, 2016;
originally announced November 2016.
-
Translating Scala Programs to Isabelle/HOL
Authors:
Lars Hupel,
Viktor Kuncak
Abstract:
We present a trustworthy connection between the Leon verification system and the Isabelle proof assistant. Leon is a system for verifying functional Scala programs. It uses a variety of automated theorem provers (ATPs) to check verification conditions (VCs) stemming from the input program. Isabelle, on the other hand, is an interactive theorem prover used to verify mathematical specifications usin…
▽ More
We present a trustworthy connection between the Leon verification system and the Isabelle proof assistant. Leon is a system for verifying functional Scala programs. It uses a variety of automated theorem provers (ATPs) to check verification conditions (VCs) stemming from the input program. Isabelle, on the other hand, is an interactive theorem prover used to verify mathematical specifications using its own input language Isabelle/Isar. Users specify (inductive) definitions and write proofs about them manually, albeit with the help of semi-automated tactics. The integration of these two systems allows us to exploit Isabelle's rich standard library and give greater confidence guarantees in the correctness of analysed programs.
△ Less
Submitted 6 July, 2016;
originally announced July 2016.
-
Proceedings Fourth Workshop on Synthesis
Authors:
Pavol Černý,
Viktor Kuncak,
Madhusudan Parthasarathy
Abstract:
The SYNT workshop aims to bring together researchers interested in the broad area of synthesis of computing systems. The goal is to foster the development of frontier techniques in automating the development of computing system. Contributions of interest include algorithms, complexity and decidability analysis, as well as reproducible heuristics, implemented tools, and experimental evaluation. Ap…
▽ More
The SYNT workshop aims to bring together researchers interested in the broad area of synthesis of computing systems. The goal is to foster the development of frontier techniques in automating the development of computing system. Contributions of interest include algorithms, complexity and decidability analysis, as well as reproducible heuristics, implemented tools, and experimental evaluation. Application domains include software, hardware, embedded, and cyberphysical systems. Computation models include functional, reactive, hybrid and timed systems. Identifying, formalizing, and evaluating synthesis in particular application domains is encouraged.
The fourth iteration of the workshop took place in San Francisco, CA, USA. It was co-located with the 27th International Conference on Computer Aided Verification. The workshop included five contributed talks and two invited talks. In addition, it featured a special session about the Syntax-Guided Synthesis Competition (SyGuS) and the SyntComp Synthesis competition.
△ Less
Submitted 1 February, 2016;
originally announced February 2016.
-
An Instantiation-Based Approach for Solving Quantified Linear Arithmetic
Authors:
Andrew Reynolds,
Tim King,
Viktor Kuncak
Abstract:
This paper presents a framework to derive instantiation-based decision procedures for satisfiability of quantified formulas in first-order theories, including its correctness, implementation, and evaluation. Using this framework we derive decision procedures for linear real arithmetic (LRA) and linear integer arithmetic (LIA) formulas with one quantifier alternation. Our procedure can be integrate…
▽ More
This paper presents a framework to derive instantiation-based decision procedures for satisfiability of quantified formulas in first-order theories, including its correctness, implementation, and evaluation. Using this framework we derive decision procedures for linear real arithmetic (LRA) and linear integer arithmetic (LIA) formulas with one quantifier alternation. Our procedure can be integrated into the solving architecture used by typical SMT solvers. Experimental results on standardized benchmarks from model checking, static analysis, and synthesis show that our implementation of the procedure in the SMT solver CVC4 outperforms existing tools for quantified linear arithmetic.
△ Less
Submitted 11 February, 2016; v1 submitted 9 October, 2015;
originally announced October 2015.
-
On Counterexample Guided Quantifier Instantiation for Synthesis in CVC4
Authors:
Andrew Reynolds,
Morgan Deters,
Viktor Kuncak,
Cesare Tinelli,
Clark Barrett
Abstract:
We introduce the first program synthesis engine implemented inside an SMT solver. We present an approach that extracts solution functions from unsatisfiability proofs of the negated form of synthesis conjectures. We also discuss novel counterexample-guided techniques for quantifier instantiation that we use to make finding such proofs practically feasible. A particularly important class of specifi…
▽ More
We introduce the first program synthesis engine implemented inside an SMT solver. We present an approach that extracts solution functions from unsatisfiability proofs of the negated form of synthesis conjectures. We also discuss novel counterexample-guided techniques for quantifier instantiation that we use to make finding such proofs practically feasible. A particularly important class of specifications are single-invocation properties, for which we present a dedicated algorithm. To support syntax restrictions on generated solutions, our approach can transform a solution found without restrictions into the desired syntactic form. As an alternative, we show how to use evaluation function axioms to embed syntactic restrictions into constraints over algebraic datatypes, and then use an algebraic datatype decision procedure to drive synthesis. Our experimental evaluation on syntax-guided synthesis benchmarks shows that our implementation in the CVC4 SMT solver is competitive with state-of-the-art tools for synthesis.
△ Less
Submitted 23 June, 2015; v1 submitted 16 February, 2015;
originally announced February 2015.
-
Towards a Compiler for Reals
Authors:
Eva Darulova,
Viktor Kuncak
Abstract:
Numerical software, common in scientific computing or embedded systems, inevitably uses an approximation of the real arithmetic in which most algorithms are designed. In many domains, roundoff errors are not the only source of inaccuracy and measurement and other input errors further increase the uncertainty of the computed results. Adequate tools are needed to help users select suitable approxima…
▽ More
Numerical software, common in scientific computing or embedded systems, inevitably uses an approximation of the real arithmetic in which most algorithms are designed. In many domains, roundoff errors are not the only source of inaccuracy and measurement and other input errors further increase the uncertainty of the computed results. Adequate tools are needed to help users select suitable approximations, especially for safety-critical applications.
We present the source-to-source compiler Rosa which takes as input a real-valued program with error specifications and synthesizes code over an appropriate floating-point or fixed-point data type. The main challenge of such a compiler is a fully automated, sound and yet accurate enough numerical error estimation. We present a unified technique for floating-point and fixed-point arithmetic of various precisions which can handle nonlinear arithmetic, determine closed- form symbolic invariants for unbounded loops and quantify the effects of discontinuities on numerical errors. We evaluate Rosa on a number of benchmarks from scientific computing and embedded systems and, comparing it to state-of-the-art in automated error estimation, show it presents an interesting trade-off between accuracy and performance.
△ Less
Submitted 11 March, 2016; v1 submitted 1 October, 2014;
originally announced October 2014.
-
On Sound Compilation of Reals
Authors:
Eva Darulova,
Viktor Kuncak
Abstract:
Writing accurate numerical software is hard because of many sources of unavoidable uncertainties, including finite numerical precision of implementations. We present a programming model where the user writes a program in a real-valued implementation and specification language that explicitly includes different types of uncertainties. We then present a compilation algorithm that generates a convent…
▽ More
Writing accurate numerical software is hard because of many sources of unavoidable uncertainties, including finite numerical precision of implementations. We present a programming model where the user writes a program in a real-valued implementation and specification language that explicitly includes different types of uncertainties. We then present a compilation algorithm that generates a conventional implementation that is guaranteed to meet the desired precision with respect to real numbers. Our verification step generates verification conditions that treat different uncertainties in a unified way and encode reasoning about floating-point roundoff errors into reasoning about real numbers. Such verification conditions can be used as a standardized format for verifying the precision and the correctness of numerical programs. Due to their often non-linear nature, precise reasoning about such verification conditions remains difficult. We show that current state-of-the art SMT solvers do not scale well to solving such verification conditions. We propose a new procedure that combines exact SMT solving over reals with approximate and sound affine and interval arithmetic. We show that this approach overcomes scalability limitations of SMT solvers while providing improved precision over affine and interval arithmetic. Using our initial implementation we show the usefullness and effectiveness of our approach on several examples, including those containing non-linear computation.
△ Less
Submitted 10 September, 2013;
originally announced September 2013.
-
On Integrating Deductive Synthesis and Verification Systems
Authors:
Etienne Kneuss,
Viktor Kuncak,
Ivan Kuraj,
Philippe Suter
Abstract:
We describe techniques for synthesis and verification of recursive functional programs over unbounded domains. Our techniques build on top of an algorithm for satisfiability modulo recursive functions, a framework for deductive synthesis, and complete synthesis procedures for algebraic data types. We present new counterexample-guided algorithms for constructing verified programs. We have implement…
▽ More
We describe techniques for synthesis and verification of recursive functional programs over unbounded domains. Our techniques build on top of an algorithm for satisfiability modulo recursive functions, a framework for deductive synthesis, and complete synthesis procedures for algebraic data types. We present new counterexample-guided algorithms for constructing verified programs. We have implemented these algorithms in an integrated environment for interactive verification and synthesis from relational specifications. Our system was able to synthesize a number of useful recursive functions that manipulate unbounded numbers and data structures.
△ Less
Submitted 20 April, 2013;
originally announced April 2013.
-
The Relationship between Craig Interpolation and Recursion-Free Horn Clauses
Authors:
Philipp Rümmer,
Hossein Hojjat,
Viktor Kuncak
Abstract:
Despite decades of research, there are still a number of concepts commonly found in software programs that are considered challenging for verification: among others, such concepts include concurrency, and the compositional analysis of programs with procedures. As a promising direction to overcome such difficulties, recently the use of Horn constraints as intermediate representation of software pro…
▽ More
Despite decades of research, there are still a number of concepts commonly found in software programs that are considered challenging for verification: among others, such concepts include concurrency, and the compositional analysis of programs with procedures. As a promising direction to overcome such difficulties, recently the use of Horn constraints as intermediate representation of software programs has been proposed. Horn constraints are related to Craig interpolation, which is one of the main techniques used to construct and refine abstractions in verification, and to synthesise inductive loop invariants. We give a survey of the different forms of Craig interpolation found in literature, and show that all of them correspond to natural fragments of (recursion-free) Horn constraints. We also discuss techniques for solving systems of recursion-free Horn constraints.
△ Less
Submitted 18 February, 2013;
originally announced February 2013.
-
Disjunctive Interpolants for Horn-Clause Verification (Extended Technical Report)
Authors:
Philipp Rümmer,
Hossein Hojjat,
Viktor Kuncak
Abstract:
One of the main challenges in software verification is efficient and precise compositional analysis of programs with procedures and loops. Interpolation methods remain one of the most promising techniques for such verification, and are closely related to solving Horn clause constraints. We introduce a new notion of interpolation, disjunctive interpolation, which solve a more general class of probl…
▽ More
One of the main challenges in software verification is efficient and precise compositional analysis of programs with procedures and loops. Interpolation methods remain one of the most promising techniques for such verification, and are closely related to solving Horn clause constraints. We introduce a new notion of interpolation, disjunctive interpolation, which solve a more general class of problems in one step compared to previous notions of interpolants, such as tree interpolants or inductive sequences of interpolants. We present algorithms and complexity for construction of disjunctive interpolants, as well as their use within an abstraction-refinement loop. We have implemented Horn clause verification algorithms that use disjunctive interpolants and evaluate them on benchmarks expressed as Horn clauses over the theory of integer linear arithmetic.
△ Less
Submitted 21 January, 2013;
originally announced January 2013.
-
Software Verification and Graph Similarity for Automated Evaluation of Students' Assignments
Authors:
Milena Vujosevic-Janicic,
Mladen Nikolic,
Dusan Tosic,
Viktor Kuncak
Abstract:
In this paper we promote introducing software verification and control flow graph similarity measurement in automated evaluation of students' programs. We present a new grading framework that merges results obtained by combination of these two approaches with results obtained by automated testing, leading to improved quality and precision of automated grading. These two approaches are also useful…
▽ More
In this paper we promote introducing software verification and control flow graph similarity measurement in automated evaluation of students' programs. We present a new grading framework that merges results obtained by combination of these two approaches with results obtained by automated testing, leading to improved quality and precision of automated grading. These two approaches are also useful in providing a comprehensible feedback that can help students to improve the quality of their programs We also present our corresponding tools that are publicly available and open source. The tools are based on LLVM low-level intermediate code representation, so they could be applied to a number of programming languages. Experimental evaluation of the proposed grading framework is performed on a corpus of university students' programs written in programming language C. Results of the experiments show that automatically generated grades are highly correlated with manually determined grades suggesting that the presented tools can find real-world applications in studying and grading.
△ Less
Submitted 29 June, 2012;
originally announced June 2012.
-
On Verifying Complex Properties using Symbolic Shape Analysis
Authors:
Thomas Wies,
Viktor Kuncak,
Karen Zee,
Andreas Podelski,
Martin Rinard
Abstract:
One of the main challenges in the verification of software systems is the analysis of unbounded data structures with dynamic memory allocation, such as linked data structures and arrays. We describe Bohne, a new analysis for verifying data structures. Bohne verifies data structure operations and shows that 1) the operations preserve data structure invariants and 2) the operations satisfy their s…
▽ More
One of the main challenges in the verification of software systems is the analysis of unbounded data structures with dynamic memory allocation, such as linked data structures and arrays. We describe Bohne, a new analysis for verifying data structures. Bohne verifies data structure operations and shows that 1) the operations preserve data structure invariants and 2) the operations satisfy their specifications expressed in terms of changes to the set of objects stored in the data structure. During the analysis, Bohne infers loop invariants in the form of disjunctions of universally quantified Boolean combinations of formulas. To synthesize loop invariants of this form, Bohne uses a combination of decision procedures for Monadic Second-Order Logic over trees, SMT-LIB decision procedures (currently CVC Lite), and an automated reasoner within the Isabelle interactive theorem prover. This architecture shows that synthesized loop invariants can serve as a useful communication mechanism between different decision procedures. Using Bohne, we have verified operations on data structures such as linked lists with iterators and back pointers, trees with and without parent pointers, two-level skip lists, array data structures, and sorted lists. We have deployed Bohne in the Hob and Jahob data structure analysis systems, enabling us to combine Bohne with analyses of data structure clients and apply it in the context of larger programs. This report describes the Bohne algorithm as well as techniques that Bohne uses to reduce the ammount of annotations and the running time of the analysis.
△ Less
Submitted 18 September, 2006;
originally announced September 2006.
-
On Algorithms and Complexity for Sets with Cardinality Constraints
Authors:
Bruno Marnette,
Viktor Kuncak,
Martin Rinard
Abstract:
Typestate systems ensure many desirable properties of imperative programs, including initialization of object fields and correct use of stateful library interfaces. Abstract sets with cardinality constraints naturally generalize typestate properties: relationships between the typestates of objects can be expressed as subset and disjointness relations on sets, and elements of sets can be represen…
▽ More
Typestate systems ensure many desirable properties of imperative programs, including initialization of object fields and correct use of stateful library interfaces. Abstract sets with cardinality constraints naturally generalize typestate properties: relationships between the typestates of objects can be expressed as subset and disjointness relations on sets, and elements of sets can be represented as sets of cardinality one. Motivated by these applications, this paper presents new algorithms and new complexity results for constraints on sets and their cardinalities. We study several classes of constraints and demonstrate a trade-off between their expressive power and their complexity.
Our first result concerns a quantifier-free fragment of Boolean Algebra with Presburger Arithmetic. We give a nondeterministic polynomial-time algorithm for reducing the satisfiability of sets with symbolic cardinalities to constraints on constant cardinalities, and give a polynomial-space algorithm for the resulting problem.
In a quest for more efficient fragments, we identify several subclasses of sets with cardinality constraints whose satisfiability is NP-hard. Finally, we identify a class of constraints that has polynomial-time satisfiability and entailment problems and can serve as a foundation for efficient program analysis.
△ Less
Submitted 28 August, 2005;
originally announced August 2005.
-
On Spatial Conjunction as Second-Order Logic
Authors:
Viktor Kuncak,
Martin Rinard
Abstract:
Spatial conjunction is a powerful construct for reasoning about dynamically allocated data structures, as well as concurrent, distributed and mobile computation. While researchers have identified many uses of spatial conjunction, its precise expressive power compared to traditional logical constructs was not previously known. In this paper we establish the expressive power of spatial conjunction…
▽ More
Spatial conjunction is a powerful construct for reasoning about dynamically allocated data structures, as well as concurrent, distributed and mobile computation. While researchers have identified many uses of spatial conjunction, its precise expressive power compared to traditional logical constructs was not previously known. In this paper we establish the expressive power of spatial conjunction. We construct an embedding from first-order logic with spatial conjunction into second-order logic, and more surprisingly, an embedding from full second order logic into first-order logic with spatial conjunction. These embeddings show that the satisfiability of formulas in first-order logic with spatial conjunction is equivalent to the satisfiability of formulas in second-order logic. These results explain the great expressive power of spatial conjunction and can be used to show that adding unrestricted spatial conjunction to a decidable logic leads to an undecidable logic. As one example, we show that adding unrestricted spatial conjunction to two-variable logic leads to undecidability. On the side of decidability, the embedding into second-order logic immediately implies the decidability of first-order logic with a form of spatial conjunction over trees. The embedding into spatial conjunction also has useful consequences: because a restricted form of spatial conjunction in two-variable logic preserves decidability, we obtain that a correspondingly restricted form of second-order quantification in two-variable logic is decidable. The resulting language generalizes the first-order theory of boolean algebra over sets and is useful in reasoning about the contents of data structures in object-oriented languages.
△ Less
Submitted 28 October, 2004;
originally announced October 2004.
-
On computing the fixpoint of a set of boolean equations
Authors:
Viktor Kuncak,
K. Rustan M. Leino
Abstract:
This paper presents a method for computing a least fixpoint of a system of equations over booleans. The resulting computation can be significantly shorter than the result of iteratively evaluating the entire system until a fixpoint is reached.
This paper presents a method for computing a least fixpoint of a system of equations over booleans. The resulting computation can be significantly shorter than the result of iteratively evaluating the entire system until a fixpoint is reached.
△ Less
Submitted 19 August, 2004;
originally announced August 2004.
-
On Generalized Records and Spatial Conjunction in Role Logic
Authors:
Viktor Kuncak,
Martin Rinard
Abstract:
We have previously introduced role logic as a notation for describing properties of relational structures in shape analysis, databases and knowledge bases. A natural fragment of role logic corresponds to two-variable logic with counting and is therefore decidable. We show how to use role logic to describe open and closed records, as well the dual of records, inverse records. We observe that the…
▽ More
We have previously introduced role logic as a notation for describing properties of relational structures in shape analysis, databases and knowledge bases. A natural fragment of role logic corresponds to two-variable logic with counting and is therefore decidable. We show how to use role logic to describe open and closed records, as well the dual of records, inverse records. We observe that the spatial conjunction operation of separation logic naturally models record concatenation. Moreover, we show how to eliminate the spatial conjunction of formulas of quantifier depth one in first-order logic with counting. As a result, allowing spatial conjunction of formulas of quantifier depth one preserves the decidability of two-variable logic with counting. This result applies to two-variable role logic fragment as well. The resulting logic smoothly integrates type system and predicate calculus notation and can be viewed as a natural generalization of the notation for constraints arising in role analysis and similar shape analysis approaches.
△ Less
Submitted 5 August, 2004;
originally announced August 2004.
-
On Role Logic
Authors:
Viktor Kuncak,
Martin Rinard
Abstract:
We present role logic, a notation for describing properties of relational structures in shape analysis, databases, and knowledge bases. We construct role logic using the ideas of de Bruijn's notation for lambda calculus, an encoding of first-order logic in lambda calculus, and a simple rule for implicit arguments of unary and binary predicates. The unrestricted version of role logic has the expr…
▽ More
We present role logic, a notation for describing properties of relational structures in shape analysis, databases, and knowledge bases. We construct role logic using the ideas of de Bruijn's notation for lambda calculus, an encoding of first-order logic in lambda calculus, and a simple rule for implicit arguments of unary and binary predicates. The unrestricted version of role logic has the expressive power of first-order logic with transitive closure. Using a syntactic restriction on role logic formulas, we identify a natural fragment RL^2 of role logic. We show that the RL^2 fragment has the same expressive power as two-variable logic with counting C^2 and is therefore decidable. We present a translation of an imperative language into the decidable fragment RL^2, which allows compositional verification of programs that manipulate relational structures. In addition, we show how RL^2 encodes boolean shape analysis constraints and an expressive description logic.
△ Less
Submitted 5 August, 2004;
originally announced August 2004.
-
On the Theory of Structural Subty**
Authors:
Viktor Kuncak,
Martin Rinard
Abstract:
We show that the first-order theory of structural subty** of non-recursive types is decidable. Let $Σ$ be a language consisting of function symbols (representing type constructors) and $C$ a decidable structure in the relational language $L$ containing a binary relation $\leq$. $C$ represents primitive types; $\leq$ represents a subtype ordering. We introduce the notion of $Σ$-term-power of…
▽ More
We show that the first-order theory of structural subty** of non-recursive types is decidable. Let $Σ$ be a language consisting of function symbols (representing type constructors) and $C$ a decidable structure in the relational language $L$ containing a binary relation $\leq$. $C$ represents primitive types; $\leq$ represents a subtype ordering. We introduce the notion of $Σ$-term-power of $C$, which generalizes the structure arising in structural subty**. The domain of the $Σ$-term-power of $C$ is the set of $Σ$-terms over the set of elements of $C$. We show that the decidability of the first-order theory of $C$ implies the decidability of the first-order theory of the $Σ$-term-power of $C$. Our decision procedure makes use of quantifier elimination for term algebras and Feferman-Vaught theorem. Our result implies the decidability of the first-order theory of structural subty** of non-recursive types.
△ Less
Submitted 5 August, 2004;
originally announced August 2004.
-
Typestate Checking and Regular Graph Constraints
Authors:
Viktor Kuncak,
Martin Rinard
Abstract:
We introduce regular graph constraints and explore their decidability properties. The motivation for regular graph constraints is 1) type checking of changing types of objects in the presence of linked data structures, 2) shape analysis techniques, and 3) generalization of similar constraints over trees and grids. We define a subclass of graphs called heaps as an abstraction of the data structur…
▽ More
We introduce regular graph constraints and explore their decidability properties. The motivation for regular graph constraints is 1) type checking of changing types of objects in the presence of linked data structures, 2) shape analysis techniques, and 3) generalization of similar constraints over trees and grids. We define a subclass of graphs called heaps as an abstraction of the data structures that a program constructs during its execution. We prove that determining the validity of implication for regular graph constraints over the class of heaps is undecidable. We show undecidability by exhibiting a characterization of certain "corresponder graphs" in terms of presence and absence of homomorphisms to a finite number of fixed graphs. The undecidability of implication of regular graph constraints implies that there is no algorithm that will verify that procedure preconditions are met or that the invariants are maintained when these properties are expressed in any specification language at least as expressive as regular graph constraints.
△ Less
Submitted 4 August, 2004;
originally announced August 2004.
-
Roles Are Really Great!
Authors:
Viktor Kuncak,
Patrick Lam,
Martin Rinard
Abstract:
We present a new role system for specifying changing referencing relationships of heap objects. The role of an object depends, in large part, on its aliasing relationships with other objects, with the role of each object changing as its aliasing relationships change. Roles therefore capture important object and data structure properties and provide useful information about how the actions of the…
▽ More
We present a new role system for specifying changing referencing relationships of heap objects. The role of an object depends, in large part, on its aliasing relationships with other objects, with the role of each object changing as its aliasing relationships change. Roles therefore capture important object and data structure properties and provide useful information about how the actions of the program interact with these properties. Our role system enables the programmer to specify the legal aliasing relationships that define the set of roles that objects may play, the roles of procedure parameters and object fields, and the role changes that procedures perform while manipulating objects. We present an interprocedural, compositional, and context-sensitive role analysis algorithm that verifies that a program respects the role constraints.
△ Less
Submitted 4 August, 2004;
originally announced August 2004.
-
The First-Order Theory of Sets with Cardinality Constraints is Decidable
Authors:
Viktor Kuncak,
Martin Rinard
Abstract:
We show that the decidability of the first-order theory of the language that combines Boolean algebras of sets of uninterpreted elements with Presburger arithmetic operations. We thereby disprove a recent conjecture that this theory is undecidable. Our language allows relating the cardinalities of sets to the values of integer variables, and can distinguish finite and infinite sets. We use quant…
▽ More
We show that the decidability of the first-order theory of the language that combines Boolean algebras of sets of uninterpreted elements with Presburger arithmetic operations. We thereby disprove a recent conjecture that this theory is undecidable. Our language allows relating the cardinalities of sets to the values of integer variables, and can distinguish finite and infinite sets. We use quantifier elimination to show the decidability and obtain an elementary upper bound on the complexity.
Precise program analyses can use our decidability result to verify representation invariants of data structures that use an integer field to represent the number of stored elements.
△ Less
Submitted 3 October, 2004; v1 submitted 17 July, 2004;
originally announced July 2004.