Showing 1–50 of 67 results for author: Soudjani, S

Specification-guided temporal logic control for stochastic systems: a multi-layered approach

Authors: Birgit C. van Huijgevoort, Ruohan Wang, Sadegh Soudjani, Sofie Haesaert

Abstract: Designing controllers to satisfy temporal requirements has proven to be challenging for dynamical systems that are affected by uncertainty. This is mainly due to the states evolving in a continuous uncountable space, the stochastic evolution of the states, and infinite-horizon temporal requirements on the system evolution, all of which makes closed-form solutions generally inaccessible. A promisin… ▽ More Designing controllers to satisfy temporal requirements has proven to be challenging for dynamical systems that are affected by uncertainty. This is mainly due to the states evolving in a continuous uncountable space, the stochastic evolution of the states, and infinite-horizon temporal requirements on the system evolution, all of which makes closed-form solutions generally inaccessible. A promising approach for designing provably correct controllers on such systems is to utilize the concept of abstraction, which is based on building simplified abstract models that can be used to approximate optimal controllers with provable closeness guarantees. The available abstraction-based methods are further divided into discretization-based approaches that build a finite abstract model by discretizing the continuous space of the system, and discretization-free approaches that work directly on the continuous state space without the need for building a finite space. To reduce the conservatism in the sub-optimality of the designed controller originating from the abstraction step, this paper develops an approach that naturally has the flexibility to combine different abstraction techniques from the aforementioned classes and to combine the same abstraction technique with different parameters. First, we develop a multi-layered discretization-based approach with variable precision by combining abstraction layers with different precision parameters. Then, we exploit the advantages of both classes of abstraction-based methods by extending this multi-layered approach guided by the specification to combinations of layers with respectively discretization-based and discretization-free abstractions. We achieve an efficient implementation that is less conservative and improves the computation time and memory usage. We illustrate the benefits of the proposed multi-layered approach on several case studies. △ Less

Submitted 4 July, 2024; originally announced July 2024.

T-Count Optimizing Genetic Algorithm for Quantum State Preparation

Authors: Andrew Wright, Marco Lewis, Paolo Zuliani, Sadegh Soudjani

Abstract: Quantum state preparation is a crucial process within numerous quantum algorithms, and the need for efficient initialization of quantum registers is ever increasing as demand for useful quantum computing grows. The problem arises as the number of qubits to be initialized grows, the circuits required to implement the desired state also exponentially increase in size leading to loss of fidelity to n… ▽ More Quantum state preparation is a crucial process within numerous quantum algorithms, and the need for efficient initialization of quantum registers is ever increasing as demand for useful quantum computing grows. The problem arises as the number of qubits to be initialized grows, the circuits required to implement the desired state also exponentially increase in size leading to loss of fidelity to noise. This is mainly due to the susceptibility to environmental effects of the non-Clifford T gate, whose use should thus be reduced as much as possible. In this paper, we present and utilize a genetic algorithm for state preparation circuits consisting of gates from the Clifford + T gate set and optimize them in T-Count as to reduce the impact of noise. Whilst the method presented here does not always produce the most accurate circuits in terms of fidelity, it can generate high-fidelity, non-trivial quantum states such as quantum Fourier transform states. In addition, our algorithm does automatically generate fault tolerantly implementable solutions where the number of the most error prone components is reduced. We present an evaluation of the algorithm when trialed against preparing random, Poisson probability distribution, W, GHZ, and quantum Fourier transform states. We also experimentally demonstrate the scalability issues as qubit count increases, which highlights the need for further optimization of the search process. △ Less

Submitted 6 June, 2024; originally announced June 2024.

Comments: To appear in IEEE QSW 2024 proceedings

Automated Verification of Silq Quantum Programs using SMT Solvers

Authors: Marco Lewis, Paolo Zuliani, Sadegh Soudjani

Abstract: We present SilVer (Silq Verification), an automated tool for verifying behaviors of quantum programs written in Silq, which is a high-level programming language for quantum computing. The goal of the verification is to ensure correctness of the Silq quantum program against user-defined specifications using SMT solvers. We introduce a programming model that is based on a quantum RAM-style computer… ▽ More We present SilVer (Silq Verification), an automated tool for verifying behaviors of quantum programs written in Silq, which is a high-level programming language for quantum computing. The goal of the verification is to ensure correctness of the Silq quantum program against user-defined specifications using SMT solvers. We introduce a programming model that is based on a quantum RAM-style computer as an interface between Silq programs and SMT proof obligations, allowing for control of quantum operations using both classical and quantum conditions. Additionally, users can employ measurement flags within the specification to easily specify conditions that measurement results require to satisfy for being a valid behavior. We provide case studies on the verification of generating entangled states and multiple oracle-based algorithms. △ Less

Submitted 5 June, 2024; originally announced June 2024.

Comments: 10 pages, to appear in the proceedings of IEEE QSW 2024

Vagus nerve stimulation: Laying the groundwork for predictive network-based computer models

Authors: John F. Ingham, Frances Hutchings, Paolo Zuliani, Yujiang Wang, Sadegh Soudjani, Peter N. Taylor

Abstract: Vagus Nerve Stimulation (VNS) is an established palliative treatment for drug resistant epilepsy. While effective for many patients, its mechanism of action is incompletely understood. Predicting individuals' response, or optimum stimulation parameters, is challenging. Computational modelling has informed other problems in epilepsy but, to our knowledge, has not been applied to VNS. We started w… ▽ More Vagus Nerve Stimulation (VNS) is an established palliative treatment for drug resistant epilepsy. While effective for many patients, its mechanism of action is incompletely understood. Predicting individuals' response, or optimum stimulation parameters, is challenging. Computational modelling has informed other problems in epilepsy but, to our knowledge, has not been applied to VNS. We started with an established, four-population neural mass model (NMM), capable of reproducing the seizure-like dynamics of a thalamocortical circuit. We extended this to include 18 further neural populations, representing nine other brain regions relevant to VNS, with connectivity based on existing literature. We modelled stimulated afferent vagal fibres as projecting to the nucleus tractus solitarius (NTS), which receives input from the vagus nerve in vivo. Bifurcation analysis of a deterministic version of the model showed higher background NTS input made the model monostable at a fixed point (FP), representing normal activity, while lower inputs produce bistability between the FP and a limit cycle (LC), representing the seizure state. Adding noise produced transitions between seizure and normal states. This stochastic model spent decreasing time in the seizure state with increasing background NTS input, until seizures were abolished, consistent with the deterministic model. Simulated VNS stimulation, modelled as a 30 Hz square wave, was summed with the background input to the NTS and was found to reduce total seizure duration in a dose-dependent manner, similar to expectations in vivo. We have successfully produced an in silico model of VNS in epilepsy, capturing behaviour seen in vivo. This may aid understanding therapeutic mechanisms of VNS in epilepsy and provides a starting point to (i) determine which patients might respond best to VNS, and (ii) optimise individuals' treatments. △ Less

Submitted 4 June, 2024; originally announced June 2024.

Rare Collision Risk Estimation of Autonomous Vehicles with Multi-Agent Situation Awareness

Authors: Mahdieh Zaker, Henk A. P. Blom, Sadegh Soudjani, Abolfazl Lavaei

Abstract: This paper offers a formal framework for the rare collision risk estimation of autonomous vehicles (AVs) with multi-agent situation awareness, affected by different sources of noise in a complex dynamic environment. In our proposed setting, the situation awareness is considered for one of the ego vehicles by aggregating a range of diverse information gathered from other vehicles into a vector. We… ▽ More This paper offers a formal framework for the rare collision risk estimation of autonomous vehicles (AVs) with multi-agent situation awareness, affected by different sources of noise in a complex dynamic environment. In our proposed setting, the situation awareness is considered for one of the ego vehicles by aggregating a range of diverse information gathered from other vehicles into a vector. We model AVs equipped with the situation awareness as general stochastic hybrid systems (GSHS) and assess the probability of collision in a lane-change scenario where two self-driving vehicles simultaneously intend to switch lanes into a shared one, while utilizing the time-to-collision measure for decision-making as required. Due to the substantial data requirements of simulation-based methods for the rare collision risk estimation, we leverage a multi-level importance splitting technique, known as interacting particle system-based estimation with fixed assignment splitting (IPS-FAS). This approach allows us to estimate the probability of a rare event by employing a group of interacting particles. Specifically, each particle embodies a system trajectory and engages with others through resampling and branching, focusing computational resources on trajectories with the highest probability of encountering the rare event. The effectiveness of our proposed approach is demonstrated through an extensive simulation of a lane-change scenario. △ Less

Submitted 2 May, 2024; originally announced May 2024.

Temporal Logic Resilience for Dynamical Systems

Authors: Adnane Saoud, Pushpak Jagtap, Sadegh Soudjani

Abstract: We consider the notion of resilience for cyber-physical systems, that is, the ability of the system to withstand adverse events while maintaining acceptable functionality. We use finite temporal logic to express the requirements on the acceptable functionality and define the resilience metric as the maximum disturbance under which the system satisfies the temporal requirements. We fix a parameteri… ▽ More We consider the notion of resilience for cyber-physical systems, that is, the ability of the system to withstand adverse events while maintaining acceptable functionality. We use finite temporal logic to express the requirements on the acceptable functionality and define the resilience metric as the maximum disturbance under which the system satisfies the temporal requirements. We fix a parameterized template for the set of disturbances and form a robust optimization problem under the system dynamics and the temporal specifications to find the maximum value of the parameter. Additionally, we introduce two novel classes of specifications: closed and convex finite temporal logics specifications, offering a comprehensive analysis of the resilience metric within these specific frameworks. From a computational standpoint, we present an exact solution for linear systems and exact-time reachability and finite-horizon safety, complemented by an approximate solution for finite-horizon reachability. Extending our findings to nonlinear systems, we leverage linear approximations and SMT-based approaches to offer viable computational methodologies. The theoretical results are demonstrated on the temperature regulation of buildings, adaptive cruise control and DC motors. △ Less

Submitted 29 April, 2024; originally announced April 2024.

Safe Reach Set Computation via Neural Barrier Certificates

Authors: Alessandro Abate, Sergiy Bogomolov, Alec Edwards, Kostiantyn Potomkin, Sadegh Soudjani, Paolo Zuliani

Abstract: We present a novel technique for online safety verification of autonomous systems, which performs reachability analysis efficiently for both bounded and unbounded horizons by employing neural barrier certificates. Our approach uses barrier certificates given by parameterized neural networks that depend on a given initial set, unsafe sets, and time horizon. Such networks are trained efficiently off… ▽ More We present a novel technique for online safety verification of autonomous systems, which performs reachability analysis efficiently for both bounded and unbounded horizons by employing neural barrier certificates. Our approach uses barrier certificates given by parameterized neural networks that depend on a given initial set, unsafe sets, and time horizon. Such networks are trained efficiently offline using system simulations sampled from regions of the state space. We then employ a meta-neural network to generalize the barrier certificates to state space regions that are outside the training set. These certificates are generated and validated online as sound over-approximations of the reachable states, thus either ensuring system safety or activating appropriate alternative actions in unsafe scenarios. We demonstrate our technique on case studies from linear models to nonlinear control-dependent models for online autonomous driving scenarios. △ Less

Submitted 29 April, 2024; originally announced April 2024.

Comments: IFAC Conference on Analysis and Design of Hybrid Systems

Data-Driven Distributionally Robust Safety Verification Using Barrier Certificates and Conditional Mean Embeddings

Authors: Oliver Schön, Zhengang Zhong, Sadegh Soudjani

Abstract: Algorithmic verification of realistic systems to satisfy safety and other temporal requirements has suffered from poor scalability of the employed formal approaches. To design systems with rigorous guarantees, many approaches still rely on exact models of the underlying systems. Since this assumption can rarely be met in practice, models have to be inferred from measurement data or are bypassed co… ▽ More Algorithmic verification of realistic systems to satisfy safety and other temporal requirements has suffered from poor scalability of the employed formal approaches. To design systems with rigorous guarantees, many approaches still rely on exact models of the underlying systems. Since this assumption can rarely be met in practice, models have to be inferred from measurement data or are bypassed completely. Whilst former usually requires the model structure to be known a-priori and immense amounts of data to be available, latter gives rise to a plethora of restrictive mathematical assumptions about the unknown dynamics. In a pursuit of develo** scalable formal verification algorithms without shifting the problem to unrealistic assumptions, we employ the concept of barrier certificates, which can guarantee safety of the system, and learn the certificate directly from a compact set of system trajectories. We use conditional mean embeddings to embed data from the system into a reproducing kernel Hilbert space (RKHS) and construct an RKHS ambiguity set that can be inflated to robustify the result w.r.t. a set of plausible transition kernels. We show how to solve the resulting program efficiently using sum-of-squares optimization and a Gaussian process envelope. Our approach lifts the need for restrictive assumptions on the system dynamics and uncertainty, and suggests an improvement in the sample complexity of verifying the safety of a system on a tested case study compared to a state-of-the-art approach. △ Less

Submitted 15 March, 2024; originally announced March 2024.

Comments: 7 pages, 2 figures, accepted to American Control Conference (ACC) 2024

Formal Verification of Unknown Stochastic Systems via Non-parametric Estimation

Authors: Zhi Zhang, Chenyu Ma, Saleh Soudijani, Sadegh Soudjani

Abstract: A novel data-driven method for formal verification is proposed to study complex systems operating in safety-critical domains. The proposed approach is able to formally verify discrete-time stochastic dynamical systems against temporal logic specifications only using observation samples and without the knowledge of the model, and provide a probabilistic guarantee on the satisfaction of the specific… ▽ More A novel data-driven method for formal verification is proposed to study complex systems operating in safety-critical domains. The proposed approach is able to formally verify discrete-time stochastic dynamical systems against temporal logic specifications only using observation samples and without the knowledge of the model, and provide a probabilistic guarantee on the satisfaction of the specification. We first propose the theoretical results for using non-parametric estimation to estimate an asymptotic upper bound for the \emph{Lipschitz constant} of the stochastic system, which can determine a finite abstraction of the system. Our results prove that the asymptotic convergence rate of the estimation is $O(n^{-\frac{1}{3+d}})$, where $d$ is the dimension of the system and $n$ is the data scale. We then construct interval Markov decision processes using two different data-driven methods, namely non-parametric estimation and empirical estimation of transition probabilities, to perform formal verification against a given temporal logic specification. Multiple case studies are presented to validate the effectiveness of the proposed methods. △ Less

Submitted 8 March, 2024; originally announced March 2024.

Awareness in robotics: An early perspective from the viewpoint of the EIC Pathfinder Challenge "Awareness Inside''

Authors: Cosimo Della Santina, Carlos Hernandez Corbato, Burak Sisman, Luis A. Leiva, Ioannis Arapakis, Michalis Vakalellis, Jean Vanderdonckt, Luis Fernando D'Haro, Guido Manzi, Cristina Becchio, Aïda Elamrani, Mohsen Alirezaei, Ginevra Castellano, Dimos V. Dimarogonas, Arabinda Ghosh, Sofie Haesaert, Sadegh Soudjani, Sybert Stroeve, Paul Verschure, Davide Bacciu, Ophelia Deroy, Bahador Bahrami, Claudio Gallicchio, Sabine Hauert, Ricardo Sanz , et al. (6 additional authors not shown)

Abstract: Consciousness has been historically a heavily debated topic in engineering, science, and philosophy. On the contrary, awareness had less success in raising the interest of scholars in the past. However, things are changing as more and more researchers are getting interested in answering questions concerning what awareness is and how it can be artificially generated. The landscape is rapidly evolvi… ▽ More Consciousness has been historically a heavily debated topic in engineering, science, and philosophy. On the contrary, awareness had less success in raising the interest of scholars in the past. However, things are changing as more and more researchers are getting interested in answering questions concerning what awareness is and how it can be artificially generated. The landscape is rapidly evolving, with multiple voices and interpretations of the concept being conceived and techniques being developed. The goal of this paper is to summarize and discuss the ones among these voices connected with projects funded by the EIC Pathfinder Challenge called ``Awareness Inside'', a nonrecurring call for proposals within Horizon Europe designed specifically for fostering research on natural and synthetic awareness. In this perspective, we dedicate special attention to challenges and promises of applying synthetic awareness in robotics, as the development of mature techniques in this new field is expected to have a special impact on generating more capable and trustworthy embodied systems. △ Less

Submitted 14 February, 2024; originally announced February 2024.

Assume-Guarantee Reinforcement Learning

Authors: Milad Kazemi, Mateo Perez, Fabio Somenzi, Sadegh Soudjani, Ashutosh Trivedi, Alvaro Velasquez

Abstract: We present a modular approach to \emph{reinforcement learning} (RL) in environments consisting of simpler components evolving in parallel. A monolithic view of such modular environments may be prohibitively large to learn, or may require unrealizable communication between the components in the form of a centralized controller. Our proposed approach is based on the assume-guarantee paradigm where t… ▽ More We present a modular approach to \emph{reinforcement learning} (RL) in environments consisting of simpler components evolving in parallel. A monolithic view of such modular environments may be prohibitively large to learn, or may require unrealizable communication between the components in the form of a centralized controller. Our proposed approach is based on the assume-guarantee paradigm where the optimal control for the individual components is synthesized in isolation by making \emph{assumptions} about the behaviors of neighboring components, and providing \emph{guarantees} about their own behavior. We express these \emph{assume-guarantee contracts} as regular languages and provide automatic translations to scalar rewards to be used in RL. By combining local probabilities of satisfaction for each component, we provide a lower bound on the probability of satisfaction of the complete system. By solving a Markov game for each component, RL can produce a controller for each component that maximizes this lower bound. The controller utilizes the information it receives through communication, observations, and any knowledge of a coarse model of other agents. We experimentally demonstrate the efficiency of the proposed approach on a variety of case studies. △ Less

Submitted 15 December, 2023; originally announced December 2023.

Comments: This is the extended version of the paper accepted in the SRRAI Special Track at the Conference on Artificial Intelligence (AAAI-24)

Using Knowledge Awareness to improve Safety of Autonomous Driving

Authors: Andrea Calvagna, Arabinda Ghosh, Sadegh Soudjani

Abstract: We present a method, which incorporates knowledge awareness into the symbolic computation of discrete controllers for reactive cyber physical systems, to improve decision making about the unknown operating environment under uncertain/incomplete inputs. Assuming an abstract model of the system and the environment, we translate the knowledge awareness of the operating context into linear temporal lo… ▽ More We present a method, which incorporates knowledge awareness into the symbolic computation of discrete controllers for reactive cyber physical systems, to improve decision making about the unknown operating environment under uncertain/incomplete inputs. Assuming an abstract model of the system and the environment, we translate the knowledge awareness of the operating context into linear temporal logic formulas and incorporate them into the system specifications to synthesize a controller. The knowledge base is built upon an ontology model of the environment objects and behavioural rules, which includes also symbolic models of partial input features. The resulting symbolic controller support smoother, early reactions, which improves the security of the system over existing approaches based on incremental symbolic perception. A motion planning case study for an autonomous vehicle has been implemented to validate the approach, and presented results show significant improvements with respect to safety of state-of-the-art symbolic controllers for reactive systems. △ Less

Submitted 25 October, 2023; originally announced October 2023.

Generalized Stochastic Dynamic Aggregative Game for Demand-Side Management in Microgrids with Shared Battery

Authors: Shahram Yadollahi, Hamed Kebriaei, Sadegh Soudjani

Abstract: In this paper, we focus on modeling and analysis of demand-side management in a microgrid where agents utilize grid energy and a shared battery charged by renewable energy sources. We model the problem as a generalized stochastic dynamic aggregative game with chance constraints that capture the effects of uncertainties in the renewable generation and agents' demands. Computing the solution of the… ▽ More In this paper, we focus on modeling and analysis of demand-side management in a microgrid where agents utilize grid energy and a shared battery charged by renewable energy sources. We model the problem as a generalized stochastic dynamic aggregative game with chance constraints that capture the effects of uncertainties in the renewable generation and agents' demands. Computing the solution of the game is a complex task due to probabilistic and coupling constraints among the agents through the state of charge of the shared battery. We investigate the Nash equilibrium of this game under uncertainty considering both the uniqueness of the solution and the effect of uncertainty on the solution. Simulation results demonstrate that the presented stochastic method is superior to deterministic methods. △ Less

Submitted 10 October, 2023; v1 submitted 4 October, 2023; originally announced October 2023.

Safety Barrier Certificates for Stochastic Control Systems with Wireless Communication Networks

Authors: Omid Akbarzadeh, Sadegh Soudjani, Abolfazl Lavaei

Abstract: This work is concerned with a formal approach for safety controller synthesis of stochastic control systems with both process and measurement noises while considering wireless communication networks between sensors, controllers, and actuators. The proposed scheme is based on control barrier certificates (CBC), which allows us to provide safety certifications for wirelessly-connected stochastic con… ▽ More This work is concerned with a formal approach for safety controller synthesis of stochastic control systems with both process and measurement noises while considering wireless communication networks between sensors, controllers, and actuators. The proposed scheme is based on control barrier certificates (CBC), which allows us to provide safety certifications for wirelessly-connected stochastic control systems. Despite the available literature on designing control barrier certificates, there has been unfortunately no consideration of wireless communication networks to capture potential packet losses and end-to-end delays, which is absolutely crucial in safety-critical real-world applications. In our proposed setting, the key objective is to construct a control barrier certificate together with a safety controller while providing a lower bound on the satisfaction probability of the safety property over a finite time horizon. We propose a systematic approach in the form of sum-of-squares optimization and matrix inequalities for the synthesis of CBC and its associated controller. We demonstrate the efficacy of our approach on a permanent magnet synchronous motor. For the application of automotive electric steering under a wireless communication network, we design a CBC together with a safety controller to maintain the electrical current of the motor in a safe set within a finite time horizon while providing a formal probabilistic guarantee. △ Less

Submitted 11 September, 2023; originally announced September 2023.

Verifying the Unknown: Correct-by-Design Control Synthesis for Networks of Stochastic Uncertain Systems

Authors: Oliver Schön, Birgit van Huijgevoort, Sofie Haesaert, Sadegh Soudjani

Abstract: In this paper, we present an approach for designing correct-by-design controllers for cyber-physical systems composed of multiple dynamically interconnected uncertain systems. We consider networked discrete-time uncertain nonlinear systems with additive stochastic noise and model parametric uncertainty. Such settings arise when multiple systems interact in an uncertain environment and only observa… ▽ More In this paper, we present an approach for designing correct-by-design controllers for cyber-physical systems composed of multiple dynamically interconnected uncertain systems. We consider networked discrete-time uncertain nonlinear systems with additive stochastic noise and model parametric uncertainty. Such settings arise when multiple systems interact in an uncertain environment and only observational data is available. We address two limitations of existing approaches for formal synthesis of controllers for networks of uncertain systems satisfying complex temporal specifications. Firstly, whilst existing approaches rely on the stochasticity to be Gaussian, the heterogeneous nature of composed systems typically yields a more complex stochastic behavior. Secondly, exact models of the systems involved are generally not available or difficult to acquire. To address these challenges, we show how abstraction-based control synthesis for uncertain systems based on sub-probability couplings can be extended to networked systems. We design controllers based on parameter uncertainty sets identified from observational data and approximate possibly arbitrary noise distributions using Gaussian mixture models whilst quantifying the incurred stochastic coupling. Finally, we demonstrate the effectiveness of our approach on a nonlinear package delivery case study with a complex specification, and a platoon of cars. △ Less

Submitted 3 September, 2023; originally announced September 2023.

Comments: 9 pages, 4 figures, accepted to CDC 2023

Verification of Quantum Systems using Barrier Certificates

Authors: Marco Lewis, Paolo Zuliani, Sadegh Soudjani

Abstract: Various techniques have been used in recent years for verifying quantum computers, that is, for determining whether a quantum computer/system satisfies a given formal specification of correctness. Barrier certificates are a recent novel concept developed for verifying properties of dynamical systems. In this article, we investigate the usage of barrier certificates as a means for verifying behavio… ▽ More Various techniques have been used in recent years for verifying quantum computers, that is, for determining whether a quantum computer/system satisfies a given formal specification of correctness. Barrier certificates are a recent novel concept developed for verifying properties of dynamical systems. In this article, we investigate the usage of barrier certificates as a means for verifying behaviours of quantum systems. To do this, we extend the notion of barrier certificates from real to complex variables. We then develop a computational technique based on linear programming to automatically generate polynomial barrier certificates with complex variables taking real values. Finally, we apply our technique to several simple quantum systems to demonstrate their usage. △ Less

Submitted 14 July, 2023; originally announced July 2023.

Comments: 18 pages, 5 figures

Neural Abstraction-Based Controller Synthesis and Deployment

Authors: Rupak Majumdar, Mahmoud Salamati, Sadegh Soudjani

Abstract: Abstraction-based techniques are an attractive approach for synthesizing correct-by-construction controllers to satisfy high-level temporal requirements. A main bottleneck for successful application of these techniques is the memory requirement, both during controller synthesis and in controller deployment. We propose memory-efficient methods for mitigating the high memory demands of the abstrac… ▽ More Abstraction-based techniques are an attractive approach for synthesizing correct-by-construction controllers to satisfy high-level temporal requirements. A main bottleneck for successful application of these techniques is the memory requirement, both during controller synthesis and in controller deployment. We propose memory-efficient methods for mitigating the high memory demands of the abstraction-based techniques using neural network representations. To perform synthesis for reach-avoid specifications, we propose an on-the-fly algorithm that relies on compressed neural network representations of the forward and backward dynamics of the system. In contrast to usual applications of neural representations, our technique maintains soundness of the end-to-end process. To ensure this, we correct the output of the trained neural network such that the corrected output representations are sound with respect to the finite abstraction. For deployment, we provide a novel training algorithm to find a neural network representation of the synthesized controller and experimentally show that the controller can be correctly represented as a combination of a neural network and a look-up table that requires a substantially smaller memory. We demonstrate experimentally that our approach significantly reduces the memory requirements of abstraction-based methods. For the selected benchmarks, our approach reduces the memory requirements respectively for the synthesis and deployment by a factor of $1.31\times 10^5$ and $7.13\times 10^3$ on average, and up to $7.54\times 10^5$ and $3.18\times 10^4$. Although this reduction is at the cost of increased off-line computations to train the neural networks, all the steps of our approach are parallelizable and can be implemented on machines with higher number of processing units to reduce the required computational time. △ Less

Submitted 7 July, 2023; originally announced July 2023.

Formal Control of New England 39-Bus Test System: An Assume-Guarantee Approach

Authors: Ben Wooding, Abolfazl Lavaei, Sadegh Soudjani

Abstract: This work is concerned with an assume-guarantee approach to compositionally control a New England 39-bus Test System (NETS). The proposed scheme is based on the new notion of robust simulation functions with disturbance refinement alongside the composition of multiple subsystems to tackle the difficulties associated with scalability, also known as the curse of dimensionality. In our proposed setti… ▽ More This work is concerned with an assume-guarantee approach to compositionally control a New England 39-bus Test System (NETS). The proposed scheme is based on the new notion of robust simulation functions with disturbance refinement alongside the composition of multiple subsystems to tackle the difficulties associated with scalability, also known as the curse of dimensionality. In our proposed setting, we approximate concrete subsystems with abstractions that have lower dimensions (a.k.a. reduced-order models) while providing mathematical guarantees on controller synthesis through the quantification of an upper bound on the closeness between output trajectories of original systems and their abstractions. We propose two control methods to provide guarantees for NETS: one using the principle of interconnected synchronous machines and another considering the power flows in the network between neighbouring subsystems. △ Less

Submitted 7 July, 2023; originally announced July 2023.

Comments: 31 pages, 13 figures

Bayesian Formal Synthesis of Unknown Systems via Robust Simulation Relations

Authors: Oliver Schön, Birgit van Huijgevoort, Sofie Haesaert, Sadegh Soudjani

Abstract: This paper addresses the problem of data-driven computation of controllers that are correct by design for safety-critical systems and can provably satisfy (complex) functional requirements. With a focus on continuous-space stochastic systems with parametric uncertainty, we propose a two-stage approach that decomposes the problem into a learning stage and a robust formal controller synthesis stage.… ▽ More This paper addresses the problem of data-driven computation of controllers that are correct by design for safety-critical systems and can provably satisfy (complex) functional requirements. With a focus on continuous-space stochastic systems with parametric uncertainty, we propose a two-stage approach that decomposes the problem into a learning stage and a robust formal controller synthesis stage. The first stage utilizes available Bayesian regression results to compute robust credible sets for the true parameters of the system. For the second stage, we introduce methods for systems subject to both stochastic and parametric uncertainties. We provide simulation relations for enabling correct-by-design control refinement that are founded on coupling uncertainties of stochastic systems via sub-probability measures. The presented relations are essential for constructing abstract models that are related to not only one model but to a set of parameterized models. The results are demonstrated on three case studies, including a nonlinear and a high-dimensional system. △ Less

Submitted 12 February, 2024; v1 submitted 14 April, 2023; originally announced April 2023.

Comments: arXiv admin note: substantial text overlap with arXiv:2210.08269

SySCoRe: Synthesis via Stochastic Coupling Relations

Authors: Birgit van Huijgevoort, Oliver Schön, Sadegh Soudjani, Sofie Haesaert

Abstract: We present SySCoRe, a MATLAB toolbox that synthesizes controllers for stochastic continuous-state systems to satisfy temporal logic specifications. Starting from a system description and a co-safe temporal logic specification, SySCoRe provides all necessary functions for synthesizing a robust controller and quantifying the associated formal robustness guarantees. It distinguishes itself from other… ▽ More We present SySCoRe, a MATLAB toolbox that synthesizes controllers for stochastic continuous-state systems to satisfy temporal logic specifications. Starting from a system description and a co-safe temporal logic specification, SySCoRe provides all necessary functions for synthesizing a robust controller and quantifying the associated formal robustness guarantees. It distinguishes itself from other available tools by supporting nonlinear dynamics, complex co-safe temporal logic specifications over infinite horizons and model-order reduction. To achieve this, SySCoRe generates a finite-state abstraction of the provided model and performs probabilistic model checking. Then, it establishes a probabilistic coupling to the original stochastic system encoded in an approximate simulation relation, based on which a lower bound on the satisfaction probability is computed. SySCoRe provides non-trivial lower bounds for infinite-horizon properties and unbounded disturbances since its computed error does not grow linearly in the horizon of the specification. It exploits a tensor representation to facilitate the efficient computation of transition probabilities. We showcase these features on several benchmarks and compare the performance of the tool with existing tools. △ Less

Submitted 23 February, 2023; originally announced February 2023.

Correct-by-Design Control of Parametric Stochastic Systems

Authors: Oliver Schön, Birgit van Huijgevoort, Sofie Haesaert, Sadegh Soudjani

Abstract: This paper addresses the problem of computing controllers that are correct by design for safety-critical systems and can provably satisfy (complex) functional requirements. We develop new methods for models of systems subject to both stochastic and parametric uncertainties. We provide for the first time novel simulation relations for enabling correct-by-design control refinement, that are founded… ▽ More This paper addresses the problem of computing controllers that are correct by design for safety-critical systems and can provably satisfy (complex) functional requirements. We develop new methods for models of systems subject to both stochastic and parametric uncertainties. We provide for the first time novel simulation relations for enabling correct-by-design control refinement, that are founded on coupling uncertainties of stochastic systems via sub-probability measures. Such new relations are essential for constructing abstract models that are related to not only one model but to a set of parameterized models. We provide theoretical results for establishing this new class of relations and the associated closeness guarantees for both linear and nonlinear parametric systems with additive Gaussian uncertainty. The results are demonstrated on a linear model and the nonlinear model of the Van der Pol Oscillator. △ Less

Submitted 15 October, 2022; originally announced October 2022.

Comments: 8 pages, 6 figures, accepted to CDC 2022

Compositional Reinforcement Learning for Discrete-Time Stochastic Control Systems

Authors: Abolfazl Lavaei, Mateo Perez, Milad Kazemi, Fabio Somenzi, Sadegh Soudjani, Ashutosh Trivedi, Majid Zamani

Abstract: We propose a compositional approach to synthesize policies for networks of continuous-space stochastic control systems with unknown dynamics using model-free reinforcement learning (RL). The approach is based on implicitly abstracting each subsystem in the network with a finite Markov decision process with unknown transition probabilities, synthesizing a strategy for each abstract model in an assu… ▽ More We propose a compositional approach to synthesize policies for networks of continuous-space stochastic control systems with unknown dynamics using model-free reinforcement learning (RL). The approach is based on implicitly abstracting each subsystem in the network with a finite Markov decision process with unknown transition probabilities, synthesizing a strategy for each abstract model in an assume-guarantee fashion using RL, and then map** the results back over the original network with approximate optimality guarantees. We provide lower bounds on the satisfaction probability of the overall network based on those over individual subsystems. A key contribution is to leverage the convergence results for adversarial RL (minimax Q-learning) on finite stochastic arenas to provide control strategies maximizing the probability of satisfaction over the network of continuous-space systems. We consider finite-horizon properties expressed in the syntactically co-safe fragment of linear temporal logic. These properties can readily be converted into automata-based reward functions, providing scalar reward signals suitable for RL. Since such reward functions are often sparse, we supply a potential-based reward sha** technique to accelerate learning by producing dense rewards. The effectiveness of the proposed approaches is demonstrated via two physical benchmarks including regulation of a room temperature network and control of a road traffic network. △ Less

Submitted 6 August, 2022; originally announced August 2022.

Safety Barrier Certificates for Stochastic Hybrid Systems

Authors: Abolfazl Lavaei, Sadegh Soudjani, Emilio Frazzoli

Abstract: This work is concerned with the safety controller synthesis of stochastic hybrid systems, in which continuous evolutions are described by stochastic differential equations with both Brownian motions and Poisson processes, and instantaneous jumps are governed by stochastic difference equations with additive noises. Our proposed framework leverages the notion of control barrier certificates (CBC), a… ▽ More This work is concerned with the safety controller synthesis of stochastic hybrid systems, in which continuous evolutions are described by stochastic differential equations with both Brownian motions and Poisson processes, and instantaneous jumps are governed by stochastic difference equations with additive noises. Our proposed framework leverages the notion of control barrier certificates (CBC), as a discretization-free approach, to synthesize safety controllers for stochastic hybrid systems while providing safety guarantees in finite time horizons. In our proposed scheme, we first provide an augmented framework to characterize each stochastic hybrid system containing continuous evolutions and instantaneous jumps with a unified system covering both scenarios. We then introduce an augmented control barrier certificate (ACBC) for augmented systems and propose sufficient conditions to construct an ACBC based on CBC of original hybrid systems. By utilizing the constructed ACBC, we quantify upper bounds on the probability that the stochastic hybrid system reaches certain unsafe regions in a finite time horizon. The proposed approach is verified over a nonlinear case study. △ Less

Submitted 6 August, 2022; originally announced August 2022.

Comments: This work has been accepted at American Control Conference (ACC 2022)

Constructing MDP Abstractions Using Data with Formal Guarantees

Authors: Abolfazl Lavaei, Sadegh Soudjani, Emilio Frazzoli, Majid Zamani

Abstract: This paper is concerned with a data-driven technique for constructing finite Markov decision processes (MDPs) as finite abstractions of discrete-time stochastic control systems with unknown dynamics while providing formal closeness guarantees. The proposed scheme is based on notions of stochastic bisimulation functions (SBF) to capture the probabilistic distance between state trajectories of an un… ▽ More This paper is concerned with a data-driven technique for constructing finite Markov decision processes (MDPs) as finite abstractions of discrete-time stochastic control systems with unknown dynamics while providing formal closeness guarantees. The proposed scheme is based on notions of stochastic bisimulation functions (SBF) to capture the probabilistic distance between state trajectories of an unknown stochastic system and those of finite MDP. In our proposed setting, we first reformulate corresponding conditions of SBF as a robust convex program (RCP). We then propose a scenario convex program (SCP) associated to the original RCP by collecting a finite number of data from trajectories of the system. We ultimately construct an SBF between the data-driven finite MDP and the unknown stochastic system with a given confidence level by establishing a probabilistic relation between optimal values of the SCP and the RCP. We also propose two different approaches for the construction of finite MDPs from data. We illustrate the efficacy of our results over a nonlinear jet engine compressor with unknown dynamics. We construct a data-driven finite MDP as a suitable substitute of the original system to synthesize controllers maintaining the system in a safe set with some probability of satisfaction and a desirable confidence level. △ Less

Submitted 29 June, 2022; originally announced June 2022.

Comments: This work has been accepted at IEEE Control Systems Letters

Data-Driven Abstraction-Based Control Synthesis

Authors: Milad Kazemi, Rupak Majumdar, Mahmoud Salamati, Sadegh Soudjani, Ben Wooding

Abstract: This paper studies formal synthesis of controllers for continuous-space systems with unknown dynamics to satisfy requirements expressed as linear temporal logic formulas. Formal abstraction-based synthesis schemes rely on a precise mathematical model of the system to build a finite abstract model, which is then used to design a controller. The abstraction-based schemes are not applicable when the… ▽ More This paper studies formal synthesis of controllers for continuous-space systems with unknown dynamics to satisfy requirements expressed as linear temporal logic formulas. Formal abstraction-based synthesis schemes rely on a precise mathematical model of the system to build a finite abstract model, which is then used to design a controller. The abstraction-based schemes are not applicable when the dynamics of the system are unknown. We propose a data-driven approach that computes the growth bound of the system using a finite number of trajectories. The growth bound together with the sampled trajectories are then used to construct the abstraction and synthesise a controller. Our approach casts the computation of the growth bound as a robust convex optimisation program (RCP). Since the unknown dynamics appear in the optimisation, we formulate a scenario convex program (SCP) corresponding to the RCP using a finite number of sampled trajectories. We establish a sample complexity result that gives a lower bound for the number of sampled trajectories to guarantee the correctness of the growth bound computed from the SCP with a given confidence. We also provide a sample complexity result for the satisfaction of the specification on the system in closed loop with the designed controller for a given confidence. Our results are founded on estimating a bound on the Lipschitz constant of the system and provide guarantees on satisfaction of both finite and infinite-horizon specifications. We show that our data-driven approach can be readily used as a model-free abstraction refinement scheme by modifying the formulation of the growth bound and providing similar sample complexity results. The performance of our approach is shown on three case studies. △ Less

Submitted 16 June, 2022; originally announced June 2022.

Matrix Representation of Arbitrarily Controlled Quantum Gates

Authors: Marco Lewis, Sadegh Soudjani, Paolo Zuliani

Abstract: Controlled operations allow for the entanglement of quantum registers. In particular, a controlled-$U$ gate allows an operation, $U$, to be applied to the target register and entangle the results to certain values in the control register. This can be generalised by making use of the classical notion of conditional statements, where if a value (or state) satisfies some condition then a sequence of… ▽ More Controlled operations allow for the entanglement of quantum registers. In particular, a controlled-$U$ gate allows an operation, $U$, to be applied to the target register and entangle the results to certain values in the control register. This can be generalised by making use of the classical notion of conditional statements, where if a value (or state) satisfies some condition then a sequence of operations can be performed. A method is introduced to represent these generalised controlled operations that are based on classical conditional statements. Throughout examples are given to highlight the use of introduced gates. △ Less

Submitted 5 May, 2022; originally announced May 2022.

Fast Symbolic Algorithms for Omega-Regular Games under Strong Transition Fairness

Authors: Tamajit Banerjee, Rupak Majumdar, Kaushik Mallik, Anne-Kathrin Schmuck, Sadegh Soudjani

Abstract: We consider fixpoint algorithms for two-player games on graphs with $ω$-regular winning conditions, where the environment is constrained by a strong transition fairness assumption. Strong transition fairness is a widely occurring special case of strong fairness, which requires that any execution is strongly fair with respect to a specified set of live edges: whenever the source vertex of a live ed… ▽ More We consider fixpoint algorithms for two-player games on graphs with $ω$-regular winning conditions, where the environment is constrained by a strong transition fairness assumption. Strong transition fairness is a widely occurring special case of strong fairness, which requires that any execution is strongly fair with respect to a specified set of live edges: whenever the source vertex of a live edge is visited infinitely often along a play, the edge itself is traversed infinitely often along the play as well. We show that, surprisingly, strong transition fairness retains the algorithmic characteristics of the fixpoint algorithms for $ω$-regular games -- the new algorithms have the same alternation depth as the classical algorithms but invoke a new type of predecessor operator. For Rabin games with $k$ pairs, the complexity of the new algorithm is $O(n^{k+2}k!)$ symbolic steps, which is independent of the number of live edges in the strong transition fairness assumption. Further, we show that GR(1) specifications with strong transition fairness assumptions can be solved with a 3-nested fixpoint algorithm, same as the usual algorithm. In contrast, strong fairness necessarily requires increasing the alternation depth depending on the number of fairness assumptions. We get symbolic algorithms for (generalized) Rabin, parity and GR(1) objectives under strong transition fairness assumptions as well as a direct symbolic algorithm for qualitative winning in stochastic $ω$-regular games that runs in $O(n^{k+2}k!)$ symbolic steps, improving the state of the art. Finally, we have implemented a BDD-based synthesis engine based on our algorithm. We show on a set of synthetic and real benchmarks that our algorithm is scalable, parallelizable, and outperforms previous algorithms by orders of magnitude. △ Less

Submitted 23 February, 2023; v1 submitted 15 February, 2022; originally announced February 2022.

Journal ref: TheoretiCS, Volume 2 (February 24, 2023) theoretics:9088

Data-driven Safety Verification of Stochastic Systems via Barrier Certificates

Authors: Ali Salamati, Abolfazl Lavaei, Sadegh Soudjani, Majid Zamani

Abstract: In this paper, we propose a data-driven approach to formally verify the safety of (potentially) unknown discrete-time continuous-space stochastic systems. The proposed framework is based on a notion of barrier certificates together with data collected from trajectories of unknown systems. We first reformulate the barrier-based safety verification as a robust convex problem (RCP). Solving the acqui… ▽ More In this paper, we propose a data-driven approach to formally verify the safety of (potentially) unknown discrete-time continuous-space stochastic systems. The proposed framework is based on a notion of barrier certificates together with data collected from trajectories of unknown systems. We first reformulate the barrier-based safety verification as a robust convex problem (RCP). Solving the acquired RCP is hard in general because not only the state of the system lives in a continuous set, but also and more problematic, the unknown model appears in one of the constraints of RCP. Instead, we leverage a finite number of data, and accordingly, the RCP is casted as a scenario convex problem (SCP). We then relate the optimizer of the SCP to that of the RCP, and consequently, we provide a safety guarantee over the unknown stochastic system with a priori guaranteed confidence. We apply our approach to an unknown room temperature system by collecting sampled data from trajectories of the system and verify formally that temperature of the room lies in a comfort zone for a finite time horizon with a desired confidence. △ Less

Submitted 23 December, 2021; originally announced December 2021.

Data-driven verification and synthesis of stochastic systems via barrier certificates

Authors: Ali Salamati, Abolfazl Lavaei, Sadegh Soudjani, Majid Zamani

Abstract: In this work, we study verification and synthesis problems for safety specifications over unknown discrete-time stochastic systems. When a model of the system is available, barrier certificates have been successfully applied for ensuring the satisfaction of safety specifications. In this work, we formulate the computation of barrier certificates as a robust convex program (RCP). Solving the acquir… ▽ More In this work, we study verification and synthesis problems for safety specifications over unknown discrete-time stochastic systems. When a model of the system is available, barrier certificates have been successfully applied for ensuring the satisfaction of safety specifications. In this work, we formulate the computation of barrier certificates as a robust convex program (RCP). Solving the acquired RCP is hard in general because the model of the system that appears in one of the constraints of the RCP is unknown. We propose a data-driven approach that replaces the uncountable number of constraints in the RCP with a finite number of constraints by taking finitely many random samples from the trajectories of the system. We thus replace the original RCP with a scenario convex program (SCP) and show how to relate their optimizers. We guarantee that the solution of the SCP is a solution of the RCP with a priori guaranteed confidence when the number of samples is larger than a pre-computed value. This provides a lower bound on the safety probability of the original unknown system together with a controller in the case of synthesis. We also discuss an extension of our verification approach to a case where the associated robust program is non-convex and show how a similar methodology can be applied. Finally, the applicability of our proposed approach is illustrated through three case studies. △ Less

Submitted 9 September, 2023; v1 submitted 19 November, 2021; originally announced November 2021.

Formal Verification of Quantum Programs: Theory, Tools and Challenges

Authors: Marco Lewis, Sadegh Soudjani, Paolo Zuliani

Abstract: Over the past 27 years, quantum computing has seen a huge rise in interest from both academia and industry. At the current rate, quantum computers are growing in size rapidly backed up by the increase of research in the field. Significant efforts are being made to improve the reliability of quantum hardware and to develop suitable software to program quantum computers. In contrast, the verificatio… ▽ More Over the past 27 years, quantum computing has seen a huge rise in interest from both academia and industry. At the current rate, quantum computers are growing in size rapidly backed up by the increase of research in the field. Significant efforts are being made to improve the reliability of quantum hardware and to develop suitable software to program quantum computers. In contrast, the verification of quantum programs has received relatively less attention. Verifying programs is especially important in the quantum setting due to how difficult it is to program complex algorithms correctly on resource-constrained and error-prone quantum hardware. Research into creating verification frameworks for quantum programs has seen recent development, with a variety of tools implemented using a collection of theoretical ideas. This survey aims to be a short introduction into the area of formal verification of quantum programs, bringing together theory and tools developed to date. Further, this survey examines some of the challenges that the field may face in the future, namely the development of complex quantum algorithms. △ Less

Submitted 12 December, 2022; v1 submitted 4 October, 2021; originally announced October 2021.

Comments: 32 pages; changes to Sections 2, 3, 4, 5.5, Appendix

Journal ref: ACM Transactions on Quantum Computing 5, 1, Article 1 (March 2024), 35 pages

Stochastic Simulation Techniques for Inference and Sensitivity Analysis of Bayesian Attack Graphs

Authors: Isaac Matthews, Sadegh Soudjani, Aad van Moorsel

Abstract: A vulnerability scan combined with information about a computer network can be used to create an attack graph, a model of how the elements of a network could be used in an attack to reach specific states or goals in the network. These graphs can be understood probabilistically by turning them into Bayesian attack graphs, making it possible to quantitatively analyse the security of large networks.… ▽ More A vulnerability scan combined with information about a computer network can be used to create an attack graph, a model of how the elements of a network could be used in an attack to reach specific states or goals in the network. These graphs can be understood probabilistically by turning them into Bayesian attack graphs, making it possible to quantitatively analyse the security of large networks. In the event of an attack, probabilities on the graph change depending on the evidence discovered (e.g., by an intrusion detection system or knowledge of a host's activity). Since such scenarios are difficult to solve through direct computation, we discuss and compare three stochastic simulation techniques for updating the probabilities dynamically based on the evidence and compare their speed and accuracy. From our experiments we conclude that likelihood weighting is most efficient for most uses. We also consider sensitivity analysis of BAGs, to identify the most critical nodes for protection of the network and solve the uncertainty problem in the assignment of priors to nodes. Since sensitivity analysis can easily become computationally expensive, we present and demonstrate an efficient sensitivity analysis approach that exploits a quantitative relation with stochastic inference. △ Less

Submitted 18 March, 2021; originally announced March 2021.

Automated Verification and Synthesis of Stochastic Hybrid Systems: A Survey

Authors: Abolfazl Lavaei, Sadegh Soudjani, Alessandro Abate, Majid Zamani

Abstract: Stochastic hybrid systems have received significant attentions as a relevant modelling framework describing many systems, from engineering to the life sciences: they enable the study of numerous applications, including transportation networks, biological systems and chemical reaction networks, smart energy and power grids, and beyond. Automated verification and policy synthesis for stochastic hybr… ▽ More Stochastic hybrid systems have received significant attentions as a relevant modelling framework describing many systems, from engineering to the life sciences: they enable the study of numerous applications, including transportation networks, biological systems and chemical reaction networks, smart energy and power grids, and beyond. Automated verification and policy synthesis for stochastic hybrid systems can be inherently challenging: this is due to the heterogeneity of their dynamics (presence of continuous and discrete components), the presence of uncertainty, and in some applications the large dimension of state and input sets. Over the past few years, a few hundred articles have investigated these models, and developed diverse and powerful approaches to mitigate difficulties encountered in the analysis and synthesis of such complex stochastic systems. In this survey, we overview the most recent results in the literature and discuss different approaches, including (in)finite abstractions, verification and synthesis for temporal logic specifications, stochastic similarity relations, (control) barrier certificates, compositional techniques, and a selection of results on continuous-time stochastic systems; we finally survey recently developed software tools that implement the discussed approaches. Throughout the manuscript we discuss a few open topics to be considered as potential future research directions: we hope that this survey will guide younger researchers through a comprehensive understanding of the various challenges, tools, and solutions in this enticing and rich scientific area. △ Less

Submitted 10 March, 2022; v1 submitted 19 January, 2021; originally announced January 2021.

Symbolic Control for Stochastic Systems via Finite Parity Games

Authors: Rupak Majumdar, Kaushik Mallik, Anne-Kathrin Schmuck, Sadegh Soudjani

Abstract: We consider the problem of computing the maximal probability of satisfying an omega-regular specification for stochastic nonlinear systems evolving in discrete time. The problem reduces, after automata-theoretic constructions, to finding the maximal probability of satisfying a parity condition on a (possibly hybrid) state space. While characterizing the exact satisfaction probability is open, we s… ▽ More We consider the problem of computing the maximal probability of satisfying an omega-regular specification for stochastic nonlinear systems evolving in discrete time. The problem reduces, after automata-theoretic constructions, to finding the maximal probability of satisfying a parity condition on a (possibly hybrid) state space. While characterizing the exact satisfaction probability is open, we show that a lower bound on this probability can be obtained by (I) computing an under-approximation of the qualitative winning region, i.e., states from which the parity condition can be enforced almost surely, and (II) computing the maximal probability of reaching this qualitative winning region. The heart of our approach is a technique to symbolically compute the under-approximation of the qualitative winning region in step (I) via a finite-state abstraction of the original system as a 2.5-player parity game. Our abstraction procedure uses only the support of the probabilistic evolution; it does not use precise numerical transition probabilities. We prove that the winning set in the abstract 2.5-player game induces an under-approximation of the qualitative winning region in the original synthesis problem, along with a policy to solve it. By combining these contributions with (a) a symbolic fixpoint algorithm to solve 2.5-player games and (b) existing techniques for reachability policy synthesis in stochastic nonlinear systems, we get an abstraction-based algorithm for finding a lower bound on the maximal satisfaction probability. We have implemented the abstraction-based algorithm in Mascot-SDS (Majumdar et al., 2020), where we combined the outlined abstraction step with our recent tool FairSyn. We evaluated our implementation on the nonlinear model of a perturbed bistable switch from the literature. We outperform a recently proposed tool for solving this problem by a large margin. △ Less

Submitted 29 September, 2022; v1 submitted 4 January, 2021; originally announced January 2021.

Comments: 54 pages, under review in the Elsevier journal of Nonlinear Analysis: Hybrid Systems

ACM Class: G.3; I.1.2; I.1.4

Compositional Construction of Control Barrier Functions for Continuous-Time Stochastic Hybrid Systems

Authors: Ameneh Nejati, Sadegh Soudjani, Majid Zamani

Abstract: In this work, we propose a compositional framework for the construction of control barrier functions for networks of continuous-time stochastic hybrid systems enforcing complex logic specifications expressed by finite-state automata. The proposed scheme is based on a notion of so-called pseudo-barrier functions computed for subsystems, by employing which one can synthesize hybrid controllers for i… ▽ More In this work, we propose a compositional framework for the construction of control barrier functions for networks of continuous-time stochastic hybrid systems enforcing complex logic specifications expressed by finite-state automata. The proposed scheme is based on a notion of so-called pseudo-barrier functions computed for subsystems, by employing which one can synthesize hybrid controllers for interconnected systems enforcing complex specifications over a finite-time horizon. Particularly, we first leverage sufficient small-gain type conditions to compositionally construct control barrier functions for interconnected systems based on the corresponding pseudo-barrier functions computed for subsystems. Then, using the constructed control barrier functions, we provide probabilistic guarantees on the satisfaction of given complex specifications in a bounded time horizon. In this respect, we decompose the given complex specification to simpler reachability tasks based on automata representing the complements of original finite-state automata. We then provide systematic approaches to solve those simpler reachability tasks by computing corresponding pseudo-barrier functions. Two different systematic techniques are provided based on (i) the sum-of-squares (SOS) optimization program and (ii) counter-example guided inductive synthesis (CEGIS) to search for pseudo-barrier functions of subsystems while synthesizing local controllers. We demonstrate the effectiveness of our proposed results by applying them to a fully-interconnected Kuramoto network of 100 nonlinear oscillators with Markovian switching signals. △ Less

Submitted 5 June, 2022; v1 submitted 14 December, 2020; originally announced December 2020.

On Decidability of Time-bounded Reachability in CTMDPs

Authors: Rupak Majumdar, Mahmoud Salamati, Sadegh Soudjani

Abstract: We consider the time-bounded reachability problem for continuous-time Markov decision processes. We show that the problem is decidable subject to Schanuel's conjecture. Our decision procedure relies on the structure of optimal policies and the conditional decidability (under Schanuel's conjecture) of the theory of reals extended with exponential and trigonometric functions over bounded domains. We… ▽ More We consider the time-bounded reachability problem for continuous-time Markov decision processes. We show that the problem is decidable subject to Schanuel's conjecture. Our decision procedure relies on the structure of optimal policies and the conditional decidability (under Schanuel's conjecture) of the theory of reals extended with exponential and trigonometric functions over bounded domains. We further show that any unconditional decidability result would imply unconditional decidability of the bounded continuous Skolem problem, or equivalently, the problem of checking if an exponential polynomial has a non-tangential zero in a bounded interval. We note that the latter problems are also decidable subject to Schanuel's conjecture but finding unconditional decision procedures remain longstanding open problems. △ Less

Submitted 9 June, 2020; originally announced June 2020.

Cyclic Bayesian Attack Graphs: A Systematic Computational Approach

Authors: Isaac Matthews, John Mace, Sadegh Soudjani, Aad van Moorsel

Abstract: Attack graphs are commonly used to analyse the security of medium-sized to large networks. Based on a scan of the network and likelihood information of vulnerabilities, attack graphs can be transformed into Bayesian Attack Graphs (BAGs). These BAGs are used to evaluate how security controls affect a network and how changes in topology affect security. A challenge with these automatically generated… ▽ More Attack graphs are commonly used to analyse the security of medium-sized to large networks. Based on a scan of the network and likelihood information of vulnerabilities, attack graphs can be transformed into Bayesian Attack Graphs (BAGs). These BAGs are used to evaluate how security controls affect a network and how changes in topology affect security. A challenge with these automatically generated BAGs is that cycles arise naturally, which make it impossible to use Bayesian network theory to calculate state probabilities. In this paper we provide a systematic approach to analyse and perform computations over cyclic Bayesian attack graphs. %thus providing a generic approach to handle cycles as well as unifying the theory of Bayesian attack graphs. Our approach first formally introduces two commonly used versions of Bayesian attack graphs and compares their expressiveness. We then present an interpretation of Bayesian attack graphs based on combinational logic circuits, which facilitates an intuitively attractive systematic treatment of cycles. We prove properties of the associated logic circuit and present an algorithm that computes state probabilities without altering the attack graphs (e.g., remove an arc to remove a cycle). Moreover, our algorithm deals seamlessly with all cycles without the need to identify their types. A set of experiments using synthetically created networks demonstrates the scalability of the algorithm on computer networks with hundreds of machines, each with multiple vulnerabilities. △ Less

Submitted 13 May, 2020; originally announced May 2020.

AMYTISS: Parallelized Automated Controller Synthesis for Large-Scale Stochastic Systems

Authors: Abolfazl Lavaei, Mahmoud Khaled, Sadegh Soudjani, Majid Zamani

Abstract: In this paper, we propose a software tool, called AMYTISS, implemented in C++/OpenCL, for designing correct-by-construction controllers for large-scale discrete-time stochastic systems. This tool is employed to (i) build finite Markov decision processes (MDPs) as finite abstractions of given original systems, and (ii) synthesize controllers for the constructed finite MDPs satisfying bounded-time h… ▽ More In this paper, we propose a software tool, called AMYTISS, implemented in C++/OpenCL, for designing correct-by-construction controllers for large-scale discrete-time stochastic systems. This tool is employed to (i) build finite Markov decision processes (MDPs) as finite abstractions of given original systems, and (ii) synthesize controllers for the constructed finite MDPs satisfying bounded-time high-level properties including safety, reachability and reach-avoid specifications. In AMYTISS, scalable parallel algorithms are designed such that they support the parallel execution within CPUs, GPUs and hardware accelerators (HWAs). Unlike all existing tools for stochastic systems, AMYTISS can utilize high-performance computing (HPC) platforms and cloud-computing services to mitigate the effects of the state-explosion problem, which is always present in analyzing large-scale stochastic systems. We benchmark AMYTISS against the most recent tools in the literature using several physical case studies including robot examples, room temperature and road traffic networks. We also apply our algorithms to a 3-dimensional autonomous vehicle and 7-dimensional nonlinear model of a BMW 320i car by synthesizing an autonomous parking controller. △ Less

Submitted 13 May, 2020; originally announced May 2020.

Comments: This work is accepted at the 32nd International Conference on Computer-Aided Verification (CAV) as a tool paper

Data-Driven Verification under Signal Temporal Logic Constraints

Authors: Ali Salamati, Sadegh Soudjani, Majid Zamani

Abstract: We consider systems under uncertainty whose dynamics are partially unknown. Our aim is to study satisfaction of temporal logic properties by trajectories of such systems. We express these properties as signal temporal logic formulas and check if the probability of satisfying the property is at least a given threshold. Since the dynamics are parameterized and partially unknown, we collect data from… ▽ More We consider systems under uncertainty whose dynamics are partially unknown. Our aim is to study satisfaction of temporal logic properties by trajectories of such systems. We express these properties as signal temporal logic formulas and check if the probability of satisfying the property is at least a given threshold. Since the dynamics are parameterized and partially unknown, we collect data from the system and employ Bayesian inference techniques to associate a confidence value to the satisfaction of the property. The main novelty of our approach is to combine both data-driven and model-based techniques in order to have a two-layer probabilistic reasoning over the behavior of the system: one layer is related to the stochastic noise inside the system and the next layer is related to the noisy data collected from the system. We provide approximate algorithms for computing the confidence for linear dynamical systems. △ Less

Submitted 8 May, 2020; originally announced May 2020.

Formal Controller Synthesis for Frequency Regulation Utilising Electric Vehicles

Authors: Ben Wooding, Vahid Vahidinasab, Sadegh Soudjani

Abstract: In this paper, we propose a formal controller synthesis approach for integrating a population of plug-in electric vehicles in frequency regulation of power systems. This approach is the first application of formal methods to the smart grids in particular to the frequency regulation of power systems. We design and simulate a novel symbolic controller for the Great Britain power system. The proposed… ▽ More In this paper, we propose a formal controller synthesis approach for integrating a population of plug-in electric vehicles in frequency regulation of power systems. This approach is the first application of formal methods to the smart grids in particular to the frequency regulation of power systems. We design and simulate a novel symbolic controller for the Great Britain power system. The proposed controller enhances the frequency response behaviour of the system when encountered with a large outage event. The symbolic controller guarantees the settlement of the after-event's frequency in the specified safe interval and ensures other requirements on the frequency are met. △ Less

Submitted 8 May, 2020; originally announced May 2020.

Comments: 9 Pages, 12 figures, 2 tables

Formal Policy Synthesis for Continuous-Space Systems via Reinforcement Learning

Authors: Milad Kazemi, Sadegh Soudjani

Abstract: This paper studies satisfaction of temporal properties on unknown stochastic processes that have continuous state spaces. We show how reinforcement learning (RL) can be applied for computing policies that are finite-memory and deterministic using only the paths of the stochastic process. We address properties expressed in linear temporal logic (LTL) and use their automaton representation to give a… ▽ More This paper studies satisfaction of temporal properties on unknown stochastic processes that have continuous state spaces. We show how reinforcement learning (RL) can be applied for computing policies that are finite-memory and deterministic using only the paths of the stochastic process. We address properties expressed in linear temporal logic (LTL) and use their automaton representation to give a path-dependent reward function maximised via the RL algorithm. We develop the required assumptions and theories for the convergence of the learned policy to the optimal policy in the continuous state space. To improve the performance of the learning on the constructed sparse reward function, we propose a sequential learning procedure based on a sequence of labelling functions obtained from the positive normal form of the LTL specification. We use this procedure to guide the RL algorithm towards a policy that converges to an optimal policy under suitable assumptions on the process. We demonstrate the approach on a 4-dim cart-pole system and 6-dim boat driving problem. △ Less

Submitted 27 September, 2020; v1 submitted 4 May, 2020; originally announced May 2020.

Comments: This is the extended version of the paper accepted in the 16th International Conference on integrated Formal Methods (iFM)

Compositional Construction of Control Barrier Certificates for Large-Scale Stochastic Switched Systems

Authors: Ameneh Nejati, Sadegh Soudjani, Majid Zamani

Abstract: In this paper, we propose a compositional framework for the construction of control barrier certificates for large-scale stochastic switched systems accepting multiple control barrier certificates with some dwell-time conditions. The proposed scheme is based on a notion of so-called augmented pseudo-barrier certificates computed for each switched subsystem, using which one can compositionally synt… ▽ More In this paper, we propose a compositional framework for the construction of control barrier certificates for large-scale stochastic switched systems accepting multiple control barrier certificates with some dwell-time conditions. The proposed scheme is based on a notion of so-called augmented pseudo-barrier certificates computed for each switched subsystem, using which one can compositionally synthesize state-feedback controllers for interconnected systems enforcing safety specifications over a finite-time horizon. In particular, we first leverage sufficient max-type small-gain conditions to compositionally construct augmented control barrier certificates for interconnected systems based on the corresponding augmented pseudo-barrier certificates of subsystems. Then we quantify upper bounds on exit probabilities - the probability that an interconnected system reaches certain unsafe regions - in a finite-time horizon using the constructed augmented barrier certificates. We employ a technique based on a counter-example guided inductive synthesis (CEGIS) approach to search for control barrier certificates of each mode while synthesizing safety controllers providing switching signals. We demonstrate our proposed results by applying them first to a room temperature network containing 1000 rooms. Finally, we apply our techniques to a network of 500 switched subsystems (totally 1000 dimensions) accepting multiple barrier certificates with a dwell-time condition, and provide upper bounds on the probability that the interconnected system reaches some unsafe region in a finite-time horizon. △ Less

Submitted 4 May, 2020; originally announced May 2020.

Comments: This work is accepted at the IEEE Control Systems Letters. arXiv admin note: text overlap with arXiv:1912.11481

Formal Controller Synthesis for Continuous-Space MDPs via Model-Free Reinforcement Learning

Authors: Abolfazl Lavaei, Fabio Somenzi, Sadegh Soudjani, Ashutosh Trivedi, Majid Zamani

Abstract: A novel reinforcement learning scheme to synthesize policies for continuous-space Markov decision processes (MDPs) is proposed. This scheme enables one to apply model-free, off-the-shelf reinforcement learning algorithms for finite MDPs to compute optimal strategies for the corresponding continuous-space MDPs without explicitly constructing the finite-state abstraction. The proposed approach is ba… ▽ More A novel reinforcement learning scheme to synthesize policies for continuous-space Markov decision processes (MDPs) is proposed. This scheme enables one to apply model-free, off-the-shelf reinforcement learning algorithms for finite MDPs to compute optimal strategies for the corresponding continuous-space MDPs without explicitly constructing the finite-state abstraction. The proposed approach is based on abstracting the system with a finite MDP (without constructing it explicitly) with unknown transition probabilities, synthesizing strategies over the abstract MDP, and then map** the results back over the concrete continuous-space MDP with approximate optimality guarantees. The properties of interest for the system belong to a fragment of linear temporal logic, known as syntactically co-safe linear temporal logic (scLTL), and the synthesis requirement is to maximize the probability of satisfaction within a given bounded time horizon. A key contribution of the paper is to leverage the classical convergence results for reinforcement learning on finite MDPs and provide control strategies maximizing the probability of satisfaction over unknown, continuous-space MDPs while providing probabilistic closeness guarantees. Automata-based reward functions are often sparse; we present a novel potential-based reward sha** technique to produce dense rewards to speed up learning. The effectiveness of the proposed approach is demonstrated by applying it to three physical benchmarks concerning the regulation of a room's temperature, control of a road traffic cell, and of a 7-dimensional nonlinear model of a BMW 320i car. △ Less

Submitted 2 March, 2020; originally announced March 2020.

Comments: This work is accepted at the 11th ACM/IEEE Conference on Cyber-Physical Systems (ICCPS)

Compositional Abstraction-based Synthesis for Networks of Stochastic Switched Systems

Authors: Abolfazl Lavaei, Sadegh Soudjani, Majid Zamani

Abstract: In this paper, we provide a compositional approach for constructing finite abstractions (a.k.a. finite Markov decision processes (MDPs)) of interconnected discrete-time stochastic switched systems. The proposed framework is based on a notion of stochastic simulation functions, using which one can employ an abstract system as a substitution of the original one in the controller design process with… ▽ More In this paper, we provide a compositional approach for constructing finite abstractions (a.k.a. finite Markov decision processes (MDPs)) of interconnected discrete-time stochastic switched systems. The proposed framework is based on a notion of stochastic simulation functions, using which one can employ an abstract system as a substitution of the original one in the controller design process with guaranteed error bounds on their output trajectories. To this end, we first provide probabilistic closeness guarantees between the interconnection of stochastic switched subsystems and that of their finite abstractions via stochastic simulation functions. We then leverage sufficient small-gain type conditions to show compositionality results of this work. Afterwards, we show that under standard assumptions ensuring incremental input-to-state stability of switched systems (i.e., existence of common incremental Lyapunov functions, or multiple incremental Lyapunov functions with dwell-time), one can construct finite MDPs for the general setting of nonlinear stochastic switched systems. We also propose an approach to construct finite MDPs for a particular class of nonlinear stochastic switched systems. To demonstrate the effectiveness of our proposed results, we first apply our approaches to a road traffic network in a circular cascade ring composed of 200 cells, and construct compositionally a finite MDP of the network. We employ the constructed finite abstractions as substitutes to compositionally synthesize policies kee** the density of the traffic lower than 20 vehicles per cell. We then apply our proposed techniques to a fully interconnected network of 500 nonlinear subsystems (totally 1000 dimensions), and construct their finite MDPs with guaranteed error bounds. We compare our proposed results with those available in the literature. △ Less

Submitted 24 December, 2019; originally announced December 2019.

Comments: This work is accepted as a regular paper at Automatica. arXiv admin note: text overlap with arXiv:1902.01223, arXiv:1808.00893

Symbolic Controller Synthesis for Büchi Specifications on Stochastic Systems

Authors: Rupak Majumdar, Kaushik Mallik, Sadegh Soudjani

Abstract: We consider the policy synthesis problem for continuous-state controlled Markov processes evolving in discrete time, when the specification is given as a Büchi condition (visit a set of states infinitely often). We decompose computation of the maximal probability of satisfying the Büchi condition into two steps. The first step is to compute the maximal qualitative winning set, from where the Büchi… ▽ More We consider the policy synthesis problem for continuous-state controlled Markov processes evolving in discrete time, when the specification is given as a Büchi condition (visit a set of states infinitely often). We decompose computation of the maximal probability of satisfying the Büchi condition into two steps. The first step is to compute the maximal qualitative winning set, from where the Büchi condition can be enforced with probability one. The second step is to find the maximal probability of reaching the already computed qualitative winning set. In contrast with finite-state models, we show that such a computation only gives a lower bound on the maximal probability where the gap can be non-zero. In this paper we focus on approximating the qualitative winning set, while pointing out that the existing approaches for unbounded reachability computation can solve the second step. We provide an abstraction-based technique to approximate the qualitative winning set by simultaneously using an over- and under-approximation of the probabilistic transition relation. Since we are interested in qualitative properties, the abstraction is non-probabilistic; instead, the probabilistic transitions are assumed to be under the control of a (fair) adversary. Thus, we reduce the original policy synthesis problem to a Büchi game under a fairness assumption and characterize upper and lower bounds on winning sets as nested fixed point expressions in the $μ$-calculus. This characterization immediately provides a symbolic algorithm scheme. Further, a winning strategy computed on the abstract game can be refined to a policy on the controlled Markov process. We describe a concrete abstraction procedure and demonstrate our algorithm on two case studies. △ Less

Submitted 14 February, 2020; v1 submitted 26 October, 2019; originally announced October 2019.

Journal ref: 23rd ACM International Conference on Hybrid Systems: Computation and Control (HSCC '20), April 22--24, 2020, Sydney, NSW, Australia

A Lyapunov Approach for Time Bounded Reachability of CTMCs and CTMDPs

Authors: Mahmoud Salamati, Sadegh Soudjani, Rupak Majumdar

Abstract: Time bounded reachability is a fundamental problem in model checking continuous-time Markov chains (CTMCs) and Markov decision processes (CTMDPs) for specifications in continuous stochastic logics. It can be computed by numerically solving a characteristic linear dynamical system but the procedure is computationally expensive. We take a control-theoretic approach and propose a reduction technique… ▽ More Time bounded reachability is a fundamental problem in model checking continuous-time Markov chains (CTMCs) and Markov decision processes (CTMDPs) for specifications in continuous stochastic logics. It can be computed by numerically solving a characteristic linear dynamical system but the procedure is computationally expensive. We take a control-theoretic approach and propose a reduction technique that finds another dynamical system of lower dimension (number of variables), such that numerically solving the reduced dynamical system provides an approximation to the solution of the original system with guaranteed error bounds. Our technique generalises lumpability (or probabilistic bisimulation) to a quantitative setting. Our main result is a Lyapunov function characterisation of the difference in the trajectories of the two dynamics that depends on the initial mismatch and exponentially decreases over time. In particular, the Lyapunov function enables us to compute an error bound between the two dynamics as well as a convergence rate. Finally, we show that the search for the reduced dynamics can be computed in polynomial time using a Schur decomposition of the transition matrix. This enables us to efficiently solve the reduced dynamical system by computing the exponential of an upper-triangular matrix characterising the reduced dynamics. For CTMDPs, we generalise our approach using piecewise quadratic Lyapunov functions for switched affine dynamical systems. We synthesise a policy for the CTMDP via its reduced-order switched system that guarantees the time bounded reachability probability lies above a threshold. We provide error bounds that depend on the minimum dwell time of the policy. We demonstrate the technique on examples from queueing networks, for which lumpability does not produce any state space reduction but our technique synthesises policies using reduced version of the model. △ Less

Submitted 6 January, 2020; v1 submitted 13 September, 2019; originally announced September 2019.

Comments: To be published at in ACM Transactions on Modeling and Performance Evaluation of Computing Systems (TOMPECS)

Compositional Abstraction-based Synthesis of General MDPs via Approximate Probabilistic Relations

Authors: Abolfazl Lavaei, Sadegh Soudjani, Majid Zamani

Abstract: We propose a compositional approach for constructing abstractions of general Markov decision processes using approximate probabilistic relations. The abstraction framework is based on the notion of $δ$-lifted relations, using which one can quantify the distance in probability between the interconnected gMDPs and that of their abstractions. This new approximate relation unifies compositionality res… ▽ More We propose a compositional approach for constructing abstractions of general Markov decision processes using approximate probabilistic relations. The abstraction framework is based on the notion of $δ$-lifted relations, using which one can quantify the distance in probability between the interconnected gMDPs and that of their abstractions. This new approximate relation unifies compositionality results in the literature by incorporating the dependencies between state transitions explicitly and by allowing abstract models to have either finite or infinite state spaces. Accordingly, one can leverage the proposed results to perform analysis and synthesis over abstract models, and then carry the results over concrete ones. To this end, we first propose our compositionality results using the new approximate probabilistic relation which is based on lifting. We then focus on a class of stochastic nonlinear dynamical systems and construct their abstractions using both model order reduction and space discretization in a unified framework. We provide conditions for simultaneous existence of relations incorporating the structure of the network. Finally, we demonstrate the effectiveness of the proposed results by considering a network of four nonlinear dynamical subsystems (together 12 dimensions) and constructing finite abstractions from their reduced-order versions (together 4 dimensions) in a unified compositional framework. We benchmark our results against the compositional abstraction techniques that construct both infinite abstractions (reduced-order models) and finite MDPs in two consecutive steps. We show that our approach is much less conservative than the ones available in the literature. △ Less

Submitted 7 June, 2019; originally announced June 2019.

Formal Synthesis of Stochastic Systems via Control Barrier Certificates

Authors: Pushpak Jagtap, Sadegh Soudjani, Majid Zamani

Abstract: This paper focuses on synthesizing control policies for discrete-time stochastic control systems together with a lower bound on the probability that the systems satisfy the complex temporal properties. The desired properties of the system are expressed as linear temporal logic (LTL) specifications over finite traces. In particular, our approach decomposes the given specification into simpler reach… ▽ More This paper focuses on synthesizing control policies for discrete-time stochastic control systems together with a lower bound on the probability that the systems satisfy the complex temporal properties. The desired properties of the system are expressed as linear temporal logic (LTL) specifications over finite traces. In particular, our approach decomposes the given specification into simpler reachability tasks based on its automata representation. We then propose the use of so-called \emph{control barrier certificate} to solve those simpler reachability tasks along with computing the corresponding controllers and probability bounds. Finally, we combine those controllers to obtain a hybrid control policy solving the considered problem. Under some assumptions, we also provide two systematic approaches for uncountable and finite input sets to search for control barrier certificates. We demonstrate the effectiveness of the proposed approach on a room temperature control and lane-kee** of a vehicle modeled as a four-dimensional single-track kinematic model. We compare our results with the discretization-based methods in the literature. △ Less

Submitted 5 August, 2020; v1 submitted 11 May, 2019; originally announced May 2019.

Comments: 22 pages, 11 figures. arXiv admin note: text overlap with arXiv:1807.00064

Journal ref: IEEE Transaction on Automatic Control, 2020

Compositional Abstraction of Large-Scale Stochastic Systems: A Relaxed Dissipativity Approach

Authors: Abolfazl Lavaei, Sadegh Soudjani, Majid Zamani

Abstract: In this paper, we propose a compositional approach for the construction of finite abstractions (a.k.a. finite Markov decision processes (MDPs)) for networks of discrete-time stochastic control subsystems that are not necessarily stabilizable. The proposed approach leverages the interconnection topology and a notion of finite-step stochastic storage functions, that describes joint dissipativity-typ… ▽ More In this paper, we propose a compositional approach for the construction of finite abstractions (a.k.a. finite Markov decision processes (MDPs)) for networks of discrete-time stochastic control subsystems that are not necessarily stabilizable. The proposed approach leverages the interconnection topology and a notion of finite-step stochastic storage functions, that describes joint dissipativity-type properties of subsystems and their abstractions, and establishes a finite-step stochastic simulation function as a relation between the network and its abstraction. To this end, we first develop a new type of compositionality conditions which is less conservative than the existing ones. In particular, using a relaxation via a finite-step stochastic simulation function, it is possible to construct finite abstractions such that stabilizability of each subsystem is not necessarily required. We then propose an approach to construct finite MDPs together with their corresponding finite-step storage functions for general discrete-time stochastic control systems satisfying an incremental passivablity property. We also construct finite MDPs for a particular class of nonlinear stochastic control systems. To demonstrate the effectiveness of the proposed results, we apply our results on three different case studies. △ Less

Submitted 11 February, 2020; v1 submitted 1 February, 2019; originally announced February 2019.

Comments: This work is accepted at Nonlinear Analysis: Hybrid Systems. arXiv admin note: text overlap with arXiv:1712.07793

Perception-in-the-Loop Adversarial Examples

Authors: Mahmoud Salamati, Sadegh Soudjani, Rupak Majumdar

Abstract: We present a scalable, black box, perception-in-the-loop technique to find adversarial examples for deep neural network classifiers. Black box means that our procedure only has input-output access to the classifier, and not to the internal structure, parameters, or intermediate confidence values. Perception-in-the-loop means that the notion of proximity between inputs can be directly queried from… ▽ More We present a scalable, black box, perception-in-the-loop technique to find adversarial examples for deep neural network classifiers. Black box means that our procedure only has input-output access to the classifier, and not to the internal structure, parameters, or intermediate confidence values. Perception-in-the-loop means that the notion of proximity between inputs can be directly queried from human participants rather than an arbitrarily chosen metric. Our technique is based on covariance matrix adaptation evolution strategy (CMA-ES), a black box optimization approach. CMA-ES explores the search space iteratively in a black box manner, by generating populations of candidates according to a distribution, choosing the best candidates according to a cost function, and updating the posterior distribution to favor the best candidates. We run CMA-ES using human participants to provide the fitness function, using the insight that the choice of best candidates in CMA-ES can be naturally modeled as a perception task: pick the top $k$ inputs perceptually closest to a fixed input. We empirically demonstrate that finding adversarial examples is feasible using small populations and few iterations. We compare the performance of CMA-ES on the MNIST benchmark with other black-box approaches using $L_p$ norms as a cost function, and show that it performs favorably both in terms of success in finding adversarial examples and in minimizing the distance between the original and the adversarial input. In experiments on the MNIST, CIFAR10, and GTSRB benchmarks, we demonstrate that CMA-ES can find perceptually similar adversarial inputs with a small number of iterations and small population sizes when using perception-in-the-loop. Finally, we show that networks trained specifically to be robust against $L_\infty$ norm can still be susceptible to perceptually similar adversarial examples. △ Less

Submitted 21 January, 2019; originally announced January 2019.

Comments: 13 pages, 13 figures, 6 tables

Automated Synthesis of Safe Digital Controllers for Sampled-Data Stochastic Nonlinear Systems

Authors: Fedor Shmarov, Sadegh Soudjani, Nicola Paoletti, Ezio Bartocci, Shan Lin, Scott A. Smolka, Paolo Zuliani

Abstract: We present a new method for the automated synthesis of digital controllers with formal safety guarantees for systems with nonlinear dynamics, noisy output measurements, and stochastic disturbances. Our method derives digital controllers such that the corresponding closed-loop system, modeled as a sampled-data stochastic control system, satisfies a safety specification with probability above a give… ▽ More We present a new method for the automated synthesis of digital controllers with formal safety guarantees for systems with nonlinear dynamics, noisy output measurements, and stochastic disturbances. Our method derives digital controllers such that the corresponding closed-loop system, modeled as a sampled-data stochastic control system, satisfies a safety specification with probability above a given threshold. The proposed synthesis method alternates between two steps: generation of a candidate controller pc, and verification of the candidate. pc is found by maximizing a Monte Carlo estimate of the safety probability, and by using a non-validated ODE solver for simulating the system. Such a candidate is therefore sub-optimal but can be generated very rapidly. To rule out unstable candidate controllers, we prove and utilize Lyapunov's indirect method for instability of sampled-data nonlinear systems. In the subsequent verification step, we use a validated solver based on SMT (Satisfiability Modulo Theories) to compute a numerically and statistically valid confidence interval for the safety probability of pc. If the probability so obtained is not above the threshold, we expand the search space for candidates by increasing the controller degree. We evaluate our technique on three case studies: an artificial pancreas model, a powertrain control model, and a quadruple-tank process. △ Less

Submitted 10 January, 2019; originally announced January 2019.

Comments: 12 pages, 4 figures, 4 tables

MSC Class: 68N30