Showing 1–9 of 9 results for author: Banse, A

A Cantor-Kantorovich Metric Between Markov Decision Processes with Application to Transfer Learning

Authors: Adrien Banse, Venkatraman Renganathan, Raphaël M. Jungers

Abstract: We extend the notion of Cantor-Kantorovich distance between Markov chains introduced by (Banse et al., 2023) in the context of Markov Decision Processes (MDPs). The proposed metric is well-defined and can be efficiently approximated given a finite horizon. Then, we provide numerical evidences that the latter metric can lead to interesting applications in the field of reinforcement learning. In par… ▽ More We extend the notion of Cantor-Kantorovich distance between Markov chains introduced by (Banse et al., 2023) in the context of Markov Decision Processes (MDPs). The proposed metric is well-defined and can be efficiently approximated given a finite horizon. Then, we provide numerical evidences that the latter metric can lead to interesting applications in the field of reinforcement learning. In particular, we show that it could be used for forecasting the performance of transfer learning algorithms. △ Less

Submitted 11 July, 2024; originally announced July 2024.

Comments: Presented at the 26th International Symposium on Mathematical Theory of Networks and Systems (Cambridge, UK)

Data-driven memory-dependent abstractions of dynamical systems via a Cantor-Kantorovich metric

Authors: Adrien Banse, Licio Romao, Alessandro Abate, Raphaël M. Jungers

Abstract: Abstractions of dynamical systems enable their verification and the design of feedback controllers using simpler, usually discrete, models. In this paper, we propose a data-driven abstraction mechanism based on a novel metric between Markov models. Our approach is based purely on observing output labels of the underlying dynamics, thus opening the road for a fully data-driven approach to construct… ▽ More Abstractions of dynamical systems enable their verification and the design of feedback controllers using simpler, usually discrete, models. In this paper, we propose a data-driven abstraction mechanism based on a novel metric between Markov models. Our approach is based purely on observing output labels of the underlying dynamics, thus opening the road for a fully data-driven approach to construct abstractions. Another feature of the proposed approach is the use of memory to better represent the dynamics in a given region of the state space. We show through numerical examples the usefulness of the proposed methodology. △ Less

Submitted 14 May, 2024; originally announced May 2024.

Comments: Submitted to IEEE Transactions on Automatic Control

Dionysos.jl: a Modular Platform for Smart Symbolic Control

Authors: Julien Calbert, Adrien Banse, Benoît Legat, Raphaël M. Jungers

Abstract: We introduce Dionysos.jl, a modular package for solving optimal control problems for complex dynamical systems using state-of-the-art and experimental techniques from symbolic control, optimization, and learning. More often than not with Cyber-Physical systems, the only sensible way of develo** a controller is by discretizing the different variables, thus transforming the control task into a pur… ▽ More We introduce Dionysos.jl, a modular package for solving optimal control problems for complex dynamical systems using state-of-the-art and experimental techniques from symbolic control, optimization, and learning. More often than not with Cyber-Physical systems, the only sensible way of develo** a controller is by discretizing the different variables, thus transforming the control task into a purely combinatorial problem on a finite-state mathematical object, called an abstraction of this system. Although this approach offers a safety-critical framework, the available techniques suffer important scalability issues. In order to render these techniques practical, it is necessary to construct smarter abstractions that differ from classical techniques by partitioning the state-space in a non trivial way. △ Less

Submitted 22 April, 2024; originally announced April 2024.

Comments: Submitted to the Proceedings of the JuliaCon Conferences

Federated Learning with Differential Privacy

Authors: Adrien Banse, Jan Kreischer, Xavier Oliva i Jürgens

Abstract: Federated learning (FL), as a type of distributed machine learning, is capable of significantly preserving client's private data from being shared among different parties. Nevertheless, private information can still be divulged by analyzing uploaded parameter weights from clients. In this report, we showcase our empirical benchmark of the effect of the number of clients and the addition of differe… ▽ More Federated learning (FL), as a type of distributed machine learning, is capable of significantly preserving client's private data from being shared among different parties. Nevertheless, private information can still be divulged by analyzing uploaded parameter weights from clients. In this report, we showcase our empirical benchmark of the effect of the number of clients and the addition of differential privacy (DP) mechanisms on the performance of the model on different types of data. Our results show that non-i.i.d and small datasets have the highest decrease in performance in a distributed and differentially private setting. △ Less

Submitted 3 February, 2024; originally announced February 2024.

Comments: Machine Learning (ML) & Federated Learning (FL); 4 pages, 3 figures

ACM Class: I.2.11

Data-driven abstractions via adaptive refinements and a Kantorovich metric [extended version]

Authors: Adrien Banse, Licio Romao, Alessandro Abate, Raphaël M. Jungers

Abstract: We introduce an adaptive refinement procedure for smart, and scalable abstraction of dynamical systems. Our technique relies on partitioning the state space depending on the observation of future outputs. However, this knowledge is dynamically constructed in an adaptive, asymmetric way. In order to learn the optimal structure, we define a Kantorovich-inspired metric between Markov chains, and we u… ▽ More We introduce an adaptive refinement procedure for smart, and scalable abstraction of dynamical systems. Our technique relies on partitioning the state space depending on the observation of future outputs. However, this knowledge is dynamically constructed in an adaptive, asymmetric way. In order to learn the optimal structure, we define a Kantorovich-inspired metric between Markov chains, and we use it as a loss function. Our technique is prone to data-driven frameworks, but not restricted to. We also study properties of the above mentioned metric between Markov chains, which we believe could be of application for wider purpose. We propose an algorithm to approximate it, and we show that our method yields a much better computational complexity than using classical linear programming techniques. △ Less

Submitted 30 October, 2023; v1 submitted 30 March, 2023; originally announced March 2023.

Comments: This paper is an extended version of a CDC2023 submission

Learning stability guarantees for constrained switching linear systems from noisy observations

Authors: Adrien Banse, Zheming Wang, Raphaël M. Jungers

Abstract: We present a data-driven framework based on Lyapunov theory to provide stability guarantees for a family of hybrid systems. In particular, we are interested in the asymptotic stability of switching linear systems whose switching sequence is constrained by labeled graphs, namely constrained switching linear systems. In order to do so, we provide chance-constrained bounds on stability guarantees, th… ▽ More We present a data-driven framework based on Lyapunov theory to provide stability guarantees for a family of hybrid systems. In particular, we are interested in the asymptotic stability of switching linear systems whose switching sequence is constrained by labeled graphs, namely constrained switching linear systems. In order to do so, we provide chance-constrained bounds on stability guarantees, that can be obtained from a finite number of noisy observations. We first present a method providing stability guarantees from sampled trajectories in the hybrid state-space of the system. We then study the harder situation where one only observes the continuous part of the hybrid states. We show that in this case, one may still obtain formal chance-constrained stability guarantees. For this latter result we provide a new upper bound of general interest, also for model-based stability analysis △ Less

Submitted 10 February, 2023; originally announced February 2023.

Comments: Submitted to Nonlinear Analysis: Hybrid Systems

Data-driven memory-dependent abstractions of dynamical systems

Authors: Adrien Banse, Licio Romao, Alessandro Abate, Raphaël M. Jungers

Abstract: We propose a sample-based, sequential method to abstract a (potentially black-box) dynamical system with a sequence of memory-dependent Markov chains of increasing size. We show that this approximation allows to alleviating a correlation bias that has been observed in sample-based abstractions. We further propose a methodology to detect on the fly the memory length resulting in an abstraction with… ▽ More We propose a sample-based, sequential method to abstract a (potentially black-box) dynamical system with a sequence of memory-dependent Markov chains of increasing size. We show that this approximation allows to alleviating a correlation bias that has been observed in sample-based abstractions. We further propose a methodology to detect on the fly the memory length resulting in an abstraction with sufficient accuracy. We prove that under reasonable assumptions, the method converges to a sound abstraction in some precise sense, and we showcase it on two case studies. △ Less

Submitted 4 December, 2022; originally announced December 2022.

Black-box stability analysis of hybrid systems with sample-based multiple Lyapunov functions

Authors: Adrien Banse, Zheming Wang, Raphaël M. Jungers

Abstract: We present a framework based on multiple Lyapunov functions to find probabilistic data-driven guarantees on the stability of unknown constrained switching linear systems (CSLS), which are switching linear systems whose switching signal is constrained by an automaton. The stability of a CSLS is characterized by its constrained joint spectral radius (CJSR). Inspired by the scenario approach and prev… ▽ More We present a framework based on multiple Lyapunov functions to find probabilistic data-driven guarantees on the stability of unknown constrained switching linear systems (CSLS), which are switching linear systems whose switching signal is constrained by an automaton. The stability of a CSLS is characterized by its constrained joint spectral radius (CJSR). Inspired by the scenario approach and previous work on unconstrained switching systems, we characterize the number of observations needed to find sufficient conditions on the (in-)stability of a CSLS using the notion of CJSR. More precisely, our contribution is the following: we derive a probabilistic upper bound on the CJSR of an unknown CSLS from a finite number of observations. We also derive a deterministic lower bound on the CJSR. From this we obtain a probabilistic method to characterize the stability of an unknown CSLS. △ Less

Submitted 2 May, 2022; originally announced May 2022.

Comments: CDC2022 pre-print, 6 pages

Learning stability guarantees for data-driven constrained switching linear systems

Authors: Adrien Banse, Zheming Wang, Raphaël M. Jungers

Abstract: We consider stability analysis of constrained switching linear systems in which the dynamics is unknown and whose switching signal is constrained by an automaton. We propose a data-driven Lyapunov framework for providing probabilistic stability guarantees based on data harvested from observations of the system. By generalizing previous results on arbitrary switching linear systems, we show that, b… ▽ More We consider stability analysis of constrained switching linear systems in which the dynamics is unknown and whose switching signal is constrained by an automaton. We propose a data-driven Lyapunov framework for providing probabilistic stability guarantees based on data harvested from observations of the system. By generalizing previous results on arbitrary switching linear systems, we show that, by sampling a finite number of observations, we are able to construct an approximate Lyapunov function for the underlying system. Moreover, we show that the entropy of the language accepted by the automaton allows to bound the number of samples needed in order to reach some pre-specified accuracy. △ Less

Submitted 14 July, 2022; v1 submitted 2 May, 2022; originally announced May 2022.

Comments: Extended abstract (4 pages), accepted final version