Showing 1–50 of 123 results for author: Müller, M

Model-Based Deep Learning for Music Information Research

Authors: Gael Richard, Vincent Lostanlen, Yi-Hsuan Yang, Meinard Müller

Abstract: In this article, we investigate the notion of model-based deep learning in the realm of music information research (MIR). Loosely speaking, we refer to the term model-based deep learning for approaches that combine traditional knowledge-based methods with data-driven techniques, especially those based on deep learning, within a diff erentiable computing framework. In music, prior knowledge for ins… ▽ More In this article, we investigate the notion of model-based deep learning in the realm of music information research (MIR). Loosely speaking, we refer to the term model-based deep learning for approaches that combine traditional knowledge-based methods with data-driven techniques, especially those based on deep learning, within a diff erentiable computing framework. In music, prior knowledge for instance related to sound production, music perception or music composition theory can be incorporated into the design of neural networks and associated loss functions. We outline three specifi c scenarios to illustrate the application of model-based deep learning in MIR, demonstrating the implementation of such concepts and their potential. △ Less

Submitted 17 June, 2024; originally announced June 2024.

Comments: IEEE Signal Processing Magazine, In press

Harder or Different? Understanding Generalization of Audio Deepfake Detection

Authors: Nicolas M. Müller, Nicholas Evans, Hemlata Tak, Philip Sperl, Konstantin Böttinger

Abstract: Recent research has highlighted a key issue in speech deepfake detection: models trained on one set of deepfakes perform poorly on others. The question arises: is this due to the continuously improving quality of Text-to-Speech (TTS) models, i.e., are newer DeepFakes just 'harder' to detect? Or, is it because deepfakes generated with one model are fundamentally different to those generated using a… ▽ More Recent research has highlighted a key issue in speech deepfake detection: models trained on one set of deepfakes perform poorly on others. The question arises: is this due to the continuously improving quality of Text-to-Speech (TTS) models, i.e., are newer DeepFakes just 'harder' to detect? Or, is it because deepfakes generated with one model are fundamentally different to those generated using another model? We answer this question by decomposing the performance gap between in-domain and out-of-domain test data into 'hardness' and 'difference' components. Experiments performed using ASVspoof databases indicate that the hardness component is practically negligible, with the performance gap being attributed primarily to the difference component. This has direct implications for real-world deepfake detection, highlighting that merely increasing model capacity, the currently-dominant research trend, may not effectively address the generalization challenge. △ Less

Submitted 12 June, 2024; v1 submitted 5 June, 2024; originally announced June 2024.

Journal ref: Interspeech 2024

An input-output continuous-time version of Willems' lemma

Authors: Victor G. Lopez, Matthias A. Müller, Paolo Rapisarda

Abstract: We illustrate a novel version of Willems' lemma for data-based representation of continuous-time systems. The main novelties compared to previous works are two. First, the proposed framework relies only on measured input-output trajectories from the system and no internal (state) information is required. Second, our system representation makes use of exact system trajectories, without resorting to… ▽ More We illustrate a novel version of Willems' lemma for data-based representation of continuous-time systems. The main novelties compared to previous works are two. First, the proposed framework relies only on measured input-output trajectories from the system and no internal (state) information is required. Second, our system representation makes use of exact system trajectories, without resorting to orthogonal bases representations and consequent approximations. We first establish sufficient and necessary conditions for data-based generation of system trajectories in terms of suitable latent variables. Subsequently, we reformulate these conditions using measured input-output data and show how to span the full behavior of the system. Furthermore, we show how to use the developed framework to solve the data-based continuous-time simulation problem. △ Less

Submitted 24 May, 2024; originally announced May 2024.

Comments: 6 pages

On discount functions for economic model predictive control without terminal conditions

Authors: Lukas Schwenkel, Daniel Briem, Matthias A. Müller, Frank Allgöwer

Abstract: In this paper, we investigate discounted economic model predictive control (E-MPC) schemes without terminal conditions in scenarios where the optimal operating behavior is a periodic orbit. For such a setting, it is known that a linearly discounted stage cost guarantees asymptotic stability of any arbitrarily small neighborhood of the optimal orbit if the prediction horizon is sufficiently long. H… ▽ More In this paper, we investigate discounted economic model predictive control (E-MPC) schemes without terminal conditions in scenarios where the optimal operating behavior is a periodic orbit. For such a setting, it is known that a linearly discounted stage cost guarantees asymptotic stability of any arbitrarily small neighborhood of the optimal orbit if the prediction horizon is sufficiently long. However, in some examples very long prediction horizons are needed to achieve the desired performance. In this work, we extend these results by providing the same qualitative stability guarantees for a large class of discount functions. Numerical examples illustrate the influence of the discount function and show that with suitable discounting we can achieve significantly better performance than the linearly discounted E-MPC, even for short prediction horizons. △ Less

Submitted 23 May, 2024; originally announced May 2024.

Locality Regularized Reconstruction: Structured Sparsity and Delaunay Triangulations

Authors: Marshall Mueller, James M. Murphy, Abiy Tasissa

Abstract: Linear representation learning is widely studied due to its conceptual simplicity and empirical utility in tasks such as compression, classification, and feature extraction. Given a set of points $[\mathbf{x}_1, \mathbf{x}_2, \ldots, \mathbf{x}_n] = \mathbf{X} \in \mathbb{R}^{d \times n}$ and a vector $\mathbf{y} \in \mathbb{R}^d$, the goal is to find coefficients $\mathbf{w} \in \mathbb{R}^n$ so… ▽ More Linear representation learning is widely studied due to its conceptual simplicity and empirical utility in tasks such as compression, classification, and feature extraction. Given a set of points $[\mathbf{x}_1, \mathbf{x}_2, \ldots, \mathbf{x}_n] = \mathbf{X} \in \mathbb{R}^{d \times n}$ and a vector $\mathbf{y} \in \mathbb{R}^d$, the goal is to find coefficients $\mathbf{w} \in \mathbb{R}^n$ so that $\mathbf{X} \mathbf{w} \approx \mathbf{y}$, subject to some desired structure on $\mathbf{w}$. In this work we seek $\mathbf{w}$ that forms a local reconstruction of $\mathbf{y}$ by solving a regularized least squares regression problem. We obtain local solutions through a locality function that promotes the use of columns of $\mathbf{X}$ that are close to $\mathbf{y}$ when used as a regularization term. We prove that, for all levels of regularization and under a mild condition that the columns of $\mathbf{X}$ have a unique Delaunay triangulation, the optimal coefficients' number of non-zero entries is upper bounded by $d+1$, thereby providing local sparse solutions when $d \ll n$. Under the same condition we also show that for any $\mathbf{y}$ contained in the convex hull of $\mathbf{X}$ there exists a regime of regularization parameter such that the optimal coefficients are supported on the vertices of the Delaunay simplex containing $\mathbf{y}$. This provides an interpretation of the sparsity as having structure obtained implicitly from the Delaunay triangulation of $\mathbf{X}$. We demonstrate that our locality regularized problem can be solved in comparable time to other methods that identify the containing Delaunay simplex. △ Less

Submitted 1 May, 2024; originally announced May 2024.

Comments: 26 pages, 8 figures

Data-Based System Representation and Synchronization for Multiagent Systems

Authors: Victor G. Lopez, Matthias A. Müller

Abstract: This paper presents novel solutions of the data-based synchronization problem for continuous-time multiagent systems. We consider the cases of homogeneous and heterogeneous systems. First, a data-based representation of the synchronization error dynamics is obtained for homogeneous systems, using input-state data collected from the agents. Then, we show how to extend existing data-based stabilizat… ▽ More This paper presents novel solutions of the data-based synchronization problem for continuous-time multiagent systems. We consider the cases of homogeneous and heterogeneous systems. First, a data-based representation of the synchronization error dynamics is obtained for homogeneous systems, using input-state data collected from the agents. Then, we show how to extend existing data-based stabilization results to the multiagent case to stabilize the obtained synchronization errors. The proposed method relies on the solution of a set of linear matrix inequalities that are shown to be feasible. Then, we solve the synchronization problem for heterogeneous systems by means of dynamic controllers. Different from existing results, we do not require model knowledge for the followers and the leader. The theoretical results are finally validated using numerical simulations. △ Less

Submitted 22 April, 2024; originally announced April 2024.

Comments: 8 pages

Disturbance feedback-based model predictive control in uncertain dynamic environments

Authors: Philipp Buschermöhle, Taouba Jouini, Torsten Lilge, Matthias A. Müller

Abstract: This paper presents a robust MPC scheme for linear systems subject to time-varying, uncertain constraints that arise from uncertain environments. The predicted input sequence is parameterized over future environment states to guarantee constraint satisfaction despite an imprecise environment prediction and unknown evolution of the future constraints. We provide theoretical guarantees for recursive… ▽ More This paper presents a robust MPC scheme for linear systems subject to time-varying, uncertain constraints that arise from uncertain environments. The predicted input sequence is parameterized over future environment states to guarantee constraint satisfaction despite an imprecise environment prediction and unknown evolution of the future constraints. We provide theoretical guarantees for recursive feasibility and asymptotic convergence. Finally, a brief simulation example showcases our results. △ Less

Submitted 15 April, 2024; originally announced April 2024.

Comments: 7 pages, 2 figures, Accepted for publication at 8th IFAC Conference on Nonlinear Model Predictive Control 2024

Moving horizon estimation for nonlinear systems with time-varying parameters

Authors: Julian D. Schiller, Matthias A. Müller

Abstract: We propose a moving horizon estimation scheme for estimating the states and time-varying parameters of nonlinear systems. We consider the case where observability of the parameters depends on the excitation of the system and may be absent during operation, with the parameter dynamics fulfilling a weak incremental bounded-energy bounded-state property to ensure boundedness of the estimation error (… ▽ More We propose a moving horizon estimation scheme for estimating the states and time-varying parameters of nonlinear systems. We consider the case where observability of the parameters depends on the excitation of the system and may be absent during operation, with the parameter dynamics fulfilling a weak incremental bounded-energy bounded-state property to ensure boundedness of the estimation error (with respect to the disturbance energy). The proposed estimation scheme involves a standard quadratic cost function with an adaptive regularization term depending on the current parameter observability. We develop robustness guarantees for the overall estimation error that are valid for all times, and that improve the more often the parameters are detected to be observable during operation. The theoretical results are illustrated by a simulation example. △ Less

Submitted 15 April, 2024; originally announced April 2024.

Comments: accepted for presentation at NMPC 2024

Data-Based Control of Continuous-Time Linear Systems with Performance Specifications

Authors: Victor G. Lopez, Matthias A. Müller

Abstract: The design of direct data-based controllers has become a fundamental part of control theory research in the last few years. In this paper, we consider three classes of data-based state feedback control problems for linear systems. These control problems are such that, besides stabilization, some additional performance requirements must be satisfied. First, we formulate and solve a trajectory-refer… ▽ More The design of direct data-based controllers has become a fundamental part of control theory research in the last few years. In this paper, we consider three classes of data-based state feedback control problems for linear systems. These control problems are such that, besides stabilization, some additional performance requirements must be satisfied. First, we formulate and solve a trajectory-reference control problem, on which desired closed-loop trajectories are known and a controller that allows the system to closely follow those trajectories is computed. Then, in the area of data-based optimal control, we solve two different problems: the inverse problem of optimal control, and the solution of the LQR problem for continuous-time systems. Finally, we consider the case in which the precise position of the desired poles of the closed-loop system is known, and introduce a data-based variant of a robust pole-placement procedure. Although we focus on continuous-time systems, all of the presented methods can also be easily formulated for the discrete-time case. The applicability of the proposed methods is tested using numerical simulations. △ Less

Submitted 1 March, 2024; originally announced March 2024.

Comments: 16 pages, 1 figure

A New Approach to Voice Authenticity

Authors: Nicolas M. Müller, Piotr Kawa, Shen Hu, Matthias Neu, Jennifer Williams, Philip Sperl, Konstantin Böttinger

Abstract: Voice faking, driven primarily by recent advances in text-to-speech (TTS) synthesis technology, poses significant societal challenges. Currently, the prevailing assumption is that unaltered human speech can be considered genuine, while fake speech comes from TTS synthesis. We argue that this binary distinction is oversimplified. For instance, altered playback speeds can be used for malicious purpo… ▽ More Voice faking, driven primarily by recent advances in text-to-speech (TTS) synthesis technology, poses significant societal challenges. Currently, the prevailing assumption is that unaltered human speech can be considered genuine, while fake speech comes from TTS synthesis. We argue that this binary distinction is oversimplified. For instance, altered playback speeds can be used for malicious purposes, like in the 'Drunken Nancy Pelosi' incident. Similarly, editing of audio clips can be done ethically, e.g., for brevity or summarization in news reporting or podcasts, but editing can also create misleading narratives. In this paper, we propose a conceptual shift away from the binary paradigm of audio being either 'fake' or 'real'. Instead, our focus is on pinpointing 'voice edits', which encompass traditional modifications like filters and cuts, as well as TTS synthesis and VC systems. We delineate 6 categories and curate a new challenge dataset rooted in the M-AILABS corpus, for which we present baseline detection systems. And most importantly, we argue that merely categorizing audio as fake or real is a dangerous over-simplification that will fail to move the field of speech technology forward. △ Less

Submitted 9 February, 2024; originally announced February 2024.

Gaussian Process-Based Nonlinear Moving Horizon Estimation

Authors: Tobias M. Wolff, Victor G. Lopez, Matthias A. Müller

Abstract: In this paper, we propose a novel Gaussian process-based moving horizon estimation (MHE) framework for unknown nonlinear systems. In the proposed scheme, we take advantage of the properties of Gaussian processes. On the one hand, we approximate the system dynamics by the posterior means of the learned Gaussian processes (GPs). On the other hand, we exploit the posterior variances of the Gaussian p… ▽ More In this paper, we propose a novel Gaussian process-based moving horizon estimation (MHE) framework for unknown nonlinear systems. In the proposed scheme, we take advantage of the properties of Gaussian processes. On the one hand, we approximate the system dynamics by the posterior means of the learned Gaussian processes (GPs). On the other hand, we exploit the posterior variances of the Gaussian processes to design the weighting matrices in the MHE cost function and account for the uncertainty in the learned system dynamics. The data collection and the tuning of the hyperparameters are done offline. We prove robust stability of the GP-based MHE scheme using a Lyapunov-based proof technique. Furthermore, as additional contribution, we analyze under which conditions incremental input/output-to-state stability (a nonlinear detectability notion) is preserved when approximating the system dynamics using, e.g., machine learning techniques. Finally, we illustrate the performance of the GP-based MHE scheme in a simulation case study and show how the chosen weighting matrices can lead to an improved performance compared to standard cost functions. △ Less

Submitted 7 February, 2024; originally announced February 2024.

Comments: 16 pages

MLAAD: The Multi-Language Audio Anti-Spoofing Dataset

Authors: Nicolas M. Müller, Piotr Kawa, Wei Herng Choong, Edresson Casanova, Eren Gölge, Thorsten Müller, Piotr Syga, Philip Sperl, Konstantin Böttinger

Abstract: Text-to-Speech (TTS) technology brings significant advantages, such as giving a voice to those with speech impairments, but also enables audio deepfakes and spoofs. The former mislead individuals and may propagate misinformation, while the latter undermine voice biometric security systems. AI-based detection can help to address these challenges by automatically differentiating between genuine and… ▽ More Text-to-Speech (TTS) technology brings significant advantages, such as giving a voice to those with speech impairments, but also enables audio deepfakes and spoofs. The former mislead individuals and may propagate misinformation, while the latter undermine voice biometric security systems. AI-based detection can help to address these challenges by automatically differentiating between genuine and fabricated voice recordings. However, these models are only as good as their training data, which currently is severely limited due to an overwhelming concentration on English and Chinese audio in anti-spoofing databases, thus restricting its worldwide effectiveness. In response, this paper presents the Multi-Language Audio Anti-Spoof Dataset (MLAAD), created using 54 TTS models, comprising 21 different architectures, to generate 163.9 hours of synthetic voice in 23 different languages. We train and evaluate three state-of-the-art deepfake detection models with MLAAD, and observe that MLAAD demonstrates superior performance over comparable datasets like InTheWild or FakeOrReal when used as a training resource. Furthermore, in comparison with the renowned ASVspoof 2019 dataset, MLAAD proves to be a complementary resource. In tests across eight datasets, MLAAD and ASVspoof 2019 alternately outperformed each other, both excelling on four datasets. By publishing MLAAD and making trained models accessible via an interactive webserver , we aim to democratize antispoofing technology, making it accessible beyond the realm of specialists, thus contributing to global efforts against audio spoofing and deepfakes. △ Less

Submitted 16 April, 2024; v1 submitted 17 January, 2024; originally announced January 2024.

Comments: IJCNN 2024

Online convex optimization for robust control of constrained dynamical systems

Authors: Marko Nonhoff, Emiliano Dall'Anese, Matthias A. Müller

Abstract: This article investigates the problem of controlling linear time-invariant systems subject to time-varying and a priori unknown cost functions, state and input constraints, and exogenous disturbances. We combine the online convex optimization framework with tools from robust model predictive control to propose an algorithm that is able to guarantee robust constraint satisfaction. The performance o… ▽ More This article investigates the problem of controlling linear time-invariant systems subject to time-varying and a priori unknown cost functions, state and input constraints, and exogenous disturbances. We combine the online convex optimization framework with tools from robust model predictive control to propose an algorithm that is able to guarantee robust constraint satisfaction. The performance of the closed loop emerging from application of our framework is studied in terms of its dynamic regret, which is proven to be bounded linearly by the variation of the cost functions and the magnitude of the disturbances. We corroborate our theoretical findings and illustrate implementational aspects of the proposed algorithm by a numerical case study of a tracking control problem of an autonomous vehicle. △ Less

Submitted 9 January, 2024; originally announced January 2024.

Comments: 16 pages

Sample-based nonlinear detectability for discrete-time systems

Authors: Isabelle Krauss, Victor G. Lopez, Matthias A. Müller

Abstract: This paper introduces two sample-based formulations of incremental input/output-to-state stability (i-IOSS), a suitable detectability notion for general nonlinear systems. In this work we consider the case of limited output information, i.e., measurements are only infrequently and/or irregularly available. The output-dependent term of the sample-based i-IOSS bound is properly modified to yield a c… ▽ More This paper introduces two sample-based formulations of incremental input/output-to-state stability (i-IOSS), a suitable detectability notion for general nonlinear systems. In this work we consider the case of limited output information, i.e., measurements are only infrequently and/or irregularly available. The output-dependent term of the sample-based i-IOSS bound is properly modified to yield a characterization for detectability in presence of incomplete output sequences. We provide both a non-timediscounted and a time-discounted formulation of samplebased i-IOSS. Furthermore, conditions for an i-IOSS system to be also sample-based i-IOSS are given and the relation between the two formulations of sample-based i-IOSS is shown. △ Less

Submitted 21 December, 2023; originally announced December 2023.

Nonlinear moving horizon estimation for robust state and parameter estimation

Authors: Julian D. Schiller, Matthias A. Müller

Abstract: We propose a moving horizon estimation scheme to estimate the states and the unknown constant parameters of general nonlinear uncertain discrete time systems. The proposed framework and analysis explicitly do not involve the a priori verification of a particular excitation condition for the parameters. Instead, we use online information about the actual excitation of the parameters at any time dur… ▽ More We propose a moving horizon estimation scheme to estimate the states and the unknown constant parameters of general nonlinear uncertain discrete time systems. The proposed framework and analysis explicitly do not involve the a priori verification of a particular excitation condition for the parameters. Instead, we use online information about the actual excitation of the parameters at any time during operation and ensure that the regularization term in the cost function is always automatically selected appropriately. This ensures that the state and parameter estimation error is bounded for all times, even if the parameters are never (or only rarely) excited during operation. Additionally, the more often sufficient excitation is detected, the better (i.e., smaller) the bound becomes. Robust exponential stability of the state and parameter estimation error emerges under an additional uniform condition on the maximum duration of insufficient excitation. The theoretical results are illustrated by a numerical example. △ Less

Submitted 20 December, 2023; originally announced December 2023.

Fractional Chirp-Slope-Shift-Keying for SDR-based Search and Rescue Applications

Authors: Daniel Gil Gaviria, Marcus Müller, Felix Artmann, Laurent Schmalen

Abstract: The use of modern software-defined radio (SDR) devices enables the implementation of efficient communication systems in numerous scenarios. Such technology comes especially handy in the context of search and rescue (SAR) systems, enabling the incorporation of additional communication data transmission into the otherwise sub-optimally used SAR bands at 121.5 and 243~MHz. In this work, we propose a… ▽ More The use of modern software-defined radio (SDR) devices enables the implementation of efficient communication systems in numerous scenarios. Such technology comes especially handy in the context of search and rescue (SAR) systems, enabling the incorporation of additional communication data transmission into the otherwise sub-optimally used SAR bands at 121.5 and 243~MHz. In this work, we propose a novel low-complexity, energy-efficient modulation scheme that allows transmission of additional data within chirped homing signals, while still meeting the standards of international SAR systems such as COSPAS-SARSAT. The proposed method modulates information onto small deviations of the chirp slope with respect to the required unmodulated chirp, which can be easily detected at the receiver side using digital signal processing. △ Less

Submitted 8 December, 2023; originally announced December 2023.

An efficient data-based off-policy Q-learning algorithm for optimal output feedback control of linear systems

Authors: Mohammad Alsalti, Victor G. Lopez, Matthias A. Müller

Abstract: In this paper, we present a Q-learning algorithm to solve the optimal output regulation problem for discrete-time LTI systems. This off-policy algorithm only relies on using persistently exciting input-output data, measured offline. No model knowledge or state measurements are needed and the obtained optimal policy only uses past input-output information. Moreover, our formulation of the proposed… ▽ More In this paper, we present a Q-learning algorithm to solve the optimal output regulation problem for discrete-time LTI systems. This off-policy algorithm only relies on using persistently exciting input-output data, measured offline. No model knowledge or state measurements are needed and the obtained optimal policy only uses past input-output information. Moreover, our formulation of the proposed algorithm renders it computationally efficient. We provide conditions that guarantee the convergence of the algorithm to the optimal solution. Finally, the performance of our method is compared to existing algorithms in the literature. △ Less

Submitted 6 December, 2023; originally announced December 2023.

Notes on data-driven output-feedback control of linear MIMO systems

Authors: Mohammad Alsalti, Victor G. Lopez, Matthias A. Müller

Abstract: Recent works have approached the data-driven design of output-feedback controllers for discrete-time LTI systems by constructing non-minimal state vectors composed of past inputs and outputs. Depending on the system's complexity (order, lag and number of inputs), it was observed in several works that such an approach presents certain limitations, but no methods were proposed to overcome them. In t… ▽ More Recent works have approached the data-driven design of output-feedback controllers for discrete-time LTI systems by constructing non-minimal state vectors composed of past inputs and outputs. Depending on the system's complexity (order, lag and number of inputs), it was observed in several works that such an approach presents certain limitations, but no methods were proposed to overcome them. In this note, we clarify these limitations and solve them by proposing the construction of (alternative) non-minimal state vectors that facilitate output-feedback control of MIMO discrete-time LTI systems. △ Less

Submitted 29 November, 2023; originally announced November 2023.

Automated Layout Design and Control of Robust Cooperative Grasped-Load Aerial Transportation Systems

Authors: Carlo Bosio, Jerry Tang, Ting-Hao Wang, Mark W. Mueller

Abstract: We present a novel approach to cooperative aerial transportation through a team of drones, using optimal control theory and a hierarchical control strategy. We assume the drones are connected to the payload through rigid attachments, essentially transforming the whole system into a larger flying object with "thrust modules" at the attachment locations of the drones. We investigate the optimal arra… ▽ More We present a novel approach to cooperative aerial transportation through a team of drones, using optimal control theory and a hierarchical control strategy. We assume the drones are connected to the payload through rigid attachments, essentially transforming the whole system into a larger flying object with "thrust modules" at the attachment locations of the drones. We investigate the optimal arrangement of the thrust modules around the payload, so that the resulting system is robust to disturbances. We choose the $\mathcal{H}_2$ norm as a measure of robustness, and propose an iterative optimization routine to compute the optimal layout of the vehicles around the object. We experimentally validate our approach using four drones and comparing the disturbance rejection performances achieved by two different layouts (the optimal one and a sub-optimal one), and observe that the results match our predictions. △ Less

Submitted 28 February, 2024; v1 submitted 11 October, 2023; originally announced October 2023.

Comments: 7 pages, 7 figures, conference paper

Performance Conditioning for Diffusion-Based Multi-Instrument Music Synthesis

Authors: Ben Maman, Johannes Zeitler, Meinard Müller, Amit H. Bermano

Abstract: Generating multi-instrument music from symbolic music representations is an important task in Music Information Retrieval (MIR). A central but still largely unsolved problem in this context is musically and acoustically informed control in the generation process. As the main contribution of this work, we propose enhancing control of multi-instrument synthesis by conditioning a generative model on… ▽ More Generating multi-instrument music from symbolic music representations is an important task in Music Information Retrieval (MIR). A central but still largely unsolved problem in this context is musically and acoustically informed control in the generation process. As the main contribution of this work, we propose enhancing control of multi-instrument synthesis by conditioning a generative model on a specific performance and recording environment, thus allowing for better guidance of timbre and style. Building on state-of-the-art diffusion-based music generative models, we introduce performance conditioning - a simple tool indicating the generative model to synthesize music with style and timbre of specific instruments taken from specific performances. Our prototype is evaluated using uncurated performances with diverse instrumentation and achieves state-of-the-art FAD realism scores while allowing novel timbre and style control. Our project page, including samples and demonstrations, is available at benadar293.github.io/midipm △ Less

Submitted 21 September, 2023; originally announced September 2023.

Comments: 5 pages, project page available at benadar293.github.io/midipm

Sample- and computationally efficient data-driven predictive control

Authors: Mohammad Alsalti, Manuel Barkey, Victor G. Lopez, Matthias A. Müller

Abstract: Recently proposed data-driven predictive control schemes for LTI systems use non-parametric representations based on the image of a Hankel matrix of previously collected, persistently exciting, input-output data. Persistence of excitation necessitates that the data is sufficiently long and, hence, the computational complexity of the corresponding finite-horizon optimal control problem increases. I… ▽ More Recently proposed data-driven predictive control schemes for LTI systems use non-parametric representations based on the image of a Hankel matrix of previously collected, persistently exciting, input-output data. Persistence of excitation necessitates that the data is sufficiently long and, hence, the computational complexity of the corresponding finite-horizon optimal control problem increases. In this paper, we propose an efficient data-driven predictive control (eDDPC) scheme which is both more sample efficient (requires less offline data) and computationally efficient (uses less decision variables) compared to existing schemes. This is done by leveraging an alternative data-based representation of the trajectories of LTI systems. We analytically and numerically compare the performance of this scheme to existing ones from the literature. △ Less

Submitted 6 March, 2024; v1 submitted 20 September, 2023; originally announced September 2023.

Comments: accepted for presentation at the 22nd European Control Conference (ECC) in Stockholm, Sweden

Complex-valued neural networks for voice anti-spoofing

Authors: Nicolas M. Müller, Philip Sperl, Konstantin Böttinger

Abstract: Current anti-spoofing and audio deepfake detection systems use either magnitude spectrogram-based features (such as CQT or Melspectrograms) or raw audio processed through convolution or sinc-layers. Both methods have drawbacks: magnitude spectrograms discard phase information, which affects audio naturalness, and raw-feature-based models cannot use traditional explainable AI methods. This paper pr… ▽ More Current anti-spoofing and audio deepfake detection systems use either magnitude spectrogram-based features (such as CQT or Melspectrograms) or raw audio processed through convolution or sinc-layers. Both methods have drawbacks: magnitude spectrograms discard phase information, which affects audio naturalness, and raw-feature-based models cannot use traditional explainable AI methods. This paper proposes a new approach that combines the benefits of both methods by using complex-valued neural networks to process the complex-valued, CQT frequency-domain representation of the input audio. This method retains phase information and allows for explainable AI methods. Results show that this approach outperforms previous methods on the "In-the-Wild" anti-spoofing dataset and enables interpretation of the results through explainable AI. Ablation studies confirm that the model has learned to use phase information to detect voice spoofing. △ Less

Submitted 22 August, 2023; originally announced August 2023.

Comments: Interspeech 2023

Local Periodicity-Based Beat Tracking for Expressive Classical Piano Music

Authors: Ching-Yu Chiu, Meinard Müller, Matthew E. P. Davies, Alvin Wen-Yu Su, Yi-Hsuan Yang

Abstract: To model the periodicity of beats, state-of-the-art beat tracking systems use "post-processing trackers" (PPTs) that rely on several empirically determined global assumptions for tempo transition, which work well for music with a steady tempo. For expressive classical music, however, these assumptions can be too rigid. With two large datasets of Western classical piano music, namely the Aligned Sc… ▽ More To model the periodicity of beats, state-of-the-art beat tracking systems use "post-processing trackers" (PPTs) that rely on several empirically determined global assumptions for tempo transition, which work well for music with a steady tempo. For expressive classical music, however, these assumptions can be too rigid. With two large datasets of Western classical piano music, namely the Aligned Scores and Performances (ASAP) dataset and a dataset of Chopin's Mazurkas (Maz-5), we report on experiments showing the failure of existing PPTs to cope with local tempo changes, thus calling for new methods. In this paper, we propose a new local periodicity-based PPT, called predominant local pulse-based dynamic programming (PLPDP) tracking, that allows for more flexible tempo transitions. Specifically, the new PPT incorporates a method called "predominant local pulses" (PLP) in combination with a dynamic programming (DP) component to jointly consider the locally detected periodicity and beat activation strength at each time instant. Accordingly, PLPDP accounts for the local periodicity, rather than relying on a global tempo assumption. Compared to existing PPTs, PLPDP particularly enhances the recall values at the cost of a lower precision, resulting in an overall improvement of F1-score for beat tracking in ASAP (from 0.473 to 0.493) and Maz-5 (from 0.595 to 0.838). △ Less

Submitted 20 August, 2023; originally announced August 2023.

Comments: Accepted to IEEE/ACM Transactions on Audio, Speech, and Language Processing (July 2023)

Stabilizing Training with Soft Dynamic Time War**: A Case Study for Pitch Class Estimation with Weakly Aligned Targets

Authors: Johannes Zeitler, Simon Deniffel, Michael Krause, Meinard Müller

Abstract: Soft dynamic time war** (SDTW) is a differentiable loss function that allows for training neural networks from weakly aligned data. Typically, SDTW is used to iteratively compute and refine soft alignments that compensate for temporal deviations between the training data and its weakly annotated targets. One major problem is that a mismatch between the estimated soft alignments and the reference… ▽ More Soft dynamic time war** (SDTW) is a differentiable loss function that allows for training neural networks from weakly aligned data. Typically, SDTW is used to iteratively compute and refine soft alignments that compensate for temporal deviations between the training data and its weakly annotated targets. One major problem is that a mismatch between the estimated soft alignments and the reference alignments in the early training stage leads to incorrect parameter updates, making the overall training procedure unstable. In this paper, we investigate such stability issues by considering the task of pitch class estimation from music recordings as an illustrative case study. In particular, we introduce and discuss three conceptually different strategies (a hyperparameter scheduling, a diagonal prior, and a sequence unfolding strategy) with the objective of stabilizing intermediate soft alignment results. Finally, we report on experiments that demonstrate the effectiveness of the strategies and discuss efficiency and implementation issues. △ Less

Submitted 10 August, 2023; originally announced August 2023.

Comments: Accepted for ISMIR 2023, Milano, Italy

Model predictive control for the prescription of antithyroid agents

Authors: Maylin Menzel, Tobias M. Wolff, Johannes W. Dietrich, Matthias A. Müller

Abstract: Although hyperthyroidism is a common disease, the pharmaceutical therapy is based on a trial-and-error approach. We extend a mathematical model of the pituitary-thyroid feedback loop such that the intake of one antithyroid agent, namely methimazole (MMI), can be considered and use a model predictive control (MPC) scheme to determine suitable dosages. Although hyperthyroidism is a common disease, the pharmaceutical therapy is based on a trial-and-error approach. We extend a mathematical model of the pituitary-thyroid feedback loop such that the intake of one antithyroid agent, namely methimazole (MMI), can be considered and use a model predictive control (MPC) scheme to determine suitable dosages. △ Less

Submitted 31 July, 2023; originally announced July 2023.

Comments: 2 pages

Data-based system representations from irregularly measured data

Authors: Mohammad Alsalti, Ivan Markovsky, Victor G. Lopez, Matthias A. Müller

Abstract: Non-parametric representations of dynamical systems based on the image of a Hankel matrix of data are extensively used for data-driven control. However, if samples of data are missing, obtaining such representations becomes a difficult task. By exploiting the kernel structure of Hankel matrices of irregularly measured data generated by a linear time-invariant system, we provide computational metho… ▽ More Non-parametric representations of dynamical systems based on the image of a Hankel matrix of data are extensively used for data-driven control. However, if samples of data are missing, obtaining such representations becomes a difficult task. By exploiting the kernel structure of Hankel matrices of irregularly measured data generated by a linear time-invariant system, we provide computational methods for which any complete finite-length behavior of the system can be obtained. For the special case of periodically missing outputs, we provide conditions on the input such that the former result is guaranteed. In the presence of noise in the data, our method returns an approximate finite-length behavior of the system. We illustrate our result with several examples, including its use for approximate data completion in real-world applications and compare it to alternative methods. △ Less

Submitted 24 February, 2024; v1 submitted 21 July, 2023; originally announced July 2023.

Comments: submitted to IEEE Transactions on Automatic Control

Analysis and design of model predictive control frameworks for dynamic operation -- An overview

Authors: Johannes Köhler, Matthas A. Müller, Frank Allgöwer

Abstract: This article provides an overview of model predictive control (MPC) frameworks for dynamic operation of nonlinear constrained systems. Dynamic operation is often an integral part of the control objective, ranging from tracking of reference signals to the general economic operation of a plant under online changing time-varying operating conditions. We focus on the particular challenges that arise w… ▽ More This article provides an overview of model predictive control (MPC) frameworks for dynamic operation of nonlinear constrained systems. Dynamic operation is often an integral part of the control objective, ranging from tracking of reference signals to the general economic operation of a plant under online changing time-varying operating conditions. We focus on the particular challenges that arise when dealing with such more general control goals and present methods that have emerged in the literature to address these issues. The goal of this article is to present an overview of the state-of-the-art techniques, providing a diverse toolkit to apply and further develop MPC formulations that can handle the challenges intrinsic to dynamic operation. We also critically assess the applicability of the different research directions, discussing limitations and opportunities for further research. △ Less

Submitted 9 January, 2024; v1 submitted 6 July, 2023; originally announced July 2023.

Comments: This is the accepted version of the paper in Annual Reviews in Control, 2024

Journal ref: Annual Reviews in Control (2024)

Model-based demosaicking for acquisitions by a RGBW color filter array

Authors: Matthieu Muller, Daniele Picone, Mauro Dalla Mura, Magnus O Ulfarsson

Abstract: Microsatellites and drones are often equipped with digital cameras whose sensing system is based on color filter arrays (CFAs), which define a pattern of color filter overlaid over the focal plane. Recent commercial cameras have started implementing RGBW patterns, which include some filters with a wideband spectral response together with the more classical RGB ones. This allows for additional ligh… ▽ More Microsatellites and drones are often equipped with digital cameras whose sensing system is based on color filter arrays (CFAs), which define a pattern of color filter overlaid over the focal plane. Recent commercial cameras have started implementing RGBW patterns, which include some filters with a wideband spectral response together with the more classical RGB ones. This allows for additional light energy to be captured by the relevant pixels and increases the overall SNR of the acquisition. Demosaicking defines reconstructing a multi-spectral image from the raw image and recovering the full color components for all pixels. However, this operation is often tailored for the most widespread patterns, such as the Bayer pattern. Consequently, less common patterns that are still employed in commercial cameras are often neglected. In this work, we present a generalized framework to represent the image formation model of such cameras. This model is then exploited by our proposed demosaicking algorithm to reconstruct the datacube of interest with a Bayesian approach, using a total variation regularizer as prior. Some preliminary experimental results are also presented, which apply to the reconstruction of acquisitions of various RGBW cameras. △ Less

Submitted 2 June, 2023; originally announced June 2023.

Robust stability of moving horizon estimation for continuous-time systems

Authors: Julian D. Schiller, Matthias A. Müller

Abstract: We consider a moving horizon estimation (MHE) scheme involving a discounted least squares objective for general nonlinear continuous-time systems. Provided that the system is detectable (incrementally integral input/output-to-state stable, i-iIOSS), we show that there exists a sufficiently long estimation horizon that guarantees robust global exponential stability of the estimation error in a time… ▽ More We consider a moving horizon estimation (MHE) scheme involving a discounted least squares objective for general nonlinear continuous-time systems. Provided that the system is detectable (incrementally integral input/output-to-state stable, i-iIOSS), we show that there exists a sufficiently long estimation horizon that guarantees robust global exponential stability of the estimation error in a time-discounted $L^2$-to-$L^\infty$ sense. In addition, we show that i-iIOSS Lyapunov functions can be efficiently constructed by verifying certain linear matrix inequality conditions. In combination, we propose a flexible Lyapunov-based MHE framework in continuous time, which particularly offers more tuning possibilities than its discrete-time analog, and provide sufficient conditions for stability that can be easily verified in practice. Our results are illustrated by a numerical example. △ Less

Submitted 31 January, 2024; v1 submitted 11 May, 2023; originally announced May 2023.

Comments: Replaced by accepted version

Journal ref: at-Automatisierungstechnik 2024; 72(2): 120-133

On an integral variant of incremental input/output-to-state stability and its use as a notion of nonlinear detectability

Authors: Julian D. Schiller, Matthias A. Müller

Abstract: We propose a time-discounted integral variant of incremental input/output-to-state stability (i-iIOSS) together with an equivalent Lyapunov function characterization. Continuity of the i-iIOSS Lyapunov function is ensured if the system satisfies a certain continuity assumption involving the Osgood condition. We show that the proposed i-iIOSS notion is a necessary condition for the existence of a r… ▽ More We propose a time-discounted integral variant of incremental input/output-to-state stability (i-iIOSS) together with an equivalent Lyapunov function characterization. Continuity of the i-iIOSS Lyapunov function is ensured if the system satisfies a certain continuity assumption involving the Osgood condition. We show that the proposed i-iIOSS notion is a necessary condition for the existence of a robustly globally asymptotically stable observer map** in a time-discounted ``$L^2$-to-$L^\infty$'' sense. In combination, our results provide a general framework for a Lyapunov-based robust stability analysis of observers for continuous-time systems, which in particular is crucial for the use of optimization-based state estimators (such as moving horizon estimation). △ Less

Submitted 19 June, 2023; v1 submitted 9 May, 2023; originally announced May 2023.

Comments: replaced with accepted version

Segmentation of glioblastomas in early post-operative multi-modal MRI with deep neural networks

Authors: Ragnhild Holden Helland, Alexandros Ferles, André Pedersen, Ivar Kommers, Hilko Ardon, Frederik Barkhof, Lorenzo Bello, Mitchel S. Berger, Tora Dunås, Marco Conti Nibali, Julia Furtner, Shawn Hervey-Jumper, Albert J. S. Idema, Barbara Kiesel, Rishi Nandoe Tewari, Emmanuel Mandonnet, Domenique M. J. Müller, Pierre A. Robe, Marco Rossi, Lisa M. Sagberg, Tommaso Sciortino, Tom Aalders, Michiel Wagemakers, Georg Widhalm, Marnix G. Witte , et al. (8 additional authors not shown)

Abstract: Extent of resection after surgery is one of the main prognostic factors for patients diagnosed with glioblastoma. To achieve this, accurate segmentation and classification of residual tumor from post-operative MR images is essential. The current standard method for estimating it is subject to high inter- and intra-rater variability, and an automated method for segmentation of residual tumor in ear… ▽ More Extent of resection after surgery is one of the main prognostic factors for patients diagnosed with glioblastoma. To achieve this, accurate segmentation and classification of residual tumor from post-operative MR images is essential. The current standard method for estimating it is subject to high inter- and intra-rater variability, and an automated method for segmentation of residual tumor in early post-operative MRI could lead to a more accurate estimation of extent of resection. In this study, two state-of-the-art neural network architectures for pre-operative segmentation were trained for the task. The models were extensively validated on a multicenter dataset with nearly 1000 patients, from 12 hospitals in Europe and the United States. The best performance achieved was a 61\% Dice score, and the best classification performance was about 80\% balanced accuracy, with a demonstrated ability to generalize across hospitals. In addition, the segmentation performance of the best models was on par with human expert raters. The predicted segmentations can be used to accurately classify the patients into those with residual tumor, and those with gross total resection. △ Less

Submitted 18 April, 2023; originally announced April 2023.

Comments: 13 pages, 4 figures, 4 tables

ACM Class: I.4.6; J.3

Robust Stability of Gaussian Process Based Moving Horizon Estimation

Authors: Tobias M. Wolff, Victor G. Lopez, Matthias A. Müller

Abstract: In this paper, we introduce a Gaussian process based moving horizon estimation (MHE) framework. The scheme is based on offline collected data and offline hyperparameter optimization. In particular, compared to standard MHE schemes, we replace the mathematical model of the system by the posterior mean of the Gaussian process. To account for the uncertainty of the learned model, we exploit the poste… ▽ More In this paper, we introduce a Gaussian process based moving horizon estimation (MHE) framework. The scheme is based on offline collected data and offline hyperparameter optimization. In particular, compared to standard MHE schemes, we replace the mathematical model of the system by the posterior mean of the Gaussian process. To account for the uncertainty of the learned model, we exploit the posterior variance of the learned Gaussian process in the weighting matrices of the cost function of the proposed MHE scheme. We prove practical robust exponential stability of the resulting estimator using a recently proposed Lyapunov-based proof technique. Finally, the performance of the Gaussian process based MHE scheme is illustrated via a nonlinear system. △ Less

Submitted 14 June, 2023; v1 submitted 13 April, 2023; originally announced April 2023.

Comments: 8 pages

Soft Dynamic Time War** for Multi-Pitch Estimation and Beyond

Authors: Michael Krause, Christof Weiß, Meinard Müller

Abstract: Many tasks in music information retrieval (MIR) involve weakly aligned data, where exact temporal correspondences are unknown. The connectionist temporal classification (CTC) loss is a standard technique to learn feature representations based on weakly aligned training data. However, CTC is limited to discrete-valued target sequences and can be difficult to extend to multi-label problems. In this… ▽ More Many tasks in music information retrieval (MIR) involve weakly aligned data, where exact temporal correspondences are unknown. The connectionist temporal classification (CTC) loss is a standard technique to learn feature representations based on weakly aligned training data. However, CTC is limited to discrete-valued target sequences and can be difficult to extend to multi-label problems. In this article, we show how soft dynamic time war** (SoftDTW), a differentiable variant of classical DTW, can be used as an alternative to CTC. Using multi-pitch estimation as an example scenario, we show that SoftDTW yields results on par with a state-of-the-art multi-label extension of CTC. In addition to being more elegant in terms of its algorithmic formulation, SoftDTW naturally extends to real-valued target sequences. △ Less

Submitted 11 April, 2023; originally announced April 2023.

Comments: Accepted at ICASSP 2023

Distributed Model Predictive Control for Periodic Cooperation of Multi-Agent Systems

Authors: Matthias Köhler, Matthias A. Müller, Frank Allgöwer

Abstract: We consider multi-agent systems with heterogeneous, nonlinear agents subject to individual constraints that want to achieve a periodic, dynamic cooperative control goal which can be characterised by a set and a suitable cost. We propose a sequential distributed model predictive control (MPC) scheme in which agents sequentially solve an individual optimisation problem to track an artificial periodi… ▽ More We consider multi-agent systems with heterogeneous, nonlinear agents subject to individual constraints that want to achieve a periodic, dynamic cooperative control goal which can be characterised by a set and a suitable cost. We propose a sequential distributed model predictive control (MPC) scheme in which agents sequentially solve an individual optimisation problem to track an artificial periodic output trajectory. The optimisation problems are coupled through these artificial periodic output trajectories, which are communicated and penalised using the cost that characterises the cooperative goal. The agents communicate only their artificial trajectories and only once per time step. We show that under suitable assumptions, the agents can incrementally move their artificial output trajectories towards the cooperative goal, and, hence, their closed-loop output trajectories asymptotically achieve it. We illustrate the scheme with a simulation example. △ Less

Submitted 6 April, 2023; originally announced April 2023.

An Efficient Off-Policy Reinforcement Learning Algorithm for the Continuous-Time LQR Problem

Authors: Victor G. Lopez, Matthias A. Müller

Abstract: In this paper, an off-policy reinforcement learning algorithm is designed to solve the continuous-time LQR problem using only input-state data measured from the system. Different from other algorithms in the literature, we propose the use of a specific persistently exciting input as the exploration signal during the data collection step. We then show that, using this persistently excited data, the… ▽ More In this paper, an off-policy reinforcement learning algorithm is designed to solve the continuous-time LQR problem using only input-state data measured from the system. Different from other algorithms in the literature, we propose the use of a specific persistently exciting input as the exploration signal during the data collection step. We then show that, using this persistently excited data, the solution of the matrix equation in our algorithm is guaranteed to exist and to be unique at every iteration. Convergence of the algorithm to the optimal control input is also proven. Moreover, we formulate the policy evaluation step as the solution of a Sylvester-transpose equation, which increases the efficiency of its solution. Finally, a method to determine a stabilizing policy to initialize the algorithm using only measured data is proposed. △ Less

Submitted 31 March, 2023; originally announced March 2023.

Comments: 7 pages

Transient Performance of MPC for Tracking

Authors: Matthias Köhler, Lisa Krügel, Lars Grüne, Matthias A. Müller, Frank Allgöwer

Abstract: We analyse the closed-loop performance of a model predictive control (MPC) for tracking formulation with artificial references. It has been shown that such a scheme guarantees closed-loop stability and recursive feasibility for any externally supplied reference, even if it is unreachable or time-varying. The basic idea is to consider an artificial reference as an additional decision variable and t… ▽ More We analyse the closed-loop performance of a model predictive control (MPC) for tracking formulation with artificial references. It has been shown that such a scheme guarantees closed-loop stability and recursive feasibility for any externally supplied reference, even if it is unreachable or time-varying. The basic idea is to consider an artificial reference as an additional decision variable and to formulate generalised terminal ingredients with respect to it. In addition, its offset is penalised in the MPC optimisation problem, leading to closed-loop convergence to the best reachable reference. In this paper, we provide a transient performance bound on the closed loop using MPC for tracking. We employ mild assumptions on the offset cost and scale it with the prediction horizon. In this case, an increasing horizon in MPC for tracking recovers the infinite horizon optimal solution. △ Less

Submitted 24 January, 2024; v1 submitted 17 March, 2023; originally announced March 2023.

Journal ref: IEEE Control Systems Letters, vol. 7, pp. 2545-2550, 2023

On the design of persistently exciting inputs for data-driven control of linear and nonlinear systems

Authors: Mohammad Alsalti, Victor G. Lopez, Matthias A. Müller

Abstract: In the context of data-driven control, persistence of excitation (PE) of an input sequence is defined in terms of a rank condition on the Hankel matrix of the input data. For nonlinear systems, recent results employed rank conditions involving collected input and state/output data, for which no guidelines are available on how to satisfy them a priori. In this paper, we first show that a set of dis… ▽ More In the context of data-driven control, persistence of excitation (PE) of an input sequence is defined in terms of a rank condition on the Hankel matrix of the input data. For nonlinear systems, recent results employed rank conditions involving collected input and state/output data, for which no guidelines are available on how to satisfy them a priori. In this paper, we first show that a set of discrete impulses is guaranteed to be persistently exciting for any controllable LTI system. Based on this result, for certain classes of nonlinear systems, we guarantee persistence of excitation of sequences of basis functions a priori, by design of the physical input only. △ Less

Submitted 19 June, 2023; v1 submitted 15 March, 2023; originally announced March 2023.

Journal ref: IEEE Control Systems Letters, 2023

Treating Hyperthyroidism: Model Predictive Control for the Prescription of Antithyroid Agents

Authors: Tobias M. Wolff, Maylin Menzel, Johannes W. Dietrich, Matthias A. Müller

Abstract: In this work, we propose an approach to determine the dosages of antithyroid agents to treat hyperthyroid patients. Instead of relying on a trial-and-error approach as it is commonly done in clinical practice, we suggest to determine the dosages by means of a model predictive control (MPC) scheme. To this end, we extend a mathematical model of the pituitary-thyroid feedback loop such that the inta… ▽ More In this work, we propose an approach to determine the dosages of antithyroid agents to treat hyperthyroid patients. Instead of relying on a trial-and-error approach as it is commonly done in clinical practice, we suggest to determine the dosages by means of a model predictive control (MPC) scheme. To this end, we extend a mathematical model of the pituitary-thyroid feedback loop such that the intake of methimazole, a common antithyroid agent, can be considered. Based on this extension, we develop an MPC scheme to determine suitable dosages. In numerical simulations, we consider scenarios in which (i) patients are affected by Graves' disease and take the medication orally, (ii) patients are additionally affected by high intrathyroidal iodide concentrations and take the medication orally and, (iii) patients suffering from a life-threatening thyrotoxicosis, in which the medication is usually given intravenously. Our results suggest that determining the medication dosages by means of an MPC scheme is a promising alternative to the currently applied trial-and-error approach. △ Less

Submitted 20 December, 2022; originally announced December 2022.

Comments: Submitted to IFAC World Congress 2023

Comparing Approaches to Distributed Control of Fluid Systems based on Multi-Agent Systems

Authors: Kevin T. Logan, J. Marius Stürmer, Tim M. Müller, Peter F. Pelz

Abstract: Conventional control of fluid systems does not consider system-wide knowledge for optimising energy efficient operation. Distributed control of fluid systems combines reliable local control of components while using system-wide cooperation to ensure energy efficient operation. The presented work compares three approaches to distributed control based on multi-agent systems, distributed model predic… ▽ More Conventional control of fluid systems does not consider system-wide knowledge for optimising energy efficient operation. Distributed control of fluid systems combines reliable local control of components while using system-wide cooperation to ensure energy efficient operation. The presented work compares three approaches to distributed control based on multi-agent systems, distributed model predictive control (DMPC), multi-agent deep reinforcement learning (MADRL) and market mechanism design. These approaches were applied to a generic fluid system and evaluated with regard to functionality, energy efficient operation, modeling effort, reliability in the face of disruptions, and transparency of control decisions. All approaches were shown to fulfil the functionality, though a trade-off between functional quality and energy efficiency was identified. Increased modeling effort was shown to improve the performance slightly while a strong interdependence of information caused by excessive information sharing has proven to be disadvantageous. DMPC and partially observable MADRL were less sensitive to disruptions than market mechanism. In conclusion, agent-based control of fluid systems achieves greater energy efficiency than conventional methods, with values similar to centralized optimal control and thus represent a viable design approach of fluid system control. △ Less

Submitted 25 April, 2023; v1 submitted 16 December, 2022; originally announced December 2022.

Comments: 42 pages, 10 figures updated DOI in Data Availability Statement; updated links in figure captions to files in new version of dataset; dataset changed due to correcting links to raw data files in figure.json files

Informed Circular Fields for Global Reactive Obstacle Avoidance of Robotic Manipulators

Authors: Marvin Becker, Philipp Caspers, Tom Hattendorf, Torsten Lilge, Sami Haddadin, Matthias A. Müller

Abstract: In this paper a global reactive motion planning framework for robotic manipulators in complex dynamic environments is presented. In particular, the circular field predictions (CFP) planner from Becker et al. (2021) is extended to ensure obstacle avoidance of the whole structure of a robotic manipulator. Towards this end, a motion planning framework is developed that leverages global information ab… ▽ More In this paper a global reactive motion planning framework for robotic manipulators in complex dynamic environments is presented. In particular, the circular field predictions (CFP) planner from Becker et al. (2021) is extended to ensure obstacle avoidance of the whole structure of a robotic manipulator. Towards this end, a motion planning framework is developed that leverages global information about promising avoidance directions from arbitrary configuration space motion planners, resulting in improved global trajectories while reactively avoiding dynamic obstacles and decreasing the required computational power. The resulting motion planning framework is tested in multiple simulations with complex and dynamic obstacles and demonstrates great potential compared to existing motion planning approaches. △ Less

Submitted 4 August, 2023; v1 submitted 12 December, 2022; originally announced December 2022.

Comments: Accepted to IFAC World Congress 2023

Localized Shortcut Removal

Authors: Nicolas M. Müller, Jochen Jacobs, Jennifer Williams, Konstantin Böttinger

Abstract: Machine learning is a data-driven field, and the quality of the underlying datasets plays a crucial role in learning success. However, high performance on held-out test data does not necessarily indicate that a model generalizes or learns anything meaningful. This is often due to the existence of machine learning shortcuts - features in the data that are predictive but unrelated to the problem at… ▽ More Machine learning is a data-driven field, and the quality of the underlying datasets plays a crucial role in learning success. However, high performance on held-out test data does not necessarily indicate that a model generalizes or learns anything meaningful. This is often due to the existence of machine learning shortcuts - features in the data that are predictive but unrelated to the problem at hand. To address this issue for datasets where the shortcuts are smaller and more localized than true features, we propose a novel approach to detect and remove them. We use an adversarially trained lens to detect and eliminate highly predictive but semantically unconnected clues in images. In our experiments on both synthetic and real-world data, we show that our proposed approach reliably identifies and neutralizes such shortcuts without causing degradation of model performance on clean data. We believe that our approach can lead to more meaningful and generalizable machine learning models, especially in scenarios where the quality of the underlying datasets is crucial. △ Less

Submitted 23 May, 2023; v1 submitted 24 November, 2022; originally announced November 2022.

Comments: Accepted at XAI4CV @ CVPR2023

Robust peak-to-peak gain analysis using integral quadratic constraints

Authors: Lukas Schwenkel, Johannes Köhler, Matthias A. Müller, Frank Allgöwer

Abstract: This work provides a framework to compute an upper bound on the robust peak-to-peak gain of discrete-time uncertain linear systems using integral quadratic constraints (IQCs). Such bounds are of particular interest in the computation of reachable sets and the $\ell_1$-norm, as well as when safety-critical constraints need to be satisfied pointwise in time. The use of $ρ$-hard IQCs with a terminal… ▽ More This work provides a framework to compute an upper bound on the robust peak-to-peak gain of discrete-time uncertain linear systems using integral quadratic constraints (IQCs). Such bounds are of particular interest in the computation of reachable sets and the $\ell_1$-norm, as well as when safety-critical constraints need to be satisfied pointwise in time. The use of $ρ$-hard IQCs with a terminal cost enables us to deal with a wide variety of uncertainty classes, for example, we provide $ρ$-hard IQCs with a terminal cost for the class of parametric uncertainties. This approach unifies, generalizes, and significantly improves state-of-the-art methods, which is also demonstrated in a numerical example. △ Less

Submitted 17 November, 2022; originally announced November 2022.

Comments: 6 pages, submitted to IFAC WC 2023

Online convex optimization for constrained control of linear systems using a reference governor

Authors: Marko Nonhoff, Johannes Köhler, Matthias A. Müller

Abstract: In this work, we propose a control scheme for linear systems subject to pointwise in time state and input constraints that aims to minimize time-varying and a priori unknown cost functions. The proposed controller is based on online convex optimization and a reference governor. In particular, we apply online gradient descent to track the time-varying and a priori unknown optimal steady state of th… ▽ More In this work, we propose a control scheme for linear systems subject to pointwise in time state and input constraints that aims to minimize time-varying and a priori unknown cost functions. The proposed controller is based on online convex optimization and a reference governor. In particular, we apply online gradient descent to track the time-varying and a priori unknown optimal steady state of the system. Moreover, we use a $λ$-contractive set to enforce constraint satisfaction and a sufficient convergence rate of the closed-loop system to the optimal steady state. We prove that the proposed scheme is recursively feasible, ensures that the state and input constraints are satisfied at all times, and achieves a dynamic regret that is linearly bounded by the variation of the cost functions. The algorithm's performance and constraint satisfaction is illustrated by means of a simulation example. △ Less

Submitted 15 June, 2023; v1 submitted 16 November, 2022; originally announced November 2022.

Comments: Accepted for publication in the proceedings of the 2023 IFAC World Congress

A moving horizon state and parameter estimation scheme with guaranteed robust convergence

Authors: Julian D. Schiller, Matthias A. Müller

Abstract: We propose a moving horizon estimation scheme for joint state and parameter estimation for nonlinear uncertain discrete-time systems. We establish robust exponential convergence of the combined estimation error subject to process disturbances and measurement noise. We employ a joint incremental input/output-to-state stability ($δ$-IOSS) Lyapunov function to characterize nonlinear detectability for… ▽ More We propose a moving horizon estimation scheme for joint state and parameter estimation for nonlinear uncertain discrete-time systems. We establish robust exponential convergence of the combined estimation error subject to process disturbances and measurement noise. We employ a joint incremental input/output-to-state stability ($δ$-IOSS) Lyapunov function to characterize nonlinear detectability for the states and (constant) parameters of the system. Sufficient conditions for the construction of a joint $δ$-IOSS Lyapunov function are provided for a special class of nonlinear systems using a persistence of excitation condition. The theoretical results are illustrated by a numerical example. △ Less

Submitted 22 December, 2023; v1 submitted 16 November, 2022; originally announced November 2022.

Comments: Replaced by final version. Presented at IFAC World Congress 2023, Yokohama, Japan

Journal ref: IFAC-PapersOnLine, Volume 56, Issue 2, 2023, Pages 6759-6764

Data-driven Nonlinear Predictive Control for Feedback Linearizable Systems

Authors: Mohammad Alsalti, Victor G. Lopez, Julian Berberich, Frank Allgöwer, Matthias A. Müller

Abstract: We present a data-driven nonlinear predictive control approach for the class of discrete-time multi-input multi-output feedback linearizable nonlinear systems. The scheme uses a non-parametric predictive model based only on input and noisy output data along with a set of basis functions that approximate the unknown nonlinearities. Despite the noisy output data as well as the mismatch caused by the… ▽ More We present a data-driven nonlinear predictive control approach for the class of discrete-time multi-input multi-output feedback linearizable nonlinear systems. The scheme uses a non-parametric predictive model based only on input and noisy output data along with a set of basis functions that approximate the unknown nonlinearities. Despite the noisy output data as well as the mismatch caused by the use of basis functions, we show that the proposed multistep robust data-driven nonlinear predictive control scheme is recursively feasible and renders the closed-loop system practically exponentially stable. We illustrate our results on a model of a fully-actuated double inverted pendulum. △ Less

Submitted 27 March, 2023; v1 submitted 11 November, 2022; originally announced November 2022.

Comments: accepted to IFAC World Congress 2023. arXiv admin note: substantial text overlap with arXiv:2204.01148

Motion Planning using Reactive Circular Fields: A 2D Analysis of Collision Avoidance and Goal Convergence

Authors: Marvin Becker, Johannes Köhler, Sami Haddadin, Matthias A. Müller

Abstract: Recently, many reactive trajectory planning approaches were suggested in the literature because of their inherent immediate adaption in the ever more demanding cluttered and unpredictable environments of robotic systems. However, typically those approaches are only locally reactive without considering global path planning and no guarantees for simultaneous collision avoidance and goal convergence… ▽ More Recently, many reactive trajectory planning approaches were suggested in the literature because of their inherent immediate adaption in the ever more demanding cluttered and unpredictable environments of robotic systems. However, typically those approaches are only locally reactive without considering global path planning and no guarantees for simultaneous collision avoidance and goal convergence can be given. In this paper, we study a recently developed circular field (CF)-based motion planner that combines local reactive control with global trajectory generation by adapting an artificial magnetic field such that multiple trajectories around obstacles can be evaluated. In particular, we provide a mathematically rigorous analysis of this planner in a planar environment to ensure safe motion of the controlled robot. Contrary to existing results, the derived collision avoidance analysis covers the entire CF motion planning algorithm including attractive forces for goal convergence and is not limited to a specific choice of the rotation field, i.e., our guarantees are not limited to a specific potentially suboptimal trajectory. Our Lyapunov-type collision avoidance analysis is based on the definition of an (equivalent) two-dimensional auxiliary system, which enables us to provide tight, if and only if conditions for the case of a collision with point obstacles. Furthermore, we show how this analysis naturally extends to multiple obstacles and we specify sufficient conditions for goal convergence. Finally, we provide a challenging simulation scenario with multiple non-convex point cloud obstacles and demonstrate collision avoidance and goal convergence. △ Less

Submitted 3 November, 2023; v1 submitted 28 October, 2022; originally announced October 2022.

Comments: Published in IEEE Transactions on Automatic Control (Early Access)

Geometric Sparse Coding in Wasserstein Space

Authors: Marshall Mueller, Shuchin Aeron, James M. Murphy, Abiy Tasissa

Abstract: Wasserstein dictionary learning is an unsupervised approach to learning a collection of probability distributions that generate observed distributions as Wasserstein barycentric combinations. Existing methods for Wasserstein dictionary learning optimize an objective that seeks a dictionary with sufficient representation capacity via barycentric interpolation to approximate the observed training da… ▽ More Wasserstein dictionary learning is an unsupervised approach to learning a collection of probability distributions that generate observed distributions as Wasserstein barycentric combinations. Existing methods for Wasserstein dictionary learning optimize an objective that seeks a dictionary with sufficient representation capacity via barycentric interpolation to approximate the observed training data, but without imposing additional structural properties on the coefficients associated to the dictionary. This leads to dictionaries that densely represent the observed data, which makes interpretation of the coefficients challenging and may also lead to poor empirical performance when using the learned coefficients in downstream tasks. In contrast and motivated by sparse dictionary learning in Euclidean spaces, we propose a geometrically sparse regularizer for Wasserstein space that promotes representations of a data point using only nearby dictionary elements. We show this approach leads to sparse representations in Wasserstein space and addresses the problem of non-uniqueness of barycentric representation. Moreover, when data is generated as Wasserstein barycenters of fixed distributions, this regularizer facilitates the recovery of the generating distributions in cases that are ill-posed for unregularized Wasserstein dictionary learning. Through experimentation on synthetic and real data, we show that our geometrically regularized approach yields sparser and more interpretable dictionaries in Wasserstein space, which perform better in downstream applications. △ Less

Submitted 21 October, 2022; originally announced October 2022.

Comments: 24 pages

Distributed MPC for Self-Organized Cooperation of Multiagent Systems -- Extended Version

Authors: Matthias Köhler, Matthias A. Müller, Frank Allgöwer

Abstract: We present a sequential distributed model predictive control (MPC) scheme for cooperative control of multi-agent systems with dynamically decoupled heterogeneous nonlinear agents subject to individual constraints. In the scheme, we explore the idea of using tracking MPC with artificial references to let agents coordinate their cooperation without external guidance. Each agent combines a tracking M… ▽ More We present a sequential distributed model predictive control (MPC) scheme for cooperative control of multi-agent systems with dynamically decoupled heterogeneous nonlinear agents subject to individual constraints. In the scheme, we explore the idea of using tracking MPC with artificial references to let agents coordinate their cooperation without external guidance. Each agent combines a tracking MPC with artificial references, the latter penalized by a suitable coupling cost. They solve an individual optimization problem for this artificial reference and an input that tracks it, only communicating the former to its neighbors in a communication graph. This puts the cooperative problem on a different layer than the handling of the dynamics and constraints, loosening the connection between the two. We provide sufficient conditions on the formulation of the cooperative problem and the coupling cost for the closed-loop system to asymptotically achieve it. Since the dynamics and the cooperative problem are only loosely connected, classical results from distributed optimization can be used to this end. We illustrate the scheme's application to consensus and formation control. △ Less

Submitted 12 June, 2024; v1 submitted 18 October, 2022; originally announced October 2022.

Comments: Extended version of https://doi.org/10.1109/TAC.2024.3407633 containing a technical proof and additional numerical examples

Robust Data-Driven Moving Horizon Estimation for Linear Discrete-Time Systems

Authors: Tobias M. Wolff, Victor G. Lopez, Matthias A. Müller

Abstract: In this paper, a robust data-driven moving horizon estimation (MHE) scheme for linear time-invariant discrete-time systems is introduced. The scheme solely relies on offline collected data without employing any system identification step. We prove practical robust exponential stability for the setting where both the online measurements and the offline collected data are corrupted by non-vanishing… ▽ More In this paper, a robust data-driven moving horizon estimation (MHE) scheme for linear time-invariant discrete-time systems is introduced. The scheme solely relies on offline collected data without employing any system identification step. We prove practical robust exponential stability for the setting where both the online measurements and the offline collected data are corrupted by non-vanishing and bounded noise. The behavior of the novel robust data-driven MHE scheme is illustrated by means of simulation examples and compared to a standard model-based MHE scheme, where the model is identified using the same offline data as for the data-driven MHE scheme. △ Less

Submitted 28 February, 2024; v1 submitted 17 October, 2022; originally announced October 2022.

Comments: 13 pages

An Analysis Method for Metric-Level Switching in Beat Tracking

Authors: Ching-Yu Chiu, Meinard Müller, Matthew E. P. Davies, Alvin Wen-Yu Su, Yi-Hsuan Yang

Abstract: For expressive music, the tempo may change over time, posing challenges to tracking the beats by an automatic model. The model may first tap to the correct tempo, but then may fail to adapt to a tempo change, or switch between several incorrect but perceptually plausible ones (e.g., half- or double-tempo). Existing evaluation metrics for beat tracking do not reflect such behaviors, as they typical… ▽ More For expressive music, the tempo may change over time, posing challenges to tracking the beats by an automatic model. The model may first tap to the correct tempo, but then may fail to adapt to a tempo change, or switch between several incorrect but perceptually plausible ones (e.g., half- or double-tempo). Existing evaluation metrics for beat tracking do not reflect such behaviors, as they typically assume a fixed relationship between the reference beats and estimated beats. In this paper, we propose a new performance analysis method, called annotation coverage ratio (ACR), that accounts for a variety of possible metric-level switching behaviors of beat trackers. The idea is to derive sequences of modified reference beats of all metrical levels for every two consecutive reference beats, and compare every sequence of modified reference beats to the subsequences of estimated beats. We show via experiments on three datasets of different genres the usefulness of ACR when utilized alongside existing metrics, and discuss the new insights to be gained. △ Less

Submitted 13 October, 2022; originally announced October 2022.

Comments: Accepted to IEEE Signal Processing Letters (Oct. 2022)