Showing 1–50 of 99 results for author: Shames, I

Improving Visual Place Recognition Based Robot Navigation Through Verification of Localization Estimates

Authors: Owen Claxton, Connor Malone, Helen Carson, Jason Ford, Gabe Bolton, Iman Shames, Michael Milford

Abstract: Visual Place Recognition (VPR) systems often have imperfect performance, which affects robot navigation decisions. This research introduces a novel Multi-Layer Perceptron (MLP) integrity monitor for VPR which demonstrates improved performance and generalizability over the previous state-of-the-art SVM approach, removing per-environment training and reducing manual tuning requirements. We test our… ▽ More Visual Place Recognition (VPR) systems often have imperfect performance, which affects robot navigation decisions. This research introduces a novel Multi-Layer Perceptron (MLP) integrity monitor for VPR which demonstrates improved performance and generalizability over the previous state-of-the-art SVM approach, removing per-environment training and reducing manual tuning requirements. We test our proposed system in extensive real-world experiments, where we also present two real-time integrity-based VPR verification methods: an instantaneous rejection method for a robot navigating to a goal zone (Experiment 1); and a historical method that takes a best, verified, match from its recent trajectory and uses an odometer to extrapolate forwards to a current position estimate (Experiment 2). Noteworthy results for Experiment 1 include a decrease in aggregate mean along-track goal error from ~9.8m to ~3.1m in missions the robot pursued to completion, and an increase in the aggregate rate of successful mission completion from ~41% to ~55%. Experiment 2 showed a decrease in aggregate mean along-track localization error from ~2.0m to ~0.5m, and an increase in the aggregate precision of localization attempts from ~97% to ~99%. Overall, our results demonstrate the practical usefulness of a VPR integrity monitor in real-world robotics to improve VPR localization and consequent navigation performance. △ Less

Submitted 10 July, 2024; originally announced July 2024.

Comments: Currently Under Review

Online Non-Stationary Stochastic Quasar-Convex Optimization

Authors: Yuen-Man Pun, Iman Shames

Abstract: Recent research has shown that quasar-convexity can be found in applications such as identification of linear dynamical systems and generalized linear models. Such observations have in turn spurred exciting developments in design and analysis algorithms that exploit quasar-convexity. In this work, we study the online stochastic quasar-convex optimization problems in a dynamic environment. We estab… ▽ More Recent research has shown that quasar-convexity can be found in applications such as identification of linear dynamical systems and generalized linear models. Such observations have in turn spurred exciting developments in design and analysis algorithms that exploit quasar-convexity. In this work, we study the online stochastic quasar-convex optimization problems in a dynamic environment. We establish regret bounds of online gradient descent in terms of cumulative path variation and cumulative gradient variance for losses satisfying quasar-convexity and strong quasar-convexity. We then apply the results to generalized linear models (GLM) when the underlying parameter is time-varying. We establish regret bounds of online gradient descent when applying to GLMs with leaky ReLU activation function, logistic activation function, and ReLU activation function. Numerical results are presented to corroborate our findings. △ Less

Submitted 3 July, 2024; originally announced July 2024.

Minimisation of Polyak-Łojasewicz Functions Using Random Zeroth-Order Oracles

Authors: Amir Ali Farzin, Iman Shames

Abstract: The application of a zeroth-order scheme for minimising Polyak-Łojasewicz (PL) functions is considered. The framework is based on exploiting a random oracle to estimate the function gradient. The convergence of the algorithm to a global minimum in the unconstrained case and to a neighbourhood of the global minimum in the constrained case along with their corresponding complexity bounds are present… ▽ More The application of a zeroth-order scheme for minimising Polyak-Łojasewicz (PL) functions is considered. The framework is based on exploiting a random oracle to estimate the function gradient. The convergence of the algorithm to a global minimum in the unconstrained case and to a neighbourhood of the global minimum in the constrained case along with their corresponding complexity bounds are presented. The theoretical results are demonstrated via numerical examples. △ Less

Submitted 15 May, 2024; originally announced May 2024.

Self-Tuning Network Control Architectures with Joint Sensor and Actuator Selection

Authors: Karthik Ganapathy, Iman Shames, Mathias Hudoba de Badyn, Tyler Summers

Abstract: We formulate a mathematical framework for designing a self-tuning network control architecture, and propose a computationally-feasible greedy algorithm for online architecture optimization. In this setting, the locations of active sensors and actuators in the network, as well as the feedback control policy are jointly adapted using all available information about the network states and dynamics to… ▽ More We formulate a mathematical framework for designing a self-tuning network control architecture, and propose a computationally-feasible greedy algorithm for online architecture optimization. In this setting, the locations of active sensors and actuators in the network, as well as the feedback control policy are jointly adapted using all available information about the network states and dynamics to optimize a performance criterion. We show that the case with full-state feedback can be solved with dynamic programming, and in the linear-quadratic setting, the optimal cost functions and policies are piecewise quadratic and piecewise linear, respectively. Our framework is extended for joint sensor and actuator selection for dynamic output feedback control with both control performance and architecture costs. For large networks where exhaustive architecture search is prohibitive, we describe a greedy heuristic for actuator selection and propose a greedy swap** algorithm for joint sensor and actuator selection. Via numerical experiments, we demonstrate a dramatic performance improvement of greedy self-tuning architectures over fixed architectures. Our general formulation provides an extremely rich and challenging problem space with opportunities to apply a wide variety of approximation methods from stochastic control, system identification, reinforcement learning, and static architecture design for practical model-based control. △ Less

Submitted 19 January, 2024; originally announced February 2024.

Comments: 12 pages, submitted to IEEE-TCNS. arXiv admin note: text overlap with arXiv:2301.06699

On Distributed Nonconvex Optimisation Via Modified ADMM

Authors: Behnam Mafakheri, Jonathan H. Manton, Iman Shames

Abstract: This paper addresses the problem of nonconvex nonsmooth decentralised optimisation in multi-agent networks with undirected connected communication graphs. Our contribution lies in introducing an algorithmic framework designed for the distributed minimisation of the sum of a smooth (possibly nonconvex and non-separable) function and a convex (possibly nonsmooth and non-separable) regulariser. The p… ▽ More This paper addresses the problem of nonconvex nonsmooth decentralised optimisation in multi-agent networks with undirected connected communication graphs. Our contribution lies in introducing an algorithmic framework designed for the distributed minimisation of the sum of a smooth (possibly nonconvex and non-separable) function and a convex (possibly nonsmooth and non-separable) regulariser. The proposed algorithm can be seen as a modified version of the ADMM algorithm where, at each step, an "inner loop" needs to be iterated for a number of iterations. The role of the inner loop is to aggregate and disseminate information across the network. We observe that a naive decentralised approach (one iteration of the inner loop) may not converge. We establish the asymptotic convergence of the proposed algorithm to the set of stationary points of the nonconvex problem where the number of iterations of the inner loop increases logarithmically with the step count of the ADMM algorithm. We present numerical results demonstrating the proposed method's correctness and performance. △ Less

Submitted 6 December, 2023; originally announced December 2023.

Comments: 6 pages, 1 Figure

Distributionally Time-Varying Online Stochastic Optimization under Polyak-Łojasiewicz Condition with Application in Conditional Value-at-Risk Statistical Learning

Authors: Yuen-Man Pun, Farhad Farokhi, Iman Shames

Abstract: In this work, we consider a sequence of stochastic optimization problems following a time-varying distribution via the lens of online optimization. Assuming that the loss function satisfies the Polyak-Łojasiewicz condition, we apply online stochastic gradient descent and establish its dynamic regret bound that is composed of cumulative distribution drifts and cumulative gradient biases caused by s… ▽ More In this work, we consider a sequence of stochastic optimization problems following a time-varying distribution via the lens of online optimization. Assuming that the loss function satisfies the Polyak-Łojasiewicz condition, we apply online stochastic gradient descent and establish its dynamic regret bound that is composed of cumulative distribution drifts and cumulative gradient biases caused by stochasticity. The distribution metric we adopt here is Wasserstein distance, which is well-defined without the absolute continuity assumption or with a time-varying support set. We also establish a regret bound of online stochastic proximal gradient descent when the objective function is regularized. Moreover, we show that the above framework can be applied to the Conditional Value-at-Risk (CVaR) learning problem. Particularly, we improve an existing proof on the discovery of the PL condition of the CVaR problem, resulting in a regret bound of online stochastic gradient descent. △ Less

Submitted 17 September, 2023; originally announced September 2023.

Online Interior-point Methods for Time-varying Equality-constrained Optimization

Authors: Jean-Luc Lupien, Iman Shames, Antoine Lesage-Landry

Abstract: An important challenge in the online convex optimization (OCO) setting is to incorporate generalized inequalities and time-varying constraints. The inclusion of constraints in OCO widens the applicability of such algorithms to dynamic and safety-critical settings such as the online optimal power flow (OPF) problem. In this work, we propose the first projection-free OCO algorithm admitting time-var… ▽ More An important challenge in the online convex optimization (OCO) setting is to incorporate generalized inequalities and time-varying constraints. The inclusion of constraints in OCO widens the applicability of such algorithms to dynamic and safety-critical settings such as the online optimal power flow (OPF) problem. In this work, we propose the first projection-free OCO algorithm admitting time-varying linear constraints and convex generalized inequalities: the online interior-point method for time-varying equality constraints (OIPM-TEC). We derive simultaneous sublinear dynamic regret and constraint violation bounds for OIPM-TEC under standard assumptions. For applications where a given tolerance around optima is accepted, we employ an alternative OCO performance metric -- the epsilon-regret -- and a more computationally efficient algorithm, the epsilon-OIPM-TEC, that possesses sublinear bounds under this metric. Finally, we showcase the performance of these two algorithms on an online OPF problem and compare them to another OCO algorithm from the literature. △ Less

Submitted 30 May, 2024; v1 submitted 30 July, 2023; originally announced July 2023.

A Robust Control Approach to Asymptotic Optimality of the Heavy Ball Method for Optimization of Quadratic Functions

Authors: V. Ugrinovskii, I. R. Petersen, I. Shames

Abstract: Among first order optimization methods, Polyak's heavy ball method has long been known to guarantee the asymptotic rate of convergence matching Nesterov's lower bound for functions defined in an infinite-dimensional space. In this paper, we use results on the robust gain margin of linear uncertain feedback control systems to show that the heavy ball method is provably worst-case asymptotically opt… ▽ More Among first order optimization methods, Polyak's heavy ball method has long been known to guarantee the asymptotic rate of convergence matching Nesterov's lower bound for functions defined in an infinite-dimensional space. In this paper, we use results on the robust gain margin of linear uncertain feedback control systems to show that the heavy ball method is provably worst-case asymptotically optimal when applied to quadratic functions in a finite dimensional space. △ Less

Submitted 11 May, 2023; originally announced May 2023.

Comments: Accepted for publication in Automatica

Distributed Block Coordinate Moving Horizon Estimation for 2D Visual-Inertial-Odometry SLAM

Authors: Emilien Flayac, Iman Shames

Abstract: This paper presents a Visual Inertial Odometry Landmark-based Simultaneous Localisation and Map** algorithm based on a distributed block coordinate nonlinear Moving Horizon Estimation scheme. The main advantage of the proposed method is that the updates on the position of the landmarks are based on a Bundle Adjustment technique that can be parallelised over the landmarks. The performance of the… ▽ More This paper presents a Visual Inertial Odometry Landmark-based Simultaneous Localisation and Map** algorithm based on a distributed block coordinate nonlinear Moving Horizon Estimation scheme. The main advantage of the proposed method is that the updates on the position of the landmarks are based on a Bundle Adjustment technique that can be parallelised over the landmarks. The performance of the method is demonstrated in simulations in different environments and with different types of robot trajectory. Circular and wiggling patterns in the trajectory lead to better estimation performance than straight ones, confirming what is expected from recent nonlinear observability theory. △ Less

Submitted 4 April, 2023; originally announced April 2023.

Nonlinear Dual control based on Fast Moving Horizon estimation and Model Predictive Control with an observability constraint

Authors: Emilien Flayac, Girish Nair, Iman Shames

Abstract: This paper proposes an algorithm that combines Fast Moving Horizon Parameter Estimation and Model Predictive Control subject to an observability constraint designed to ensure a lower bound on the performance of the parameter estimator. Output-feedback stability is proved through input-to-state stability of the state/error system under a small noise and initial error assumption. Numerical experimen… ▽ More This paper proposes an algorithm that combines Fast Moving Horizon Parameter Estimation and Model Predictive Control subject to an observability constraint designed to ensure a lower bound on the performance of the parameter estimator. Output-feedback stability is proved through input-to-state stability of the state/error system under a small noise and initial error assumption. Numerical experiments have been carried out in the case of Active Simultaneous Localisation and Map** (SLAM). △ Less

Submitted 24 March, 2023; originally announced March 2023.

Regret Analysis of Online LQR Control via Trajectory Prediction and Tracking: Extended Version

Authors: Yitian Chen, Timothy L. Molloy, Tyler Summers, Iman Shames

Abstract: In this paper, we propose and analyze a new method for online linear quadratic regulator (LQR) control with a priori unknown time-varying cost matrices. The cost matrices are revealed sequentially with the potential for future values to be previewed over a short window. Our novel method involves using the available cost matrices to predict the optimal trajectory, and a tracking controller to drive… ▽ More In this paper, we propose and analyze a new method for online linear quadratic regulator (LQR) control with a priori unknown time-varying cost matrices. The cost matrices are revealed sequentially with the potential for future values to be previewed over a short window. Our novel method involves using the available cost matrices to predict the optimal trajectory, and a tracking controller to drive the system towards it. We adopted the notion of dynamic regret to measure the performance of this proposed online LQR control method, with our main result being that the (dynamic) regret of our method is upper bounded by a constant. Moreover, the regret upper bound decays exponentially with the preview window length, and is extendable to systems with disturbances. We show in simulations that our proposed method offers improved performance compared to other previously proposed online LQR methods. △ Less

Submitted 20 February, 2023; originally announced February 2023.

Comments: Submitted to L4DC2023

MSC Class: 49N10; 49M05

The Attractor of the Replicator Dynamic in Zero-Sum Games

Authors: Oliver Biggar, Iman Shames

Abstract: In this paper we characterise the long-run behaviour of the replicator dynamic in zero-sum games (symmetric or non-symmetric). Specifically, we prove that every zero-sum game possesses a unique global replicator attractor, which we then characterise. Most surprisingly, this attractor depends only on each player's preference order over their own strategies and not on the cardinal payoff values, def… ▽ More In this paper we characterise the long-run behaviour of the replicator dynamic in zero-sum games (symmetric or non-symmetric). Specifically, we prove that every zero-sum game possesses a unique global replicator attractor, which we then characterise. Most surprisingly, this attractor depends only on each player's preference order over their own strategies and not on the cardinal payoff values, defined by a finite directed graph we call the game's preference graph. When the game is symmetric, this graph is a tournament whose nodes are strategies; when the game is not symmetric, this graph is the game's response graph. We discuss the consequences of our results on chain recurrence and Nash equilibria. △ Less

Submitted 5 February, 2024; v1 submitted 1 February, 2023; originally announced February 2023.

Comments: 17 pages, 2 figures. Accepted version, to appear at ALT 2024

Self-Tuning Network Control Architectures

Authors: Tyler Summers, Karthik Ganapathy, Iman Shames, Mathias Hudoba de Badyn

Abstract: We formulate a general mathematical framework for self-tuning network control architecture design. This problem involves jointly adapting the locations of active sensors and actuators in the network and the feedback control policy to all available information about the time-varying network state and dynamics to optimize a performance criterion. We propose a general solution structure analogous to… ▽ More We formulate a general mathematical framework for self-tuning network control architecture design. This problem involves jointly adapting the locations of active sensors and actuators in the network and the feedback control policy to all available information about the time-varying network state and dynamics to optimize a performance criterion. We propose a general solution structure analogous to the classical self-tuning regulator from adaptive control. We show that a special case with full-state feedback can be solved in principle with dynamic programming, and in the linear quadratic setting the optimal cost functions and policies are piecewise quadratic and piecewise linear, respectively. For large networks where exhaustive architecture search is prohibitive, we describe a greedy heuristic for joint architecture-policy design. We demonstrate in numerical experiments that self-tuning architectures can provide dramatically improved performance over fixed architectures. Our general formulation provides an extremely rich and challenging problem space with opportunities to apply a wide variety of approximation methods from stochastic control, system identification, reinforcement learning, and static architecture design. △ Less

Submitted 16 January, 2023; originally announced January 2023.

Comments: 6 pages, 5 figures

Journal ref: 61st Conference on Decision and Control, pp 5876-5881, 2022

First order online optimisation using forward gradients in over-parameterised systems

Authors: Behnam Mafakheri, Iman Shames, Jonathan H. Manton

Abstract: The success of deep learning over the past decade mainly relies on gradient-based optimisation and backpropagation. This paper focuses on analysing the performance of first-order gradient-based optimisation algorithms, gradient descent and proximal gradient, with time-varying non-convex cost function under (proximal) Polyak-Łojasiewicz condition. Specifically, we focus on using the forward mode of… ▽ More The success of deep learning over the past decade mainly relies on gradient-based optimisation and backpropagation. This paper focuses on analysing the performance of first-order gradient-based optimisation algorithms, gradient descent and proximal gradient, with time-varying non-convex cost function under (proximal) Polyak-Łojasiewicz condition. Specifically, we focus on using the forward mode of automatic differentiation to compute gradients in the fast-changing problems where calculating gradients using the backpropagation algorithm is either impossible or inefficient. Upper bounds for tracking and asymptotic errors are derived for various cases, showing the linear convergence to a solution or a neighbourhood of an optimal solution, where the convergence rate decreases with the increase in the dimension of the problem. We show that for a solver with constraints on computing resources, the number of forward gradient iterations at each step can be a design parameter that trades off between the tracking performance and computing constraints. △ Less

Submitted 6 December, 2022; v1 submitted 28 November, 2022; originally announced November 2022.

Comments: 18 pages, 1 figure

Minimum-Time Escape from a Circular Region for a Dubins Car

Authors: Timothy L. Molloy, Iman Shames

Abstract: We investigate the problem of finding paths that enable a robot modeled as a Dubins car (i.e., a constant-speed finite-turn-rate unicycle) to escape from a circular region of space in minimum time. This minimum-time escape problem arises in marine, aerial, and ground robotics in situations where a safety region has been violated and must be exited before a potential negative consequence occurs (e.… ▽ More We investigate the problem of finding paths that enable a robot modeled as a Dubins car (i.e., a constant-speed finite-turn-rate unicycle) to escape from a circular region of space in minimum time. This minimum-time escape problem arises in marine, aerial, and ground robotics in situations where a safety region has been violated and must be exited before a potential negative consequence occurs (e.g., a collision). Using the tools of nonlinear optimal control theory, we show that a surprisingly simple closed-form feedback control law solves this minimum-time escape problem, and that the minimum-time paths have an elegant geometric interpretation. △ Less

Submitted 3 November, 2022; originally announced November 2022.

Comments: 7 pages, 5 figures, accepted for 12th IFAC Symposium on Nonlinear Control Systems (NOLCOS)

The Replicator Dynamic, Chain Components and the Response Graph

Authors: Oliver Biggar, Iman Shames

Abstract: In this paper we examine the relationship between the flow of the replicator dynamic, the continuum limit of Multiplicative Weights Update, and a game's response graph. We settle an open problem establishing that under the replicator, sink chain components -- a topological notion of long-run outcome of a dynamical system -- always exist and are approximated by the sink connected components of the… ▽ More In this paper we examine the relationship between the flow of the replicator dynamic, the continuum limit of Multiplicative Weights Update, and a game's response graph. We settle an open problem establishing that under the replicator, sink chain components -- a topological notion of long-run outcome of a dynamical system -- always exist and are approximated by the sink connected components of the game's response graph. More specifically, each sink chain component contains a sink connected component of the response graph, as well as all mixed strategy profiles whose support consists of pure profiles in the same connected component, a set we call the content of the connected component. As a corollary, all profiles are chain recurrent in games with strongly connected response graphs. In any two-player game sharing a response graph with a zero-sum game, the sink chain component is unique. In two-player zero-sum and potential games the sink chain components and sink connected components are in a one-to-one correspondence, and we conjecture that this holds in all games. △ Less

Submitted 31 January, 2023; v1 submitted 30 September, 2022; originally announced September 2022.

Comments: 22 pages, 2 figures. Accepted version. To appear in Algorithmic Learning Theory 2023

The graph structure of two-player games

Authors: Oliver Biggar, Iman Shames

Abstract: In this paper we analyse two-player games by their response graphs. The response graph has nodes which are strategy profiles, with an arc between profiles if they differ in the strategy of a single player, with the direction of the arc indicating the preferred option for that player. Response graphs, and particularly their sink strongly connected components, play an important role in modern techni… ▽ More In this paper we analyse two-player games by their response graphs. The response graph has nodes which are strategy profiles, with an arc between profiles if they differ in the strategy of a single player, with the direction of the arc indicating the preferred option for that player. Response graphs, and particularly their sink strongly connected components, play an important role in modern techniques in evolutionary game theory and multi-agent learning. We show that the response graph is a simple and well-motivated model of strategic interaction which captures many non-trivial properties of a game, despite not depending on cardinal payoffs. We characterise the games which share a response graph with a zero-sum or potential game respectively, and demonstrate a duality between these sets. This allows us to understand the influence of these properties on the response graph. The response graphs of Matching Pennies and Coordination are shown to play a key role in all two-player games: every non-iteratively-dominated strategy takes part in a subgame with these graph structures. As a corollary, any game sharing a response graph with both a zero-sum game and potential game must be dominance-solvable. Finally, we demonstrate our results on some larger games. △ Less

Submitted 13 February, 2023; v1 submitted 21 September, 2022; originally announced September 2022.

Comments: 16 pages, 11 figures

Journal ref: Scientific Reports 13 (2023), 1833

A Logistic Regression Approach to Field Estimation Using Binary Measurements

Authors: Alex S. Leong, Mohammad Zamani, Iman Shames

Abstract: In this letter, we consider the problem of field estimation using binary measurements. Previous work has formulated the problem as a parameter estimation problem, with the parameter estimation carried out in an online manner using sequential Monte Carlo techniques. In the current work, we consider an alternative approach to the parameter estimation based on online logistic regression. The develope… ▽ More In this letter, we consider the problem of field estimation using binary measurements. Previous work has formulated the problem as a parameter estimation problem, with the parameter estimation carried out in an online manner using sequential Monte Carlo techniques. In the current work, we consider an alternative approach to the parameter estimation based on online logistic regression. The developed algorithm is less computationally intensive than the sequential Monte Carlo approach, while having more reliable estimation performance. △ Less

Submitted 1 June, 2022; originally announced June 2022.

Robust Data-Driven Output Feedback Control via Bootstrapped Multiplicative Noise

Authors: Benjamin Gravell, Iman Shames, Tyler Summers

Abstract: We propose a robust data-driven output feedback control algorithm that explicitly incorporates inherent finite-sample model estimate uncertainties into the control design. The algorithm has three components: (1) a subspace identification nominal model estimator; (2) a bootstrap resampling method that quantifies non-asymptotic variance of the nominal model estimate; and (3) a non-conventional robus… ▽ More We propose a robust data-driven output feedback control algorithm that explicitly incorporates inherent finite-sample model estimate uncertainties into the control design. The algorithm has three components: (1) a subspace identification nominal model estimator; (2) a bootstrap resampling method that quantifies non-asymptotic variance of the nominal model estimate; and (3) a non-conventional robust control design method comprising a coupled optimal dynamic output feedback filter and controller with multiplicative noise. A key advantage of the proposed approach is that the system identification and robust control design procedures both use stochastic uncertainty representations, so that the actual inherent statistical estimation uncertainty directly aligns with the uncertainty the robust controller is being designed against. Moreover, the control design method accommodates a highly structured uncertainty representation that can capture uncertainty shape more effectively than existing approaches. We show through numerical experiments that the proposed robust data-driven output feedback controller can significantly outperform a certainty equivalent controller on various measures of sample complexity and stability robustness. △ Less

Submitted 10 May, 2022; originally announced May 2022.

MITL Verification Under Timing Uncertainty

Authors: Daniel Selvaratnam, Michael Cantoni, J. M. Davoren, Iman Shames

Abstract: A Metric Interval Temporal Logic (MITL) verification algorithm is presented. It verifies continuous-time signals without relying on high frequency sampling. Instead, it is assumed that collections of over- and under-approximating intervals are available for the times at which the individual atomic propositions hold true for a given signal. These are combined inductively to generate corresponding o… ▽ More A Metric Interval Temporal Logic (MITL) verification algorithm is presented. It verifies continuous-time signals without relying on high frequency sampling. Instead, it is assumed that collections of over- and under-approximating intervals are available for the times at which the individual atomic propositions hold true for a given signal. These are combined inductively to generate corresponding over- and under-approximations for the specified MITL formula. The gap between the over- and under-approximations reflects timing uncertainty with respect to the signal being verified, thereby providing a quantitative measure of the conservativeness of the algorithm. The verification is exact when the over-approximations for the atomic propositions coincide with the under-approximations. Numerical examples are provided to illustrate. △ Less

Submitted 16 August, 2022; v1 submitted 22 April, 2022; originally announced April 2022.

Comments: To appear in proceedings of FORMATS 2022

Risk-Bounded Temporal Logic Control of Continuous-Time Stochastic Systems

Authors: Sleiman Safaoui, Lars Lindemann, Iman Shames, Tyler H. Summers

Abstract: Motivated by the recent interest in risk-aware control, we study a continuous-time control synthesis problem to bound the risk that a stochastic linear system violates a given specification. We use risk signal temporal logic as a specification formalism in which distributionally robust risk predicates are considered and equipped with the usual Boolean and temporal operators. Our control approach r… ▽ More Motivated by the recent interest in risk-aware control, we study a continuous-time control synthesis problem to bound the risk that a stochastic linear system violates a given specification. We use risk signal temporal logic as a specification formalism in which distributionally robust risk predicates are considered and equipped with the usual Boolean and temporal operators. Our control approach relies on reformulating these risk predicates as deterministic predicates over mean and covariance states of the system. We then obtain a timed sequence of sets of mean and covariance states from the timed automata representation of the specification. To avoid an explosion in the number of automata states, we propose heuristics to find candidate sequences effectively. To execute and check dynamic feasibility of these sequences, we present a sampled-data control technique based on time discretization and constraint tightening that allows to perform timed transitions while satisfying the continuous-time constraints. △ Less

Submitted 8 April, 2022; originally announced April 2022.

Comments: 8 pages, 4 figures, contributed paper at the 2022 American Control Conference (ACC) in Atlanta, GA

Probabilistic Data Association for Semantic SLAM at Scale

Authors: Elad Michael, Tyler Summers, Tony A. Wood, Chris Manzie, Iman Shames

Abstract: With advances in image processing and machine learning, it is now feasible to incorporate semantic information into the problem of simultaneous localisation and map** (SLAM). Previously, SLAM was carried out using lower level geometric features (points, lines, and planes) which are often view-point dependent and error prone in visually repetitive environments. Semantic information can improve th… ▽ More With advances in image processing and machine learning, it is now feasible to incorporate semantic information into the problem of simultaneous localisation and map** (SLAM). Previously, SLAM was carried out using lower level geometric features (points, lines, and planes) which are often view-point dependent and error prone in visually repetitive environments. Semantic information can improve the ability to recognise previously visited locations, as well as maintain sparser maps for long term SLAM applications. However, SLAM in repetitive environments has the critical problem of assigning measurements to the landmarks which generated them. In this paper, we use k-best assignment enumeration to compute marginal assignment probabilities for each measurement landmark pair, in real time. We present numerical studies on the KITTI dataset to demonstrate the effectiveness and speed of the proposed framework. △ Less

Submitted 25 February, 2022; originally announced February 2022.

Comments: 6 Pages, 3 figures, submitted to Robotics and Automation Letters and the IROS 2020 conference

MSC Class: 4104 (Primary); 05-08 (Secondary)

Global convergence and asymptotic optimality of the heavy ball method for a class of non-convex optimization problems

Authors: Valery Ugrinovskii, Ian R. Petersen, Iman Shames

Abstract: In this letter we revisit the famous heavy ball method and study its global convergence for a class of non-convex problems with sector-bounded gradient. We characterize the parameters that render the method globally convergent and yield the best $R$-convergence factor. We show that for this family of functions, this convergence factor is superior to the factor obtained from the triple momentum met… ▽ More In this letter we revisit the famous heavy ball method and study its global convergence for a class of non-convex problems with sector-bounded gradient. We characterize the parameters that render the method globally convergent and yield the best $R$-convergence factor. We show that for this family of functions, this convergence factor is superior to the factor obtained from the triple momentum method. △ Less

Submitted 24 March, 2022; v1 submitted 6 February, 2022; originally announced February 2022.

Comments: 6 pages, 4 figures, to appear in CSS Letters

MSC Class: 90C26 (Primary); 93D09 (Secondary); 65K05

Online Convex Optimization Using Coordinate Descent Algorithms

Authors: Yankai Lin, Iman Shames, Dragan Nešić

Abstract: This paper considers the problem of online optimization where the objective function is time-varying. In particular, we extend coordinate descent type algorithms to the online case, where the objective function varies after a finite number of iterations of the algorithm. Instead of solving the problem exactly at each time step, we only apply a finite number of iterations at each time step. Commonl… ▽ More This paper considers the problem of online optimization where the objective function is time-varying. In particular, we extend coordinate descent type algorithms to the online case, where the objective function varies after a finite number of iterations of the algorithm. Instead of solving the problem exactly at each time step, we only apply a finite number of iterations at each time step. Commonly used notions of regret are used to measure the performance of the online algorithm. Moreover, coordinate descent algorithms with different updating rules are considered, including both deterministic and stochastic rules that are developed in the literature of classical offline optimization. A thorough regret analysis is given for each case. Finally, numerical simulations are provided to illustrate the theoretical results. △ Less

Submitted 25 April, 2024; v1 submitted 24 January, 2022; originally announced January 2022.

Comments: Accepted for publication in Automatica

MSC Class: 68Q32 (Primary); 68T05; 90C25 (Secondary)

Journal ref: Automatica, vol. 165, Article 111681, 2024

Risk Bounded Nonlinear Robot Motion Planning With Integrated Perception & Control

Authors: Venkatraman Renganathan, Sleiman Safaoui, Aadi Kothari, Benjamin Gravell, Iman Shames, Tyler Summers

Abstract: Robust autonomy stacks require tight integration of perception, motion planning, and control layers, but these layers often inadequately incorporate inherent perception and prediction uncertainties, either ignoring them altogether or making questionable assumptions of Gaussianity. Robots with nonlinear dynamics and complex sensing modalities operating in an uncertain environment demand more carefu… ▽ More Robust autonomy stacks require tight integration of perception, motion planning, and control layers, but these layers often inadequately incorporate inherent perception and prediction uncertainties, either ignoring them altogether or making questionable assumptions of Gaussianity. Robots with nonlinear dynamics and complex sensing modalities operating in an uncertain environment demand more careful consideration of how uncertainties propagate across stack layers. We propose a framework to integrate perception, motion planning, and control by explicitly incorporating perception and prediction uncertainties into planning so that risks of constraint violation can be mitigated. Specifically, we use a nonlinear model predictive control based steering law coupled with a decorrelation scheme based Unscented Kalman Filter for state and environment estimation to propagate the robot state and environment uncertainties. Subsequently, we use distributionally robust risk constraints to limit the risk in the presence of these uncertainties. Finally, we present a layered autonomy stack consisting of a nonlinear steering-based distributionally robust motion planning module and a reference trajectory tracking module. Our numerical experiments with nonlinear robot models and an urban driving simulator show the effectiveness of our proposed approaches. △ Less

Submitted 5 January, 2022; originally announced January 2022.

Comments: arXiv admin note: text overlap with arXiv:2002.02928

Gradient Free Cooperative Seeking of a Moving Source

Authors: Elad Michael, Chris Manzie, Tony A. Wood, Daniel Zelazo, Iman Shames

Abstract: In this paper, we consider the optimisation of a time varying scalar field by a network of agents with no gradient information. We propose a composite control law, blending extremum seeking with formation control in order to converge to the extrema faster by minimising the gradient estimation error. By formalising the relationship between the formation and the gradient estimation error, we provide… ▽ More In this paper, we consider the optimisation of a time varying scalar field by a network of agents with no gradient information. We propose a composite control law, blending extremum seeking with formation control in order to converge to the extrema faster by minimising the gradient estimation error. By formalising the relationship between the formation and the gradient estimation error, we provide a novel analysis to prove the convergence of the network to a bounded neighbourhood of the field's time varying extrema. We assume the time-varying field satisfies the Polyak Lojasiewicz inequality and the gradient is Lipschitz continuous at each iteration. Numerical studies and comparisons are provided to support the theoretical results. △ Less

Submitted 30 June, 2022; v1 submitted 2 January, 2022; originally announced January 2022.

Comments: 13 pages, 6 figures, submitted to Automatica

The Anomalous Formation of Irradiation Induced Nitrogen-Vacancy Centers in 5-Nanometer-Sized Detonation Nanodiamonds

Authors: Frederick T. -K. So, Alexander I. Shames, Daiki Terada, Takuya Genjo, Hiroki Morishita, Izuru Ohki, Takeshi Ohshima, Shinobu Onoda, Hideaki Takashima, Shigeki Takeuchi, Norikazu Mizuochi, Ryuji Igarashi, Masahiro Shirakawa, Takuya F. Segawa

Abstract: Nanodiamonds containing negatively charged nitrogen-vacancy (NV$^-$) centers are versatile room-temperature quantum sensors in a growing field of research. Yet, knowledge regarding the NV-formation mechanism in very small particles is still limited. This study focuses on the formation of the smallest NV$^-$-containing diamonds, 5 nm detonation nanodiamonds (DNDs). As a reliable method to quantify… ▽ More Nanodiamonds containing negatively charged nitrogen-vacancy (NV$^-$) centers are versatile room-temperature quantum sensors in a growing field of research. Yet, knowledge regarding the NV-formation mechanism in very small particles is still limited. This study focuses on the formation of the smallest NV$^-$-containing diamonds, 5 nm detonation nanodiamonds (DNDs). As a reliable method to quantify NV$^-$ centers in nanodiamonds, half-field signals in electron paramagnetic resonance (EPR) spectroscopy are recorded. By comparing the NV$^-$ concentration with a series of nanodiamonds from high-pressure high-temperature (HPHT) synthesis (10 - 100 nm), it is shown that the formation process in 5 nm DNDs is unique in several aspects. NV$^-$ centers in DNDs are already formed at the stage of electron irradiation, without the need for high-temperature annealing. The effect is explained in terms of "self-annealing", where size and type dependent effects enable vacancy migration close to room temperature. Although our experiments show that NV$^-$ concentration generally increases with particle size, remarkably, the NV$^-$ concentration in 5 nm DNDs surpasses that of 20 nm-sized nanodiamonds. Using Monte Carlo simulations, we show that the ten times higher substitutional nitrogen concentration in DNDs compensates the vacancy loss induced by the large relative particle surface. Upon electron irradiation at a fluence of $1.5 \times 10 ^{19}$ e$^-$/cm$^2$, DNDs show a 12.5-fold increment in the NV$^-$ concentration with no sign of saturation. These findings can be of interest for the creation of defects in other very small semiconductor nanoparticles beyond NV-nanodiamonds as quantum sensors. △ Less

Submitted 12 December, 2021; originally announced December 2021.

Modular Decomposition of Hierarchical Finite State Machines

Authors: Oliver Biggar, Mohammad Zamani, Iman Shames

Abstract: In this paper we develop an analogue of the graph-theoretic `modular decomposition' in automata theory. This decomposition allows us to identify hierarchical finite state machines (HFSMs) equivalent to a given finite state machine (FSM). We first define a module of an FSM, which is a collection of nodes which can be treated as a nested FSM. We then identify a natural subset of FSM modules called t… ▽ More In this paper we develop an analogue of the graph-theoretic `modular decomposition' in automata theory. This decomposition allows us to identify hierarchical finite state machines (HFSMs) equivalent to a given finite state machine (FSM). We first define a module of an FSM, which is a collection of nodes which can be treated as a nested FSM. We then identify a natural subset of FSM modules called thin modules, which are algebraically well-behaved. We construct a linear-space directed graph, which uniquely represents every thin module, and hence every equivalent (thin) HFSM. We call this graph the modular decomposition. The modular decomposition makes clear the significant common structure underlying equivalent HFSMs, and allows us to efficiently construct equivalent HFSMs. Finally, we provide an $O(n^2k)$ algorithm for constructing the modular decomposition of an $n$-state $k$-symbol FSM. △ Less

Submitted 20 July, 2023; v1 submitted 8 November, 2021; originally announced November 2021.

Comments: 28 pages, 5 figures. This version shortens and restructures the paper

Learning Safety Filters for Unknown Discrete-Time Linear Systems

Authors: Farhad Farokhi, Alex S. Leong, Mohammad Zamani, Iman Shames

Abstract: A learning-based safety filter is developed for discrete-time linear time-invariant systems with unknown models subject to Gaussian noises with unknown covariance. Safety is characterized using polytopic constraints on the states and control inputs. The empirically learned model and process noise covariance with their confidence bounds are used to construct a robust optimization problem for minima… ▽ More A learning-based safety filter is developed for discrete-time linear time-invariant systems with unknown models subject to Gaussian noises with unknown covariance. Safety is characterized using polytopic constraints on the states and control inputs. The empirically learned model and process noise covariance with their confidence bounds are used to construct a robust optimization problem for minimally modifying nominal control actions to ensure safety with high probability. The optimization problem relies on tightening the original safety constraints. The magnitude of the tightening is larger at the beginning since there is little information to construct reliable models, but shrinks with time as more data becomes available. △ Less

Submitted 8 May, 2023; v1 submitted 31 October, 2021; originally announced November 2021.

Toward nonlinear dynamic control over encrypted data for infinite time horizon

Authors: Junsoo Kim, Farhad Farokhi, Iman Shames, Hyungbo Shim

Abstract: Recent studies on encrypted control using homomorphic encryption allow secure operation by directly performing computations on encrypted data without decryption. Implementing dynamic controllers on encrypted data presents unique challenges due to limitations on the number of operations on an encrypted message. Hence, it may not be possible to perform the recursive operations for an infinite time h… ▽ More Recent studies on encrypted control using homomorphic encryption allow secure operation by directly performing computations on encrypted data without decryption. Implementing dynamic controllers on encrypted data presents unique challenges due to limitations on the number of operations on an encrypted message. Hence, it may not be possible to perform the recursive operations for an infinite time horizon. In this note, we demonstrate that it is possible to run a dynamic controller over encrypted data for an infinite time horizon if the output of the controller can be represented as a function of a fixed number of previous inputs and outputs. The presented implementation requires encryption at both input and output of the plant. We identify a class of nonlinear systems that can accommodate the proposed implementation. The closed-loop performance can be guaranteed using the proposed encrypted controller by ensuring that quantization error is made arbitrarily small with appropriate choice of parameters. We show that the proposed method is amenable to linear systems (as a subset of the said nonlinear systems) with performance guarantees. △ Less

Submitted 12 October, 2021; originally announced October 2021.

Comments: 3 pages, previously presented at the 21st IFAC World Congress, 2020

Zeroth-order optimisation on subsets of symmetric matrices with application to MPC tuning

Authors: Alejandro I. Maass, Chris Manzie, Iman Shames, Hayato Nakada

Abstract: This paper provides a zeroth-order optimisation framework for non-smooth and possibly non-convex cost functions with matrix parameters that are real and symmetric. We provide complexity bounds on the number of iterations required to ensure a given accuracy level for both the convex and non-convex case. The derived complexity bounds for the convex case are less conservative than available bounds in… ▽ More This paper provides a zeroth-order optimisation framework for non-smooth and possibly non-convex cost functions with matrix parameters that are real and symmetric. We provide complexity bounds on the number of iterations required to ensure a given accuracy level for both the convex and non-convex case. The derived complexity bounds for the convex case are less conservative than available bounds in the literature since we exploit the symmetric structure of the underlying matrix space. Moreover, the non-convex complexity bounds are novel for the class of optimisation problems we consider. The utility of the framework is evident in the suite of applications that use symmetric matrices as tuning parameters. Of primary interest here is the challenge of tuning the gain matrices in model predictive controllers, as this is a challenge known to be inhibiting industrial implementation of these architectures. To demonstrate the framework we consider the problem of MIMO diesel air-path control, and consider implementing the framework iteratively ``in-the-loop'' to reduce tracking error on the output channels. Both simulations and experimental results are included to illustrate the effectiveness of the proposed framework over different engine drive cycles. △ Less

Submitted 27 June, 2021; originally announced June 2021.

Comments: 27 pages, 11 figures

Temporal Logic Planning for Minimum-Time Positioning of Multiple Threat-Seduction Decoys

Authors: Tony A. Wood, Mitchell Khoo, Elad Michael, Chris Manzie, Iman Shames

Abstract: Reusable decoys offer a cost-effective alternative to the single-use hardware commonly applied to protect surface assets from threats. Such decoys portray fake assets to lure threats away from the true asset. To deceive a threat, a decoy first has to position itself such that it can break the radar lock. Considering multiple simultaneous threats, this paper introduces an approach for controlling m… ▽ More Reusable decoys offer a cost-effective alternative to the single-use hardware commonly applied to protect surface assets from threats. Such decoys portray fake assets to lure threats away from the true asset. To deceive a threat, a decoy first has to position itself such that it can break the radar lock. Considering multiple simultaneous threats, this paper introduces an approach for controlling multiple decoys to minimise the time required to break the locks of all the threats. The method includes the optimal allocation of one decoy to every threat with an assignment procedure that provides local position constraints to guarantee collision avoidance and thereby decouples the control of the decoys. A crude model of a decoy with uncertainty is considered for motion planning. The task of a decoy reaching a state in which the lock of the assigned threat can be broken is formulated as a temporal logic specification. To this end, the requirements to complete the task are modelled as time-varying set-membership constraints. The temporal and logical combination of the constraints is encoded in a mixed-integer optimisation problem. To demonstrate the results a simulated case study is provided. △ Less

Submitted 17 June, 2021; originally announced June 2021.

Ordinal Optimisation and the Offline Multiple Noisy Secretary Problem

Authors: Robert Chin, Jonathan E. Rowe, Iman Shames, Chris Manzie, Dragan Nešić

Abstract: We study the success probability for a variant of the secretary problem, with noisy observations and multiple offline selection. Our formulation emulates, and is motivated by, problems involving noisy selection arising in the disciplines of stochastic simulation and simulation-based optimisation. In addition, we employ the philosophy of ordinal optimisation - involving an ordinal selection rule, a… ▽ More We study the success probability for a variant of the secretary problem, with noisy observations and multiple offline selection. Our formulation emulates, and is motivated by, problems involving noisy selection arising in the disciplines of stochastic simulation and simulation-based optimisation. In addition, we employ the philosophy of ordinal optimisation - involving an ordinal selection rule, and a percentile notion of goal softening for the success probability. As a result, it is shown that the success probability only depends on the underlying copula of the problem. Other general properties for the success probability are also presented. Specialising to the case of Gaussian copulas, we also derive an analytic lower bound for the success probability, which may then be inverted to find sufficiently large sample sizes that guarantee a high success probability arbitrarily close to one. △ Less

Submitted 10 June, 2021; v1 submitted 2 June, 2021; originally announced June 2021.

Comments: 10 pages plus 9 pages of appendices

Fast Spline Trajectory Planning: Minimum Snap and Beyond

Authors: Declan Burke, Airlie Chapman, Iman Shames

Abstract: In this paper, we study spline trajectory generation via the solution of two optimisation problems: (i) a quadratic program (QP) with linear equality constraints and (ii) a nonlinear and nonconvex optimisation program. We propose an efficient algorithm to solve (i), which we then leverage to use in an iterative algorithm to solve (ii). Both the first algorithm and each iteration of the second algo… ▽ More In this paper, we study spline trajectory generation via the solution of two optimisation problems: (i) a quadratic program (QP) with linear equality constraints and (ii) a nonlinear and nonconvex optimisation program. We propose an efficient algorithm to solve (i), which we then leverage to use in an iterative algorithm to solve (ii). Both the first algorithm and each iteration of the second algorithm have linear computational complexity in the number of spline segments. The scaling of each algorithm is such that we are able to solve the two problems faster than state-of-the-art methods and in times amenable to real-time trajectory generation requirements. The trajectories we generate are applicable to differentially flat systems, a broad class of mechanical systems, which we demonstrate by planning trajectories for a quadrotor. △ Less

Submitted 4 May, 2021; originally announced May 2021.

Comments: 11 pages, 5 figures, to be submitted to IEEE Transactions on Robotics

Non-uniform Observability for Moving Horizon Estimation and stability with respect to additive perturbation

Authors: Emilien Flayac, Iman Shames

Abstract: This paper formalises the concepts of weakly and weakly regularly persistent input trajectory as well as their link to the Observability Grammian and the existence and uniqueness of solutions of Moving Horizon Estimation (MHE) problems. Additionally, thanks to a new time-uniform Implicit Function Theorem, these notions are proved to imply the stability of MHE solutions with respect to small additi… ▽ More This paper formalises the concepts of weakly and weakly regularly persistent input trajectory as well as their link to the Observability Grammian and the existence and uniqueness of solutions of Moving Horizon Estimation (MHE) problems. Additionally, thanks to a new time-uniform Implicit Function Theorem, these notions are proved to imply the stability of MHE solutions with respect to small additive perturbation in the measurements and in the dynamics, both uniformly and non-uniformly in time. Finally, examples and counter-examples of weakly persistent and weakly regularly persistent input trajectories are given in the case of 2D bearing-only navigation. △ Less

Submitted 9 June, 2023; v1 submitted 25 April, 2021; originally announced April 2021.

Sampling Polynomial Trajectories for LTL Verification

Authors: Daniel Selvaratnam, Michael Cantoni, J. M. Davoren, Iman Shames

Abstract: This paper concerns the verification of continuous-time polynomial spline trajectories against linear temporal logic specifications (LTL without 'next'). Each atomic proposition is assumed to represent a state space region described by a multivariate polynomial inequality. The proposed approach samples a trajectory strategically, to capture every one of its region transitions. This yields a discre… ▽ More This paper concerns the verification of continuous-time polynomial spline trajectories against linear temporal logic specifications (LTL without 'next'). Each atomic proposition is assumed to represent a state space region described by a multivariate polynomial inequality. The proposed approach samples a trajectory strategically, to capture every one of its region transitions. This yields a discrete word called a trace, which is amenable to established formal methods for path checking. The original continuous-time trajectory is shown to satisfy the specification if and only if its trace does. General topological conditions on the sample points are derived that ensure a trace is recorded for arbitrary continuous paths, given arbitrary region descriptions. Using techniques from computer algebra, a trace generation algorithm is developed to satisfy these conditions when the path and region boundaries are defined by polynomials. The proposed PolyTrace algorithm has polynomial complexity in the number of atomic propositions, and is guaranteed to produce a trace of any polynomial path. Its performance is demonstrated via numerical examples and a case study from robotics. △ Less

Submitted 21 January, 2022; v1 submitted 18 April, 2021; originally announced April 2021.

Comments: Published in Theoretical Computer Science. Definition 7 and Remark 4 have been updated to correct an oversight in the published version

ACM Class: F.4.1; F.3.1

An expressiveness hierarchy of Behavior Trees and related architectures

Authors: Oliver Biggar, Mohammad Zamani, Iman Shames

Abstract: In this paper we provide a formal framework for comparing the expressive power of Behavior Trees (BTs) to other action selection architectures. Taking inspiration from the analogous comparisons of structural programming methodologies, we formalise the concept of `expressiveness'. This leads us to an expressiveness hierarchy of control architectures, which includes BTs, Decision Trees (DTs), Teleo-… ▽ More In this paper we provide a formal framework for comparing the expressive power of Behavior Trees (BTs) to other action selection architectures. Taking inspiration from the analogous comparisons of structural programming methodologies, we formalise the concept of `expressiveness'. This leads us to an expressiveness hierarchy of control architectures, which includes BTs, Decision Trees (DTs), Teleo-reactive Programs (TRs) and Finite State Machines (FSMs). By distinguishing between BTs with auxiliary variables and those without, we demonstrate the existence of a trade-off in BT design between readability and expressiveness. We discuss what this means for BTs in practice. △ Less

Submitted 16 April, 2021; originally announced April 2021.

Comments: 8 pages, 2 figures. Accepted to IEEE Robotics and Automation Letters

Sensitivity Analysis for Bottleneck Assignment Problems

Authors: Elad Michael, Tony A. Wood, Chris Manzie, Iman Shames

Abstract: In assignment problems, decision makers are often interested in not only the optimal assignment, but also the sensitivity of the optimal assignment to perturbations in the assignment weights. Typically, only perturbations to individual assignment weights are considered. We present a novel extension of the traditional sensitivity analysis by allowing for simultaneous variations in all assignment we… ▽ More In assignment problems, decision makers are often interested in not only the optimal assignment, but also the sensitivity of the optimal assignment to perturbations in the assignment weights. Typically, only perturbations to individual assignment weights are considered. We present a novel extension of the traditional sensitivity analysis by allowing for simultaneous variations in all assignment weights. Focusing on the bottleneck assignment problem, we provide two different methods of quantifying the sensitivity of the optimal assignment, and present algorithms for each. Numerical examples as well as a discussion of the complexity for all algorithms are provided. △ Less

Submitted 10 February, 2022; v1 submitted 1 April, 2021; originally announced April 2021.

Comments: 28 pages

Safe Learning of Uncertain Environments

Authors: Farhad Farokhi, Alex Leong, Iman Shames, Mohammad Zamani

Abstract: In many learning based control methodologies, learning the unknown dynamic model precedes the control phase, while the aim is to control the system such that it remains in some safe region of the state space. In this work, our aim is to guarantee safety while learning and control proceed simultaneously. Specifically, we consider the problem of safe learning in nonlinear control-affine systems subj… ▽ More In many learning based control methodologies, learning the unknown dynamic model precedes the control phase, while the aim is to control the system such that it remains in some safe region of the state space. In this work, our aim is to guarantee safety while learning and control proceed simultaneously. Specifically, we consider the problem of safe learning in nonlinear control-affine systems subject to unknown additive uncertainty. We first model the uncertainty as a Gaussian noise and use state measurements to learn its mean and covariance. We provide rigorous time-varying bounds on the mean and covariance of the uncertainty and employ them to modify the control input via an optimization program with potentially time-varying safety constraints. We show that with an arbitrarily large probability we can guarantee that the state will remain in the safe set, while learning and control are carried out simultaneously, provided that a feasible solution exists for the optimization problem. We provide a secondary formulation of this optimization that is computationally more efficient. This is based on tightening the safety constraints to counter the uncertainty about the learned mean and covariance. The magnitude of the tightening can be decreased as our confidence in the learned mean and covariance increases (i.e., as we gather more measurements about the environment). Extensions of the method are provided for non-Gaussian process noise with unknown mean and covariance as well as Gaussian uncertainties with state-dependent mean and covariance to accommodate more general environments. △ Less

Submitted 13 May, 2021; v1 submitted 1 March, 2021; originally announced March 2021.

A Sequential Learning Algorithm for Probabilistically Robust Controller Tuning

Authors: Robert Chin, Chris Manzie, Iman Shames, Dragan Nešić, Jonathan E. Rowe

Abstract: We introduce a sequential learning algorithm to address a robust controller tuning problem, which in effect, finds (with high probability) a candidate solution satisfying the internal performance constraint to a chance-constrained program which has black-box functions. The algorithm leverages ideas from the areas of randomised algorithms and ordinal optimisation, and also draws comparisons with th… ▽ More We introduce a sequential learning algorithm to address a robust controller tuning problem, which in effect, finds (with high probability) a candidate solution satisfying the internal performance constraint to a chance-constrained program which has black-box functions. The algorithm leverages ideas from the areas of randomised algorithms and ordinal optimisation, and also draws comparisons with the scenario approach; these have all been previously applied to finding approximate solutions for difficult design problems. By exploiting statistical correlations through black-box sampling, we formally prove that our algorithm yields a controller meeting the prescribed probabilistic performance specification. Additionally, we characterise the computational requirement of the algorithm with a probabilistic lower bound on the algorithm's stop** time. To validate our work, the algorithm is then demonstrated for tuning model predictive controllers on a diesel engine air-path across a fleet of vehicles. The algorithm successfully tuned a single controller to meet a desired tracking error performance, even in the presence of the plant uncertainty inherent across the fleet. Moreover, the algorithm was shown to exhibit a sample complexity comparable to the scenario approach. △ Less

Submitted 17 October, 2021; v1 submitted 18 February, 2021; originally announced February 2021.

Comments: 17 pages including appendices and references

Asynchronous Distributed Optimization via Dual Decomposition and Block Coordinate Subgradient Methods

Authors: Yankai Lin, Iman Shames, Dragan Nesic

Abstract: We study the problem of minimizing the sum of potentially non-differentiable convex cost functions with partially overlap** dependences in an asynchronous manner, where communication in the network is not coordinated. We study the behavior of an asynchronous algorithm based on dual decomposition and block coordinate subgradient methods under assumptions weaker than those used in the literature.… ▽ More We study the problem of minimizing the sum of potentially non-differentiable convex cost functions with partially overlap** dependences in an asynchronous manner, where communication in the network is not coordinated. We study the behavior of an asynchronous algorithm based on dual decomposition and block coordinate subgradient methods under assumptions weaker than those used in the literature. At the same time, we allow different agents to use local stepsizes with no global coordination. Sufficient conditions are provided for almost sure convergence to the solution of the optimization problem. Under additional assumptions, we establish a sublinear convergence rate that in turn can be strengthened to linear convergence rate if the problem is strongly convex and has Lipschitz gradients. We also extend available results in the literature by allowing multiple and potentially overlap** blocks to be updated at the same time with non-uniform and potentially time varying probabilities assigned to different blocks. A numerical example is provided to illustrate the effectiveness of the algorithm. △ Less

Submitted 15 February, 2021; originally announced February 2021.

MSC Class: 93D99 (primary); 90C25 (secondary); 49M29

Rigid-profile input scheduling under constrained dynamics with a water network application

Authors: Adair Lang, Michael Cantoni, Farhad Farokhi, Iman Shames

Abstract: The motivation for this work stems from the problem of scheduling requests for flow at supply points along an automated network of open-water channels. The off-take flows are rigid-profile inputs to the system dynamics. In particular, the channel operator can only shift orders in time to satisfy constraints on the automatic response to changes in the load. This leads to a non-convex semi-infinite… ▽ More The motivation for this work stems from the problem of scheduling requests for flow at supply points along an automated network of open-water channels. The off-take flows are rigid-profile inputs to the system dynamics. In particular, the channel operator can only shift orders in time to satisfy constraints on the automatic response to changes in the load. This leads to a non-convex semi-infinite programming problem, with sum-separable cost that encodes the collective sensitivity of end users to scheduling delays. The constraints encode the linear time-invariant continuous-time dynamics and limits on the state across a \al{continuous} scheduling horizon. Discretization is used to arrive at a more manageable approximation of the semi-infinite program. A method for parsimoniously refining the discretization is applied to ensure continuous-time feasibility for solutions of the approximate problem. It is then shown how to improve cost without loss of feasibility. Supporting analysis is provided, along with simulation results for a realistic irrigation channel setup to illustrate the approach. △ Less

Submitted 2 December, 2020; originally announced December 2020.

MSC Class: 93A10 (Primary); 90C34 (Secondary); 90B35

Journal ref: IEEE Transactions on Control Systems Technology. Early Access (2020) 1-16

Approximate Midpoint Policy Iteration for Linear Quadratic Control

Authors: Benjamin Gravell, Iman Shames, Tyler Summers

Abstract: We present a midpoint policy iteration algorithm to solve linear quadratic optimal control problems in both model-based and model-free settings. The algorithm is a variation of Newton's method, and we show that in the model-based setting it achieves cubic convergence, which is superior to standard policy iteration and policy gradient algorithms that achieve quadratic and linear convergence, respec… ▽ More We present a midpoint policy iteration algorithm to solve linear quadratic optimal control problems in both model-based and model-free settings. The algorithm is a variation of Newton's method, and we show that in the model-based setting it achieves cubic convergence, which is superior to standard policy iteration and policy gradient algorithms that achieve quadratic and linear convergence, respectively. We also demonstrate that the algorithm can be approximately implemented without knowledge of the dynamics model by using least-squares estimates of the state-action value function from trajectory data, from which policy improvements can be obtained. With sufficient trajectory data, the policy iterates converge cubically to approximately optimal policies, and this occurs with the same available sample budget as the approximate standard policy iteration. Numerical experiments demonstrate effectiveness of the proposed algorithms. △ Less

Submitted 15 February, 2022; v1 submitted 28 November, 2020; originally announced November 2020.

A Distributed Augmenting Path Approach for the Bottleneck Assignment Problem

Authors: Mitchell Khoo, Tony A. Wood, Chris Manzie, Iman Shames

Abstract: We develop an algorithm to solve the Bottleneck Assignment Problem (BAP) that is amenable to having computation distributed over a network of agents. This consists of exploring how each component of the algorithm can be distributed, with a focus on one component in particular, i.e., the function to search for an augmenting path. An augmenting path is a common tool used in most BAP algorithms and p… ▽ More We develop an algorithm to solve the Bottleneck Assignment Problem (BAP) that is amenable to having computation distributed over a network of agents. This consists of exploring how each component of the algorithm can be distributed, with a focus on one component in particular, i.e., the function to search for an augmenting path. An augmenting path is a common tool used in most BAP algorithms and poses a particular challenge for this distributed approach. Given this significance, we compare two different methods to search for an augmenting path in a bipartite graph. We also exploit properties of the augmenting paths to formalise conditions for which the solution from subsets of the sets of agents and tasks can be used to solve the BAP with the full sets of agents and tasks. In the end, we evaluate and compare the derived approaches with a numerical analysis. △ Less

Submitted 18 November, 2020; originally announced November 2020.

Tracking and regret bounds for online zeroth-order Euclidean and Riemannian optimisation

Authors: Alejandro I. Maass, Chris Manzie, Dragan Nesic, Jonathan H. Manton, Iman Shames

Abstract: We study numerical optimisation algorithms that use zeroth-order information to minimise time-varying geodesically-convex cost functions on Riemannian manifolds. In the Euclidean setting, zeroth-order algorithms have received a lot of attention in both the time-varying and time-invariant cases. However, the extension to Riemannian manifolds is much less developed. We focus on Hadamard manifolds, w… ▽ More We study numerical optimisation algorithms that use zeroth-order information to minimise time-varying geodesically-convex cost functions on Riemannian manifolds. In the Euclidean setting, zeroth-order algorithms have received a lot of attention in both the time-varying and time-invariant cases. However, the extension to Riemannian manifolds is much less developed. We focus on Hadamard manifolds, which are a special class of Riemannian manifolds with global nonpositive curvature that offer convenient grounds for the generalisation of convexity notions. Specifically, we derive bounds on the expected instantaneous tracking error, and we provide algorithm parameter values that minimise the algorithm's performance. Our results illustrate how the manifold geometry in terms of the sectional curvature affects these bounds. Additionally, we provide dynamic regret bounds for this online optimisation setting. To the best of our knowledge, these are the first regret bounds even for the Euclidean version of the problem. Lastly, via numerical simulations, we demonstrate the applicability of our algorithm on an online Karcher mean problem. △ Less

Submitted 13 February, 2022; v1 submitted 1 October, 2020; originally announced October 2020.

Comments: 27 pages, 2 figures

MSC Class: 68T05; 68Q32 (Primary); 90C25; 90C56 (Secondary)

Cyber Attack and Machine Induced Fault Detection and Isolation Methodologies for Cyber-Physical Systems

Authors: Mahdi Taheri, Khashayar Khorasani, Iman Shames, Nader Meskin

Abstract: In this paper, the problem of simultaneous cyber attack and fault detection and isolation (CAFDI) in cyber-physical systems (CPS) is studied. The proposed solution methodology consists of two filters on the plant and the command and control (C\&C) sides of the CPS and an unknown input observer (UIO) based detector on the plant side. Conditions under which the proposed methodology can detect decept… ▽ More In this paper, the problem of simultaneous cyber attack and fault detection and isolation (CAFDI) in cyber-physical systems (CPS) is studied. The proposed solution methodology consists of two filters on the plant and the command and control (C\&C) sides of the CPS and an unknown input observer (UIO) based detector on the plant side. Conditions under which the proposed methodology can detect deception attacks, such as covert attacks, zero dynamics attacks, and replay attacks are characterized. An advantage of the proposed methodology is that one does not require a fully secured communication link which implies that the communication link can be compromised by the adversary while it is used to transmit the C\&C side observer estimates. Also, it is assumed that adversaries have access to parameters of the system, filters, and the UIO-based detector, however, they do not have access to all the communication link channels. Conditions under which, using the communication link cyber attacks, the adversary cannot eliminate the impact of actuator and sensor cyber attacks are investigated. To illustrate the capabilities and effectiveness of the proposed CAFDI methodologies, simulation case studies are provided and comparisons with detection methods that are available in the literature are included to demonstrate the advantages and benefits of our proposed solutions. △ Less

Submitted 14 September, 2020; originally announced September 2020.

Mitigation and Resiliency of Multi-Agent Systems Subject to Malicious Cyber Attacks on Communication Links

Authors: Mahdi Taheri, Khashayar Khorasani, Iman Shames, Nader Meskin

Abstract: This paper aims at investigating a novel type of cyber attack that is injected to multi-agent systems (MAS) having an underlying directed graph. The cyber attack, which is designated as the controllability attack, is injected by the malicious adversary into the communication links among the agents. The adversary, leveraging the compromised communication links disguises the cyber attack signals and… ▽ More This paper aims at investigating a novel type of cyber attack that is injected to multi-agent systems (MAS) having an underlying directed graph. The cyber attack, which is designated as the controllability attack, is injected by the malicious adversary into the communication links among the agents. The adversary, leveraging the compromised communication links disguises the cyber attack signals and attempts to take control over the entire network of MAS. The adversary aims at achieving this by directly attacking only a subset of the multi-agents. Conditions under which the malicious hacker has control over the entire MAS network are provided. Two notions of security controllability indices are proposed and developed. These notions are utilized as metrics to evaluate the controllability that each agent provides to the adversary for executing the malicious cyber attack. Furthermore, the possibility of introducing zero dynamics cyber attacks on the MAS through compromising the communication links is also investigated. Finally, an illustrative numerical example is provided to demonstrate the effectiveness of our proposed methods. △ Less

Submitted 14 September, 2020; originally announced September 2020.

Undetectable Cyber Attacks on Communication Links in Multi-Agent Cyber-Physical Systems

Authors: Mahdi Taheri, Khashayar Khorasani, Iman Shames, Nader Meskin

Abstract: The objective in this paper is to study and develop conditions for a network of multi-agent cyber-physical systems (MAS) where a malicious adversary can utilize vulnerabilities in order to ensure and maintain cyber attacks undetectable. We classify these cyber attacks as undetectable in the sense that their impact cannot be observed in the generated residuals. It is shown if an agent that is the r… ▽ More The objective in this paper is to study and develop conditions for a network of multi-agent cyber-physical systems (MAS) where a malicious adversary can utilize vulnerabilities in order to ensure and maintain cyber attacks undetectable. We classify these cyber attacks as undetectable in the sense that their impact cannot be observed in the generated residuals. It is shown if an agent that is the root of a rooted spanning tree in the MAS graph is under a cyber attack, the attack is undetectable by the entire network. Next we investigate if a non-root agent is compromised, then under certain conditions cyber attacks can become detectable. Moreover, a novel cyber attack that is designated as quasi-covert cyber attack is introduced that can be used to eliminate detectable impacts of cyber attacks to the entire network and maintain these attacks as undetected. Finally, an event-triggered based detector is proposed that can be used to detect the quasi-covert cyber attacks. Numerical simulations are provided to illustrate the effectiveness and capabilities of our proposed methodologies. △ Less

Submitted 13 September, 2020; originally announced September 2020.

On modularity in reactive control architectures, with an application to formal verification

Authors: Oliver Biggar, Mohammad Zamani, Iman Shames

Abstract: Modularity is a central principle throughout the design process for cyber-physical systems. Modularity reduces complexity and increases reuse of behavior. In this paper we pose and answer the following question: how can we identify independent `modules' within the structure of reactive control architectures? To this end, we propose a graph-structured control architecture we call a decision structu… ▽ More Modularity is a central principle throughout the design process for cyber-physical systems. Modularity reduces complexity and increases reuse of behavior. In this paper we pose and answer the following question: how can we identify independent `modules' within the structure of reactive control architectures? To this end, we propose a graph-structured control architecture we call a decision structure, and show how it generalises some reactive control architectures which are popular in Artificial Intelligence (AI) and robotics, specifically Teleo-Reactive programs (TRs), Decision Trees (DTs), Behavior Trees (BTs) and Generalised Behavior Trees ($k$-BTs). Inspired by the definition of a module in graph theory, we define modules in decision structures and show how each decision structure possesses a canonical decomposition into its modules. We can naturally characterise each of the BTs, $k$-BTs, DTs and TRs by properties of their module decomposition. This allows us to recognise which decision structures are equivalent to each of these architectures in quadratic time. Our proposed concept of modules extends to formal verification, under any verification scheme capable of verifying a decision structure. Namely, we prove that a modification to a module within a decision structure has no greater flow-on effects than a modification to an individual action within that structure. This enables verification on modules to be done locally and hierarchically, where structures can be verified and then repeatedly locally modified, with modules replaced by modules while preserving correctness. To illustrate the findings, we present an example of a solar-powered drone controlled by a decision structure. We use a Linear Temporal Logic-based verification scheme to verify the correctness of this structure, and then show how one can modify modules while preserving its correctness. △ Less

Submitted 30 January, 2022; v1 submitted 28 August, 2020; originally announced August 2020.

Comments: Accepted to ACM Transactions on Cyber-Physical Systems. 26 pages, 9 figures. Version 3 changes: accepted version, minor changes

A Greedy and Distributable Approach to the Lexicographic Bottleneck Assignment Problem with Conditions on Exactness

Authors: Mitchell Khoo, Tony A. Wood, Chris Manzie, Iman Shames

Abstract: Solving the Lexicographic Bottleneck Assignment Problem (LexBAP) typically relies on centralised computation with order quartic complexity. We consider the Sequential Bottleneck Assignment Problem (SeqBAP), which yields a greedy solution to the LexBAP and discuss the relationship between the SeqBAP, the LexBAP, and the Bottleneck Assignment Problem (BAP). In particular, we reexamine tools used to… ▽ More Solving the Lexicographic Bottleneck Assignment Problem (LexBAP) typically relies on centralised computation with order quartic complexity. We consider the Sequential Bottleneck Assignment Problem (SeqBAP), which yields a greedy solution to the LexBAP and discuss the relationship between the SeqBAP, the LexBAP, and the Bottleneck Assignment Problem (BAP). In particular, we reexamine tools used to analyse the structure of the BAP, and apply them to derive an algorithm that solves the SeqBAP with cubic complexity. We show that the set of solutions of the LexBAP is a subset of the solutions of the SeqBAP and analyse the conditions for which the solutions sets are identical. Furthermore, we provide a method to verify the satisfaction of these conditions. In cases where the conditions are satisfied, the proposed algorithm for solving the SeqBAP solves the LexBAP with computation that has lower complexity and can be distributed over a network of computing agents. The applicability of the approach is demonstrated with a case study where mobile robots are assigned to goal locations. △ Less

Submitted 7 January, 2022; v1 submitted 28 August, 2020; originally announced August 2020.