Showing 1–50 of 58 results for author: Shames, I

Online Non-Stationary Stochastic Quasar-Convex Optimization

Authors: Yuen-Man Pun, Iman Shames

Abstract: Recent research has shown that quasar-convexity can be found in applications such as identification of linear dynamical systems and generalized linear models. Such observations have in turn spurred exciting developments in design and analysis algorithms that exploit quasar-convexity. In this work, we study the online stochastic quasar-convex optimization problems in a dynamic environment. We estab… ▽ More Recent research has shown that quasar-convexity can be found in applications such as identification of linear dynamical systems and generalized linear models. Such observations have in turn spurred exciting developments in design and analysis algorithms that exploit quasar-convexity. In this work, we study the online stochastic quasar-convex optimization problems in a dynamic environment. We establish regret bounds of online gradient descent in terms of cumulative path variation and cumulative gradient variance for losses satisfying quasar-convexity and strong quasar-convexity. We then apply the results to generalized linear models (GLM) when the underlying parameter is time-varying. We establish regret bounds of online gradient descent when applying to GLMs with leaky ReLU activation function, logistic activation function, and ReLU activation function. Numerical results are presented to corroborate our findings. △ Less

Submitted 3 July, 2024; originally announced July 2024.

Minimisation of Polyak-Łojasewicz Functions Using Random Zeroth-Order Oracles

Authors: Amir Ali Farzin, Iman Shames

Abstract: The application of a zeroth-order scheme for minimising Polyak-Łojasewicz (PL) functions is considered. The framework is based on exploiting a random oracle to estimate the function gradient. The convergence of the algorithm to a global minimum in the unconstrained case and to a neighbourhood of the global minimum in the constrained case along with their corresponding complexity bounds are present… ▽ More The application of a zeroth-order scheme for minimising Polyak-Łojasewicz (PL) functions is considered. The framework is based on exploiting a random oracle to estimate the function gradient. The convergence of the algorithm to a global minimum in the unconstrained case and to a neighbourhood of the global minimum in the constrained case along with their corresponding complexity bounds are presented. The theoretical results are demonstrated via numerical examples. △ Less

Submitted 15 May, 2024; originally announced May 2024.

On Distributed Nonconvex Optimisation Via Modified ADMM

Authors: Behnam Mafakheri, Jonathan H. Manton, Iman Shames

Abstract: This paper addresses the problem of nonconvex nonsmooth decentralised optimisation in multi-agent networks with undirected connected communication graphs. Our contribution lies in introducing an algorithmic framework designed for the distributed minimisation of the sum of a smooth (possibly nonconvex and non-separable) function and a convex (possibly nonsmooth and non-separable) regulariser. The p… ▽ More This paper addresses the problem of nonconvex nonsmooth decentralised optimisation in multi-agent networks with undirected connected communication graphs. Our contribution lies in introducing an algorithmic framework designed for the distributed minimisation of the sum of a smooth (possibly nonconvex and non-separable) function and a convex (possibly nonsmooth and non-separable) regulariser. The proposed algorithm can be seen as a modified version of the ADMM algorithm where, at each step, an "inner loop" needs to be iterated for a number of iterations. The role of the inner loop is to aggregate and disseminate information across the network. We observe that a naive decentralised approach (one iteration of the inner loop) may not converge. We establish the asymptotic convergence of the proposed algorithm to the set of stationary points of the nonconvex problem where the number of iterations of the inner loop increases logarithmically with the step count of the ADMM algorithm. We present numerical results demonstrating the proposed method's correctness and performance. △ Less

Submitted 6 December, 2023; originally announced December 2023.

Comments: 6 pages, 1 Figure

Distributionally Time-Varying Online Stochastic Optimization under Polyak-Łojasiewicz Condition with Application in Conditional Value-at-Risk Statistical Learning

Authors: Yuen-Man Pun, Farhad Farokhi, Iman Shames

Abstract: In this work, we consider a sequence of stochastic optimization problems following a time-varying distribution via the lens of online optimization. Assuming that the loss function satisfies the Polyak-Łojasiewicz condition, we apply online stochastic gradient descent and establish its dynamic regret bound that is composed of cumulative distribution drifts and cumulative gradient biases caused by s… ▽ More In this work, we consider a sequence of stochastic optimization problems following a time-varying distribution via the lens of online optimization. Assuming that the loss function satisfies the Polyak-Łojasiewicz condition, we apply online stochastic gradient descent and establish its dynamic regret bound that is composed of cumulative distribution drifts and cumulative gradient biases caused by stochasticity. The distribution metric we adopt here is Wasserstein distance, which is well-defined without the absolute continuity assumption or with a time-varying support set. We also establish a regret bound of online stochastic proximal gradient descent when the objective function is regularized. Moreover, we show that the above framework can be applied to the Conditional Value-at-Risk (CVaR) learning problem. Particularly, we improve an existing proof on the discovery of the PL condition of the CVaR problem, resulting in a regret bound of online stochastic gradient descent. △ Less

Submitted 17 September, 2023; originally announced September 2023.

Online Interior-point Methods for Time-varying Equality-constrained Optimization

Authors: Jean-Luc Lupien, Iman Shames, Antoine Lesage-Landry

Abstract: An important challenge in the online convex optimization (OCO) setting is to incorporate generalized inequalities and time-varying constraints. The inclusion of constraints in OCO widens the applicability of such algorithms to dynamic and safety-critical settings such as the online optimal power flow (OPF) problem. In this work, we propose the first projection-free OCO algorithm admitting time-var… ▽ More An important challenge in the online convex optimization (OCO) setting is to incorporate generalized inequalities and time-varying constraints. The inclusion of constraints in OCO widens the applicability of such algorithms to dynamic and safety-critical settings such as the online optimal power flow (OPF) problem. In this work, we propose the first projection-free OCO algorithm admitting time-varying linear constraints and convex generalized inequalities: the online interior-point method for time-varying equality constraints (OIPM-TEC). We derive simultaneous sublinear dynamic regret and constraint violation bounds for OIPM-TEC under standard assumptions. For applications where a given tolerance around optima is accepted, we employ an alternative OCO performance metric -- the epsilon-regret -- and a more computationally efficient algorithm, the epsilon-OIPM-TEC, that possesses sublinear bounds under this metric. Finally, we showcase the performance of these two algorithms on an online OPF problem and compare them to another OCO algorithm from the literature. △ Less

Submitted 30 May, 2024; v1 submitted 30 July, 2023; originally announced July 2023.

A Robust Control Approach to Asymptotic Optimality of the Heavy Ball Method for Optimization of Quadratic Functions

Authors: V. Ugrinovskii, I. R. Petersen, I. Shames

Abstract: Among first order optimization methods, Polyak's heavy ball method has long been known to guarantee the asymptotic rate of convergence matching Nesterov's lower bound for functions defined in an infinite-dimensional space. In this paper, we use results on the robust gain margin of linear uncertain feedback control systems to show that the heavy ball method is provably worst-case asymptotically opt… ▽ More Among first order optimization methods, Polyak's heavy ball method has long been known to guarantee the asymptotic rate of convergence matching Nesterov's lower bound for functions defined in an infinite-dimensional space. In this paper, we use results on the robust gain margin of linear uncertain feedback control systems to show that the heavy ball method is provably worst-case asymptotically optimal when applied to quadratic functions in a finite dimensional space. △ Less

Submitted 11 May, 2023; originally announced May 2023.

Comments: Accepted for publication in Automatica

Distributed Block Coordinate Moving Horizon Estimation for 2D Visual-Inertial-Odometry SLAM

Authors: Emilien Flayac, Iman Shames

Abstract: This paper presents a Visual Inertial Odometry Landmark-based Simultaneous Localisation and Map** algorithm based on a distributed block coordinate nonlinear Moving Horizon Estimation scheme. The main advantage of the proposed method is that the updates on the position of the landmarks are based on a Bundle Adjustment technique that can be parallelised over the landmarks. The performance of the… ▽ More This paper presents a Visual Inertial Odometry Landmark-based Simultaneous Localisation and Map** algorithm based on a distributed block coordinate nonlinear Moving Horizon Estimation scheme. The main advantage of the proposed method is that the updates on the position of the landmarks are based on a Bundle Adjustment technique that can be parallelised over the landmarks. The performance of the method is demonstrated in simulations in different environments and with different types of robot trajectory. Circular and wiggling patterns in the trajectory lead to better estimation performance than straight ones, confirming what is expected from recent nonlinear observability theory. △ Less

Submitted 4 April, 2023; originally announced April 2023.

Nonlinear Dual control based on Fast Moving Horizon estimation and Model Predictive Control with an observability constraint

Authors: Emilien Flayac, Girish Nair, Iman Shames

Abstract: This paper proposes an algorithm that combines Fast Moving Horizon Parameter Estimation and Model Predictive Control subject to an observability constraint designed to ensure a lower bound on the performance of the parameter estimator. Output-feedback stability is proved through input-to-state stability of the state/error system under a small noise and initial error assumption. Numerical experimen… ▽ More This paper proposes an algorithm that combines Fast Moving Horizon Parameter Estimation and Model Predictive Control subject to an observability constraint designed to ensure a lower bound on the performance of the parameter estimator. Output-feedback stability is proved through input-to-state stability of the state/error system under a small noise and initial error assumption. Numerical experiments have been carried out in the case of Active Simultaneous Localisation and Map** (SLAM). △ Less

Submitted 24 March, 2023; originally announced March 2023.

Regret Analysis of Online LQR Control via Trajectory Prediction and Tracking: Extended Version

Authors: Yitian Chen, Timothy L. Molloy, Tyler Summers, Iman Shames

Abstract: In this paper, we propose and analyze a new method for online linear quadratic regulator (LQR) control with a priori unknown time-varying cost matrices. The cost matrices are revealed sequentially with the potential for future values to be previewed over a short window. Our novel method involves using the available cost matrices to predict the optimal trajectory, and a tracking controller to drive… ▽ More In this paper, we propose and analyze a new method for online linear quadratic regulator (LQR) control with a priori unknown time-varying cost matrices. The cost matrices are revealed sequentially with the potential for future values to be previewed over a short window. Our novel method involves using the available cost matrices to predict the optimal trajectory, and a tracking controller to drive the system towards it. We adopted the notion of dynamic regret to measure the performance of this proposed online LQR control method, with our main result being that the (dynamic) regret of our method is upper bounded by a constant. Moreover, the regret upper bound decays exponentially with the preview window length, and is extendable to systems with disturbances. We show in simulations that our proposed method offers improved performance compared to other previously proposed online LQR methods. △ Less

Submitted 20 February, 2023; originally announced February 2023.

Comments: Submitted to L4DC2023

MSC Class: 49N10; 49M05

Self-Tuning Network Control Architectures

Authors: Tyler Summers, Karthik Ganapathy, Iman Shames, Mathias Hudoba de Badyn

Abstract: We formulate a general mathematical framework for self-tuning network control architecture design. This problem involves jointly adapting the locations of active sensors and actuators in the network and the feedback control policy to all available information about the time-varying network state and dynamics to optimize a performance criterion. We propose a general solution structure analogous to… ▽ More We formulate a general mathematical framework for self-tuning network control architecture design. This problem involves jointly adapting the locations of active sensors and actuators in the network and the feedback control policy to all available information about the time-varying network state and dynamics to optimize a performance criterion. We propose a general solution structure analogous to the classical self-tuning regulator from adaptive control. We show that a special case with full-state feedback can be solved in principle with dynamic programming, and in the linear quadratic setting the optimal cost functions and policies are piecewise quadratic and piecewise linear, respectively. For large networks where exhaustive architecture search is prohibitive, we describe a greedy heuristic for joint architecture-policy design. We demonstrate in numerical experiments that self-tuning architectures can provide dramatically improved performance over fixed architectures. Our general formulation provides an extremely rich and challenging problem space with opportunities to apply a wide variety of approximation methods from stochastic control, system identification, reinforcement learning, and static architecture design. △ Less

Submitted 16 January, 2023; originally announced January 2023.

Comments: 6 pages, 5 figures

Journal ref: 61st Conference on Decision and Control, pp 5876-5881, 2022

First order online optimisation using forward gradients in over-parameterised systems

Authors: Behnam Mafakheri, Iman Shames, Jonathan H. Manton

Abstract: The success of deep learning over the past decade mainly relies on gradient-based optimisation and backpropagation. This paper focuses on analysing the performance of first-order gradient-based optimisation algorithms, gradient descent and proximal gradient, with time-varying non-convex cost function under (proximal) Polyak-Łojasiewicz condition. Specifically, we focus on using the forward mode of… ▽ More The success of deep learning over the past decade mainly relies on gradient-based optimisation and backpropagation. This paper focuses on analysing the performance of first-order gradient-based optimisation algorithms, gradient descent and proximal gradient, with time-varying non-convex cost function under (proximal) Polyak-Łojasiewicz condition. Specifically, we focus on using the forward mode of automatic differentiation to compute gradients in the fast-changing problems where calculating gradients using the backpropagation algorithm is either impossible or inefficient. Upper bounds for tracking and asymptotic errors are derived for various cases, showing the linear convergence to a solution or a neighbourhood of an optimal solution, where the convergence rate decreases with the increase in the dimension of the problem. We show that for a solver with constraints on computing resources, the number of forward gradient iterations at each step can be a design parameter that trades off between the tracking performance and computing constraints. △ Less

Submitted 6 December, 2022; v1 submitted 28 November, 2022; originally announced November 2022.

Comments: 18 pages, 1 figure

Robust Data-Driven Output Feedback Control via Bootstrapped Multiplicative Noise

Authors: Benjamin Gravell, Iman Shames, Tyler Summers

Abstract: We propose a robust data-driven output feedback control algorithm that explicitly incorporates inherent finite-sample model estimate uncertainties into the control design. The algorithm has three components: (1) a subspace identification nominal model estimator; (2) a bootstrap resampling method that quantifies non-asymptotic variance of the nominal model estimate; and (3) a non-conventional robus… ▽ More We propose a robust data-driven output feedback control algorithm that explicitly incorporates inherent finite-sample model estimate uncertainties into the control design. The algorithm has three components: (1) a subspace identification nominal model estimator; (2) a bootstrap resampling method that quantifies non-asymptotic variance of the nominal model estimate; and (3) a non-conventional robust control design method comprising a coupled optimal dynamic output feedback filter and controller with multiplicative noise. A key advantage of the proposed approach is that the system identification and robust control design procedures both use stochastic uncertainty representations, so that the actual inherent statistical estimation uncertainty directly aligns with the uncertainty the robust controller is being designed against. Moreover, the control design method accommodates a highly structured uncertainty representation that can capture uncertainty shape more effectively than existing approaches. We show through numerical experiments that the proposed robust data-driven output feedback controller can significantly outperform a certainty equivalent controller on various measures of sample complexity and stability robustness. △ Less

Submitted 10 May, 2022; originally announced May 2022.

Global convergence and asymptotic optimality of the heavy ball method for a class of non-convex optimization problems

Authors: Valery Ugrinovskii, Ian R. Petersen, Iman Shames

Abstract: In this letter we revisit the famous heavy ball method and study its global convergence for a class of non-convex problems with sector-bounded gradient. We characterize the parameters that render the method globally convergent and yield the best $R$-convergence factor. We show that for this family of functions, this convergence factor is superior to the factor obtained from the triple momentum met… ▽ More In this letter we revisit the famous heavy ball method and study its global convergence for a class of non-convex problems with sector-bounded gradient. We characterize the parameters that render the method globally convergent and yield the best $R$-convergence factor. We show that for this family of functions, this convergence factor is superior to the factor obtained from the triple momentum method. △ Less

Submitted 24 March, 2022; v1 submitted 6 February, 2022; originally announced February 2022.

Comments: 6 pages, 4 figures, to appear in CSS Letters

MSC Class: 90C26 (Primary); 93D09 (Secondary); 65K05

Online Convex Optimization Using Coordinate Descent Algorithms

Authors: Yankai Lin, Iman Shames, Dragan Nešić

Abstract: This paper considers the problem of online optimization where the objective function is time-varying. In particular, we extend coordinate descent type algorithms to the online case, where the objective function varies after a finite number of iterations of the algorithm. Instead of solving the problem exactly at each time step, we only apply a finite number of iterations at each time step. Commonl… ▽ More This paper considers the problem of online optimization where the objective function is time-varying. In particular, we extend coordinate descent type algorithms to the online case, where the objective function varies after a finite number of iterations of the algorithm. Instead of solving the problem exactly at each time step, we only apply a finite number of iterations at each time step. Commonly used notions of regret are used to measure the performance of the online algorithm. Moreover, coordinate descent algorithms with different updating rules are considered, including both deterministic and stochastic rules that are developed in the literature of classical offline optimization. A thorough regret analysis is given for each case. Finally, numerical simulations are provided to illustrate the theoretical results. △ Less

Submitted 25 April, 2024; v1 submitted 24 January, 2022; originally announced January 2022.

Comments: Accepted for publication in Automatica

MSC Class: 68Q32 (Primary); 68T05; 90C25 (Secondary)

Journal ref: Automatica, vol. 165, Article 111681, 2024

Gradient Free Cooperative Seeking of a Moving Source

Authors: Elad Michael, Chris Manzie, Tony A. Wood, Daniel Zelazo, Iman Shames

Abstract: In this paper, we consider the optimisation of a time varying scalar field by a network of agents with no gradient information. We propose a composite control law, blending extremum seeking with formation control in order to converge to the extrema faster by minimising the gradient estimation error. By formalising the relationship between the formation and the gradient estimation error, we provide… ▽ More In this paper, we consider the optimisation of a time varying scalar field by a network of agents with no gradient information. We propose a composite control law, blending extremum seeking with formation control in order to converge to the extrema faster by minimising the gradient estimation error. By formalising the relationship between the formation and the gradient estimation error, we provide a novel analysis to prove the convergence of the network to a bounded neighbourhood of the field's time varying extrema. We assume the time-varying field satisfies the Polyak Lojasiewicz inequality and the gradient is Lipschitz continuous at each iteration. Numerical studies and comparisons are provided to support the theoretical results. △ Less

Submitted 30 June, 2022; v1 submitted 2 January, 2022; originally announced January 2022.

Comments: 13 pages, 6 figures, submitted to Automatica

Learning Safety Filters for Unknown Discrete-Time Linear Systems

Authors: Farhad Farokhi, Alex S. Leong, Mohammad Zamani, Iman Shames

Abstract: A learning-based safety filter is developed for discrete-time linear time-invariant systems with unknown models subject to Gaussian noises with unknown covariance. Safety is characterized using polytopic constraints on the states and control inputs. The empirically learned model and process noise covariance with their confidence bounds are used to construct a robust optimization problem for minima… ▽ More A learning-based safety filter is developed for discrete-time linear time-invariant systems with unknown models subject to Gaussian noises with unknown covariance. Safety is characterized using polytopic constraints on the states and control inputs. The empirically learned model and process noise covariance with their confidence bounds are used to construct a robust optimization problem for minimally modifying nominal control actions to ensure safety with high probability. The optimization problem relies on tightening the original safety constraints. The magnitude of the tightening is larger at the beginning since there is little information to construct reliable models, but shrinks with time as more data becomes available. △ Less

Submitted 8 May, 2023; v1 submitted 31 October, 2021; originally announced November 2021.

Zeroth-order optimisation on subsets of symmetric matrices with application to MPC tuning

Authors: Alejandro I. Maass, Chris Manzie, Iman Shames, Hayato Nakada

Abstract: This paper provides a zeroth-order optimisation framework for non-smooth and possibly non-convex cost functions with matrix parameters that are real and symmetric. We provide complexity bounds on the number of iterations required to ensure a given accuracy level for both the convex and non-convex case. The derived complexity bounds for the convex case are less conservative than available bounds in… ▽ More This paper provides a zeroth-order optimisation framework for non-smooth and possibly non-convex cost functions with matrix parameters that are real and symmetric. We provide complexity bounds on the number of iterations required to ensure a given accuracy level for both the convex and non-convex case. The derived complexity bounds for the convex case are less conservative than available bounds in the literature since we exploit the symmetric structure of the underlying matrix space. Moreover, the non-convex complexity bounds are novel for the class of optimisation problems we consider. The utility of the framework is evident in the suite of applications that use symmetric matrices as tuning parameters. Of primary interest here is the challenge of tuning the gain matrices in model predictive controllers, as this is a challenge known to be inhibiting industrial implementation of these architectures. To demonstrate the framework we consider the problem of MIMO diesel air-path control, and consider implementing the framework iteratively ``in-the-loop'' to reduce tracking error on the output channels. Both simulations and experimental results are included to illustrate the effectiveness of the proposed framework over different engine drive cycles. △ Less

Submitted 27 June, 2021; originally announced June 2021.

Comments: 27 pages, 11 figures

Temporal Logic Planning for Minimum-Time Positioning of Multiple Threat-Seduction Decoys

Authors: Tony A. Wood, Mitchell Khoo, Elad Michael, Chris Manzie, Iman Shames

Abstract: Reusable decoys offer a cost-effective alternative to the single-use hardware commonly applied to protect surface assets from threats. Such decoys portray fake assets to lure threats away from the true asset. To deceive a threat, a decoy first has to position itself such that it can break the radar lock. Considering multiple simultaneous threats, this paper introduces an approach for controlling m… ▽ More Reusable decoys offer a cost-effective alternative to the single-use hardware commonly applied to protect surface assets from threats. Such decoys portray fake assets to lure threats away from the true asset. To deceive a threat, a decoy first has to position itself such that it can break the radar lock. Considering multiple simultaneous threats, this paper introduces an approach for controlling multiple decoys to minimise the time required to break the locks of all the threats. The method includes the optimal allocation of one decoy to every threat with an assignment procedure that provides local position constraints to guarantee collision avoidance and thereby decouples the control of the decoys. A crude model of a decoy with uncertainty is considered for motion planning. The task of a decoy reaching a state in which the lock of the assigned threat can be broken is formulated as a temporal logic specification. To this end, the requirements to complete the task are modelled as time-varying set-membership constraints. The temporal and logical combination of the constraints is encoded in a mixed-integer optimisation problem. To demonstrate the results a simulated case study is provided. △ Less

Submitted 17 June, 2021; originally announced June 2021.

Ordinal Optimisation and the Offline Multiple Noisy Secretary Problem

Authors: Robert Chin, Jonathan E. Rowe, Iman Shames, Chris Manzie, Dragan Nešić

Abstract: We study the success probability for a variant of the secretary problem, with noisy observations and multiple offline selection. Our formulation emulates, and is motivated by, problems involving noisy selection arising in the disciplines of stochastic simulation and simulation-based optimisation. In addition, we employ the philosophy of ordinal optimisation - involving an ordinal selection rule, a… ▽ More We study the success probability for a variant of the secretary problem, with noisy observations and multiple offline selection. Our formulation emulates, and is motivated by, problems involving noisy selection arising in the disciplines of stochastic simulation and simulation-based optimisation. In addition, we employ the philosophy of ordinal optimisation - involving an ordinal selection rule, and a percentile notion of goal softening for the success probability. As a result, it is shown that the success probability only depends on the underlying copula of the problem. Other general properties for the success probability are also presented. Specialising to the case of Gaussian copulas, we also derive an analytic lower bound for the success probability, which may then be inverted to find sufficiently large sample sizes that guarantee a high success probability arbitrarily close to one. △ Less

Submitted 10 June, 2021; v1 submitted 2 June, 2021; originally announced June 2021.

Comments: 10 pages plus 9 pages of appendices

Non-uniform Observability for Moving Horizon Estimation and stability with respect to additive perturbation

Authors: Emilien Flayac, Iman Shames

Abstract: This paper formalises the concepts of weakly and weakly regularly persistent input trajectory as well as their link to the Observability Grammian and the existence and uniqueness of solutions of Moving Horizon Estimation (MHE) problems. Additionally, thanks to a new time-uniform Implicit Function Theorem, these notions are proved to imply the stability of MHE solutions with respect to small additi… ▽ More This paper formalises the concepts of weakly and weakly regularly persistent input trajectory as well as their link to the Observability Grammian and the existence and uniqueness of solutions of Moving Horizon Estimation (MHE) problems. Additionally, thanks to a new time-uniform Implicit Function Theorem, these notions are proved to imply the stability of MHE solutions with respect to small additive perturbation in the measurements and in the dynamics, both uniformly and non-uniformly in time. Finally, examples and counter-examples of weakly persistent and weakly regularly persistent input trajectories are given in the case of 2D bearing-only navigation. △ Less

Submitted 9 June, 2023; v1 submitted 25 April, 2021; originally announced April 2021.

Sensitivity Analysis for Bottleneck Assignment Problems

Authors: Elad Michael, Tony A. Wood, Chris Manzie, Iman Shames

Abstract: In assignment problems, decision makers are often interested in not only the optimal assignment, but also the sensitivity of the optimal assignment to perturbations in the assignment weights. Typically, only perturbations to individual assignment weights are considered. We present a novel extension of the traditional sensitivity analysis by allowing for simultaneous variations in all assignment we… ▽ More In assignment problems, decision makers are often interested in not only the optimal assignment, but also the sensitivity of the optimal assignment to perturbations in the assignment weights. Typically, only perturbations to individual assignment weights are considered. We present a novel extension of the traditional sensitivity analysis by allowing for simultaneous variations in all assignment weights. Focusing on the bottleneck assignment problem, we provide two different methods of quantifying the sensitivity of the optimal assignment, and present algorithms for each. Numerical examples as well as a discussion of the complexity for all algorithms are provided. △ Less

Submitted 10 February, 2022; v1 submitted 1 April, 2021; originally announced April 2021.

Comments: 28 pages

Safe Learning of Uncertain Environments

Authors: Farhad Farokhi, Alex Leong, Iman Shames, Mohammad Zamani

Abstract: In many learning based control methodologies, learning the unknown dynamic model precedes the control phase, while the aim is to control the system such that it remains in some safe region of the state space. In this work, our aim is to guarantee safety while learning and control proceed simultaneously. Specifically, we consider the problem of safe learning in nonlinear control-affine systems subj… ▽ More In many learning based control methodologies, learning the unknown dynamic model precedes the control phase, while the aim is to control the system such that it remains in some safe region of the state space. In this work, our aim is to guarantee safety while learning and control proceed simultaneously. Specifically, we consider the problem of safe learning in nonlinear control-affine systems subject to unknown additive uncertainty. We first model the uncertainty as a Gaussian noise and use state measurements to learn its mean and covariance. We provide rigorous time-varying bounds on the mean and covariance of the uncertainty and employ them to modify the control input via an optimization program with potentially time-varying safety constraints. We show that with an arbitrarily large probability we can guarantee that the state will remain in the safe set, while learning and control are carried out simultaneously, provided that a feasible solution exists for the optimization problem. We provide a secondary formulation of this optimization that is computationally more efficient. This is based on tightening the safety constraints to counter the uncertainty about the learned mean and covariance. The magnitude of the tightening can be decreased as our confidence in the learned mean and covariance increases (i.e., as we gather more measurements about the environment). Extensions of the method are provided for non-Gaussian process noise with unknown mean and covariance as well as Gaussian uncertainties with state-dependent mean and covariance to accommodate more general environments. △ Less

Submitted 13 May, 2021; v1 submitted 1 March, 2021; originally announced March 2021.

A Sequential Learning Algorithm for Probabilistically Robust Controller Tuning

Authors: Robert Chin, Chris Manzie, Iman Shames, Dragan Nešić, Jonathan E. Rowe

Abstract: We introduce a sequential learning algorithm to address a robust controller tuning problem, which in effect, finds (with high probability) a candidate solution satisfying the internal performance constraint to a chance-constrained program which has black-box functions. The algorithm leverages ideas from the areas of randomised algorithms and ordinal optimisation, and also draws comparisons with th… ▽ More We introduce a sequential learning algorithm to address a robust controller tuning problem, which in effect, finds (with high probability) a candidate solution satisfying the internal performance constraint to a chance-constrained program which has black-box functions. The algorithm leverages ideas from the areas of randomised algorithms and ordinal optimisation, and also draws comparisons with the scenario approach; these have all been previously applied to finding approximate solutions for difficult design problems. By exploiting statistical correlations through black-box sampling, we formally prove that our algorithm yields a controller meeting the prescribed probabilistic performance specification. Additionally, we characterise the computational requirement of the algorithm with a probabilistic lower bound on the algorithm's stop** time. To validate our work, the algorithm is then demonstrated for tuning model predictive controllers on a diesel engine air-path across a fleet of vehicles. The algorithm successfully tuned a single controller to meet a desired tracking error performance, even in the presence of the plant uncertainty inherent across the fleet. Moreover, the algorithm was shown to exhibit a sample complexity comparable to the scenario approach. △ Less

Submitted 17 October, 2021; v1 submitted 18 February, 2021; originally announced February 2021.

Comments: 17 pages including appendices and references

Asynchronous Distributed Optimization via Dual Decomposition and Block Coordinate Subgradient Methods

Authors: Yankai Lin, Iman Shames, Dragan Nesic

Abstract: We study the problem of minimizing the sum of potentially non-differentiable convex cost functions with partially overlap** dependences in an asynchronous manner, where communication in the network is not coordinated. We study the behavior of an asynchronous algorithm based on dual decomposition and block coordinate subgradient methods under assumptions weaker than those used in the literature.… ▽ More We study the problem of minimizing the sum of potentially non-differentiable convex cost functions with partially overlap** dependences in an asynchronous manner, where communication in the network is not coordinated. We study the behavior of an asynchronous algorithm based on dual decomposition and block coordinate subgradient methods under assumptions weaker than those used in the literature. At the same time, we allow different agents to use local stepsizes with no global coordination. Sufficient conditions are provided for almost sure convergence to the solution of the optimization problem. Under additional assumptions, we establish a sublinear convergence rate that in turn can be strengthened to linear convergence rate if the problem is strongly convex and has Lipschitz gradients. We also extend available results in the literature by allowing multiple and potentially overlap** blocks to be updated at the same time with non-uniform and potentially time varying probabilities assigned to different blocks. A numerical example is provided to illustrate the effectiveness of the algorithm. △ Less

Submitted 15 February, 2021; originally announced February 2021.

MSC Class: 93D99 (primary); 90C25 (secondary); 49M29

Rigid-profile input scheduling under constrained dynamics with a water network application

Authors: Adair Lang, Michael Cantoni, Farhad Farokhi, Iman Shames

Abstract: The motivation for this work stems from the problem of scheduling requests for flow at supply points along an automated network of open-water channels. The off-take flows are rigid-profile inputs to the system dynamics. In particular, the channel operator can only shift orders in time to satisfy constraints on the automatic response to changes in the load. This leads to a non-convex semi-infinite… ▽ More The motivation for this work stems from the problem of scheduling requests for flow at supply points along an automated network of open-water channels. The off-take flows are rigid-profile inputs to the system dynamics. In particular, the channel operator can only shift orders in time to satisfy constraints on the automatic response to changes in the load. This leads to a non-convex semi-infinite programming problem, with sum-separable cost that encodes the collective sensitivity of end users to scheduling delays. The constraints encode the linear time-invariant continuous-time dynamics and limits on the state across a \al{continuous} scheduling horizon. Discretization is used to arrive at a more manageable approximation of the semi-infinite program. A method for parsimoniously refining the discretization is applied to ensure continuous-time feasibility for solutions of the approximate problem. It is then shown how to improve cost without loss of feasibility. Supporting analysis is provided, along with simulation results for a realistic irrigation channel setup to illustrate the approach. △ Less

Submitted 2 December, 2020; originally announced December 2020.

MSC Class: 93A10 (Primary); 90C34 (Secondary); 90B35

Journal ref: IEEE Transactions on Control Systems Technology. Early Access (2020) 1-16

Approximate Midpoint Policy Iteration for Linear Quadratic Control

Authors: Benjamin Gravell, Iman Shames, Tyler Summers

Abstract: We present a midpoint policy iteration algorithm to solve linear quadratic optimal control problems in both model-based and model-free settings. The algorithm is a variation of Newton's method, and we show that in the model-based setting it achieves cubic convergence, which is superior to standard policy iteration and policy gradient algorithms that achieve quadratic and linear convergence, respec… ▽ More We present a midpoint policy iteration algorithm to solve linear quadratic optimal control problems in both model-based and model-free settings. The algorithm is a variation of Newton's method, and we show that in the model-based setting it achieves cubic convergence, which is superior to standard policy iteration and policy gradient algorithms that achieve quadratic and linear convergence, respectively. We also demonstrate that the algorithm can be approximately implemented without knowledge of the dynamics model by using least-squares estimates of the state-action value function from trajectory data, from which policy improvements can be obtained. With sufficient trajectory data, the policy iterates converge cubically to approximately optimal policies, and this occurs with the same available sample budget as the approximate standard policy iteration. Numerical experiments demonstrate effectiveness of the proposed algorithms. △ Less

Submitted 15 February, 2022; v1 submitted 28 November, 2020; originally announced November 2020.

A Distributed Augmenting Path Approach for the Bottleneck Assignment Problem

Authors: Mitchell Khoo, Tony A. Wood, Chris Manzie, Iman Shames

Abstract: We develop an algorithm to solve the Bottleneck Assignment Problem (BAP) that is amenable to having computation distributed over a network of agents. This consists of exploring how each component of the algorithm can be distributed, with a focus on one component in particular, i.e., the function to search for an augmenting path. An augmenting path is a common tool used in most BAP algorithms and p… ▽ More We develop an algorithm to solve the Bottleneck Assignment Problem (BAP) that is amenable to having computation distributed over a network of agents. This consists of exploring how each component of the algorithm can be distributed, with a focus on one component in particular, i.e., the function to search for an augmenting path. An augmenting path is a common tool used in most BAP algorithms and poses a particular challenge for this distributed approach. Given this significance, we compare two different methods to search for an augmenting path in a bipartite graph. We also exploit properties of the augmenting paths to formalise conditions for which the solution from subsets of the sets of agents and tasks can be used to solve the BAP with the full sets of agents and tasks. In the end, we evaluate and compare the derived approaches with a numerical analysis. △ Less

Submitted 18 November, 2020; originally announced November 2020.

Tracking and regret bounds for online zeroth-order Euclidean and Riemannian optimisation

Authors: Alejandro I. Maass, Chris Manzie, Dragan Nesic, Jonathan H. Manton, Iman Shames

Abstract: We study numerical optimisation algorithms that use zeroth-order information to minimise time-varying geodesically-convex cost functions on Riemannian manifolds. In the Euclidean setting, zeroth-order algorithms have received a lot of attention in both the time-varying and time-invariant cases. However, the extension to Riemannian manifolds is much less developed. We focus on Hadamard manifolds, w… ▽ More We study numerical optimisation algorithms that use zeroth-order information to minimise time-varying geodesically-convex cost functions on Riemannian manifolds. In the Euclidean setting, zeroth-order algorithms have received a lot of attention in both the time-varying and time-invariant cases. However, the extension to Riemannian manifolds is much less developed. We focus on Hadamard manifolds, which are a special class of Riemannian manifolds with global nonpositive curvature that offer convenient grounds for the generalisation of convexity notions. Specifically, we derive bounds on the expected instantaneous tracking error, and we provide algorithm parameter values that minimise the algorithm's performance. Our results illustrate how the manifold geometry in terms of the sectional curvature affects these bounds. Additionally, we provide dynamic regret bounds for this online optimisation setting. To the best of our knowledge, these are the first regret bounds even for the Euclidean version of the problem. Lastly, via numerical simulations, we demonstrate the applicability of our algorithm on an online Karcher mean problem. △ Less

Submitted 13 February, 2022; v1 submitted 1 October, 2020; originally announced October 2020.

Comments: 27 pages, 2 figures

MSC Class: 68T05; 68Q32 (Primary); 90C25; 90C56 (Secondary)

A Greedy and Distributable Approach to the Lexicographic Bottleneck Assignment Problem with Conditions on Exactness

Authors: Mitchell Khoo, Tony A. Wood, Chris Manzie, Iman Shames

Abstract: Solving the Lexicographic Bottleneck Assignment Problem (LexBAP) typically relies on centralised computation with order quartic complexity. We consider the Sequential Bottleneck Assignment Problem (SeqBAP), which yields a greedy solution to the LexBAP and discuss the relationship between the SeqBAP, the LexBAP, and the Bottleneck Assignment Problem (BAP). In particular, we reexamine tools used to… ▽ More Solving the Lexicographic Bottleneck Assignment Problem (LexBAP) typically relies on centralised computation with order quartic complexity. We consider the Sequential Bottleneck Assignment Problem (SeqBAP), which yields a greedy solution to the LexBAP and discuss the relationship between the SeqBAP, the LexBAP, and the Bottleneck Assignment Problem (BAP). In particular, we reexamine tools used to analyse the structure of the BAP, and apply them to derive an algorithm that solves the SeqBAP with cubic complexity. We show that the set of solutions of the LexBAP is a subset of the solutions of the SeqBAP and analyse the conditions for which the solutions sets are identical. Furthermore, we provide a method to verify the satisfaction of these conditions. In cases where the conditions are satisfied, the proposed algorithm for solving the SeqBAP solves the LexBAP with computation that has lower complexity and can be distributed over a network of computing agents. The applicability of the approach is demonstrated with a case study where mobile robots are assigned to goal locations. △ Less

Submitted 7 January, 2022; v1 submitted 28 August, 2020; originally announced August 2020.

Exploiting Structure in the Bottleneck Assignment Problem

Authors: Mitchell Khoo, Tony A. Wood, Chris Manzie, Iman Shames

Abstract: An assignment problem arises when there exists a set of tasks that must be allocated to a set of agents. The bottleneck assignment problem (BAP) has the objective of minimising the most costly allocation of a task to an agent. Under certain conditions the structure of the BAP can be exploited such that subgroups of tasks are assigned separately with lower complexity and then merged to form a combi… ▽ More An assignment problem arises when there exists a set of tasks that must be allocated to a set of agents. The bottleneck assignment problem (BAP) has the objective of minimising the most costly allocation of a task to an agent. Under certain conditions the structure of the BAP can be exploited such that subgroups of tasks are assigned separately with lower complexity and then merged to form a combined assignment. In particular, we discuss merging the assignments from two separate BAPs and use the solution of the subproblems to bound the solution of the combined problem. We also provide conditions for cases where the solution of the subproblems produces an exact solution to the BAP over the combined problem. We then introduce a particular algorithm for solving the BAP that takes advantage of this insight. The methods are demonstrated in a numerical case study. △ Less

Submitted 25 August, 2020; originally announced August 2020.

Comments: Accepted in IFAC World Congress 2020

Online Stochastic Convex Optimization: Wasserstein Distance Variation

Authors: Iman Shames, Farhad Farokhi

Abstract: Distributionally-robust optimization is often studied for a fixed set of distributions rather than time-varying distributions that can drift significantly over time (which is, for instance, the case in finance and sociology due to underlying expansion of economy and evolution of demographics). This motivates understanding conditions on probability distributions, using the Wasserstein distance, tha… ▽ More Distributionally-robust optimization is often studied for a fixed set of distributions rather than time-varying distributions that can drift significantly over time (which is, for instance, the case in finance and sociology due to underlying expansion of economy and evolution of demographics). This motivates understanding conditions on probability distributions, using the Wasserstein distance, that can be used to model time-varying environments. We can then use these conditions in conjunction with online stochastic optimization to adapt the decisions. We considers an online proximal-gradient method to track the minimizers of expectations of smooth convex functions parameterised by a random variable whose probability distributions continuously evolve over time at a rate similar to that of the rate at which the decision maker acts. We revisit the concepts of estimation and tracking error inspired by systems and control literature and provide bounds for them under strong convexity, Lipschitzness of the gradient, and bounds on the probability distribution drift. Further, noting that computing projections for a general feasible sets might not be amenable to online implementation (due to computational constraints), we propose an exact penalty method. Doing so allows us to relax the uniform boundedness of the gradient and establish dynamic regret bounds for tracking and estimation error. We further introduce a constraint-tightening approach and relate the amount of tightening to the probability of satisfying the constraints. △ Less

Submitted 29 September, 2020; v1 submitted 2 June, 2020; originally announced June 2020.

Uncertainty Intervals for Robust Bottleneck Assignment

Authors: Elad Michael, Tony A. Wood, Chris Manzie, Iman Shames

Abstract: We examine the robustness of bottleneck assignment problems to perturbations in the assignment weights. We derive two algorithms that provide uncertainty bounds for robust assignment. We prove that the bottleneck assignment is guaranteed to be invariant to perturbations which lie within the provided bounds. We apply the method to an example of task assignment for a multi-agent system. We examine the robustness of bottleneck assignment problems to perturbations in the assignment weights. We derive two algorithms that provide uncertainty bounds for robust assignment. We prove that the bottleneck assignment is guaranteed to be invariant to perturbations which lie within the provided bounds. We apply the method to an example of task assignment for a multi-agent system. △ Less

Submitted 24 May, 2020; originally announced May 2020.

Comments: 6 pages, 1 figure, accepted at the European Control Conference

MSC Class: 90B80 (Primary) 93B35 (Secondary)

Journal ref: 2019 18th European Control Conference (ECC), Naples, Italy, 2019, pp. 4204-4209

Global Sensitivity Analysis for the Linear Assignment Problem

Authors: Elad Michael, Tony A. Wood, Chris Manzie, Iman Shames

Abstract: In this paper, the following question is addressed: given a linear assignment problem, how much can the all of the individual assignment weights be perturbed without changing the optimal assignment? The extension of results involving perturbations in just one edge or one row/column are presented. Algorithms for the derivation of these bounds are provided. We also show how these bounds may be used… ▽ More In this paper, the following question is addressed: given a linear assignment problem, how much can the all of the individual assignment weights be perturbed without changing the optimal assignment? The extension of results involving perturbations in just one edge or one row/column are presented. Algorithms for the derivation of these bounds are provided. We also show how these bounds may be used to prevent assignment churning in a multi-vehicle guidance scenario. △ Less

Submitted 24 May, 2020; originally announced May 2020.

Comments: 6 pages, 1 figure, accepted in the American Control Conference,

MSC Class: 90B80 (Primary) 93B35 (Secondary)

Interior Point Differential Dynamic Programming

Authors: Andrei Pavlov, Iman Shames, Chris Manzie

Abstract: This paper introduces a novel Differential Dynamic Programming (DDP) algorithm for solving discrete-time finite-horizon optimal control problems with inequality constraints. Two variants, namely Feasible- and Infeasible-IPDDP algorithms, are developed using primal-dual interior-point methodology, and their local quadratic convergence properties are characterised. We show that the stationary points… ▽ More This paper introduces a novel Differential Dynamic Programming (DDP) algorithm for solving discrete-time finite-horizon optimal control problems with inequality constraints. Two variants, namely Feasible- and Infeasible-IPDDP algorithms, are developed using primal-dual interior-point methodology, and their local quadratic convergence properties are characterised. We show that the stationary points of the algorithms are the perturbed KKT points, and thus can be moved arbitrarily close to a locally optimal solution. Being free from the burden of the active-set methods, it can handle nonlinear state and input inequality constraints without a discernible increase in its computational complexity relative to the unconstrained case. The performance of the proposed algorithms is demonstrated using numerical experiments on three different problems: control-limited inverted pendulum, car-parking, and unicycle motion control and obstacle avoidance. △ Less

Submitted 20 October, 2020; v1 submitted 27 April, 2020; originally announced April 2020.

Comments: 8 pages, 8 figures

Collision Avoidance Based on Robust Lexicographic Task Assignment

Authors: Tony A. Wood, Mitchell Khoo, Elad Michael, Chris Manzie, Iman Shames

Abstract: Traditional task assignment approaches for multi-agent motion control do not take the possibility of collisions into account. This can lead to challenging requirements for path planning. We derive an assignment method that not only minimises the largest distance between an agent and its assigned destination but also provides local constraints for guaranteed collision avoidance. To this end, we int… ▽ More Traditional task assignment approaches for multi-agent motion control do not take the possibility of collisions into account. This can lead to challenging requirements for path planning. We derive an assignment method that not only minimises the largest distance between an agent and its assigned destination but also provides local constraints for guaranteed collision avoidance. To this end, we introduce a sequential bottleneck optimisation problem and define a notion of robustness of an optimising assignment to changes of individual assignment costs. Conditioned on a sufficient level of robustness in relation to the size of the agents, we construct time-varying position bounds for every individual agent. These local constraints are a direct byproduct of the assignment procedure and only depend on the initial agent positions, the destinations that are to be visited, and a timing parameter. We prove that no agent that is assigned to move to one of the target locations collides with any other agent if all agents satisfy their local position constraints. We demonstrate the method in a illustrative case study. △ Less

Submitted 17 July, 2020; v1 submitted 25 February, 2020; originally announced February 2020.

Tuning of multivariable model predictive controllersthrough expert bandit feedback

Authors: Alex. S. Ira, Chris Manzie, Iman Shames, Robert Chin, Dragan Nesic, Hayato Nakada, Takeshi Sano

Abstract: For certain industrial control applications an explicit function capturing the nontrivial trade-off between competing objectives in closed loop performance is not available. In such scenarios it is common practice to use the human innate ability to implicitly learn such a relationship and manually tune the corresponding controller to achieve the desirable closed loop performance. This approach has… ▽ More For certain industrial control applications an explicit function capturing the nontrivial trade-off between competing objectives in closed loop performance is not available. In such scenarios it is common practice to use the human innate ability to implicitly learn such a relationship and manually tune the corresponding controller to achieve the desirable closed loop performance. This approach has its deficiencies because of individual variations due to experience levels and preferences in the absence of an explicit calibration metric. Moreover, as the complexity of the underlying system and/or the controller increase, in the effort to achieve better performance, so does the tuning time and the associated tuning cost. To reduce the overall tuning cost, a tuning framework is proposed herein, whereby a supervised machine learning is used to extract the human-learned cost function and an optimization algorithm that can efficiently deal with a large number of variables, is used for optimizing the extracted cost function. Given the interest in the implementation across many industrial domains and the associated high degree of freedom present in the corresponding tuning process, a Model Predictive Controller applied to air path control in a diesel engine is tuned for the purpose of demonstrating the potential of the framework. △ Less

Submitted 9 February, 2020; originally announced February 2020.

MSC Class: 93C83; 93C85; 93C95; 90C56; 90C90

Second-order Online Nonconvex Optimization

Authors: Antoine Lesage-Landry, Joshua A. Taylor, Iman Shames

Abstract: We present the online Newton's method, a single-step second-order method for online nonconvex optimization. We analyze its performance and obtain a dynamic regret bound that is linear in the cumulative variation between round optima. We show that if the variation between round optima is limited, the method leads to a constant regret bound. In the general case, the online Newton's method outperform… ▽ More We present the online Newton's method, a single-step second-order method for online nonconvex optimization. We analyze its performance and obtain a dynamic regret bound that is linear in the cumulative variation between round optima. We show that if the variation between round optima is limited, the method leads to a constant regret bound. In the general case, the online Newton's method outperforms online convex optimization algorithms for convex functions and performs similarly to a specialized algorithm for strongly convex functions. We simulate the performance of the online Newton's method on a nonlinear, nonconvex moving target localization example and find that it outperforms a first-order approach. △ Less

Submitted 9 June, 2020; v1 submitted 27 January, 2020; originally announced January 2020.

Ordinal Optimisation for the Gaussian Copula Model

Authors: Robert Chin, Jonathan E. Rowe, Iman Shames, Chris Manzie, Dragan Nešić

Abstract: We present results on the estimation and evaluation of success probabilities for ordinal optimisation over uncountable sets (such as subsets of $\mathbb{R}^{d}$). Our formulation invokes an assumption of a Gaussian copula model, and we show that the success probability can be equivalently computed by assuming a special case of additive noise. We formally prove a lower bound on the success probabil… ▽ More We present results on the estimation and evaluation of success probabilities for ordinal optimisation over uncountable sets (such as subsets of $\mathbb{R}^{d}$). Our formulation invokes an assumption of a Gaussian copula model, and we show that the success probability can be equivalently computed by assuming a special case of additive noise. We formally prove a lower bound on the success probability under the Gaussian copula model, and numerical experiments demonstrate that the lower bound yields a reasonable approximation to the actual success probability. Lastly, we showcase the utility of our results by guaranteeing high success probabilities with ordinal optimisation. △ Less

Submitted 13 May, 2021; v1 submitted 5 November, 2019; originally announced November 2019.

Comments: 18 pages, including appendices and references

Predictive Online Convex Optimization

Authors: Antoine Lesage-Landry, Iman Shames, Joshua A. Taylor

Abstract: We incorporate future information in the form of the estimated value of future gradients in online convex optimization. This is motivated by demand response in power systems, where forecasts about the current round, e.g., the weather or the loads' behavior, can be used to improve on predictions made with only past observations. Specifically, we introduce an additional predictive step that follows… ▽ More We incorporate future information in the form of the estimated value of future gradients in online convex optimization. This is motivated by demand response in power systems, where forecasts about the current round, e.g., the weather or the loads' behavior, can be used to improve on predictions made with only past observations. Specifically, we introduce an additional predictive step that follows the standard online convex optimization step when certain conditions on the estimated gradient and descent direction are met. We show that under these conditions and without any assumptions on the predictability of the environment, the predictive update strictly improves on the performance of the standard update. We give two types of predictive update for various family of loss functions. We provide a regret bound for each of our predictive online convex optimization algorithms. Finally, we apply our framework to an example based on demand response which demonstrates its superior performance to a standard online convex optimization algorithm. △ Less

Submitted 29 November, 2019; v1 submitted 15 May, 2019; originally announced May 2019.

Journal ref: Automatica, 113: 108771, March 2020

Implementing Homomorphic Encryption Based Secure Feedback Control for Physical Systems

Authors: Julian Tran, Farhad Farokhi, Michael Cantoni, Iman Shames

Abstract: This paper is about an encryption based approach to the secure implementation of feedback controllers for physical systems. Specifically, Paillier's homomorphic encryption is used to digitally implement a class of linear dynamic controllers, which includes the commonplace static gain and PID type feedback control laws as special cases. The developed implementation is amenable to Field Programmable… ▽ More This paper is about an encryption based approach to the secure implementation of feedback controllers for physical systems. Specifically, Paillier's homomorphic encryption is used to digitally implement a class of linear dynamic controllers, which includes the commonplace static gain and PID type feedback control laws as special cases. The developed implementation is amenable to Field Programmable Gate Array (FPGA) realization. Experimental results, including timing analysis and resource usage characteristics for different encryption key lengths, are presented for the realization of an inverted pendulum controller; as this is an unstable plant, the control is necessarily fast. △ Less

Submitted 27 March, 2019; v1 submitted 19 February, 2019; originally announced February 2019.

Journal ref: Control Engineering Practice, Volume 97, April 2020, 104350

Secure and Private Implementation of Dynamic Controllers Using Semi-Homomorphic Encryption

Authors: Carlos Murguia, Farhad Farokhi, Iman Shames

Abstract: This paper presents a secure and private implementation of linear time-invariant dynamic controllers using Paillier's encryption, a semi-homomorphic encryption method. To avoid overflow or underflow within the encryption domain, the state of the controller is reset periodically. A control design approach is presented to ensure stability and optimize performance of the closed-loop system with encry… ▽ More This paper presents a secure and private implementation of linear time-invariant dynamic controllers using Paillier's encryption, a semi-homomorphic encryption method. To avoid overflow or underflow within the encryption domain, the state of the controller is reset periodically. A control design approach is presented to ensure stability and optimize performance of the closed-loop system with encrypted controller. △ Less

Submitted 20 June, 2019; v1 submitted 10 December, 2018; originally announced December 2018.

Comments: Improved numerical example

Preserving Privacy of Finite Impulse Response Systems

Authors: Giulio Bottegal, Farhad Farokhi, Iman Shames

Abstract: Adding input and output noises for increasing model identification error of finite impulse response (FIR) systems is considered. This is motivated by the desire to protect the model of the system as a trade secret by rendering model identification techniques ineffective. Optimal filters for constructing additive noises that maximizes the identification error subject to maintaining the closed-loop… ▽ More Adding input and output noises for increasing model identification error of finite impulse response (FIR) systems is considered. This is motivated by the desire to protect the model of the system as a trade secret by rendering model identification techniques ineffective. Optimal filters for constructing additive noises that maximizes the identification error subject to maintaining the closed-loop performance degradation below a limit are constructed. Furthermore, differential privacy is used for designing output noises that preserve the privacy of the model. △ Less

Submitted 5 June, 2017; originally announced June 2017.

On Eigenvalues of Laplacian Matrix for a Class of Directed Signed Graphs

Authors: Saeed Ahmadizadeh, Iman Shames, Samuel Martin, Dragan Nesic

Abstract: The eigenvalues of the Laplacian matrix for a class of directed graphs with both positive and negative weights are studied. First, a class of directed signed graphs is investigated in which one pair of nodes (either connected or not) is perturbed with negative weights. A necessary condition is proposed to attain the following objective for the perturbed graph: the real parts of the non-zero eigenv… ▽ More The eigenvalues of the Laplacian matrix for a class of directed graphs with both positive and negative weights are studied. First, a class of directed signed graphs is investigated in which one pair of nodes (either connected or not) is perturbed with negative weights. A necessary condition is proposed to attain the following objective for the perturbed graph: the real parts of the non-zero eigenvalues of its Laplacian matrix are positive. A sufficient condition is also presented that ensures the aforementioned objective for the unperturbed graph. It is then highlighted the case where the condition becomes necessary and sufficient. Secondly, for directed graphs, a subset of pairs of nodes are identified where if any of the pairs is connected by an edge with infinitesimal negative weight, the resulting Laplacian matrix will have at least one eigenvalue with negative real part. Illustrative examples are presented to show the applicability of our results. △ Less

Submitted 11 May, 2017; originally announced May 2017.

Private and Secure Coordination of Match-Making for Heavy-Duty Vehicle Platooning

Authors: Farhad Farokhi, Iman Shames, Karl H. Johansson

Abstract: A secure and private framework for inter-agent communication and coordination is developed. This allows an agent, in our case a fleet owner, to ask questions or submit queries in an encrypted fashion using semi-homomorphic encryption. The submitted query can be about the interest of the other fleet owners for using a road at a specific time of the day, for instance, for the purpose of collaborativ… ▽ More A secure and private framework for inter-agent communication and coordination is developed. This allows an agent, in our case a fleet owner, to ask questions or submit queries in an encrypted fashion using semi-homomorphic encryption. The submitted query can be about the interest of the other fleet owners for using a road at a specific time of the day, for instance, for the purpose of collaborative vehicle platooning. The other agents can then provide appropriate responses without knowing the content of the questions or the queries. Strong privacy and security guarantees are provided for the agent who is submitting the queries. It is also shown that the amount of the information that this agent can extract from the other agent is bounded. In fact, with submitting one query, a sophisticated agent can at most extract the answer to two queries. This secure communication platform is used subsequently to develop a distributed coordination mechanisms among fleet owners. △ Less

Submitted 27 February, 2017; originally announced February 2017.

Scalable computation for optimal control of cascade systems with constraints

Authors: Michael Cantoni, Farhad Farokhi, Eric C. Kerrigan, Iman Shames

Abstract: A method is devised for numerically solving a class of finite-horizon optimal control problems subject to cascade linear discrete-time dynamics. It is assumed that the linear state and input inequality constraints, and the quadratic measure of performance, are all separable with respect to the spatial dimension of the underlying cascade of sub-systems, as well as the temporal dimension of the dyna… ▽ More A method is devised for numerically solving a class of finite-horizon optimal control problems subject to cascade linear discrete-time dynamics. It is assumed that the linear state and input inequality constraints, and the quadratic measure of performance, are all separable with respect to the spatial dimension of the underlying cascade of sub-systems, as well as the temporal dimension of the dynamics. By virtue of this structure, the computation cost of an interior-point method for an equivalent quadratic programming formulation of the optimal control problem can be made to scale linearly with the number of sub-systems. However, the complexity of this approach grows cubically with the time horizon. As such, computational advantage becomes apparent in situations where the number of sub-systems is relatively large. In any case, the method is amenable to distributed computation with low communication overhead and only immediate upstream neighbour sharing of partial model data among processing agents. An example is presented to illustrate an application of the main results to model data for the cascade dynamics of an automated irrigation channel. △ Less

Submitted 12 October, 2017; v1 submitted 21 February, 2017; originally announced February 2017.

Journal ref: International Journal of Control, 2017

Preserving Privacy of Agents in Participatory-Sensing Schemes for Traffic Estimation

Authors: Farhad Farokhi, Iman Shames

Abstract: A measure of privacy infringement for agents (or participants) travelling across a transportation network in participatory-sensing schemes for traffic estimation is introduced. The measure is defined to be the conditional probability that an external observer assigns to the private nodes in the transportation network, e.g., location of home or office, given all the position measurements that it br… ▽ More A measure of privacy infringement for agents (or participants) travelling across a transportation network in participatory-sensing schemes for traffic estimation is introduced. The measure is defined to be the conditional probability that an external observer assigns to the private nodes in the transportation network, e.g., location of home or office, given all the position measurements that it broadcasts over time. An algorithm for finding an optimal trade-off between the measure of privacy infringement and the expected estimation error, captured by the number of the nodes over which the participant stops broadcasting its position, is proposed. The algorithm searches over a family of policies in which an agent stops transmitting its position measurements if its distance (in terms of the number of hops) to the privacy sensitive node is smaller than a prescribed threshold. Employing such symmetric policies are advantageous in terms of the resources required for implementation and the ease of computation. The results are expanded to more general policies. Further, the effect of the heterogeneity of the population density on the optimal policy is explored. Finally, the relationship between the betweenness measure of centrality and the optimal privacy-preserving policy of the agents is numerically explored. △ Less

Submitted 5 September, 2016; originally announced September 2016.

Budget-Constrained Contract Design for Effort-Averse Sensors in Averaging Based Estimation

Authors: Farhad Farokhi, Iman Shames, Michael Cantoni

Abstract: Consider a group of effort-averse, or lazy, sensors that seek to minimize the effort invested to collect measurements of a variable. Increasing the effort invested by the sensors improves the quality of the measurements provided to the central planner but this incurs increased costs to the sensors. The central planner, which processes the sensor measurements, employs an averaging estimator. It als… ▽ More Consider a group of effort-averse, or lazy, sensors that seek to minimize the effort invested to collect measurements of a variable. Increasing the effort invested by the sensors improves the quality of the measurements provided to the central planner but this incurs increased costs to the sensors. The central planner, which processes the sensor measurements, employs an averaging estimator. It also determines contracts for rewarding sensors based on the measurements obtained. The problem of designing a contract that yields an estimation-error based quality-of-service level in return for the reward extended to sensors is investigated in this paper. To this end, a game is formulated between the central planner and the sensors. Conditions for the existence and uniqueness of an equilibrium are identified. The equilibrium is constructed explicitly and its properties in response to a reward based contract are studied. It turns out that the central planner, while not being able to directly measure the effort invested by the sensors, can enhance the estimation quality by rewarding each sensor based on the distance of its measurements from the output of the averaging estimator. Ultimately, optimal contracts are designed from the perspective of the budget required for achieving a specified level of estimation error. △ Less

Submitted 14 February, 2016; v1 submitted 28 September, 2015; originally announced September 2015.

Comments: Improved literature review

On Reconstructability of Quadratic Utility Functions from the Iterations in Gradient Methods

Authors: Farhad Farokhi, Iman Shames, Michael G. Rabbat, Mikael Johansson

Abstract: In this paper, we consider a scenario where an eavesdropper can read the content of messages transmitted over a network. The nodes in the network are running a gradient algorithm to optimize a quadratic utility function where such a utility optimization is a part of a decision making process by an administrator. We are interested in understanding the conditions under which the eavesdropper can rec… ▽ More In this paper, we consider a scenario where an eavesdropper can read the content of messages transmitted over a network. The nodes in the network are running a gradient algorithm to optimize a quadratic utility function where such a utility optimization is a part of a decision making process by an administrator. We are interested in understanding the conditions under which the eavesdropper can reconstruct the utility function or a scaled version of it and, as a result, gain insight into the decision-making process. We establish that if the parameter of the gradient algorithm, i.e.,~the step size, is chosen appropriately, the task of reconstruction becomes practically impossible for a class of Bayesian filters with uniform priors. We establish what step-size rules should be employed to ensure this. △ Less

Submitted 17 September, 2015; originally announced September 2015.

Scheduling Rigid Demands on Continuous-Time Linear Shift-Invariant Systems

Authors: Farhad Farokhi, Michael Cantoni, Iman Shames

Abstract: We consider load scheduling on constrained continuous-time linear dynamical systems, such as automated irrigation and other distribution networks. The requested loads are rigid, i.e., the shapes cannot be changed. Hence, it is only possible to shift the order back-and-forth in time to arrive at a feasible schedule. We present a numerical algorithm based on using log-barrier functions to include th… ▽ More We consider load scheduling on constrained continuous-time linear dynamical systems, such as automated irrigation and other distribution networks. The requested loads are rigid, i.e., the shapes cannot be changed. Hence, it is only possible to shift the order back-and-forth in time to arrive at a feasible schedule. We present a numerical algorithm based on using log-barrier functions to include the state constraints in the social cost function (i.e., an appropriate function of the scheduling delays). This algorithm requires a feasible initialization. Further, in another algorithm, we treat the state constraints as soft constraints and heavily penalize the constraint violations. This algorithm can even be initialized at an infeasible point. The applicability of both these numerical algorithms is demonstrated on an automated irrigation network with two pools and six farms. △ Less

Submitted 17 September, 2015; originally announced September 2015.

Comments: Accepted for Presentation at the 54th IEEE Conference on Decision and Control (CDC 2015)

Quadratic Gaussian Privacy Games

Authors: Farhad Farokhi, Henrik Sandberg, Iman Shames, Michael Cantoni

Abstract: A game-theoretic model for analysing the effects of privacy on strategic communication between agents is devised. In the model, a sender wishes to provide an accurate measurement of the state to a receiver while also protecting its private information (which is correlated with the state) private from a malicious agent that may eavesdrop on its communications with the receiver. A family of nontrivi… ▽ More A game-theoretic model for analysing the effects of privacy on strategic communication between agents is devised. In the model, a sender wishes to provide an accurate measurement of the state to a receiver while also protecting its private information (which is correlated with the state) private from a malicious agent that may eavesdrop on its communications with the receiver. A family of nontrivial equilibria, in which the communicated messages carry information, is constructed and its properties are studied. △ Less

Submitted 17 September, 2015; originally announced September 2015.

Comments: Accepted for Presentation at the 54th IEEE Conference on Decision and Control (CDC 2015)