-
Convex neural network synthesis for robustness in the 1-norm
Authors:
Ross Drummond,
Chris Guiver,
Matthew C. Turner
Abstract:
With neural networks being used to control safety-critical systems, they increasingly have to be both accurate (in the sense of matching inputs to outputs) and robust. However, these two properties are often at odds with each other and a trade-off has to be navigated. To address this issue, this paper proposes a method to generate an approximation of a neural network which is certifiably more robu…
▽ More
With neural networks being used to control safety-critical systems, they increasingly have to be both accurate (in the sense of matching inputs to outputs) and robust. However, these two properties are often at odds with each other and a trade-off has to be navigated. To address this issue, this paper proposes a method to generate an approximation of a neural network which is certifiably more robust. Crucially, the method is fully convex and posed as a semi-definite programme. An application to robustifying model predictive control is used to demonstrate the results. The aim of this work is to introduce a method to navigate the neural network robustness/accuracy trade-off.
△ Less
Submitted 29 May, 2024;
originally announced May 2024.
-
Bounding the difference between model predictive control and neural networks
Authors:
Ross Drummond,
Stephen R. Duncan,
Matthew C. Turner,
Patricia Pauli,
Frank Allgöwer
Abstract:
There is a growing debate on whether the future of feedback control systems will be dominated by data-driven or model-driven approaches. Each of these two approaches has their own complimentary set of advantages and disadvantages, however, only limited attempts have, so far, been developed to bridge the gap between them. To address this issue, this paper introduces a method to bound the worst-case…
▽ More
There is a growing debate on whether the future of feedback control systems will be dominated by data-driven or model-driven approaches. Each of these two approaches has their own complimentary set of advantages and disadvantages, however, only limited attempts have, so far, been developed to bridge the gap between them. To address this issue, this paper introduces a method to bound the worst-case error between feedback control policies based upon model predictive control (MPC) and neural networks (NNs). This result is leveraged into an approach to automatically synthesize MPC policies minimising the worst-case error with respect to a NN. Numerical examples highlight the application of the bounds, with the goal of the paper being to encourage a more quantitative understanding of the relationship between data-driven and model-driven control.
△ Less
Submitted 13 April, 2022;
originally announced April 2022.
-
Zames-Falb Multipliers: don't panic
Authors:
Matthew C. Turner
Abstract:
Zames-Falb multipliers are mathematical constructs which can be used to prove stability of so-called Lur'e systems: systems that consist of a feedback interconnection of a linear element and a static nonlinear element. The main advantage of Zames-Falb multipliers is that they enable "passivity"-like results to be obtained but with a level of conservatism much lower than \emph{pure} passivity resul…
▽ More
Zames-Falb multipliers are mathematical constructs which can be used to prove stability of so-called Lur'e systems: systems that consist of a feedback interconnection of a linear element and a static nonlinear element. The main advantage of Zames-Falb multipliers is that they enable "passivity"-like results to be obtained but with a level of conservatism much lower than \emph{pure} passivity results. However, some of the papers describing the development of the Zames-Falb multiplier machinery are somewhat abstruse and not entirely clear. This article attempts to provide a relatively simple construction of Zames and Falb's main results which will hopefully be understandable to most graduate-level control engineers.
△ Less
Submitted 14 December, 2022; v1 submitted 30 June, 2021;
originally announced June 2021.
-
Reduced-Order Neural Network Synthesis with Robustness Guarantees
Authors:
Ross Drummond,
Mathew C. Turner,
Stephen R. Duncan
Abstract:
In the wake of the explosive growth in smartphones and cyberphysical systems, there has been an accelerating shift in how data is generated away from centralised data towards on-device generated data. In response, machine learning algorithms are being adapted to run locally on board, potentially hardware limited, devices to improve user privacy, reduce latency and be more energy efficient. However…
▽ More
In the wake of the explosive growth in smartphones and cyberphysical systems, there has been an accelerating shift in how data is generated away from centralised data towards on-device generated data. In response, machine learning algorithms are being adapted to run locally on board, potentially hardware limited, devices to improve user privacy, reduce latency and be more energy efficient. However, our understanding of how these device orientated algorithms behave and should be trained is still fairly limited. To address this issue, a method to automatically synthesize reduced-order neural networks (having fewer neurons) approximating the input/output map** of a larger one is introduced. The reduced-order neural network's weights and biases are generated from a convex semi-definite programme that minimises the worst-case approximation error with respect to the larger network. Worst-case bounds for this approximation error are obtained and the approach can be applied to a wide variety of neural networks architectures. What differentiates the proposed approach to existing methods for generating small neural networks, e.g. pruning, is the inclusion of the worst-case approximation error directly within the training cost function, which should add robustness. Numerical examples highlight the potential of the proposed approach. The overriding goal of this paper is to generalise recent results in the robustness analysis of neural networks to a robust synthesis problem for their weights and biases.
△ Less
Submitted 7 December, 2021; v1 submitted 18 February, 2021;
originally announced February 2021.
