-
Adaptive Kalman Filtering Developed from Recursive Least Squares Forgetting Algorithms
Authors:
Brian Lai,
Dennis S. Bernstein
Abstract:
Recursive least squares (RLS) is derived as the recursive minimizer of the least-squares cost function. Moreover, it is well known that RLS is a special case of the Kalman filter. This work presents the Kalman filter least squares (KFLS) cost function, whose recursive minimizer gives the Kalman filter. KFLS is an extension of generalized forgetting recursive least squares (GF-RLS), a general frame…
▽ More
Recursive least squares (RLS) is derived as the recursive minimizer of the least-squares cost function. Moreover, it is well known that RLS is a special case of the Kalman filter. This work presents the Kalman filter least squares (KFLS) cost function, whose recursive minimizer gives the Kalman filter. KFLS is an extension of generalized forgetting recursive least squares (GF-RLS), a general framework which contains various extensions of RLS from the literature as special cases. This then implies that extensions of RLS are also special cases of the Kalman filter. Motivated by this connection, we propose an algorithm that combines extensions of RLS with the Kalman filter, resulting in a new class of adaptive Kalman filters. A numerical example shows that one such adaptive Kalman filter provides improved state estimation for a mass-spring-damper with intermittent, unmodeled collisions. This example suggests that such adaptive Kalman filtering may provide potential benefits for systems with non-classical disturbances.
△ Less
Submitted 16 April, 2024;
originally announced April 2024.
-
Efficient Batch and Recursive Least Squares for Matrix Parameter Estimation
Authors:
Brian Lai,
Dennis S. Bernstein
Abstract:
Traditionally, batch least squares (BLS) and recursive least squares (RLS) are used for identification of a vector of parameters that form a linear model. In some situations, however, it is of interest to identify parameters in a matrix structure. In this case, a common approach is to transform the problem into standard vector form using the vectorization (vec) operator and the Kronecker product,…
▽ More
Traditionally, batch least squares (BLS) and recursive least squares (RLS) are used for identification of a vector of parameters that form a linear model. In some situations, however, it is of interest to identify parameters in a matrix structure. In this case, a common approach is to transform the problem into standard vector form using the vectorization (vec) operator and the Kronecker product, known as vec-permutation. However, the use of the Kronecker product introduces extraneous zero terms in the regressor, resulting in unnecessary additional computational and space requirements. This work derives matrix BLS and RLS formulations which, under mild assumptions, minimize the same cost as the vec-permutation approach. This new approach requires less computational complexity and space complexity than vec-permutation in both BLS and RLS identification. It is also shown that persistent excitation guarantees convergence to the true matrix parameters. This method can used to improve computation time in the online identification of multiple-input, multiple-output systems for indirect adaptive model predictive control.
△ Less
Submitted 9 June, 2024; v1 submitted 16 April, 2024;
originally announced April 2024.
-
Convergence of Recursive Least Squares Based Input/Output System Identification with Model Order Mismatch
Authors:
Brian Lai,
Dennis S. Bernstein
Abstract:
Discrete-time input/output models, also called infinite impulse response (IIR) models or autoregressive moving average (ARMA) models, are useful for online identification as they can be efficiently updated using recursive least squares (RLS) as new data is collected. Several works have studied the convergence of the input/output model coefficients identified using RLS under the assumption that the…
▽ More
Discrete-time input/output models, also called infinite impulse response (IIR) models or autoregressive moving average (ARMA) models, are useful for online identification as they can be efficiently updated using recursive least squares (RLS) as new data is collected. Several works have studied the convergence of the input/output model coefficients identified using RLS under the assumption that the order of the identified model is the same as that of the true system. However, the case of model order mismatch is not as well addressed. This work begins by introducing the notion of \textit{equivalence} of input/output models of different orders. Next, this work analyzes online identification of input/output models in the case where the order of the identified model is higher than that of the true system. It is shown that, given persistently exciting data, the higher-order identified model converges to the model equivalent to the true system that minimizes the regularization term of RLS.
△ Less
Submitted 16 April, 2024;
originally announced April 2024.
-
SIFt-RLS: Subspace of Information Forgetting Recursive Least Squares
Authors:
Brian Lai,
Dennis S. Bernstein
Abstract:
This paper presents subspace of information forgetting recursive least squares (SIFt-RLS), a directional forgetting algorithm which, at each step, forgets only in row space of the regressor matrix, or the \textit{information subspace}. As a result, SIFt-RLS tracks parameters that are in excited directions while not changing parameter estimation in unexcited directions. It is shown that SIFt-RLS gu…
▽ More
This paper presents subspace of information forgetting recursive least squares (SIFt-RLS), a directional forgetting algorithm which, at each step, forgets only in row space of the regressor matrix, or the \textit{information subspace}. As a result, SIFt-RLS tracks parameters that are in excited directions while not changing parameter estimation in unexcited directions. It is shown that SIFt-RLS guarantees an upper and lower bound of the covariance matrix, without assumptions of persistent excitation, and explicit bounds are given. Furthermore, sufficient conditions are given for the uniform Lyapunov stability and global uniform exponential stability of parameter estimation error in SIFt-RLS when estimating fixed parameters without noise. SIFt-RLS is compared to other RLS algorithms from the literature in a numerical example without persistently exciting data.
△ Less
Submitted 16 April, 2024;
originally announced April 2024.
-
Stability Analysis of Adaptive Model Predictive Control Using the Circle and Tsypkin Criteria
Authors:
Juan A. Paredes,
Dennis S. Bernstein
Abstract:
Absolute stability is a technique for analyzing the stability of Lur'e systems, which arise in diverse applications, such as oscillators with nonlinear dam** or nonlinear stiffness. A special class of Lur'e systems consists of self-excited systems (SES), in which bounded oscillations arise from constant inputs. In many cases, SES can be stabilized by linear controllers, which motivates the prese…
▽ More
Absolute stability is a technique for analyzing the stability of Lur'e systems, which arise in diverse applications, such as oscillators with nonlinear dam** or nonlinear stiffness. A special class of Lur'e systems consists of self-excited systems (SES), in which bounded oscillations arise from constant inputs. In many cases, SES can be stabilized by linear controllers, which motivates the present work, where the goal is to evaluate the effectiveness of adaptive model predictive control for Lur'e systems. In particular, the present paper considers predictive cost adaptive control (PCAC), which is equivalent to a linear, time-variant (LTV) controller. A closed-loop Lur'e system comprised of the positive feedback interconnection of the Lur'e system and the PCAC-based controller can thus be derived at each step. In this work, the circle and Tsypkin criteria are used to evaluate the absolute stability of the closed-loop Lur'e system, where the adaptive controller is viewed as instantaneously linear time-invariant. When the controller converges, the absolute stability criteria guarantee global asymptotic stability of the asymptotic closed-loop dynamics.
△ Less
Submitted 5 April, 2024;
originally announced April 2024.
-
Retrospective Cost-based Extremum Seeking Control with Vanishing Perturbation for Online Output Minimization
Authors:
Juan A. Paredes,
Jhon Manuel Portella,
Dennis S. Bernstein,
Ankit Goel
Abstract:
Extremum seeking control (ESC) constitutes a powerful technique for online optimization with theoretical guarantees for convergence to the neighborhood of the optimizer under well-understood conditions. However, ESC requires a nonconstant perturbation signal to provide persistent excitation to the target system to yield convergent results, which usually results in steady state oscillations. While…
▽ More
Extremum seeking control (ESC) constitutes a powerful technique for online optimization with theoretical guarantees for convergence to the neighborhood of the optimizer under well-understood conditions. However, ESC requires a nonconstant perturbation signal to provide persistent excitation to the target system to yield convergent results, which usually results in steady state oscillations. While certain techniques have been proposed to eliminate perturbations once the neighborhood of the minimizer is reached, system modifications and environmental perturbations can suddenly change the minimizer and nonconstant perturbations would once more be required to convergence to the new minimizer. Hence, this paper develops a retrospective cost-based ESC(RC/ESC) technique for online output minimization with a vanishing perturbation, that is, a perturbation that becomes zero as time increases independently from the state of the controller or the controlled system. The performance of the proposed algorithm is illustrated via numerical examples.
△ Less
Submitted 6 February, 2024;
originally announced February 2024.
-
A Data-Driven Autopilot for Fixed-Wing Aircraft Based on Model Predictive Control
Authors:
Riley J. Richards,
Juan A. Paredes,
Dennis S. Bernstein
Abstract:
Autopilots for fixed-wing aircraft are typically designed based on linearized aerodynamic models consisting of stability and control derivatives obtained from wind-tunnel testing. The resulting local controllers are then pieced together using gain scheduling. For applications in which the aerodynamics are unmodeled, the present paper proposes an autopilot based on predictive cost adaptive control…
▽ More
Autopilots for fixed-wing aircraft are typically designed based on linearized aerodynamic models consisting of stability and control derivatives obtained from wind-tunnel testing. The resulting local controllers are then pieced together using gain scheduling. For applications in which the aerodynamics are unmodeled, the present paper proposes an autopilot based on predictive cost adaptive control (PCAC). As an indirect adaptive control extension of model predictive control, PCAC uses recursive least squares (RLS) with variable-rate forgetting for online, closed-loop system identification. At each time step, RLS-based system identification updates the coefficients of an input-output model whose order is a hyperparameter specified by the user. For MPC, the receding-horizon optimization can be performed by either the backward-propagating Riccati equation or quadratic programming. The present paper investigates the performance of PCAC for fixed-wing aircraft without the use of any aerodynamic modeling or offline/prior data collection.
△ Less
Submitted 1 February, 2024;
originally announced February 2024.
-
Experimental Application of Predictive Cost Adaptive Control to Thermoacoustic Oscillations in a Rijke Tube
Authors:
Juan A. Paredes,
Dennis S. Bernstein
Abstract:
Model predictive control (MPC) has been used successfully in diverse applications. As its name suggests, MPC requires a model for predictive optimization. The present paper focuses on the application of MPC to a Rijke tube, in which a heating source and acoustic dynamics interact to produce self-excited oscillations. Since the dynamics of a Rijke tube are difficult to model to a high level of accu…
▽ More
Model predictive control (MPC) has been used successfully in diverse applications. As its name suggests, MPC requires a model for predictive optimization. The present paper focuses on the application of MPC to a Rijke tube, in which a heating source and acoustic dynamics interact to produce self-excited oscillations. Since the dynamics of a Rijke tube are difficult to model to a high level of accuracy, the implementation of MPC requires leveraging data from the physical setup as well as knowledge about thermoacoustics, which is labor intensive and requires domain expertise. With this motivation, the present paper uses predictive cost adaptive control (PCAC) for sampled-data control of an experimental Rijke-tube setup. PCAC performs online closed-loop linear model identification for receding-horizon optimization based on the backward propagating Riccati equation. In place of analytical modeling, open-loop experiments are used to create a simple emulation model, which is used for choosing PCAC hyperparameters. PCAC is applied to the Rijke-tube setup under various experimental scenarios.
△ Less
Submitted 1 February, 2024;
originally announced February 2024.
-
Adaptive Output-Feedback Model Predictive Control of Hammerstein Systems with Unknown Linear Dynamics
Authors:
Mohammadreza Kamaldar,
Dennis S. Bernstein
Abstract:
This paper considers model predictive control of Hammerstein systems, where the linear dynamics are a priori unknown and the input nonlinearity is known. Predictive cost adaptive control (PCAC) is applied to this system using recursive least squares for online, closed-loop system identification with optimization over a receding horizon performed by quadratic programming (QP). In order to account f…
▽ More
This paper considers model predictive control of Hammerstein systems, where the linear dynamics are a priori unknown and the input nonlinearity is known. Predictive cost adaptive control (PCAC) is applied to this system using recursive least squares for online, closed-loop system identification with optimization over a receding horizon performed by quadratic programming (QP). In order to account for the input nonlinearity, the input matrix is defined to be control dependent, and the optimization is performed iteratively. This technique is applied to output stabilization of a chain of integrators with unknown dynamics under control saturation and deadzone input nonlinearity.
△ Less
Submitted 28 September, 2023;
originally announced September 2023.
-
Adaptive Real-Time Numerical Differentiation with Variable-Rate Forgetting and Exponential Resetting
Authors:
Shashank Verma,
Brian Lai,
Dennis S. Bernstein
Abstract:
Digital PID control requires a differencing operation to implement the D gain. In order to suppress the effects of noisy data, the traditional approach is to filter the data, where the frequency response of the filter is adjusted manually based on the characteristics of the sensor noise. The present paper considers the case where the characteristics of the sensor noise change over time in an unkno…
▽ More
Digital PID control requires a differencing operation to implement the D gain. In order to suppress the effects of noisy data, the traditional approach is to filter the data, where the frequency response of the filter is adjusted manually based on the characteristics of the sensor noise. The present paper considers the case where the characteristics of the sensor noise change over time in an unknown way. This problem is addressed by applying adaptive real-time numerical differentiation based on adaptive input and state estimation (AISE). The contribution of this paper is to extend AISE to include variable-rate forgetting with exponential resetting, which allows AISE to more rapidly respond to changing noise characteristics while enforcing the boundedness of the covariance matrix used in recursive least squares.
△ Less
Submitted 28 September, 2023;
originally announced September 2023.
-
Output-Feedback Nonlinear Model Predictive Control with Iterative State- and Control-Dependent Coefficients
Authors:
Mohammadreza Kamaldar,
Dennis S. Bernstein
Abstract:
By optimizing the predicted performance over a receding horizon, model predictive control (MPC) provides the ability to enforce state and control constraints. The present paper considers an extension of MPC for nonlinear systems that can be written in pseudo-linear form with state- and control-dependent coefficients. The main innovation is to apply quadratic programming iteratively over the horizo…
▽ More
By optimizing the predicted performance over a receding horizon, model predictive control (MPC) provides the ability to enforce state and control constraints. The present paper considers an extension of MPC for nonlinear systems that can be written in pseudo-linear form with state- and control-dependent coefficients. The main innovation is to apply quadratic programming iteratively over the horizon, where the predicted state trajectory is updated based on the updated control sequence. Output-feedback control is facilitated by using the block-observable canonical form for linear, time-varying dynamics. This control technique is illustrated on various numerical examples, including the Kapitza pendulum with slider-crank actuation, the nonholonomic integrator, the electromagnetically controlled oscillator, and the triple integrator with control-magnitude saturation.
△ Less
Submitted 28 September, 2023; v1 submitted 20 September, 2023;
originally announced September 2023.
-
Real-Time Kinematics-Based Sensor-Fault Detection for Autonomous Vehicles Using Single and Double Transport with Adaptive Numerical Differentiation
Authors:
Shashank Verma,
Dennis S. Bernstein
Abstract:
Sensor-fault detection is crucial for the safe operation of autonomous vehicles. This paper introduces a novel kinematics-based approach for detecting and identifying faulty sensors, which is model-independent, rule-free, and applicable to ground and aerial vehicles. This method, called kinematics-based sensor fault detection (KSFD), relies on kinematic relations, sensor measurements, and real-tim…
▽ More
Sensor-fault detection is crucial for the safe operation of autonomous vehicles. This paper introduces a novel kinematics-based approach for detecting and identifying faulty sensors, which is model-independent, rule-free, and applicable to ground and aerial vehicles. This method, called kinematics-based sensor fault detection (KSFD), relies on kinematic relations, sensor measurements, and real-time single and double numerical differentiation. Using onboard data from radar, rate gyros, magnetometers, and accelerometers, KSFD uniquely identifies a single faulty sensor in real time. To achieve this, adaptive input and state estimation (AISE) is used for real-time single and double numerical differentiation of the sensor data, and the single and double transport theorems are used to evaluate the consistency of data. Unlike model-based and knowledge-based methods, KSFD relies solely on sensor signals, kinematic relations, and AISE for real-time numerical differentiation. For ground vehicles, KSFD requires six kinematics-based error metrics, whereas, for aerial vehicles, nine error metrics are used. Simulated and experimental examples are provided to evaluate the effectiveness of KSFD.
△ Less
Submitted 1 July, 2024; v1 submitted 10 September, 2023;
originally announced September 2023.
-
Real-Time Numerical Differentiation of Sampled Data Using Adaptive Input and State Estimation
Authors:
Shashank Verma,
Sneha Sanjeevini,
E. Dogan Sumer,
Dennis S. Bernstein
Abstract:
Real-time numerical differentiation plays a crucial role in many digital control algorithms, such as PID control, which requires numerical differentiation to implement derivative action. This paper addresses the problem of numerical differentiation for real-time implementation with minimal prior information about the signal and noise using adaptive input and state estimation. Adaptive input estima…
▽ More
Real-time numerical differentiation plays a crucial role in many digital control algorithms, such as PID control, which requires numerical differentiation to implement derivative action. This paper addresses the problem of numerical differentiation for real-time implementation with minimal prior information about the signal and noise using adaptive input and state estimation. Adaptive input estimation with adaptive state estimation (AIE/ASE) is based on retrospective cost input estimation, while adaptive state estimation is based on an adaptive Kalman filter in which the input-estimation error covariance and the measurement-noise covariance are updated online. The accuracy of AIE/ASE is compared numerically to several conventional numerical differentiation methods. Finally, AIE/ASE is applied to simulated vehicle position data generated from CarSim.
△ Less
Submitted 15 August, 2023;
originally announced August 2023.
-
Generalized Forgetting Recursive Least Squares: Stability and Robustness Guarantees
Authors:
Brian Lai,
Dennis S. Bernstein
Abstract:
This work presents generalized forgetting recursive least squares (GF-RLS), a generalization of recursive least squares (RLS) that encompasses many extensions of RLS as special cases. First, sufficient conditions are presented for the 1) Lyapunov stability, 2) uniform Lyapunov stability, 3) global asymptotic stability, and 4) global uniform exponential stability of parameter estimation error in GF…
▽ More
This work presents generalized forgetting recursive least squares (GF-RLS), a generalization of recursive least squares (RLS) that encompasses many extensions of RLS as special cases. First, sufficient conditions are presented for the 1) Lyapunov stability, 2) uniform Lyapunov stability, 3) global asymptotic stability, and 4) global uniform exponential stability of parameter estimation error in GF-RLS when estimating fixed parameters without noise. Second, robustness guarantees are derived for the estimation of time-varying parameters in the presence of measurement noise and regressor noise. These robustness guarantees are presented in terms of global uniform ultimate boundedness of the parameter estimation error. A specialization of this result gives a bound to the asymptotic bias of least squares estimators in the errors-in-variables problem. Lastly, a survey is presented to show how GF-RLS can be used to analyze various extensions of RLS from the literature.
△ Less
Submitted 6 May, 2024; v1 submitted 8 August, 2023;
originally announced August 2023.
-
Self-Excited Dynamics of Discrete-Time Lur'e Models with Affinely Constrained, Piecewise-C1 Feedback Nonlinearities
Authors:
Juan Paredes,
Omran Kouba,
Dennis S. Bernstein
Abstract:
Self-excited systems (SES) arise in numerous applications, such as fluid-structure interaction, combustion, and biochemical systems. In support of system identification and digital control of SES, this paper analyzes discrete-time Lur'e models with affinely constrained, piecewise-C1 feedback nonlinearities. The main result provides sufficient conditions under which a discrete-time Lur'e model is s…
▽ More
Self-excited systems (SES) arise in numerous applications, such as fluid-structure interaction, combustion, and biochemical systems. In support of system identification and digital control of SES, this paper analyzes discrete-time Lur'e models with affinely constrained, piecewise-C1 feedback nonlinearities. The main result provides sufficient conditions under which a discrete-time Lur'e model is self-excited in the sense that its response is 1) bounded for all initial conditions, and 2) nonconvergent for almost all initial conditions.
△ Less
Submitted 7 July, 2023;
originally announced July 2023.
-
Experimental Flight Testing of an Adaptive Autopilot with Parameter Drift Mitigation
Authors:
Yin Yong Chee,
Parham Oveissi,
Siyuan Shao,
Joonghyun Lee,
Juan A. Paredes,
Dennis S. Bernstein,
Ankit Goel
Abstract:
This paper modifies an adaptive multicopter autopilot to mitigate instabilities caused by adaptive parameter drift and presents simulation and experimental results to validate the modified autopilot. The modified adaptive controller is obtained by including a static nonlinearity in the adaptive loop, updated by the retrospective cost adaptive control algorithm. It is shown in simulation and physic…
▽ More
This paper modifies an adaptive multicopter autopilot to mitigate instabilities caused by adaptive parameter drift and presents simulation and experimental results to validate the modified autopilot. The modified adaptive controller is obtained by including a static nonlinearity in the adaptive loop, updated by the retrospective cost adaptive control algorithm. It is shown in simulation and physical test experiments that the adaptive autopilot with proposed modifications can continually improve the fixed-gain autopilot as well as prevent the drift of the adaptive parameters, thus improving the robustness of the adaptive autopilot.
△ Less
Submitted 20 April, 2023;
originally announced April 2023.
-
Euler's Equation via Lagrangian Dynamics with Generalized Coordinates
Authors:
Dennis S. Bernstein,
Ankit Goel,
Omran Kouba
Abstract:
Euler's equation relates the change in angular momentum of a rigid body to the applied torque. This paper fills a gap in the literature by using Lagrangian dynamics to derive Euler's equation in terms of generalized coordinates. This is done by parameterizing the angular velocity vector in terms of 3-2-1 and 3-1-3 Euler angles as well as Euler parameters, that is, unit quaternions.
Euler's equation relates the change in angular momentum of a rigid body to the applied torque. This paper fills a gap in the literature by using Lagrangian dynamics to derive Euler's equation in terms of generalized coordinates. This is done by parameterizing the angular velocity vector in terms of 3-2-1 and 3-1-3 Euler angles as well as Euler parameters, that is, unit quaternions.
△ Less
Submitted 22 December, 2022;
originally announced December 2022.
-
Experimental Flight Testing of a Fault-Tolerant Adaptive Autopilot for Fixed-Wing Aircraft
Authors:
Joonghyun Lee,
John Spencer,
Siyuan Shao,
Juan Augusto Paredes,
Dennis S. Bernstein,
Ankit Goel
Abstract:
This paper presents an adaptive autopilot for fixed-wing aircraft and compares its performance with a fixed-gain autopilot. The adaptive autopilot is constructed by augmenting the autopilot architecture with adaptive control laws that are updated using retrospective cost adaptive control. In order to investigate the performance of the adaptive autopilot, the default gains of the fixed-gain autopil…
▽ More
This paper presents an adaptive autopilot for fixed-wing aircraft and compares its performance with a fixed-gain autopilot. The adaptive autopilot is constructed by augmenting the autopilot architecture with adaptive control laws that are updated using retrospective cost adaptive control. In order to investigate the performance of the adaptive autopilot, the default gains of the fixed-gain autopilot are scaled to degrade its performance. This scenario provides a venue for determining the ability of the adaptive autopilot to compensate for the degraded fixed-gain autopilot. Next, the performance of the adaptive autopilot is examined under failure conditions by simulating a scenario where one of the control surfaces is assumed to be stuck at an unknown angle. The adaptive autopilot is also tested in physical flight experiments under degraded-nominal conditions, and the resulting performance improvement is examined.
△ Less
Submitted 24 October, 2022;
originally announced October 2022.
-
On the Accuracy of the One-step UKF and the Two-step UKF
Authors:
Ankit Goel,
Dennis S. Bernstein
Abstract:
The most accurate version of the unscented Kalman filter (UKF) involves the construction of two ensembles. To reduce computational cost, however, UKF is often implemented without the second ensemble. This simplification comes at a price, however, since, for linear systems, the one-step variation of the two-step UKF does not specialize to the classical Kalman filter, with an associated loss of accu…
▽ More
The most accurate version of the unscented Kalman filter (UKF) involves the construction of two ensembles. To reduce computational cost, however, UKF is often implemented without the second ensemble. This simplification comes at a price, however, since, for linear systems, the one-step variation of the two-step UKF does not specialize to the classical Kalman filter, with an associated loss of accuracy. This paper remedies this drawback by develo** a modified one-step UKF that recovers the classical Kalman filter for linear systems. Numerical examples show that the modified one-step UKF also recovers the accuracy of the two-step UKF in nonlinear systems with linear outputs.
△ Less
Submitted 18 August, 2022;
originally announced August 2022.
-
Retrospective Cost Parameter Estimation with Application to Space Weather Modeling
Authors:
Ankit Goel,
Dennis S. Bernstein
Abstract:
This chapter reviews standard parameter-estimation techniques and presents a novel gradient-, ensemble-, adjoint-free data-driven parameter estimation technique in the DDDAS framework. This technique, called retrospective cost parameter estimation (RCPE), is motivated by large-scale complex estimation models characterized by high-dimensional nonlinear dynamics, nonlinear parameterizations, and rep…
▽ More
This chapter reviews standard parameter-estimation techniques and presents a novel gradient-, ensemble-, adjoint-free data-driven parameter estimation technique in the DDDAS framework. This technique, called retrospective cost parameter estimation (RCPE), is motivated by large-scale complex estimation models characterized by high-dimensional nonlinear dynamics, nonlinear parameterizations, and representational models. RCPE is illustrated by estimating unknown parameters in three examples. In the first example, salient features of RCPE are investigated by considering parameter estimation problem in a low-order nonlinear system. In the second example, RCPE is used to estimate the convective coefficient and the viscosity in the generalized Burgers equation by using a scalar measurement. In the final example, RCPE is used to estimate thermal conductivity coefficients that relate temporal temperature variation with the vertical gradient of the temperature in the atmosphere.
△ Less
Submitted 15 August, 2022;
originally announced August 2022.
-
An Adaptive Digital Autopilot for Fixed-Wing Aircraft with Actuator Faults
Authors:
Joonghyun Lee,
John Spencer,
Juan Augusto Paredes,
Sai Ravela,
Dennis S. Bernstein,
Ankit Goel
Abstract:
This paper develops an adaptive digital autopilot for a fixed-wing aircraft and compares its performance with a fixed-gain autopilot. The adaptive digital autopilot is constructed by augmenting the autopilot architecture implemented in PX4 flight stack with adaptive digital control laws that are updated using the retrospective cost adaptive control algorithm. In order to investigate the performanc…
▽ More
This paper develops an adaptive digital autopilot for a fixed-wing aircraft and compares its performance with a fixed-gain autopilot. The adaptive digital autopilot is constructed by augmenting the autopilot architecture implemented in PX4 flight stack with adaptive digital control laws that are updated using the retrospective cost adaptive control algorithm. In order to investigate the performance of the adaptive digital autopilot, the default gains of the fixed-gain autopilot are scaled down to degrade its performance. This scenario provides a venue for determining the ability of the adaptive digital autopilot to compensate for the detuned fixed-gain autopilot. Next, the performance of the adaptive autopilot is examined under failure conditions by simulating a scenario where one of the control surfaces is assumed to be stuck at an unknown angular position. The adaptive digital autopilot is tested in simulation, and the resulting performance improvements are examined.
△ Less
Submitted 21 October, 2021;
originally announced October 2021.
-
Self-Excited Dynamics of Discrete-Time Lur'e Systems
Authors:
Juan A. Paredes,
Syed Aseem Ul Islam,
Omran Kouba,
Dennis S. Bernstein
Abstract:
Self-excited systems arise in numerous applications, such as biochemical systems, fluid-structure interaction, and combustion. This paper analyzes a discrete-time Lur'e system with a piecewise-linear saturation feedback nonlinearity. The main result provides sufficient conditions under which the Lur'e system is self-excited in the sense that its response is bounded and nonconvergent.
Self-excited systems arise in numerous applications, such as biochemical systems, fluid-structure interaction, and combustion. This paper analyzes a discrete-time Lur'e system with a piecewise-linear saturation feedback nonlinearity. The main result provides sufficient conditions under which the Lur'e system is self-excited in the sense that its response is bounded and nonconvergent.
△ Less
Submitted 26 September, 2021;
originally announced September 2021.
-
Regularization-Induced Bias and Consistency in Recursive Least Squares
Authors:
Brian Lai,
Syed Aseem Ul Islam,
Dennis S. Bernstein
Abstract:
Within the context of recursive least squares (RLS) parameter estimation, the goal of the present paper is to study the effect of regularization-induced bias on the transient and asymptotic accuracy of the parameter estimates. We consider this question in three stages. First, we consider regression with random data, in which case persistency is guaranteed. Next, we apply RLS to finite-impulse-resp…
▽ More
Within the context of recursive least squares (RLS) parameter estimation, the goal of the present paper is to study the effect of regularization-induced bias on the transient and asymptotic accuracy of the parameter estimates. We consider this question in three stages. First, we consider regression with random data, in which case persistency is guaranteed. Next, we apply RLS to finite-impulse-response (FIR) system identification and, finally, to infinite-impulse-response (IIR) system identification. For each case, we relate the condition number of the regressor matrix to the transient response and rate of convergence of the parameter estimates.
△ Less
Submitted 10 August, 2021; v1 submitted 16 June, 2021;
originally announced June 2021.
-
Two Modifications of the Unscented Kalman Filter that Specialize to the Kalman Filter for Linear Systems
Authors:
Ankit Goel,
Dennis S. Bernstein
Abstract:
Although the unscented Kalman filter (UKF) is applicable to nonlinear systems, it turns out that, for linear systems, UKF does not specialize to the classical Kalman filter. This situation suggests that it may be advantageous to modify UKF in such a way that, for linear systems, the Kalman filter is recovered. The ultimate goal is thus to develop modifications of UKF that specialize to the Kalman…
▽ More
Although the unscented Kalman filter (UKF) is applicable to nonlinear systems, it turns out that, for linear systems, UKF does not specialize to the classical Kalman filter. This situation suggests that it may be advantageous to modify UKF in such a way that, for linear systems, the Kalman filter is recovered. The ultimate goal is thus to develop modifications of UKF that specialize to the Kalman filter for linear systems and have improved accuracy for nonlinear systems. With this motivation, this paper presents two modifications of UKF that specialize to the Kalman filter for linear systems. The first modification (EUKF-A) requires the Jacobian of the dynamics map, whereas the second modification (EUKF-C) requires the Jacobian of the measurement map. For various nonlinear examples, the accuracy of EUKF-A and EUKF-C is compared to the accuracy of UKF.
△ Less
Submitted 1 April, 2021;
originally announced April 2021.
-
Data-Driven Retrospective Cost Adaptive Control for Flight Control Application
Authors:
Syed Aseem Ul Islam,
Tam W. Nguyen,
Ilya V. Kolmanovsky,
Dennis S. Bernstein
Abstract:
Unlike fixed-gain robust control, which trades off performance with modeling uncertainty, direct adaptive control uses partial modeling information for online tuning. The present paper combines retrospective cost adaptive control (RCAC), a direct adaptive control technique for sampled-data systems, with online system identification based on recursive least squares (RLS) with variable-rate forgetti…
▽ More
Unlike fixed-gain robust control, which trades off performance with modeling uncertainty, direct adaptive control uses partial modeling information for online tuning. The present paper combines retrospective cost adaptive control (RCAC), a direct adaptive control technique for sampled-data systems, with online system identification based on recursive least squares (RLS) with variable-rate forgetting (VRF). The combination of RCAC and RLS-VRF constitutes data-driven RCAC (DDRCAC), where the online system identification is used to construct the target model, which defines the retrospective performance variable. This paper investigates the ability of RLS-VRF to provide the modeling information needed for the target model, especially nonminimum-phase (NMP) zeros. DDRCAC is applied to single-input, single-output (SISO) and multiple-input, multiple-output (MIMO) numerical examples with unknown NMP zeros, as well as several flight control problems, namely, unknown transition from minimum-phase to NMP lateral dynamics, flexible modes, flutter, and nonlinear planar missile dynamics.
△ Less
Submitted 8 April, 2021; v1 submitted 14 February, 2021;
originally announced February 2021.
-
Output-Feedback Model Predictive Control with Online Identification
Authors:
Tam W. Nguyen,
Syed Aseem Ul Islam,
Dennis S. Bernstein,
Ilya V. Kolmanovsky
Abstract:
Model predictive control (MPC) is a widely used modern control technique with numerous successful application in diverse areas. Much of this success is due to the ability of MPC to enforce state and control constraints, which are crucial in many applications of control. In order to avoid the need for an observer, output-feedback model predictive control with online identification (OFMPCOI) uses th…
▽ More
Model predictive control (MPC) is a widely used modern control technique with numerous successful application in diverse areas. Much of this success is due to the ability of MPC to enforce state and control constraints, which are crucial in many applications of control. In order to avoid the need for an observer, output-feedback model predictive control with online identification (OFMPCOI) uses the block observable canonical form whose state consists of past values of the control inputs and measured outputs. Online identification is performed using recursive least squares (RLS) with variable-rate forgetting. The article describes the algorithmic details of OFMPCOI and numerically investigates its performance through a collection of numerical examples that highlight various control challenges, such as model order uncertainty, sensor noise, prediction horizon, stabilization, magnitude and move-size saturation, and stabilization. The numerical examples are used to probe the performance of OFMPCOI in terms of persistency, consistency, and exigency. Since OFMPCOI does not employ a separate control perturbation to enhance persistency, the focus is on self-generated persistency during transient operation. For closed-loop identification using RLS, sensor noise gives rise to bias in the identified model, and the goal is to determine the effect of the lack of consistency. Finally, the numerical examples reveal exigency, which is the extent to which the online identification emphasizes model characteristics that are most relevant to meeting performance objectives.
△ Less
Submitted 22 September, 2020;
originally announced September 2020.
-
Identification of Self-Excited Systems Using Discrete-Time, Time-Delayed Lur'e Models
Authors:
Juan A. Paredes,
Dennis S. Bernstein
Abstract:
This paper presents a system identification technique for systems whose output is asymptotically periodic under constant inputs. The model used for system identification is a discrete-time Lur'e model consisting of asymptotically stable linear dynamics, a time delay, a washout filter, and a static nonlinear feedback map**. For all sufficiently large scalings of the loop transfer function, these…
▽ More
This paper presents a system identification technique for systems whose output is asymptotically periodic under constant inputs. The model used for system identification is a discrete-time Lur'e model consisting of asymptotically stable linear dynamics, a time delay, a washout filter, and a static nonlinear feedback map**. For all sufficiently large scalings of the loop transfer function, these components cause divergence under small signal levels and decay under large signal amplitudes, thus producing an asymptotically oscillatory output. A bias-generation mechanism is used to provide a bias in the oscillation. The contribution of the paper is a least-squares technique that estimates the coefficients of the linear model as well as the parameterization of the continuous, piecewise-linear feedback map**.
△ Less
Submitted 29 April, 2020;
originally announced April 2020.
-
Recursive Least Squares with Variable-Direction Forgetting -- Compensating for the loss of persistency
Authors:
Ankit Goel,
Adam L. Bruce,
Dennis S. Bernstein
Abstract:
Learning depends on the ability to acquire and assimilate new information. This ability depends---somewhat counterintuitively---on the ability to forget. In particular, effective forgetting requires the ability to recognize and utilize new information to order to update a system model. This article is a tutorial on forgetting within the context of recursive least squares (RLS). To do this, RLS is…
▽ More
Learning depends on the ability to acquire and assimilate new information. This ability depends---somewhat counterintuitively---on the ability to forget. In particular, effective forgetting requires the ability to recognize and utilize new information to order to update a system model. This article is a tutorial on forgetting within the context of recursive least squares (RLS). To do this, RLS is first presented in its classical form, which employs uniform-direction forgetting. Next, examples are given to motivate the need for variable-direction forgetting, especially in cases where the excitation is not persistent. Some of these results are well known, whereas others complement the prior literature. The goal is to provide a self-contained tutorial of the main ideas and techniques for students and researchers whose research may benefit from variable-direction forgetting.
△ Less
Submitted 7 March, 2020;
originally announced March 2020.
-
A Discrete-Time, Time-Delayed Lur'e Model with Biased Self-Excited Oscillations
Authors:
Juan Paredes,
Syed Aseem Ul Islam,
Omran Kouba,
Dennis S. Bernstein
Abstract:
Self-excited systems arise in many applications, such as biochemical systems, mechanical systems with fluid-structure interaction, and fuel-driven systems with combustion dynamics. This paper presents a Lur'e model that exhibits biased self-excited oscillations under constant inputs. The model involves asymptotically stable linear dynamics, time delay, a washout filter, and a saturation nonlineari…
▽ More
Self-excited systems arise in many applications, such as biochemical systems, mechanical systems with fluid-structure interaction, and fuel-driven systems with combustion dynamics. This paper presents a Lur'e model that exhibits biased self-excited oscillations under constant inputs. The model involves asymptotically stable linear dynamics, time delay, a washout filter, and a saturation nonlinearity. For all sufficiently large scalings of the loop transfer function, these components cause divergence under small signal levels and decay under large signal amplitudes, thus producing an oscillatory response. A bias-generation mechanism is used to specify the mean of the oscillation. The main contribution of the paper is a detailed analysis of a discrete-time version of this model.
△ Less
Submitted 6 March, 2020;
originally announced March 2020.
-
Convergence and Consistency of Recursive Least Squares with Variable-Rate Forgetting
Authors:
Adam L. Bruce,
Ankit Goel,
Dennis S. Bernstein
Abstract:
A recursive least squares algorithm with variable rate forgetting (VRF) is derived by minimizing a quadratic cost function.Under persistent excitation and boundedness of the forgetting factor, the minimizer given by VRF is shown to converge to the true parameters. In addition, under persistent excitation and with noisy measurements, where the noise is uncorrelated with the regressor, conditions ar…
▽ More
A recursive least squares algorithm with variable rate forgetting (VRF) is derived by minimizing a quadratic cost function.Under persistent excitation and boundedness of the forgetting factor, the minimizer given by VRF is shown to converge to the true parameters. In addition, under persistent excitation and with noisy measurements, where the noise is uncorrelated with the regressor, conditions are given under which the minimizer given by VRF is a consistent estimator of the true parameters.The results are illustrated by a numerical example involving abruptly changing parameters.
△ Less
Submitted 5 March, 2020;
originally announced March 2020.
-
Counting Colorful Necklaces and Bracelets in Three Colors
Authors:
Dennis S. Bernstein,
Omran Kouba
Abstract:
A necklace or bracelet is \textit{colorful} if no pair of adjacent beads are the same color. In addition, two necklaces are \textit{equivalent} if one results from the other by permuting its colors, and two bracelets are \textit{equivalent} if one results from the other by either permuting its colors or reversing the order of the beads; a bracelet is thus a necklace that can be turned over. This n…
▽ More
A necklace or bracelet is \textit{colorful} if no pair of adjacent beads are the same color. In addition, two necklaces are \textit{equivalent} if one results from the other by permuting its colors, and two bracelets are \textit{equivalent} if one results from the other by either permuting its colors or reversing the order of the beads; a bracelet is thus a necklace that can be turned over. This note counts the number $K(n)$ of non-equivalent colorful necklaces and the number $K'(n)$ of colorful bracelets formed with $n$-beads in at most three colors. Expressions obtained for $K'(n)$ simplify expressions given by OEIS sequence A114438, while the expressions given for $K(n)$ appear to be new and are not included in OEIS.
△ Less
Submitted 5 March, 2019; v1 submitted 30 January, 2019;
originally announced January 2019.
-
Policy Iteration for Decentralized Control of Markov Decision Processes
Authors:
Daniel S. Bernstein,
Christopher Amato,
Eric A. Hansen,
Shlomo Zilberstein
Abstract:
Coordination of distributed agents is required for problems arising in many areas, including multi-robot systems, networking and e-commerce. As a formal framework for such problems, we use the decentralized partially observable Markov decision process (DEC-POMDP). Though much work has been done on optimal dynamic programming algorithms for the single-agent version of the problem, optimal algorit…
▽ More
Coordination of distributed agents is required for problems arising in many areas, including multi-robot systems, networking and e-commerce. As a formal framework for such problems, we use the decentralized partially observable Markov decision process (DEC-POMDP). Though much work has been done on optimal dynamic programming algorithms for the single-agent version of the problem, optimal algorithms for the multiagent case have been elusive. The main contribution of this paper is an optimal policy iteration algorithm for solving DEC-POMDPs. The algorithm uses stochastic finite-state controllers to represent policies. The solution can include a correlation device, which allows agents to correlate their actions without communicating. This approach alternates between expanding the controller and performing value-preserving transformations, which modify the controller without sacrificing value. We present two efficient value-preserving transformations: one can reduce the size of the controller and the other can improve its value while kee** the size fixed. Empirical results demonstrate the usefulness of value-preserving transformations in increasing value while kee** controller size to a minimum. To broaden the applicability of the approach, we also present a heuristic version of the policy iteration algorithm, which sacrifices convergence to optimality. This algorithm further reduces the size of the controllers at each step by assuming that probability distributions over the other agents actions are known. While this assumption may not hold in general, it helps produce higher quality solutions in our test problems.
△ Less
Submitted 15 January, 2014;
originally announced January 2014.
-
The Complexity of Decentralized Control of Markov Decision Processes
Authors:
Daniel S Bernstein,
Shlomo Zilberstein,
Neil Immerman
Abstract:
Planning for distributed agents with partial state information is considered from a decision- theoretic perspective. We describe generalizations of both the MDP and POMDP models that allow for decentralized control. For even a small number of agents, the finite-horizon problems corresponding to both of our models are complete for nondeterministic exponential time. These complexity results illus…
▽ More
Planning for distributed agents with partial state information is considered from a decision- theoretic perspective. We describe generalizations of both the MDP and POMDP models that allow for decentralized control. For even a small number of agents, the finite-horizon problems corresponding to both of our models are complete for nondeterministic exponential time. These complexity results illustrate a fundamental difference between centralized and decentralized control of Markov processes. In contrast to the MDP and POMDP problems, the problems we consider provably do not admit polynomial-time algorithms and most likely require doubly exponential time to solve in the worst case. We have thus provided mathematical evidence corresponding to the intuition that decentralized planning problems cannot easily be reduced to centralized problems and solved exactly using established techniques.
△ Less
Submitted 16 January, 2013;
originally announced January 2013.
-
Optimizing Memory-Bounded Controllers for Decentralized POMDPs
Authors:
Christopher Amato,
Daniel S Bernstein,
Shlomo Zilberstein
Abstract:
We present a memory-bounded optimization approach for solving infinite-horizon decentralized POMDPs. Policies for each agent are represented by stochastic finite state controllers. We formulate the problem of optimizing these policies as a nonlinear program, leveraging powerful existing nonlinear optimization techniques for solving the problem. While existing solvers only guarantee locally optimal…
▽ More
We present a memory-bounded optimization approach for solving infinite-horizon decentralized POMDPs. Policies for each agent are represented by stochastic finite state controllers. We formulate the problem of optimizing these policies as a nonlinear program, leveraging powerful existing nonlinear optimization techniques for solving the problem. While existing solvers only guarantee locally optimal solutions, we show that our formulation produces higher quality controllers than the state-of-the-art approach. We also incorporate a shared source of randomness in the form of a correlation device to further increase solution quality with only a limited increase in space and time. Our experimental results show that nonlinear optimization can be used to provide high quality, concise solutions to decentralized decision problems under uncertainty.
△ Less
Submitted 20 June, 2012;
originally announced June 2012.