Showing 201–211 of 211 results for author: Lygeros, J

Stochastic receding horizon control with output feedback and bounded control inputs

Authors: Peter Hokayem, Eugenio Cinquemani, Debasish Chatterjee, Federico Ramponi, John Lygeros

Abstract: We provide a solution to the problem of receding horizon control for stochastic discrete-time systems with bounded control inputs and imperfect state measurements. For a suitable choice of control policies, we show that the finite-horizon optimization problem to be solved on-line is convex and successively feasible. Due to the inherent nonlinearity of the feedback loop, a slight extension of the… ▽ More We provide a solution to the problem of receding horizon control for stochastic discrete-time systems with bounded control inputs and imperfect state measurements. For a suitable choice of control policies, we show that the finite-horizon optimization problem to be solved on-line is convex and successively feasible. Due to the inherent nonlinearity of the feedback loop, a slight extension of the Kalman filter is exploited to estimate the state optimally in mean-square sense. We show that the receding horizon implementation of the resulting control policies renders the state of the overall system mean-square bounded under mild assumptions. Finally, we discuss how some of the quantities required by the finite-horizon optimization problem can be computed off-line, reducing the on-line computation, and present some numerical examples. △ Less

Submitted 14 April, 2010; v1 submitted 18 January, 2010; originally announced January 2010.

Comments: 25 pages, 4 figures

Hamilton-Jacobi formulation for reach-avoid differential games

Authors: Kostas Margellos, John Lygeros

Abstract: A new framework for formulating reachability problems with competing inputs, nonlinear dynamics and state constraints as optimal control problems is developed. Such reach-avoid problems arise in, among others, the study of safety problems in hybrid systems. Earlier approaches to reach-avoid computations are either restricted to linear systems, or face numerical difficulties due to possible disco… ▽ More A new framework for formulating reachability problems with competing inputs, nonlinear dynamics and state constraints as optimal control problems is developed. Such reach-avoid problems arise in, among others, the study of safety problems in hybrid systems. Earlier approaches to reach-avoid computations are either restricted to linear systems, or face numerical difficulties due to possible discontinuities in the Hamiltonian of the optimal control problem. The main advantage of the approach proposed in this paper is that it can be applied to a general class of target hitting continuous dynamic games with nonlinear dynamics, and has very good properties in terms of its numerical solution, since the value function and the Hamiltonian of the system are both continuous. The performance of the proposed method is demonstrated by applying it to a two aircraft collision avoidance scenario under target window constraints and in the presence of wind disturbance. Target Windows are a novel concept in air traffic management, and represent spatial and temporal constraints, that the aircraft have to respect to meet their schedule. △ Less

Submitted 24 November, 2009; originally announced November 2009.

Comments: 12 figures, 21 pages

On the connections between PCTL and Dynamic Programming

Authors: Federico Ramponi, Debasish Chatterjee, Sean Summers, John Lygeros

Abstract: Probabilistic Computation Tree Logic (PCTL) is a well-known modal logic which has become a standard for expressing temporal properties of finite-state Markov chains in the context of automated model checking. In this paper, we give a definition of PCTL for noncountable-space Markov chains, and we show that there is a substantial affinity between certain of its operators and problems of Dynamic P… ▽ More Probabilistic Computation Tree Logic (PCTL) is a well-known modal logic which has become a standard for expressing temporal properties of finite-state Markov chains in the context of automated model checking. In this paper, we give a definition of PCTL for noncountable-space Markov chains, and we show that there is a substantial affinity between certain of its operators and problems of Dynamic Programming. After proving some uniqueness properties of the solutions to the latter, we conclude the paper with two examples to show that some recovery strategies in practical applications, which are naturally stated as reach-avoid problems, can be actually viewed as particular cases of PCTL formulas. △ Less

Submitted 25 October, 2009; originally announced October 2009.

Comments: Submitted

MSC Class: 60J10

Journal ref: HSCC Stockholm, 2010, pages 253-262

Attaining mean square boundedness of a marginally stable noisy linear system with a bounded control input

Authors: Federico Ramponi, Debasish Chatterjee, Andreas Milias-Argeitis, Peter Hokayem, John Lygeros

Abstract: We construct control policies that ensure bounded variance of a noisy marginally stable linear system in closed-loop. It is assumed that the noise sequence is a mutually independent sequence of random vectors, enters the dynamics affinely, and has bounded fourth moment. The magnitude of the control is required to be of the order of the first moment of the noise, and the policies we obtain are si… ▽ More We construct control policies that ensure bounded variance of a noisy marginally stable linear system in closed-loop. It is assumed that the noise sequence is a mutually independent sequence of random vectors, enters the dynamics affinely, and has bounded fourth moment. The magnitude of the control is required to be of the order of the first moment of the noise, and the policies we obtain are simple and computable. △ Less

Submitted 26 January, 2010; v1 submitted 9 July, 2009; originally announced July 2009.

Comments: 10 pages

MSC Class: 93E15

Journal ref: IEEE Transactions on Automatic Control, Vol 55, No. 10, pp. 2414-2418, 2010

Stochastic optimization on continuous domains with finite-time guarantees by Markov chain Monte Carlo methods

Authors: A. Lecchini-Visintini, J. Lygeros, J. Maciejowski

Abstract: We introduce bounds on the finite-time performance of Markov chain Monte Carlo algorithms in approaching the global solution of stochastic optimization problems over continuous domains. A comparison with other state-of-the-art methods having finite-time guarantees for solving stochastic programming problems is included. We introduce bounds on the finite-time performance of Markov chain Monte Carlo algorithms in approaching the global solution of stochastic optimization problems over continuous domains. A comparison with other state-of-the-art methods having finite-time guarantees for solving stochastic programming problems is included. △ Less

Submitted 7 January, 2010; v1 submitted 5 June, 2009; originally announced June 2009.

Comments: 29 pages, 6 figures. Revised version based on referees report

Journal ref: IEEE Transactions on Automatic Control, 55(12):2858-2863, 2010

On convex problems in chance-constrained stochastic model predictive control

Authors: Eugenio Cinquemani, Mayank Agarwal, Debasish Chatterjee, John Lygeros

Abstract: We investigate constrained optimal control problems for linear stochastic dynamical systems evolving in discrete time. We consider minimization of an expected value cost over a finite horizon. Hard constraints are introduced first, and then reformulated in terms of probabilistic constraints. It is shown that, for a suitable parametrization of the control policy, a wide class of the resulting opt… ▽ More We investigate constrained optimal control problems for linear stochastic dynamical systems evolving in discrete time. We consider minimization of an expected value cost over a finite horizon. Hard constraints are introduced first, and then reformulated in terms of probabilistic constraints. It is shown that, for a suitable parametrization of the control policy, a wide class of the resulting optimization problems are convex, or admit reasonable convex approximations. △ Less

Submitted 21 May, 2009; originally announced May 2009.

MSC Class: 93C05; 93E03

Maximizing the probability of attaining a target prior to extinction

Authors: Debasish Chatterjee, Eugenio Cinquemani, John Lygeros

Abstract: We present a dynamic programming-based solution to the problem of maximizing the probability of attaining a target set before hitting a cemetery set for a discrete-time Markov control process. Under mild hypotheses we establish that there exists a deterministic stationary policy that achieves the maximum value of this probability. We demonstrate how the maximization of this probability can be co… ▽ More We present a dynamic programming-based solution to the problem of maximizing the probability of attaining a target set before hitting a cemetery set for a discrete-time Markov control process. Under mild hypotheses we establish that there exists a deterministic stationary policy that achieves the maximum value of this probability. We demonstrate how the maximization of this probability can be computed through the maximization of an expected total reward until the first hitting time to either the target or the cemetery set. Martingale characterizations of thrifty, equalizing, and optimal policies in the context of our problem are also established. △ Less

Submitted 27 November, 2009; v1 submitted 27 April, 2009; originally announced April 2009.

Comments: 22 pages, 1 figure. Revised

MSC Class: 90C39; 90C40 (Primary); 93E20 (Secondary)

Stochastic model predictive control with bounded control inputs: a vector space approach

Authors: Debasish Chatterjee, Peter Hokayem, John Lygeros

Abstract: We design receding horizon control strategies for stochastic discrete-time linear systems with additive (possibly) unbounded disturbances, while obeying hard bounds on the control inputs. We pose the problem of selecting an appropriate optimal controller on vector spaces of functions and show that the resulting optimization problem has a tractable convex solution. Under the assumption that the z… ▽ More We design receding horizon control strategies for stochastic discrete-time linear systems with additive (possibly) unbounded disturbances, while obeying hard bounds on the control inputs. We pose the problem of selecting an appropriate optimal controller on vector spaces of functions and show that the resulting optimization problem has a tractable convex solution. Under the assumption that the zero-input and zero-noise system is asymptotically stable, we show that the variance of the state is bounded when enforcing hard bounds on the control inputs, for any receding horizon implementation. Throughout the article we provide several examples that illustrate how quantities needed in the formulation of the resulting optimization problems can be calculated off-line, as well as comparative examples that illustrate the effectiveness of our control strategies. △ Less

Submitted 15 October, 2009; v1 submitted 31 March, 2009; originally announced March 2009.

Comments: Revised. 28 pages, 5 figures

MSC Class: 93E20; 93C55 (Primary) 90C15 (Secondary)

On Stochastic Model Predictive Control with Bounded Control Inputs

Authors: Peter Hokayem, Debasish Chatterjee, John Lygeros

Abstract: This paper is concerned with the problem of Model Predictive Control and Rolling Horizon Control of discrete-time systems subject to possibly unbounded random noise inputs, while satisfying hard bounds on the control inputs. We use a nonlinear feedback policy with respect to noise measurements and show that the resulting mathematical program has a tractable convex solution in both cases. Moreove… ▽ More This paper is concerned with the problem of Model Predictive Control and Rolling Horizon Control of discrete-time systems subject to possibly unbounded random noise inputs, while satisfying hard bounds on the control inputs. We use a nonlinear feedback policy with respect to noise measurements and show that the resulting mathematical program has a tractable convex solution in both cases. Moreover, under the assumption that the zero-input and zero-noise system is asymptotically stable, we show that the variance of the state, under the resulting Model Predictive Control and Rolling Horizon Control policies, is bounded. Finally, we provide some numerical examples on how certain matrices in the underlying mathematical program can be calculated off-line. △ Less

Submitted 23 February, 2009; originally announced February 2009.

Comments: 8 pages

MSC Class: 93E25; 65K05

Journal ref: Proceedings of the 48th IEEE Conference on Decision & Control, 2009, pages 6359 - 6364

Stochastic control up to a hitting time: optimality and rolling-horizon implementation

Authors: Debasish Chatterjee, Eugenio Cinquemani, Giorgos Chaloulos, John Lygeros

Abstract: We present a dynamic programming-based solution to a stochastic optimal control problem up to a hitting time for a discrete-time Markov control process. Firstly, we determine an optimal control policy to steer the process toward a compact target set while simultaneously minimizing an expected discounted cost. We then provide a rolling-horizon strategy for approximating the optimal policy, togeth… ▽ More We present a dynamic programming-based solution to a stochastic optimal control problem up to a hitting time for a discrete-time Markov control process. Firstly, we determine an optimal control policy to steer the process toward a compact target set while simultaneously minimizing an expected discounted cost. We then provide a rolling-horizon strategy for approximating the optimal policy, together with quantitative characterization of its sub-optimality with respect to the optimal policy. Finally, we address related issues of asymptotic discount-optimality of the value-iteration policy. Both the state and action spaces are assumed to be Polish. △ Less

Submitted 28 September, 2009; v1 submitted 18 June, 2008; originally announced June 2008.

Comments: 20 pages, 4 figures

MSC Class: 90C39; 90C40; 93E20

Simulated Annealing: Rigorous finite-time guarantees for optimization on continuous domains

Authors: A. Lecchini-Visintini, J. Lygeros, J. Maciejowski

Abstract: Simulated annealing is a popular method for approaching the solution of a global optimization problem. Existing results on its performance apply to discrete combinatorial optimization where the optimization variables can assume only a finite set of possible values. We introduce a new general formulation of simulated annealing which allows one to guarantee finite-time performance in the optimizat… ▽ More Simulated annealing is a popular method for approaching the solution of a global optimization problem. Existing results on its performance apply to discrete combinatorial optimization where the optimization variables can assume only a finite set of possible values. We introduce a new general formulation of simulated annealing which allows one to guarantee finite-time performance in the optimization of functions of continuous variables. The results hold universally for any optimization problem on a bounded domain and establish a connection between simulated annealing and up-to-date theory of convergence of Markov chain Monte Carlo methods on continuous domains. This work is inspired by the concept of finite-time learning with known accuracy and confidence developed in statistical learning theory. △ Less

Submitted 19 September, 2007; originally announced September 2007.

Comments: 10 pages, 2 figures. Preprint. The final version will appear in: Advances in Neural Information Processing Systems 20, Proceedings of NIPS 2007, MIT Press