-
Quantum speedups for convex dynamic programming
Authors:
David Sutter,
Giacomo Nannicini,
Tobias Sutter,
Stefan Woerner
Abstract:
We present a quantum algorithm to solve dynamic programming problems with convex value functions. For linear discrete-time systems with a $d$-dimensional state space of size $N$, the proposed algorithm outputs a quantum-mechanical representation of the value function in time $O(T γ^{dT}\mathrm{polylog}(N,(T/\varepsilon)^{d}))$, where $\varepsilon$ is the accuracy of the solution, $T$ is the time h…
▽ More
We present a quantum algorithm to solve dynamic programming problems with convex value functions. For linear discrete-time systems with a $d$-dimensional state space of size $N$, the proposed algorithm outputs a quantum-mechanical representation of the value function in time $O(T γ^{dT}\mathrm{polylog}(N,(T/\varepsilon)^{d}))$, where $\varepsilon$ is the accuracy of the solution, $T$ is the time horizon, and $γ$ is a problem-specific parameter depending on the condition numbers of the cost functions. This allows us to evaluate the value function at any fixed state in time $O(T γ^{dT}\sqrt{N}\,\mathrm{polylog}(N,(T/\varepsilon)^{d}))$, and the corresponding optimal action can be recovered by solving a convex program. The class of optimization problems to which our algorithm can be applied includes provably hard stochastic dynamic programs. Finally, we show that the algorithm obtains a quadratic speedup (up to polylogarithmic factors) compared to the classical Bellman approach on some dynamic programs with continuous state space that have $γ=1$.
△ Less
Submitted 17 March, 2021; v1 submitted 23 November, 2020;
originally announced November 2020.
-
Quantitative lower bounds on the Lyapunov exponent from multivariate matrix inequalities
Authors:
Marius Lemm,
David Sutter
Abstract:
The Lyapunov exponent characterizes the asymptotic behavior of long matrix products. Recognizing scenarios where the Lyapunov exponent is strictly positive is a fundamental challenge that is relevant in many applications. In this work we establish a novel tool for this task by deriving a quantitative lower bound on the Lyapunov exponent in terms of a matrix sum which is efficiently computable in e…
▽ More
The Lyapunov exponent characterizes the asymptotic behavior of long matrix products. Recognizing scenarios where the Lyapunov exponent is strictly positive is a fundamental challenge that is relevant in many applications. In this work we establish a novel tool for this task by deriving a quantitative lower bound on the Lyapunov exponent in terms of a matrix sum which is efficiently computable in ergodic situations. Our approach combines two deep results from matrix analysis --- the $n$-matrix extension of the Golden-Thompson inequality and the Avalanche-Principle. We apply these bounds to the Lyapunov exponents of Schrödinger cocycles with certain ergodic potentials of polymer type and arbitrary correlation structure. We also derive related quantitative stability results for the Lyapunov exponent near aligned diagonal matrices and a bound for almost-commuting matrices.
△ Less
Submitted 24 January, 2020;
originally announced January 2020.
-
Bounds on Lyapunov exponents via entropy accumulation
Authors:
David Sutter,
Omar Fawzi,
Renato Renner
Abstract:
Lyapunov exponents describe the asymptotic behavior of the singular values of large products of random matrices. A direct computation of these exponents is however often infeasible. By establishing a link between Lyapunov exponents and an information theoretic tool called entropy accumulation theorem we derive an upper and a lower bound for the maximal and minimal Lyapunov exponent, respectively.…
▽ More
Lyapunov exponents describe the asymptotic behavior of the singular values of large products of random matrices. A direct computation of these exponents is however often infeasible. By establishing a link between Lyapunov exponents and an information theoretic tool called entropy accumulation theorem we derive an upper and a lower bound for the maximal and minimal Lyapunov exponent, respectively. The bounds assume independence of the random matrices, are analytical, and are tight in the commutative case as well as in other scenarios. They can be expressed in terms of an optimization problem that only involves single matrices rather than large products. The upper bound for the maximal Lyapunov exponent can be evaluated efficiently via the theory of convex optimization.
△ Less
Submitted 22 September, 2020; v1 submitted 8 May, 2019;
originally announced May 2019.
-
Generalized maximum entropy estimation
Authors:
Tobias Sutter,
David Sutter,
Peyman Mohajerin Esfahani,
John Lygeros
Abstract:
We consider the problem of estimating a probability distribution that maximizes the entropy while satisfying a finite number of moment constraints, possibly corrupted by noise. Based on duality of convex programming, we present a novel approximation scheme using a smoothed fast gradient method that is equipped with explicit bounds on the approximation error. We further demonstrate how the presente…
▽ More
We consider the problem of estimating a probability distribution that maximizes the entropy while satisfying a finite number of moment constraints, possibly corrupted by noise. Based on duality of convex programming, we present a novel approximation scheme using a smoothed fast gradient method that is equipped with explicit bounds on the approximation error. We further demonstrate how the presented scheme can be used for approximating the chemical master equation through the zero-information moment closure method, and for an approximate dynamic programming approach in the context of constrained Markov decision processes with uncountable state and action spaces.
△ Less
Submitted 8 September, 2019; v1 submitted 24 August, 2017;
originally announced August 2017.
-
Capacity of Random Channels with Large Alphabets
Authors:
Tobias Sutter,
David Sutter,
John Lygeros
Abstract:
We consider discrete memoryless channels with input alphabet size $n$ and output alphabet size $m$, where $m=$ceil$(γn)$ for some constant $γ>0$. The channel transition matrix consists of entries that, before being normalised, are independent and identically distributed nonnegative random variables $V$ and such that $E[(V \log V)^2]<\infty$. We prove that in the limit as $n\to \infty$ the capacity…
▽ More
We consider discrete memoryless channels with input alphabet size $n$ and output alphabet size $m$, where $m=$ceil$(γn)$ for some constant $γ>0$. The channel transition matrix consists of entries that, before being normalised, are independent and identically distributed nonnegative random variables $V$ and such that $E[(V \log V)^2]<\infty$. We prove that in the limit as $n\to \infty$ the capacity of such a channel converges to $Ent(V) / E[V]$ almost surely and in $L^2$, where $Ent(V):= E[V\log V]-E[V] \log E[V]$ denotes the entropy of $V$. We further show that, under slightly different model assumptions, the capacity of these random channels converges to this asymptotic value exponentially in $n$. Finally, we present an application in the context of Bayesian optimal experiment design.
△ Less
Submitted 31 March, 2016; v1 submitted 13 March, 2015;
originally announced March 2015.
-
Efficient Approximation of Channel Capacities
Authors:
Tobias Sutter,
David Sutter,
Peyman Mohajerin Esfahani,
John Lygeros
Abstract:
We propose an iterative method for approximately computing the capacity of discrete memoryless channels, possibly under additional constraints on the input distribution. Based on duality of convex programming, we derive explicit upper and lower bounds for the capacity. The presented method requires $O(M^2 N \sqrt{\log N}/\varepsilon)$ to provide an estimate of the capacity to within $\varepsilon$,…
▽ More
We propose an iterative method for approximately computing the capacity of discrete memoryless channels, possibly under additional constraints on the input distribution. Based on duality of convex programming, we derive explicit upper and lower bounds for the capacity. The presented method requires $O(M^2 N \sqrt{\log N}/\varepsilon)$ to provide an estimate of the capacity to within $\varepsilon$, where $N$ and $M$ denote the input and output alphabet size; a single iteration has a complexity $O(M N)$. We also show how to approximately compute the capacity of memoryless channels having a bounded continuous input alphabet and a countable output alphabet under some mild assumptions on the decay rate of the channel's tail. It is shown that discrete-time Poisson channels fall into this problem class. As an example, we compute sharp upper and lower bounds for the capacity of a discrete-time Poisson channel with a peak-power input constraint.
△ Less
Submitted 3 April, 2015; v1 submitted 29 July, 2014;
originally announced July 2014.
