-
Offline Estimation of Controlled Markov Chains: Minimaxity and Sample Complexity
Authors:
Imon Banerjee,
Harsha Honnappa,
Vinayak Rao
Abstract:
In this work, we study a natural nonparametric estimator of the transition probability matrices of a finite controlled Markov chain. We consider an offline setting with a fixed dataset, collected using a so-called logging policy. We develop sample complexity bounds for the estimator and establish conditions for minimaxity. Our statistical bounds depend on the logging policy through its mixing prop…
▽ More
In this work, we study a natural nonparametric estimator of the transition probability matrices of a finite controlled Markov chain. We consider an offline setting with a fixed dataset, collected using a so-called logging policy. We develop sample complexity bounds for the estimator and establish conditions for minimaxity. Our statistical bounds depend on the logging policy through its mixing properties. We show that achieving a particular statistical risk bound involves a subtle and interesting trade-off between the strength of the mixing properties and the number of samples. We demonstrate the validity of our results under various examples, such as ergodic Markov chains, weakly ergodic inhomogeneous Markov chains, and controlled Markov chains with non-stationary Markov, episodic, and greedy controls. Lastly, we use these sample complexity bounds to establish concomitant ones for offline evaluation of stationary Markov control policies.
△ Less
Submitted 26 January, 2024; v1 submitted 13 November, 2022;
originally announced November 2022.
-
Distributed Sparse Regression via Penalization
Authors:
Yao Ji,
Gesualdo Scutari,
Ying Sun,
Harsha Honnappa
Abstract:
We study sparse linear regression over a network of agents, modeled as an undirected graph (with no centralized node). The estimation problem is formulated as the minimization of the sum of the local LASSO loss functions plus a quadratic penalty of the consensus constraint -- the latter being instrumental to obtain distributed solution methods. While penalty-based consensus methods have been exten…
▽ More
We study sparse linear regression over a network of agents, modeled as an undirected graph (with no centralized node). The estimation problem is formulated as the minimization of the sum of the local LASSO loss functions plus a quadratic penalty of the consensus constraint -- the latter being instrumental to obtain distributed solution methods. While penalty-based consensus methods have been extensively studied in the optimization literature, their statistical and computational guarantees in the high dimensional setting remain unclear. This work provides an answer to this open problem. Our contribution is two-fold. First, we establish statistical consistency of the estimator: under a suitable choice of the penalty parameter, the optimal solution of the penalized problem achieves near optimal minimax rate $\mathcal{O}(s \log d/N)$ in $\ell_2$-loss, where $s$ is the sparsity value, $d$ is the ambient dimension, and $N$ is the total sample size in the network -- this matches centralized sample rates. Second, we show that the proximal-gradient algorithm applied to the penalized problem, which naturally leads to distributed implementations, converges linearly up to a tolerance of the order of the centralized statistical error -- the rate scales as $\mathcal{O}(d)$, revealing an unavoidable speed-accuracy dilemma.Numerical results demonstrate the tightness of the derived sample rate and convergence rate scalings.
△ Less
Submitted 21 June, 2023; v1 submitted 11 November, 2021;
originally announced November 2021.
-
Bayesian Joint Chance Constrained Optimization: Approximations and Statistical Consistency
Authors:
Prateek Jaiswal,
Harsha Honnappa,
Vinayak A. Rao
Abstract:
This paper considers data-driven chance-constrained stochastic optimization problems in a Bayesian framework. Bayesian posteriors afford a principled mechanism to incorporate data and prior knowledge into stochastic optimization problems. However, the computation of Bayesian posteriors is typically an intractable problem, and has spawned a large literature on approximate Bayesian computation. Here…
▽ More
This paper considers data-driven chance-constrained stochastic optimization problems in a Bayesian framework. Bayesian posteriors afford a principled mechanism to incorporate data and prior knowledge into stochastic optimization problems. However, the computation of Bayesian posteriors is typically an intractable problem, and has spawned a large literature on approximate Bayesian computation. Here, in the context of chance-constrained optimization, we focus on the question of statistical consistency (in an appropriate sense) of the optimal value, computed using an approximate posterior distribution. To this end, we rigorously prove a frequentist consistency result demonstrating the convergence of the optimal value to the optimal value of a fixed, parameterized constrained optimization problem. We augment this by also establishing a probabilistic rate of convergence of the optimal value. We also prove the convex feasibility of the approximate Bayesian stochastic optimization problem. Finally, we demonstrate the utility of our approach on an optimal staffing problem for an M/M/c queueing model.
△ Less
Submitted 30 September, 2022; v1 submitted 23 June, 2021;
originally announced June 2021.
-
Estimating Stochastic Poisson Intensities Using Deep Latent Models
Authors:
Ruixin Wang,
Prateek Jaiwal,
Harsha Honnappa
Abstract:
We present methodology for estimating the stochastic intensity of a doubly stochastic Poisson process. Statistical and theoretical analyses of traffic traces show that these processes are appropriate models of high intensity traffic arriving at an array of service systems. The statistical estimation of the underlying latent stochastic intensity process driving the traffic model involves a rather c…
▽ More
We present methodology for estimating the stochastic intensity of a doubly stochastic Poisson process. Statistical and theoretical analyses of traffic traces show that these processes are appropriate models of high intensity traffic arriving at an array of service systems. The statistical estimation of the underlying latent stochastic intensity process driving the traffic model involves a rather complicated nonlinear filtering problem. We develop a novel simulation methodology, using deep neural networks to approximate the path measures induced by the stochastic intensity process, for solving this nonlinear filtering problem. Our simulation studies demonstrate that the method is quite accurate on both in-sample estimation and on an out-of-sample performance prediction task for an infinite server queue.
△ Less
Submitted 22 July, 2020; v1 submitted 12 July, 2020;
originally announced July 2020.
-
Variational Bayesian Methods for Stochastically Constrained System Design Problems
Authors:
Prateek Jaiswal,
Harsha Honnappa,
Vinayak A. Rao
Abstract:
We study system design problems stated as parameterized stochastic programs with a chance-constraint set. We adopt a Bayesian approach that requires the computation of a posterior predictive integral which is usually intractable. In addition, for the problem to be a well-defined convex program, we must retain the convexity of the feasible set. Consequently, we propose a variational Bayes-based met…
▽ More
We study system design problems stated as parameterized stochastic programs with a chance-constraint set. We adopt a Bayesian approach that requires the computation of a posterior predictive integral which is usually intractable. In addition, for the problem to be a well-defined convex program, we must retain the convexity of the feasible set. Consequently, we propose a variational Bayes-based method to approximately compute the posterior predictive integral that ensures tractability and retains the convexity of the feasible set. Under certain regularity conditions, we also show that the solution set obtained using variational Bayes converges to the true solution set as the number of observations tends to infinity. We also provide bounds on the probability of qualifying a true infeasible point (with respect to the true constraints) as feasible under the VB approximation for a given number of samples.
△ Less
Submitted 6 January, 2020;
originally announced January 2020.
-
Asymptotic Consistency of Loss-Calibrated Variational Bayes
Authors:
Prateek Jaiswal,
Harsha Honnappa,
Vinayak A. Rao
Abstract:
This paper establishes the asymptotic consistency of the {\it loss-calibrated variational Bayes} (LCVB) method. LCVB was proposed in~\cite{LaSiGh2011} as a method for approximately computing Bayesian posteriors in a `loss aware' manner. This methodology is also highly relevant in general data-driven decision-making contexts. Here, we not only establish the asymptotic consistency of the calibrated…
▽ More
This paper establishes the asymptotic consistency of the {\it loss-calibrated variational Bayes} (LCVB) method. LCVB was proposed in~\cite{LaSiGh2011} as a method for approximately computing Bayesian posteriors in a `loss aware' manner. This methodology is also highly relevant in general data-driven decision-making contexts. Here, we not only establish the asymptotic consistency of the calibrated approximate posterior, but also the asymptotic consistency of decision rules. We also establish the asymptotic consistency of decision rules obtained from a `naive' variational Bayesian procedure.
△ Less
Submitted 4 November, 2019;
originally announced November 2019.
-
Asymptotic Consistency of $α-$Rényi-Approximate Posteriors
Authors:
Prateek Jaiswal,
Vinayak A. Rao,
Harsha Honnappa
Abstract:
We study the asymptotic consistency properties of $α$-Rényi approximate posteriors, a class of variational Bayesian methods that approximate an intractable Bayesian posterior with a member of a tractable family of distributions, the member chosen to minimize the $α$-Rényi divergence from the true posterior. Unique to our work is that we consider settings with $α> 1$, resulting in approximations th…
▽ More
We study the asymptotic consistency properties of $α$-Rényi approximate posteriors, a class of variational Bayesian methods that approximate an intractable Bayesian posterior with a member of a tractable family of distributions, the member chosen to minimize the $α$-Rényi divergence from the true posterior. Unique to our work is that we consider settings with $α> 1$, resulting in approximations that upperbound the log-likelihood, and consequently have wider spread than traditional variational approaches that minimize the Kullback-Liebler (KL) divergence from the posterior. Our primary result identifies sufficient conditions under which consistency holds, centering around the existence of a 'good' sequence of distributions in the approximating family that possesses, among other properties, the right rate of convergence to a limit distribution. We further characterize the good sequence by demonstrating that a sequence of distributions that converges too quickly cannot be a good sequence. We also extend our analysis to the setting where $α$ equals one, corresponding to the minimizer of the reverse KL divergence, and to models with local latent variables. We also illustrate the existence of good sequence with a number of examples. Our results complement a growing body of work focused on the frequentist properties of variational Bayesian methods.
△ Less
Submitted 14 August, 2020; v1 submitted 5 February, 2019;
originally announced February 2019.
-
On Transitory Queueing
Authors:
Harsha Honnappa,
Rahul Jain,
Amy R. Ward
Abstract:
We introduce a framework and develop a theory of transitory queueing models. These are models that are not only non-stationary and time-varying but also have other features such as the queueing system operates over finite time, or only a finite population arrives. Such models are relevant in many real-world settings, from queues at post-offces, DMV, concert halls and stadia to out-patient departme…
▽ More
We introduce a framework and develop a theory of transitory queueing models. These are models that are not only non-stationary and time-varying but also have other features such as the queueing system operates over finite time, or only a finite population arrives. Such models are relevant in many real-world settings, from queues at post-offces, DMV, concert halls and stadia to out-patient departments at hospitals. We develop fluid and diffusion limits for a large class of transitory queueing models. We then introduce three specific models that fit within this framework, namely, the Delta(i)/GI/1 model, the conditioned G/GI/1 model, and an arrival model of scheduled traffic with epoch uncertainty. We show that asymptotically these models are distributionally equivalent, i.e., they have the same fluid and diffusion limits. We note that our framework provides the first ever way of analyzing the standard G/GI/1 model when we condition on the number of arrivals. In obtaining these results, we provide generalizations and extensions of the Glivenko-Cantelli and Donskers Theorem for empirical processes with triangular arrays. Our analysis uses the population acceleration technique that we introduce and develop. This may be useful in analysis of other non-stationary and non-ergodic queuing models.
△ Less
Submitted 7 December, 2014;
originally announced December 2014.
-
Strategic Arrivals into Queueing Networks: The Network Concert Queueing Game
Authors:
Harsha Honnappa,
Rahul Jain
Abstract:
Queueing networks are typically modelled assuming that the arrival process is exogenous, and unaffected by admission control, scheduling policies, etc. In many situations, however, users choose the time of their arrival strategically, taking delay and other metrics into account. In this paper, we develop a framework to study such strategic arrivals into queueing networks. We start by deriving a fu…
▽ More
Queueing networks are typically modelled assuming that the arrival process is exogenous, and unaffected by admission control, scheduling policies, etc. In many situations, however, users choose the time of their arrival strategically, taking delay and other metrics into account. In this paper, we develop a framework to study such strategic arrivals into queueing networks. We start by deriving a functional strong law of large numbers (FSLLN) approximation to the queueing network. In the fluid limit derived, we then study the population game wherein users strategically choose when to arrive, and upon arrival which of the K queues to join. The queues start service at given times, which can potentially be different. We characterize the (strategic) arrival process at each of the queues, and the price of anarchy of the ensuing strategic arrival game. We then extend the analysis to multiple populations of users, each with a different cost metric. The equilibrium arrival profile and price of anarchy are derived. Finally, we present the methodology for exact equilibrium analysis. This, however, is tractable for only some simple cases such as two users arriving at a two node queueing network, which we then present.
△ Less
Submitted 13 December, 2011;
originally announced December 2011.
