-
Uses of Sub-sample Estimates to Reduce Errors in Stochastic Optimization Models
Authors:
John R. Birge
Abstract:
Optimization software enables the solution of problems with millions of variables and associated parameters. These parameters are, however, often uncertain and represented with an analytical description of the parameter's distribution or with some form of sample. With large numbers of such parameters, optimization of the resulting model is often driven by mis-specifications or extreme sample chara…
▽ More
Optimization software enables the solution of problems with millions of variables and associated parameters. These parameters are, however, often uncertain and represented with an analytical description of the parameter's distribution or with some form of sample. With large numbers of such parameters, optimization of the resulting model is often driven by mis-specifications or extreme sample characteristics, resulting in solutions that are far from a true optimum. This paper describes how asymptotic convergence results may not be useful in large-scale problems and how the optimization of problems based on sub-sample estimates may achieve improved results over models using full-sample solution estimates. A motivating example and numerical results from a portfolio optimization problem demonstrate the potential improvement. A theoretical analysis also provides insight into the structure of problems where sub-sample optimization may be most beneficial.
△ Less
Submitted 10 October, 2023;
originally announced October 2023.
-
On the Convergence of L-shaped Algorithms for Two-Stage Stochastic Programming
Authors:
John R. Birge,
Haihao Lu,
Baoyu Zhou
Abstract:
In this paper, we design, analyze, and implement a variant of the two-loop L-shaped algorithms for solving two-stage stochastic programming problems that arise from important application areas including revenue management and power systems. We consider the setting in which it is intractable to compute exact objective function and (sub)gradient information, and instead, only estimates of objective…
▽ More
In this paper, we design, analyze, and implement a variant of the two-loop L-shaped algorithms for solving two-stage stochastic programming problems that arise from important application areas including revenue management and power systems. We consider the setting in which it is intractable to compute exact objective function and (sub)gradient information, and instead, only estimates of objective function and (sub)gradient values are available. Under common assumptions including fixed recourse and bounded (sub)gradients, the algorithm generates a sequence of iterates that converge to a neighborhood of optimality, where the radius of the convergence neighborhood depends on the level of the inexactness of objective function estimates. The number of outer and inner iterations needed to find an approximate optimal iterate is provided. Finally, we show a sample complexity result for the algorithm with a Polyak-type step-size policy that can be extended to analyze other situations. We also present a numerical study that verifies our theoretical results and demonstrates the superior empirical performance of our proposed algorithms over classic solvers.
△ Less
Submitted 3 September, 2023;
originally announced September 2023.
-
Quantification of Market Power Mitigation via Efficient Aggregation of Distributed Energy Resources
Authors:
Zuguang Gao,
Khaled Alshehri,
John R. Birge
Abstract:
Distributed energy resources (DERs) such as solar panels have small supply capacities and cannot be directly integrated into wholesale markets. So, the presence of an intermediary is critical. The intermediary could be a profit-seeking entity (called the aggregator) that buys DER supply from prosumers, and then sells them in the wholesale electricity market. Thus, DER integration has an influence…
▽ More
Distributed energy resources (DERs) such as solar panels have small supply capacities and cannot be directly integrated into wholesale markets. So, the presence of an intermediary is critical. The intermediary could be a profit-seeking entity (called the aggregator) that buys DER supply from prosumers, and then sells them in the wholesale electricity market. Thus, DER integration has an influence on wholesale market prices, demand, and supply. The purpose of this article is to shed light onto the impact of efficient DER aggregation on the market power of conventional generators. Firstly, under efficient DER aggregation, we quantify the social welfare gap between two cases: when conventional generators are truthful, and when they are strategic. We also do the same when DERs are not present. Secondly, we show that the gap due to market power of generators in the presence of DERs is smaller than the one when there is no DER participation. Finally, we provide explicit expressions of the gaps and conduct numerical experiments to gain deeper insights. The main message of this article is that market power of conventional generators can be mitigated by adopting an efficient DER aggregation model.
△ Less
Submitted 14 September, 2022;
originally announced September 2022.
-
Learning from Stochastically Revealed Preference
Authors:
John R. Birge,
Xiaocheng Li,
Chunlin Sun
Abstract:
We study the learning problem of revealed preference in a stochastic setting: a learner observes the utility-maximizing actions of a set of agents whose utility follows some unknown distribution, and the learner aims to infer the distribution through the observations of actions. The problem can be viewed as a single-constraint special case of the inverse linear optimization problem. Existing works…
▽ More
We study the learning problem of revealed preference in a stochastic setting: a learner observes the utility-maximizing actions of a set of agents whose utility follows some unknown distribution, and the learner aims to infer the distribution through the observations of actions. The problem can be viewed as a single-constraint special case of the inverse linear optimization problem. Existing works all assume that all the agents share one common utility which can easily be violated under practical contexts. In this paper, we consider two settings for the underlying utility distribution: a Gaussian setting where the customer utility follows the von Mises-Fisher distribution, and a $δ$-corruption setting where the customer utility distribution concentrates on one fixed vector with high probability and is arbitrarily corrupted otherwise. We devise Bayesian approaches for parameter estimation and develop theoretical guarantees for the recovery of the true parameter. We illustrate the algorithm performance through numerical experiments.
△ Less
Submitted 3 June, 2022;
originally announced June 2022.
-
Finite-Sample Analysis of Decentralized Q-Learning for Stochastic Games
Authors:
Zuguang Gao,
Qianqian Ma,
Tamer Başar,
John R. Birge
Abstract:
Learning in stochastic games is arguably the most standard and fundamental setting in multi-agent reinforcement learning (MARL). In this paper, we consider decentralized MARL in stochastic games in the non-asymptotic regime. In particular, we establish the finite-sample complexity of fully decentralized Q-learning algorithms in a significant class of general-sum stochastic games (SGs) - weakly acy…
▽ More
Learning in stochastic games is arguably the most standard and fundamental setting in multi-agent reinforcement learning (MARL). In this paper, we consider decentralized MARL in stochastic games in the non-asymptotic regime. In particular, we establish the finite-sample complexity of fully decentralized Q-learning algorithms in a significant class of general-sum stochastic games (SGs) - weakly acyclic SGs, which includes the common cooperative MARL setting with an identical reward to all agents (a Markov team problem) as a special case. We focus on the practical while challenging setting of fully decentralized MARL, where neither the rewards nor the actions of other agents can be observed by each agent. In fact, each agent is completely oblivious to the presence of other decision makers. Both the tabular and the linear function approximation cases have been considered. In the tabular setting, we analyze the sample complexity for the decentralized Q-learning algorithm to converge to a Markov perfect equilibrium (Nash equilibrium). With linear function approximation, the results are for convergence to a linear approximated equilibrium - a new notion of equilibrium that we propose - which describes that each agent's policy is a best reply (to other agents) within a linear space. Numerical experiments are also provided for both settings to demonstrate the results.
△ Less
Submitted 16 December, 2021; v1 submitted 14 December, 2021;
originally announced December 2021.
-
Approximation Schemes for Multiperiod Binary Knapsack Problems
Authors:
Zuguang Gao,
John R. Birge,
Varun Gupta
Abstract:
An instance of the multiperiod binary knapsack problem (MPBKP) is given by a horizon length $T$, a non-decreasing vector of knapsack sizes $(c_1, \ldots, c_T)$ where $c_t$ denotes the cumulative size for periods $1,\ldots,t$, and a list of $n$ items. Each item is a triple $(r, q, d)$ where $r$ denotes the reward of the item, $q$ its size, and $d$ its time index (or, deadline). The goal is to choos…
▽ More
An instance of the multiperiod binary knapsack problem (MPBKP) is given by a horizon length $T$, a non-decreasing vector of knapsack sizes $(c_1, \ldots, c_T)$ where $c_t$ denotes the cumulative size for periods $1,\ldots,t$, and a list of $n$ items. Each item is a triple $(r, q, d)$ where $r$ denotes the reward of the item, $q$ its size, and $d$ its time index (or, deadline). The goal is to choose, for each deadline $t$, which items to include to maximize the total reward, subject to the constraints that for all $t=1,\ldots,T$, the total size of selected items with deadlines at most $t$ does not exceed the cumulative capacity of the knapsack up to time $t$. We also consider the multiperiod binary knapsack problem with soft capacity constraints (MPBKP-S) where the capacity constraints are allowed to be violated by paying a penalty that is linear in the violation. The goal is to maximize the total profit, i.e., the total reward of selected items less the total penalty. Finally, we consider the multiperiod binary knapsack problem with soft stochastic capacity constraints (MPBKP-SS), where the non-decreasing vector of knapsack sizes $(c_1, \ldots, c_T)$ follow some arbitrary joint distribution but we are given access to the profit as an oracle, and we choose a subset of items to maximize the total expected profit, i.e., the total reward less the total expected penalty. For MPBKP, we exhibit a fully polynomial-time approximation scheme with runtime $\tilde{\mathcal{O}}\left(\min\left\{n+\frac{T^{3.25}}{ε^{2.25}},n+\frac{T^{2}}{ε^{3}},\frac{nT}{ε^2},\frac{n^2}ε\right\}\right)$ that achieves $(1+ε)$ approximation; for MPBKP-S, the $(1+ε)$ approximation can be achieved in $\mathcal{O}\left(\frac{n\log n}ε\cdot\min\left\{\frac{T}ε,n\right\}\right)$; for MPBKP-SS, a greedy algorithm is a 2-approximation when items have the same size.
△ Less
Submitted 31 March, 2021;
originally announced April 2021.
-
On Efficient Aggregation of Distributed Energy Resources
Authors:
Zuguang Gao,
Khaled Alshehri,
John R. Birge
Abstract:
The rapid expansion of distributed energy resources (DERs) is one of the most significant changes to electricity systems around the world. Examples of DERs include solar panels, small natural gas-fueled generators, combined heat and power plants, etc. Due to the small supply capacities of these DERs, it is impractical for them to participate directly in the wholesale electricity market. We study i…
▽ More
The rapid expansion of distributed energy resources (DERs) is one of the most significant changes to electricity systems around the world. Examples of DERs include solar panels, small natural gas-fueled generators, combined heat and power plants, etc. Due to the small supply capacities of these DERs, it is impractical for them to participate directly in the wholesale electricity market. We study in this paper an efficient aggregation model where a profit-maximizing aggregator procures electricity from DERs, and sells them in the wholesale market. The interaction between the aggregator and the DER owners is modeled as a Stackelberg game: the aggregator adopts two-part pricing by announcing a participation fee and a per-unit price of procurement for each DER owner, and the DER owner responds by choosing her payoff-maximizing energy supplies. We show that our proposed model preserves full market efficiency, i.e., the social welfare achieved by the aggregation model is the same as that when DERs participate directly in the wholesale market. We also note that two-part pricing is critical for market efficiency, and illustrate via an example that with one-part pricing, there will be an efficiency loss from DER aggregation, due to the profit-seeking behavior of the aggregator.
△ Less
Submitted 26 March, 2021;
originally announced March 2021.
-
Flexibility from Networks of Data Centers: A Market Clearing Formulation with Virtual Links
Authors:
Weiqi Zhang,
Line A. Roald,
Andrew A. Chien,
John R. Birge,
Victor M. Zavala
Abstract:
Data centers owned and operated by large companies have a high power consumption and this is expected to increase in the future. However, the ability to shift computing loads geographically and in time can provide flexibility to the power grid. We introduce the concept of virtual links to capture space-time load flexibility provided by geographically-distributed data centers in market clearing pro…
▽ More
Data centers owned and operated by large companies have a high power consumption and this is expected to increase in the future. However, the ability to shift computing loads geographically and in time can provide flexibility to the power grid. We introduce the concept of virtual links to capture space-time load flexibility provided by geographically-distributed data centers in market clearing procedures. We show that the virtual link abstraction fits well into existing market clearing frameworks and can help analyze and establish market design properties. This is demonstrated using illustrative case studies.
△ Less
Submitted 11 October, 2019;
originally announced October 2019.
-
Optimisation via Slice Sampling
Authors:
John R. Birge,
Nicholas G. Polson
Abstract:
In this paper, we develop a simulation-based approach to optimisation with multi-modal functions using slice sampling. Our method specifies the objective function as an energy potential in a Boltzmann distribution and then we use auxiliary exponential slice variables to provide samples for a variety of energy levels. Our slice sampler draws uniformly over the augmented slice region. We identify th…
▽ More
In this paper, we develop a simulation-based approach to optimisation with multi-modal functions using slice sampling. Our method specifies the objective function as an energy potential in a Boltzmann distribution and then we use auxiliary exponential slice variables to provide samples for a variety of energy levels. Our slice sampler draws uniformly over the augmented slice region. We identify the global modes by projecting the path of the chain back to the underlying space. Four standard test functions are used to illustrate the methodology: Rosenbrock, Himmelblau, Rastrigin, and Shubert. These functions demonstrate the flexibility of our approach as they include functions with long ridges (Rosenbrock), multi-modality (Himmelblau, Shubert) and many local modes dominated by one global (Rastrigin). The methods described here are implemented in the {\tt R} package {\tt McmcOpt}.
△ Less
Submitted 10 December, 2012;
originally announced December 2012.
-
Split Sampling: Expectations, Normalisation and Rare Events
Authors:
John R. Birge,
Changgee Chang,
Nicholas G. Polson
Abstract:
In this paper we develop a methodology that we call split sampling methods to estimate high dimensional expectations and rare event probabilities. Split sampling uses an auxiliary variable MCMC simulation and expresses the expectation of interest as an integrated set of rare event probabilities. We derive our estimator from a Rao-Blackwellised estimate of a marginal auxiliary variable distribution…
▽ More
In this paper we develop a methodology that we call split sampling methods to estimate high dimensional expectations and rare event probabilities. Split sampling uses an auxiliary variable MCMC simulation and expresses the expectation of interest as an integrated set of rare event probabilities. We derive our estimator from a Rao-Blackwellised estimate of a marginal auxiliary variable distribution. We illustrate our method with two applications. First, we compute a shortest network path rare event probability and compare our method to estimation to a cross entropy approach. Then, we compute a normalisation constant of a high dimensional mixture of Gaussians and compare our estimate to one based on nested sampling. We discuss the relationship between our method and other alternatives such as the product of conditional probability estimator and importance sampling. The methods developed here are available in the R package: SplitSampling.
△ Less
Submitted 31 October, 2013; v1 submitted 3 December, 2012;
originally announced December 2012.
