-
Differentiable Distributionally Robust Optimization Layers
Authors:
Xutao Ma,
Chao Ning,
Wenli Du
Abstract:
In recent years, there has been a growing research interest in decision-focused learning, which embeds optimization problems as a layer in learning pipelines and demonstrates a superior performance than the prediction-focused approach. However, for distributionally robust optimization (DRO), a popular paradigm for decision-making under uncertainty, it is still unknown how to embed it as a layer, i…
▽ More
In recent years, there has been a growing research interest in decision-focused learning, which embeds optimization problems as a layer in learning pipelines and demonstrates a superior performance than the prediction-focused approach. However, for distributionally robust optimization (DRO), a popular paradigm for decision-making under uncertainty, it is still unknown how to embed it as a layer, i.e., how to differentiate decisions with respect to an ambiguity set. In this paper, we develop such differentiable DRO layers for generic mixed-integer DRO problems with parameterized second-order conic ambiguity sets and discuss its extension to Wasserstein ambiguity sets. To differentiate the mixed-integer decisions, we propose a novel dual-view methodology by handling continuous and discrete parts of decisions via different principles. Specifically, we construct a differentiable energy-based surrogate to implement the dual-view methodology and use importance sampling to estimate its gradient. We further prove that such a surrogate enjoys the asymptotic convergency under regularization. As an application of the proposed differentiable DRO layers, we develop a novel decision-focused learning pipeline for contextual distributionally robust decision-making tasks and compare it with the prediction-focused approach in experiments.
△ Less
Submitted 24 June, 2024;
originally announced June 2024.
-
Data-Driven Bayesian Nonparametric Wasserstein Distributionally Robust Optimization
Authors:
Chao Ning,
Xutao Ma
Abstract:
In this work, we develop a novel data-driven Bayesian nonparametric Wasserstein distributionally robust optimization (BNWDRO) framework for decision-making under uncertainty. The proposed framework unifies a Bayesian nonparametric method and the Wasserstein metric to decipher the global-local features of uncertainty data and encode these features into a novel data-driven ambiguity set. By establis…
▽ More
In this work, we develop a novel data-driven Bayesian nonparametric Wasserstein distributionally robust optimization (BNWDRO) framework for decision-making under uncertainty. The proposed framework unifies a Bayesian nonparametric method and the Wasserstein metric to decipher the global-local features of uncertainty data and encode these features into a novel data-driven ambiguity set. By establishing the theoretical connection between this data-driven ambiguity set and the conventional Wasserstein ambiguity set, we prove that the proposed framework enjoys the finite sample guarantee and asymptotic consistency. To efficiently solve the resulting distributionally robust optimization problem under the BNWDRO framework, we derive for this optimization problem an equivalent reformulation, which is kept tractable for many practical scenarios. Numerical experiments on a unit commitment problem verify the effectiveness of the proposed BNWDRO framework compared with existing methods.
△ Less
Submitted 6 November, 2023;
originally announced November 2023.
-
A Unified Early Termination Technique for Primal-dual Algorithms in Mixed Integer Conic Programming
Authors:
Yuwen Chen,
Catherine Ning,
Paul Goulart
Abstract:
We propose an early termination technique for mixed integer conic programming for use within branch-and-bound based solvers. Our approach generalizes previous early termination results for ADMM-based solvers to a broader class of primal-dual algorithms, including both operator splitting methods and interior point methods. The complexity for checking early termination is $O(n)$ for each termination…
▽ More
We propose an early termination technique for mixed integer conic programming for use within branch-and-bound based solvers. Our approach generalizes previous early termination results for ADMM-based solvers to a broader class of primal-dual algorithms, including both operator splitting methods and interior point methods. The complexity for checking early termination is $O(n)$ for each termination check assuming a bounded problem domain. We show that this domain restriction can be relaxed for problems whose data satisfies a simple rank condition, in which case each check requires an $O(n^2)$ solve using a linear system that must be factored only once at the root node. We further show how this approach can be used in hybrid model predictive control as long as system inputs are bounded. Numerical results show that our method leads to a moderate reduction in the total iterations required for branch-and-bound conic solvers with interior-point based subsolvers.
△ Less
Submitted 16 March, 2023;
originally announced March 2023.
-
Online Learning Based Risk-Averse Stochastic MPC of Constrained Linear Uncertain Systems
Authors:
Chao Ning,
Fengqi You
Abstract:
This paper investigates the problem of designing data-driven stochastic Model Predictive Control (MPC) for linear time-invariant systems under additive stochastic disturbance, whose probability distribution is unknown but can be partially inferred from data. We propose a novel online learning based risk-averse stochastic MPC framework in which Conditional Value-at-Risk (CVaR) constraints on system…
▽ More
This paper investigates the problem of designing data-driven stochastic Model Predictive Control (MPC) for linear time-invariant systems under additive stochastic disturbance, whose probability distribution is unknown but can be partially inferred from data. We propose a novel online learning based risk-averse stochastic MPC framework in which Conditional Value-at-Risk (CVaR) constraints on system states are required to hold for a family of distributions called an ambiguity set. The ambiguity set is constructed from disturbance data by leveraging a Dirichlet process mixture model that is self-adaptive to the underlying data structure and complexity. Specifically, the structural property of multimodality is exploit-ed, so that the first- and second-order moment information of each mixture component is incorporated into the ambiguity set. A novel constraint tightening strategy is then developed based on an equivalent reformulation of distributionally ro-bust CVaR constraints over the proposed ambiguity set. As more data are gathered during the runtime of the controller, the ambiguity set is updated online using real-time disturbance data, which enables the risk-averse stochastic MPC to cope with time-varying disturbance distributions. The online variational inference algorithm employed does not require all collected data be learned from scratch, and therefore the proposed MPC is endowed with the guaranteed computational complexity of online learning. The guarantees on recursive feasibility and closed-loop stability of the proposed MPC are established via a safe update scheme. Numerical examples are used to illustrate the effectiveness and advantages of the proposed MPC.
△ Less
Submitted 20 November, 2020;
originally announced November 2020.
-
Optimization under Uncertainty in the Era of Big Data and Deep Learning: When Machine Learning Meets Mathematical Programming
Authors:
Chao Ning,
Fengqi You
Abstract:
This paper reviews recent advances in the field of optimization under uncertainty via a modern data lens, highlights key research challenges and promise of data-driven optimization that organically integrates machine learning and mathematical programming for decision-making under uncertainty, and identifies potential research opportunities. A brief review of classical mathematical programming tech…
▽ More
This paper reviews recent advances in the field of optimization under uncertainty via a modern data lens, highlights key research challenges and promise of data-driven optimization that organically integrates machine learning and mathematical programming for decision-making under uncertainty, and identifies potential research opportunities. A brief review of classical mathematical programming techniques for hedging against uncertainty is first presented, along with their wide spectrum of applications in Process Systems Engineering. A comprehensive review and classification of the relevant publications on data-driven distributionally robust optimization, data-driven chance constrained program, data-driven robust optimization, and data-driven scenario-based optimization is then presented. This paper also identifies fertile avenues for future research that focuses on a closed-loop data-driven optimization framework, which allows the feedback from mathematical programming to machine learning, as well as scenario-based optimization leveraging the power of deep learning techniques. Perspectives on online learning-based data-driven multistage optimization with a learning-while-optimizing scheme is presented.
△ Less
Submitted 3 April, 2019;
originally announced April 2019.
-
A Transformation-Proximal Bundle Algorithm for Multistage Adaptive Robust Optimization and Application to Constrained Robust Optimal Control
Authors:
Chao Ning,
Fengqi You
Abstract:
This paper presents a novel transformation-proximal bundle algorithm for multistage adaptive robust optimization problems. By partitioning recourse decisions into state and control decisions, the proposed algorithm applies affine control policy only to state decisions and allows control decisions to be fully adaptive, thus transforming the original problem into an equivalent two-stage Adaptive Rob…
▽ More
This paper presents a novel transformation-proximal bundle algorithm for multistage adaptive robust optimization problems. By partitioning recourse decisions into state and control decisions, the proposed algorithm applies affine control policy only to state decisions and allows control decisions to be fully adaptive, thus transforming the original problem into an equivalent two-stage Adaptive Robust Optimization (ARO) problem. Importantly, this multi-to-two transformation is general enough to be employed with any two-stage ARO solution algorithms, thus opening a new avenue for a variety of multistage ARO algorithms. The proximal bundle method is developed for the resulting two-stage problem along with convergence analysis. In an inventory control application, the affine disturbance-feedback control policy suffers from a severe suboptimality with an average gap of 34.88%, while the proposed algorithm generates an average gap of merely 1.68%.
△ Less
Submitted 29 December, 2019; v1 submitted 13 October, 2018;
originally announced October 2018.
-
Instability of the solitary wave solutions for the generalized derivative nonlinear Schrödinger equation in the endpoint case
Authors:
Bing Li,
Cui Ning
Abstract:
We consider the stability theory of solitary wave solutions for the generalized derivative nonlinear Schrödinger equation
$$
i\partial_{t}u+\partial_{x}^{2}u+i|u|^{2σ}\partial_x u=0,
$$
where $1<σ<2$.
The equation has a two-parameter family of solitary wave solutions of the form…
▽ More
We consider the stability theory of solitary wave solutions for the generalized derivative nonlinear Schrödinger equation
$$
i\partial_{t}u+\partial_{x}^{2}u+i|u|^{2σ}\partial_x u=0,
$$
where $1<σ<2$.
The equation has a two-parameter family of solitary wave solutions of the form
$$ u_{ω,c}(t,x)=e^{iωt+i\frac c2(x-ct)-\frac{i}{2σ+2}\int_{-\infty}^{x-ct}\varphi^{2σ}_{ω,c}(y)dy}\varphi_{ω,c}(x-ct).
$$
The stability theory in the frequency region of $|c|<2\sqrtω$ was studied previously. In this paper, we prove the instability of the solitary wave solutions in the endpoint case $c=2\sqrtω$.
△ Less
Submitted 8 April, 2018;
originally announced April 2018.
-
Instability of the solitary wave solutions for the genenalized derivative Nonlinear Schrödinger equation in the critical frequency case
Authors:
Zihua Guo,
Cui Ning,
Yifei Wu
Abstract:
We study the stability theory of solitary wave solutions for the generalized derivative nonlinear Schrödinger equation $$ i\partial_{t}u+\partial_{x}^{2}u+i|u|^{2σ}\partial_x u=0. $$ The equation has a two-parameter family of solitary wave solutions of the form \begin{align*} φ_{ω,c}(x)=\varphi_{ω,c}(x)\exp{\big\{ i\frac c2 x-\frac{i}{2σ+2}\int_{-\infty}^{x}\varphi^{2σ}_{ω,c}(y)dy\big\}}. \end{ali…
▽ More
We study the stability theory of solitary wave solutions for the generalized derivative nonlinear Schrödinger equation $$ i\partial_{t}u+\partial_{x}^{2}u+i|u|^{2σ}\partial_x u=0. $$ The equation has a two-parameter family of solitary wave solutions of the form \begin{align*} φ_{ω,c}(x)=\varphi_{ω,c}(x)\exp{\big\{ i\frac c2 x-\frac{i}{2σ+2}\int_{-\infty}^{x}\varphi^{2σ}_{ω,c}(y)dy\big\}}. \end{align*} Here $ \varphi_{ω,c}$ is some real-valued function. It was proved in \cite{LiSiSu1} that the solitary wave solutions are stable if $-2\sqrt{ω}<c <2z_0\sqrt{ω}$, and unstable if $2z_0\sqrt{ω}<c <2\sqrt{ω}$ for some $z_0\in(0,1)$. We prove the instability at the borderline case $c =2z_0\sqrt{ω}$ for $1<σ<2$, improving the previous results in \cite{Fu-16-DNLS} where $3/2<σ<2$.
△ Less
Submitted 20 March, 2018;
originally announced March 2018.
-
Data-Driven Stochastic Robust Optimization: A General Computational Framework and Algorithm for Optimization under Uncertainty in the Big Data Era
Authors:
Chao Ning,
Fengqi You
Abstract:
A novel data-driven stochastic robust optimization (DDSRO) framework is proposed for optimization under uncertainty leveraging labeled multi-class uncertainty data. Uncertainty data in large datasets are often collected from various conditions, which are encoded by class labels. Machine learning methods including Dirichlet process mixture model and maximum likelihood estimation are employed for un…
▽ More
A novel data-driven stochastic robust optimization (DDSRO) framework is proposed for optimization under uncertainty leveraging labeled multi-class uncertainty data. Uncertainty data in large datasets are often collected from various conditions, which are encoded by class labels. Machine learning methods including Dirichlet process mixture model and maximum likelihood estimation are employed for uncertainty modeling. A DDSRO framework is further proposed based on the data-driven uncertainty model through a bi-level optimization structure. The outer optimization problem follows a two-stage stochastic programming approach to optimize the expected objective across different data classes; adaptive robust optimization is nested as the inner problem to ensure the robustness of the solution while maintaining computational tractability. A decomposition-based algorithm is further developed to solve the resulting multi-level optimization problem efficiently. Case studies on process network design and planning are presented to demonstrate the applicability of the proposed framework and algorithm.
△ Less
Submitted 29 December, 2017; v1 submitted 28 July, 2017;
originally announced July 2017.
