Showing 1–35 of 35 results for author: Bienstock, D

Accurate and Warm-Startable Linear Cutting-Plane Relaxations for ACOPF

Authors: Daniel Bienstock, Matias Villagra

Abstract: We present a linear cutting-plane relaxation approach that rapidly proves tight lower bounds for the Alternating Current Optimal Power Flow Problem (ACOPF). Our method leverages outer-envelope linear cuts for well-known second-order cone relaxations for ACOPF along with modern cut management techniques. These techniques prove effective on a broad family of ACOPF instances, including the largest on… ▽ More We present a linear cutting-plane relaxation approach that rapidly proves tight lower bounds for the Alternating Current Optimal Power Flow Problem (ACOPF). Our method leverages outer-envelope linear cuts for well-known second-order cone relaxations for ACOPF along with modern cut management techniques. These techniques prove effective on a broad family of ACOPF instances, including the largest ones publicly available, quickly and robustly yielding sharp bounds. Our primary focus concerns the (frequent) case where an ACOPF instance is considered following a small or moderate change in problem data, e.g., load changes and generator or branch shut-offs. We provide significant computational evidence that the cuts computed on the prior instance provide an effective warm-start for our algorithm. △ Less

Submitted 4 February, 2024; originally announced March 2024.

Comments: arXiv admin note: substantial text overlap with arXiv:2312.04251

Accurate Linear Cutting-Plane Relaxations for ACOPF

Authors: Daniel Bienstock, Matias Villagra

Abstract: We present a pure linear cutting-plane relaxation approach for rapidly proving tight and accurate lower bounds for the Alternating Current Optimal Power Flow Problem (ACOPF) and its multi-period extension with ram** constraints. Our method leverages outer-envelope linear cuts for well-known second-order cone relaxations for ACOPF together with modern cut management techniques and reformulations… ▽ More We present a pure linear cutting-plane relaxation approach for rapidly proving tight and accurate lower bounds for the Alternating Current Optimal Power Flow Problem (ACOPF) and its multi-period extension with ram** constraints. Our method leverages outer-envelope linear cuts for well-known second-order cone relaxations for ACOPF together with modern cut management techniques and reformulations to attain numerical stability. These techniques prove effective on a broad family of ACOPF instances, including the largest ones publicly available, quickly and robustly yielding tight bounds. Additionally, we consider the (frequent) case where an ACOPF instance is handled following a small or moderate change in problem data, e.g., load changes and generator or branch shut-offs. We provide significant computational evidence, on single and multi-period ACOPF instances, that the cuts computed on the prior instance provide very good lower bounds when warm-starting our algorithm on the perturbed instance. △ Less

Submitted 25 May, 2024; v1 submitted 7 December, 2023; originally announced December 2023.

Risk-Aware Security-Constrained Unit Commitment: Taming the Curse of Real-Time Volatility and Consumer Exposure

Authors: Daniel Bienstock, Yury Dvorkin, Cheng Guo, Robert Mieth, Jiayi Wang

Abstract: We propose an enhancement to wholesale electricity markets whereby the exposure of consumers to increasingly large and volatile consumer payments arising as a byproduct of volatile real-time net loads -- i.e., loads minus renewable outputs -- and prices, both compared to day-ahead cleared values. We incorporate a robust estimate of such excess payments into the day-ahead computation and specifical… ▽ More We propose an enhancement to wholesale electricity markets whereby the exposure of consumers to increasingly large and volatile consumer payments arising as a byproduct of volatile real-time net loads -- i.e., loads minus renewable outputs -- and prices, both compared to day-ahead cleared values. We incorporate a robust estimate of such excess payments into the day-ahead computation and specifically seek to account for volatility in real-time net loads and renewable generation. Our model features a data-driven uncertainty set based on principal component analysis, which accommodates both load and wind production volatility and captures locational correlation of uncertain data. To solve the model more efficiently, we develop a decomposition algorithm that can handle nonconvex subproblems. Our extensive experiments on a realistic NYISO data set show that the risk-aware model protects the consumers from potential high costs caused by adverse circumstances. △ Less

Submitted 17 April, 2024; v1 submitted 28 November, 2023; originally announced November 2023.

Incentivizing Investment and Reliability: A Study on Electricity Capacity Markets

Authors: Cheng Guo, Christian Kroer, Yury Dvorkin, Daniel Bienstock

Abstract: The capacity market, a marketplace to exchange available generation capacity for electricity production, provides a major revenue stream for generators and is adopted in several U.S. regions. A subject of ongoing debate, the capacity market is viewed by its proponents as a crucial mechanism to ensure system reliability, while critics highlight its drawbacks such as market distortion. Under a novel… ▽ More The capacity market, a marketplace to exchange available generation capacity for electricity production, provides a major revenue stream for generators and is adopted in several U.S. regions. A subject of ongoing debate, the capacity market is viewed by its proponents as a crucial mechanism to ensure system reliability, while critics highlight its drawbacks such as market distortion. Under a novel analytical framework, we rigorously evaluate the impact of the capacity market on generators' revenue and system reliability. More specifically, based on market designs at New York Independent System Operator (NYISO), we propose market equilibrium-based models to capture salient aspects of the capacity market and its interaction with the energy market. We also develop a leader-follower model to study market power. We show that the capacity market incentivizes the investment of generators with lower net cost of new entry. It also facilitates reliability by preventing significant physical withholding when the demand is relatively high. Nevertheless, the capacity market may not provide enough revenue for peaking plants. Moreover, it is susceptible to market power, which necessitates tailored market power mitigation measures depending on market dynamics. We provide further insights via large-scale experiments on data from NYISO markets. △ Less

Submitted 10 November, 2023; originally announced November 2023.

A Physics-Informed Machine Learning for Electricity Markets: A NYISO Case Study

Authors: Robert Ferrando, Laurent Pagnier, Robert Mieth, Zhirui Liang, Yury Dvorkin, Daniel Bienstock, Michael Chertkov

Abstract: This paper addresses the challenge of efficiently solving the optimal power flow problem in real-time electricity markets. The proposed solution, named Physics-Informed Market-Aware Active Set learning OPF (PIMA-AS-OPF), leverages physical constraints and market properties to ensure physical and economic feasibility of market-clearing outcomes. Specifically, PIMA-AS-OPF employs the active set lear… ▽ More This paper addresses the challenge of efficiently solving the optimal power flow problem in real-time electricity markets. The proposed solution, named Physics-Informed Market-Aware Active Set learning OPF (PIMA-AS-OPF), leverages physical constraints and market properties to ensure physical and economic feasibility of market-clearing outcomes. Specifically, PIMA-AS-OPF employs the active set learning technique and expands its capabilities to account for curtailment in load or renewable power generation, which is a common challenge in real-world power systems. The core of PIMA-AS-OPF is a fully-connected neural network that takes the net load and the system topology as input. The outputs of this neural network include active constraints such as saturated generators and transmission lines, as well as non-zero load shedding and wind curtailments. These outputs allow for reducing the original market-clearing optimization to a system of linear equations, which can be solved efficiently and yield both the dispatch decisions and the locational marginal prices (LMPs). The dispatch decisions and LMPs are then tested for their feasibility with respect to the requirements for efficient market-clearing results. The accuracy and scalability of the proposed method is tested on a realistic 1814-bus NYISO system with current and future renewable energy penetration levels. △ Less

Submitted 31 March, 2023; originally announced April 2023.

On a Computationally Ill-Behaved Bilevel Problem with a Continuous and Nonconvex Lower Level

Authors: Yasmine Beck, Daniel Bienstock, Martin Schmidt, Johannes Thürauf

Abstract: It is well known that bilevel optimization problems are hard to solve both in theory and practice. In this paper, we highlight a further computational difficulty when it comes to solving bilevel problems with continuous but nonconvex lower levels. Even if the lower-level problem is solved to $\varepsilon$-feasibility regarding its nonlinear constraints for an arbitrarily small but positive… ▽ More It is well known that bilevel optimization problems are hard to solve both in theory and practice. In this paper, we highlight a further computational difficulty when it comes to solving bilevel problems with continuous but nonconvex lower levels. Even if the lower-level problem is solved to $\varepsilon$-feasibility regarding its nonlinear constraints for an arbitrarily small but positive $\varepsilon$, the obtained bilevel solution as well as its objective value may be arbitrarily far away from the actual bilevel solution and its actual objective value. This result even holds for bilevel problems for which the nonconvex lower level is uniquely solvable, for which the strict complementarity condition holds, for which the feasible set is convex, and for which Slater's constraint qualification is satisfied for all feasible upper-level decisions. Since the consideration of $\varepsilon$-feasibility cannot be avoided when solving nonconvex problems to global optimality, our result shows that computational bilevel optimization with continuous and nonconvex lower levels needs to be done with great care. Finally, we illustrate that the nonlinearities in the lower level are the key reason for the observed bad behavior by showing that linear bilevel problems behave much better at least on the level of feasible solutions. △ Less

Submitted 3 May, 2023; v1 submitted 2 February, 2022; originally announced February 2022.

MSC Class: 90-XX; 90C26; 90C31

Cardinality Minimization, Constraints, and Regularization: A Survey

Authors: Andreas M. Tillmann, Daniel Bienstock, Andrea Lodi, Alexandra Schwartz

Abstract: We survey optimization problems that involve the cardinality of variable vectors in constraints or the objective function. We provide a unified viewpoint on the general problem classes and models, and give concrete examples from diverse application fields such as signal and image processing, portfolio selection, or machine learning. The paper discusses general-purpose modeling techniques and broad… ▽ More We survey optimization problems that involve the cardinality of variable vectors in constraints or the objective function. We provide a unified viewpoint on the general problem classes and models, and give concrete examples from diverse application fields such as signal and image processing, portfolio selection, or machine learning. The paper discusses general-purpose modeling techniques and broadly applicable as well as problem-specific exact and heuristic solution approaches. While our perspective is that of mathematical optimization, a main goal of this work is to reach out to and build bridges between the different communities in which cardinality optimization problems are frequently encountered. In particular, we highlight that modern mixed-integer programming, which is often regarded as impractical due to commonly unsatisfactory behavior of black-box solvers applied to generic problem formulations, can in fact produce provably high-quality or even optimal solutions for cardinality optimization problems, even in large-scale real-world settings. Achieving such performance typically draws on the merits of problem-specific knowledge that may stem from different fields of application and, e.g., shed light on structural properties of a model or its solutions, or lead to the development of efficient heuristics; we also provide some illustrative examples. △ Less

Submitted 8 August, 2022; v1 submitted 17 June, 2021; originally announced June 2021.

Complexity, Exactness, and Rationality in Polynomial Optimization

Authors: Daniel Bienstock, Alberto del Pia, Robert Hildebrand

Abstract: We focus on rational solutions or nearly-feasible rational solutions that serve as certificates of feasibility for polynomial optimization problems. We show that, under some separability conditions, certain cubic polynomially constrained sets admit rational solutions. However, we show in other cases that it is NP Hard to detect if rational solutions exist or if they exist of any reasonable size. W… ▽ More We focus on rational solutions or nearly-feasible rational solutions that serve as certificates of feasibility for polynomial optimization problems. We show that, under some separability conditions, certain cubic polynomially constrained sets admit rational solutions. However, we show in other cases that it is NP Hard to detect if rational solutions exist or if they exist of any reasonable size. We extend this idea to various settings including near feasible, but super optimal solutions and detecting rational rays on which a cubic function is unbounded. Lastly, we show that in fixed dimension, the feasibility problem over a set defined by polynomial inequalities is in NP by providing a simple certificate to verify feasibility. We conclude with several related examples of irrationality and encoding size issues in QCQPs and SOCPs. △ Less

Submitted 14 April, 2022; v1 submitted 16 November, 2020; originally announced November 2020.

Mathematical Programming formulations for the Alternating Current Optimal Power Flow problem

Authors: Daniel Bienstock, Mauro Escobar, Claudio Gentile, Leo Liberti

Abstract: Power flow refers to the injection of power on the lines of an electrical grid, so that all the injections at the nodes form a consistent flow within the network. Optimality, in this setting, is usually intended as the minimization of the cost of generating power. Current can either be direct or alternating: while the former yields approximate linear programming formulations, the latter yields for… ▽ More Power flow refers to the injection of power on the lines of an electrical grid, so that all the injections at the nodes form a consistent flow within the network. Optimality, in this setting, is usually intended as the minimization of the cost of generating power. Current can either be direct or alternating: while the former yields approximate linear programming formulations, the latter yields formulations of a much more interesting sort: namely, nonconvex nonlinear programs in complex numbers. In this technical survey, we derive formulation variants and relaxations of the alternating current optimal power flow problem. △ Less

Submitted 10 July, 2020; originally announced July 2020.

MSC Class: 90C90; 90C26

Journal ref: 4OR - A Quarterly Journal of Operations Research (2020)

Robust Streaming PCA

Authors: Daniel Bienstock, Minchan Jeong, Apurv Shukla, Se-Young Yun

Abstract: We consider streaming principal component analysis when the stochastic data-generating model is subject to perturbations. While existing models assume a fixed covariance, we adopt a robust perspective where the covariance matrix belongs to a temporal uncertainty set. Under this setting, we provide fundamental limits on convergence of any algorithm recovering principal components. We analyze the co… ▽ More We consider streaming principal component analysis when the stochastic data-generating model is subject to perturbations. While existing models assume a fixed covariance, we adopt a robust perspective where the covariance matrix belongs to a temporal uncertainty set. Under this setting, we provide fundamental limits on convergence of any algorithm recovering principal components. We analyze the convergence of the noisy power method and Oja's algorithm, both studied for the stationary data generating model, and argue that the noisy power method is rate-optimal in our setting. Finally, we demonstrate the validity of our analysis through numerical experiments on synthetic and real-world dataset. △ Less

Submitted 12 October, 2022; v1 submitted 8 February, 2019; originally announced February 2019.

Comments: The authors are ordered alphabetically. 36th Conference on Neural Information Processing Systems (NeurIPS 2022)

Learning from power system data stream: phasor-detective approach

Authors: Mauro Escobar, Daniel Bienstock, Michael Chertkov

Abstract: Assuming access to synchronized stream of Phasor Measurement Unit (PMU) data over a significant portion of a power system interconnect, say controlled by an Independent System Operator (ISO), what can you extract about past, current and future state of the system? We have focused on answering this practical questions pragmatically - empowered with nothing but standard tools of data analysis, such… ▽ More Assuming access to synchronized stream of Phasor Measurement Unit (PMU) data over a significant portion of a power system interconnect, say controlled by an Independent System Operator (ISO), what can you extract about past, current and future state of the system? We have focused on answering this practical questions pragmatically - empowered with nothing but standard tools of data analysis, such as PCA, filtering and cross-correlation analysis. Quite surprisingly we have found that even during the quiet "no significant events" period this standard set of statistical tools allows the "phasor-detective" to extract from the data important hidden anomalies, such as problematic control loops at loads and wind farms, and mildly malfunctioning assets, such as transformers and generators. We also discuss and sketch future challenges a mature phasor-detective can possibly tackle by adding machine learning and physics modeling sophistication to the basic approach. △ Less

Submitted 17 December, 2018; v1 submitted 17 November, 2018; originally announced November 2018.

Journal ref: 2019 IEEE Milan PowerTech, Milan, Italy, 2019, pp. 1-6

Principled Deep Neural Network Training through Linear Programming

Authors: Daniel Bienstock, Gonzalo Muñoz, Sebastian Pokutta

Abstract: Deep learning has received much attention lately due to the impressive empirical performance achieved by training algorithms. Consequently, a need for a better theoretical understanding of these problems has become more evident in recent years. In this work, using a unified framework, we show that there exists a polyhedron which encodes simultaneously all possible deep neural network training prob… ▽ More Deep learning has received much attention lately due to the impressive empirical performance achieved by training algorithms. Consequently, a need for a better theoretical understanding of these problems has become more evident in recent years. In this work, using a unified framework, we show that there exists a polyhedron which encodes simultaneously all possible deep neural network training problems that can arise from a given architecture, activation functions, loss function, and sample-size. Notably, the size of the polyhedral representation depends only linearly on the sample-size, and a better dependency on several other network parameters is unlikely (assuming $P\neq NP$). Additionally, we use our polyhedral representation to obtain new and better computational complexity results for training problems of well-known neural network architectures. Our results provide a new perspective on training problems through the lens of polyhedral theory and reveal a strong structure arising from these problems. △ Less

Submitted 1 March, 2022; v1 submitted 7 October, 2018; originally announced October 2018.

Stochastic defense against complex grid attacks

Authors: Daniel Bienstock, Mauro Escobar

Abstract: We describe defense mechanisms designed to detect sophisticated grid attacks involving both physical actions (including load modification) and sensor output alteration, with the latter performed in a sparse manner and also so as to take into account grid response mechanisms (secondary response). The attacks aim to be both undetectable even under a full AC power flow model, and to hide equipment ov… ▽ More We describe defense mechanisms designed to detect sophisticated grid attacks involving both physical actions (including load modification) and sensor output alteration, with the latter performed in a sparse manner and also so as to take into account grid response mechanisms (secondary response). The attacks aim to be both undetectable even under a full AC power flow model, and to hide equipment overload. We demonstrate that such attacks, while perhaps difficult to implement, nevertheless are easily computed even on systems with a large number of installed sensors, and can, in a static setting, successfuly hide large line overloads. Furthermore an attacker that understands the ongoing stochastic nature of sensor signals can extend the attack so as to remain effective for a nontrivial time period. To counteract such "ideal" or "perfect" attacks, we demonstrate a set of data-driven stochastic techniques to thwart the attacker and detect that an attack has taken place. △ Less

Submitted 3 August, 2019; v1 submitted 17 July, 2018; originally announced July 2018.

Journal ref: IEEE Transactions on Control of Network Systems, vol. 7, no. 2, pp. 842-854, June 2020

Simpler derivation of bounded pitch inequalities for set covering, and minimum knapsack sets

Authors: Daniel Bienstock, Mark Zuckerberg

Abstract: A valid inequality α^Tx \ge α_0 for a set covering problem is said to have pitch <= k ( a positive integer) if the k smallest positive α_j sum to at least alpha_0. This paper presents a new, simple derivation of a relaxation for set covering problems whose solutions satisfy all valid inequalities of pitch and is of polynomial size, for each fixed . We also consider the minimum knapsack problem, an… ▽ More A valid inequality α^Tx \ge α_0 for a set covering problem is said to have pitch <= k ( a positive integer) if the k smallest positive α_j sum to at least alpha_0. This paper presents a new, simple derivation of a relaxation for set covering problems whose solutions satisfy all valid inequalities of pitch and is of polynomial size, for each fixed . We also consider the minimum knapsack problem, and show that for each fixed integer p > 0 and 0 < ε< 1 one can separate, within additive tolerance ε, from the relaxation defined by the valid inequalities with coefficients in {0, 1, . . . , p} in time polynomial in the number of variables and 1/ε. △ Less

Submitted 19 June, 2018; originally announced June 2018.

Variance-Aware Optimal Power Flow

Authors: Daniel Bienstock, Apurv Shukla

Abstract: The incorporation of stochastic loads and generation into the operation of power grids gives rise to an exposure to stochastic risk. This risk has been addressed in prior work through a variety of mechanisms, such as scenario generation or chance constraints, that can be incorporated into OPF computations. Nevertheless, numerical experiments reveal that the resulting operational decisions can prod… ▽ More The incorporation of stochastic loads and generation into the operation of power grids gives rise to an exposure to stochastic risk. This risk has been addressed in prior work through a variety of mechanisms, such as scenario generation or chance constraints, that can be incorporated into OPF computations. Nevertheless, numerical experiments reveal that the resulting operational decisions can produce power flows with very high variance. In this paper we introduce a variety of convex variants of OPF that explicitly address the interplay of (power flow) variance with cost minimization, and present numerical experiments that highlight our contributions. △ Less

Submitted 2 November, 2017; originally announced November 2017.

Chance-Constrained Unit Commitment with N-1 Security and Wind Uncertainty

Authors: Kaarthik Sundar, Harsha Nagarajan, Line Roald, Sidhant Misra, Russell Bent, Daniel Bienstock

Abstract: As renewable wind energy penetration rates continue to increase, one of the major challenges facing grid operators is the question of how to control transmission grids in a reliable and a cost-efficient manner. The stochastic nature of wind forces an alteration of traditional methods for solving day-ahead and look-ahead unit commitment and dispatch. In particular, the variability of wind generatio… ▽ More As renewable wind energy penetration rates continue to increase, one of the major challenges facing grid operators is the question of how to control transmission grids in a reliable and a cost-efficient manner. The stochastic nature of wind forces an alteration of traditional methods for solving day-ahead and look-ahead unit commitment and dispatch. In particular, the variability of wind generation increases the risk of unexpected overloads and cascading events. To address these questions, we present an N-1 Security and Chance-Constrained Unit Commitment (SCCUC) that includes models of generation reserves that respond to wind fluctuations and component outages. We formulate the SCCUC as a mixed-integer, second-order cone problem that limits the probability of failure. We develop a modified Benders decomposition algorithm to solve the problem to optimality and present detailed case studies on the IEEE RTS-96 three-area and the IEEE 300 NESTA test systems. The case studies assess the economic impacts of contingencies and various degrees of wind power penetration and demonstrate the effectiveness and scalability of the algorithm. △ Less

Submitted 20 November, 2020; v1 submitted 15 March, 2017; originally announced March 2017.

Comments: 24 pages, figures. arXiv admin note: substantial text overlap with arXiv:1602.00079

Robust linear control of nonconvex battery operation in transmission systems

Authors: Daniel Bienstock, Carsten Matke, Gonzalo Munoz, Shuoguang Yang

Abstract: We describe a robust multiperiod transmission plan- ning model including renewables and batteries, where battery output is used to partly offset renewable output deviations from forecast. A central element is a nonconvex battery operation model which is used with a robust model of forecast errors and a linear control scheme. Even though the problem is nonconvex we provide an efficient and theoreti… ▽ More We describe a robust multiperiod transmission plan- ning model including renewables and batteries, where battery output is used to partly offset renewable output deviations from forecast. A central element is a nonconvex battery operation model which is used with a robust model of forecast errors and a linear control scheme. Even though the problem is nonconvex we provide an efficient and theoretically valid algorithm that effectively solves cases on large transmission systems. Index Terms. Batteries, control, robust optimization. △ Less

Submitted 6 November, 2016; v1 submitted 28 October, 2016; originally announced October 2016.

Outer-Product-Free Sets for Polynomial Optimization and Oracle-Based Cuts

Authors: Daniel Bienstock, Chen Chen, Gonzalo Muñoz

Abstract: This paper introduces cutting planes that involve minimal structural assumptions, enabling the generation of strong polyhedral relaxations for a broad class of problems. We consider valid inequalities for the set $S\cap P$, where $S$ is a closed set, and $P$ is a polyhedron. Given an oracle that provides the distance from a point to $S$, we construct a pure cutting plane algorithm which is shown t… ▽ More This paper introduces cutting planes that involve minimal structural assumptions, enabling the generation of strong polyhedral relaxations for a broad class of problems. We consider valid inequalities for the set $S\cap P$, where $S$ is a closed set, and $P$ is a polyhedron. Given an oracle that provides the distance from a point to $S$, we construct a pure cutting plane algorithm which is shown to converge if the initial relaxation is a polyhedron. These cuts are generated from convex forbidden zones, or $S$-free sets, derived from the oracle. We also consider the special case of polynomial optimization. Accordingly we develop a theory of \emph{outer-product-free} sets, where $S$ is the set of real, symmetric matrices of the form $xx^T$. All maximal outer-product-free sets of full dimension are shown to be convex cones and we identify several families of such sets. These families are used to generate strengthened intersection cuts that can separate any infeasible extreme point of a linear programming relaxation efficiently. Computational experiments demonstrate the promise of our approach. △ Less

Submitted 30 January, 2020; v1 submitted 14 October, 2016; originally announced October 2016.

Comments: 48 pages, 4 figures

MSC Class: 90C30; 90C57

Unit Commitment with N-1 Security and Wind Uncertainty

Authors: Kaarthik Sundar, Harsha Nagarajan, Miles Lubin, Line Roald, Sidhant Misra, Russell Bent, Daniel Bienstock

Abstract: As renewable wind energy penetration rates continue to increase, one of the major challenges facing grid operators is the question of how to control transmission grids in a reliable and a cost-efficient manner. The stochastic nature of wind forces an alteration of traditional methods for solving day-ahead and look-ahead unit commitment and dispatch. In particular, uncontrollable wind generation in… ▽ More As renewable wind energy penetration rates continue to increase, one of the major challenges facing grid operators is the question of how to control transmission grids in a reliable and a cost-efficient manner. The stochastic nature of wind forces an alteration of traditional methods for solving day-ahead and look-ahead unit commitment and dispatch. In particular, uncontrollable wind generation increases the risk of random component failures. To address these questions, we present an N-1 Security and Chance-Constrained Unit Commitment (SCCUC) that includes the modeling of generation reserves that respond to wind fluctuations and tertiary reserves to account for single component outages. The basic formulation is reformulated as a mixed-integer second-order cone problem to limit the probability of failure. We develop three different algorithms to solve the problem to optimality and present a detailed case study on the IEEE RTS-96 single area system. The case study assesses the economic impacts due to contingencies and various degrees of wind power penetration into the system and also corroborates the effectiveness of the algorithms. △ Less

Submitted 30 January, 2016; originally announced February 2016.

Comments: 7 pages, 3 figures

Strong NP-hardness of AC power flows feasibility

Authors: Daniel Bienstock, Abhinav Verma

Abstract: We present a rigorous proof of strong NP-hardness of the AC-OPF problem. We present a rigorous proof of strong NP-hardness of the AC-OPF problem. △ Less

Submitted 9 April, 2019; v1 submitted 22 December, 2015; originally announced December 2015.

Models for managing the impact of an epidemic

Authors: Daniel Bienstock, A. Cecilia Zenteno

Abstract: In this paper we consider robust models for emergency staff deployment in the event of a flu pandemic. We focus on managing critical staff levels at organizations that must remain operational during such an event, and develop methodologies for managing emergency resources with the goal of minimizing the impact of the pandemic. We present numerical experiments using realistic data to study the effe… ▽ More In this paper we consider robust models for emergency staff deployment in the event of a flu pandemic. We focus on managing critical staff levels at organizations that must remain operational during such an event, and develop methodologies for managing emergency resources with the goal of minimizing the impact of the pandemic. We present numerical experiments using realistic data to study the effectiveness of our approach. The underlying methodology is that of robust optimization; we model the problem as an infinite linear program which is approximately solved using a variant of Benders decomposition. △ Less

Submitted 30 July, 2015; originally announced July 2015.

Two-sided linear chance constraints and extensions

Authors: Miles Lubin, Daniel Bienstock, Juan Pablo Vielma

Abstract: We examine the convexity and tractability of the two-sided linear chance constraint model under Gaussian uncertainty. We show that these constraints can be applied directly to model a larger class of nonlinear chance constraints as well as provide a reasonable approximation for a challenging class of quadratic chance constraints of direct interest for applications in power systems. With a view tow… ▽ More We examine the convexity and tractability of the two-sided linear chance constraint model under Gaussian uncertainty. We show that these constraints can be applied directly to model a larger class of nonlinear chance constraints as well as provide a reasonable approximation for a challenging class of quadratic chance constraints of direct interest for applications in power systems. With a view towards practical computations, we develop a second-order cone outer approximation of the two-sided chance constraint with provably small approximation error. △ Less

Submitted 26 February, 2016; v1 submitted 7 July, 2015; originally announced July 2015.

Robust Control of Cascading Power Grid Failures using Stochastic Approximation

Authors: Daniel Bienstock, Guy Grebla

Abstract: Cascading failure of a power transmission system are initiated by an exogenous event that disable a set of elements (e.g., lines) followed by a sequence of interrelated failures (or more precisely, trips) of overloaded elements caused by the combination of physics of power flows in the changed system topology, and controls. Should this sequence accelerate it can lead to a large system failure with… ▽ More Cascading failure of a power transmission system are initiated by an exogenous event that disable a set of elements (e.g., lines) followed by a sequence of interrelated failures (or more precisely, trips) of overloaded elements caused by the combination of physics of power flows in the changed system topology, and controls. Should this sequence accelerate it can lead to a large system failure with significant loss of load. In previous work we have analyzed deterministic algorithms that in an online fashion (i.e., responding to observed data) selectively shed load so as to minimize the amount of lost load at termination of the cascade. In this work we present a rigorous methodology for incorporating noise and model errors, based on the Sample Average Approximation methodology for stochastic optimization. △ Less

Submitted 18 July, 2015; v1 submitted 3 April, 2015; originally announced April 2015.

Vulnerability Analysis of Power Systems

Authors: Taedong Kim, Stephen J. Wright, Daniel Bienstock, Sean Harnett

Abstract: Potential vulnerabilities in a power grid can be exposed by identifying those transmission lines on which attacks (in the form of interference with their transmission capabilities) causes maximum disruption to the grid. In this study, we model the grid by (nonlinear) AC power flow equations, and assume that attacks take the form of increased impedance along transmission lines. We quantify disrupti… ▽ More Potential vulnerabilities in a power grid can be exposed by identifying those transmission lines on which attacks (in the form of interference with their transmission capabilities) causes maximum disruption to the grid. In this study, we model the grid by (nonlinear) AC power flow equations, and assume that attacks take the form of increased impedance along transmission lines. We quantify disruption in several different ways, including (a) overall deviation of the voltages at the buses from 1.0 per unit (p.u.), and (b) the minimal amount of load that must be shed in order to restore the grid to stable operation. We describe optimization formulations of the problem of finding the most disruptive attack, which are either nonlinear programing problems or nonlinear bilevel optimization problems, and describe customized algorithms for solving these problems. Experimental results on the IEEE 118-Bus system and a Polish 2383-Bus system are presented. △ Less

Submitted 8 March, 2015; originally announced March 2015.

LP approximations to mixed-integer polynomial optimization problems

Authors: Daniel Bienstock, Gonzalo Munoz

Abstract: We present a class of linear programming approximations for constrained optimization problems. In the case of mixed-integer polynomial optimization problems, if the intersection graph of the constraints has bounded tree-width our construction yields a class of linear size formulations that attain any desired tolerance. As a result, we obtain an approximation scheme for the "AC-OPF" problem on grap… ▽ More We present a class of linear programming approximations for constrained optimization problems. In the case of mixed-integer polynomial optimization problems, if the intersection graph of the constraints has bounded tree-width our construction yields a class of linear size formulations that attain any desired tolerance. As a result, we obtain an approximation scheme for the "AC-OPF" problem on graphs with bounded tree-width. We also describe a more general construction for pure binary optimization problems where individual constraints are available through a membership oracle; if the intersection graph for the constraints has bounded tree-width our construction is of linear size and exact. This improves on a number of results in the literature, both from the perspective of formulation size and generality. △ Less

Submitted 18 October, 2016; v1 submitted 1 January, 2015; originally announced January 2015.

On linear relaxations of OPF problems

Authors: Daniel Bienstock, Gonzalo Munoz

Abstract: We present lifted linear relaxations of the OPF problem. We present lifted linear relaxations of the OPF problem. △ Less

Submitted 4 November, 2014; originally announced November 2014.

Comments: 1 figure

Efficient Synchronization Stability Metrics for Fault Clearing

Authors: Scott Backhaus, Russell Bent, Daniel Bienstock, Michael Chertkov, Dvijotham Krishnamurthy

Abstract: Direct methods can provide rapid screening of the dynamical security of large numbers fault and contingency scenarios by avoiding extensive time simulation. We introduce a computationally-efficient direct method based on optimization that leverages efficient cutting plane techniques. The method considers both unstable equilibrium points and the effects of additional relay trip** on dynamical sec… ▽ More Direct methods can provide rapid screening of the dynamical security of large numbers fault and contingency scenarios by avoiding extensive time simulation. We introduce a computationally-efficient direct method based on optimization that leverages efficient cutting plane techniques. The method considers both unstable equilibrium points and the effects of additional relay trip** on dynamical security\cite{01SH}. Similar to other direct methods, our approach yields conservative results for dynamical security, however, the optimization formulation potentially lends itself to the inclusion of additional constraints to reduce this conservatism. △ Less

Submitted 15 September, 2014; originally announced September 2014.

Comments: 7 pages, 4 figures

Report number: LA-UR-14-20227

A note on polynomial solvability of the CDT problem

Authors: Daniel Bienstock

Abstract: We describe a simple polynomial-time algorithm for the CDT problem that relies on a construction of Barvinok. We describe a simple polynomial-time algorithm for the CDT problem that relies on a construction of Barvinok. △ Less

Submitted 23 February, 2015; v1 submitted 24 June, 2014; originally announced June 2014.

Approximation Algorithms for the Incremental Knapsack Problem via Disjunctive Programming

Authors: Daniel Bienstock, Jay Sethuraman, Chun Ye

Abstract: In the incremental knapsack problem ($\IK$), we are given a knapsack whose capacity grows weakly as a function of time. There is a time horizon of $T$ periods and the capacity of the knapsack is $B_t$ in period $t$ for $t = 1, \ldots, T$. We are also given a set $S$ of $N$ items to be placed in the knapsack. Item $i$ has a value of $v_i$ and a weight of $w_i$ that is independent of the time period… ▽ More In the incremental knapsack problem ($\IK$), we are given a knapsack whose capacity grows weakly as a function of time. There is a time horizon of $T$ periods and the capacity of the knapsack is $B_t$ in period $t$ for $t = 1, \ldots, T$. We are also given a set $S$ of $N$ items to be placed in the knapsack. Item $i$ has a value of $v_i$ and a weight of $w_i$ that is independent of the time period. At any time period $t$, the sum of the weights of the items in the knapsack cannot exceed the knapsack capacity $B_t$. Moreover, once an item is placed in the knapsack, it cannot be removed from the knapsack at a later time period. We seek to maximize the sum of (discounted) knapsack values over time subject to the capacity constraints. We first give a constant factor approximation algorithm for $\IK$, under mild restrictions on the growth rate of $B_t$ (the constant factor depends on the growth rate). We then give a PTAS for $\IIK$, the special case of $\IK$ with no discounting, when $T = O(\sqrt{\log N})$. △ Less

Submitted 18 November, 2013; originally announced November 2013.

Comments: Key words: Approximation Algorithms, Integer Programming, Disjunctive Programming

Synchronization-Aware and Algorithm-Efficient Chance Constrained Optimal Power Flow

Authors: Russell Bent, Daniel Bienstock, Michael Chertkov

Abstract: One of the most common control decisions faced by power system operators is the question of how to dispatch generation to meet demand for power. This is a complex optimization problem that includes many nonlinear, non convex constraints as well as inherent uncertainties about future demand for power and available generation. In this paper we develop convex formulations to appropriately model cruci… ▽ More One of the most common control decisions faced by power system operators is the question of how to dispatch generation to meet demand for power. This is a complex optimization problem that includes many nonlinear, non convex constraints as well as inherent uncertainties about future demand for power and available generation. In this paper we develop convex formulations to appropriately model crucial classes of nonlinearities and stochastic effects. We focus on solving a nonlinear optimal power flow (OPF) problem that includes loss of synchrony constraints and models wind-farm caused fluctuations. In particular, we develop (a) a convex formulation of the deterministic phase-difference nonlinear Optimum Power Flow (OPF) problem; and (b) a probabilistic chance constrained OPF for angular stability, thermal overloads and generation limits that is computationally tractable. △ Less

Submitted 12 June, 2013; originally announced June 2013.

Comments: 11 pages, 3 figures

Chance Constrained Optimal Power Flow: Risk-Aware Network Control under Uncertainty

Authors: Daniel Bienstock, Michael Chertkov, Sean Harnett

Abstract: When uncontrollable resources fluctuate, Optimum Power Flow (OPF), routinely used by the electric power industry to re-dispatch hourly controllable generation (coal, gas and hydro plants) over control areas of transmission networks, can result in grid instability, and, potentially, cascading outages. This risk arises because OPF dispatch is computed without awareness of major uncertainty, in parti… ▽ More When uncontrollable resources fluctuate, Optimum Power Flow (OPF), routinely used by the electric power industry to re-dispatch hourly controllable generation (coal, gas and hydro plants) over control areas of transmission networks, can result in grid instability, and, potentially, cascading outages. This risk arises because OPF dispatch is computed without awareness of major uncertainty, in particular fluctuations in renewable output. As a result, grid operation under OPF with renewable variability can lead to frequent conditions where power line flow ratings are significantly exceeded. Such a condition, which is borne by simulations of real grids, would likely resulting in automatic line trip** to protect lines from thermal stress, a risky and undesirable outcome which compromises stability. Smart grid goals include a commitment to large penetration of highly fluctuating renewables, thus calling to reconsider current practices, in particular the use of standard OPF. Our Chance Constrained (CC) OPF corrects the problem and mitigates dangerous renewable fluctuations with minimal changes in the current operational procedure. Assuming availability of a reliable wind forecast parameterizing the distribution function of the uncertain generation, our CC-OPF satisfies all the constraints with high probability while simultaneously minimizing the cost of economic re-dispatch. CC-OPF allows efficient implementation, e.g. solving a typical instance over the 2746-bus Polish network in 20 seconds on a standard laptop. △ Less

Submitted 4 February, 2013; v1 submitted 25 September, 2012; originally announced September 2012.

Comments: 36 pages, 11 figures

Power Grid Vulnerability to Geographically Correlated Failures - Analysis and Control Implications

Authors: Andrey Bernstein, Daniel Bienstock, David Hay, Meric Uzunoglu, Gil Zussman

Abstract: We consider power line outages in the transmission system of the power grid, and specifically those caused by a natural disaster or a large scale physical attack. In the transmission system, an outage of a line may lead to overload on other lines, thereby eventually leading to their outage. While such cascading failures have been studied before, our focus is on cascading failures that follow an ou… ▽ More We consider power line outages in the transmission system of the power grid, and specifically those caused by a natural disaster or a large scale physical attack. In the transmission system, an outage of a line may lead to overload on other lines, thereby eventually leading to their outage. While such cascading failures have been studied before, our focus is on cascading failures that follow an outage of several lines in the same geographical area. We provide an analytical model of such failures, investigate the model's properties, and show that it differs from other models used to analyze cascades in the power grid (e.g., epidemic/percolation-based models). We then show how to identify the most vulnerable locations in the grid and perform extensive numerical experiments with real grid data to investigate the various effects of geographically correlated outages and the resulting cascades. These results allow us to gain insights into the relationships between various parameters and performance metrics, such as the size of the original event, the final number of connected components, and the fraction of demand (load) satisfied after the cascade. In particular, we focus on the timing and nature of optimal control actions used to reduce the impact of a cascade, in real time. We also compare results obtained by our model to the results of a real cascade that occurred during a major blackout in the San Diego area on Sept. 2011. The analysis and results presented in this paper will have implications both on the design of new power grids and on identifying the locations for shielding, strengthening, and monitoring efforts in grid upgrades. △ Less

Submitted 5 June, 2012; originally announced June 2012.

Report number: Columbia University, Electrical Engineering, Technical Report #2011-05-06, Nov. 2011

Optimizing convex functions over nonconvex sets

Authors: Daniel Bienstock, Alexander Michalka

Abstract: In this paper we derive strong linear inequalities for sets of the form {(x, q) \in Rd \times R : q \geq Q(x), x \in Rd - int(P)}, where Q(x) : Rd \rightarrow R is a quadratic function, P \subset Rd and "int" denotes interior. Of particular but not exclusive interest is the case where P denotes a closed convex set. In this paper, we present several cases where it is possible to characterize the co… ▽ More In this paper we derive strong linear inequalities for sets of the form {(x, q) \in Rd \times R : q \geq Q(x), x \in Rd - int(P)}, where Q(x) : Rd \rightarrow R is a quadratic function, P \subset Rd and "int" denotes interior. Of particular but not exclusive interest is the case where P denotes a closed convex set. In this paper, we present several cases where it is possible to characterize the convex hull by efficiently separable linear inequalities. △ Less

Submitted 3 August, 2019; v1 submitted 14 December, 2011; originally announced December 2011.

Comments: 9 pages

MSC Class: 90C26

Optimal adaptive control of cascading power grid failures

Authors: Daniel Bienstock

Abstract: We present theoretical results and experiments with parallel algorithms for computing an adaptive, online control with the objective of attenuating a power grid cascading failure. We present theoretical results and experiments with parallel algorithms for computing an adaptive, online control with the objective of attenuating a power grid cascading failure. △ Less

Submitted 17 December, 2010; originally announced December 2010.

Comments: 2 figures

MSC Class: 90; 93

The N-K Problem in Power Grids: New Models, Formulations and Numerical Experiments (extended version)

Authors: Daniel Bienstock, Abhinav Verma

Abstract: Given a power grid modeled by a network together with equations describing the power flows, power generation and consumption, and the laws of physics, the so-called N-k problem asks whether there exists a set of k or fewer arcs whose removal will cause the system to fail. The case where k is small is of practical interest. We present theoretical and computational results involving a mixed-intege… ▽ More Given a power grid modeled by a network together with equations describing the power flows, power generation and consumption, and the laws of physics, the so-called N-k problem asks whether there exists a set of k or fewer arcs whose removal will cause the system to fail. The case where k is small is of practical interest. We present theoretical and computational results involving a mixed-integer model and a continuous nonlinear model related to this question. △ Less

Submitted 31 December, 2009; v1 submitted 28 December, 2009; originally announced December 2009.

Comments: 40 pages 3 figures

MSC Class: 90-08