Showing 1–12 of 12 results for author: Biagioni, D

Generation Expansion Equilibria with Predictive Dispatch Model

Authors: Sourabh Dalvi, David Biagioni, Muhammad Bashar Anwar, Gord Stephen, Bethany Frew

Abstract: This paper proposes a methodology to solve generation expansion equilibrium problems by using a predictive model to represent the equilibrium in a simplified network constrained electricity market. The investment problem for each generation company (Genco) is a bi-level problem with the investment decision made in the upper level and market clearing condition in the lower level, which traditionall… ▽ More This paper proposes a methodology to solve generation expansion equilibrium problems by using a predictive model to represent the equilibrium in a simplified network constrained electricity market. The investment problem for each generation company (Genco) is a bi-level problem with the investment decision made in the upper level and market clearing condition in the lower level, which traditionally is represented as a Mathematical Program with Equilibrium Constraint (MPEC). The predictive model is trained for estimating the system-wide revenues for each technology type across energy, ancillary services and capacity markets given the amount of technology-specific installed capacity on the grid. The profit maximization investment problem for each Genco is solved using a global search algorithm, which uses the predictive model to evaluate the objective function. To solve for the strategic equilibrium, each Genco's problem is plugged into a diagonalization algorithm that is generally used in multi-leader, single-follower bi-level problems. The methodology presented here enables significant computational improvements while still capturing the desired market characteristics and dynamics of traditional equilibrium modeling approaches △ Less

Submitted 1 July, 2024; originally announced July 2024.

Non-Stationary Policy Learning for Multi-Timescale Multi-Agent Reinforcement Learning

Authors: Patrick Emami, Xiangyu Zhang, David Biagioni, Ahmed S. Zamzam

Abstract: In multi-timescale multi-agent reinforcement learning (MARL), agents interact across different timescales. In general, policies for time-dependent behaviors, such as those induced by multiple timescales, are non-stationary. Learning non-stationary policies is challenging and typically requires sophisticated or inefficient algorithms. Motivated by the prevalence of this control problem in real-worl… ▽ More In multi-timescale multi-agent reinforcement learning (MARL), agents interact across different timescales. In general, policies for time-dependent behaviors, such as those induced by multiple timescales, are non-stationary. Learning non-stationary policies is challenging and typically requires sophisticated or inefficient algorithms. Motivated by the prevalence of this control problem in real-world complex systems, we introduce a simple framework for learning non-stationary policies for multi-timescale MARL. Our approach uses available information about agent timescales to define a periodic time encoding. In detail, we theoretically demonstrate that the effects of non-stationarity introduced by multiple timescales can be learned by a periodic multi-agent policy. To learn such policies, we propose a policy gradient algorithm that parameterizes the actor and critic with phase-functioned neural networks, which provide an inductive bias for periodicity. The framework's ability to effectively learn multi-timescale policies is validated on a gridworld and building energy management environment. △ Less

Submitted 17 July, 2023; originally announced July 2023.

Comments: Accepted at IEEE CDC'23. 7 pages, 6 figures

Plug & Play Directed Evolution of Proteins with Gradient-based Discrete MCMC

Authors: Patrick Emami, Aidan Perreault, Jeffrey Law, David Biagioni, Peter C. St. John

Abstract: A long-standing goal of machine-learning-based protein engineering is to accelerate the discovery of novel mutations that improve the function of a known protein. We introduce a sampling framework for evolving proteins in silico that supports mixing and matching a variety of unsupervised models, such as protein language models, and supervised models that predict protein function from sequence. By… ▽ More A long-standing goal of machine-learning-based protein engineering is to accelerate the discovery of novel mutations that improve the function of a known protein. We introduce a sampling framework for evolving proteins in silico that supports mixing and matching a variety of unsupervised models, such as protein language models, and supervised models that predict protein function from sequence. By composing these models, we aim to improve our ability to evaluate unseen mutations and constrain search to regions of sequence space likely to contain functional proteins. Our framework achieves this without any model fine-tuning or re-training by constructing a product of experts distribution directly in discrete protein space. Instead of resorting to brute force search or random sampling, which is typical of classic directed evolution, we introduce a fast MCMC sampler that uses gradients to propose promising mutations. We conduct in silico directed evolution experiments on wide fitness landscapes and across a range of different pre-trained unsupervised models, including a 650M parameter protein language model. Our results demonstrate an ability to efficiently discover variants with high evolutionary likelihood as well as estimated activity multiple mutations away from a wild type protein, suggesting our sampler provides a practical and effective new paradigm for machine-learning-based protein engineering. △ Less

Submitted 6 April, 2023; v1 submitted 19 December, 2022; originally announced December 2022.

Comments: 31 pages, 8 figures. To appear in the Machine Learning: Science & Technology (ML:S&T) journal. Code is available at https://github.com/pemami4911/ppde. A short version of this work appeared at the NeurIPS 2022 Machine Learning in Structural Biology Workshop

From Model-Based to Model-Free: Learning Building Control for Demand Response

Authors: David Biagioni, Xiangyu Zhang, Christiane Adcock, Michael Sinner, Peter Graf, Jennifer King

Abstract: Grid-interactive building control is a challenging and important problem for reducing carbon emissions, increasing energy efficiency, and supporting the electric power grid. Currently researchers and practitioners are confronted with a choice of control strategies ranging from model-free (purely data-driven) to model-based (directly incorporating physical knowledge) to hybrid methods that combine… ▽ More Grid-interactive building control is a challenging and important problem for reducing carbon emissions, increasing energy efficiency, and supporting the electric power grid. Currently researchers and practitioners are confronted with a choice of control strategies ranging from model-free (purely data-driven) to model-based (directly incorporating physical knowledge) to hybrid methods that combine data and models. In this work, we identify state-of-the-art methods that span this methodological spectrum and evaluate their performance for multi-zone building HVAC control in the context of three demand response programs. We demonstrate, in this context, that hybrid methods offer many benefits over both purely model-free and model-based methods as long as certain requirements are met. In particular, hybrid controllers are relatively sample efficient, fast online, and high accuracy so long as the test case falls within the distribution of training data. Like all data-driven methods, hybrid controllers are still subject to generalization errors when applied to out-of-sample scenarios. Key takeaways for control strategies are summarized and the developed software framework is open-sourced. △ Less

Submitted 18 October, 2022; originally announced October 2022.

PowerGridworld: A Framework for Multi-Agent Reinforcement Learning in Power Systems

Authors: David Biagioni, Xiangyu Zhang, Dylan Wald, Deepthi Vaidhynathan, Rohit Chintala, Jennifer King, Ahmed S. Zamzam

Abstract: We present the PowerGridworld software package to provide users with a lightweight, modular, and customizable framework for creating power-systems-focused, multi-agent Gym environments that readily integrate with existing training frameworks for reinforcement learning (RL). Although many frameworks exist for training multi-agent RL (MARL) policies, none can rapidly prototype and develop the enviro… ▽ More We present the PowerGridworld software package to provide users with a lightweight, modular, and customizable framework for creating power-systems-focused, multi-agent Gym environments that readily integrate with existing training frameworks for reinforcement learning (RL). Although many frameworks exist for training multi-agent RL (MARL) policies, none can rapidly prototype and develop the environments themselves, especially in the context of heterogeneous (composite, multi-device) power systems where power flow solutions are required to define grid-level variables and costs. PowerGridworld is an open-source software package that helps to fill this gap. To highlight PowerGridworld's key features, we present two case studies and demonstrate learning MARL policies using both OpenAI's multi-agent deep deterministic policy gradient (MADDPG) and RLLib's proximal policy optimization (PPO) algorithms. In both cases, at least some subset of agents incorporates elements of the power flow solution at each time step as part of their reward (negative cost) structures. △ Less

Submitted 10 November, 2021; originally announced November 2021.

A Comparison of Model-Free and Model Predictive Control for Price Responsive Water Heaters

Authors: David J. Biagioni, Xiangyu Zhang, Peter Graf, Devon Sigler, Wesley Jones

Abstract: We present a careful comparison of two model-free control algorithms, Evolution Strategies (ES) and Proximal Policy Optimization (PPO), with receding horizon model predictive control (MPC) for operating simulated, price responsive water heaters. Four MPC variants are considered: a one-shot controller with perfect forecasting yielding optimal control; a limited-horizon controller with perfect forec… ▽ More We present a careful comparison of two model-free control algorithms, Evolution Strategies (ES) and Proximal Policy Optimization (PPO), with receding horizon model predictive control (MPC) for operating simulated, price responsive water heaters. Four MPC variants are considered: a one-shot controller with perfect forecasting yielding optimal control; a limited-horizon controller with perfect forecasting; a mean forecasting-based controller; and a two-stage stochastic programming controller using historical scenarios. In all cases, the MPC model for water temperature and electricity price are exact; only water demand is uncertain. For comparison, both ES and PPO learn neural network-based policies by directly interacting with the simulated environment under the same scenarios used by MPC. All methods are then evaluated on a separate one-week continuation of the demand time series. We demonstrate that optimal control for this problem is challenging, requiring more than 8-hour lookahead for MPC with perfect forecasting to attain the minimum cost. Despite this challenge, both ES and PPO learn good general purpose policies that outperform mean forecast and two-stage stochastic MPC controllers in terms of average cost and are more than two orders of magnitude faster at computing actions. We show that ES in particular can leverage parallelism to learn a policy in under 90 seconds using 1150 CPU cores. △ Less

Submitted 8 November, 2021; originally announced November 2021.

Comments: All authors are with the Computational Science Center at the National Renewable Energy Laboratory. Corresponding author: David Biagioni ([email protected])

Journal ref: In Proceedings of the 1st International Workshop on Reinforcement Learning for Energy Management in Buildings & Cities, pp. 29-33. 2020

A Modular and Transferable Reinforcement Learning Framework for the Fleet Rebalancing Problem

Authors: Erotokritos Skordilis, Yi Hou, Charles Tripp, Matthew Moniot, Peter Graf, David Biagioni

Abstract: Mobility on demand (MoD) systems show great promise in realizing flexible and efficient urban transportation. However, significant technical challenges arise from operational decision making associated with MoD vehicle dispatch and fleet rebalancing. For this reason, operators tend to employ simplified algorithms that have been demonstrated to work well in a particular setting. To help bridge the… ▽ More Mobility on demand (MoD) systems show great promise in realizing flexible and efficient urban transportation. However, significant technical challenges arise from operational decision making associated with MoD vehicle dispatch and fleet rebalancing. For this reason, operators tend to employ simplified algorithms that have been demonstrated to work well in a particular setting. To help bridge the gap between novel and existing methods, we propose a modular framework for fleet rebalancing based on model-free reinforcement learning (RL) that can leverage an existing dispatch method to minimize system cost. In particular, by treating dispatch as part of the environment dynamics, a centralized agent can learn to intermittently direct the dispatcher to reposition free vehicles and mitigate against fleet imbalance. We formulate RL state and action spaces as distributions over a grid partitioning of the operating area, making the framework scalable and avoiding the complexities associated with multiagent RL. Numerical experiments, using real-world trip and network data, demonstrate that this approach has several distinct advantages over baseline methods including: improved system cost; high degree of adaptability to the selected dispatch method; and the ability to perform scale-invariant transfer learning between problem instances with similar vehicle and request distributions. △ Less

Submitted 27 May, 2021; originally announced May 2021.

On the Computational Viability of Quantum Optimization for PMU Placement

Authors: Eric B. Jones, Eliot Kapit, Chin-Yao Chang, David Biagioni, Deepthi Vaidhynathan, Peter Graf, Wesley Jones

Abstract: Using optimal phasor measurement unit placement as a prototypical problem, we assess the computational viability of the current generation D-Wave Systems 2000Q quantum annealer for power systems design problems. We reformulate minimum dominating set for the annealer hardware, solve the reformulation for a standard set of IEEE test systems, and benchmark solution quality and time to solution agains… ▽ More Using optimal phasor measurement unit placement as a prototypical problem, we assess the computational viability of the current generation D-Wave Systems 2000Q quantum annealer for power systems design problems. We reformulate minimum dominating set for the annealer hardware, solve the reformulation for a standard set of IEEE test systems, and benchmark solution quality and time to solution against the CPLEX Optimizer and simulated annealing. For some problem instances the 2000Q outpaces CPLEX. For instances where the 2000Q underperforms with respect to CPLEX and simulated annealing, we suggest hardware improvements for the next generation of quantum annealers. △ Less

Submitted 13 January, 2020; originally announced January 2020.

Learning-Accelerated ADMM for Distributed Optimal Power Flow

Authors: David Biagioni, Peter Graf, Xiangyu Zhang, Ahmed Zamzam, Kyri Baker, Jennifer King

Abstract: We propose a novel data-driven method to accelerate the convergence of Alternating Direction Method of Multipliers (ADMM) for solving distributed DC optimal power flow (DC-OPF) where lines are shared between independent network partitions. Using previous observations of ADMM trajectories for a given system under varying load, the method trains a recurrent neural network (RNN) to predict the conver… ▽ More We propose a novel data-driven method to accelerate the convergence of Alternating Direction Method of Multipliers (ADMM) for solving distributed DC optimal power flow (DC-OPF) where lines are shared between independent network partitions. Using previous observations of ADMM trajectories for a given system under varying load, the method trains a recurrent neural network (RNN) to predict the converged values of dual and consensus variables. Given a new realization of system load, a small number of initial ADMM iterations is taken as input to infer the converged values and directly inject them into the iteration. We empirically demonstrate that the online injection of these values into the ADMM iteration accelerates convergence by a significant factor for partitioned 14-, 118- and 2848-bus test systems under differing load scenarios. The proposed method has several advantages: it maintains the security of private decision variables inherent in consensus ADMM; inference is fast and so may be used in online settings; RNN-generated predictions can dramatically improve time to convergence but, by construction, can never result in infeasible ADMM subproblems; it can be easily integrated into existing software implementations. While we focus on the ADMM formulation of distributed DC-OPF in this paper, the ideas presented are naturally extended to other distributed optimization problems. △ Less

Submitted 15 September, 2020; v1 submitted 7 November, 2019; originally announced November 2019.

Synthesis of a mixed-valent tin nitride and considerations of its possible crystal structures

Authors: Christopher M. Caskey, Aaron Holder, Sarah Shulda, Steve Christensen, David Diercks, Craig P. Schwartz, David Biagioni, Dennis Nordlund, Alon Kukliansky, Amir Natan, David Prendergast, Bernardo Orvananos, Wenhao Sun, Xiuwen Zhang, Gerbrand Ceder, William Tumas, David S. Ginley, John D. Perkins, Vladan Stevanovic, Svitlana Pylypenko, Stephan Lany, Ryan M. Richards, Andriy Zakutayev

Abstract: Recent advances in theoretical structure prediction methods and high-throughput computational techniques are revolutionizing experimental discovery of the thermodynamically stable inorganic materials. Metastable materials represent a new frontier for studies, since even simple binary non ground state compounds of common elements may be awaiting discovery. However, there are significant research ch… ▽ More Recent advances in theoretical structure prediction methods and high-throughput computational techniques are revolutionizing experimental discovery of the thermodynamically stable inorganic materials. Metastable materials represent a new frontier for studies, since even simple binary non ground state compounds of common elements may be awaiting discovery. However, there are significant research challenges related to non-equilibrium thin film synthesis and crystal structure predictions, such as small strained crystals in the experimental samples and energy minimization based theoretical algorithms. Here we report on experimental synthesis and characterization, as well as theoretical first-principles calculations of a previously unreported mixed-valent binary tin nitride. Thin film experiments indicate that this novel material is N-deficient SnN with tin in the mixed II/IV valence state and a small low-symmetry unit cell. Theoretical calculations suggest that the most likely crystal structure has the space group 2 (SG2) related to the distorted delafossite (SG166), which is nearly 0.1 eV/atom above the ground state SnN polymorph. This observation is rationalized by the structural similarity of the SnN distorted delafossite to the chemically related Sn3N4 spinel compound, which provides a fresh scientific insight into the reasons for growth of polymorphs of the metastable material. In addition to reporting on the discovery of the simple binary SnN compound, this paper illustrates a possible way of combining a wide range of advanced characterization techniques with the first-principle property calculation methods, to elucidate the most likely crystal structure of the previously unreported metastable materials. △ Less

Submitted 18 January, 2016; originally announced January 2016.

Randomized Interpolative Decomposition of Separated Representations

Authors: David J. Biagioni, Daniel Beylkin, Gregory Beylkin

Abstract: We introduce tensor Interpolative Decomposition (tensor ID) for the reduction of the separation rank of Canonical Tensor Decompositions (CTDs). Tensor ID selects, for a user-defined accuracy ε, a near optimal subset of terms of a CTD to represent the remaining terms via a linear combination of the selected terms. Tensor ID can be used as an alternative to or a step of the Alternating Least Squares… ▽ More We introduce tensor Interpolative Decomposition (tensor ID) for the reduction of the separation rank of Canonical Tensor Decompositions (CTDs). Tensor ID selects, for a user-defined accuracy ε, a near optimal subset of terms of a CTD to represent the remaining terms via a linear combination of the selected terms. Tensor ID can be used as an alternative to or a step of the Alternating Least Squares (ALS) algorithm. In addition, we briefly discuss Q-factorization to reduce the size of components within an ALS iteration. Combined, tensor ID and Q-factorization lead to a new paradigm for the reduction of the separation rank of CTDs. In this context, we also discuss the spectral norm as a computational alternative to the Frobenius norm. We reduce the problem of finding tensor IDs to that of constructing Interpolative Decompositions of certain matrices. These matrices are generated via either randomized projection or randomized sampling of the given tensor. We provide cost estimates and several examples of the new approach to the reduction of separation rank. △ Less

Submitted 16 December, 2013; v1 submitted 20 June, 2013; originally announced June 2013.

Kee** greed good: sparse regression under design uncertainty with application to biomass characterization

Authors: David J. Biagioni, Ryan Elmore, Wesley Jones

Abstract: In this paper, we consider the classic measurement error regression scenario in which our independent, or design, variables are observed with several sources of additive noise. We will show that our motivating example's replicated measurements on both the design and dependent variables may be leveraged to enhance a sparse regression algorithm. Specifically, we estimate the variance and use it to s… ▽ More In this paper, we consider the classic measurement error regression scenario in which our independent, or design, variables are observed with several sources of additive noise. We will show that our motivating example's replicated measurements on both the design and dependent variables may be leveraged to enhance a sparse regression algorithm. Specifically, we estimate the variance and use it to scale our design variables. We demonstrate the efficacy of scaling from several points of view and validate it empirically with a biomass characterization data set using two of the most widely used sparse algorithms: least angle regression (LARS) and the Dantzig selector (DS). △ Less

Submitted 8 July, 2012; originally announced July 2012.

Comments: 22 pages, 5 figures