-
Constrained Stein Variational Trajectory Optimization
Authors:
Thomas Power,
Dmitry Berenson
Abstract:
We present Constrained Stein Variational Trajectory Optimization (CSVTO), an algorithm for performing trajectory optimization with constraints on a set of trajectories in parallel. We frame constrained trajectory optimization as a novel form of constrained functional minimization over trajectory distributions, which avoids treating the constraints as a penalty in the objective and allows us to gen…
▽ More
We present Constrained Stein Variational Trajectory Optimization (CSVTO), an algorithm for performing trajectory optimization with constraints on a set of trajectories in parallel. We frame constrained trajectory optimization as a novel form of constrained functional minimization over trajectory distributions, which avoids treating the constraints as a penalty in the objective and allows us to generate diverse sets of constraint-satisfying trajectories. Our method uses Stein Variational Gradient Descent (SVGD) to find a set of particles that approximates a distribution over low-cost trajectories while obeying constraints. CSVTO is applicable to problems with arbitrary equality and inequality constraints and includes a novel particle resampling step to escape local minima. By explicitly generating diverse sets of trajectories, CSVTO is better able to avoid poor local minima and is more robust to initialization. We demonstrate that CSVTO outperforms baselines in challenging highly-constrained tasks, such as a 7DoF wrench manipulation task, where CSVTO succeeds in 20/20 trials vs 13/20 for the closest baseline. Our results demonstrate that generating diverse constraint-satisfying trajectories improves robustness to disturbances and initialization over baselines.
△ Less
Submitted 23 December, 2023; v1 submitted 23 August, 2023;
originally announced August 2023.
-
Variational Inference MPC using Normalizing Flows and Out-of-Distribution Projection
Authors:
Thomas Power,
Dmitry Berenson
Abstract:
We propose a Model Predictive Control (MPC) method for collision-free navigation that uses amortized variational inference to approximate the distribution of optimal control sequences by training a normalizing flow conditioned on the start, goal and environment. This representation allows us to learn a distribution that accounts for both the dynamics of the robot and complex obstacle geometries. W…
▽ More
We propose a Model Predictive Control (MPC) method for collision-free navigation that uses amortized variational inference to approximate the distribution of optimal control sequences by training a normalizing flow conditioned on the start, goal and environment. This representation allows us to learn a distribution that accounts for both the dynamics of the robot and complex obstacle geometries. We can then sample from this distribution to produce control sequences which are likely to be both goal-directed and collision-free as part of our proposed FlowMPPI sampling-based MPC method. However, when deploying this method, the robot may encounter an out-of-distribution (OOD) environment, i.e. one which is radically different from those used in training. In such cases, the learned flow cannot be trusted to produce low-cost control sequences. To generalize our method to OOD environments we also present an approach that performs projection on the representation of the environment as part of the MPC process. This projection changes the environment representation to be more in-distribution while also optimizing trajectory quality in the true environment. Our simulation results on a 2D double-integrator and a 3D 12DoF underactuated quadrotor suggest that FlowMPPI with projection outperforms state-of-the-art MPC baselines on both in-distribution and OOD environments, including OOD environments generated from real-world data.
△ Less
Submitted 10 May, 2022;
originally announced May 2022.
-
Keep it Simple: Data-efficient Learning for Controlling Complex Systems with Simple Models
Authors:
Thomas Power,
Dmitry Berenson
Abstract:
When manipulating a novel object with complex dynamics, a state representation is not always available, for example for deformable objects. Learning both a representation and dynamics from observations requires large amounts of data. We propose Learned Visual Similarity Predictive Control (LVSPC), a novel method for data-efficient learning to control systems with complex dynamics and high-dimensio…
▽ More
When manipulating a novel object with complex dynamics, a state representation is not always available, for example for deformable objects. Learning both a representation and dynamics from observations requires large amounts of data. We propose Learned Visual Similarity Predictive Control (LVSPC), a novel method for data-efficient learning to control systems with complex dynamics and high-dimensional state spaces from images. LVSPC leverages a given simple model approximation from which image observations can be generated. We use these images to train a perception model that estimates the simple model state from observations of the complex system online. We then use data from the complex system to fit the parameters of the simple model and learn where this model is inaccurate, also online. Finally, we use Model Predictive Control and bias the controller away from regions where the simple model is inaccurate and thus where the controller is less reliable. We evaluate LVSPC on two tasks; manipulating a tethered mass and a rope. We find that our method performs comparably to state-of-the-art reinforcement learning methods with an order of magnitude less data. LVSPC also completes the rope manipulation task on a real robot with 80% success rate after only 10 trials, despite using a perception system trained only on images from simulation.
△ Less
Submitted 17 February, 2021; v1 submitted 4 February, 2021;
originally announced February 2021.
-
Learning When to Trust a Dynamics Model for Planning in Reduced State Spaces
Authors:
Dale McConachie,
Thomas Power,
Peter Mitrano,
Dmitry Berenson
Abstract:
When the dynamics of a system are difficult to model and/or time-consuming to evaluate, such as in deformable object manipulation tasks, motion planning algorithms struggle to find feasible plans efficiently. Such problems are often reduced to state spaces where the dynamics are straightforward to model and evaluate. However, such reductions usually discard information about the system for the ben…
▽ More
When the dynamics of a system are difficult to model and/or time-consuming to evaluate, such as in deformable object manipulation tasks, motion planning algorithms struggle to find feasible plans efficiently. Such problems are often reduced to state spaces where the dynamics are straightforward to model and evaluate. However, such reductions usually discard information about the system for the benefit of computational efficiency, leading to cases where the true and reduced dynamics disagree on the result of an action. This paper presents a formulation for planning in reduced state spaces that uses a classifier to bias the planner away from state-action pairs that are not reliably feasible under the true dynamics. We present a method to generate and label data to train such a classifier, as well as an application of our framework to rope manipulation, where we use a Virtual Elastic Band (VEB) approximation to the true dynamics. Our experiments with rope manipulation demonstrate that the classifier significantly improves the success rate of our RRT-based planner in several difficult scenarios which are designed to cause the VEB to produce incorrect predictions in key parts of the environment.
△ Less
Submitted 29 January, 2020;
originally announced January 2020.
-
A Novel Probabilistic Framework to Study the Impact of PV-battery Systems on Low-Voltage Distribution Networks
Authors:
Yiju Ma,
Donald Azuatalam,
Thomas Power,
Gregor Verbic,
Archie C. Chapman
Abstract:
Battery storage, particularly residential battery storage coupled with rooftop PV, is emerging as an essential component of the smart grid technology mix. However, including battery storage and other flexible resources like electric vehicles and loads with thermal inertia into a probabilistic analysis based on Monte Carlo (MC) simulation is challenging, because their operational profiles are deter…
▽ More
Battery storage, particularly residential battery storage coupled with rooftop PV, is emerging as an essential component of the smart grid technology mix. However, including battery storage and other flexible resources like electric vehicles and loads with thermal inertia into a probabilistic analysis based on Monte Carlo (MC) simulation is challenging, because their operational profiles are determined by computationally intensive optimization. Additionally, MC analysis requires a large pool of statistically-representative demand profiles to sample from. As a result, the analysis of the network impact of PV-battery systems has attracted little attention in the existing literature. To fill these knowledge gaps, this paper proposes a novel probabilistic framework to study the impact of PV-battery systems on low-voltage distribution networks. Specifically, the framework incorporates home energy management(HEM) operational decisions within the MC time series power flow analysis. First, using available smart meter data, we use a Bayesian nonparametric model to generate statistically-representative synthetic demand and PV profiles. Second, a policy function approximation that emulates battery scheduling decisions is used to make the simulation of optimization-based HEM feasible within the MC framework. The efficacy of our method is demonstrated on three representative low-voltage feeders, where the computation time to execute our MC framework is 5% of that when using explicit optimization methods in each MC sample. The assessment results show that uncoordinated battery scheduling has a limited beneficial impact, which is against the conjecture that batteries will serendipitously mitigate the technical problems induced by PV generation.
△ Less
Submitted 12 April, 2019; v1 submitted 20 September, 2018;
originally announced September 2018.
-
A Nonparametric Bayesian Methodology for Synthesizing Residential Solar Generation and Demand Data
Authors:
Thomas Power,
Gregor Verbič,
Archie C. Chapman
Abstract:
The uptake of behind-the-meter distributed energy resources in low-voltage distribution networks has reached a level where network issues have started to emerge, which requires new tools for operation and planning. In this paper, we propose a methodology for synthesizing stochastic demand and generation profiles for unobserved customers with rooftop PV, called prosumers. The proposed model bridges…
▽ More
The uptake of behind-the-meter distributed energy resources in low-voltage distribution networks has reached a level where network issues have started to emerge, which requires new tools for operation and planning. In this paper, we propose a methodology for synthesizing stochastic demand and generation profiles for unobserved customers with rooftop PV, called prosumers. The proposed model bridges the gap between the limited available empirical data, and the large amount of high-quality, stochastic demand and generation data required for probabilistic analysis. The approach employs clustering analysis and a Dirichlet-categorical hierarchical model of the features of unobserved prosumers. Based on the data of clusters of prosumers, Markov chain models of demand and generation profiles are constructed from empirical data, and synthetic demand profiles are subsequently sampled from these. The sampled traces are cross-validated and show a good statistical fit to the observed data. Two case studies are considered to confirm the validity of the proposed methodology. The first studies the impact of behavioral differences on the synthetic demand profiles, while the second looks at the impact of varying solar generation penetration on demand profiles.
△ Less
Submitted 18 December, 2018; v1 submitted 1 August, 2018;
originally announced August 2018.
