Showing 1–13 of 13 results for author: Carlberg, K

Accelerating Look-ahead in Bayesian Optimization: Multilevel Monte Carlo is All you Need

Authors: Shangda Yang, Vitaly Zankin, Maximilian Balandat, Stefan Scherer, Kevin Carlberg, Neil Walton, Kody J. H. Law

Abstract: We leverage multilevel Monte Carlo (MLMC) to improve the performance of multi-step look-ahead Bayesian optimization (BO) methods that involve nested expectations and maximizations. Often these expectations must be computed by Monte Carlo (MC). The complexity rate of naive MC degrades for nested operations, whereas MLMC is capable of achieving the canonical MC convergence rate for this type of prob… ▽ More We leverage multilevel Monte Carlo (MLMC) to improve the performance of multi-step look-ahead Bayesian optimization (BO) methods that involve nested expectations and maximizations. Often these expectations must be computed by Monte Carlo (MC). The complexity rate of naive MC degrades for nested operations, whereas MLMC is capable of achieving the canonical MC convergence rate for this type of problem, independently of dimension and without any smoothness assumptions. Our theoretical study focuses on the approximation improvements for twoand three-step look-ahead acquisition functions, but, as we discuss, the approach is generalizable in various ways, including beyond the context of BO. Our findings are verified numerically and the benefits of MLMC for BO are illustrated on several benchmark examples. Code is available at https://github.com/Shangda-Yang/MLMCBO . △ Less

Submitted 25 June, 2024; v1 submitted 3 February, 2024; originally announced February 2024.

Comments: Preprint ICML 2024

Neural Stress Fields for Reduced-order Elastoplasticity and Fracture

Authors: Zeshun Zong, Xuan Li, Minchen Li, Maurizio M. Chiaramonte, Wojciech Matusik, Eitan Grinspun, Kevin Carlberg, Chenfanfu Jiang, Peter Yichen Chen

Abstract: We propose a hybrid neural network and physics framework for reduced-order modeling of elastoplasticity and fracture. State-of-the-art scientific computing models like the Material Point Method (MPM) faithfully simulate large-deformation elastoplasticity and fracture mechanics. However, their long runtime and large memory consumption render them unsuitable for applications constrained by computati… ▽ More We propose a hybrid neural network and physics framework for reduced-order modeling of elastoplasticity and fracture. State-of-the-art scientific computing models like the Material Point Method (MPM) faithfully simulate large-deformation elastoplasticity and fracture mechanics. However, their long runtime and large memory consumption render them unsuitable for applications constrained by computation time and memory usage, e.g., virtual reality. To overcome these barriers, we propose a reduced-order framework. Our key innovation is training a low-dimensional manifold for the Kirchhoff stress field via an implicit neural representation. This low-dimensional neural stress field (NSF) enables efficient evaluations of stress values and, correspondingly, internal forces at arbitrary spatial locations. In addition, we also train neural deformation and affine fields to build low-dimensional manifolds for the deformation and affine momentum fields. These neural stress, deformation, and affine fields share the same low-dimensional latent space, which uniquely embeds the high-dimensional simulation state. After training, we run new simulations by evolving in this single latent space, which drastically reduces the computation time and memory consumption. Our general continuum-mechanics-based reduced-order framework is applicable to any phenomena governed by the elastodynamics equation. To showcase the versatility of our framework, we simulate a wide range of material behaviors, including elastica, sand, metal, non-Newtonian fluids, fracture, contact, and collision. We demonstrate dimension reduction by up to 100,000X and time savings by up to 10X. △ Less

Submitted 26 October, 2023; originally announced October 2023.

LiCROM: Linear-Subspace Continuous Reduced Order Modeling with Neural Fields

Authors: Yue Chang, Peter Yichen Chen, Zhecheng Wang, Maurizio M. Chiaramonte, Kevin Carlberg, Eitan Grinspun

Abstract: Linear reduced-order modeling (ROM) simplifies complex simulations by approximating the behavior of a system using a simplified kinematic representation. Typically, ROM is trained on input simulations created with a specific spatial discretization, and then serves to accelerate simulations with the same discretization. This discretization-dependence is restrictive. Becoming independent of a spec… ▽ More Linear reduced-order modeling (ROM) simplifies complex simulations by approximating the behavior of a system using a simplified kinematic representation. Typically, ROM is trained on input simulations created with a specific spatial discretization, and then serves to accelerate simulations with the same discretization. This discretization-dependence is restrictive. Becoming independent of a specific discretization would provide flexibility to mix and match mesh resolutions, connectivity, and type (tetrahedral, hexahedral) in training data; to accelerate simulations with novel discretizations unseen during training; and to accelerate adaptive simulations that temporally or parametrically change the discretization. We present a flexible, discretization-independent approach to reduced-order modeling. Like traditional ROM, we represent the configuration as a linear combination of displacement fields. Unlike traditional ROM, our displacement fields are continuous maps from every point on the reference domain to a corresponding displacement vector; these maps are represented as implicit neural fields. With linear continuous ROM (LiCROM), our training set can include multiple geometries undergoing multiple loading conditions, independent of their discretization. This opens the door to novel applications of reduced order modeling. We can now accelerate simulations that modify the geometry at runtime, for instance via cutting, hole punching, and even swap** the entire mesh. We can also accelerate simulations of geometries unseen during training. We demonstrate one-shot generalization, training on a single geometry and subsequently simulating various unseen geometries. △ Less

Submitted 24 October, 2023; originally announced October 2023.

Learning a Visually Grounded Memory Assistant

Authors: Meera Hahn, Kevin Carlberg, Ruta Desai, James Hillis

Abstract: We introduce a novel interface for large scale collection of human memory and assistance. Using the 3D Matterport simulator we create a realistic indoor environments in which we have people perform specific embodied memory tasks that mimic household daily activities. This interface was then deployed on Amazon Mechanical Turk allowing us to test and record human memory, navigation and needs for ass… ▽ More We introduce a novel interface for large scale collection of human memory and assistance. Using the 3D Matterport simulator we create a realistic indoor environments in which we have people perform specific embodied memory tasks that mimic household daily activities. This interface was then deployed on Amazon Mechanical Turk allowing us to test and record human memory, navigation and needs for assistance at a large scale that was previously impossible. Using the interface we collect the `The Visually Grounded Memory Assistant Dataset' which is aimed at develo** our understanding of (1) the information people encode during navigation of 3D environments and (2) conditions under which people ask for memory assistance. Additionally we experiment with with predicting when people will ask for assistance using models trained on hand-selected visual and semantic features. This provides an opportunity to build stronger ties between the machine-learning and cognitive-science communities through learned models of human perception, memory, and cognition. △ Less

Submitted 7 October, 2022; originally announced October 2022.

CROM: Continuous Reduced-Order Modeling of PDEs Using Implicit Neural Representations

Authors: Peter Yichen Chen, **xu Xiang, Dong Heon Cho, Yue Chang, G A Pershing, Henrique Teles Maia, Maurizio M. Chiaramonte, Kevin Carlberg, Eitan Grinspun

Abstract: The long runtime of high-fidelity partial differential equation (PDE) solvers makes them unsuitable for time-critical applications. We propose to accelerate PDE solvers using reduced-order modeling (ROM). Whereas prior ROM approaches reduce the dimensionality of discretized vector fields, our continuous reduced-order modeling (CROM) approach builds a low-dimensional embedding of the continuous vec… ▽ More The long runtime of high-fidelity partial differential equation (PDE) solvers makes them unsuitable for time-critical applications. We propose to accelerate PDE solvers using reduced-order modeling (ROM). Whereas prior ROM approaches reduce the dimensionality of discretized vector fields, our continuous reduced-order modeling (CROM) approach builds a low-dimensional embedding of the continuous vector fields themselves, not their discretization. We represent this reduced manifold using continuously differentiable neural fields, which may train on any and all available numerical solutions of the continuous system, even when they are obtained using diverse methods or discretizations. We validate our approach on an extensive range of PDEs with training data from voxel grids, meshes, and point clouds. Compared to prior discretization-dependent ROM methods, such as linear subspace proper orthogonal decomposition (POD) and nonlinear manifold neural-network-based autoencoders, CROM features higher accuracy, lower memory consumption, dynamically adaptive resolutions, and applicability to any discretization. For equal latent space dimension, CROM exhibits 79$\times$ and 49$\times$ better accuracy, and 39$\times$ and 132$\times$ smaller memory footprint, than POD and autoencoder methods, respectively. Experiments demonstrate 109$\times$ and 89$\times$ wall-clock speedups over unreduced models on CPUs and GPUs, respectively. Videos and codes are available on the project page: https://crom-pde.github.io △ Less

Submitted 3 March, 2023; v1 submitted 6 June, 2022; originally announced June 2022.

Preconditioned Least-Squares Petrov-Galerkin Reduced Order Models

Authors: Payton Lindsay, Jeffrey Fike, Irina Tezaur, Kevin Carlberg

Abstract: This paper introduces a methodology for improving the accuracy and efficiency of reduced order models (ROMs) constructed using the least-squares Petrov-Galerkin (LSPG) projection method through the introduction of preconditioning. Unlike prior related work, which focuses on preconditioning the linear systems arising within the ROM numerical solution procedure to improve linear solver performance,… ▽ More This paper introduces a methodology for improving the accuracy and efficiency of reduced order models (ROMs) constructed using the least-squares Petrov-Galerkin (LSPG) projection method through the introduction of preconditioning. Unlike prior related work, which focuses on preconditioning the linear systems arising within the ROM numerical solution procedure to improve linear solver performance, our approach leverages a preconditioning matrix directly within the LSPG minimization problem. Applying preconditioning in this way can improve ROM accuracy for several reasons. First, preconditioning the LSPG formulation changes the norm defining the residual minimization, which can improve the residual-based stability constant bounding the ROM solution's error. The incorporation of a preconditioner into the LSPG formulation can have the additional effect of scaling the components of the residual being minimized, which can be beneficial for problems with disparate scales. Importantly, we demonstrate that an 'ideal preconditioned' LSPG ROM (a ROM preconditioned with the inverse of the Jacobian of its corresponding full order model, or FOM) emulates projection of the FOM solution increment onto the reduced basis, a lower bound on the ROM solution error for a given reduced basis. By designing preconditioners that approximate the Jacobian inverse, a ROM whose error approaches this lower bound can be obtained. The proposed approach is evaluated in the predictive regime on several mechanical and thermo-mechanical problems within the Albany HPC code. We demonstrate numerically that the introduction of simple Jacobi, Gauss-Seidel and ILU preconditioners into the Proper Orthogonal Decomposition/LSPG formulation reduces significantly the ROM solution error, the reduced Jacobian condition number, the number of nonlinear iterations required to reach convergence, and the wall time. △ Less

Submitted 22 March, 2022; originally announced March 2022.

Model reduction for the material point method via an implicit neural representation of the deformation map

Authors: Peter Yichen Chen, Maurizio M. Chiaramonte, Eitan Grinspun, Kevin Carlberg

Abstract: This work proposes a model-reduction approach for the material point method on nonlinear manifolds. Our technique approximates the $\textit{kinematics}$ by approximating the deformation map using an implicit neural representation that restricts deformation trajectories to reside on a low-dimensional manifold. By explicitly approximating the deformation map, its spatiotemporal gradients -- in parti… ▽ More This work proposes a model-reduction approach for the material point method on nonlinear manifolds. Our technique approximates the $\textit{kinematics}$ by approximating the deformation map using an implicit neural representation that restricts deformation trajectories to reside on a low-dimensional manifold. By explicitly approximating the deformation map, its spatiotemporal gradients -- in particular the deformation gradient and the velocity -- can be computed via analytical differentiation. In contrast to typical model-reduction techniques that construct a linear or nonlinear manifold to approximate the (finite number of) degrees of freedom characterizing a given spatial discretization, the use of an implicit neural representation enables the proposed method to approximate the $\textit{continuous}$ deformation map. This allows the kinematic approximation to remain agnostic to the discretization. Consequently, the technique supports dynamic discretizations -- including resolution changes -- during the course of the online reduced-order-model simulation. To generate $\textit{dynamics}$ for the generalized coordinates, we propose a family of projection techniques. At each time step, these techniques: (1) Calculate full-space kinematics at quadrature points, (2) Calculate the full-space dynamics for a subset of `sample' material points, and (3) Calculate the reduced-space dynamics by projecting the updated full-space position and velocity onto the low-dimensional manifold and tangent space, respectively. We achieve significant computational speedup via hyper-reduction that ensures all three steps execute on only a small subset of the problem's spatial domain. Large-scale numerical examples with millions of material points illustrate the method's ability to gain an order of magnitude computational-cost saving -- indeed $\textit{real-time simulations}$ -- with negligible errors. △ Less

Submitted 9 February, 2023; v1 submitted 25 September, 2021; originally announced September 2021.

Projection-tree reduced order modeling for fast N-body computations

Authors: Steven N. Rodriguez, Athanasios P. Iliopoulos, Kevin T. Carlberg, Steven L. Brunton, John C. Steuben, John G. Michopoulos

Abstract: This work presents a data-driven reduced-order modeling framework to accelerate the computations of $N$-body dynamical systems and their pair-wise interactions. The proposed framework differs from traditional acceleration methods, like the Barnes-Hut method, which requires online tree building of the state space, or the fast-multipole method, which requires rigorous $a$ $priori$ analysis of govern… ▽ More This work presents a data-driven reduced-order modeling framework to accelerate the computations of $N$-body dynamical systems and their pair-wise interactions. The proposed framework differs from traditional acceleration methods, like the Barnes-Hut method, which requires online tree building of the state space, or the fast-multipole method, which requires rigorous $a$ $priori$ analysis of governing kernels and online tree building. Our approach combines Barnes-Hut hierarchical decomposition, dimensional compression via the least-squares Petrov-Galerkin (LSPG) projection, and hyper-reduction by way of the Gauss-Newton with approximated tensor (GNAT) approach. The resulting $projection-tree$ reduced order model (PTROM) enables a drastic reduction in operational count complexity by constructing sparse hyper-reduced pairwise interactions of the $N$-body dynamical system. As a result, the presented framework is capable of achieving an operational count complexity that is independent of $N$, the number of bodies in the numerical domain. Capabilities of the PTROM method are demonstrated on the two-dimensional fluid-dynamic Biot-Savart kernel within a parametric and reproductive setting. Results show the PTROM is capable of achieving over 2000$\times$ wall-time speed-up with respect to the full-order model, where the speed-up increases with $N$. The resulting solution delivers quantities of interest with errors that are less than 0.1$\%$ with respect to full-order model. △ Less

Submitted 17 May, 2021; v1 submitted 2 March, 2021; originally announced March 2021.

Preserving general physical properties in model reduction of dynamical systems via constrained-optimization projection

Authors: A. Schein, K. T. Carlberg, M. J. Zahr

Abstract: Model-reduction techniques aim to reduce the computational complexity of simulating dynamical systems by applying a (Petrov-)Galerkin projection process that enforces the dynamics to evolve in a low-dimensional subspace of the original state space. Frequently, the resulting reduced-order model (ROM) violates intrinsic physical properties of the original full-order model (FOM) (e.g., global conserv… ▽ More Model-reduction techniques aim to reduce the computational complexity of simulating dynamical systems by applying a (Petrov-)Galerkin projection process that enforces the dynamics to evolve in a low-dimensional subspace of the original state space. Frequently, the resulting reduced-order model (ROM) violates intrinsic physical properties of the original full-order model (FOM) (e.g., global conservation, Lagrangian structure, state-variable bounds) because the projection process does not generally ensure preservation of these properties. However, in many applications, ensuring the ROM preserves such intrinsic properties can enable the ROM to retain physical meaning and lead to improved accuracy and stability properties. In this work, we present a general constrained-optimization formulation for projection-based model reduction that can be used as a template to enforce the ROM to satisfy specific properties on the kinematics and dynamics. We introduce constrained-optimization formulations at both the time-continuous (i.e., ODE) level, which leads to a constrained Galerkin projection, and at the time-discrete level, which leads to a least-squares Petrov-Galerkin (LSPG) projection, in the context of linear multistep schemes. We demonstrate the ability of the proposed formulations to equip ROMs with desired properties such as global energy conservation and bounds on the total variation. △ Less

Submitted 1 April, 2021; v1 submitted 27 November, 2020; originally announced November 2020.

Optimal Assistance for Object-Rearrangement Tasks in Augmented Reality

Authors: Benjamin Newman, Kevin Carlberg, Ruta Desai

Abstract: Augmented-reality (AR) glasses that will have access to onboard sensors and an ability to display relevant information to the user present an opportunity to provide user assistance in quotidian tasks. Many such tasks can be characterized as object-rearrangement tasks. We introduce a novel framework for computing and displaying AR assistance that consists of (1) associating an optimal action sequen… ▽ More Augmented-reality (AR) glasses that will have access to onboard sensors and an ability to display relevant information to the user present an opportunity to provide user assistance in quotidian tasks. Many such tasks can be characterized as object-rearrangement tasks. We introduce a novel framework for computing and displaying AR assistance that consists of (1) associating an optimal action sequence with the policy of an embodied agent and (2) presenting this sequence to the user as suggestions in the AR system's heads-up display. The embodied agent comprises a "hybrid" between the AR system and the user, with the AR system's observation space (i.e., sensors) and the user's action space (i.e., task-execution actions); its policy is learned by minimizing the task-completion time. In this initial study, we assume that the AR system's observations include the environment's map and localization of the objects and the user. These choices allow us to formalize the problem of computing AR assistance for any object-rearrangement task as a planning problem, specifically as a capacitated vehicle-routing problem. Further, we introduce a novel AR simulator that can enable web-based evaluation of AR-like assistance and associated at-scale data collection via the Habitat simulator for embodied artificial intelligence. Finally, we perform a study that evaluates user response to the proposed form of AR assistance on a specific quotidian object-rearrangement task, house cleaning, using our proposed AR simulator on mechanical turk. In particular, we study the effect of the proposed AR assistance on users' task performance and sense of agency over a range of task difficulties. Our results indicate that providing users with such assistance improves their overall performance and while users report a negative impact to their agency, they may still prefer the proposed assistance to having no assistance at all. △ Less

Submitted 14 October, 2020; originally announced October 2020.

Comments: 19 pages including supplementary. Under review for ACM IUI 2021

Pressio: Enabling projection-based model reduction for large-scale nonlinear dynamical systems

Authors: Francesco Rizzi, Patrick J. Blonigan, Eric J. Parish, Kevin T. Carlberg

Abstract: This work introduces Pressio, an open-source project aimed at enabling leading-edge projection-based reduced order models (ROMs) for large-scale nonlinear dynamical systems in science and engineering. Pressio provides model-reduction methods that can reduce both the number of spatial and temporal degrees of freedom for any dynamical system expressible as a system of parameterized ordinary differen… ▽ More This work introduces Pressio, an open-source project aimed at enabling leading-edge projection-based reduced order models (ROMs) for large-scale nonlinear dynamical systems in science and engineering. Pressio provides model-reduction methods that can reduce both the number of spatial and temporal degrees of freedom for any dynamical system expressible as a system of parameterized ordinary differential equations (ODEs). We leverage this simple, expressive mathematical framework as a pivotal design choice to enable a minimal application programming interface (API) that is natural to dynamical systems. The core component of Pressio is a C++11 header-only library that leverages generic programming to support applications with arbitrary data types and arbitrarily complex programming models. This is complemented with Python bindings to expose these C++ functionalities to Python users with negligible overhead and no user-required binding code. We discuss the distinguishing characteristics of Pressio relative to existing model-reduction libraries, outline its key design features, describe how the user interacts with it, and present two test cases -- including one with over 20 million degrees of freedom -- that highlight the performance results of Pressio and illustrate the breath of problems that can be addressed with it. △ Less

Submitted 1 September, 2021; v1 submitted 17 March, 2020; originally announced March 2020.

Comments: 32 pages, 5 figures, supplement of 6 pages; Added references in intro, corrected fig5b, few more clarifications in sec4.2

Report number: SAND2020-1279 J, SAND2020-1445 J

Algorithms and Statistical Models for Scientific Discovery in the Petabyte Era

Authors: Brian Nord, Andrew J. Connolly, Jamie Kinney, Jeremy Kubica, Gautaum Narayan, Joshua E. G. Peek, Chad Schafer, Erik J. Tollerud, Camille Avestruz, G. Jogesh Babu, Simon Birrer, Douglas Burke, João Caldeira, Douglas A. Caldwell, Joleen K. Carlberg, Yen-Chi Chen, Chuanfei Dong, Eric D. Feigelson, V. Zach Golkhou, Vinay Kashyap, T. S. Li, Thomas Loredo, Luisa Lucie-Smith, Kaisey S. Mandel, J. R. Martínez-Galarza , et al. (13 additional authors not shown)

Abstract: The field of astronomy has arrived at a turning point in terms of size and complexity of both datasets and scientific collaboration. Commensurately, algorithms and statistical models have begun to adapt --- e.g., via the onset of artificial intelligence --- which itself presents new challenges and opportunities for growth. This white paper aims to offer guidance and ideas for how we can evolve our… ▽ More The field of astronomy has arrived at a turning point in terms of size and complexity of both datasets and scientific collaboration. Commensurately, algorithms and statistical models have begun to adapt --- e.g., via the onset of artificial intelligence --- which itself presents new challenges and opportunities for growth. This white paper aims to offer guidance and ideas for how we can evolve our technical and collaborative frameworks to promote efficient algorithmic development and take advantage of opportunities for scientific discovery in the petabyte era. We discuss challenges for discovery in large and complex data sets; challenges and requirements for the next stage of development of statistical methodologies and algorithmic tool sets; how we might change our paradigms of collaboration and education; and the ethical implications of scientists' contributions to widely applicable algorithms and computational modeling. We start with six distinct recommendations that are supported by the commentary following them. This white paper is related to a larger corpus of effort that has taken place within and around the Petabytes to Science Workshops (https://petabytestoscience.github.io/). △ Less

Submitted 4 November, 2019; originally announced November 2019.

Comments: arXiv admin note: substantial text overlap with arXiv:1905.05116

Report number: FERMILAB-FN-1093-A-AE-SCD

Preserving Lagrangian structure in nonlinear model reduction with application to structural dynamics

Authors: Kevin Carlberg, Ray Tuminaro, Paul Boggs

Abstract: This work proposes a model-reduction methodology that preserves Lagrangian structure (equivalently Hamiltonian structure) and achieves computational efficiency in the presence of high-order nonlinearities and arbitrary parameter dependence. As such, the resulting reduced-order model retains key properties such as energy conservation and symplectic time-evolution maps. We focus on parameterized sim… ▽ More This work proposes a model-reduction methodology that preserves Lagrangian structure (equivalently Hamiltonian structure) and achieves computational efficiency in the presence of high-order nonlinearities and arbitrary parameter dependence. As such, the resulting reduced-order model retains key properties such as energy conservation and symplectic time-evolution maps. We focus on parameterized simple mechanical systems subjected to Rayleigh dam** and external forces, and consider an application to nonlinear structural dynamics. To preserve structure, the method first approximates the system's `Lagrangian ingredients'---the Riemannian metric, the potential-energy function, the dissipation function, and the external force---and subsequently derives reduced-order equations of motion by applying the (forced) Euler--Lagrange equation with these quantities. From the algebraic perspective, key contributions include two efficient techniques for approximating parameterized reduced matrices while preserving symmetry and positive definiteness: matrix gappy POD and reduced-basis sparsification (RBS). Results for a parameterized truss-structure problem demonstrate the importance of preserving Lagrangian structure and illustrate the proposed method's merits: it reduces computation time while maintaining high accuracy and stability, in contrast to existing nonlinear model-reduction techniques that do not preserve structure. △ Less

Submitted 30 January, 2014; originally announced January 2014.

Comments: Submitted to SIAM Journal on Scientific Computing (SISC)

Journal ref: SIAM Journal on Scientific Computing, Vol. 37, No. 2, p. B153-B184 (2015)