Showing 1–28 of 28 results for author: Roberts, S

A Structure-Preserving Domain Decomposition Method for Data-Driven Modeling

Authors: Shuai Jiang, Jonas Actor, Scott Roberts, Nathaniel Trask

Abstract: We present a domain decomposition strategy for develo** structure-preserving finite element discretizations from data when exact governing equations are unknown. On subdomains, trainable Whitney form elements are used to identify structure-preserving models from data, providing a Dirichlet-to-Neumann map which may be used to globally construct a mortar method. The reduced-order local elements ma… ▽ More We present a domain decomposition strategy for develo** structure-preserving finite element discretizations from data when exact governing equations are unknown. On subdomains, trainable Whitney form elements are used to identify structure-preserving models from data, providing a Dirichlet-to-Neumann map which may be used to globally construct a mortar method. The reduced-order local elements may be trained offline to reproduce high-fidelity Dirichlet data in cases where first principles model derivation is either intractable, unknown, or computationally prohibitive. In such cases, particular care must be taken to preserve structure on both local and mortar levels without knowledge of the governing equations, as well as to ensure well-posedness and stability of the resulting monolithic data-driven system. This strategy provides a flexible means of both scaling to large systems and treating complex geometries, and is particularly attractive for multiscale problems with complex microstructure geometry. While consistency is traditionally obtained in finite element methods via quasi-optimality results and the Bramble-Hilbert lemma as the local element diameter $h\rightarrow0$, our analysis establishes notions of accuracy and stability for finite h with accuracy coming from matching data. Numerical experiments and analysis establish properties for $H(\operatorname{div})$ problems in small data limits ($\mathcal{O}(1)$ reference solutions). △ Less

Submitted 8 June, 2024; originally announced June 2024.

Comments: 38 pages, 18 figures

Strongly stable dual-pairing summation by parts finite difference schemes for the vector invariant nonlinear shallow water equations -- I: Numerical scheme and validation on the plane

Authors: Justin Kin Jun Hew, Kenneth Duru, Stephen Roberts, Christopher Zoppou, Kieran Ricardo

Abstract: We present an energy/entropy stable and high order accurate finite difference method for solving the nonlinear (rotating) shallow water equations (SWE) in vector invariant form using the newly developed dual-pairing and dispersion-relation preserving summation-by-parts finite difference operators. We derive new well-posed boundary conditions for the SWE in one space dimension, formulated in terms… ▽ More We present an energy/entropy stable and high order accurate finite difference method for solving the nonlinear (rotating) shallow water equations (SWE) in vector invariant form using the newly developed dual-pairing and dispersion-relation preserving summation-by-parts finite difference operators. We derive new well-posed boundary conditions for the SWE in one space dimension, formulated in terms of fluxes and applicable to linear and nonlinear SWEs. For the nonlinear SWE, entropy stability ensures the boundedness of numerical solution but does not guarantee convergence. Adequate amount of numerical dissipation is necessary to control high frequency errors which could negatively impact accuracy in the numerical simulations. Using the dual-pairing summation by parts framework, we derive high order accurate and nonlinear hyper-viscosity operator which dissipates entropy and enstrophy. The hyper-viscosity operator effectively minimises oscillations from shocks and discontinuities, and eliminates high frequency grid-scale errors. The numerical method is most suitable for the simulations of subcritical flows typically observed in atmospheric and geostrophic flow problems. We prove a priori error estimates for the semi-discrete approximations of both linear and nonlinear SWE. Convergence, accuracy, and well-balanced properties are verified via the method of manufactured solutions and canonical test problems such as the dam break and lake at rest. Numerical simulations in two-dimensions are presented which include the rotating and merging vortex problem and barotropic shear instability, with fully developed turbulence. △ Less

Submitted 8 April, 2024; v1 submitted 19 October, 2023; originally announced October 2023.

Comments: 37 pages, 14 figures, comments are welcome

Lipschitz Interpolation: Non-parametric Convergence under Bounded Stochastic Noise

Authors: Julien Walden Huang, Stephen Roberts, Jan-Peter Calliess

Abstract: This paper examines the asymptotic convergence properties of Lipschitz interpolation methods within the context of bounded stochastic noise. In the first part of the paper, we establish probabilistic consistency guarantees of the classical approach in a general setting and derive upper bounds on the uniform convergence rates. These bounds align with well-established optimal rates of non-parametric… ▽ More This paper examines the asymptotic convergence properties of Lipschitz interpolation methods within the context of bounded stochastic noise. In the first part of the paper, we establish probabilistic consistency guarantees of the classical approach in a general setting and derive upper bounds on the uniform convergence rates. These bounds align with well-established optimal rates of non-parametric regression obtained in related settings and provide new precise upper bounds on the non-parametric regression problem under bounded noise assumptions. Practically, they can serve as a theoretical tool for comparing Lipschitz interpolation to alternative non-parametric regression methods, providing a condition on the behaviour of the noise at the boundary of its support which indicates when Lipschitz interpolation should be expected to asymptotically outperform or underperform other approaches. In the second part, we expand upon these results to include asymptotic guarantees for online learning of dynamics in discrete-time stochastic systems and illustrate their utility in deriving closed-loop stability guarantees of a simple controller. We also explore applications where the main assumption of prior knowledge of the Lipschitz constant is removed by adopting the LACKI framework (Calliess et al. (2020)) and deriving general asymptotic consistency. △ Less

Submitted 10 October, 2023; originally announced October 2023.

Comments: Preprint

Explicit Runge Kutta Methods that Alleviate Order Reduction

Authors: Abhijit Biswas, David I. Ketcheson, Steven Roberts, Benjamin Seibold, David Shirokoff

Abstract: Explicit Runge--Kutta (RK) methods are susceptible to a reduction in the observed order of convergence when applied to initial-boundary value problem with time-dependent boundary conditions. We study conditions on explicit RK methods that guarantee high-order convergence for linear problems; we refer to these conditions as weak stage order conditions. We prove a general relationship between the me… ▽ More Explicit Runge--Kutta (RK) methods are susceptible to a reduction in the observed order of convergence when applied to initial-boundary value problem with time-dependent boundary conditions. We study conditions on explicit RK methods that guarantee high-order convergence for linear problems; we refer to these conditions as weak stage order conditions. We prove a general relationship between the method's order, weak stage order, and number of stages. We derive explicit RK methods with high weak stage order and demonstrate, through numerical tests, that they avoid the order reduction phenomenon up to any order for linear problems and up to order three for nonlinear problems. △ Less

Submitted 4 October, 2023; v1 submitted 4 October, 2023; originally announced October 2023.

MSC Class: 65L04; 65L20; 65M12

On well-posed boundary conditions and energy stable finite volume method for the linear shallow water wave equation

Authors: Rudi Prihandoko, Kenneth Duru, Stephen Roberts, Christopher Zoppou

Abstract: We derive and analyse well-posed boundary conditions for the linear shallow water wave equation. The analysis is based on the energy method and it identifies the number, location and form of the boundary conditions so that the initial boundary value problem is well-posed. A finite volume method is developed based on the summation-by-parts framework with the boundary conditions implemented weakly u… ▽ More We derive and analyse well-posed boundary conditions for the linear shallow water wave equation. The analysis is based on the energy method and it identifies the number, location and form of the boundary conditions so that the initial boundary value problem is well-posed. A finite volume method is developed based on the summation-by-parts framework with the boundary conditions implemented weakly using penalties. Stability is proven by deriving a discrete energy estimate analogous to the continuous estimate. The continuous and discrete analysis covers all flow regimes. Numerical experiments are presented verifying the analysis. △ Less

Submitted 27 September, 2023; originally announced September 2023.

Comments: 23 pages, 4 figures

MSC Class: 65M12 (Primary); 65M08 (Secondary) ACM Class: G.1.8

Leveraging Mixed Precision in Exponential Time Integration Methods

Authors: Cody J. Balos, Steven Roberts, David J. Gardner

Abstract: The machine learning explosion has created a prominent trend in modern computer hardware towards low precision floating-point operations. In response, there have been growing efforts to use low and mixed precision in general scientific computing. One important area that has received limited exploration is time-integration methods, which are used for solving differential equations that are ubiquito… ▽ More The machine learning explosion has created a prominent trend in modern computer hardware towards low precision floating-point operations. In response, there have been growing efforts to use low and mixed precision in general scientific computing. One important area that has received limited exploration is time-integration methods, which are used for solving differential equations that are ubiquitous in science and engineering applications. In this work, we develop two new approaches for leveraging mixed precision in exponential time integration methods. The first approach is based on a reformulation of the exponential Rosenbrock--Euler method allowing for low precision computations in matrix exponentials independent of the particular algorithm for matrix exponentiation. The second approach is based on an inexact and incomplete Arnoldi procedure in Krylov approximation methods for computing matrix exponentials and is agnostic to the chosen integration method. We show that both approaches improve accuracy compared to using purely low precision and offer better efficiency than using only double precision when solving an advection-diffusion-reaction partial differential equation. △ Less

Submitted 18 July, 2023; originally announced July 2023.

Report number: LLNL-PROC-851497 MSC Class: 65; 34 ACM Class: G.1.7; G.4

Journal ref: 2023 IEEE High Performance Extreme Computing Conference (HPEC)

Well-posed boundary conditions and energy stable discontinuous Galerkin spectral element method for the linearized Serre equations

Authors: Kenny Wiratama, Kenneth Duru, Stephen Roberts, Christopher Zoppou

Abstract: We derive well-posed boundary conditions for the linearized Serre equations in one spatial dimension by utilizing the energy method. An energy stable and conservative discontinuous Galerkin spectral element method with simple upwind numerical fluxes is proposed for solving the initial boundary value problem. We derive discrete energy estimates for the numerical approximation and prove a priori err… ▽ More We derive well-posed boundary conditions for the linearized Serre equations in one spatial dimension by utilizing the energy method. An energy stable and conservative discontinuous Galerkin spectral element method with simple upwind numerical fluxes is proposed for solving the initial boundary value problem. We derive discrete energy estimates for the numerical approximation and prove a priori error estimates in the energy norm. Detailed numerical examples are provided to verify the theoretical analysis and show convergence of numerical errors. △ Less

Submitted 25 April, 2023; v1 submitted 20 March, 2023; originally announced March 2023.

MSC Class: 65M12; 65M70; 65M15; 65M60

Implicit-Explicit Multirate Infinitesimal Stage-Restart Methods

Authors: Alex C. Fish, Daniel R. Reynolds, Steven B. Roberts

Abstract: Implicit-Explicit (IMEX) methods are flexible numerical time integration methods which solve an initial-value problem (IVP) that is partitioned into stiff and nonstiff processes with the goal of lower computational costs than a purely implicit or explicit approach. A complementary form of flexible IVP solvers are multirate infinitesimal methods for problems partitioned into fast- and slow-changing… ▽ More Implicit-Explicit (IMEX) methods are flexible numerical time integration methods which solve an initial-value problem (IVP) that is partitioned into stiff and nonstiff processes with the goal of lower computational costs than a purely implicit or explicit approach. A complementary form of flexible IVP solvers are multirate infinitesimal methods for problems partitioned into fast- and slow-changing dynamics, that solve a multirate IVP by evolving a sequence of ``fast'' IVPs using any suitably accurate algorithm. This article introduces a new class of high-order implicit-explicit multirate methods that are designed for multirate IVPs in which the slow-changing dynamics are further partitioned in an IMEX fashion. This new class, which we call implicit-explicit multirate stage-restart (IMEX-MRI-SR), both improves upon the previous implicit-explicit multirate generalized-structure additive Runge Kutta (IMEX-MRI-GARK) methods, and extends multirate exponential Runge Kutta (MERK) methods into the IMEX context. We leverage GARK theory to derive conditions guaranteeing orders of accuracy up to four. We provide second-, third-, and fourth-order accurate example methods and perform numerical simulations demonstrating convergence rates and computational performance in both fixed-step and adaptive-step settings. △ Less

Submitted 2 January, 2023; originally announced January 2023.

MSC Class: 65L20; 65L05; 65L06

Eliminating Order Reduction on Linear, Time-Dependent ODEs with GARK Methods

Authors: Steven Roberts, Adrian Sandu

Abstract: When applied to stiff, linear differential equations with time-dependent forcing, Runge-Kutta methods can exhibit convergence rates lower than predicted by the classical order condition theory. Commonly, this order reduction phenomenon is addressed by using an expensive, fully implicit Runge-Kutta method with high stage order or a specialized scheme satisfying additional order conditions. This wor… ▽ More When applied to stiff, linear differential equations with time-dependent forcing, Runge-Kutta methods can exhibit convergence rates lower than predicted by the classical order condition theory. Commonly, this order reduction phenomenon is addressed by using an expensive, fully implicit Runge-Kutta method with high stage order or a specialized scheme satisfying additional order conditions. This work develops a flexible approach of augmenting an arbitrary Runge-Kutta method with a fully implicit method used to treat the forcing such as to maintain the classical order of the base scheme. Our methods and analyses are based on the general-structure additive Runge-Kutta framework. Numerical experiments using diagonally implicit, fully implicit, and even explicit Runge-Kutta methods confirm that the new approach eliminates order reduction for the class of problems under consideration, and the base methods achieve their theoretical orders of convergence. △ Less

Submitted 13 February, 2022; v1 submitted 19 January, 2022; originally announced January 2022.

MSC Class: 65L04; 65L20

A unified formulation of splitting-based implicit time integration schemes

Authors: Severiano González-Pinto, Domingo Hernández-Abreu, Maria S. Pérez-Rodríguez, Arash Sarshar, Steven Roberts, Adrian Sandu

Abstract: Splitting-based time integration approaches such as fractional steps, alternating direction implicit, operator splitting, and locally one-dimensional methods partition the system of interest into components and solve individual components implicitly in a cost-effective way. This work proposes a unified formulation of splitting time integration schemes in the framework of general-structure additive… ▽ More Splitting-based time integration approaches such as fractional steps, alternating direction implicit, operator splitting, and locally one-dimensional methods partition the system of interest into components and solve individual components implicitly in a cost-effective way. This work proposes a unified formulation of splitting time integration schemes in the framework of general-structure additive Runge-Kutta (GARK) methods. Specifically, we develop implicit-implicit (IMIM) GARK schemes, provide the order conditions and stability analysis for this class, and explain their application to partitioned systems of ordinary differential equations. We show that classical splitting methods belong to the IMIM GARK family, and therefore can be studied in this unified framework. New IMIM-GARK splitting methods are developed and tested using parabolic systems. △ Less

Submitted 20 December, 2021; v1 submitted 1 March, 2021; originally announced March 2021.

Report number: CSL-TR-2021-3 MSC Class: 65L05; 65L07

OffCon$^3$: What is state of the art anyway?

Authors: Philip J. Ball, Stephen J. Roberts

Abstract: Two popular approaches to model-free continuous control tasks are SAC and TD3. At first glance these approaches seem rather different; SAC aims to solve the entropy-augmented MDP by minimising the KL-divergence between a stochastic proposal policy and a hypotheical energy-basd soft Q-function policy, whereas TD3 is derived from DPG, which uses a deterministic policy to perform policy gradient asce… ▽ More Two popular approaches to model-free continuous control tasks are SAC and TD3. At first glance these approaches seem rather different; SAC aims to solve the entropy-augmented MDP by minimising the KL-divergence between a stochastic proposal policy and a hypotheical energy-basd soft Q-function policy, whereas TD3 is derived from DPG, which uses a deterministic policy to perform policy gradient ascent along the value function. In reality, both approaches are remarkably similar, and belong to a family of approaches we call `Off-Policy Continuous Generalized Policy Iteration'. This illuminates their similar performance in most continuous control benchmarks, and indeed when hyperparameters are matched, their performance can be statistically indistinguishable. To further remove any difference due to implementation, we provide OffCon$^3$ (Off-Policy Continuous Control: Consolidated), a code base featuring state-of-the-art versions of both algorithms. △ Less

Submitted 14 March, 2021; v1 submitted 27 January, 2021; originally announced January 2021.

A fast time-step** strategy for dynamical systems equipped with a surrogate model

Authors: Steven Roberts, Andrey A Popov, Arash Sarshar, Adrian Sandu

Abstract: Simulation of complex dynamical systems arising in many applications is computationally challenging due to their size and complexity. Model order reduction, machine learning, and other types of surrogate modeling techniques offer cheaper and simpler ways to describe the dynamics of these systems but are inexact and introduce additional approximation errors. In order to overcome the computational d… ▽ More Simulation of complex dynamical systems arising in many applications is computationally challenging due to their size and complexity. Model order reduction, machine learning, and other types of surrogate modeling techniques offer cheaper and simpler ways to describe the dynamics of these systems but are inexact and introduce additional approximation errors. In order to overcome the computational difficulties of the full complex models, on one hand, and the limitations of surrogate models, on the other, this work proposes a new accelerated time-step** strategy that combines information from both. This approach is based on the multirate infinitesimal general-structure additive Runge-Kutta (MRI-GARK) framework. The inexpensive surrogate model is integrated with a small timestep to guide the solution trajectory, and the full model is treated with a large timestep to occasionally correct for the surrogate model error and ensure convergence. We provide a theoretical error analysis, and several numerical experiments, to show that this approach can be significantly more efficient than using only the full or only the surrogate model for the integration. △ Less

Submitted 14 February, 2022; v1 submitted 6 November, 2020; originally announced November 2020.

Report number: CSL-TR-20-6 MSC Class: 65L05; 65F99

Linearly implicit GARK schemes

Authors: Adrian Sandu, Michael Günther, Steven Roberts

Abstract: Systems driven by multiple physical processes are central to many areas of science and engineering. Time discretization of multiphysics systems is challenging, since different processes have different levels of stiffness and characteristic time scales. The multimethod approach discretizes each physical process with an appropriate numerical method; the methods are coupled appropriately such that th… ▽ More Systems driven by multiple physical processes are central to many areas of science and engineering. Time discretization of multiphysics systems is challenging, since different processes have different levels of stiffness and characteristic time scales. The multimethod approach discretizes each physical process with an appropriate numerical method; the methods are coupled appropriately such that the overall solution has the desired accuracy and stability properties. The authors developed the general-structure additive Runge-Kutta (GARK) framework, which constructs multimethods based on Runge-Kutta schemes. This paper constructs the new GARK-ROS/GARK-ROW families of multimethods based on linearly implicit Rosenbrock/Rosenbrock-W schemes. For ordinary differential equation models, we develop a general order condition theory for linearly implicit methods with any number of partitions, using exact or approximate Jacobians. We generalize the order condition theory to two-way partitioned index-1 differential-algebraic equations. Applications of the framework include decoupled linearly implicit, linearly implicit/explicit, and linearly implicit/implicit methods. Practical GARK-ROS and GARK-ROW schemes of order up to four are constructed. △ Less

Submitted 11 October, 2020; v1 submitted 4 August, 2020; originally announced August 2020.

Report number: CSL-TR-20-9 MSC Class: 65L05; 65L06; 65L07; 65L20

Parallel implicit-explicit general linear methods

Authors: Steven Roberts, Arash Sarshar, Adrian Sandu

Abstract: High-order discretizations of partial differential equations (PDEs) necessitate high-order time integration schemes capable of handling both stiff and nonstiff operators in an efficient manner. Implicit-explicit (IMEX) integration based on general linear methods (GLMs) offers an attractive solution due to their high stage and method order, as well as excellent stability properties. The IMEX charac… ▽ More High-order discretizations of partial differential equations (PDEs) necessitate high-order time integration schemes capable of handling both stiff and nonstiff operators in an efficient manner. Implicit-explicit (IMEX) integration based on general linear methods (GLMs) offers an attractive solution due to their high stage and method order, as well as excellent stability properties. The IMEX characteristic allows stiff terms to be treated implicitly and nonstiff terms to be efficiently integrated explicitly. This work develops two systematic approaches for the development of IMEX GLMs of arbitrary order with stages that can be solved in parallel. The first approach is based on diagonally implicit multistage integration methods (DIMSIMs) of types 3 and 4. The second is a parallel generalization of IMEX Euler and has the interesting feature that the linear stability is independent of the order of accuracy. Numerical experiments confirm the theoretical rates of convergence and reveal that the new schemes are more efficient than serial IMEX GLMs and IMEX Runge-Kutta methods. △ Less

Submitted 21 April, 2020; v1 submitted 3 February, 2020; originally announced February 2020.

Report number: CSL-TR-19-12 MSC Class: 65L05; 65L20; 65L80

Polynomial Valuations on Vector Lattices

Authors: Gerard Buskes, Stephan Roberts

Abstract: We prove that polynomial valuations on vector lattices correspond to orthosymmetric multilinear maps. As a consequence we obtain a concise proof of the equivalence of orthosymmetry and orthogonal additivity. We prove that polynomial valuations on vector lattices correspond to orthosymmetric multilinear maps. As a consequence we obtain a concise proof of the equivalence of orthosymmetry and orthogonal additivity. △ Less

Submitted 1 November, 2019; v1 submitted 31 October, 2019; originally announced October 2019.

Implicit multirate GARK methods

Authors: Steven Roberts, John Loffeld, Arash Sarshar, Carol S. Woodward, Adrian Sandu

Abstract: This work considers multirate generalized-structure additively partitioned Runge-Kutta (MrGARK) methods for solving stiff systems of ordinary differential equations (ODEs) with multiple time scales. These methods treat different partitions of the system with different timesteps for a more targeted and efficient solution compared to monolithic single rate approaches. With implicit methods used acro… ▽ More This work considers multirate generalized-structure additively partitioned Runge-Kutta (MrGARK) methods for solving stiff systems of ordinary differential equations (ODEs) with multiple time scales. These methods treat different partitions of the system with different timesteps for a more targeted and efficient solution compared to monolithic single rate approaches. With implicit methods used across all partitions, methods must find a balance between stability and the cost of solving nonlinear equations for the stages. In order to characterize this important trade-off, we explore multirate coupling strategies, problems for assessing linear stability, and techniques to efficiently implement Newton iterations for stage equations. Unlike much of the existing multirate stability analysis which is limited in scope to particular methods, we present general statements on stability and describe fundamental limitations for certain types of multirate schemes. New implicit multirate methods up to fourth order are derived, and their accuracy and efficiency properties are verified with numerical tests. △ Less

Submitted 30 November, 2020; v1 submitted 30 October, 2019; originally announced October 2019.

Report number: CSL-TR-19-5 MSC Class: 65L06; 65L20

Stratified Space Learning: Reconstructing Embedded Graphs

Authors: Yossi Bokor, Daniel Grixti-Cheng, Markus Hegland, Stephen Roberts, Katharine Turner

Abstract: Many data-rich industries are interested in the efficient discovery and modelling of structures underlying large data sets, as it allows for the fast triage and dimension reduction of large volumes of data embedded in high dimensional spaces. The modelling of these underlying structures is also beneficial for the creation of simulated data that better represents real data. In particular, for syste… ▽ More Many data-rich industries are interested in the efficient discovery and modelling of structures underlying large data sets, as it allows for the fast triage and dimension reduction of large volumes of data embedded in high dimensional spaces. The modelling of these underlying structures is also beneficial for the creation of simulated data that better represents real data. In particular, for systems testing in cases where the use of real data streams might prove impractical or otherwise undesirable. We seek to discover and model the structure by combining methods from topological data analysis with numerical modelling. As a first step in combining these two areas, we examine the recovery of the abstract graph $G$ structure, and model a linear embedding $|G|$ given only a noisy point cloud sample $X$ of $|G|$. △ Less

Submitted 26 September, 2019; originally announced September 2019.

Comments: 7 pages, 3 figures, accepted for MODSIM 2019 conference

Alternating Directions Implicit Integration in a General Linear Method Framework

Authors: Arash Sarshar, Steven Roberts, Adrian Sandu

Abstract: Alternating Directions Implicit (ADI) integration is an operator splitting approach to solve parabolic and elliptic partial differential equations in multiple dimensions based on solving sequentially a set of related one-dimensional equations. Classical ADI methods have order at most two, due to the splitting errors. Moreover, when the time discretization of stiff one-dimensional problems is based… ▽ More Alternating Directions Implicit (ADI) integration is an operator splitting approach to solve parabolic and elliptic partial differential equations in multiple dimensions based on solving sequentially a set of related one-dimensional equations. Classical ADI methods have order at most two, due to the splitting errors. Moreover, when the time discretization of stiff one-dimensional problems is based on Runge-Kutta schemes, additional order reduction may occur. This work proposes a new ADI approach based on the partitioned General Linear Methods framework. This approach allows the construction of high order ADI methods. Due to their high stage order, the proposed methods can alleviate the order reduction phenomenon seen with other schemes. Numerical experiments are shown to provide further insight into the accuracy, stability, and applicability of these new methods. △ Less

Submitted 3 December, 2019; v1 submitted 1 February, 2019; originally announced February 2019.

Report number: CSL-TR-2019-1 MSC Class: 65L05; 65L07

Journal ref: Journal of Computational and Applied Mathematics, 2019, 112619, ISSN 0377-0427

ODE Test Problems: a MATLAB suite of initial value problems

Authors: Steven Roberts, Andrey A. Popov, Adrian Sandu

Abstract: ODE Test Problems (OTP) is an object-oriented MATLAB package offering a broad range of initial value problems which can be used to test numerical methods such as time integration methods and data assimilation (DA) methods. It includes problems that are linear and nonlinear, homogeneous and nonhomogeneous, autonomous and nonautonomous, scalar and high-dimensional, stiff and nonstiff, and chaotic an… ▽ More ODE Test Problems (OTP) is an object-oriented MATLAB package offering a broad range of initial value problems which can be used to test numerical methods such as time integration methods and data assimilation (DA) methods. It includes problems that are linear and nonlinear, homogeneous and nonhomogeneous, autonomous and nonautonomous, scalar and high-dimensional, stiff and nonstiff, and chaotic and nonchaotic. Many are real-world problems from fields such as chemistry, astrophysics, meteorology, and electrical engineering. OTP also supports partitioned ODEs for testing IMEX methods, multirate methods, and other multimethods. Functions for plotting solutions and creating movies are available for all problems, and exact solutions are provided when available. OTP is desgined for ease of use-meaning that working with and modifying problems is simple and intuitive. △ Less

Submitted 13 January, 2019; originally announced January 2019.

Report number: CSL-TR-19-1

Coupled Multirate Infinitesimal GARK Schemes for Stiff Systems with Multiple Time Scales

Authors: Steven Roberts, Arash Sarshar, Adrian Sandu

Abstract: Traditional time discretization methods use a single timestep for the entire system of interest and can perform poorly when the dynamics of the system exhibits a wide range of time scales. Multirate infinitesimal step (MIS) methods (Knoth and Wolke, 1998) offer an elegant and flexible approach to efficiently integrate such systems. The slow components are discretized by a Runge-Kutta method, and t… ▽ More Traditional time discretization methods use a single timestep for the entire system of interest and can perform poorly when the dynamics of the system exhibits a wide range of time scales. Multirate infinitesimal step (MIS) methods (Knoth and Wolke, 1998) offer an elegant and flexible approach to efficiently integrate such systems. The slow components are discretized by a Runge-Kutta method, and the fast components are resolved by solving modified fast differential equations. Sandu (2018) developed the Multirate Infinitesimal General-structure Additive Runge-Kutta (MRI-GARK) family of methods that includes traditional MIS schemes as a subset. The MRI-GARK framework allowed the construction of the first fourth order MIS schemes. This framework also enabled the introduction of implicit methods, which are decoupled in the sense that any implicitness lies entirely within the fast or slow integrations. It was shown by Sandu that the stability of decoupled implicit MRI-GARK methods has limitations when both the fast and slow components are stiff and interact strongly. This work extends the MRI-GARK framework by introducing coupled implicit methods to solve stiff multiscale systems. The coupled approach has the potential to considerably improve the overall stability of the scheme, at the price of requiring implicit stage calculations over the entire system. Two coupling strategies are considered. The first computes coupled Runge-Kutta stages before solving a single differential equation to refine the fast solution. The second alternates between computing coupled Runge-Kutta stages and solving fast differential equations. We derive order conditions and perform the stability analysis for both strategies. The new coupled methods offer improved stability compared to the decoupled MRI-GARK schemes. The theoretical properties of the new methods are validated with numerical experiments. △ Less

Submitted 2 March, 2020; v1 submitted 30 November, 2018; originally announced December 2018.

Comments: arXiv admin note: substantial text overlap with arXiv:1808.02759

Report number: CSL-TR-18-7 MSC Class: 65L05; 65L06

Design of High-Order Decoupled Multirate GARK Schemes

Authors: Arash Sarshar, Steven Roberts, Adrian Sandu

Abstract: Multirate time integration methods apply different step sizes to resolve different components of the system based on the local activity levels. This local selection of step sizes allows increased computational efficiency while achieving the desired solution accuracy. While the multirate idea is elegant and has been around for decades, multirate methods are not yet widely used in applications. This… ▽ More Multirate time integration methods apply different step sizes to resolve different components of the system based on the local activity levels. This local selection of step sizes allows increased computational efficiency while achieving the desired solution accuracy. While the multirate idea is elegant and has been around for decades, multirate methods are not yet widely used in applications. This is due, in part, to the difficulties raised by the construction of high order multirate schemes. Seeking to overcome these challenges, this work focuses on the design of practical high-order multirate methods using the theoretical framework of generalized additive Runge-Kutta (MrGARK) methods, which provides the generic order conditions and the linear and nonlinear stability analyses. A set of design criteria for practical multirate methods is defined herein: method coefficients should be generic in the step size ratio, but should not depend strongly on this ratio; unnecessary coupling between the fast and the slow components should be avoided; and the step size controllers should adjust both the micro- and the macro-steps. Using these criteria, we develop MrGARK schemes of up to order four that are explicit-explicit (both the fast and slow component are treated explicitly), implicit-explicit (implicit in the fast component and explicit in the slow one), and explicit-implicit (explicit in the fast component and implicit in the slow one). Numerical experiments illustrate the performance of these new schemes. △ Less

Submitted 20 December, 2021; v1 submitted 20 April, 2018; originally announced April 2018.

Report number: CSL-TR-18-4 MSC Class: 65L05; 65L06; 65L20

Journal ref: SIAM Journal on Scientific Computing, Vol. 41, No. 2, 2019, PP. A816-A847

Behaviour of the Serre Equations in the Presence of Steep Gradients Revisited

Authors: Jordan Pitt, Christopher Zoppou, Stephen Roberts

Abstract: We use numerical methods to study the behaviour of the Serre equations in the presence of steep gradients because there are no known analytical solutions for these problems. In kee** with the literature we study a class of initial condition problems that are a smooth approximation to the initial conditions of the dam-break problem. This class of initial condition problems allow us to observe the… ▽ More We use numerical methods to study the behaviour of the Serre equations in the presence of steep gradients because there are no known analytical solutions for these problems. In kee** with the literature we study a class of initial condition problems that are a smooth approximation to the initial conditions of the dam-break problem. This class of initial condition problems allow us to observe the behaviour of the Serre equations with varying steepness of the initial conditions. The numerical solutions of the Serre equations are justified by demonstrating that as the resolution increases they converge to a solution with little error in conservation of mass, momentum and energy independent of the numerical method. We observe four different structures of the converged numerical solutions depending on the steepness of the initial conditions. Two of these structures were observed in the literature, with the other two not being commonly found in the literature. The numerical solutions are then used to assess how well the analytical solution of the shallow water wave equations captures the mean behaviour of the solution of the Serre equations for the dam-break problem. Lastly the numerical solutions are used to evaluate the usefulness of asymptotic results in the literature to approximate the depth and location of the front of an undular bore. △ Less

Submitted 26 June, 2017; originally announced June 2017.

Comments: 28 pages, 36 individual figure files, compiled into 20 figures and 1 table

Entropic Trace Estimates for Log Determinants

Authors: Jack Fitzsimons, Diego Granziol, Kurt Cutajar, Michael Osborne, Maurizio Filippone, Stephen Roberts

Abstract: The scalable calculation of matrix determinants has been a bottleneck to the widespread application of many machine learning methods such as determinantal point processes, Gaussian processes, generalised Markov random fields, graph models and many others. In this work, we estimate log determinants under the framework of maximum entropy, given information in the form of moment constraints from stoc… ▽ More The scalable calculation of matrix determinants has been a bottleneck to the widespread application of many machine learning methods such as determinantal point processes, Gaussian processes, generalised Markov random fields, graph models and many others. In this work, we estimate log determinants under the framework of maximum entropy, given information in the form of moment constraints from stochastic trace estimation. The estimates demonstrate a significant improvement on state-of-the-art alternative methods, as shown on a wide variety of UFL sparse matrices. By taking the example of a general Markov random field, we also demonstrate how this approach can significantly accelerate inference in large-scale learning methods involving the log determinant. △ Less

Submitted 24 April, 2017; originally announced April 2017.

Comments: 16 pages, 4 figures, 2 tables, 2 algorithms

Bayesian Inference of Log Determinants

Authors: Jack Fitzsimons, Kurt Cutajar, Michael Osborne, Stephen Roberts, Maurizio Filippone

Abstract: The log-determinant of a kernel matrix appears in a variety of machine learning problems, ranging from determinantal point processes and generalized Markov random fields, through to the training of Gaussian processes. Exact calculation of this term is often intractable when the size of the kernel matrix exceeds a few thousand. In the spirit of probabilistic numerics, we reinterpret the problem of… ▽ More The log-determinant of a kernel matrix appears in a variety of machine learning problems, ranging from determinantal point processes and generalized Markov random fields, through to the training of Gaussian processes. Exact calculation of this term is often intractable when the size of the kernel matrix exceeds a few thousand. In the spirit of probabilistic numerics, we reinterpret the problem of computing the log-determinant as a Bayesian inference problem. In particular, we combine prior knowledge in the form of bounds from matrix theory and evidence derived from stochastic trace estimation to obtain probabilistic estimates for the log-determinant and its associated uncertainty within a given computational budget. Beyond its novelty and theoretic appeal, the performance of our proposal is competitive with state-of-the-art approaches to approximating the log-determinant, while also quantifying the uncertainty due to budget-constrained evidence. △ Less

Submitted 5 April, 2017; originally announced April 2017.

Comments: 12 pages, 3 figures

Improved stochastic trace estimation using mutually unbiased bases

Authors: J. K. Fitzsimons, M. A. Osborne, S. J. Roberts, J. F. Fitzsimons

Abstract: We examine the problem of estimating the trace of a matrix $A$ when given access to an oracle which computes $x^\dagger A x$ for an input vector $x$. We make use of the basis vectors from a set of mutually unbiased bases, widely studied in the field of quantum information processing, in the selection of probing vectors $x$. This approach offers a new state of the art single shot sampling variance… ▽ More We examine the problem of estimating the trace of a matrix $A$ when given access to an oracle which computes $x^\dagger A x$ for an input vector $x$. We make use of the basis vectors from a set of mutually unbiased bases, widely studied in the field of quantum information processing, in the selection of probing vectors $x$. This approach offers a new state of the art single shot sampling variance while requiring only $O(\log(n))$ random bits to generate each vector. This significantly improves on traditional methods such as Hutchinson's and Gaussian estimators in terms of the number of random bits required and worst case sample variance. △ Less

Submitted 30 July, 2016; originally announced August 2016.

Comments: 5 pages, 1 figure, 2 tables. Comments welcome

Third-order Finite Volume/Finite Element Solution of the Fully Nonlinear Weakly Dispersive Serre Equations

Authors: Christopher Zoppou, Jordan Pitt, Stephen G. Roberts

Abstract: The nonlinear weakly dispersive Serre equations contain higher-order dispersive terms. This includes a mixed derivative flux term which is difficult to handle numerically. The mix spatial and temporal derivative dispersive term is replaced by a combination of temporal and spatial terms. The Serre equations are re-written so that the system of equations contain homogeneous derivative terms only. Th… ▽ More The nonlinear weakly dispersive Serre equations contain higher-order dispersive terms. This includes a mixed derivative flux term which is difficult to handle numerically. The mix spatial and temporal derivative dispersive term is replaced by a combination of temporal and spatial terms. The Serre equations are re-written so that the system of equations contain homogeneous derivative terms only. The reformulated Serre equations involve the water depth and a new quantity as the conserved variables which are evolved using the finite volume method. The remaining primitive variable, the velocity is obtained by solving a second-order elliptic equation using the finite element method. To avoid the introduction of numerical dispersion that may dominate the physical dispersion, the hybrid scheme has third-order accuracy. Using analytical solutions, laboratory flume data and by simulating the dam-break problem, the proposed scheme is shown to be accurate, simple to implement and stable for a range of problems, including discontinuous flows. △ Less

Submitted 27 July, 2016; originally announced July 2016.

MSC Class: 76B15; 35L65; 65M08

Dynamic Bayesian Combination of Multiple Imperfect Classifiers

Authors: Edwin Simpson, Stephen Roberts, Ioannis Psorakis, Arfon Smith

Abstract: Classifier combination methods need to make best use of the outputs of multiple, imperfect classifiers to enable higher accuracy classifications. In many situations, such as when human decisions need to be combined, the base decisions can vary enormously in reliability. A Bayesian approach to such uncertain combination allows us to infer the differences in performance between individuals and to in… ▽ More Classifier combination methods need to make best use of the outputs of multiple, imperfect classifiers to enable higher accuracy classifications. In many situations, such as when human decisions need to be combined, the base decisions can vary enormously in reliability. A Bayesian approach to such uncertain combination allows us to infer the differences in performance between individuals and to incorporate any available prior knowledge about their abilities when training data is sparse. In this paper we explore Bayesian classifier combination, using the computationally efficient framework of variational Bayesian inference. We apply the approach to real data from a large citizen science project, Galaxy Zoo Supernovae, and show that our method far outperforms other established approaches to imperfect decision combination. We go on to analyse the putative community structure of the decision makers, based on their inferred decision making strategies, and show that natural grou**s are formed. Finally we present a dynamic Bayesian classifier combination approach and investigate the changes in base classifier performance over time. △ Less

Submitted 8 June, 2012; originally announced June 2012.

Comments: 35 pages, 12 figures

Local and Dimension Adaptive Sparse Grid Interpolation and Quadrature

Authors: John D. Jakeman, Stephen G. Roberts

Abstract: In this paper we present a locally and dimension-adaptive sparse grid method for interpolation and integration of high-dimensional functions with discontinuities. The proposed algorithm combines the strengths of the generalised sparse grid algorithm and hierarchical surplus-guided local adaptivity. A high-degree basis is used to obtain a high-order method which, given sufficient smoothness, perfor… ▽ More In this paper we present a locally and dimension-adaptive sparse grid method for interpolation and integration of high-dimensional functions with discontinuities. The proposed algorithm combines the strengths of the generalised sparse grid algorithm and hierarchical surplus-guided local adaptivity. A high-degree basis is used to obtain a high-order method which, given sufficient smoothness, performs significantly better than the piecewise-linear basis. The underlying generalised sparse grid algorithm greedily selects the dimensions and variable interactions that contribute most to the variability of a function. The hierarchical surplus of points within the sparse grid is used as an error criterion for local refinement with the aim of concentrating computational effort within rapidly varying or discontinuous regions. This approach limits the number of points that are invested in `unimportant' dimensions and regions within the high-dimensional domain. We show the utility of the proposed method for non-smooth functions with hundreds of variables. △ Less

Submitted 30 September, 2011; originally announced October 2011.