Showing 1–50 of 80 results for author: Cai, W

Deep Neural Networks with Symplectic Preservation Properties

Authors: Qing He, Wei Cai

Abstract: We propose a deep neural network architecture designed such that its output forms an invertible symplectomorphism of the input. This design draws an analogy to the real-valued non-volume-preserving (real NVP) method used in normalizing flow techniques. Utilizing this neural network type allows for learning tasks on unknown Hamiltonian systems without breaking the inherent symplectic structure of t… ▽ More We propose a deep neural network architecture designed such that its output forms an invertible symplectomorphism of the input. This design draws an analogy to the real-valued non-volume-preserving (real NVP) method used in normalizing flow techniques. Utilizing this neural network type allows for learning tasks on unknown Hamiltonian systems without breaking the inherent symplectic structure of the phase space. △ Less

Submitted 28 June, 2024; originally announced July 2024.

MSC Class: 37J11; 70H15; 68T07

Ordered bases, order-preserving automorphisms and bi-orderable link groups

Authors: Tommy Wuxing Cai, Adam Clay, Dale Rolfsen

Abstract: We give a new criterion which guarantees that a free group admits a bi-ordering that is invariant under a given automorphism. As an application, we show that the fundamental group of the "magic manifold" is bi-orderable, answering a question of Kin and Rolfsen. We give a new criterion which guarantees that a free group admits a bi-ordering that is invariant under a given automorphism. As an application, we show that the fundamental group of the "magic manifold" is bi-orderable, answering a question of Kin and Rolfsen. △ Less

Submitted 27 June, 2024; originally announced June 2024.

Comments: 19 pages, 2 figures

MSC Class: 06F15; 20F60; 57M05; 57K30

Approaches to iterative algorithms for solving nonlinear equations with an application in tomographic absorption spectroscopy

Authors: F. J. Aragón-Artacho, W. Cai, Y. Censor, A. Gibali, C. Shui, D. Torregrosa-Belén

Abstract: In this paper we propose an approach for solving systems of nonlinear equations without computing function derivatives. Motivated by the application area of tomographic absorption spectroscopy, which is a highly-nonlinear problem with variables coupling, we consider a situation where straightforward translation to a fixed point problem is not possible because the operators that represent the relev… ▽ More In this paper we propose an approach for solving systems of nonlinear equations without computing function derivatives. Motivated by the application area of tomographic absorption spectroscopy, which is a highly-nonlinear problem with variables coupling, we consider a situation where straightforward translation to a fixed point problem is not possible because the operators that represent the relevant systems of nonlinear equations are not self-map**s, i.e., they operate between spaces of different dimensions. To overcome this difficulty we suggest an "alternating common fixed points algorithm" that acts alternatingly on the different vector variables. This approach translates the original problem to a common fixed point problem for which iterative algorithms are abound and exhibits a viable alternative to translation to an optimization problem, which usually requires derivatives information. However, to apply any of these iterative algorithms requires to ascertain the conditions that appear in their convergence theorems. To circumvent the need to verify conditions for convergence, we propose and motivate a derivative-free algorithm that better suits the tomographic absorption spectroscopy problem at hand and is even further improved by applying to it the superiorization approach. This is presented along with experimental results that demonstrate our approach. △ Less

Submitted 14 May, 2024; originally announced May 2024.

Comments: Accepted for publication in the journal: Communications in Optimization Theory

MSC Class: 35Q99; 47J05; 65K05; 90C30

SOC-MartNet: A Martingale Neural Network for the Hamilton-Jacobi-Bellman Equation without Explicit inf H in Stochastic Optimal Controls

Authors: Wei Cai, Shuixin Fang, Tao Zhou

Abstract: In this work, we propose a martingale based neural network, SOC-MartNet, for solving high-dimensional Hamilton-Jacobi-Bellman (HJB) equations where no explicit expression is needed for the Hamiltonian $\inf_{u \in U} H(t,x,u, z,p)$, and stochastic optimal control problems with controls on both drift and volatility. We reformulate the HJB equations into a stochastic neural network learning process,… ▽ More In this work, we propose a martingale based neural network, SOC-MartNet, for solving high-dimensional Hamilton-Jacobi-Bellman (HJB) equations where no explicit expression is needed for the Hamiltonian $\inf_{u \in U} H(t,x,u, z,p)$, and stochastic optimal control problems with controls on both drift and volatility. We reformulate the HJB equations into a stochastic neural network learning process, i.e., training a control network and a value network such that the associated Hamiltonian process is minimized and the cost process becomes a martingale.To enforce the martingale property for the cost process, we employ an adversarial network and construct a loss function based on the projection property of conditional expectations. Then, the control/value networks and the adversarial network are trained adversarially, such that the cost process is driven towards a martingale and the minimum principle is satisfied for the control.Numerical results show that the proposed SOC-MartNet is effective and efficient for solving HJB-type equations and SOCP with a dimension up to $500$ in a small number of training epochs. △ Less

Submitted 6 May, 2024; originally announced May 2024.

On the First Passage Times of Branching Random Walks in $\mathbb R^d$

Authors: Jose Blanchet, Wei Cai, Shaswat Mohanty, Zhenyuan Zhang

Abstract: We study the first passage times of discrete-time branching random walks in ${\mathbb R}^d$ where $d\geq 1$. Here, the genealogy of the particles follows a supercritical Galton-Watson process. We provide asymptotics of the first passage times to a ball of radius one with a distance $x$ from the origin, conditioned upon survival. We provide explicitly the linear dominating term and the logarithmic… ▽ More We study the first passage times of discrete-time branching random walks in ${\mathbb R}^d$ where $d\geq 1$. Here, the genealogy of the particles follows a supercritical Galton-Watson process. We provide asymptotics of the first passage times to a ball of radius one with a distance $x$ from the origin, conditioned upon survival. We provide explicitly the linear dominating term and the logarithmic correction term as a function of $x$. The asymptotics are precise up to an order of $o_{\mathbb P}(\log x)$ for general jump distributions and up to $O_{\mathbb P}(\log\log x)$ for spherically symmetric jumps. A crucial ingredient of both results is the tightness of first passage times. We also discuss an extension of the first passage time analysis to a modified branching random walk model that has been proven to successfully capture shortest path statistics in polymer networks. △ Less

Submitted 13 April, 2024; originally announced April 2024.

Comments: 40 pages, 8 figures

MSC Class: 60G70; 60J80; 60J85; 60G50

DeepMartNet -- A Martingale Based Deep Neural Network Learning Method for Dirichlet BVPs and Eigenvalue Problems of Elliptic PDEs in R^d

Authors: Wei Cai, Andrew He, Daniel Margolis

Abstract: In this paper, we propose DeepMartNet - a Martingale based deep neural network learning method for solving Dirichlet boundary value problems (BVPs) and eigenvalue problems for elliptic partial differential equations (PDEs) in high dimensions or domains with complex geometries. The method is based on Varadhan's Martingale problem formulation for the BVPs/eigenvalue problems where a loss function en… ▽ More In this paper, we propose DeepMartNet - a Martingale based deep neural network learning method for solving Dirichlet boundary value problems (BVPs) and eigenvalue problems for elliptic partial differential equations (PDEs) in high dimensions or domains with complex geometries. The method is based on Varadhan's Martingale problem formulation for the BVPs/eigenvalue problems where a loss function enforcing the Martingale property for the PDE solution is used for an efficient optimization by sampling the stochastic processes associated with corresponding elliptic operators. High dimensional numerical results for BVPs of the linear and nonlinear Poisson-Boltzmann equation and eigenvalue problems of the Laplace equation and a Fokker-Planck equation demonstrate the capability of the proposed DeepMartNet learning method in solving high dimensional PDE problems. △ Less

Submitted 20 December, 2023; v1 submitted 15 November, 2023; originally announced November 2023.

Modeling Shortest Paths in Polymeric Networks using Spatial Branching Processes

Authors: Zhenyuan Zhang, Shaswat Mohanty, Jose Blanchet, Wei Cai

Abstract: Recent studies have established a connection between the macroscopic mechanical response of polymeric materials and the statistics of the shortest path (SP) length between distant nodes in the polymer network. Since these statistics can be costly to compute and difficult to study theoretically, we introduce a branching random walk (BRW) model to describe the SP statistics from the coarse-grained m… ▽ More Recent studies have established a connection between the macroscopic mechanical response of polymeric materials and the statistics of the shortest path (SP) length between distant nodes in the polymer network. Since these statistics can be costly to compute and difficult to study theoretically, we introduce a branching random walk (BRW) model to describe the SP statistics from the coarse-grained molecular dynamics (CGMD) simulations of polymer networks. We postulate that the first passage time (FPT) of the BRW to a given termination site can be used to approximate the statistics of the SP between distant nodes in the polymer network. We develop a theoretical framework for studying the FPT of spatial branching processes and obtain an analytical expression for estimating the FPT distribution as a function of the cross-link density. We demonstrate by extensive numerical calculations that the distribution of the FPT of the BRW model agrees well with the SP distribution from the CGMD simulations. The theoretical estimate and the corresponding numerical implementations of BRW provide an efficient way of approximating the SP distribution in a polymer network. Our results have the physical meaning that by accounting for the realistic topology of polymer networks, extensive bond-breaking is expected to occur at a much smaller stretch than that expected from idealized models assuming periodic network structures. Our work presents the first analysis of polymer networks as a BRW and sets the framework for develo** a generalizable spatial branching model for studying the macroscopic evolution of polymeric systems. △ Less

Submitted 30 March, 2024; v1 submitted 27 October, 2023; originally announced October 2023.

Comments: 37 pages, 17 figures

Optimal $L^2$ error estimates of mass- and energy-conserved FE schemes for a nonlinear Schrödinger-type system

Authors: Zhuoyue Zhang, Wentao Cai

Abstract: In this paper, we present an implicit Crank-Nicolson finite element (FE) scheme for solving a nonlinear Schrödinger-type system, which includes Schrödinger-Helmholz system and Schrödinger-Poisson system. In our numerical scheme, we employ an implicit Crank-Nicolson method for time discretization and a conforming FE method for spatial discretization. The proposed method is proved to be well-posedne… ▽ More In this paper, we present an implicit Crank-Nicolson finite element (FE) scheme for solving a nonlinear Schrödinger-type system, which includes Schrödinger-Helmholz system and Schrödinger-Poisson system. In our numerical scheme, we employ an implicit Crank-Nicolson method for time discretization and a conforming FE method for spatial discretization. The proposed method is proved to be well-posedness and ensures mass and energy conservation at the discrete level. Furthermore, we prove optimal $L^2$ error estimates for the fully discrete solutions. Finally, some numerical examples are provided to verify the convergence rate and conservation properties. △ Less

Submitted 10 May, 2024; v1 submitted 12 September, 2023; originally announced September 2023.

MSC Class: 65M60; 65N15; 65N30

Error analysis of energy-conservative BDF2-FE scheme for the 2D Navier-Stokes equations with variable density

Authors: **g**g Pan, Wentao Cai

Abstract: In this paper, we present an error estimate of a second-order linearized finite element (FE) method for the 2D Navier-Stokes equations with variable density. In order to get error estimates, we first introduce an equivalent form of the original system. Later, we propose a general BDF2-FE method for solving this equivalent form, where the Taylor-Hood FE space is used for discretizing the Navier-Sto… ▽ More In this paper, we present an error estimate of a second-order linearized finite element (FE) method for the 2D Navier-Stokes equations with variable density. In order to get error estimates, we first introduce an equivalent form of the original system. Later, we propose a general BDF2-FE method for solving this equivalent form, where the Taylor-Hood FE space is used for discretizing the Navier-Stokes equations and conforming FE space is used for discretizing density equation. We show that our scheme ensures discrete energy dissipation. Under the assumption of sufficient smoothness of strong solutions, an error estimate is presented for our numerical scheme for variable density incompressible flow in two dimensions. Finally, some numerical examples are provided to confirm our theoretical results. △ Less

Submitted 28 July, 2023; originally announced July 2023.

Comments: 22 pages, 1 figures

DeepMartNet -- A Martingale based Deep Neural Network Learning Algorithm for Eigenvalue/BVP Problems and Optimal Stochastic Controls

Authors: Wei Cai

Abstract: In this paper, we propose a neural network learning algorithm for solving eigenvalue problems and boundary value problems (BVPs) for elliptic operators and initial BVPs (IBVPs) of quasi-linear parabolic equations in high dimensions as well as optimal stochastic controls. The method is based on the Martingale property in the stochastic representation for the eigenvalue/BVP/IBVP problems and marting… ▽ More In this paper, we propose a neural network learning algorithm for solving eigenvalue problems and boundary value problems (BVPs) for elliptic operators and initial BVPs (IBVPs) of quasi-linear parabolic equations in high dimensions as well as optimal stochastic controls. The method is based on the Martingale property in the stochastic representation for the eigenvalue/BVP/IBVP problems and martingale principle for optimal stochastic controls. A loss function based on the Martingale property can be used for efficient optimization by sampling the stochastic processes associated with the elliptic operators or value process for stochastic controls. The proposed algorithm can be used for eigenvalue problems and BVPs and IBVPs with Dirichlet, Neumann, and Robin boundaries in bounded or unbounded domains and some feedback stochastic control problems. △ Less

Submitted 23 August, 2023; v1 submitted 21 July, 2023; originally announced July 2023.

Comments: update the loss function to enforce full Martingale property

A novel high-order linearly implicit and energy-stable additive Runge-Kutta methods for gradient flow models

Authors: Xuelong Gu, Wenjun Cai, Yushun Wang

Abstract: This paper introduces a novel paradigm for constructing linearly implicit and high-order unconditionally energy-stable schemes for general gradient flows, utilizing the scalar auxiliary variable (SAV) approach and the additive Runge-Kutta (ARK) methods. We provide a rigorous proof of energy stability, unique solvability, and convergence. The proposed schemes generalizes some recently developed hig… ▽ More This paper introduces a novel paradigm for constructing linearly implicit and high-order unconditionally energy-stable schemes for general gradient flows, utilizing the scalar auxiliary variable (SAV) approach and the additive Runge-Kutta (ARK) methods. We provide a rigorous proof of energy stability, unique solvability, and convergence. The proposed schemes generalizes some recently developed high-order, energy-stable schemes and address their shortcomings. On the one other hand, the proposed schemes can incorporate existing SAV-RK type methods after judiciously selecting the Butcher tables of ARK methods \cite{sav_li,sav_nlsw}. The order of a SAV-RKPC method can thus be confirmed theoretically by the order conditions of the corresponding ARK method. Several new schemes are constructed based on our framework, which perform to be more stable than existing SAV-RK type methods. On the other hand, the proposed schemes do not limit to a specific form of the nonlinear part of the free energy and can achieve high order with fewer intermediate stages compared to the convex splitting ARK methods \cite{csrk}. Numerical experiments demonstrate stability and efficiency of proposed schemes. △ Less

Submitted 8 July, 2023; originally announced July 2023.

Energy stable and maximum bound principle preserving schemes for the Allen-Cahn equation based on the Saul'yev methods

Authors: Xuelong Gu, Yushun Wang, Wenjun Cai

Abstract: The energy dissipation law and maximum bound principle are significant characteristics of the Allen-Chan equation. To preserve discrete counterpart of these properties, the linear part of the target system is usually discretized implicitly, resulting in a large linear or nonlinear system of equations. The Fast Fourier Transform (FFT) algorithm is commonly used to solve the resulting linear or nonl… ▽ More The energy dissipation law and maximum bound principle are significant characteristics of the Allen-Chan equation. To preserve discrete counterpart of these properties, the linear part of the target system is usually discretized implicitly, resulting in a large linear or nonlinear system of equations. The Fast Fourier Transform (FFT) algorithm is commonly used to solve the resulting linear or nonlinear systems with computational costs of $\mathcal{O}(M^d log M)$ at each time step, where $M$ is the number of spatial grid points in each direction, and $d$ is the dimension of the problem. Combining the Saul'yev methods and the stabilized technique, we propose and analyze novel first- and second-order numerical schemes for the Allen-Cahn equation in this paper. In contrast to the traditional methods, the proposed methods can be solved by components, requiring only $\mathcal{O}(M^d)$ computational costs per time step. Additionally, they preserve the maximum bound principle and original energy dissipation law at the discrete level. We also propose rigorous analysis of their consistency and convergence. Numerical experiments are conducted to confirm the theoretical analysis and demonstrate the efficiency of the proposed methods. △ Less

Submitted 30 May, 2023; originally announced May 2023.

Evaluating the Transferability of Machine-Learned Force Fields for Material Property Modeling

Authors: Shaswat Mohanty, Sanghyuk Yoo, Keonwook Kang, Wei Cai

Abstract: Machine-learned force fields have generated significant interest in recent years as a tool for molecular dynamics (MD) simulations, with the aim of develo** accurate and efficient models that can replace classical interatomic potentials. However, before these models can be confidently applied to materials simulations, they must be thoroughly tested and validated. The existing tests on the radial… ▽ More Machine-learned force fields have generated significant interest in recent years as a tool for molecular dynamics (MD) simulations, with the aim of develo** accurate and efficient models that can replace classical interatomic potentials. However, before these models can be confidently applied to materials simulations, they must be thoroughly tested and validated. The existing tests on the radial distribution function and mean-squared displacements are insufficient in assessing the transferability of these models. Here we present a more comprehensive set of benchmarking tests for evaluating the transferability of machine-learned force fields. We use a graph neural network (GNN)-based force field coupled with the OpenMM package to carry out MD simulations for Argon as a test case. Our tests include computational X-ray photon correlation spectroscopy (XPCS) signals, which capture the density fluctuation at various length scales in the liquid phase, as well as phonon density-of-state in the solid phase and the liquid-solid phase transition behavior. Our results show that the model can accurately capture the behavior of the solid phase only when the configurations from the solid phase are included in the training dataset. This underscores the importance of appropriately selecting the training data set when develo** machine-learned force fields. The tests presented in this work provide a necessary foundation for the development and application of machine-learned force fields for materials simulations. △ Less

Submitted 15 January, 2023; v1 submitted 9 January, 2023; originally announced January 2023.

Comments: 27 pages, 14 figures, under review

On Spectral Bias Reduction of Multi-scale Neural Networks for Regression Problems

Authors: Bo Wang, Heng Yuan, Lizuo Liu, Wenzhong Zhang, Wei Cai

Abstract: In this paper, we derive diffusion equation models in the spectral domain for the evolution of training errors of two-layer multi-scale deep neural networks (MscaleDNN) \cite{caixu2019,liu2020multi}, designed to reduce the spectral bias of fully connected deep neural networks in approximating oscillatory functions. The diffusion models are obtained from the spectral form of the error equation of t… ▽ More In this paper, we derive diffusion equation models in the spectral domain for the evolution of training errors of two-layer multi-scale deep neural networks (MscaleDNN) \cite{caixu2019,liu2020multi}, designed to reduce the spectral bias of fully connected deep neural networks in approximating oscillatory functions. The diffusion models are obtained from the spectral form of the error equation of the MscaleDNN, derived with a neural tangent kernel approach and gradient descent training and a sine activation function, assuming a vanishing learning rate and infinite network width and domain size. The involved diffusion coefficients are shown to have larger supports if more scales are used in the MscaleDNN, and thus, the proposed diffusion equation models in the frequency domain explain the MscaleDNN's spectral bias reduction capability. Numerical results of the diffusion models for a two-layer MscaleDNN training match with the error evolution of actual gradient descent training with a reasonably large network width, thus validating the effectiveness of the diffusion models. Meanwhile, the numerical results for MscaleDNN show error decay over a wide frequency range and confirm the advantage of using the MscaleDNN in approximating functions with a wide range of frequencies. △ Less

Submitted 20 October, 2023; v1 submitted 6 December, 2022; originally announced December 2022.

Unconditional convergence of conservative spectral Galerkin methods for the coupled fractional nonlinear Klein-Gordon-Schrödinger equations

Authors: Dongdong Hu, Yayun Fu, Wenjun Cai, Yushun Wang

Abstract: In this work, two novel classes of structure-preserving spectral Galerkin methods are proposed which based on the Crank-Nicolson scheme and the exponential scalar auxiliary variable method respectively, for solving the coupled fractional nonlinear Klein-Gordon-Schrödinger equation. The paper focuses on the theoretical analyses and computational efficiency of the proposed schemes, the Crank-Nicolos… ▽ More In this work, two novel classes of structure-preserving spectral Galerkin methods are proposed which based on the Crank-Nicolson scheme and the exponential scalar auxiliary variable method respectively, for solving the coupled fractional nonlinear Klein-Gordon-Schrödinger equation. The paper focuses on the theoretical analyses and computational efficiency of the proposed schemes, the Crank-Nicoloson scheme is proved to be unconditionally convergent and has the maximum-norm boundness of numerical solutions. The exponential scalar auxiliary variable scheme is linearly implicit and decoupled, but lack of the maximum-norm boundness, also, the energy structure has been modified. Subsequently, the efficient implementations of the proposed schemes are introduced in detail. Both the theoretical analyses and the numerical comparisons show that the proposed spectral Galerkin methods have high efficiency in long-time computations. △ Less

Submitted 5 October, 2022; originally announced October 2022.

A Causality-DeepONet for Causal Responses of Linear Dynamical Systems

Authors: Lizuo Liu, Kamaljyoti Nath, Wei Cai

Abstract: In this paper, we propose a DeepONet structure with causality to represent the causal linear operators between Banach spaces of time-dependent signals. The theorem of universal approximations to nonlinear operators proposed in \cite{tian**chen1995} is extended to operators with causalities, and the proposed Causality-DeepONet implements the physical causality in its framework. The proposed Causa… ▽ More In this paper, we propose a DeepONet structure with causality to represent the causal linear operators between Banach spaces of time-dependent signals. The theorem of universal approximations to nonlinear operators proposed in \cite{tian**chen1995} is extended to operators with causalities, and the proposed Causality-DeepONet implements the physical causality in its framework. The proposed Causality-DeepONet considers causality (the state of the system at the current time is not affected by that of the future, but only by its current state and past history) and uses a convolution-type weight in its design. To demonstrate its effectiveness in handling the causal response of a physical system, the Causality-DeepONet is applied to learn the operator representing the response of a building due to earthquake ground accelerations. Extensive numerical tests and comparisons with some existing variants of DeepONet are carried out, and the Causality-DeepONet clearly shows its unique capability to learn the retarded dynamic responses of the seismic response operator with good accuracy. △ Less

Submitted 17 September, 2022; originally announced September 2022.

MSC Class: 65R20; 65Z05; 78M25

Linearly implicit energy-preserving integrating factor methods for the 2D nonlinear Schrödinger equation with wave operator and convergence analysis

Authors: Xuelong Gu, Wenjun Cai, Chaolong Jiang, Yushun Wang

Abstract: In this paper, we develop a novel class of linear energy-preserving integrating factor methods for the 2D nonlinear Schrödinger equation with wave operator (NLSW), combining the scalar auxiliary variable approach and the integrating factor methods. A second-order scheme is first proposed, which is rigorously proved to be energy-preserving. By using the energy methods, we analyze its optimal conver… ▽ More In this paper, we develop a novel class of linear energy-preserving integrating factor methods for the 2D nonlinear Schrödinger equation with wave operator (NLSW), combining the scalar auxiliary variable approach and the integrating factor methods. A second-order scheme is first proposed, which is rigorously proved to be energy-preserving. By using the energy methods, we analyze its optimal convergence in the $H^1$ norm without any restrictions on the grid ratio, where a novel technique and an improved induction argument are proposed to overcome the difficulty posed by the unavailability of a priori $L^\infty$ estimates of numerical solutions. Based on the integrating factor Runge-Kutta methods, we extend the proposed scheme to arbitrarily high order, which is also linear and conservative. Numerical experiments are presented to confirm the theoretical analysis and demonstrate the advantages of the proposed methods. △ Less

Submitted 25 September, 2022; v1 submitted 16 September, 2022; originally announced September 2022.

On Mixing Distributions Via Random Orthogonal Matrices and the Spectrum of the Singular Values of Multi-Z Shaped Graph Matrices

Authors: Wenjun Cai, Aaron Potechin

Abstract: In this paper, we introduce and analyze a new operation $\circ_{R}$ which mixes two distributions $Ω$ and $Ω'$ via a random orthogonal matrix. In particular, we take $Ω\circ_R Ω'$ to be the limit as $n \to \infty$ of the distribution of singular values of $DRD'$ where $D$ and $D'$ are $n \times n$ diagonal matrices whose diagonal entries have distributions $Ω$ and $Ω'$ respectively and $R$ is a ra… ▽ More In this paper, we introduce and analyze a new operation $\circ_{R}$ which mixes two distributions $Ω$ and $Ω'$ via a random orthogonal matrix. In particular, we take $Ω\circ_R Ω'$ to be the limit as $n \to \infty$ of the distribution of singular values of $DRD'$ where $D$ and $D'$ are $n \times n$ diagonal matrices whose diagonal entries have distributions $Ω$ and $Ω'$ respectively and $R$ is a random $n \times n$ orthogonal matrix. We show that $\circ_R$ has several nice properties. We first observe that $\circ_R$ is commutative and associative and compute the moments of $Ω\circ_R Ω'$ in terms of the moments of $Ω$ and $Ω'$. We then show that $\circ_R$ interacts very nicely with the spectrum of the singular values of Z-shaped and multi-Z-shaped graph matrices. This allows us to answer the question posed by our previous paper of how to describe the spectrum of the singular values of Z-shaped and multi-Z-shaped graph matrices when the input distribution is not $\{-1,1\}$. In our analysis, we show that the moments of our distributions are closely connected to non-crossing partitions and prove a number of new results on non-crossing partitions which may be of independent interest. △ Less

Submitted 28 December, 2022; v1 submitted 5 June, 2022; originally announced June 2022.

Comments: Updates from the previous version: the paper is rewritten to highlight the connection between our results and free probability theory. In particular, Section 3 is now written in the language of free probability theory and our original result for the \circ_R operation is stated as a corollary

Computational Approaches to Model X-ray Photon Correlation Spectroscopy from Molecular Dynamics

Authors: Shaswat Mohanty, Christopher B. Cooper, Hui Wang, Mengning Liang, Wei Cai

Abstract: X-ray photon correlation spectroscopy (XPCS) allows for the resolution of dynamic processes within a material across a wide range of length and time scales. X-ray speckle visibility spectroscopy (XSVS) is a related method that uses a single diffraction pattern to probe ultrafast dynamics. Interpretation of the XPCS and XSVS data in terms of underlying physical processes is necessary to establish t… ▽ More X-ray photon correlation spectroscopy (XPCS) allows for the resolution of dynamic processes within a material across a wide range of length and time scales. X-ray speckle visibility spectroscopy (XSVS) is a related method that uses a single diffraction pattern to probe ultrafast dynamics. Interpretation of the XPCS and XSVS data in terms of underlying physical processes is necessary to establish the connection between the macroscopic responses and the microstructural dynamics. To aid the interpretation of the XPCS and XSVS data, we present a computational framework to model these experiments by computing the X-ray scattering intensity directly from the atomic positions obtained from molecular dynamics (MD) simulations. We compare the efficiency and accuracy of two alternative computational methods: the direct method computing the intensity at each diffraction vector separately, and a method based on fast Fourier transform that computes the intensities at all diffraction vectors at once. The computed X-ray speckle patterns capture the density fluctuations over a range of length and time scales and are shown to reproduce the known properties and relations of experimental XPCS and XSVS for liquids. △ Less

Submitted 5 January, 2023; v1 submitted 27 April, 2022; originally announced April 2022.

Comments: 31 pages, 11 figures, submitted to Modelling and Simulations in Materials Science and Engineering

Journal ref: Modelling and Simulation in Materials Science and Engineering 30 (2022) 075004

DeepPropNet -- A Recursive Deep Propagator Neural Network for Learning Evolution PDE Operators

Authors: Lizuo Liu, Wei Cai

Abstract: In this paper, we propose a deep neural network approximation to the evolution operator for time dependent PDE systems over long time period by recursively using one single neural network propagator, in the form of POD-DeepONet with built-in causality feature, for a small time interval. The trained DeepPropNet of moderate size is shown to give accurate prediction of wave solutions over the whole t… ▽ More In this paper, we propose a deep neural network approximation to the evolution operator for time dependent PDE systems over long time period by recursively using one single neural network propagator, in the form of POD-DeepONet with built-in causality feature, for a small time interval. The trained DeepPropNet of moderate size is shown to give accurate prediction of wave solutions over the whole time interval. △ Less

Submitted 27 February, 2022; originally announced February 2022.

Multiscale DeepONet for Nonlinear Operators in Oscillatory Function Spaces for Building Seismic Wave Responses

Authors: Lizuo Liu, Wei Cai

Abstract: In this paper, we propose a multiscale DeepONet to represent nonlinear operator between Banach spaces of highly oscillatory continuous functions. The multiscale deep neural network (DNN) utilizes a multiple scaling technique to convert high frequency function to lower frequency functions before using a DNN to learn a specific range of frequency of the function. The multi-scale concept is integrate… ▽ More In this paper, we propose a multiscale DeepONet to represent nonlinear operator between Banach spaces of highly oscillatory continuous functions. The multiscale deep neural network (DNN) utilizes a multiple scaling technique to convert high frequency function to lower frequency functions before using a DNN to learn a specific range of frequency of the function. The multi-scale concept is integrated into the DeepONet which is based on a universal approximation theory of nonlinear operators. The resulting multi-scale DeepONet is shown to be effective to represent building seismic response operator which maps oscillatory seismic excitation to the oscillatory building responses. △ Less

Submitted 8 November, 2021; originally announced November 2021.

Efficient energy-preserving exponential integrators for multi-components Hamiltonian systems

Authors: X. Gu, C. Jiang, Y. Wang, W. Cai

Abstract: In this paper, we develop a framework to construct energy-preserving methods for multi-components Hamiltonian systems, combining the exponential integrator and the partitioned averaged vector field method. This leads to numerical schemes with both advantages of long-time stability and excellent behavior for highly oscillatory or stiff problems. Compared to the existing energy-preserving exponentia… ▽ More In this paper, we develop a framework to construct energy-preserving methods for multi-components Hamiltonian systems, combining the exponential integrator and the partitioned averaged vector field method. This leads to numerical schemes with both advantages of long-time stability and excellent behavior for highly oscillatory or stiff problems. Compared to the existing energy-preserving exponential integrators (EP-EI) in practical implementation, our proposed methods are much efficient which can at least be computed by subsystem instead of handling a nonlinear coupling system at a time. Moreover, for most cases, such as the Klein-Gordon-Schrödinger equations and the Klein-Gordon-Zakharov equations considered in this paper, the computational cost can be further reduced. Specifically, one part of the derived schemes is totally explicit, and the other is linearly implicit. In addition, we present rigorous proof of conserving the original energy of Hamiltonian systems, in which an alternative technique is utilized so that no additional assumptions are required, in contrast to the proof strategies used for the existing EP-EI. Numerical experiments are provided to demonstrate the significant advantages in accuracy, computational efficiency, and the ability to capture highly oscillatory solutions. △ Less

Submitted 5 November, 2021; v1 submitted 8 October, 2021; originally announced October 2021.

Comments: 29 pages, 68 figures

Linearized Learning Methods with Multiscale Deep Neural Networks for Stationary Navier-Stokes Equations with Oscillatory Solutions

Authors: Lizuo Liu, Bo Wang, Wei Cai

Abstract: In this paper, we present linearized learning methods to accelerate the convergence of training for stationary nonlinear Navier-Stokes equations. To solve the stationary nonlinear Navier-Stokes (NS) equation, we integrate the procedure of linearization of the nonlinear convection term in the NS equation into the training process of multi-scale deep neural network approximation of the NS solution.… ▽ More In this paper, we present linearized learning methods to accelerate the convergence of training for stationary nonlinear Navier-Stokes equations. To solve the stationary nonlinear Navier-Stokes (NS) equation, we integrate the procedure of linearization of the nonlinear convection term in the NS equation into the training process of multi-scale deep neural network approximation of the NS solution. Four forms of linearizations are considered. After a benchmark problem, we solve the highly oscillating stationary flows utilizing the proposed linearized learning with multi-scale neural network for complex domains. The results show that multiscale deep neural network combining with the linearized schemes can be trained fast and accurately. △ Less

Submitted 5 April, 2021; v1 submitted 5 February, 2021; originally announced February 2021.

FBSDE based Neural Network Algorithms for High-Dimensional Quasilinear Parabolic PDEs

Authors: Wenzhong Zhang, Wei Cai

Abstract: In this paper, we propose forward and backward stochastic differential equations (FBSDEs) based deep neural network (DNN) learning algorithms for the solution of high dimensional quasilinear parabolic partial differential equations (PDEs), which are related to the FBSDEs by the Pardoux-Peng theory. The algorithms rely on a learning process by minimizing the pathwise difference between two discrete… ▽ More In this paper, we propose forward and backward stochastic differential equations (FBSDEs) based deep neural network (DNN) learning algorithms for the solution of high dimensional quasilinear parabolic partial differential equations (PDEs), which are related to the FBSDEs by the Pardoux-Peng theory. The algorithms rely on a learning process by minimizing the pathwise difference between two discrete stochastic processes, defined by the time discretization of the FBSDEs and the DNN representation of the PDE solutions, respectively. The proposed algorithms are shown to generate DNN solutions for a 100-dimensional Black--Scholes--Barenblatt equation, accurate in a finite region in the solution space, and has a convergence rate similar to that of the Euler--Maruyama discretization used for the FBSDEs. As a result, a Richardson extrapolation technique over time discretizations can be used to enhance the accuracy of the DNN solutions. For time oscillatory solutions, a multiscale DNN is shown to improve the performance of the FBSDE DNN for high frequencies. △ Less

Submitted 6 May, 2021; v1 submitted 14 December, 2020; originally announced December 2020.

Arbitrary high-order linearly implicit energy-preserving algorithms for Hamiltonian PDEs

Authors: Yonghui Bo, Yushun Wang, Wenjun Cai

Abstract: In this paper, we present a novel strategy to systematically construct linearly implicit energy-preserving schemes with arbitrary order of accuracy for Hamiltonian PDEs. Such novel strategy is based on the newly developed exponential scalar variable (ESAV) approach that can remove the bounded-from-blew restriction of nonlinear terms in the Hamiltonian functional and provides a totally explicit dis… ▽ More In this paper, we present a novel strategy to systematically construct linearly implicit energy-preserving schemes with arbitrary order of accuracy for Hamiltonian PDEs. Such novel strategy is based on the newly developed exponential scalar variable (ESAV) approach that can remove the bounded-from-blew restriction of nonlinear terms in the Hamiltonian functional and provides a totally explicit discretization of the auxiliary variable without computing extra inner products, which make it more effective and applicable than the traditional scalar auxiliary variable (SAV) approach. To achieve arbitrary high-order accuracy and energy preservation, we utilize the symplectic Runge-Kutta method for both solution variables and the auxiliary variable, where the values of internal stages in nonlinear terms are explicitly derived via an extrapolation from numerical solutions already obtained in the preceding calculation. A prediction-correction strategy is proposed to further improve the accuracy. Fourier pseudo-spectral method is then employed to obtain fully discrete schemes. Compared with the SAV schemes, the solution variables and the auxiliary variable in these ESAV schemes are now decoupled. Moreover, when the linear terms are of constant coefficients, the solution variables can be explicitly solved by using the fast Fourier transform. Numerical experiments are carried out for three Hamiltonian PDEs to demonstrate the efficiency and conservation of the ESAV schemes. △ Less

Submitted 16 November, 2020; originally announced November 2020.

Journal ref: Numerical Algorithms 90 (2022) 1519-1546

Multi-scale Deep Neural Network (MscaleDNN) Methods for Oscillatory Stokes Flows in Complex Domains

Authors: Bo Wang, Wenzhong Zhang, Wei Cai

Abstract: In this paper, we study a multi-scale deep neural network (MscaleDNN) as a meshless numerical method for computing oscillatory Stokes flows in complex domains. The MscaleDNN employs a multi-scale structure in the design of its DNN using radial scalings to convert the approximation of high frequency components of the highly oscillatory Stokes solution to one of lower frequencies. The MscaleDNN solu… ▽ More In this paper, we study a multi-scale deep neural network (MscaleDNN) as a meshless numerical method for computing oscillatory Stokes flows in complex domains. The MscaleDNN employs a multi-scale structure in the design of its DNN using radial scalings to convert the approximation of high frequency components of the highly oscillatory Stokes solution to one of lower frequencies. The MscaleDNN solution to the Stokes problem is obtained by minimizing a loss function in terms of L2 normof the residual of the Stokes equation. Three forms of loss functions are investigated based on vorticity-velocity-pressure, velocity-stress-pressure, and velocity-gradient of velocity-pressure formulations of the Stokes equation. We first conduct a systematic study of the MscaleDNN methods with various loss functions on the Kovasznay flow in comparison with normal fully connected DNNs. Then, Stokes flows with highly oscillatory solutions in a 2-D domain with six randomly placed holes are simulated by the MscaleDNN. The results show that MscaleDNN has faster convergence and consistent error decays in the simulation of Kovasznay flow for all four tested loss functions. More importantly, the MscaleDNN is capable of learning highly oscillatory solutions when the normal DNNs fail to converge. △ Less

Submitted 28 October, 2020; v1 submitted 26 September, 2020; originally announced September 2020.

An explicit and practically invariants-preserving method for conservative systems

Authors: Wenjun Cai, Yuezheng Gong, Yushun Wang

Abstract: An explicit numerical strategy that practically preserves invariants is derived for conservative systems by combining an explicit high-order Runge-Kutta (RK) scheme with a simple modification of the standard projection approach, which is named the explicit invariants-preserving (EIP) method. The proposed approach is shown to have the same order as the underlying RK method, while the error of invar… ▽ More An explicit numerical strategy that practically preserves invariants is derived for conservative systems by combining an explicit high-order Runge-Kutta (RK) scheme with a simple modification of the standard projection approach, which is named the explicit invariants-preserving (EIP) method. The proposed approach is shown to have the same order as the underlying RK method, while the error of invariants is analyzed in the order of $\mathcal{O}\left(h^{2(p+1)}\right),$ where $h$ is the time step and $p$ represents the order of the method. When $p$ is appropriately large, the EIP method is practically invariants-conserving because the error of invariants can reach the machine accuracy. The method is illustrated for the cases of single and multiple invariants, with regard to both ODEs and high-dimensional PDEs. Extensive numerical experiments are presented to verify our theoretical results and demonstrate the superior behaviors of the proposed method in a long time numerical simulation. Numerical results suggest that the fourth-order EIP method preserves much better the qualitative properties of the flow than the standard fourth-order RK method and it is more efficient in practice than the fully implicit integrators. △ Less

Submitted 15 September, 2020; originally announced September 2020.

Comments: 25 pages, 61 figures

A Matrix Basis Formulation For The Green's Functions Of Maxwell's Equations And The Elastic Wave Equations In Layered Media

Authors: Wenzhong Zhang, Bo Wang, Wei Cai

Abstract: A matrix basis formulation is introduced to represent the 3 x 3 dyadic Green's functions in the frequency domain for the Maxwell's equations and the elastic wave equation in layered media. The formulation can be used to decompose the Maxwell's Green's functions into independent TE and TM components, each satisfying a Helmholtz equation, and decompose the elastic wave Green's function into the S-wa… ▽ More A matrix basis formulation is introduced to represent the 3 x 3 dyadic Green's functions in the frequency domain for the Maxwell's equations and the elastic wave equation in layered media. The formulation can be used to decompose the Maxwell's Green's functions into independent TE and TM components, each satisfying a Helmholtz equation, and decompose the elastic wave Green's function into the S-wave and the P-wave components. In addition, a derived vector basis formulation is applied to the case for acoustic wave sources from a non-viscous fluid layer. △ Less

Submitted 3 August, 2020; originally announced August 2020.

Exponential convergence for multipole and local expansions and their translations for sources in layered media: three-dimensional Laplace equation

Authors: Bo Wang, Wenzhong Zhang, Wei Cai

Abstract: In this paper, we prove the exponential convergence of the multipole and local expansions, shifting and translation operators used in fast multipole methods (FMMs) for 3-dimensional Laplace equations in layered media. These theoretical results ensure the exponential convergence of the FMM which has been shown by the numerical results recently reported in [9]. As the free space components are calcu… ▽ More In this paper, we prove the exponential convergence of the multipole and local expansions, shifting and translation operators used in fast multipole methods (FMMs) for 3-dimensional Laplace equations in layered media. These theoretical results ensure the exponential convergence of the FMM which has been shown by the numerical results recently reported in [9]. As the free space components are calculated by the classic FMM, this paper will focus on the analysis for the reaction components of the Green's function for the Laplace equation in layered media. We first prove that the density functions in the integral representations of the reaction components are analytic and bounded in the right half complex plane. Then, using the Cagniard-de Hoop transform and contour deformations, estimate for the remainder terms of the truncated expansions is given, and, as a result, the exponential convergence for the expansions and translation operators is proven. △ Less

Submitted 2 August, 2020; originally announced August 2020.

Dynamics of many species through competition for resources

Authors: Wenli Cai, Hailiang Liu

Abstract: This paper is concerned with a mathematical model of competition for resource where species consume noninteracting resources. This system of differential equations is formally obtained by renormalizing the MacArthur's competition model at equilibrium, and agrees with the trait-continuous model studied by Mirrahimi S, Perthame B, Wakano JY [J. Math. Biol. 64(7): 1189-1223, 2012]. As a dynamical sys… ▽ More This paper is concerned with a mathematical model of competition for resource where species consume noninteracting resources. This system of differential equations is formally obtained by renormalizing the MacArthur's competition model at equilibrium, and agrees with the trait-continuous model studied by Mirrahimi S, Perthame B, Wakano JY [J. Math. Biol. 64(7): 1189-1223, 2012]. As a dynamical system, self-organized generation of distinct species occurs. The necessary conditions for survival are given. We prove the existence of the evolutionary stable distribution (ESD) through an optimization problem and present an independent algorithm to compute the ESD directly. Under certain structural conditions, solutions of the system are shown to approach the discrete ESD as time evolves. The time discretization of the system is proven to satisfy two desired properties: positivity and energy dissipation. Numerical examples are given to illustrate certain interesting biological phenomena. △ Less

Submitted 2 July, 2020; originally announced July 2020.

MSC Class: 37N25; 65M08; 92D15

Multi-scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domains

Authors: Ziqi Liu, Wei Cai, Zhi-Qin John Xu

Abstract: In this paper, we propose multi-scale deep neural networks (MscaleDNNs) using the idea of radial scaling in frequency domain and activation functions with compact support. The radial scaling converts the problem of approximation of high frequency contents of PDEs' solutions to a problem of learning about lower frequency functions, and the compact support activation functions facilitate the separat… ▽ More In this paper, we propose multi-scale deep neural networks (MscaleDNNs) using the idea of radial scaling in frequency domain and activation functions with compact support. The radial scaling converts the problem of approximation of high frequency contents of PDEs' solutions to a problem of learning about lower frequency functions, and the compact support activation functions facilitate the separation of frequency contents of the target function to be approximated by corresponding DNNs. As a result, the MscaleDNNs achieve fast uniform convergence over multiple scales. The proposed MscaleDNNs are shown to be superior to traditional fully connected DNNs and be an effective mesh-less numerical method for Poisson-Boltzmann equations with ample frequency contents over complex and singular domains. △ Less

Submitted 28 September, 2020; v1 submitted 22 July, 2020; originally announced July 2020.

The Spectrum of the Singular Values of Z-Shaped Graph Matrices

Authors: Wenjun Cai, Aaron Potechin

Abstract: Graph matrices are a type of matrix which has played a crucial role in analyzing the sum of squares hierarchy on average case problems. However, except for rough norm bounds, little is known about graph matrices. In this paper, we take a step towards better understanding graph matrices by determining the limiting distribution of the spectrum of the singular values of Z-shaped graph matrices. We th… ▽ More Graph matrices are a type of matrix which has played a crucial role in analyzing the sum of squares hierarchy on average case problems. However, except for rough norm bounds, little is known about graph matrices. In this paper, we take a step towards better understanding graph matrices by determining the limiting distribution of the spectrum of the singular values of Z-shaped graph matrices. We then give a partial generalization of our results for $m$-layer Z-shaped graph matrices. △ Less

Submitted 25 June, 2024; v1 submitted 24 June, 2020; originally announced June 2020.

Deformation of Cayley's hyperdeterminants

Authors: Tommy Wuxing Cai, Naihuan **g

Abstract: We introduce a deformation of Cayley's second hyperdeterminant for even-dimensional hypermatrices. As an application, we formulate a generalization of the Jacobi-Trudi formula for Macdonald functions of rectangular shapes generalizing Matsumoto's formula for Jack functions. We introduce a deformation of Cayley's second hyperdeterminant for even-dimensional hypermatrices. As an application, we formulate a generalization of the Jacobi-Trudi formula for Macdonald functions of rectangular shapes generalizing Matsumoto's formula for Jack functions. △ Less

Submitted 5 June, 2020; v1 submitted 2 June, 2020; originally announced June 2020.

Comments: 9 pages, 0 figures

MSC Class: Primary: 05E05; Secondary: 17B69; 05E10

Journal ref: Elec. J. Combin. 27(2) (2020) P2.50

Fast Multipole Method for 3-D Linearized Poisson-Boltzmann Equation in Layered Media

Authors: Bo Wang, Wen Zhong Zhang, Wei Cai

Abstract: In this paper, we propose a fast multipole method (FMM) for 3-D linearized Poisson-Boltzmann (PB) equation in layered media. The main framework of the algorithm is analogous to the FMM for Helmholtz and Laplace equation in layered media [1,2], using an extension of the Funk-Hecke formula for pure imaginary wave number. Moreover, a recurrence formula is provided for the run-time computation of the… ▽ More In this paper, we propose a fast multipole method (FMM) for 3-D linearized Poisson-Boltzmann (PB) equation in layered media. The main framework of the algorithm is analogous to the FMM for Helmholtz and Laplace equation in layered media [1,2], using an extension of the Funk-Hecke formula for pure imaginary wave number. Moreover, a recurrence formula is provided for the run-time computation of the Sommerfeld-type integrals used in the FMM algorithm. Due to the similarity between Helmholtz and linearized PB equation, the recurrence formula can also be used for the FMM of Helmholtz equation in layered media with minor changes as mentioned in [1] . Numerical results validate that the FMM for interactions of charges under screen's potentials in layered media has the same accuracy and CPU complexity as the classic FMM for charge interactions in free space. △ Less

Submitted 3 February, 2020; originally announced February 2020.

Comments: arXiv admin note: substantial text overlap with arXiv:1908.10863, arXiv:1902.05132

Correlation functions of charged free boson and fermion systems

Authors: Naihuan **g, Zhijun Li, Tommy Wuxing Cai

Abstract: Using the idea of the quantum inverse scattering method, we introduce the operators $\mathbf{B}(x), \mathbf{C}(x)$ and $\mathbf{\tilde{B}}(x), \mathbf{\tilde{C}}(x)$ corresponding to the off-diagonal entries of the monodromy matrix $T$ for the phase model and $i$-boson model in terms of bc fermions and neutral fermions respectively, thus giving alternative treatment of the KP and BKP hierarchies.… ▽ More Using the idea of the quantum inverse scattering method, we introduce the operators $\mathbf{B}(x), \mathbf{C}(x)$ and $\mathbf{\tilde{B}}(x), \mathbf{\tilde{C}}(x)$ corresponding to the off-diagonal entries of the monodromy matrix $T$ for the phase model and $i$-boson model in terms of bc fermions and neutral fermions respectively, thus giving alternative treatment of the KP and BKP hierarchies. We also introduce analogous operators $\mathbf{B}^{*}(x)$ and $\mathbf{C}^{*}(x)$ for the charged free boson system and show that they are in complete analogy to those of $bc$ fermionic fields. It is proved that the correlation function $\langle 0|\mathbf{C}(x_N)\cdots\mathbf{C}(x_1)\mathbf{B}(y_1)\cdots $ $\mathbf{B}(y_N)|0\rangle$ in the $bc$ fermionic fields is the inverse of the correlation function $\langle 0|\mathbf{C}^{*}(x_N)\cdots\mathbf{C}^{*}(x_1)\mathbf{B}^{*}(y_1)\cdots \mathbf{B}^{*}(y_N)|0\rangle$ in the charged free bosons. △ Less

Submitted 16 June, 2020; v1 submitted 9 December, 2019; originally announced December 2019.

Comments: 26 pages. Final version for J. Stat. Mech

MSC Class: Primary: 17B37; Secondary: 58A17; 15A75; 15B33; 15A15; 05E05

Journal ref: J. Stat. Mech. (2020), 083101, 27pp

Two novel classes of arbitrary high-order structure-preserving algorithms for canonical Hamiltonian systems

Authors: Yonghui Bo, Wenjun Cai, Yushun Wang

Abstract: In this paper, we systematically construct two classes of structure-preserving schemes with arbitrary order of accuracy for canonical Hamiltonian systems. The one class is the symplectic scheme, which contains two new families of parameterized symplectic schemes that are derived by basing on the generating function method and the symmetric composition method, respectively. Each member in these sch… ▽ More In this paper, we systematically construct two classes of structure-preserving schemes with arbitrary order of accuracy for canonical Hamiltonian systems. The one class is the symplectic scheme, which contains two new families of parameterized symplectic schemes that are derived by basing on the generating function method and the symmetric composition method, respectively. Each member in these schemes is symplectic for any fixed parameter. A more general form of generating functions is introduced, which generalizes the three classical generating functions that are widely used to construct symplectic algorithms. The other class is a novel family of energy and quadratic invariants preserving schemes, which is devised by adjusting the parameter in parameterized symplectic schemes to guarantee energy conservation at each time step. The existence of the solutions of these schemes is verified. Numerical experiments demonstrate the theoretical analysis and conservation of the proposed schemes. △ Less

Submitted 24 December, 2020; v1 submitted 2 December, 2019; originally announced December 2019.

Journal ref: Journal of Computational Mathematics 41 (2023) 375-394

Structure-preserving algorithms for multi-dimensional fractional Klein-Gordon-Schrödinger equation

Authors: Yayun Fu Wenjun Cai, Yushun Wang

Abstract: This paper aims to construct structure-preserving numerical schemes for multi-dimensional space fractional Klein-Gordon-Schrödinger equation, which are based on the newly developed partitioned averaged vector field methods. First, we derive an equivalent equation, and reformulate the equation as a canonical Hamiltonian system by virtue of the variational derivative of the functional with fractiona… ▽ More This paper aims to construct structure-preserving numerical schemes for multi-dimensional space fractional Klein-Gordon-Schrödinger equation, which are based on the newly developed partitioned averaged vector field methods. First, we derive an equivalent equation, and reformulate the equation as a canonical Hamiltonian system by virtue of the variational derivative of the functional with fractional Laplacian. Then, we develop a semi-discrete conservative scheme via using the Fourier pseudo-spectral method to discrete the equation in space direction. Further applying the partitioned averaged vector field methods on the temporal direction gives a class of fully-discrete schemes that can preserve the mass and energy exactly. Numerical examples are provided to confirm our theoretical analysis results at last. △ Less

Submitted 26 November, 2019; v1 submitted 25 November, 2019; originally announced November 2019.

Comments: 26 pages, 13 figures

A structure-preserving algorithm for the fractional nonlinear Schrödinger equation based on the SAV approach

Authors: Yayun Fu, Wenjun Cai, Yushun Wang

Abstract: The main objective of this paper is to present an efficient structure-preserving scheme, which is based on the idea of the scalar auxiliary variable approach, for solving the space fractional nonlinear Schrödinger equation. First, we reformulate the equation as a Hamiltonian system, and obtain a new equivalent system via introducing a scalar variable. Then, we construct a semi-discrete energy-pres… ▽ More The main objective of this paper is to present an efficient structure-preserving scheme, which is based on the idea of the scalar auxiliary variable approach, for solving the space fractional nonlinear Schrödinger equation. First, we reformulate the equation as a Hamiltonian system, and obtain a new equivalent system via introducing a scalar variable. Then, we construct a semi-discrete energy-preserving scheme by using the Fourier pseudo-spectral method to discretize the equivalent system in space direction. After that, applying the Crank-Nicolson method on the temporal direction gives a linear implicit scheme in the fully-discrete version. As expected, the proposed scheme can preserve the energy exactly and more efficient in the sense that only decoupled equations with constant coefficients need to be solved at each time step. Finally, numerical experiments are provided to demonstrate the effectiveness and conservation of the scheme. △ Less

Submitted 17 November, 2019; originally announced November 2019.

Comments: 24 pages,7 figures

A linearly implicit structure-preserving scheme for the fractional sine-Gordon equation based on the IEQ approach

Authors: Yayun Fu, Wenjun Cai, Yushun Wang

Abstract: This paper aims to develop a linearly implicit structure-preserving numerical scheme for the space fractional sine-Gordon equation, which is based on the newly developed invariant energy quadratization method. First, we reformulate the equation as a canonical Hamiltonian system by virtue of the variational derivative of the functional with fractional Laplacian. Then, we utilize the fractional cent… ▽ More This paper aims to develop a linearly implicit structure-preserving numerical scheme for the space fractional sine-Gordon equation, which is based on the newly developed invariant energy quadratization method. First, we reformulate the equation as a canonical Hamiltonian system by virtue of the variational derivative of the functional with fractional Laplacian. Then, we utilize the fractional centered difference formula to discrete the equivalent system derived by the invariant energy quadratization method in space direction, and obtain a conservative semi-discrete scheme. Subsequently, the linearly implicit structure-preserving method is applied for the resulting semi-discrete system to arrive at a fully-discrete conservative scheme. The stability, solvability and convergence in the maximum norm of the numerical scheme are given. Furthermore, a fast algorithm based on the fast Fourier transformation technique is used to reduce the computational complexity in practical computation. Finally, numerical examples are provided to confirm our theoretical analysis results. △ Less

Submitted 16 November, 2019; originally announced November 2019.

Comments: 28pages, 5 figures

Multi-scale Deep Neural Networks for Solving High Dimensional PDEs

Authors: Wei Cai, Zhi-Qin John Xu

Abstract: In this paper, we propose the idea of radial scaling in frequency domain and activation functions with compact support to produce a multi-scale DNN (MscaleDNN), which will have the multi-scale capability in approximating high frequency and high dimensional functions and speeding up the solution of high dimensional PDEs. Numerical results on high dimensional function fitting and solutions of high d… ▽ More In this paper, we propose the idea of radial scaling in frequency domain and activation functions with compact support to produce a multi-scale DNN (MscaleDNN), which will have the multi-scale capability in approximating high frequency and high dimensional functions and speeding up the solution of high dimensional PDEs. Numerical results on high dimensional function fitting and solutions of high dimensional PDEs, using loss functions with either Ritz energy or least squared PDE residuals, have validated the increased power of multi-scale resolution and high frequency capturing of the proposed MscaleDNN. △ Less

Submitted 25 October, 2019; originally announced October 2019.

Optimal error estimate of a conservative Fourier pseudo-spectral method for the space fractional nonlinear Schrödinger equation

Authors: Zhuangzhi Xu, Wenjun Cai, Chaolong Jiang, Yushun Wang

Abstract: In this paper, we consider the error analysis of a conservative Fourier pseudo-spectral method that conserves mass and energy for the space fractional nonlinear Schrödinger equation. We give a new fractional Sobolev norm that can construct the discrete fractional Sobolev space, and we also can prove some important lemmas for the new fractional Sobolev norm. Based on these lemmas and energy method,… ▽ More In this paper, we consider the error analysis of a conservative Fourier pseudo-spectral method that conserves mass and energy for the space fractional nonlinear Schrödinger equation. We give a new fractional Sobolev norm that can construct the discrete fractional Sobolev space, and we also can prove some important lemmas for the new fractional Sobolev norm. Based on these lemmas and energy method, a priori error estimate for the method can be established. Then, we are able to prove that the Fourier pseudo-spectral method is unconditionally convergent with order $O(τ^{2}+N^{α/2-r})$ in the discrete $L^{\infty}$ norm, where $τ$ is the time step and $N$ is the number of collocation points used in the spectral method. Numerical examples are presented to verify the theoretical analysis. △ Less

Submitted 21 October, 2019; v1 submitted 15 October, 2019; originally announced October 2019.

A Phase Shift Deep Neural Network for High Frequency Approximation and Wave Problems

Authors: Wei Cai, Xiaoguang Li, Lizuo Liu

Abstract: In this paper, we propose a phase shift deep neural network (PhaseDNN), which provides a uniform wideband convergence in approximating high frequency functions and solutions of wave equations. The PhaseDNN makes use of the fact that common DNNs often achieve convergence in the low frequency range first, and a series of moderately-sized DNNs are constructed and trained for selected high frequency r… ▽ More In this paper, we propose a phase shift deep neural network (PhaseDNN), which provides a uniform wideband convergence in approximating high frequency functions and solutions of wave equations. The PhaseDNN makes use of the fact that common DNNs often achieve convergence in the low frequency range first, and a series of moderately-sized DNNs are constructed and trained for selected high frequency ranges. With the help of phase shifts in the frequency domain, each of the DNNs will be trained to approximate the function's higher frequency content over a specific range at the the speed of convergence as in the low frequency range. As a result, the proposed PhaseDNN is able to convert high frequency learning to low frequency one, allowing a uniform learning to wideband functions. The PhaseDNN will then be applied to find the solution of high frequency wave equations in inhomogeneous media through both differential and integral equation formulations with least square residual loss functions. Numerical results have demonstrated the capability of the PhaseDNN in learning high frequency functions and oscillatory solutions of interior and exterior Helmholtz equations. △ Less

Submitted 13 December, 2019; v1 submitted 23 September, 2019; originally announced September 2019.

Comments: arXiv admin note: substantial text overlap with arXiv:1905.01389

Fast multipole method for 3-D Laplace equation in layered media

Authors: Bo Wang, Wen Zhong Zhang, Wei Cai

Abstract: In this paper, a fast multipole method (FMM) is proposed for 3-D Laplace equation in layered media. The potential due to charges embedded in layered media is decomposed into a free space component and four types of reaction field components, and the latter can be associated with the potential of a polarization source defined for each type. New multipole expansions (MEs) and local expansions (LEs),… ▽ More In this paper, a fast multipole method (FMM) is proposed for 3-D Laplace equation in layered media. The potential due to charges embedded in layered media is decomposed into a free space component and four types of reaction field components, and the latter can be associated with the potential of a polarization source defined for each type. New multipole expansions (MEs) and local expansions (LEs), as well as the multipole to local (M2L) translation operators are derived for the reaction components, based on which the FMMs for reaction components are then proposed. The resulting FMM for charge interactions in layered media is a combination of using the classic FMM for the free space components and the new FMMs for the reaction field components. With the help of a recurrence formula for the run-time computation of the Sommerfeld-type integrals used in M2L translation operators, pre-computations of a large number of tables are avoided. The new FMMs for the reaction components are found to be much faster than the classic FMM for the free space components due to the separation of equivalent polarization charges and the associated target charges by a material interface. As a result, the FMM for potential in layered media costs almost the same as the classic FMM in the free space case. Numerical results validate the fast convergence of the MEs for the reaction components, and the O(N) complexity of the FMM with a given truncation number p for charge interactions in 3-D layered media. △ Less

Submitted 22 May, 2020; v1 submitted 27 August, 2019; originally announced August 2019.

Comments: arXiv admin note: text overlap with arXiv:1902.05132

A linearly implicit energy-preserving exponential integrator for the nonlinear Klein-Gordon equation

Authors: Chaolong Jiang, Yushun Wang, Wenjun Cai

Abstract: In this paper, we generalize the exponential energy-preserving integrator proposed in the recent paper [SIAM J. Sci. Comput. 38(2016) A1876-A1895] for conservative systems, which now becomes linearly implicit by further utilizing the idea of the scalar auxiliary variable approach. Comparing with the original exponential energy-preserving integrator which usually leads to a nonlinear algebraic syst… ▽ More In this paper, we generalize the exponential energy-preserving integrator proposed in the recent paper [SIAM J. Sci. Comput. 38(2016) A1876-A1895] for conservative systems, which now becomes linearly implicit by further utilizing the idea of the scalar auxiliary variable approach. Comparing with the original exponential energy-preserving integrator which usually leads to a nonlinear algebraic system, our new method only involve a linear system with constant coefficient matrix. Taking the nonlinear Klein-Gordon equation for example, we derive the concrete energy-preserving scheme and demonstrate its high efficiency through numerical experiments. △ Less

Submitted 16 March, 2020; v1 submitted 27 August, 2019; originally announced August 2019.

Comments: 21 pages, 13 figures

Efficient Estimation of Pathwise Differentiable Target Parameters with the Undersmoothed Highly Adaptive Lasso

Authors: Mark J. van der Laan, David Benkeser, Weixin Cai

Abstract: We consider estimation of a functional parameter of a realistically modeled data distribution based on observing independent and identically distributed observations. We define an $m$-th order Spline Highly Adaptive Lasso Minimum Loss Estimator (Spline HAL-MLE) of a functional parameter that is defined by minimizing the empirical risk function over an $m$-th order smoothness class of functions. We… ▽ More We consider estimation of a functional parameter of a realistically modeled data distribution based on observing independent and identically distributed observations. We define an $m$-th order Spline Highly Adaptive Lasso Minimum Loss Estimator (Spline HAL-MLE) of a functional parameter that is defined by minimizing the empirical risk function over an $m$-th order smoothness class of functions. We show that this $m$-th order smoothness class consists of all functions that can be represented as an infinitesimal linear combination of tensor products of $\leq m$-th order spline-basis functions, and involves assuming $m$-derivatives in each coordinate. By selecting $m$ with cross-validation we obtain a Spline-HAL-MLE that is able to adapt to the underlying unknown smoothness of the true function, while guaranteeing a rate of convergence faster than $n^{-1/4}$, as long as the true function is cadlag (right-continuous with left-hand limits) and has finite sectional variation norm. The $m=0$-smoothness class consists of all cadlag functions with finite sectional variation norm and corresponds with the original HAL-MLE defined in van der Laan (2015). In this article we establish that this Spline-HAL-MLE yields an asymptotically efficient estimator of any smooth feature of the functional parameter under an easily verifiable global undersmoothing condition. A sufficient condition for the latter condition is that the minimum of the empirical mean of the selected basis functions is smaller than a constant times $n^{-1/2}$, which is not parameter specific and enforces the selection of the $L_1$-norm in the lasso to be large enough to include sparsely supported basis. We demonstrate our general result for the $m=0$-HAL-MLE of the average treatment effect and of the integral of the square of the data density. We also present simulations for these two examples confirming the theory. △ Less

Submitted 2 July, 2021; v1 submitted 14 August, 2019; originally announced August 2019.

A Path Integral Monte Carlo Method based on Feynman-Kac Formula for Electrical Impedance Tomography

Authors: Yi**g Zhou, Wei Cai

Abstract: A path integral Monte Carlo method (PIMC) based on Feynman-Kac formula for mixed boundary conditions of elliptic equations is proposed to solve the forward problem of electrical impedance tomography (EIT) on the boundary to obtain electrical potentials. The forward problem is an important part for iterative algorithms of the inverse problem of EIT, which has attracted continual interest due to its… ▽ More A path integral Monte Carlo method (PIMC) based on Feynman-Kac formula for mixed boundary conditions of elliptic equations is proposed to solve the forward problem of electrical impedance tomography (EIT) on the boundary to obtain electrical potentials. The forward problem is an important part for iterative algorithms of the inverse problem of EIT, which has attracted continual interest due to its applications in medical imaging and material testing of materials. By simulating reflecting Brownian motion with walk-on-sphere techniques and calculating its corresponding local time, we are able to obtain accurate voltage-to-current map for the conductivity equation with mixed boundary conditions for a 3-D spherical object with eight electrodes. Due to the local property of the PIMC method, the solution of the map can be done locally for each electrode in a parallel manner. △ Less

Submitted 29 July, 2019; originally announced July 2019.

Comments: arXiv admin note: text overlap with arXiv:1506.03385

A linearly implicit structure-preserving scheme for the Camassa-Holm equation based on multiple scalar auxiliary variables approach

Authors: Chaolong Jiang, Yuezheng Gong, Wenjun Cai, Yushun Wang

Abstract: In this paper, we present a linearly implicit energy-preserving scheme for the Camassa-Holm equation by using the multiple scalar auxiliary variables approach, which is first developed to construct efficient and robust energy stable schemes for gradient systems. The Camassa-Holm equation is first reformulated into an equivalent system by utilizing the multiple scalar auxiliary variables approach,… ▽ More In this paper, we present a linearly implicit energy-preserving scheme for the Camassa-Holm equation by using the multiple scalar auxiliary variables approach, which is first developed to construct efficient and robust energy stable schemes for gradient systems. The Camassa-Holm equation is first reformulated into an equivalent system by utilizing the multiple scalar auxiliary variables approach, which inherits a modified energy. Then, the system is discretized in space aided by the standard Fourier pseudo-spectral method and a semi-discrete system is obtained, which is proven to preserve a semi-discrete modified energy. Subsequently, the linearized Crank-Nicolson method is applied for the resulting semi-discrete system to arrive at a fully discrete scheme. The main feature of the new scheme is to form a linear system with a constant coefficient matrix at each time step and produce numerical solutions along which the modified energy is precisely conserved, as is the case with the analytical solution. Several numerical results are addressed to confirm accuracy and efficiency of the proposed scheme. △ Less

Submitted 16 March, 2020; v1 submitted 29 June, 2019; originally announced July 2019.

Comments: 21 pages, 13 figures

Nonparametric Bootstrap Inference for the Targeted Highly Adaptive LASSO Estimator

Authors: Weixin Cai, Mark van der Laan

Abstract: The Highly-Adaptive-LASSO Targeted Minimum Loss Estimator (HAL-TMLE) is an efficient plug-in estimator of a pathwise differentiable parameter in a statistical model that at minimal (and possibly only) assumes that the sectional variation norm of the true nuisance functional parameters (i.e., the relevant part of data distribution) are finite. It relies on an initial estimator (HAL-MLE) of the nuis… ▽ More The Highly-Adaptive-LASSO Targeted Minimum Loss Estimator (HAL-TMLE) is an efficient plug-in estimator of a pathwise differentiable parameter in a statistical model that at minimal (and possibly only) assumes that the sectional variation norm of the true nuisance functional parameters (i.e., the relevant part of data distribution) are finite. It relies on an initial estimator (HAL-MLE) of the nuisance functional parameters by minimizing the empirical risk over the parameter space under the constraint that the sectional variation norm of the candidate functions are bounded by a constant, where this constant can be selected with cross-validation. In this article, we establish that the nonparametric bootstrap for the HAL-TMLE, fixing the value of the sectional variation norm at a value larger or equal than the cross-validation selector, provides a consistent method for estimating the normal limit distribution of the HAL-TMLE. In order to optimize the finite sample coverage of the nonparametric bootstrap confidence intervals, we propose a selection method for this sectional variation norm that is based on running the nonparametric bootstrap for all values of the sectional variation norm larger than the one selected by cross-validation, and subsequently determining a value at which the width of the resulting confidence intervals reaches a plateau. We demonstrate our method for 1) nonparametric estimation of the average treatment effect based on observing on each unit a covariate vector, binary treatment, and outcome, and for 2) nonparametric estimation of the integral of the square of the multivariate density of the data distribution. In addition, we also present simulation results for these two examples demonstrating the excellent finite sample coverage of bootstrap-based confidence intervals. △ Less

Submitted 7 February, 2020; v1 submitted 23 May, 2019; originally announced May 2019.

Comments: arXiv admin note: substantial text overlap with arXiv:1708.09502

A linearized energy--conservative finite element method for the nonlinear Schrödinger equation with wave operator

Authors: Wentao Cai, Dongdong He, Kejia Pan

Abstract: In this paper, we propose a linearized finite element method (FEM) for solving the cubic nonlinear Schrödinger equation with wave operator. In this method, a modified leap-frog scheme is applied for time discretization and a Galerkin finite element method is applied for spatial discretization. We prove that the proposed method keeps the energy conservation in the given discrete norm. Comparing wit… ▽ More In this paper, we propose a linearized finite element method (FEM) for solving the cubic nonlinear Schrödinger equation with wave operator. In this method, a modified leap-frog scheme is applied for time discretization and a Galerkin finite element method is applied for spatial discretization. We prove that the proposed method keeps the energy conservation in the given discrete norm. Comparing with non-conservative schemes, our algorithm keeps higher stability. Meanwhile, an optimal error estimate for the proposed scheme is given by an error splitting technique. That is, we split the error into two parts, one from temporal discretization and the other from spatial discretization. First, by introducing a time-discrete system, we prove the uniform boundedness for the solution of this time-discrete system in some strong norms and obtain error estimates in temporal direction. With the help of the preliminary temporal estimates, we then prove the pointwise uniform boundedness of the finite element solution, and obtain the optimal $L^2$-norm error estimates in the sense that the time step size is not related to spatial mesh size. Finally, numerical examples are provided to validate the convergence-order, unconditional stability and energy conservation. △ Less

Submitted 22 February, 2019; originally announced February 2019.

Comments: 18 pages, 5 figures

Taylor expansion based fast Multipole Methods for 3-D Helmholtz equations in Layered Media

Authors: Bo Wanga, Duan Chen, Bo Zhang, Wenzhong Zhang, Min Hyung Cho, Wei Cai

Abstract: In this paper, we develop fast multipole methods for 3D Helmholtz kernel in layered media. Two algorithms based on different forms of Taylor expansion of layered media Green's function are developed. A key component of the first algorithm is an efficient algorithm based on discrete complex image approximation and recurrence formula for the calculation of the layered media Green's function and its… ▽ More In this paper, we develop fast multipole methods for 3D Helmholtz kernel in layered media. Two algorithms based on different forms of Taylor expansion of layered media Green's function are developed. A key component of the first algorithm is an efficient algorithm based on discrete complex image approximation and recurrence formula for the calculation of the layered media Green's function and its derivatives, which are given in terms of Sommerfeld integrals. The second algorithm uses symmetric derivatives in the Taylor expansion to reduce the size of precomputed tables for the derivatives of layered media Green's function. Numerical tests in layered media have validated the accuracy and O(N) complexity of the proposed algorithms. △ Less

Submitted 15 February, 2019; originally announced February 2019.