Showing 1–50 of 68 results for author: Dong, S

DCDILP: a distributed learning method for large-scale causal structure learning

Authors: Shuyu Dong, Michèle Sebag, Kento Uemura, Akito Fujii, Shuang Chang, Yusuke Koyanagi, Koji Maruhashi

Abstract: This paper presents a novel approach to causal discovery through a divide-and-conquer framework. By decomposing the problem into smaller subproblems defined on Markov blankets, the proposed DCDILP method first explores in parallel the local causal graphs of these subproblems. However, this local discovery phase encounters systematic challenges due to the presence of hidden confounders (variables w… ▽ More This paper presents a novel approach to causal discovery through a divide-and-conquer framework. By decomposing the problem into smaller subproblems defined on Markov blankets, the proposed DCDILP method first explores in parallel the local causal graphs of these subproblems. However, this local discovery phase encounters systematic challenges due to the presence of hidden confounders (variables within each Markov blanket may be influenced by external variables). Moreover, aggregating these local causal graphs in a consistent global graph defines a large size combinatorial optimization problem. DCDILP addresses these challenges by: i) restricting the local subgraphs to causal links only related with the central variable of the Markov blanket; ii) formulating the reconciliation of local causal graphs as an integer linear programming method. The merits of the approach, in both terms of causal discovery accuracy and scalability in the size of the problem, are showcased by experiments and comparisons with the state of the art. △ Less

Submitted 14 June, 2024; originally announced June 2024.

Error Analysis and Numerical Algorithm for PDE Approximation with Hidden-Layer Concatenated Physics Informed Neural Networks

Authors: Yianxia Qian, Yongchao Zhang, Suchuan Dong

Abstract: We present the hidden-layer concatenated physics informed neural network (HLConcPINN) method, which combines hidden-layer concatenated feed-forward neural networks, a modified block time marching strategy, and a physics informed approach for approximating partial differential equations (PDEs). We analyze the convergence properties and establish the error bounds of this method for two types of PDEs… ▽ More We present the hidden-layer concatenated physics informed neural network (HLConcPINN) method, which combines hidden-layer concatenated feed-forward neural networks, a modified block time marching strategy, and a physics informed approach for approximating partial differential equations (PDEs). We analyze the convergence properties and establish the error bounds of this method for two types of PDEs: parabolic (exemplified by the heat and Burgers' equations) and hyperbolic (exemplified by the wave and nonlinear Klein-Gordon equations). We show that its approximation error of the solution can be effectively controlled by the training loss for dynamic simulations with long time horizons. The HLConcPINN method in principle allows an arbitrary number of hidden layers not smaller than two and any of the commonly-used smooth activation functions for the hidden layers beyond the first two, with theoretical guarantees. This generalizes several recent neural-network techniques, which have theoretical guarantees but are confined to two hidden layers in the network architecture and the $\tanh$ activation function. Our theoretical analyses subsequently inform the formulation of appropriate training loss functions for these PDEs, leading to physics informed neural network (PINN) type computational algorithms that differ from the standard PINN formulation. Ample numerical experiments are presented based on the proposed algorithm to validate the effectiveness of this method and confirm aspects of the theoretical analyses. △ Less

Submitted 10 June, 2024; originally announced June 2024.

Comments: 40 pages, 10 tables, 18 figures

Large data global existence for coupled massive-massless wave-type systems

Authors: Yuan Cai, Shijie Dong, Kuijie Li, **gya Zhao

Abstract: We consider 3D Klein-Gordon-Zakharov (KGZ) and Dirac-Klein-Gordon (DKG) systems, where a common feature is that there exist both massless and massive fields in each system. We establish global existence and asymptotic behavior for both systems with a class of large data. More precisely, in the KGZ system, we allow the massless field to be large, while in the DKG system we allow the massive field t… ▽ More We consider 3D Klein-Gordon-Zakharov (KGZ) and Dirac-Klein-Gordon (DKG) systems, where a common feature is that there exist both massless and massive fields in each system. We establish global existence and asymptotic behavior for both systems with a class of large data. More precisely, in the KGZ system, we allow the massless field to be large, while in the DKG system we allow the massive field to be large. △ Less

Submitted 10 June, 2024; v1 submitted 9 June, 2024; originally announced June 2024.

Comments: All comments are welcome. 58 pages

Fibonacci and Lucas Sequences in Aperiodic Monotile Supertiles

Authors: Shiying Dong

Abstract: This paper first discusses the size and orientation of hat supertiles. Fibonacci and Lucas sequences, as well as a third integer sequence linearly related to the Lucas sequence are involved. The result is then generalized to any aperiodic tile in the hat family. This paper first discusses the size and orientation of hat supertiles. Fibonacci and Lucas sequences, as well as a third integer sequence linearly related to the Lucas sequence are involved. The result is then generalized to any aperiodic tile in the hat family. △ Less

Submitted 30 April, 2024; originally announced April 2024.

Comments: 10 pages, 21 figures

MSC Class: 05B45 ACM Class: G.2.1

A Functionally Connected Element Method for Solving Boundary Value Problems

Authors: Jielin Yang, Suchuan Dong

Abstract: We present the general forms of piece-wise functions on partitioned domains satisfying an intrinsic $C^0$ or $C^1$ continuity across the sub-domain boundaries. These general forms are constructed based on a strategy stemming from the theory of functional connections, and we refer to partitioned domains endowed with these general forms as functionally connected elements (FCE). We further present a… ▽ More We present the general forms of piece-wise functions on partitioned domains satisfying an intrinsic $C^0$ or $C^1$ continuity across the sub-domain boundaries. These general forms are constructed based on a strategy stemming from the theory of functional connections, and we refer to partitioned domains endowed with these general forms as functionally connected elements (FCE). We further present a method, incorporating functionally connected elements and a least squares collocation approach, for solving boundary and initial value problems. This method exhibits a spectral-like accuracy, with the free functions involved in the FCE form represented by polynomial bases or by non-polynomial bases of quasi-random sinusoidal functions. The FCE method offers a unique advantage over traditional element-based methods for boundary value problems involving relative boundary conditions. A number of linear and nonlinear numerical examples in one and two dimensions are presented to demonstrate the performance of the FCE method developed herein. △ Less

Submitted 10 March, 2024; originally announced March 2024.

Comments: 44 pages, 10 figures, 8 tables

Global smooth solutions to 2D semilinear wave equations with large data

Authors: Bingbing Ding, Shijie Dong, Gang Xu

Abstract: We are interested in coupled semi-linear wave equations satisfying the null condition in two space dimensions, a basic model in nonlinear wave equations. Our aim is to establish global existence of smooth solutions to this system with large initial data of short pulse type. Major difficulties arise due to the largeness of initial data and the slow decay nature of 2D wave equations. To overcome the… ▽ More We are interested in coupled semi-linear wave equations satisfying the null condition in two space dimensions, a basic model in nonlinear wave equations. Our aim is to establish global existence of smooth solutions to this system with large initial data of short pulse type. Major difficulties arise due to the largeness of initial data and the slow decay nature of 2D wave equations. To overcome the difficulties, by careful examination of the local solutions, we adapt various vector-field methods to different spacetime regions with several novel weighted energy estimates. △ Less

Submitted 13 December, 2023; originally announced December 2023.

Cubic Dirac equations with a class of large data

Authors: Shijie Dong, Kuijie Li, **gya Zhao

Abstract: We are interested in massless cubic Dirac equations in two and three space dimensions, known as the Soler model. The solution to this model is known as a wave function, which has the unit $L^2$ norm. We aim to show global existence and asymptotic behavior for the cubic Dirac model with a class of initial data that can be large in $L^2$. We are interested in massless cubic Dirac equations in two and three space dimensions, known as the Soler model. The solution to this model is known as a wave function, which has the unit $L^2$ norm. We aim to show global existence and asymptotic behavior for the cubic Dirac model with a class of initial data that can be large in $L^2$. △ Less

Submitted 26 December, 2023; v1 submitted 12 December, 2023; originally announced December 2023.

Comments: 30 pages, comments are welcome

An Extreme Learning Machine-Based Method for Computational PDEs in Higher Dimensions

Authors: Yiran Wang, Suchuan Dong

Abstract: We present two effective methods for solving high-dimensional partial differential equations (PDE) based on randomized neural networks. Motivated by the universal approximation property of this type of networks, both methods extend the extreme learning machine (ELM) approach from low to high dimensions. With the first method the unknown solution field in $d$ dimensions is represented by a randomiz… ▽ More We present two effective methods for solving high-dimensional partial differential equations (PDE) based on randomized neural networks. Motivated by the universal approximation property of this type of networks, both methods extend the extreme learning machine (ELM) approach from low to high dimensions. With the first method the unknown solution field in $d$ dimensions is represented by a randomized feed-forward neural network, in which the hidden-layer parameters are randomly assigned and fixed while the output-layer parameters are trained. The PDE and the boundary/initial conditions, as well as the continuity conditions (for the local variant of the method), are enforced on a set of random interior/boundary collocation points. The resultant linear or nonlinear algebraic system, through its least squares solution, provides the trained values for the network parameters. With the second method the high-dimensional PDE problem is reformulated through a constrained expression based on an Approximate variant of the Theory of Functional Connections (A-TFC), which avoids the exponential growth in the number of terms of TFC as the dimension increases. The free field function in the A-TFC constrained expression is represented by a randomized neural network and is trained by a procedure analogous to the first method. We present ample numerical simulations for a number of high-dimensional linear/nonlinear stationary/dynamic PDEs to demonstrate their performance. These methods can produce accurate solutions to high-dimensional PDEs, in particular with their errors reaching levels not far from the machine accuracy for relatively lower dimensions. Compared with the physics-informed neural network (PINN) method, the current method is both cost-effective and more accurate for high-dimensional PDEs. △ Less

Submitted 13 September, 2023; originally announced September 2023.

Comments: 38 pages, 17 tables, 25 figures

Polytopes with Bounded Integral Slack Matrices Have Sub-Exponential Extension Complexity

Authors: Sally Dong, Thomas Rothvoss

Abstract: We show that any bounded integral function $f : A \times B \mapsto \{0,1, \dots, Δ\}$ with rank $r$ has deterministic communication complexity $Δ^{O(Δ)} \cdot \sqrt{r} \cdot \log r$, where the rank of $f$ is defined to be the rank of the $A \times B$ matrix whose entries are the function values. As a corollary, we show that any $n$-dimensional polytope that admits a slack matrix with entries from… ▽ More We show that any bounded integral function $f : A \times B \mapsto \{0,1, \dots, Δ\}$ with rank $r$ has deterministic communication complexity $Δ^{O(Δ)} \cdot \sqrt{r} \cdot \log r$, where the rank of $f$ is defined to be the rank of the $A \times B$ matrix whose entries are the function values. As a corollary, we show that any $n$-dimensional polytope that admits a slack matrix with entries from $\{0,1,\dots,Δ\}$ has extension complexity at most $\exp(Δ^{O(Δ)} \cdot \sqrt{n} \cdot \log n)$. △ Less

Submitted 20 March, 2024; v1 submitted 30 July, 2023; originally announced July 2023.

Comments: 9 pages, to appear at IPCO 2024

Phase Field Modeling and Numerical Algorithm for Two-Phase Dielectric Fluid Flows

Authors: Jielin Yang, Ivan C. Christov, Suchuan Dong

Abstract: We develop a method for modeling and simulating a class of two-phase flows consisting of two immiscible incompressible dielectric fluids and their interactions with imposed external electric fields in two and three dimensions. We first present a thermodynamically-consistent and reduction-consistent phase field model for two-phase dielectric fluids. The model honors the conservation laws and thermo… ▽ More We develop a method for modeling and simulating a class of two-phase flows consisting of two immiscible incompressible dielectric fluids and their interactions with imposed external electric fields in two and three dimensions. We first present a thermodynamically-consistent and reduction-consistent phase field model for two-phase dielectric fluids. The model honors the conservation laws and thermodynamic principles, and has the property that, if only one fluid component is present in the system, the two-phase formulation will exactly reduce to that of the corresponding single-phase system. In particular, this model accommodates an equilibrium solution that is compatible with the zero-velocity requirement based on physics. This property provides a simpler method for simulating the equilibrium state of two-phase dielectric systems. We further present an efficient numerical algorithm, together with a spectral-element (for two dimensions) or a hybrid Fourier-spectral/spectral-element (for three dimensions) discretization in space, for simulating this class of problems. This algorithm computes different dynamic variables successively in an un-coupled fashion, and involves only coefficient matrices that are time-independent in the resultant linear algebraic systems upon discretization, even when the physical properties (e.g. permittivity, density, viscosity) of the two dielectric fluids are different. This property is crucial and enables us to employ fast Fourier transforms for three-dimensional problems. Ample numerical simulations of two-phase dielectric flows under imposed voltage are presented to demonstrate the performance of the method herein and to compare the simulation results with theoretical models and experimental data. △ Less

Submitted 15 May, 2023; originally announced May 2023.

Comments: 38 pages, 19 figures, 2 tables

Journal ref: Journal of Computational Physics 514 (2024) 113228

Error Analysis of Physics-Informed Neural Networks for Approximating Dynamic PDEs of Second Order in Time

Authors: Yanxia Qian, Yongchao Zhang, Yunqing Huang, Suchuan Dong

Abstract: We consider the approximation of a class of dynamic partial differential equations (PDE) of second order in time by the physics-informed neural network (PINN) approach, and provide an error analysis of PINN for the wave equation, the Sine-Gordon equation and the linear elastodynamic equation. Our analyses show that, with feed-forward neural networks having two hidden layers and the $\tanh$ activat… ▽ More We consider the approximation of a class of dynamic partial differential equations (PDE) of second order in time by the physics-informed neural network (PINN) approach, and provide an error analysis of PINN for the wave equation, the Sine-Gordon equation and the linear elastodynamic equation. Our analyses show that, with feed-forward neural networks having two hidden layers and the $\tanh$ activation function, the PINN approximation errors for the solution field, its time derivative and its gradient field can be effectively bounded by the training loss and the number of training data points (quadrature points). Our analyses further suggest new forms for the training loss function, which contain certain residuals that are crucial to the error estimate but would be absent from the canonical PINN loss formulation. Adopting these new forms for the loss function leads to a variant PINN algorithm. We present ample numerical experiments with the new PINN algorithm for the wave equation, the Sine-Gordon equation and the linear elastodynamic equation, which show that the method can capture the solution well. △ Less

Submitted 21 March, 2023; originally announced March 2023.

Comments: 46 pages, 14 figures, 3 tables

Generically sharp decay for quasilinear wave equations with null condition

Authors: Shijie Dong, Siyuan Ma, Yue Ma, Xu Yuan

Abstract: We are interested in the three-dimensional quasilinear wave equations with null condition. Global existence and pointwise decay for this model have been proved in the celebrated works of Klainerman \cite{Klainerman86} and Christodoulou \cite{Christodoulou86} for small smooth initial data. In this work, we illustrate the precise pointwise asymptotic behavior of the solutions for initial data posed… ▽ More We are interested in the three-dimensional quasilinear wave equations with null condition. Global existence and pointwise decay for this model have been proved in the celebrated works of Klainerman \cite{Klainerman86} and Christodoulou \cite{Christodoulou86} for small smooth initial data. In this work, we illustrate the precise pointwise asymptotic behavior of the solutions for initial data posed on a hyperboloid and show that the decay rate $v^{-1}u^{-1}$ is optimal for a generic set of initial data. △ Less

Submitted 22 December, 2022; originally announced December 2022.

Comments: 40 pages, all comments are welcome

Learning Large Causal Structures from Inverse Covariance Matrix via Sparse Matrix Decomposition

Authors: Shuyu Dong, Kento Uemura, Akito Fujii, Shuang Chang, Yusuke Koyanagi, Koji Maruhashi, Michèle Sebag

Abstract: Learning causal structures from observational data is a fundamental problem facing important computational challenges when the number of variables is large. In the context of linear structural equation models (SEMs), this paper focuses on learning causal structures from the inverse covariance matrix. The proposed method, called ICID for Independence-preserving Decomposition from Inverse Covariance… ▽ More Learning causal structures from observational data is a fundamental problem facing important computational challenges when the number of variables is large. In the context of linear structural equation models (SEMs), this paper focuses on learning causal structures from the inverse covariance matrix. The proposed method, called ICID for Independence-preserving Decomposition from Inverse Covariance matrix, is based on continuous optimization of a matrix decomposition model that preserves the nonzero patterns of the inverse covariance matrix. Through theoretical and empirical evidences, we show that ICID efficiently identifies the sought directed acyclic graph (DAG) assuming the knowledge of noise variances. Moreover, ICID is shown empirically to be robust under bounded misspecification of noise variances in the case where the noise variances are non-equal. The proposed method enjoys a low complexity, as reflected by its time efficiency in the experiments, and also enables a novel regularization scheme that yields highly accurate solutions on the Simulated fMRI data (Smith et al., 2011) in comparison with state-of-the-art algorithms. △ Less

Submitted 19 February, 2024; v1 submitted 25 November, 2022; originally announced November 2022.

A Method for Computing Inverse Parametric PDE Problems with Random-Weight Neural Networks

Authors: Suchuan Dong, Yiran Wang

Abstract: We present a method for computing the inverse parameters and the solution field to inverse parametric PDEs based on randomized neural networks. This extends the local extreme learning machine technique originally developed for forward PDEs to inverse problems. We develop three algorithms for training the neural network to solve the inverse PDE problem. The first algorithm (NLLSQ) determines the in… ▽ More We present a method for computing the inverse parameters and the solution field to inverse parametric PDEs based on randomized neural networks. This extends the local extreme learning machine technique originally developed for forward PDEs to inverse problems. We develop three algorithms for training the neural network to solve the inverse PDE problem. The first algorithm (NLLSQ) determines the inverse parameters and the trainable network parameters all together by the nonlinear least squares method with perturbations (NLLSQ-perturb). The second algorithm (VarPro-F1) eliminates the inverse parameters from the overall problem by variable projection to attain a reduced problem about the trainable network parameters only. It solves the reduced problem first by the NLLSQ-perturb algorithm for the trainable network parameters, and then computes the inverse parameters by the linear least squares method. The third algorithm (VarPro-F2) eliminates the trainable network parameters from the overall problem by variable projection to attain a reduced problem about the inverse parameters only. It solves the reduced problem for the inverse parameters first, and then computes the trainable network parameters afterwards. VarPro-F1 and VarPro-F2 are reciprocal to each other in a sense. The presented method produces accurate results for inverse PDE problems, as shown by the numerical examples herein. For noise-free data, the errors for the inverse parameters and the solution field decrease exponentially as the number of collocation points or the number of trainable network parameters increases, and can reach a level close to the machine accuracy. For noisy data, the accuracy degrades compared with the case of noise-free data, but the method remains quite accurate. The presented method has been compared with the physics-informed neural network method. △ Less

Submitted 9 October, 2022; originally announced October 2022.

Comments: 40 pages, 8 figures, 34 tables

Generalization to the Natural Gradient Descent

Authors: Shaojun Dong, Fengyu Le, Meng Zhang, Si-**g Tao, Chao Wang, Yong-Jian Han, Guo-** Guo

Abstract: Optimization problem, which is aimed at finding the global minimal value of a given cost function, is one of the central problem in science and engineering. Various numerical methods have been proposed to solve this problem, among which the Gradient Descent (GD) method is the most popular one due to its simplicity and efficiency. However, the GD method suffers from two main issues: the local minim… ▽ More Optimization problem, which is aimed at finding the global minimal value of a given cost function, is one of the central problem in science and engineering. Various numerical methods have been proposed to solve this problem, among which the Gradient Descent (GD) method is the most popular one due to its simplicity and efficiency. However, the GD method suffers from two main issues: the local minima and the slow convergence especially near the minima point. The Natural Gradient Descent(NGD), which has been proved as one of the most powerful method for various optimization problems in machine learning, tensor network, variational quantum algorithms and so on, supplies an efficient way to accelerate the convergence. Here, we give a unified method to extend the NGD method to a more general situation which keeps the fast convergence by looking for a more suitable metric through introducing a 'proper' reference Riemannian manifold. Our method generalizes the NDG, and may give more insight of the optimization methods. △ Less

Submitted 6 October, 2022; originally announced October 2022.

Coupled iterative analysis for the stationary thermally coupled inductionless MHD system based on charge-conservative finite element method

Authors: Shitian Dong, Haiyan Su

Abstract: This paper mainly considers three iterations based on charge-conservative finite element approximation in Lipschitz domain for the stationary thermally coupled inductionless MHD equations. Based on the hybrid finite element method, the unknowns of hydrodynamic are discretized by the stable velocity-pressure finite element pair, and the current density along with electric potential are similarly di… ▽ More This paper mainly considers three iterations based on charge-conservative finite element approximation in Lipschitz domain for the stationary thermally coupled inductionless MHD equations. Based on the hybrid finite element method, the unknowns of hydrodynamic are discretized by the stable velocity-pressure finite element pair, and the current density along with electric potential are similarly discretized by the comforming finite element pair in $\boldsymbol{H}(\boldsymbol{div}, Ω)\times L^2(Ω)$. And on account of the strong nonlinearity of the equations, we present three coupled iterative methods, namely, the Stokes, Newton and Oseen iteration and the convergence and stability under different uniqueness conditions are analyzed strictly. It is proved especially that the error estimates of velocity, current density, temperature and pressure do not depend on potential. The theoretical analysis is validated by the given numerical results, and for the proposed methods, the applicability and effectiveness are demonstrated. △ Less

Submitted 24 September, 2022; originally announced September 2022.

Global solution to the 3D Dirac--Klein-Gordon system with uniform energy bounds

Authors: Shijie Dong, Kuijie Li, Xu Yuan

Abstract: On the (1+3) dimensional Minkowski spacetime, for small, regular initial data, it is well-known that the Dirac-Klein-Gordon system admits a global solution. In the present paper, we aim to establish the uniform boundedness of the total energy of the solution for this system. The proof relies on Klainerman's vector field and Alinhac's ghost weight methods. The main difficulty originates from the… ▽ More On the (1+3) dimensional Minkowski spacetime, for small, regular initial data, it is well-known that the Dirac-Klein-Gordon system admits a global solution. In the present paper, we aim to establish the uniform boundedness of the total energy of the solution for this system. The proof relies on Klainerman's vector field and Alinhac's ghost weight methods. The main difficulty originates from the slow decay nature of the Dirac and wave components in three space dimensions. To overcome the difficulty, a sharp understanding of the structure for this system, and a new weighted conformal energy estimate are required. In addition, we also provide a few scattering results for the system. △ Less

Submitted 30 August, 2022; originally announced August 2022.

Comments: 36 pages, four sections. All suggestions and comments are welcome

Decomposable Non-Smooth Convex Optimization with Nearly-Linear Gradient Oracle Complexity

Authors: Sally Dong, Haotian Jiang, Yin Tat Lee, Swati Padmanabhan, Guanghao Ye

Abstract: Many fundamental problems in machine learning can be formulated by the convex program \[ \min_{θ\in R^d}\ \sum_{i=1}^{n}f_{i}(θ), \] where each $f_i$ is a convex, Lipschitz function supported on a subset of $d_i$ coordinates of $θ$. One common approach to this problem, exemplified by stochastic gradient descent, involves sampling one $f_i$ term at every iteration to make progress. This approach cr… ▽ More Many fundamental problems in machine learning can be formulated by the convex program \[ \min_{θ\in R^d}\ \sum_{i=1}^{n}f_{i}(θ), \] where each $f_i$ is a convex, Lipschitz function supported on a subset of $d_i$ coordinates of $θ$. One common approach to this problem, exemplified by stochastic gradient descent, involves sampling one $f_i$ term at every iteration to make progress. This approach crucially relies on a notion of uniformity across the $f_i$'s, formally captured by their condition number. In this work, we give an algorithm that minimizes the above convex formulation to $ε$-accuracy in $\widetilde{O}(\sum_{i=1}^n d_i \log (1 /ε))$ gradient computations, with no assumptions on the condition number. The previous best algorithm independent of the condition number is the standard cutting plane method, which requires $O(nd \log (1/ε))$ gradient computations. As a corollary, we improve upon the evaluation oracle complexity for decomposable submodular minimization by Axiotis et al. (ICML 2021). Our main technical contribution is an adaptive procedure to select an $f_i$ term at every iteration via a novel combination of cutting-plane and interior-point methods. △ Less

Submitted 7 August, 2022; originally announced August 2022.

Local Randomized Neural Networks with Discontinuous Galerkin Methods for Partial Differential Equations

Authors: **gbo Sun, Suchuan Dong, Fei Wang

Abstract: Randomized neural networks (RNN) are a variation of neural networks in which the hidden-layer parameters are fixed to randomly assigned values and the output-layer parameters are obtained by solving a linear system by least squares. This improves the efficiency without degrading the accuracy of the neural network. In this paper, we combine the idea of the local RNN (LRNN) and the discontinuous Gal… ▽ More Randomized neural networks (RNN) are a variation of neural networks in which the hidden-layer parameters are fixed to randomly assigned values and the output-layer parameters are obtained by solving a linear system by least squares. This improves the efficiency without degrading the accuracy of the neural network. In this paper, we combine the idea of the local RNN (LRNN) and the discontinuous Galerkin (DG) approach for solving partial differential equations. RNNs are used to approximate the solution on the subdomains, and the DG formulation is used to glue them together. Taking the Poisson problem as a model, we propose three numerical schemes and provide the convergence analyses. Then we extend the ideas to time-dependent problems. Taking the heat equation as a model, three space-time LRNN with DG formulations are proposed. Finally, we present numerical tests to demonstrate the performance of the methods developed herein. We compare the proposed methods with the finite element method and the usual DG method. The LRNN-DG methods can achieve better accuracy under the same degrees of freedom, signifying that this new approach has a great potential for solving partial differential equations. △ Less

Submitted 11 June, 2022; originally announced June 2022.

Global Behavior of Small Data Solutions for The 2D Dirac-Klein-Gordon Equations

Authors: Shijie Dong, Kuijie Li, Yue Ma, Xu Yuan

Abstract: In this paper, we are interested in the two-dimensional Dirac-Klein-Gordon system, which is a basic model in particle physics. We investigate the global behaviors of small data solutions to this system in the case of a massive scalar field and a massless Dirac field. More precisely, our main result is twofold: 1) we show sharp time decay for the pointwise estimates of the solutions which imply the… ▽ More In this paper, we are interested in the two-dimensional Dirac-Klein-Gordon system, which is a basic model in particle physics. We investigate the global behaviors of small data solutions to this system in the case of a massive scalar field and a massless Dirac field. More precisely, our main result is twofold: 1) we show sharp time decay for the pointwise estimates of the solutions which imply the asymptotic stability of this system; 2) we show the linear scattering result of this system which is a fundamental problem when it is viewed as dispersive equations. Our result is valid for general small, high-regular initial data, in particular, there is no restriction on the support of the initial data. △ Less

Submitted 24 May, 2022; originally announced May 2022.

Comments: 42 Pages, 3 sections. All comments and suggestions are warmly welcome

RAR-PINN algorithm for the data-driven vector-soliton solutions and parameter discovery of coupled nonlinear equations

Authors: Shu-Mei Qin, Min Li, Tao Xu, Shao-Qun Dong

Abstract: This work aims to provide an effective deep learning framework to predict the vector-soliton solutions of the coupled nonlinear equations and their interactions. The method we propose here is a physics-informed neural network (PINN) combining with the residual-based adaptive refinement (RAR-PINN) algorithm. Different from the traditional PINN algorithm which takes points randomly, the RAR-PINN alg… ▽ More This work aims to provide an effective deep learning framework to predict the vector-soliton solutions of the coupled nonlinear equations and their interactions. The method we propose here is a physics-informed neural network (PINN) combining with the residual-based adaptive refinement (RAR-PINN) algorithm. Different from the traditional PINN algorithm which takes points randomly, the RAR-PINN algorithm uses an adaptive point-fetching approach to improve the training efficiency for the solutions with steep gradients. A series of experiment comparisons between the RAR-PINN and traditional PINN algorithms are implemented to a coupled generalized nonlinear Schrödinger (CGNLS) equation as an example. The results indicate that the RAR-PINN algorithm has faster convergence rate and better approximation ability, especially in modeling the shape-changing vector-soliton interactions in the coupled systems. Finally, the RAR-PINN method is applied to perform the data-driven discovery of the CGNLS equation, which shows the dispersion and nonlinear coefficients can be well approximated. △ Less

Submitted 29 April, 2022; originally announced May 2022.

Numerical Computation of Partial Differential Equations by Hidden-Layer Concatenated Extreme Learning Machine

Authors: Naxian Ni, Suchuan Dong

Abstract: The extreme learning machine (ELM) method can yield highly accurate solutions to linear/nonlinear partial differential equations (PDEs), but requires the last hidden layer of the neural network to be wide to achieve a high accuracy. If the last hidden layer is narrow, the accuracy of the existing ELM method will be poor, irrespective of the rest of the network configuration. In this paper we prese… ▽ More The extreme learning machine (ELM) method can yield highly accurate solutions to linear/nonlinear partial differential equations (PDEs), but requires the last hidden layer of the neural network to be wide to achieve a high accuracy. If the last hidden layer is narrow, the accuracy of the existing ELM method will be poor, irrespective of the rest of the network configuration. In this paper we present a modified ELM method, termed HLConcELM (hidden-layer concatenated ELM), to overcome the above drawback of the conventional ELM method. The HLConcELM method can produce highly accurate solutions to linear/nonlinear PDEs when the last hidden layer of the network is narrow and when it is wide. The new method is based on a type of modified feedforward neural networks (FNN), termed HLConcFNN (hidden-layer concatenated FNN), which incorporates a logical concatenation of the hidden layers in the network and exposes all the hidden nodes to the output-layer nodes. HLConcFNNs have the interesting property that, given a network architecture, when additional hidden layers are appended to the network or when extra nodes are added to the existing hidden layers the representation capacity of the HLConcFNN associated with the new architecture is guaranteed to be not smaller than that of the original network architecture. Here representation capacity refers to the set of all functions that can be exactly represented by the neural network of a given architecture. We present ample benchmark tests with linear/nonlinear PDEs to demonstrate the computational accuracy and performance of the HLConcELM method and the superiority of this method to the conventional ELM from previous works. △ Less

Submitted 15 May, 2022; v1 submitted 24 April, 2022; originally announced April 2022.

Comments: 36 pages, 24 figures, 5 tables

A Multi-stage Stochastic Programming Approach for Pre-positioning of Relief Supplies

Authors: Oluwasegun Olanrewaju, Shaolong Hu, Zhijie Sasha Dong

Abstract: Pre-positioning of relief supplies is an important aspect of disaster operations management that aims at decreasing the response time by advancing procurement and storage of needed supplies. In this paper we consider commodity life-time period with the related costs in kee** the commodity and removing it from the storage when it is close to expiration (i.e. holding and removal cost). We develop… ▽ More Pre-positioning of relief supplies is an important aspect of disaster operations management that aims at decreasing the response time by advancing procurement and storage of needed supplies. In this paper we consider commodity life-time period with the related costs in kee** the commodity and removing it from the storage when it is close to expiration (i.e. holding and removal cost). We develop a multi-stage stochastic programming model for pre-positioning of relief supplies, which provides relief agencies insight on how to have dynamically control inventories due to uncertain demands when disasters (e.g., hurricane or earthquake) occur. We also present a case study based on mainland United States to illustrate the proposed model as well as provide managerial insights to relief agencies. △ Less

Submitted 14 April, 2022; originally announced April 2022.

Comments: 6 pages, 4 figures

From graphs to DAGs: a low-complexity model and a scalable algorithm

Authors: Shuyu Dong, Michèle Sebag

Abstract: Learning directed acyclic graphs (DAGs) is long known a critical challenge at the core of probabilistic and causal modeling. The NoTears approach of (Zheng et al., 2018), through a differentiable function involving the matrix exponential trace $\mathrm{tr}(\exp(\cdot))$, opens up a way to learning DAGs via continuous optimization, though with a $O(d^3)$ complexity in the number $d$ of nodes. This… ▽ More Learning directed acyclic graphs (DAGs) is long known a critical challenge at the core of probabilistic and causal modeling. The NoTears approach of (Zheng et al., 2018), through a differentiable function involving the matrix exponential trace $\mathrm{tr}(\exp(\cdot))$, opens up a way to learning DAGs via continuous optimization, though with a $O(d^3)$ complexity in the number $d$ of nodes. This paper presents a low-complexity model, called LoRAM for Low-Rank Additive Model, which combines low-rank matrix factorization with a sparsification mechanism for the continuous optimization of DAGs. The main contribution of the approach lies in an efficient gradient approximation method leveraging the low-rank property of the model, and its straightforward application to the computation of projections from graph matrices onto the DAG matrix space. The proposed method achieves a reduction from a cubic complexity to quadratic complexity while handling the same DAG characteristic function as NoTears, and scales easily up to thousands of nodes for the projection problem. The experiments show that the LoRAM achieves efficiency gains of orders of magnitude compared to the state-of-the-art at the expense of a very moderate accuracy loss in the considered range of sparse matrices, and with a low sensitivity to the rank choice of the model's low-rank component. △ Less

Submitted 10 April, 2022; originally announced April 2022.

On the analysis of optimization with fixed-rank matrices: a quotient geometric view

Authors: Shuyu Dong, Bin Gao, Wen Huang, Kyle A. Gallivan

Abstract: We study a type of Riemannian gradient descent (RGD) algorithm, designed through Riemannian preconditioning, for optimization on $\mathcal{M}_k^{m\times n}$ -- the set of $m\times n$ real matrices with a fixed rank $k$. Our analysis is based on a quotient geometric view of $\mathcal{M}_k^{m\times n}$: by identifying this set with the quotient manifold of a two-term product space… ▽ More We study a type of Riemannian gradient descent (RGD) algorithm, designed through Riemannian preconditioning, for optimization on $\mathcal{M}_k^{m\times n}$ -- the set of $m\times n$ real matrices with a fixed rank $k$. Our analysis is based on a quotient geometric view of $\mathcal{M}_k^{m\times n}$: by identifying this set with the quotient manifold of a two-term product space $\mathbb{R}_*^{m\times k}\times \mathbb{R}_*^{n\times k}$ of matrices with full column rank via matrix factorization, we find an explicit form for the update rule of the RGD algorithm, which leads to a novel approach to analysing their convergence behavior in rank-constrained optimization. We then deduce some interesting properties that reflect how RGD distinguishes from other matrix factorization algorithms such as those based on the Euclidean geometry. In particular, we show that the RGD algorithm are not only faster than Euclidean gradient descent but also do not rely on the balancing technique to ensure its efficiency while the latter does. Starting from the novel results, we further show that this RGD algorithm is guaranteed to solve matrix sensing and matrix completion problems with linear convergence rate, under mild conditions related to the restricted positive definiteness property. Numerical experiments on matrix sensing and completion are provided to demonstrate these properties. △ Less

Submitted 13 March, 2022; originally announced March 2022.

MSC Class: 53B21; 41A29; 90C26; 65K05; 68Q25; 15A29; 15A83

Asymptotic behavior of 2D Wave-Klein-Gordon coupled system under null condition

Authors: Shijie Dong, Yue Ma, Xu Yuan

Abstract: We study the 2D coupled wave-Klein-Gordon systems with semi-linear null nonlinearities $Q_0$ and $Q_{αβ}$. The main result states that the solution to the 2D coupled systems exists globally provided that the initial data are small in some weighted Sobolev space, which do not necessarily have compact support, and we also show the optimal time decay of the solution. The major difficulties lie in t… ▽ More We study the 2D coupled wave-Klein-Gordon systems with semi-linear null nonlinearities $Q_0$ and $Q_{αβ}$. The main result states that the solution to the 2D coupled systems exists globally provided that the initial data are small in some weighted Sobolev space, which do not necessarily have compact support, and we also show the optimal time decay of the solution. The major difficulties lie in the slow decay nature of the wave and the Klein-Gordon components in two space dimensions, in addition, extra difficulties arise due to the presence of the null form $Q_0$ which is not of divergence form and is not compatible with the Klein-Gordon equations. To overcome the difficulties, a new observation for the structure of the null form $Q_0$ is required. △ Less

Submitted 16 February, 2022; originally announced February 2022.

Comments: 28 pages, three sections. All suggestions and comments are welcome

Numerical Approximation of Partial Differential Equations by a Variable Projection Method with Artificial Neural Networks

Authors: Suchuan Dong, Jielin Yang

Abstract: We present a method for solving linear and nonlinear PDEs based on the variable projection (VarPro) framework and artificial neural networks (ANN). For linear PDEs, enforcing the boundary/initial value problem on the collocation points leads to a separable nonlinear least squares problem about the network coefficients. We reformulate this problem by the VarPro approach to eliminate the linear outp… ▽ More We present a method for solving linear and nonlinear PDEs based on the variable projection (VarPro) framework and artificial neural networks (ANN). For linear PDEs, enforcing the boundary/initial value problem on the collocation points leads to a separable nonlinear least squares problem about the network coefficients. We reformulate this problem by the VarPro approach to eliminate the linear output-layer coefficients, leading to a reduced problem about the hidden-layer coefficients only. The reduced problem is solved first by the nonlinear least squares method to determine the hidden-layer coefficients, and then the output-layer coefficients are computed by the linear least squares method. For nonlinear PDEs, enforcing the boundary/initial value problem on the collocation points leads to a nonlinear least squares problem that is not separable, which precludes the VarPro strategy for such problems. To enable the VarPro approach for nonlinear PDEs, we first linearize the problem with a Newton iteration, using a particular form of linearization. The linearized system is solved by the VarPro framework together with ANNs. Upon convergence of the Newton iteration, the network coefficients provide the representation of the solution field to the original nonlinear problem. We present ample numerical examples with linear and nonlinear PDEs to demonstrate the performance of the method herein. For smooth field solutions, the errors of the current method decrease exponentially as the number of collocation points or the number of output-layer coefficients increases. We compare the current method with the ELM method from a previous work. Under identical conditions and network configurations, the current method exhibits an accuracy significantly superior to the ELM method. △ Less

Submitted 24 January, 2022; originally announced January 2022.

Comments: 35 pages, 18 figures, 10 tables

Superintegrability on the Dunkl oscillator model in three-Dimensional spaces of constant curvature

Authors: Shi-Hai Dong, Amene Najafizade, Hossein Panahi, Won Sang Chung, Hassan Hassanabadi

Abstract: This paper has studied the three-dimensional Dunkl oscillator models in a generalization of superintegrable Euclidean Hamiltonian systems to curved ones. These models are defined based on curved Hamiltonians, which depend on a deformation parameter of underlying space and involve reflection operators. Their symmetries are obtained by the Jordan-Schwinger representations in the family of the Cayley… ▽ More This paper has studied the three-dimensional Dunkl oscillator models in a generalization of superintegrable Euclidean Hamiltonian systems to curved ones. These models are defined based on curved Hamiltonians, which depend on a deformation parameter of underlying space and involve reflection operators. Their symmetries are obtained by the Jordan-Schwinger representations in the family of the Cayley-Klein orthogonal algebras using the creation and annihilation operators of the dynamical $sl_{-1}(2)$ algebra of the one-dimensional Dunkl oscillator. The resulting algebra is a deformation of $so_{κ_1κ_2}(4)$ with reflections, which is known as the Jordan-Schwinger-Dunkl algebra $jsd_{κ_1κ_2}(4)$. Hence, this model is shown to be maximally superintegrable. On the other hand, the superintegrability of the three-dimensional Dunkl oscillator model is studied from the factorization approach viewpoint. The spectrum of this system is derived through the separation of variables in geodesic polar coordinates, and the resulting eigenfunctions are algebraically given in terms of Jacobi polynomials. △ Less

Submitted 27 December, 2021; originally announced December 2021.

Comments: 21 pages, to appear in Journal quantum

Global solution to the cubic Dirac equation in two space dimensions

Authors: Shijie Dong, Kuijie Li

Abstract: We are interested in the cubic Dirac equation with mass $m \in [0, 1]$ in two space dimensions, which is also known as the Soler model. We conduct a thorough study on this model with initial data sufficiently small in high regularity Sobolev spaces. First, we show the global existence of the model, which is uniform-in-mass. In addition, we derive a unified pointwise decay result valid for all… ▽ More We are interested in the cubic Dirac equation with mass $m \in [0, 1]$ in two space dimensions, which is also known as the Soler model. We conduct a thorough study on this model with initial data sufficiently small in high regularity Sobolev spaces. First, we show the global existence of the model, which is uniform-in-mass. In addition, we derive a unified pointwise decay result valid for all $m \in [0, 1]$. Last but not least, we prove the cubic Dirac equations scatter linearly with an explicit scattering speed. When the mass $m=0$, we can show an improved pointwise decay result. △ Less

Submitted 7 November, 2021; originally announced November 2021.

Comments: 26 pages, all comments are welcome

Global Existence and Scattering of the Klein-Gordon-Zakharov System in Two Space Dimensions

Authors: Shijie Dong, Yue Ma

Abstract: We are interested in the Klein-Gordon-Zakharov system in $\mathbb{R}^{1+2}$, which is an important model in plasma physics with extensive mathematical studies. The system can be regarded as semilinear coupled wave and Klein-Gordon equations with nonlinearities violating the null conditions. Without the compactness assumptions on the initial data, we aim to establish the existence of small global s… ▽ More We are interested in the Klein-Gordon-Zakharov system in $\mathbb{R}^{1+2}$, which is an important model in plasma physics with extensive mathematical studies. The system can be regarded as semilinear coupled wave and Klein-Gordon equations with nonlinearities violating the null conditions. Without the compactness assumptions on the initial data, we aim to establish the existence of small global solutions, and in addition, we want to illustrate the optimal pointwise decay of the solutions. Furthermore, we show that the Klein-Gordon part of the system enjoys linear scattering while the wave part has uniformly bounded low-order energy. None of these goals is easy because of the slow pointwise decay nature of the linear wave and Klein-Gordon components in $\mathbb{R}^{1+2}$. We tackle the difficulties by carefully exploiting the properties of the wave and the Klein-Gordon components, and by relying on the ghost weight energy estimates to close higher-order energy estimates. This appears to be the first pointwise decay result and the first scattering result for the Klein-Gordon-Zakharov system in $\mathbb{R}^{1+2}$ without compactness assumptions. △ Less

Submitted 30 October, 2021; originally announced November 2021.

Comments: 37 pages. Five sections with one appendix. All suggestions and comments are welcome

On Computing the Hyperparameter of Extreme Learning Machines: Algorithm and Application to Computational PDEs, and Comparison with Classical and High-Order Finite Elements

Authors: Suchuan Dong, Jielin Yang

Abstract: We consider the use of extreme learning machines (ELM) for computational partial differential equations (PDE). In ELM the hidden-layer coefficients in the neural network are assigned to random values generated on $[-R_m,R_m]$ and fixed, where $R_m$ is a user-provided constant, and the output-layer coefficients are trained by a linear or nonlinear least squares computation. We present a method for… ▽ More We consider the use of extreme learning machines (ELM) for computational partial differential equations (PDE). In ELM the hidden-layer coefficients in the neural network are assigned to random values generated on $[-R_m,R_m]$ and fixed, where $R_m$ is a user-provided constant, and the output-layer coefficients are trained by a linear or nonlinear least squares computation. We present a method for computing the optimal value of $R_m$ based on the differential evolution algorithm. The presented method enables us to illuminate the characteristics of the optimal $R_m$ for two types of ELM configurations: (i) Single-Rm-ELM, in which a single $R_m$ is used for generating the random coefficients in all the hidden layers, and (ii) Multi-Rm-ELM, in which multiple $R_m$ constants are involved with each used for generating the random coefficients of a different hidden layer. We adopt the optimal $R_m$ from this method and also incorporate other improvements into the ELM implementation. In particular, here we compute all the differential operators involving the output fields of the last hidden layer by a forward-mode auto-differentiation, as opposed to the reverse-mode auto-differentiation in a previous work. These improvements significantly reduce the network training time and enhance the ELM performance. We systematically compare the computational performance of the current improved ELM with that of the finite element method (FEM), both the classical second-order FEM and the high-order FEM with Lagrange elements of higher degrees, for solving a number of linear and nonlinear PDEs. It is shown that the current improved ELM far outperforms the classical FEM. Its computational performance is comparable to that of the high-order FEM for smaller problem sizes, and for larger problem sizes the ELM markedly outperforms the high-order FEM. △ Less

Submitted 26 October, 2021; originally announced October 2021.

Comments: 46 pages, 35 figures

Asymptotic stability for the Dirac--Klein-Gordon system in two space dimensions

Authors: Shijie Dong, Zoe Wyatt

Abstract: We study the Dirac--Klein-Gordon system in $1+2$ spacetime dimensions. We show global existence of the solutions, as well as sharp time decay and linear scattering. One key advance is that we provide the first asymptotic stability result for the Dirac--Klein-Gordon system in $1+2$ spacetime dimensions in the case of a massive Klein-Gordon field and a massless Dirac field. The nonlinearities are be… ▽ More We study the Dirac--Klein-Gordon system in $1+2$ spacetime dimensions. We show global existence of the solutions, as well as sharp time decay and linear scattering. One key advance is that we provide the first asymptotic stability result for the Dirac--Klein-Gordon system in $1+2$ spacetime dimensions in the case of a massive Klein-Gordon field and a massless Dirac field. The nonlinearities are below-critical in two spatial dimensions, and so our method requires the identification of special structures within the system and novel weighted energy estimates. Another key advance, is that our proof allows us to weaken certain conditions on the nonlinear structures that have been assumed in the literature. △ Less

Submitted 14 November, 2023; v1 submitted 28 May, 2021; originally announced May 2021.

Comments: 37 pages

MSC Class: 35L05; 35L52; 35L71; 35Q41

Ridesharing Evacuation Model of Disaster Response

Authors: Lingyu Meng, Zhijie Sasha Dong

Abstract: Timely evacuation is crucial to disaster response, as people can avoid suffering and loss of lives when a major disaster happens. With the development of sharing economy, ridesharing has the advantage of reducing congestion, saving travel time, and optimizing transportation mode to improve disaster evacuation efficiency. The paper proposes to integrate the concept of ridesharing into evacuation an… ▽ More Timely evacuation is crucial to disaster response, as people can avoid suffering and loss of lives when a major disaster happens. With the development of sharing economy, ridesharing has the advantage of reducing congestion, saving travel time, and optimizing transportation mode to improve disaster evacuation efficiency. The paper proposes to integrate the concept of ridesharing into evacuation and develops a mixed-integer programming model for this problem. A real-world case study based on Houston is used to validate the proposed model. A series of instances are designed to compare the evacuation efficiency using two indicators, evacuation percentage and average travel distance. Results reveal that increasing the number of vehicles to help carless individuals might not be the most efficient method in this model. Moreover, this model offers a specific response strategy based on different disaster scales, which not only develops a better evacuation plan for the people but also provides relief agencies insights on resource utilization. △ Less

Submitted 26 July, 2021; v1 submitted 28 March, 2021; originally announced March 2021.

Comments: 6 pages, 3 figures. Proceeding of the 2020 IISE Annual Conference, https://www.proquest.com/openview/403992

A Modified Batch Intrinsic Plasticity Method for Pre-training the Random Coefficients of Extreme Learning Machines

Authors: Suchuan Dong, Zongwei Li

Abstract: In extreme learning machines (ELM) the hidden-layer coefficients are randomly set and fixed, while the output-layer coefficients of the neural network are computed by a least squares method. The randomly-assigned coefficients in ELM are known to influence its performance and accuracy significantly. In this paper we present a modified batch intrinsic plasticity (modBIP) method for pre-training the… ▽ More In extreme learning machines (ELM) the hidden-layer coefficients are randomly set and fixed, while the output-layer coefficients of the neural network are computed by a least squares method. The randomly-assigned coefficients in ELM are known to influence its performance and accuracy significantly. In this paper we present a modified batch intrinsic plasticity (modBIP) method for pre-training the random coefficients in the ELM neural networks. The current method is devised based on the same principle as the batch intrinsic plasticity (BIP) method, namely, by enhancing the information transmission in every node of the neural network. It differs from BIP in two prominent aspects. First, modBIP does not involve the activation function in its algorithm, and it can be applied with any activation function in the neural network. In contrast, BIP employs the inverse of the activation function in its construction, and requires the activation function to be invertible (or monotonic). The modBIP method can work with the often-used non-monotonic activation functions (e.g. Gaussian, swish, Gaussian error linear unit, and radial-basis type functions), with which BIP breaks down. Second, modBIP generates target samples on random intervals with a minimum size, which leads to highly accurate computation results when combined with ELM. The combined ELM/modBIP method is markedly more accurate than ELM/BIP in numerical simulations. Ample numerical experiments are presented with shallow and deep neural networks for function approximation and boundary/initial value problems with partial differential equations. They demonstrate that the combined ELM/modBIP method produces highly accurate simulation results, and that its accuracy is insensitive to the random-coefficient initializations in the neural network. This is in sharp contrast with the ELM results without pre-training of the random coefficients. △ Less

Submitted 14 March, 2021; originally announced March 2021.

Comments: 33 pages, 26 figures

The top-order energy of quasilinear wave equations in two space dimensions is uniformly bounded

Authors: Shijie Dong, Philippe G. LeFloch, Zhen Lei

Abstract: Alinhac solved a long-standing open problem in 2001 and established that quasilinear wave equations in two space dimensions with quadratic null nonlinearities admit global-in-time solutions, provided that the initial data are compactly supported and sufficiently small in Sobolev norm. In this work, Alinhac obtained an upper bound with polynomial growth in time for the top-order energy of the solut… ▽ More Alinhac solved a long-standing open problem in 2001 and established that quasilinear wave equations in two space dimensions with quadratic null nonlinearities admit global-in-time solutions, provided that the initial data are compactly supported and sufficiently small in Sobolev norm. In this work, Alinhac obtained an upper bound with polynomial growth in time for the top-order energy of the solutions. A natural question then arises whether the time-growth is a true phenomena, despite the possible conservation of basic energy. Analogous problems are also of central importance for Schrödinger equations and the incompressible Euler equations in two space dimensions, as studied by Bourgain, Colliander-Keel-Staffilani-Takaoka-Tao, Kiselev-Sverak, and others. In the present paper, we establish that the top-order energy of the solutions in Alinhac theorem remains globally bounded in time, which is opposite to Alinhac's blowup-at-infinity conjecture. △ Less

Submitted 14 March, 2021; originally announced March 2021.

Comments: 28 pages

MSC Class: 35L05

Stability of some two dimensional wave maps: a wave--Klein-Gordon model

Authors: Shijie Dong, Zoe Wyatt

Abstract: We are interested in the stability of a class of totally geodesic wave maps, as recently studied by Abbrescia and Chen, and later by Duan and Ma. The relevant equations of motion are a system of coupled semilinear wave and Klein-Gordon equations in $\mathbb{R}^{1+n}$ whose nonlinearities are critical when $n=2$. In this paper we use a pure energy method to show global existence when $n=2$. By care… ▽ More We are interested in the stability of a class of totally geodesic wave maps, as recently studied by Abbrescia and Chen, and later by Duan and Ma. The relevant equations of motion are a system of coupled semilinear wave and Klein-Gordon equations in $\mathbb{R}^{1+n}$ whose nonlinearities are critical when $n=2$. In this paper we use a pure energy method to show global existence when $n=2$. By carefully examining the structure of the nonlinear terms, we are able to obtain uniform energy bounds at lower orders. This allows us to prove pointwise decay estimates and also to reduce the required regularity. △ Less

Submitted 14 November, 2023; v1 submitted 9 March, 2021; originally announced March 2021.

Comments: arXiv admin note: text overlap with arXiv:2011.11990

MSC Class: 35L05; 35L52; 35L71

New Riemannian preconditioned algorithms for tensor completion via polyadic decomposition

Authors: Shuyu Dong, Bin Gao, Yu Guan, François Glineur

Abstract: We propose new Riemannian preconditioned algorithms for low-rank tensor completion via the polyadic decomposition of a tensor. These algorithms exploit a non-Euclidean metric on the product space of the factor matrices of the low-rank tensor in the polyadic decomposition form. This new metric is designed using an approximation of the diagonal blocks of the Hessian of the tensor completion cost fun… ▽ More We propose new Riemannian preconditioned algorithms for low-rank tensor completion via the polyadic decomposition of a tensor. These algorithms exploit a non-Euclidean metric on the product space of the factor matrices of the low-rank tensor in the polyadic decomposition form. This new metric is designed using an approximation of the diagonal blocks of the Hessian of the tensor completion cost function, thus has a preconditioning effect on these algorithms. We prove that the proposed Riemannian gradient descent algorithm globally converges to a stationary point of the tensor completion problem, with convergence rate estimates using the $Ł$ojasiewicz property. Numerical results on synthetic and real-world data suggest that the proposed algorithms are more efficient in memory and time compared to state-of-the-art algorithms. Moreover, the proposed algorithms display a greater tolerance for overestimated rank parameters in terms of the tensor recovery performance, thus enable a flexible choice of the rank parameter. △ Less

Submitted 2 June, 2022; v1 submitted 26 January, 2021; originally announced January 2021.

Comments: 27 Pages, 10 figures, 4 tables

MSC Class: 15A69; 90C26; 90C30; 90C52

Journal ref: SIAM Journal on Matrix Analysis and Applications, 43-2 (2022), 840-866

Global solution to the Klein-Gordon-Zakharov equations with uniform energy bounds

Authors: Shijie Dong

Abstract: We are interested in the Klein-Gordon-Zakharov equations in $\mathbb{R}^{1+3}$, and we aim to show that the energy for the global solution to the equations is uniformly bounded, and we do not require the compactness assumption on the initial data. To achieve these goals, the key is to apply Alinhac's ghost weight energy estimates adapted to the Klein-Gordon equations. We are interested in the Klein-Gordon-Zakharov equations in $\mathbb{R}^{1+3}$, and we aim to show that the energy for the global solution to the equations is uniformly bounded, and we do not require the compactness assumption on the initial data. To achieve these goals, the key is to apply Alinhac's ghost weight energy estimates adapted to the Klein-Gordon equations. △ Less

Submitted 8 January, 2021; originally announced January 2021.

Comments: 24 pages, comments are welcome

Local Extreme Learning Machines and Domain Decomposition for Solving Linear and Nonlinear Partial Differential Equations

Authors: Suchuan Dong, Zongwei Li

Abstract: We present a neural network-based method for solving linear and nonlinear partial differential equations, by combining the ideas of extreme learning machines (ELM), domain decomposition and local neural networks. The field solution on each sub-domain is represented by a local feed-forward neural network, and $C^k$ continuity is imposed on the sub-domain boundaries. Each local neural network consis… ▽ More We present a neural network-based method for solving linear and nonlinear partial differential equations, by combining the ideas of extreme learning machines (ELM), domain decomposition and local neural networks. The field solution on each sub-domain is represented by a local feed-forward neural network, and $C^k$ continuity is imposed on the sub-domain boundaries. Each local neural network consists of a small number of hidden layers, while its last hidden layer can be wide. The weight/bias coefficients in all hidden layers of the local neural networks are pre-set to random values and are fixed, and only the weight coefficients in the output layers are training parameters. The overall neural network is trained by a linear or nonlinear least squares computation, not by the back-propagation type algorithms. We introduce a block time-marching scheme together with the presented method for long-time dynamic simulations. The current method exhibits a clear sense of convergence with respect to the degrees of freedom in the neural network. Its numerical errors typically decrease exponentially or nearly exponentially as the number of degrees of freedom increases. Extensive numerical experiments have been performed to demonstrate the computational performance of the presented method. We compare the current method with the deep Galerkin method (DGM) and the physics-informed neural network (PINN) in terms of the accuracy and computational cost. The current method exhibits a clear superiority, with its numerical errors and network training time considerably smaller (typically by orders of magnitude) than those of DGM and PINN. We also compare the current method with the classical finite element method (FEM). The computational performance of the current method is on par with, and oftentimes exceeds, the FEM performance. △ Less

Submitted 4 December, 2020; originally announced December 2020.

Comments: 62 pages, 48 figures, 12 tables

Two dimensional wave--Klein-Gordon equations with semilinear nonlinearities

Authors: Shijie Dong, Zoe Wyatt

Abstract: From the work on the weak-null condition by Lindblad and Rodnianski, it is well-known that `bad' quadratic sourcing terms are allowed to appear in coupled semilinear wave equations in three spatial dimensions, provided that such terms appear as sources for `good' variables and that the good variables feed back into the system via `good' sourcing terms. Motivated by these ideas, in this paper we in… ▽ More From the work on the weak-null condition by Lindblad and Rodnianski, it is well-known that `bad' quadratic sourcing terms are allowed to appear in coupled semilinear wave equations in three spatial dimensions, provided that such terms appear as sources for `good' variables and that the good variables feed back into the system via `good' sourcing terms. Motivated by these ideas, in this paper we investigate the small data global existence and pointwise decay of solutions to two systems of coupled wave--Klein-Gordon equations in two spatial dimensions. In particular, we consider critical semilinear nonlinearities for the wave equation and below-critical semilinear nonlinearities for the Klein-Gordon equation. An interesting feature of our two systems is that if the nonlinearities of our PDEs were to be swapped, the nonlinear term in the wave equation would lead to finite-time blow-up. △ Less

Submitted 12 August, 2022; v1 submitted 24 November, 2020; originally announced November 2020.

Comments: V2: small changes

MSC Class: 35L05; 35L52; 35L70

A Nearly-Linear Time Algorithm for Linear Programs with Small Treewidth: A Multiscale Representation of Robust Central Path

Authors: Sally Dong, Yin Tat Lee, Guanghao Ye

Abstract: Arising from structural graph theory, treewidth has become a focus of study in fixed-parameter tractable algorithms in various communities including combinatorics, integer-linear programming, and numerical analysis. Many NP-hard problems are known to be solvable in $\widetilde{O}(n \cdot 2^{O(\mathrm{tw})})$ time, where $\mathrm{tw}$ is the treewidth of the input graph. Analogously, many problems… ▽ More Arising from structural graph theory, treewidth has become a focus of study in fixed-parameter tractable algorithms in various communities including combinatorics, integer-linear programming, and numerical analysis. Many NP-hard problems are known to be solvable in $\widetilde{O}(n \cdot 2^{O(\mathrm{tw})})$ time, where $\mathrm{tw}$ is the treewidth of the input graph. Analogously, many problems in P should be solvable in $\widetilde{O}(n \cdot \mathrm{tw}^{O(1)})$ time; however, due to the lack of appropriate tools, only a few such results are currently known. [Fom+18] conjectured this to hold as broadly as all linear programs; in our paper, we show this is true: Given a linear program of the form $\min_{Ax=b,\ell \leq x\leq u} c^{\top} x$, and a width-$τ$ tree decomposition of a graph $G_A$ related to $A$, we show how to solve it in time $$\widetilde{O}(n \cdot τ^2 \log (1/\varepsilon)),$$ where $n$ is the number of variables and $\varepsilon$ is the relative accuracy. Combined with recent techniques in vertex-capacitated flow [BGS21], this leads to an algorithm with $\widetilde{O}(n^{1+o(1)} \cdot \mathrm{tw}^2 \log (1/\varepsilon))$ run-time. Besides being the first of its kind, our algorithm has run-time nearly matching the fastest run-time for solving the sub-problem $Ax=b$ (under the assumption that no fast matrix multiplication is used). We obtain these results by combining recent techniques in interior-point methods (IPMs), sketching, and a novel representation of the solution under a multiscale basis similar to the wavelet basis. △ Less

Submitted 13 September, 2023; v1 submitted 10 November, 2020; originally announced November 2020.

Alternating minimization algorithms for graph regularized tensor completion

Authors: Yu Guan, Shuyu Dong, Bin Gao, P. -A. Absil, François Glineur

Abstract: We consider a Canonical Polyadic (CP) decomposition approach to low-rank tensor completion (LRTC) by incorporating external pairwise similarity relations through graph Laplacian regularization on the CP factor matrices. The usage of graph regularization entails benefits in the learning accuracy of LRTC, but at the same time, induces coupling graph Laplacian terms that hinder the optimization of th… ▽ More We consider a Canonical Polyadic (CP) decomposition approach to low-rank tensor completion (LRTC) by incorporating external pairwise similarity relations through graph Laplacian regularization on the CP factor matrices. The usage of graph regularization entails benefits in the learning accuracy of LRTC, but at the same time, induces coupling graph Laplacian terms that hinder the optimization of the tensor completion model. In order to solve graph-regularized LRTC, we propose efficient alternating minimization algorithms by leveraging the block structure of the underlying CP decomposition-based model. For the subproblems of alternating minimization, a linear conjugate gradient subroutine is specifically adapted to graph-regularized LRTC. Alternatively, we circumvent the complicating coupling effects of graph Laplacian terms by using an alternating directions method of multipliers. Based on the Kurdyka-Łojasiewicz property, we show that the sequence generated by the proposed algorithms globally converges to a critical point of the objective function. Moreover, the complexity and convergence rate are also derived. In addition, numerical experiments including synthetic data and real data show that the graph regularized tensor completion model has improved recovery results compared to those without graph regularization, and that the proposed algorithms achieve gains in time efficiency over existing algorithms. △ Less

Submitted 11 November, 2023; v1 submitted 28 August, 2020; originally announced August 2020.

MSC Class: 15A69; 49M20; 65B05; 90C26; 90C30; 90C52

A Method for Representing Periodic Functions and Enforcing Exactly Periodic Boundary Conditions with Deep Neural Networks

Authors: Suchuan Dong, Naxian Ni

Abstract: We present a simple and effective method for representing periodic functions and enforcing exactly the periodic boundary conditions for solving differential equations with deep neural networks (DNN). The method stems from some simple properties about function compositions involving periodic functions. It essentially composes a DNN-represented arbitrary function with a set of independent periodic f… ▽ More We present a simple and effective method for representing periodic functions and enforcing exactly the periodic boundary conditions for solving differential equations with deep neural networks (DNN). The method stems from some simple properties about function compositions involving periodic functions. It essentially composes a DNN-represented arbitrary function with a set of independent periodic functions with adjustable (training) parameters. We distinguish two types of periodic conditions: those imposing the periodicity requirement on the function and all its derivatives (to infinite order), and those imposing periodicity on the function and its derivatives up to a finite order $k$ ($k\geqslant 0$). The former will be referred to as $C^{\infty}$ periodic conditions, and the latter $C^{k}$ periodic conditions. We define operations that constitute a $C^{\infty}$ periodic layer and a $C^k$ periodic layer (for any $k\geqslant 0$). A deep neural network with a $C^{\infty}$ (or $C^k$) periodic layer incorporated as the second layer automatically and exactly satisfies the $C^{\infty}$ (or $C^k$) periodic conditions. We present extensive numerical experiments on ordinary and partial differential equations with $C^{\infty}$ and $C^k$ periodic boundary conditions to verify and demonstrate that the proposed method indeed enforces exactly, to the machine accuracy, the periodicity for the DNN solution and its derivatives. △ Less

Submitted 14 July, 2020; originally announced July 2020.

Comments: 31 pages, 16 figures, 8 tables

gPAV-Based Unconditionally Energy-Stable Schemes for the Cahn-Hilliard Equation: Stability and Error Analysis

Authors: Yanxia Qian, Zhiguo Yang, Fei Wang, Suchuan Dong

Abstract: We present several first-order and second-order numerical schemes for the Cahn-Hilliard equation with discrete unconditional energy stability. These schemes stem from the generalized Positive Auxiliary Variable (gPAV) idea, and require only the solution of linear algebraic systems with a constant coefficient matrix. More importantly, the computational complexity (operation count per time step) of… ▽ More We present several first-order and second-order numerical schemes for the Cahn-Hilliard equation with discrete unconditional energy stability. These schemes stem from the generalized Positive Auxiliary Variable (gPAV) idea, and require only the solution of linear algebraic systems with a constant coefficient matrix. More importantly, the computational complexity (operation count per time step) of these schemes is approximately a half of those of the gPAV and the scalar auxiliary variable (SAV) methods in previous works. We investigate the stability properties of the proposed schemes to establish stability bounds for the field function and the auxiliary variable, and also provide their error analyses. Numerical experiments are presented to verify the theoretical analyses and also demonstrate the stability of the schemes at large time step sizes. △ Less

Submitted 14 June, 2020; originally announced June 2020.

Comments: 36 pages, 7 figures

Asymptotic Behavior of the Solution to the Klein-Gordon-Zakharov Model in Dimension Two

Authors: Shijie Dong

Abstract: Consider the Klein-Gordon-Zakharov equations in $\mathbb{R}^{1+2}$, and we are interested in establishing the small global solution to the equations and in investigating the pointwise asymptotic behavior of the solution. The Klein-Gordon-Zakharov equations can be regarded as a coupled semilinear wave and Klein-Gordon system with quadratic nonlinearities which do not satisfy the null conditions, an… ▽ More Consider the Klein-Gordon-Zakharov equations in $\mathbb{R}^{1+2}$, and we are interested in establishing the small global solution to the equations and in investigating the pointwise asymptotic behavior of the solution. The Klein-Gordon-Zakharov equations can be regarded as a coupled semilinear wave and Klein-Gordon system with quadratic nonlinearities which do not satisfy the null conditions, and the fact that wave components and Klein-Gordon components do not decay sufficiently fast makes it harder to conduct the analysis. In order to conquer the difficulties, we will rely on the hyperboloidal foliation method and a minor variance of the ghost weight method. As a side result of the analysis, we are also able to show the small data global existence result for a class of quasilinear wave and Klein-Gordon system violating the null conditions. △ Less

Submitted 8 June, 2020; originally announced June 2020.

Comments: 24 pages, all comments are welcome

Global solution to the wave and Klein-Gordon system under null condition in dimension two

Authors: Shijie Dong

Abstract: We are interested in studying the coupled wave and Klein-Gordon equations with null quadratic nonlinearities in $\mathbb{R}^{2+1}$. We want to establish the small data global existence result, and in addition, we also demonstrate the pointwise asymptotic behaviour of the solution to the coupled system. The initial data are not required to have compact support, and this is achieved by applying the… ▽ More We are interested in studying the coupled wave and Klein-Gordon equations with null quadratic nonlinearities in $\mathbb{R}^{2+1}$. We want to establish the small data global existence result, and in addition, we also demonstrate the pointwise asymptotic behaviour of the solution to the coupled system. The initial data are not required to have compact support, and this is achieved by applying the Alinhac's ghost weight method to both the wave and the Klein-Gordon equations. △ Less

Submitted 10 May, 2020; originally announced May 2020.

Comments: 20 pages; all comments are welcome

The zero mass problem for Klein-Gordon equations: quadratic null interactions

Authors: Shijie Dong

Abstract: We study in $\mathbb{R}^{3+1}$ a system of nonlinearly coupled Klein-Gordon equations under null condition, with (possibly vanishing) mass varying in the interval $[0, 1]$. Our goal is three folds: 1) we want to establish the global well-posedness result to the system which is uniform in terms of the mass parameter; 2) we want to obtain unified pointwise decay result for the solution to the system… ▽ More We study in $\mathbb{R}^{3+1}$ a system of nonlinearly coupled Klein-Gordon equations under null condition, with (possibly vanishing) mass varying in the interval $[0, 1]$. Our goal is three folds: 1) we want to establish the global well-posedness result to the system which is uniform in terms of the mass parameter; 2) we want to obtain unified pointwise decay result for the solution to the system, in the sense that the solution decays more like a wave component (independent of the mass parameter) in certain range of time, while the solution decays as a Klein-Gordon component with a factor depending on the mass parameter in the other part of the time range; 3) the solution to the Klein-Gordon system converges to the solution to the corresponding wave system in certain sense when the mass parameter goes to 0. In order to achieve these goals, we will rely on both the flat and the hyperboloidal foliation of the spacetime. △ Less

Submitted 22 April, 2020; originally announced April 2020.

Comments: 23 pages, comments are welcome

Provably Efficient Reinforcement Learning with Aggregated States

Authors: Shi Dong, Benjamin Van Roy, Zhengyuan Zhou

Abstract: We establish that an optimistic variant of Q-learning applied to a fixed-horizon episodic Markov decision process with an aggregated state representation incurs regret $\tilde{\mathcal{O}}(\sqrt{H^5 M K} + εHK)$, where $H$ is the horizon, $M$ is the number of aggregate states, $K$ is the number of episodes, and $ε$ is the largest difference between any pair of optimal state-action values associate… ▽ More We establish that an optimistic variant of Q-learning applied to a fixed-horizon episodic Markov decision process with an aggregated state representation incurs regret $\tilde{\mathcal{O}}(\sqrt{H^5 M K} + εHK)$, where $H$ is the horizon, $M$ is the number of aggregate states, $K$ is the number of episodes, and $ε$ is the largest difference between any pair of optimal state-action values associated with a common aggregate state. Notably, this regret bound does not depend on the number of states or actions and indicates that asymptotic per-period regret is no greater than $ε$, independent of horizon. To our knowledge, this is the first such result that applies to reinforcement learning with nontrivial value function approximation without any restrictions on transition probabilities. △ Less

Submitted 19 February, 2020; v1 submitted 13 December, 2019; originally announced December 2019.

Stability of a wave and Klein-Gordon system with mixed coupling

Authors: Shijie Dong

Abstract: We are interested in establishing stability results for a system of semilinear wave and Klein-Gordon equations with mixed coupling nonlinearities, that is, we consider all of the possible quadratic nonlinear terms of the type of wave and Klein-Gordon interactions. The main difficulties are due to the absence of derivatives on the wave component in the nonlinearities. By doing a transformation on t… ▽ More We are interested in establishing stability results for a system of semilinear wave and Klein-Gordon equations with mixed coupling nonlinearities, that is, we consider all of the possible quadratic nonlinear terms of the type of wave and Klein-Gordon interactions. The main difficulties are due to the absence of derivatives on the wave component in the nonlinearities. By doing a transformation on the wave equation, we reveal a hidden null structure. Next by using the scaling vector field on the wave component only, which was generally avoided, we are able to get very good $L^2$--type estimates on the wave component. Then we distinguish high order and low order energies of both wave and Klein-Gordon components, which allows us to close the bootstrap argument. △ Less

Submitted 16 July, 2020; v1 submitted 11 December, 2019; originally announced December 2019.

Comments: Added references and more details in the proof

Stability of a class of semilinear waves in $2+1$ dimension under null condition

Authors: Shijie Dong

Abstract: We will show that in $\RR^{2+1}$ semilinear wave equations of the form $-\Box u = u Q(\del u; \del u)$ possess global-in-time solutions if the null condition on $Q(\del u; \del u)$ is assumed. As a consequence, we also provide a new proof, after \cite{Wong}, on the small data global solutions to the wave map equation in $\RR^{2+1}$ and no compactness assumptions on the initial data are needed. We will show that in $\RR^{2+1}$ semilinear wave equations of the form $-\Box u = u Q(\del u; \del u)$ possess global-in-time solutions if the null condition on $Q(\del u; \del u)$ is assumed. As a consequence, we also provide a new proof, after \cite{Wong}, on the small data global solutions to the wave map equation in $\RR^{2+1}$ and no compactness assumptions on the initial data are needed. △ Less

Submitted 14 April, 2020; v1 submitted 22 October, 2019; originally announced October 2019.

Comments: 14 pages, corrected typos, and added more information about the financial support