Showing 1–50 of 75 results for author: Jiang, L

A Primal-Dual-Assisted Penalty Approach to Bilevel Optimization with Coupled Constraints

Authors: Liuyuan Jiang, Quan Xiao, Victor M. Tenorio, Fernando Real-Rojas, Antonio Marques, Tianyi Chen

Abstract: Interest in bilevel optimization has grown in recent years, partially due to its applications to tackle challenging machine-learning problems. Several exciting recent works have been centered around develo** efficient gradient-based algorithms that can solve bilevel optimization problems with provable guarantees. However, the existing literature mainly focuses on bilevel problems either without… ▽ More Interest in bilevel optimization has grown in recent years, partially due to its applications to tackle challenging machine-learning problems. Several exciting recent works have been centered around develo** efficient gradient-based algorithms that can solve bilevel optimization problems with provable guarantees. However, the existing literature mainly focuses on bilevel problems either without constraints, or featuring only simple constraints that do not couple variables across the upper and lower levels, excluding a range of complex applications. Our paper studies this challenging but less explored scenario and develops a (fully) first-order algorithm, which we term BLOCC, to tackle BiLevel Optimization problems with Coupled Constraints. We establish rigorous convergence theory for the proposed algorithm and demonstrate its effectiveness on two well-known real-world applications - hyperparameter selection in support vector machine (SVM) and infrastructure planning in transportation networks using the real data from the city of Seville. △ Less

Submitted 14 June, 2024; originally announced June 2024.

Localized subspace iteration methods for elliptic multiscale problems

Authors: Xiaofei Guan, Lijian Jiang, Yajun Wang, Zihao Yang

Abstract: This paper proposes localized subspace iteration (LSI) methods to construct generalized finite element basis functions for elliptic problems with multiscale coefficients. The key components of the proposed method consist of the localization of the original differential operator and the subspace iteration of the corresponding local spectral problems, where the localization is conducted by enforcing… ▽ More This paper proposes localized subspace iteration (LSI) methods to construct generalized finite element basis functions for elliptic problems with multiscale coefficients. The key components of the proposed method consist of the localization of the original differential operator and the subspace iteration of the corresponding local spectral problems, where the localization is conducted by enforcing the local homogeneous Dirichlet condition and the partition of the unity functions. From a novel perspective, some multiscale methods can be regarded as one iteration step under approximating the eigenspace of the corresponding local spectral problems. Vice versa, new multiscale methods can be designed through subspaces of spectral problem algorithms. Then, we propose the efficient localized standard subspace iteration (LSSI) method and the localized Krylov subspace iteration (LKSI) method based on the standard subspace and Krylov subspace, respectively. Convergence analysis is carried out for the proposed method. Various numerical examples demonstrate the effectiveness of our methods. In addition, the proposed methods show significant superiority in treating long-channel cases over other well-known multiscale methods. △ Less

Submitted 14 June, 2024; originally announced June 2024.

Comments: 23 pages

MSC Class: 65N99; 65N30; 34E13

A Generalized Version of Chung's Lemma and its Applications

Authors: Li Jiang, Xiao Li, Andre Milzarek, Junwen Qiu

Abstract: Chung's lemma is a classical tool for establishing asymptotic convergence rates of (stochastic) optimization methods under strong convexity-type assumptions and appropriate polynomial diminishing step sizes. In this work, we develop a generalized version of Chung's lemma, which provides a simple non-asymptotic convergence framework for a more general family of step size rules. We demonstrate broad… ▽ More Chung's lemma is a classical tool for establishing asymptotic convergence rates of (stochastic) optimization methods under strong convexity-type assumptions and appropriate polynomial diminishing step sizes. In this work, we develop a generalized version of Chung's lemma, which provides a simple non-asymptotic convergence framework for a more general family of step size rules. We demonstrate broad applicability of the proposed generalized Chung's lemma by deriving tight non-asymptotic convergence rates for a large variety of stochastic methods. In particular, we obtain partially new non-asymptotic complexity results for stochastic optimization methods, such as stochastic gradient descent and random reshuffling, under a general $(θ,μ)$-Polyak-Lojasiewicz (PL) condition and for various step sizes strategies, including polynomial, constant, exponential, and cosine step sizes rules. Notably, as a by-product of our analysis, we observe that exponential step sizes can adapt to the objective function's geometry, achieving the optimal convergence rate without requiring exact knowledge of the underlying landscape. Our results demonstrate that the developed variant of Chung's lemma offers a versatile, systematic, and streamlined approach to establish non-asymptotic convergence rates under general step size rules. △ Less

Submitted 9 June, 2024; originally announced June 2024.

Comments: 43 pages, 5 figures

MSC Class: 90C15; 90C30; 90C26

Combining physics-informed graph neural network and finite difference for solving forward and inverse spatiotemporal PDEs

Authors: Hao Zhang, Longxiang Jiang, Xinkun Chu, Yong Wen, Luxiong Li, Yonghao Xiao, Liyuan Wang

Abstract: The great success of Physics-Informed Neural Networks (PINN) in solving partial differential equations (PDEs) has significantly advanced our simulation and understanding of complex physical systems in science and engineering. However, many PINN-like methods are poorly scalable and are limited to in-sample scenarios. To address these challenges, this work proposes a novel discrete approach termed P… ▽ More The great success of Physics-Informed Neural Networks (PINN) in solving partial differential equations (PDEs) has significantly advanced our simulation and understanding of complex physical systems in science and engineering. However, many PINN-like methods are poorly scalable and are limited to in-sample scenarios. To address these challenges, this work proposes a novel discrete approach termed Physics-Informed Graph Neural Network (PIGNN) to solve forward and inverse nonlinear PDEs. In particular, our approach seamlessly integrates the strength of graph neural networks (GNN), physical equations and finite difference to approximate solutions of physical systems. Our approach is compared with the PINN baseline on three well-known nonlinear PDEs (heat, Burgers and FitzHugh-Nagumo). We demonstrate the excellent performance of the proposed method to work with irregular meshes, longer time steps, arbitrary spatial resolutions, varying initial conditions (ICs) and boundary conditions (BCs) by conducting extensive numerical experiments. Numerical results also illustrate the superiority of our approach in terms of accuracy, time extrapolability, generalizability and scalability. The main advantage of our approach is that models trained in small domains with simple settings have excellent fitting capabilities and can be directly applied to more complex situations in large domains. △ Less

Submitted 14 June, 2024; v1 submitted 30 May, 2024; originally announced May 2024.

Mean uniformly stable function and its application to almost sure stability analysis of randomly switched time-varying systems

Authors: Qian Liu, Yong He, Lin Jiang

Abstract: This paper investigates uniform almost sure stability of randomly switched time-varying systems. To assess stability properties of diverse time-varying subsystems, mode-dependent indefinite multiple Lyapunov functions are introduced. We present a novel condition so-called mean uniformly stable function focusing on the time-varying functions in derivatives' parameters of indefinite multiple Lyapuno… ▽ More This paper investigates uniform almost sure stability of randomly switched time-varying systems. To assess stability properties of diverse time-varying subsystems, mode-dependent indefinite multiple Lyapunov functions are introduced. We present a novel condition so-called mean uniformly stable function focusing on the time-varying functions in derivatives' parameters of indefinite multiple Lyapunov functions. These conditions do not need pre-determined switching moments and provide enhanced flexibility to accommodate unstable subsystems and stable but no-exponentially decay subystems. A numerical example is provided to demonstrate the effectiveness and advantages of our approach. △ Less

Submitted 21 January, 2024; originally announced January 2024.

Box dimension of fractal interpolation surfaces with vertical scaling function

Authors: Lai Jiang

Abstract: In this paper, we first present a simple lemma which allows us to estimate the box dimension of graphs of given functions by the associated oscillation sums and oscillation vectors. Then we define vertical scaling matrices of generalized affine fractal interpolation surfaces (FISs). By using these matrices, we establish relationships between oscillation vectors of different levels, which enables u… ▽ More In this paper, we first present a simple lemma which allows us to estimate the box dimension of graphs of given functions by the associated oscillation sums and oscillation vectors. Then we define vertical scaling matrices of generalized affine fractal interpolation surfaces (FISs). By using these matrices, we establish relationships between oscillation vectors of different levels, which enables us to obtain the box dimension of generalized affine FISs under certain constraints. △ Less

Submitted 22 January, 2024; v1 submitted 23 December, 2023; originally announced December 2023.

MSC Class: 28A80; 41A30

Convergence rates for Chernoff-type approximations of convex monotone semigroups

Authors: Jonas Blessing, Lianzi Jiang, Michael Kupper, Gechun Liang

Abstract: We provide explicit convergence rates for Chernoff-type approximations of convex monotone semigroups which have the form $S(t)f=\lim_{n\to\infty}I(\frac{t}{n})^n f$ for bounded continuous functions $f$. Under suitable conditions on the one-step operators $I(t)$ regarding the time regularity and consistency of the approximation scheme, we obtain $\|S(t)f-I(\frac{t}{n})^n f\|_\infty\leq cn^{-γ}$ for… ▽ More We provide explicit convergence rates for Chernoff-type approximations of convex monotone semigroups which have the form $S(t)f=\lim_{n\to\infty}I(\frac{t}{n})^n f$ for bounded continuous functions $f$. Under suitable conditions on the one-step operators $I(t)$ regarding the time regularity and consistency of the approximation scheme, we obtain $\|S(t)f-I(\frac{t}{n})^n f\|_\infty\leq cn^{-γ}$ for bounded Lipschitz continuous functions $f$, where $c\geq 0$ and $γ>0$ are determined explicitly. Moreover, the map** $t\mapsto S(t)f$ is Hölder continuous. These results are closely related to monotone approximation schemes for viscosity solutions but are obtained independently by following a recently developed semigroup approach to Hamilton-Jacobi-Bellman equations which uniquely characterizes semigroups via their $Γ$-generators. The different approach allows to consider convex rather than sublinear equations and the results can be extended to unbounded functions by modifying the norm with a suitable weight function. Furthermore, up to possibly different consistency errors for the operators $I(t)$, the upper and lower bound for the error between the semigroup and the iterated operators are symmetric. The abstract results are applied to Nisio semigroups and limit theorems for convex expectations. △ Less

Submitted 15 October, 2023; originally announced October 2023.

MSC Class: Primary 47H20; 65M15; Secondary 47J25; 49M25; 60G65

Error estimates for the robust $α$-stable central limit theorem under sublinear expectation by discrete approximation method

Authors: Lianzi Jiang

Abstract: In this work, we develop a numerical method to study the error estimates of the $α$-stable central limit theorem under sublinear expectation with $α\in(0,2)$, whose limit distribution can be characterized by a fully nonlinear integro-differential equation (PIDE). Based on the sequence of independent random variables, we propose a discrete approximation scheme for the fully nonlinear PIDE. With the… ▽ More In this work, we develop a numerical method to study the error estimates of the $α$-stable central limit theorem under sublinear expectation with $α\in(0,2)$, whose limit distribution can be characterized by a fully nonlinear integro-differential equation (PIDE). Based on the sequence of independent random variables, we propose a discrete approximation scheme for the fully nonlinear PIDE. With the help of the nonlinear stochastic analysis techniques and numerical analysis tools, we establish the error bounds for the discrete approximation scheme, which in turn provides a general error bound for the robust $α$-stable central limit theorem, including the integrable case $α\in(1,2)$ as well as the non-integrable case $α\in(0,1]$. Finally, we provide some concrete examples to illustrate our main results and derive the precise convergence rates. △ Less

Submitted 5 October, 2023; v1 submitted 3 October, 2023; originally announced October 2023.

Box dimension of generalized affine fractal interpolation functions (II)

Authors: Lai Jiang, Huo-Jun Ruan

Abstract: Let $f$ be a generalized affine fractal interpolation function with vertical scaling functions. In this paper, we first estimate $\mathrm{dim}_B Γf$, the box dimension of the graph of $f$, by the sum function of vertical scaling functions. Then we estimate $\mathrm{dim}_B Γf$ by the limits of spectral radii of vertical scaling matrices under certain conditions. As an application, we study the box… ▽ More Let $f$ be a generalized affine fractal interpolation function with vertical scaling functions. In this paper, we first estimate $\mathrm{dim}_B Γf$, the box dimension of the graph of $f$, by the sum function of vertical scaling functions. Then we estimate $\mathrm{dim}_B Γf$ by the limits of spectral radii of vertical scaling matrices under certain conditions. As an application, we study the box dimension of the graph of a generalized Weierstrass-type function. △ Less

Submitted 15 August, 2023; originally announced August 2023.

Comments: 20 pages, 1 figure. arXiv admin note: text overlap with arXiv:2208.04126

Regularized coupling multiscale method for thermomechanical coupled problems

Authors: Xiaofei Guan, Lijian Jiang, Yajun Wang

Abstract: The coupling effects in multiphysics processes are often neglected in designing multiscale methods. The coupling may be described by a non-positive definite operator, which in turn brings significant challenges in multiscale simulations. In the paper, we develop a regularized coupling multiscale method based on the generalized multiscale finite element method (GMsFEM) to solve coupled thermomechan… ▽ More The coupling effects in multiphysics processes are often neglected in designing multiscale methods. The coupling may be described by a non-positive definite operator, which in turn brings significant challenges in multiscale simulations. In the paper, we develop a regularized coupling multiscale method based on the generalized multiscale finite element method (GMsFEM) to solve coupled thermomechanical problems, and it is referred to as the coupling generalized multiscale finite element method (CGMsFEM). The method consists of defining the coupling multiscale basis functions through local regularized coupling spectral problems in each coarse-grid block, which can be implemented by a novel design of two relaxation parameters. Compared to the standard GMsFEM, the proposed method can not only accurately capture the multiscale coupling correlation effects of multiphysics problems but also greatly improve computational efficiency with fewer multiscale basis functions. In addition, the convergence analysis is also established, and the optimal error estimates are derived, where the upper bound of errors is independent of the magnitude of the relaxation coefficient. Several numerical examples for periodic, random microstructure, and random material coefficients are presented to validate the theoretical analysis. The numerical results show that the CGMsFEM shows better robustness and efficiency than uncoupled GMsFEM. △ Less

Submitted 2 January, 2024; v1 submitted 3 February, 2023; originally announced February 2023.

Comments: 26 pages

MSC Class: 65N99; 65N30; 34E13

Journal ref: Journal of Computational Physics (2023): 112737

A robust $α$-stable central limit theorem under sublinear expectation without integrability condition

Authors: Lianzi Jiang, Gechun Liang

Abstract: This article relaxes the integrability condition imposed in the literature for the robust $α$-stable central limit theorem under sublinear expectation. Specifically, for $α\in(0,1]$, we prove that the normalized sums of i.i.d. non-integrable random variables $\big \{n^{-\frac{1}α}\sum_{i=1}^{n}Z_{i}\big \}_{n=1}^{\infty}$ converge in distribution to $\tildeζ_{1}$, where… ▽ More This article relaxes the integrability condition imposed in the literature for the robust $α$-stable central limit theorem under sublinear expectation. Specifically, for $α\in(0,1]$, we prove that the normalized sums of i.i.d. non-integrable random variables $\big \{n^{-\frac{1}α}\sum_{i=1}^{n}Z_{i}\big \}_{n=1}^{\infty}$ converge in distribution to $\tildeζ_{1}$, where $(\tildeζ_{t})_{t\in \lbrack0,1]}$ is a multidimensional nonlinear symmetric $α$-stable process with a jump uncertainty set $\mathcal{L}$. The limiting $α$-stable process is further characterized by a fully nonlinear partial integro-differential equation (PIDE) \[ \left \{ \begin{array} [c]{l}\displaystyle \partial_{t}u(t,x)-\sup \limits_{F_μ\in \mathcal{L}}\left \{ \int_{\mathbb{R}^{d}}δ_λ^αu(t,x)F_μ(dλ)\right \} =0,\\ \displaystyle u(0,x)=φ(x),\ \ \ \forall(t,x)\in \lbrack0,1]\times \mathbb{R}^{d}, \end{array} \right. \] where \[ δ_λ^α u(t,x):= \left \{ \begin{array} [c]{l} u(t,x+λ)-u(t,x)-\langle D_{x}u(t,x),λ\mathbb{1}_{\{|λ|\leq 1\}}\rangle,\ α=1,\\ u(t,x+λ)-u(t,x),\ α\in(0,1). \end{array} \right. \] The main tools are a weak convergence approach to obtain the limiting process, a Lévy-Khintchine representation of the nonlinear $α$-stable process and a truncation technique to estimate the corresponding $α$-stable Lévy measures. As a byproduct, the article also provides a probabilistic approach to prove the existence of the above fully nonlinear PIDE. △ Less

Submitted 18 January, 2023; originally announced January 2023.

Comments: 26 pages. arXiv admin note: text overlap with arXiv:2205.00203

Asymptotic normality and optimality in nonsmooth stochastic approximation

Authors: Damek Davis, Dmitriy Drusvyatskiy, Liwei Jiang

Abstract: In their seminal work, Polyak and Juditsky showed that stochastic approximation algorithms for solving smooth equations enjoy a central limit theorem. Moreover, it has since been argued that the asymptotic covariance of the method is best possible among any estimation procedure in a local minimax sense of Hájek and Le Cam. A long-standing open question in this line of work is whether similar guara… ▽ More In their seminal work, Polyak and Juditsky showed that stochastic approximation algorithms for solving smooth equations enjoy a central limit theorem. Moreover, it has since been argued that the asymptotic covariance of the method is best possible among any estimation procedure in a local minimax sense of Hájek and Le Cam. A long-standing open question in this line of work is whether similar guarantees hold for important non-smooth problems, such as stochastic nonlinear programming or stochastic variational inequalities. In this work, we show that this is indeed the case. △ Less

Submitted 16 January, 2023; originally announced January 2023.

Comments: The arxiv report arXiv:2108.11832 has been split into two parts. This is Part 2 of the original submission, augmented by a some new results and a reworked exposition

MSC Class: 65K05; 65K10; 90C15; 90C30; 90C06

Perron-Frobenius operator filter for stochastic dynamical systems

Authors: Ningxin Liu, Lijian Jiang

Abstract: The filtering problems are derived from a sequential minimization of a quadratic function representing a compromise between model and data. In this paper, we use the Perron-Frobenius operator in stochastic process to develop a Perron-Frobenius operator filter. The proposed method belongs to Bayesian filtering and works for non-Gaussian distributions for nonlinear stochastic dynamical systems. The… ▽ More The filtering problems are derived from a sequential minimization of a quadratic function representing a compromise between model and data. In this paper, we use the Perron-Frobenius operator in stochastic process to develop a Perron-Frobenius operator filter. The proposed method belongs to Bayesian filtering and works for non-Gaussian distributions for nonlinear stochastic dynamical systems. The recursion of the filtering can be characterized by the composition of Perron-Frobenius operator and likelihood operator. This gives a significant connection between the Perron-Frobenius operator and Bayesian filtering. We numerically fulfil the recursion through approximating the Perron-Frobenius operator by Ulam's method. In this way, the posterior measure is represented by a convex combination of the indicator functions in Ulam's method. To get a low rank approximation for the Perron-Frobenius operator filter, we take a spectral decomposition for the posterior measure by using the eigenfunctions of the discretized Perron-Frobenius operator. A convergence analysis is carried out and shows that the Perron-Frobenius operator filter achieves a higher convergence rate than the particle filter, which uses Dirac measures for the posterior. The proposed method is explored for the data assimilation of the stochastic dynamical systems. A few numerical examples are presented to illustrate the advantage of the Perron-Frobenius operator filter over particle filter and extend Kalman filter. △ Less

Submitted 8 January, 2023; originally announced January 2023.

Comments: 31 pages

MSC Class: 65C40; 37M99;

Data-driven probability density forecast for stochastic dynamical systems

Authors: Meng Zhao, Lijian Jiang

Abstract: In this paper, a data-driven nonparametric approach is presented for forecasting the probability density evolution of stochastic dynamical systems. The method is based on stochastic Koopman operator and extended dynamic mode decomposition (EDMD). To approximate the finite-dimensional eigendecomposition of the stochastic Koopman operator, EDMD is applied to the training data set sampled from the st… ▽ More In this paper, a data-driven nonparametric approach is presented for forecasting the probability density evolution of stochastic dynamical systems. The method is based on stochastic Koopman operator and extended dynamic mode decomposition (EDMD). To approximate the finite-dimensional eigendecomposition of the stochastic Koopman operator, EDMD is applied to the training data set sampled from the stationary distribution of the underlying stochastic dynamical system. The family of the Koopman operators form a semigroup, which is generated by the infinitesimal generator of the stochastic dynamical system. A significant connection between the generator and Fokker-Planck operator provides a way to construct an orthonormal basis of a weighted Hilbert space. A spectral decomposition of the probability density function is accomplished in this weighted space. This approach is a data-driven method and used to predict the probability density evolution and real-time moment estimation. In the limit of the large number of snapshots and observables, the data-driven probability density approximation converges to the Galerkin projection of the semigroup solution of Fokker-Planck equation on a basis adapted to an invariant measure. The proposed method shares the similar idea to diffusion forecast, but renders more accurate probability density than the diffusion forecast does. A few numerical examples are presented to illustrate the performance of the data-driven probability density forecast. △ Less

Submitted 10 October, 2022; v1 submitted 7 October, 2022; originally announced October 2022.

Comments: typos corrected

A Validation Approach to Over-parameterized Matrix and Image Recovery

Authors: Lijun Ding, Zhen Qin, Liwei Jiang, **xin Zhou, Zhihui Zhu

Abstract: In this paper, we study the problem of recovering a low-rank matrix from a number of noisy random linear measurements. We consider the setting where the rank of the ground-truth matrix is unknown a prior and use an overspecified factored representation of the matrix variable, where the global optimal solutions overfit and do not correspond to the underlying ground-truth. We then solve the associat… ▽ More In this paper, we study the problem of recovering a low-rank matrix from a number of noisy random linear measurements. We consider the setting where the rank of the ground-truth matrix is unknown a prior and use an overspecified factored representation of the matrix variable, where the global optimal solutions overfit and do not correspond to the underlying ground-truth. We then solve the associated nonconvex problem using gradient descent with small random initialization. We show that as long as the measurement operators satisfy the restricted isometry property (RIP) with its rank parameter scaling with the rank of ground-truth matrix rather than scaling with the overspecified matrix variable, gradient descent iterations are on a particular trajectory towards the ground-truth matrix and achieve nearly information-theoretically optimal recovery when stop appropriately. We then propose an efficient early stop** strategy based on the common hold-out method and show that it detects nearly optimal estimator provably. Moreover, experiments show that the proposed validation approach can also be efficiently used for image restoration with deep image prior which over-parameterizes an image with a deep network. △ Less

Submitted 21 September, 2022; originally announced September 2022.

Comments: 29 pages and 9 figures

Box dimension of generalized affine fractal interpolation functions

Authors: Lai Jiang, Huo-Jun Ruan

Abstract: Let $f$ be a generalized affine fractal interpolation function with vertical scaling function $S$. In this paper, we study $\dim_B Γf$, the box dimension of the graph of $f$, under the assumption that $S$ is a Lipschtz function. By introducing vertical scaling matrices, we estimate the upper bound and the lower bound of oscillations of $f$. As a result, we obtain explicit formula of $\dim_B Γf$ un… ▽ More Let $f$ be a generalized affine fractal interpolation function with vertical scaling function $S$. In this paper, we study $\dim_B Γf$, the box dimension of the graph of $f$, under the assumption that $S$ is a Lipschtz function. By introducing vertical scaling matrices, we estimate the upper bound and the lower bound of oscillations of $f$. As a result, we obtain explicit formula of $\dim_B Γf$ under certain constraint conditions. △ Less

Submitted 8 August, 2022; originally announced August 2022.

Comments: 18 pages, 1 figure

MSC Class: Primary 28A80; Secondary 41A30

A new extension of generalized Drazin inverse in Banach algebras

Authors: Yanxun Ren, Lining Jiang

Abstract: In this paper, we introduce and study a new generalized inverse, called ag-Drazin inverses in a Banach algebra $\mathcal{A}$ with unit $1$. An element $a\in\mathcal{A}$ is ag-Drazin invertible if there exists $x\in\mathcal{A}$ such that $ax=xa, \, xax=x \ {\rm and} \ a-axa\in\mathcal{A}^{acc}$, where… ▽ More In this paper, we introduce and study a new generalized inverse, called ag-Drazin inverses in a Banach algebra $\mathcal{A}$ with unit $1$. An element $a\in\mathcal{A}$ is ag-Drazin invertible if there exists $x\in\mathcal{A}$ such that $ax=xa, \, xax=x \ {\rm and} \ a-axa\in\mathcal{A}^{acc}$, where $\mathcal{A}^{acc}\triangleq\{a\in\mathcal{A}: a-λ1 \ {\rm is \ generalized \ Drazin\ invertible} \ {\rm for \ all} \ λ\in\mathbb{C}\backslash\{0\}\}.$ Using idempotent elements, we characterize this inverse and give some its representations. Also, we prove that $a\in\mathcal{A}$ is ag-Drazin invertible if and only if $0$ is not an accumulation point of $σ_{d}(a)$, where $σ_{d}(a)$ is the generalized Drazin spectrum of $a$. △ Less

Submitted 7 May, 2022; originally announced May 2022.

MSC Class: 46H05; 46H09; 15A99

A universal robust limit theorem for nonlinear Lévy processes under sublinear expectation

Authors: Mingshang Hu, Lianzi Jiang, Gechun Liang, Shige Peng

Abstract: This article establishes a universal robust limit theorem under a sublinear expectation framework. Under moment and consistency conditions, we show that, for $α\in(1,2)$, the i.i.d. sequence \[ \left \{ \left( \frac{1}{\sqrt{n}}\sum_{i=1}^{n}X_{i},\frac{1}{n}\sum _{i=1}^{n}Y_{i},\frac{1}{\sqrt[α]{n}}\sum_{i=1}^{n}Z_{i}\right) \right \} _{n=1}^{\infty} \] converges in distribution to… ▽ More This article establishes a universal robust limit theorem under a sublinear expectation framework. Under moment and consistency conditions, we show that, for $α\in(1,2)$, the i.i.d. sequence \[ \left \{ \left( \frac{1}{\sqrt{n}}\sum_{i=1}^{n}X_{i},\frac{1}{n}\sum _{i=1}^{n}Y_{i},\frac{1}{\sqrt[α]{n}}\sum_{i=1}^{n}Z_{i}\right) \right \} _{n=1}^{\infty} \] converges in distribution to $\tilde{L}_{1}$, where $\tilde{L}_{t}=(\tilde ξ_{t},\tildeη_{t},\tildeζ_{t})$, $t\in [0,1]$, is a multidimensional nonlinear Lévy process with an uncertainty set $Θ$ as a set of Lévy triplets. This nonlinear Lévy process is characterized by a fully nonlinear and possibly degenerate partial integro-differential equation (PIDE) \[ \left \{ \begin{array} [c]{l} \displaystyle \partial_{t}u(t,x,y,z)-\sup \limits_{(F_μ,q,Q)\in Θ}\left \{ \int_{\mathbb{R}^{d}}δ_λu(t,x,y,z)F_μ(dλ)\right. \\ \displaystyle \text{\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ }\left. +\langle D_{y}u(t,x,y,z),q\rangle+\frac{1}{2}tr[D_{x}^{2}u(t,x,y,z)Q]\right \} =0,\\ \displaystyle u(0,x,y,z)=φ(x,y,z),\ \ \forall(t,x,y,z)\in \lbrack 0,1]\times \mathbb{R}^{3d}, \end{array} \right. \] with $δ_λu(t,x,y,z):=u(t,x,y,z+λ)-u(t,x,y,z)-\langle D_{z}u(t,x,y,z),λ\rangle$. To construct the limit process $(\tilde{L}_{t})_{t\in \lbrack0,1]}$, we develop a novel weak convergence approach based on the notions of tightness and weak compactness on a sublinear expectation space. We further prove a new type of Lévy-Khintchine representation formula to characterize $(\tilde{L}_{t})_{t\in [0,1]}$. As a byproduct, we also provide a probabilistic approach to prove the existence of the above fully nonlinear degenerate PIDE. △ Less

Submitted 27 October, 2022; v1 submitted 30 April, 2022; originally announced May 2022.

Comments: 33 pages

MSC Class: 60F05; 60G51; 60G52; 60G65; 45K05

A nearly linearly convergent first-order method for nonsmooth functions with quadratic growth

Authors: Damek Davis, Liwei Jiang

Abstract: Classical results show that gradient descent converges linearly to minimizers of smooth strongly convex functions. A natural question is whether there exists a locally nearly linearly convergent method for nonsmooth functions with quadratic growth. This work designs such a method for a wide class of nonsmooth and nonconvex locally Lipschitz functions, including max-of-smooth, Shapiro's decomposabl… ▽ More Classical results show that gradient descent converges linearly to minimizers of smooth strongly convex functions. A natural question is whether there exists a locally nearly linearly convergent method for nonsmooth functions with quadratic growth. This work designs such a method for a wide class of nonsmooth and nonconvex locally Lipschitz functions, including max-of-smooth, Shapiro's decomposable class, and generic semialgebraic functions. The algorithm is parameter-free and derives from Goldstein's conceptual subgradient method. △ Less

Submitted 17 July, 2023; v1 submitted 29 April, 2022; originally announced May 2022.

Comments: 75 pages, 8 figures. Revised intro. New experiments. Simplified proofs. PyTorch code available here: https://github.com/COR-OPT/ntd.py/

MSC Class: 65K05; 65K10; 90C15; 90C30

Data-driven reduced-order modeling for nonautonomous dynamical systems in multiscale media

Authors: Mengnan Li, Lijian Jiang

Abstract: In this article, we present data-driven reduced-order modeling for nonautonomous dynamical systems in multiscale media using Koopman operators. Different from the case of autonomous dynamical systems, the Koopman operator family of nonautonomous dynamical systems significantly depend on a time pair. In order to effectively estimate the time-dependent Koopman operators, a moving time window is used… ▽ More In this article, we present data-driven reduced-order modeling for nonautonomous dynamical systems in multiscale media using Koopman operators. Different from the case of autonomous dynamical systems, the Koopman operator family of nonautonomous dynamical systems significantly depend on a time pair. In order to effectively estimate the time-dependent Koopman operators, a moving time window is used to decompose the snapshot data, and the extended dynamic mode decomposition method is applied to computing the Koopman operators in each local temporal domain. Many physical properties in multiscale media often vary in very different scales. In order to capture multiscale information well, the dimension of collected data may be high. To accurately construct the models of dynamical systems in multiscale media, we use high spatial dimension of observation data. It is challenging to compute the Koopman operators using the very high dimensional data. Thus, the strategy of reduced-order modeling is proposed to treat the difficulty. The proposed reduced-order modeling includes two stages: offline stage and online stage. In offline stage, a block-wise low rank decomposition is used to reduce the spatial dimension of initial snapshot data. For the nonautonomous dynamical systems, real-time observation data may be required to update the Koopman operators. The online reduced-order modeling is proposed to correct the offline reduced-order modeling. Three methods are developed for the online reduced-order modeling: fully online, semi-online and adaptive online. The adaptive online method automatically selects the fully online or semi-online and can achieve a good trade-off between modeling accuracy and efficiency. △ Less

Submitted 8 January, 2023; v1 submitted 27 April, 2022; originally announced April 2022.

Comments: 34 pages

MSC Class: 65N99; 65N30; 68T09; 35K55

Journal ref: Journal of Computational Physics, 474 (2023), 111799

Online multiscale model reduction for nonlinear stochastic PDEs with multiplicative noise

Authors: Lijian Jiang, Mengnan Li, Meng Zhao

Abstract: In this paper, an online multiscale model reduction method is presented for stochastic partial differential equations (SPDEs) with multiplicative noise, where the diffusion coefficient is spatially multiscale and the noise perturbation nonlinearly depends on the diffusion dynamics. It is necessary to efficiently compute all possible trajectories of the stochastic dynamics for quantifying model's u… ▽ More In this paper, an online multiscale model reduction method is presented for stochastic partial differential equations (SPDEs) with multiplicative noise, where the diffusion coefficient is spatially multiscale and the noise perturbation nonlinearly depends on the diffusion dynamics. It is necessary to efficiently compute all possible trajectories of the stochastic dynamics for quantifying model's uncertainty and statistic moments. The multiscale diffusion and nonlinearity may cause the computation intractable. To overcome the multiscale difficulty, a constraint energy minimizing generalized multiscale finite element method (CEM-GMsFEM) is used to localize the computation and obtain an effective coarse model. However, the nonlinear terms are still defined on a fine scale space after the Galerkin projection of CEM-GMsFEM is applied to the nonlinear SPDEs. This significantly impacts on the simulation efficiency by CEM-GMsFEM. To this end, a stochastic online discrete empirical interpolation method (DEIM) is proposed to treat the stochastic nonlinearity. The stochastic online DEIM incorporates offline snapshots and online snapshots. The offline snapshots consist of the nonlinear terms at the approximate mean of the stochastic dynamics and are used to construct an offline reduced model. The online snapshots contain some information of the current new trajectory and are used to correct the offline reduced model in an increment manner. The stochastic online DEIM substantially reduces the dimension of the nonlinear dynamics and enhances the prediction accuracy for the reduced model. Thus, the online multiscale model reduction is constructed by using CEM-GMsFEM and the stochastic online DEIM. A priori error analysis is carried out for the nonlinear SPDEs. We present a few numerical examples with diffusion in heterogeneous porous media and show the effectiveness of the proposed model reduction. △ Less

Submitted 25 April, 2022; originally announced April 2022.

Comments: to be published in Multiscale Modeling and Simulation

Algorithmic Regularization in Model-free Overparametrized Asymmetric Matrix Factorization

Authors: Liwei Jiang, Yudong Chen, Lijun Ding

Abstract: We study the asymmetric matrix factorization problem under a natural nonconvex formulation with arbitrary overparametrization. The model-free setting is considered, with minimal assumption on the rank or singular values of the observed matrix, where the global optima provably overfit. We show that vanilla gradient descent with small random initialization sequentially recovers the principal compone… ▽ More We study the asymmetric matrix factorization problem under a natural nonconvex formulation with arbitrary overparametrization. The model-free setting is considered, with minimal assumption on the rank or singular values of the observed matrix, where the global optima provably overfit. We show that vanilla gradient descent with small random initialization sequentially recovers the principal components of the observed matrix. Consequently, when equipped with proper early stop**, gradient descent produces the best low-rank approximation of the observed matrix without explicit regularization. We provide a sharp characterization of the relationship between the approximation error, iteration complexity, initialization size and stepsize. Our complexity bound is almost dimension-free and depends logarithmically on the approximation error, with significantly more lenient requirements on the stepsize and initialization compared to prior work. Our theoretical results provide accurate prediction for the behavior gradient descent, showing good agreement with numerical experiments. △ Less

Submitted 15 September, 2022; v1 submitted 5 March, 2022; originally announced March 2022.

Comments: 30 pages, 7 figures

Journal ref: SIAM Journal on Mathematics of Data Science , vol. 5, no. 3, pp. 723-744, 2023

Rank Overspecified Robust Matrix Recovery: Subgradient Method and Exact Recovery

Authors: Lijun Ding, Liwei Jiang, Yudong Chen, Qing Qu, Zhihui Zhu

Abstract: We study the robust recovery of a low-rank matrix from sparsely and grossly corrupted Gaussian measurements, with no prior knowledge on the intrinsic rank. We consider the robust matrix factorization approach. We employ a robust $\ell_1$ loss function and deal with the challenge of the unknown rank by using an overspecified factored representation of the matrix variable. We then solve the associat… ▽ More We study the robust recovery of a low-rank matrix from sparsely and grossly corrupted Gaussian measurements, with no prior knowledge on the intrinsic rank. We consider the robust matrix factorization approach. We employ a robust $\ell_1$ loss function and deal with the challenge of the unknown rank by using an overspecified factored representation of the matrix variable. We then solve the associated nonconvex nonsmooth problem using a subgradient method with diminishing stepsizes. We show that under a regularity condition on the sensing matrices and corruption, which we call restricted direction preserving property (RDPP), even with rank overspecified, the subgradient method converges to the exact low-rank solution at a sublinear rate. Moreover, our result is more general in the sense that it automatically speeds up to a linear rate once the factor rank matches the unknown rank. On the other hand, we show that the RDPP condition holds under generic settings, such as Gaussian measurements under independent or adversarial sparse corruptions, where the result could be of independent interest. Both the exact recovery and the convergence rate of the proposed subgradient method are numerically verified in the overspecified regime. Moreover, our experiment further shows that our particular design of diminishing stepsize effectively prevents overfitting for robust recovery under overparameterized models, such as robust matrix sensing and learning robust deep image prior. This regularization effect is worth further investigation. △ Less

Submitted 26 October, 2021; v1 submitted 23 September, 2021; originally announced September 2021.

Comments: 75 pages, 3 figures

Active manifolds, stratifications, and convergence to local minima in nonsmooth optimization

Authors: Damek Davis, Dmitriy Drusvyatskiy, Liwei Jiang

Abstract: We show that the subgradient method converges only to local minimizers when applied to generic Lipschitz continuous and subdifferentially regular functions that are definable in an o-minimal structure. At a high level, the argument we present is appealingly transparent: we interpret the nonsmooth dynamics as an approximate Riemannian gradient method on a certain distinguished submanifold that capt… ▽ More We show that the subgradient method converges only to local minimizers when applied to generic Lipschitz continuous and subdifferentially regular functions that are definable in an o-minimal structure. At a high level, the argument we present is appealingly transparent: we interpret the nonsmooth dynamics as an approximate Riemannian gradient method on a certain distinguished submanifold that captures the nonsmooth activity of the function. In the process, we develop new regularity conditions in nonsmooth analysis that parallel the stratification conditions of Whitney, Kuo, and Verdier and extend stochastic processes techniques of Pemantle. △ Less

Submitted 9 January, 2023; v1 submitted 26 August, 2021; originally announced August 2021.

Comments: Version 1 of the arxiv report has been split into two parts. Version 2 of the arxiv report is Part 1 of the original submission. Part 2 will appear as a separate arxiv submission

MSC Class: 65K05; 65K10; 90C15; 90C30; 90C06

On the rate of convergence for an $α$-stable central limit theorem under sublinear expectation

Authors: Mingshang Hu, Lianzi Jiang, Gechun Liang

Abstract: In this paper, we propose a monotone approximation scheme for a class of fully nonlinear degenerate partial integro-differential equations (PIDEs) which characterize the nonlinear $α$-stable Lévy processes under sublinear expectation space with $α\in(1,2)$. We further establish the error bounds for the monotone approximation scheme. This in turn yields an explicit Berry-Esseen bound and convergenc… ▽ More In this paper, we propose a monotone approximation scheme for a class of fully nonlinear degenerate partial integro-differential equations (PIDEs) which characterize the nonlinear $α$-stable Lévy processes under sublinear expectation space with $α\in(1,2)$. We further establish the error bounds for the monotone approximation scheme. This in turn yields an explicit Berry-Esseen bound and convergence rate for the $α$-stable central limit theorem under sublinear expectation. △ Less

Submitted 10 June, 2024; v1 submitted 23 July, 2021; originally announced July 2021.

Comments: 25 pages

MSC Class: 60F05; 60E07; 65M15

Remark on common properties of the products $ac$ and $ba$

Authors: Yanxun Ren, Lining Jiang

Abstract: In this paper, we discuss the common properties for the products $ac$ and $ba$ in various categories under the condition $a(ba)^{2}=abaca=acaba=(ac)^{2}a$. We prove that generalized Jacobson's lemma and Cline's formula are suitable for generalized n-strong Drazin invertible in rings, and generalized Jacobson's lemma is suitable for left and right Fredholm operator on Banach spaces. In this paper, we discuss the common properties for the products $ac$ and $ba$ in various categories under the condition $a(ba)^{2}=abaca=acaba=(ac)^{2}a$. We prove that generalized Jacobson's lemma and Cline's formula are suitable for generalized n-strong Drazin invertible in rings, and generalized Jacobson's lemma is suitable for left and right Fredholm operator on Banach spaces. △ Less

Submitted 22 October, 2021; v1 submitted 23 April, 2021; originally announced April 2021.

Comments: the proof of theorem is incorrect

MSC Class: 15A09; 47A53

Extensions of Jacobson's lemma for generalized inverses in a ring

Authors: Yanxun Ren, Lining Jiang

Abstract: Let $R$ be an associative ring with unit $1$, and $a, b, c\in R$ satisfy $a(ba)^{2}=abaca=acaba=(ac)^{2}a$, this paper proves that $1-ac$ has generalized Drazin inverse (Drazin inverse, pseudo Drazin inverse, respectively) if and only if $1-ba$ has generalized Drazin inverse (Drazin inverse, pseudo Drazin inverse, respectively). In particular, we obtain new common spectral properties for $ac$ and… ▽ More Let $R$ be an associative ring with unit $1$, and $a, b, c\in R$ satisfy $a(ba)^{2}=abaca=acaba=(ac)^{2}a$, this paper proves that $1-ac$ has generalized Drazin inverse (Drazin inverse, pseudo Drazin inverse, respectively) if and only if $1-ba$ has generalized Drazin inverse (Drazin inverse, pseudo Drazin inverse, respectively). In particular, we obtain new common spectral properties for $ac$ and $ba$ in Banach algebras. As applications, new extension of Jacobson's lemma for B-Fredholm elements and generalized Fredholm elements in rings is established. △ Less

Submitted 22 October, 2021; v1 submitted 20 April, 2021; originally announced April 2021.

Comments: the proof of Theorem is incorrect

MSC Class: Primary 15A09; Secondary 16B99; 16U99

Discrete-time approximation for stochastic optimal control problems under the $G$-expectation framework

Authors: Lianzi Jiang

Abstract: In this paper, we propose a class of discrete-time approximation schemes for stochastic optimal control problems under the $G$-expectation framework. The proposed schemes are constructed recursively based on piecewise constant policy. We prove the convergence of the discrete schemes and determine the convergence rates. Several numerical examples are presented to illustrate the effectiveness of the… ▽ More In this paper, we propose a class of discrete-time approximation schemes for stochastic optimal control problems under the $G$-expectation framework. The proposed schemes are constructed recursively based on piecewise constant policy. We prove the convergence of the discrete schemes and determine the convergence rates. Several numerical examples are presented to illustrate the effectiveness of the obtained results. △ Less

Submitted 3 October, 2021; v1 submitted 5 April, 2021; originally announced April 2021.

Comments: 20 pages

MSC Class: 93-08 (49L20 49M25 60H10 65K10 93E20)

Big Hankel operators on Hardy spaces of strongly pseudoconvex domains

Authors: Bo-Yong Chen, Liangying Jiang

Abstract: In this article, we investigate the (big) Hankel operators $H_f$ on Hardy spaces of strongly pseudoconvex domains with smooth boundaries in $\mathbb{C}^n$. We also give a necessary and sufficient condition for boundedness of the Hankel operator $H_f$ on the Hardy space of the unit disc, which is new in the setting of one variable. In this article, we investigate the (big) Hankel operators $H_f$ on Hardy spaces of strongly pseudoconvex domains with smooth boundaries in $\mathbb{C}^n$. We also give a necessary and sufficient condition for boundedness of the Hankel operator $H_f$ on the Hardy space of the unit disc, which is new in the setting of one variable. △ Less

Submitted 7 February, 2021; originally announced February 2021.

Comments: 23 pages

A Kaczmarz Method with Simple Random Sampling for Solving Large Linear Systems

Authors: Yutong Jiang, Gang Wu, Long Jiang

Abstract: The Kaczmarz method is a popular iterative scheme for solving large-scale linear systems. The randomized Kaczmarz method (RK) greatly improves the convergence rate of the Kaczmarz method, by using the rows of the coefficient matrix in random order rather than in their given order. An obvious disadvantage of the randomized Kaczmarz method is its probability criterion for selecting the active or wor… ▽ More The Kaczmarz method is a popular iterative scheme for solving large-scale linear systems. The randomized Kaczmarz method (RK) greatly improves the convergence rate of the Kaczmarz method, by using the rows of the coefficient matrix in random order rather than in their given order. An obvious disadvantage of the randomized Kaczmarz method is its probability criterion for selecting the active or working rows in the coefficient matrix. In [{\sc Z.Z. Bai, W. Wu}, {\em On greedy randomized Kaczmarz method for solving large sparse linear systems}, SIAM Journal on Scientific Computing, 2018, 40: A592--A606], the authors proposed a greedy randomized Kaczmarz method (GRK). However, this method may suffer from heavily computational cost when the size of the matrix is large, and the overhead will be prohibitively large for big data problems. The contribution of this work is as follows. First, from the probability significance point of view, we present a partially randomized Kaczmarz method, which can reduce the computational overhead needed in greedy randomized Kaczmarz method. Second, based on Chebyshev's law of large numbers and Z-test, we apply a simple sampling approach to the partially randomized Kaczmarz method. The convergence of the proposed method is established. Third, we apply the new strategy to the ridge regression problem, and propose a partially randomized Kaczmarz method with simple random sampling for ridge regression. Numerical experiments demonstrate the superiority of the new algorithms over many state-of-the-art randomized Kaczmarz methods for large linear systems problems and ridge regression problems. △ Less

Submitted 30 November, 2020; originally announced November 2020.

Comments: 26 pages, 6 figures

MSC Class: 65F 15; 65F 10

An Efficient Numerical Method for Forward-Backward Stochastic Differential Equations Driven by $G$-Brownian motion

Authors: Mingshang Hu, Lianzi Jiang

Abstract: In this paper, we study the numerical method for solving forward-backward stochastic differential equations driven by $G$-Brownian motion ($G$-FBSDEs) which correspond to fully nonlinear partial differential equations (PDEs). First, we give an approximate conditional $G$-expectation and obtain feasible methods to calculate the distribution of $G$-Brownian motion. On this basis, some efficient nume… ▽ More In this paper, we study the numerical method for solving forward-backward stochastic differential equations driven by $G$-Brownian motion ($G$-FBSDEs) which correspond to fully nonlinear partial differential equations (PDEs). First, we give an approximate conditional $G$-expectation and obtain feasible methods to calculate the distribution of $G$-Brownian motion. On this basis, some efficient numerical schemes for $G$-FBSDEs are then proposed. We rigorously analyze errors of the proposed schemes and prove the convergence results. Finally, several numerical experiments are given to demonstrate the accuracy of our method. △ Less

Submitted 18 May, 2022; v1 submitted 1 October, 2020; originally announced October 2020.

Comments: 30 pages

MSC Class: 60H35; 65C20; 60H10

Direct and inverse results for popular differences in trees of positive dimension

Authors: Alexander Fish, Leo Jiang, with a joint appendix with Ilya D. Shkredov

Abstract: We establish analogues for trees of results relating the density of a set $E \subset \mathbb{N}$, the density of its set of popular differences, and the structure of $E$. To obtain our results, we formalise a correspondence principle of Furstenberg and Weiss which relates combinatorial data on a tree to the dynamics of a Markov process. Our main tools are Kneser-type inverse theorems for sets of r… ▽ More We establish analogues for trees of results relating the density of a set $E \subset \mathbb{N}$, the density of its set of popular differences, and the structure of $E$. To obtain our results, we formalise a correspondence principle of Furstenberg and Weiss which relates combinatorial data on a tree to the dynamics of a Markov process. Our main tools are Kneser-type inverse theorems for sets of return times in measure-preserving systems. In the ergodic setting we use a recent result of the first author with Björklund and Shkredov and a stability-type extension (proved jointly with Shkredov); we also prove a new result for non-ergodic systems. △ Less

Submitted 9 February, 2023; v1 submitted 23 August, 2020; originally announced August 2020.

Comments: 21 pages, 1 figure

An Effective Discrete Recursive Method for Stochastic Optimal Control Problems

Authors: Mingshang Hu, Lianzi Jiang

Abstract: In this paper, we study the numerical method for stochastic optimal control problems (SOCPs). By reducing the optimal control problem to the discrete case, we derive a discrete stochastic maximum principle (SMP). With the help of this SMP, we propose an effective discrete recursive method for SOCPs with feedback control. We rigorously analyze errors of the proposed method and prove that the cost o… ▽ More In this paper, we study the numerical method for stochastic optimal control problems (SOCPs). By reducing the optimal control problem to the discrete case, we derive a discrete stochastic maximum principle (SMP). With the help of this SMP, we propose an effective discrete recursive method for SOCPs with feedback control. We rigorously analyze errors of the proposed method and prove that the cost obtained by our method is of first-order convergence. Numerical experiments are carried out to support our theoretical results. △ Less

Submitted 13 July, 2020; originally announced July 2020.

MSC Class: 60H35; 65C20; 93E20

Numerical Schemes for Backward Stochastic Differential Equations Driven by $G$-Brownian motion

Authors: Mingshang Hu, Lianzi Jiang

Abstract: We design a class of numerical schemes for backward stochastic differential equation driven by $G$-Brownian motion ($G$-BSDE), which is related to a fully nonlinear PDE. Based on Peng's central limit theorem, we employ the CLT method to approximate $G$-distributed. Rigorous stability and convergence analysis are also carried out. It is shown that the $θ$-scheme admits a half order convergence rate… ▽ More We design a class of numerical schemes for backward stochastic differential equation driven by $G$-Brownian motion ($G$-BSDE), which is related to a fully nonlinear PDE. Based on Peng's central limit theorem, we employ the CLT method to approximate $G$-distributed. Rigorous stability and convergence analysis are also carried out. It is shown that the $θ$-scheme admits a half order convergence rate in the general case. In particular, for the case of $θ_{1}\in[0,1]$ and $θ_{2}=0$, the scheme can reach first-order in the deterministic case. Several numerical tests are given to support our theoretical results. △ Less

Submitted 29 November, 2019; originally announced November 2019.

Comments: 25 pages, 2 figures

Explicit $θ$-Schemes for Solving Anticipated Backward Stochastic Differential Equations

Authors: Mingshang Hu, Lianzi Jiang

Abstract: In this paper, a class of stable explicit $θ$-schemes are proposed for solving anticipated backward stochastic differential equations (anticipated BSDEs) which generator not only contains the present values of the solutions but also the future. We subtly transform the delay process of the generator into the current measurable process, resulting in high-order convergence rate. We also analyze the s… ▽ More In this paper, a class of stable explicit $θ$-schemes are proposed for solving anticipated backward stochastic differential equations (anticipated BSDEs) which generator not only contains the present values of the solutions but also the future. We subtly transform the delay process of the generator into the current measurable process, resulting in high-order convergence rate. We also analyze the stability of our numerical schemes and strictly prove the error estimates. Various numerical tests powerful demonstrate high accuracy of the proposed numerical schemes. △ Less

Submitted 10 June, 2019; v1 submitted 4 June, 2019; originally announced June 2019.

Comments: 23 pages

Consecutive Detecting Arrays for Interaction Faults

Authors: Ce Shi, Ling Jiang, Aiyuan Tao

Abstract: The concept of detecting arrays was developed to locate and detect interaction faults arising between the factors in a component-based system during software testing. In this paper, we propose a family of consecutive detecting arrays (CDAs) in which the interactions between factors are considered to be ordered. CDAs can be used to generate test suites for locating and detecting interaction faults… ▽ More The concept of detecting arrays was developed to locate and detect interaction faults arising between the factors in a component-based system during software testing. In this paper, we propose a family of consecutive detecting arrays (CDAs) in which the interactions between factors are considered to be ordered. CDAs can be used to generate test suites for locating and detecting interaction faults between neighboring factors. We establish a general criterion for measuring the optimality of CDAs in terms of their size. Based on this optimality criterion, the equivalence between optimum CDAs and consecutive orthogonal arrays with prescribed properties is explored. Using the advantages of this equivalence, a great number of optimum CDAs are presented. In particular, the existence of optimum CDAs with few factors is almost completely determined. △ Less

Submitted 25 January, 2024; v1 submitted 26 May, 2019; originally announced May 2019.

MSC Class: 05B15; 05B20; 62K15; 94C12

Journal ref: Graphs and Combinatorics 36, 1203-1218 (2020)

Numerical simulation of a coupled system of Maxwell equations and a gas dynamic model

Authors: Maohui Lyu, Weng Cho Chew, Lijun Jiang, Maojun Li, Liwei Xu

Abstract: It is known that both linear and nonlinear optical phenomena can be produced when the plasmon in metallic nanostructures are excited by the external electromagnetic waves. In this work, a coupled system of Maxwell equations and a gas dynamic model including a quantum pressure term is employed to simulate the plasmon dynamics of free electron fluid in different metallic nanostructures using a disco… ▽ More It is known that both linear and nonlinear optical phenomena can be produced when the plasmon in metallic nanostructures are excited by the external electromagnetic waves. In this work, a coupled system of Maxwell equations and a gas dynamic model including a quantum pressure term is employed to simulate the plasmon dynamics of free electron fluid in different metallic nanostructures using a discontinuous Galerkin method in two dimensions. Numerical benchmarks demonstrate that the proposed numerical method can simulate both the high order harmonic generation and the nonlocal effect from metallic nanostructures. Based on the switch-on-and-off investigation, we can conclude that the quantum pressure term in gas dynamics is responsible for the bulk plasmon resonance. In addition, for the dielectric-filled nano-cavity, a coupled effective polarization model is further adopted to investigate the optical behavior of bound electrons. Concerning the numerical setting in this work, a strengthened influence of bound electrons on the generation of high order harmonic waves has been observed. △ Less

Submitted 17 June, 2019; v1 submitted 26 February, 2019; originally announced February 2019.

Comments: 17 pages, 11 figures

Ensemble-based implicit sampling for Bayesian inverse problems with non-Gaussian priors

Authors: Yuming Ba, Lijian Jiang

Abstract: In the paper, we develop an ensemble-based implicit sampling method for Bayesian inverse problems. For Bayesian inference, the iterative ensemble smoother (IES) and implicit sampling are integrated to obtain importance ensemble samples, which build an importance density. The proposed method shares a similar idea to importance sampling. IES is used to approximate mean and covariance of a posterior… ▽ More In the paper, we develop an ensemble-based implicit sampling method for Bayesian inverse problems. For Bayesian inference, the iterative ensemble smoother (IES) and implicit sampling are integrated to obtain importance ensemble samples, which build an importance density. The proposed method shares a similar idea to importance sampling. IES is used to approximate mean and covariance of a posterior distribution. This provides the MAP point and the inverse of Hessian matrix, which are necessary to construct the implicit map in implicit sampling. The importance samples are generated by the implicit map and the corresponding weights are the ratio between the importance density and posterior density. In the proposed method, we use the ensemble samples of IES to find the optimization solution of likelihood function and the inverse of Hessian matrix. This approach avoids the explicit computation for Jacobian matrix and Hessian matrix, which are very computationally expensive in high dimension spaces. To treat non-Gaussian models, discrete cosine transform and Gaussian mixture model are used to characterize the non-Gaussian priors. The ensemble-based implicit sampling method is extended to the non-Gaussian priors for exploring the posterior of unknowns in inverse problems. The proposed method is used for each individual Gaussian model in the Gaussian mixture model. The proposed approach substantially improves the applicability of implicit sampling method. A few numerical examples are presented to demonstrate the efficacy of the proposed method with applications of inverse problems for subsurface flow problems and anomalous diffusion models in porous media. △ Less

Submitted 2 December, 2018; originally announced December 2018.

Comments: 27 pages

An improved implicit sampling for Bayesian inverse problems of multi-term time fractional multiscale diffusion models

Authors: Xiaoyan Song, Lijian Jiang, Guanghui Zheng

Abstract: This paper presents an improved implicit sampling method for hierarchical Bayesian inverse problems. A widely used approach for sampling posterior distribution is based on Markov chain Monte Carlo (MCMC). However, the samples generated by MCMC are usually strongly correlated. This may lead to a small size of effective samples from a long Markov chain and the resultant posterior estimate may be ina… ▽ More This paper presents an improved implicit sampling method for hierarchical Bayesian inverse problems. A widely used approach for sampling posterior distribution is based on Markov chain Monte Carlo (MCMC). However, the samples generated by MCMC are usually strongly correlated. This may lead to a small size of effective samples from a long Markov chain and the resultant posterior estimate may be inaccurate. An implicit sampling method proposed in [11] can generate independent samples and capture some inherent non-Gaussian features of the posterior based on the weights of samples. In the implicit sampling method, the posterior samples are generated by constructing a map and distribute around the MAP point. However, the weights of implicit sampling in previous works may cause excessive concentration of samples and lead to ensemble collapse. To overcome this issue, we propose a new weight formulation and make resampling based on the new weights. In practice, some parameters in prior density are often unknown and a hierarchical Bayesian inference is necessary for posterior exploration. To this end, the hierarchical Bayesian formulation is used to estimate the MAP point and integrated in the implicit sampling framework. Compared to conventional implicit sampling, the proposed implicit sampling method can significantly improve the posterior estimator and the applicability for high dimensional inverse problems. The improved implicit sampling method is applied to the Bayesian inverse problems of multi-term time fractional diffusion models in heterogeneous media. To effectively capture the heterogeneity effect, we present a mixed generalized multiscale finite element method (mixed GMsFEM) to solve the time fractional diffusion models in a coarse grid, which can substantially speed up the Bayesian inversion. △ Less

Submitted 26 November, 2018; originally announced November 2018.

Comments: 32 pages

Bayesian identification of discontinuous fields with an ensemble-based variable separation multiscale method

Authors: Na Ou, Guang Lin, Lijian Jiang

Abstract: This work presents a multiscale model reduction approach to discontinuous fields identification problems in the framework of Bayesian inference. An ensemble-based variable separation (VS) method is proposed to approximate multiscale basis functions used to build a coarse model. The variable-separation expression is constructed for stochastic multiscale basis functions based on the random field, wh… ▽ More This work presents a multiscale model reduction approach to discontinuous fields identification problems in the framework of Bayesian inference. An ensemble-based variable separation (VS) method is proposed to approximate multiscale basis functions used to build a coarse model. The variable-separation expression is constructed for stochastic multiscale basis functions based on the random field, which is treated Gauss process as prior information. To this end, multiple local inhomogeneous Dirichlet boundary condition problems are required to be solved, and the ensemble-based method is used to obtain variable separation forms for the corresponding local functions. The local functions share the same interpolate rule for different physical basis functions in each coarse block. This approach significantly improves the efficiency of computation. We obtain the variable separation expression of multiscale basis functions, which can be used to the models with different boundary conditions and source terms, once the expression constructed. The proposed method is applied to discontinuous field identification problems where the hybrid of total variation and Gaussian (TG) densities are imposed as the penalty. We give a convergence analysis of the approximate posterior to the reference one with respect to the Kullback-Leibler (KL) divergence under the hybrid prior. The proposed method is applied to identify discontinuous structures in permeability fields. Two patterns of discontinuous structures are considered in numerical examples: separated blocks and nested blocks. △ Less

Submitted 30 May, 2019; v1 submitted 21 September, 2018; originally announced September 2018.

MSC Class: 65N99; 65N30; 35R60

On the translates of general dyadic systems on $\mathbb{R}$

Authors: Theresa C. Anderson, Bingyang Hu, Liwei Jiang, Connor Olson, Zeyu Wei

Abstract: Many techniques in harmonic analysis use the fact that a continuous object can be written as a sum (or an intersection) of dyadic counterparts, as long as those counterparts belong to an adjacent dyadic system. Here we generalize the notion of adjacent dyadic system and explore when it occurs, leading to some new and perhaps surprising classifications. In particular, we show that every dyadic grid… ▽ More Many techniques in harmonic analysis use the fact that a continuous object can be written as a sum (or an intersection) of dyadic counterparts, as long as those counterparts belong to an adjacent dyadic system. Here we generalize the notion of adjacent dyadic system and explore when it occurs, leading to some new and perhaps surprising classifications. In particular, we show that every dyadic grid is determined by two parameters, the \emph{shift} and the \emph{location}; moreover two dyadic grids form an adjacent dyadic system if and only if their shifts and locations satisfy readily verifiable conditions. △ Less

Submitted 19 December, 2019; v1 submitted 4 September, 2018; originally announced September 2018.

Comments: Final revised version to appear in Mathematische Annalen

A Constraint energy minimizing generalized multiscale finite element method for parabolic equations

Authors: Mengnan Li, Eric Chung, Lijian Jiang

Abstract: In this paper, we present a Constraint Energy Minimizing Generalized Multiscale Finite Element Method (CEM-GMsFEM) for parabolic equations with multiscale coefficients, arising from applications in porous media. We will present the construction of CEM-GMsFEM and rigorously analyze its convergence for the parabolic equations. The convergence rate is characterized by the coarse grid size and the eig… ▽ More In this paper, we present a Constraint Energy Minimizing Generalized Multiscale Finite Element Method (CEM-GMsFEM) for parabolic equations with multiscale coefficients, arising from applications in porous media. We will present the construction of CEM-GMsFEM and rigorously analyze its convergence for the parabolic equations. The convergence rate is characterized by the coarse grid size and the eigenvalue decay of local spectral problems, but is independent of the scale length and contrast of the media. The analysis shows that the method has a first order convergence rate with respect to coarse grid size in the energy norm and second order convergence rate with respect to coarse grid size in $L^2$ norm under some appropriate assumptions. For the temporal discretization, finite difference techniques are used and the convergence analysis of full discrete scheme is given. Moreover, a posteriori error estimator is derived and analyzed. A few numerical results for porous media applications are presented to confirm the theoretical findings and demonstrate the performance of the approach. △ Less

Submitted 12 June, 2018; originally announced June 2018.

MSC Class: 65N99; 65N30; 34E13

A two-stage ensemble Kalman filter based on multiscale model reduction for inverse problems in time fractional diffusion-wave equations

Authors: Yuming Ba, Lijian Jiang, Na Ou

Abstract: Ensemble Kalman filter (EnKF) has been widely used in state estimation and parameter estimation for the dynamic system where observational data is obtained sequentially in time. To reduce uncertainty and accelerate posterior inference, a two-stage ensemble Kalman filter is presented to improve the sequential analysis of EnKF. It is known that the final posterior ensemble may be concentrated in a… ▽ More Ensemble Kalman filter (EnKF) has been widely used in state estimation and parameter estimation for the dynamic system where observational data is obtained sequentially in time. To reduce uncertainty and accelerate posterior inference, a two-stage ensemble Kalman filter is presented to improve the sequential analysis of EnKF. It is known that the final posterior ensemble may be concentrated in a small portion of the entire support of the initial prior ensemble. It will be much more efficient if we first build a new prior by some partial observations, and construct a surrogate only over the significant region of the new prior. To this end, we construct a very coarse model using generalized multiscale finite element method (GMsFEM) and generate a new prior ensemble in the first stage. GMsFEM provides a set of hierarchical multiscale basis functions supported in coarse blocks. This gives flexibility and adaptivity to choosing degree of freedoms to construct a reduce model. In the second stage, we build an initial surrogate model based on the new prior by using GMsFEM and sparse generalized polynomial chaos (gPC)-based stochastic collocation methods. To improve the initial surrogate model, we dynamically update the surrogate model, which is adapted to the sequential availability of data and the updated analysis. The two-stage EnKF can achieve a better estimation than standard EnKF, and significantly improve the efficiency to update the ensemble analysis (posterior exploration). To enhance the applicability and flexibility in Bayesian inverse problems, we extend the two-stage EnKF to non-Gaussian models and hierarchical models. In the paper, we focus on the time fractional diffusion-wave models in porous media and investigate their Bayesian inverse problems using the proposed two-stage EnKF. △ Less

Submitted 27 February, 2018; originally announced February 2018.

Local-global model reduction method for stochastic optimal control problems constrained by partial differential equations

Authors: Lingling Ma, Qiuqi Li, Lijian Jiang

Abstract: In this paper, a local-global model reduction method is presented to solve stochastic optimal control problems governed by partial differential equations (PDEs). If the optimal control problems involve uncertainty, we need to use a few random variables to parameterize the uncertainty. The stochastic optimal control problems require solving coupled optimality system for a large number of samples in… ▽ More In this paper, a local-global model reduction method is presented to solve stochastic optimal control problems governed by partial differential equations (PDEs). If the optimal control problems involve uncertainty, we need to use a few random variables to parameterize the uncertainty. The stochastic optimal control problems require solving coupled optimality system for a large number of samples in the stochastic space to quantify the statistics of the system response and explore the uncertainty quantification. Thus the computation is prohibitively expensive. To overcome the difficulty, model reduction is necessary to significantly reduce the computation complexity. We exploit the advantages from both reduced basis method and Generalized Multiscale Finite Element Method (GMsFEM) and develop the local-global model reduction method for stochastic optimal control problems with PDE constraints. This local-global model reduction can achieve much more computation efficiency than using only local model reduction approach and only global model reduction approach. We recast the stochastic optimal problems in the framework of saddle-point problems and analyze the existence and uniqueness of the optimal solutions of the reduced model. In the local-global approach, most of computation steps are independent of each other. This is very desirable for scientific computation. Moreover, the online computation for each random sample is very fast via the proposed model reduction method. This allows us to compute the optimality system for a large number of samples. To demonstrate the performance of the local-global model reduction method, a few numerical examples are provided for different stochastic optimal control problems. △ Less

Submitted 18 December, 2017; originally announced December 2017.

Comments: 34 pages

Recovering the reaction coefficient for two dimensional time fractional diffusion equations

Authors: Xiaoyan Song, Guanghui Zheng, Lijian Jiang

Abstract: In this paper, we present an inverse problem of identifying the reaction coefficient for time fractional diffusion equations in two dimensional spaces by using boundary Neumann data. It is proved that the forward operator is continuous with respect to the unknown parameter. Because the inverse problem is often ill-posed, regularization strategies are imposed on the least fit-to-data functional to… ▽ More In this paper, we present an inverse problem of identifying the reaction coefficient for time fractional diffusion equations in two dimensional spaces by using boundary Neumann data. It is proved that the forward operator is continuous with respect to the unknown parameter. Because the inverse problem is often ill-posed, regularization strategies are imposed on the least fit-to-data functional to overcome the stability issue. There may exist various kinds of functions to reconstruct. It is crucial to choose a suitable regularization method. We present a multi-parameter regularization $L^{2}+BV$ method for the inverse problem. This can extend the applicability for reconstructing the unknown functions. Rigorous analysis is carried out for the inverse problem. In particular, we analyze the existence and stability of regularized variational problem and the convergence. To reduce the dimension in the inversion for numerical simulation, the unknown coefficient is represented by a suitable set of basis functions based on a priori information. A few numerical examples are presented for the inverse problem in time fractional diffusion equations to confirm the theoretic analysis and the efficacy of the different regularization methods. △ Less

Submitted 12 June, 2018; v1 submitted 3 July, 2017; originally announced July 2017.

Comments: 23 pages

Bayesian inference using intermediate distribution based on coarse multiscale model for time fractional diffusion equation

Authors: Lijian Jiang, Na Ou

Abstract: In the paper, we present a strategy for accelerating posterior inference for unknown inputs in time fractional diffusion models. In many inference problems, the posterior may be concentrated in a small portion of the entire prior support. It will be much more efficient if we build and simulate a surrogate only over the significant region of the posterior. To this end, we construct a coarse model u… ▽ More In the paper, we present a strategy for accelerating posterior inference for unknown inputs in time fractional diffusion models. In many inference problems, the posterior may be concentrated in a small portion of the entire prior support. It will be much more efficient if we build and simulate a surrogate only over the significant region of the posterior. To this end, we construct a coarse model using Generalized Multiscale Finite Element Method (GMsFEM), and solve a least-squares problem for the coarse model with a regularizing Levenberg-Marquart algorithm. An intermediate distribution is built based on the approximate sampling distribution. For Bayesian inference, we use GMsFEM and least-squares stochastic collocation method to obtain a reduced coarse model based on the intermediate distribution. To increase the sampling speed of Markov chain Monte Carlo, the DREAM$_\text{ZS}$ algorithm is used to explore the surrogate posterior density, which is based on the surrogate likelihood and the intermediate distribution. The proposed method with lower gPC order gives the approximate posterior as accurate as the the surrogate model directly based on the original prior. A few numerical examples for time fractional diffusion equations are carried out to demonstrate the performance of the proposed method with applications of the Bayesian inversion. △ Less

Submitted 14 June, 2017; originally announced June 2017.

Comments: 30 pages. arXiv admin note: text overlap with arXiv:1604.00138

Distributionally Robust Chance-Constrained Approximate AC-OPF with Wasserstein Metric

Authors: Chao Duan, Wanliang Fang, Lin Jiang, Li Yao, Jun Liu

Abstract: Chance constrained optimal power flow (OPF) has been recognized as a promising framework to manage the risk from variable renewable energy (VRE). In presence of VRE uncertainties, this paper discusses a distributionally robust chance constrained approximate AC-OPF. The power flow model employed in the proposed OPF formulation combines an exact AC power flow model at the nominal operation point and… ▽ More Chance constrained optimal power flow (OPF) has been recognized as a promising framework to manage the risk from variable renewable energy (VRE). In presence of VRE uncertainties, this paper discusses a distributionally robust chance constrained approximate AC-OPF. The power flow model employed in the proposed OPF formulation combines an exact AC power flow model at the nominal operation point and an approximate linear power flow model to reflect the system response under uncertainties. The ambiguity set employed in the distributionally robust formulation is the Wasserstein ball centered at the empirical distribution. The proposed OPF model minimizes the expectation of the quadratic cost function w.r.t. the worst-case probability distribution and guarantees the chance constraints satisfied for any distribution in the ambiguity set. The whole method is data-driven in the sense that the ambiguity set is constructed from historical data without any presumption on the type of the probability distribution, and more data leads to smaller ambiguity set and less conservative strategy. Moreover, special problem structures of the proposed problem formulation are exploited to develop an efficient and scalable solution approach. Case studies are carried out on IEEE 14 and 118 bus systems to show the accuracy and necessity of the approximate AC model and the attractive features of the distributionally robust optimization approach compared with other methods to deal with uncertainties. △ Less

Submitted 28 April, 2018; v1 submitted 17 June, 2017; originally announced June 2017.

On the connective eccentricity index of two types of trees

Authors: Zikai Tang, Lingyao Jiang, Hanyuan Deng

Abstract: The connective eccentricity index $ξ^{ce}=\sum^{}_{u\in V}\frac{d(u)}{\varepsilon(u)}$, where $\varepsilon(u)$ and $d(u)$ denote the eccentricity and the degree of the vertex $u$, respectively. In this paper, we first determine the extremal trees which minimize and maximize the connective eccentricity index among all trees with a given degree sequence, and then determine the extremal trees which… ▽ More The connective eccentricity index $ξ^{ce}=\sum^{}_{u\in V}\frac{d(u)}{\varepsilon(u)}$, where $\varepsilon(u)$ and $d(u)$ denote the eccentricity and the degree of the vertex $u$, respectively. In this paper, we first determine the extremal trees which minimize and maximize the connective eccentricity index among all trees with a given degree sequence, and then determine the extremal trees which minimize and maximize the connective eccentricity index among all trees with a given number of branching vertices. △ Less

Submitted 16 February, 2017; v1 submitted 15 February, 2017; originally announced February 2017.

Comments: arXiv admin note: text overlap with arXiv:1408.5865 by other authors

A novel variable-separation method based on sparse representation for stochastic partial differential equations

Authors: Qiuqi Li, Lijian Jiang

Abstract: In this paper, we propose a novel variable-separation (NVS) method for generic multivariate functions. The idea of NVS is extended to to obtain the solution in tensor product structure for stochastic partial differential equations (SPDEs). Compared with many widely used variation-separation methods, NVS shares their merits but has less computation complexity and better efficiency. NVS can be u… ▽ More In this paper, we propose a novel variable-separation (NVS) method for generic multivariate functions. The idea of NVS is extended to to obtain the solution in tensor product structure for stochastic partial differential equations (SPDEs). Compared with many widely used variation-separation methods, NVS shares their merits but has less computation complexity and better efficiency. NVS can be used to get the separated representation of the solution for SPDE in a systematic enrichment manner. No iteration is performed at each enrichment step. This is a significant improvement compared with proper generalized decomposition. Because the stochastic functions of the separated representations obtained by NVS depend on the previous terms, this impacts on the computation efficiency and brings great challenge for numerical simulation for the problems in high stochastic dimensional spaces. In order to overcome the difficulty, we propose an improved least angle regression algorithm (ILARS) and a hierarchical sparse low rank tensor approximation (HSLRTA) method based on sparse regularization. For ILARS, we explicitly give the selection of the optimal regularization parameters at each step based on least angle regression algorithm (LARS) for lasso problems such that ILARS is much more efficient. HSLRTA hierarchically decomposes a high dimensional problem into some low dimensional problems and brings an accurate approximation for the solution to SPDEs in high dimensional stochastic spaces using limited computer resource. A few numerical examples are presented to illustrate the efficacy of the proposed methods. △ Less

Submitted 13 November, 2016; originally announced November 2016.

Comments: 32 pages

Kadec-Klee property for convergence in measure of noncommutative Orlicz spaces

Authors: Zhenhua Ma, Lining Jiang, Kai Ji

Abstract: In this paper, we study the Kadec-Klee property for convergence in measure of noncommutative Orlicz spaces $L_{\varphi}(\widetilde{\mathcal{M}},τ)$, where $\widetilde{\mathcal{M}}$ is a von Neumann algebra, and $\varphi$ is an Orlicz function. We show that if $\varphi\inΔ_{2}$, $L_{\varphi}(\widetilde{\mathcal{M}},τ)$ has the Kadec-Klee property in measure. As a corollary, the dual space and refle… ▽ More In this paper, we study the Kadec-Klee property for convergence in measure of noncommutative Orlicz spaces $L_{\varphi}(\widetilde{\mathcal{M}},τ)$, where $\widetilde{\mathcal{M}}$ is a von Neumann algebra, and $\varphi$ is an Orlicz function. We show that if $\varphi\inΔ_{2}$, $L_{\varphi}(\widetilde{\mathcal{M}},τ)$ has the Kadec-Klee property in measure. As a corollary, the dual space and reflexivity of $L_{\varphi}(\widetilde{\mathcal{M}},τ)$ are given. △ Less

Submitted 13 October, 2016; originally announced October 2016.

Comments: 11