Showing 1–50 of 65 results for author: Ren, Z

Boosting e-BH via conditional calibration

Authors: Junu Lee, Zhimei Ren

Abstract: The e-BH procedure is an e-value-based multiple testing procedure that provably controls the false discovery rate (FDR) under any dependence structure between the e-values. Despite this appealing theoretical FDR control guarantee, the e-BH procedure often suffers from low power in practice. In this paper, we propose a general framework that boosts the power of e-BH without sacrificing its FDR cont… ▽ More The e-BH procedure is an e-value-based multiple testing procedure that provably controls the false discovery rate (FDR) under any dependence structure between the e-values. Despite this appealing theoretical FDR control guarantee, the e-BH procedure often suffers from low power in practice. In this paper, we propose a general framework that boosts the power of e-BH without sacrificing its FDR control under arbitrary dependence. This is achieved by the technique of conditional calibration, where we take as input the e-values and calibrate them to be a set of "boosted e-values" that are guaranteed to be no less -- and are often more -- powerful than the original ones. Our general framework is explicitly instantiated in three classes of multiple testing problems: (1) testing under parametric models, (2) conditional independence testing under the model-X setting, and (3) model-free conformalized selection. Extensive numerical experiments show that our proposed method significantly improves the power of e-BH while continuing to control the FDR. We also demonstrate the effectiveness of our method through an application to an observational study dataset for identifying individuals whose counterfactuals satisfy certain properties. △ Less

Submitted 26 April, 2024; originally announced April 2024.

TS-RSR: A provably efficient approach for batch bayesian optimization

Authors: Zhaolin Ren, Na Li

Abstract: This paper presents a new approach for batch Bayesian Optimization (BO) called Thompson Sampling-Regret to Sigma Ratio directed sampling (TS-RSR), where we sample a new batch of actions by minimizing a Thompson Sampling approximation of a regret to uncertainty ratio. Our sampling objective is able to coordinate the actions chosen in each batch in a way that minimizes redundancy between points whil… ▽ More This paper presents a new approach for batch Bayesian Optimization (BO) called Thompson Sampling-Regret to Sigma Ratio directed sampling (TS-RSR), where we sample a new batch of actions by minimizing a Thompson Sampling approximation of a regret to uncertainty ratio. Our sampling objective is able to coordinate the actions chosen in each batch in a way that minimizes redundancy between points whilst focusing on points with high predictive means or high uncertainty. Theoretically, we provide rigorous convergence guarantees on our algorithm's regret, and numerically, we demonstrate that our method attains state-of-the-art performance on a range of challenging synthetic and realistic test functions, where it outperforms several competitive benchmark batch BO algorithms. △ Less

Submitted 2 May, 2024; v1 submitted 7 March, 2024; originally announced March 2024.

Comments: Revised presentation and organization of theoretical results

Confidence on the Focal: Conformal Prediction with Selection-Conditional Coverage

Authors: Ying **, Zhimei Ren

Abstract: Conformal prediction builds marginally valid prediction intervals that cover the unknown outcome of a randomly drawn new test point with a prescribed probability. However, a common scenario in practice is that, after seeing the data, practitioners decide which test unit(s) to focus on in a data-driven manner and seek for uncertainty quantification of the focal unit(s). In such cases, marginally va… ▽ More Conformal prediction builds marginally valid prediction intervals that cover the unknown outcome of a randomly drawn new test point with a prescribed probability. However, a common scenario in practice is that, after seeing the data, practitioners decide which test unit(s) to focus on in a data-driven manner and seek for uncertainty quantification of the focal unit(s). In such cases, marginally valid conformal prediction intervals may not provide valid coverage for the focal unit(s) due to selection bias. This paper presents a general framework for constructing a prediction set with finite-sample exact coverage conditional on the unit being selected by a given procedure. The general form of our method works for arbitrary selection rules that are invariant to the permutation of the calibration units, and generalizes Mondrian Conformal Prediction to multiple test units and non-equivariant classifiers. We then work out the computationally efficient implementation of our framework for a number of realistic selection rules, including top-K selection, optimization-based selection, selection based on conformal p-values, and selection based on properties of preliminary conformal prediction sets. The performance of our methods is demonstrated via applications in drug discovery and health risk prediction. △ Less

Submitted 24 March, 2024; v1 submitted 6 March, 2024; originally announced March 2024.

Time-uniform log-Sobolev inequalities and applications to propagation of chaos

Authors: Pierre Monmarché, Zhenjie Ren, Songbo Wang

Abstract: Time-uniform log-Sobolev inequalities (LSI) satisfied by solutions of semi-linear mean-field equations have recently appeared to be a key tool to obtain time-uniform propagation of chaos estimates. This work addresses the more general settings of time-inhomogeneous Fokker-Planck equations. Time-uniform LSI are obtained in two cases, either with the bounded-Lipschitz perturbation argument with resp… ▽ More Time-uniform log-Sobolev inequalities (LSI) satisfied by solutions of semi-linear mean-field equations have recently appeared to be a key tool to obtain time-uniform propagation of chaos estimates. This work addresses the more general settings of time-inhomogeneous Fokker-Planck equations. Time-uniform LSI are obtained in two cases, either with the bounded-Lipschitz perturbation argument with respect to a reference measure, or with a coupling approach at high temperature. These arguments are then applied to mean-field equations, where, on the one hand, sharp marginal propagation of chaos estimates are obtained in smooth cases and, on the other hand, time-uniform global propagation of chaos is shown in the case of vortex interactions with quadratic confinement potential on the whole space. In this second case, an important point is to establish global gradient and Hessian estimates, which is of independent interest. We prove these bounds in the more general situation of non-attractive logarithmic and Riesz singular interactions. △ Less

Submitted 30 January, 2024; v1 submitted 15 January, 2024; originally announced January 2024.

Comments: 40 pages; proof of Theorem 4.5 expanded and Remark 4.7 clarified

Self-interacting approximation to McKean-Vlasov long-time limit: a Markov chain Monte Carlo method

Authors: Kai Du, Zhenjie Ren, Florin Suciu, Songbo Wang

Abstract: For a certain class of McKean--Vlasov processes, we introduce proxy processes that substitute the mean-field interaction with self-interaction, employing a weighted occupation measure. Our study encompasses two key achievements. First, we demonstrate the ergodicity of the self-interacting dynamics, under broad conditions, by applying the reflection coupling method. Second, in scenarios where the d… ▽ More For a certain class of McKean--Vlasov processes, we introduce proxy processes that substitute the mean-field interaction with self-interaction, employing a weighted occupation measure. Our study encompasses two key achievements. First, we demonstrate the ergodicity of the self-interacting dynamics, under broad conditions, by applying the reflection coupling method. Second, in scenarios where the drifts are negative intrinsic gradients of convex mean-field potential functionals, we use entropy and functional inequalities to demonstrate that the stationary measures of the self-interacting processes approximate the invariant measures of the corresponding McKean--Vlasov processes. As an application, we show how to learn the optimal weights of a two-layer neural network by training a single neuron. △ Less

Submitted 14 January, 2024; v1 submitted 19 November, 2023; originally announced November 2023.

Comments: 42 pages, 1 figure; contains a minor correction

Distribution learning via neural differential equations: a nonparametric statistical perspective

Authors: Youssef Marzouk, Zhi Ren, Sven Wang, Jakob Zech

Abstract: Ordinary differential equations (ODEs), via their induced flow maps, provide a powerful framework to parameterize invertible transformations for the purpose of representing complex probability distributions. While such models have achieved enormous success in machine learning, particularly for generative modeling and density estimation, little is known about their statistical properties. This work… ▽ More Ordinary differential equations (ODEs), via their induced flow maps, provide a powerful framework to parameterize invertible transformations for the purpose of representing complex probability distributions. While such models have achieved enormous success in machine learning, particularly for generative modeling and density estimation, little is known about their statistical properties. This work establishes the first general nonparametric statistical convergence analysis for distribution learning via ODE models trained through likelihood maximization. We first prove a convergence theorem applicable to arbitrary velocity field classes $\mathcal{F}$ satisfying certain simple boundary constraints. This general result captures the trade-off between approximation error (`bias') and the complexity of the ODE model (`variance'). We show that the latter can be quantified via the $C^1$-metric entropy of the class $\mathcal F$. We then apply this general framework to the setting of $C^k$-smooth target densities, and establish nearly minimax-optimal convergence rates for two relevant velocity field classes $\mathcal F$: $C^k$ functions and neural networks. The latter is the practically important case of neural ODEs. Our proof techniques require a careful synthesis of (i) analytical stability results for ODEs, (ii) classical theory for sieved M-estimators, and (iii) recent results on approximation rates and metric entropies of neural network classes. The results also provide theoretical insight on how the choice of velocity field class, and the dependence of this choice on sample size $n$ (e.g., the scaling of width, depth, and sparsity of neural network classes), impacts statistical performance. △ Less

Submitted 2 September, 2023; originally announced September 2023.

Uniform-in-time propagation of chaos for kinetic mean field Langevin dynamics

Authors: Fan Chen, Yiqing Lin, Zhenjie Ren, Songbo Wang

Abstract: We study the kinetic mean field Langevin dynamics under the functional convexity assumption of the mean field energy functional. Using hypocoercivity, we first establish the exponential convergence of the mean field dynamics and then show the corresponding $N$-particle system converges exponentially in a rate uniform in $N$ modulo a small error. Finally we study the short-time regularization effec… ▽ More We study the kinetic mean field Langevin dynamics under the functional convexity assumption of the mean field energy functional. Using hypocoercivity, we first establish the exponential convergence of the mean field dynamics and then show the corresponding $N$-particle system converges exponentially in a rate uniform in $N$ modulo a small error. Finally we study the short-time regularization effects of the dynamics and prove its uniform-in-time propagation of chaos property in both the Wasserstein and entropic sense. Our results can be applied to the training of two-layer neural networks with momentum and we include the numerical experiments. △ Less

Submitted 8 February, 2024; v1 submitted 5 July, 2023; originally announced July 2023.

Comments: 54 pages, 5 figures, 1 table; errors in the previous version were corrected; accepted version

MSC Class: 60J60; 60K35 (Primary) 35B40; 35H10; 35Q83; 35Q84 (Secondary)

Journal ref: Electron.J.Probab. 29(17) (2024) 1-43

Stochastic Nonlinear Control via Finite-dimensional Spectral Dynamic Embedding

Authors: Tongzheng Ren, Zhaolin Ren, Haitong Ma, Na Li, Bo Dai

Abstract: This paper presents an approach, Spectral Dynamics Embedding Control (SDEC), to optimal control for nonlinear stochastic systems. This method leverages an infinite-dimensional feature to linearly represent the state-action value function and exploits finite-dimensional truncation approximation for practical implementation. To characterize the effectiveness of these finite dimensional approximation… ▽ More This paper presents an approach, Spectral Dynamics Embedding Control (SDEC), to optimal control for nonlinear stochastic systems. This method leverages an infinite-dimensional feature to linearly represent the state-action value function and exploits finite-dimensional truncation approximation for practical implementation. To characterize the effectiveness of these finite dimensional approximations, we provide an in-depth theoretical analysis to characterize the approximation error induced by the finite-dimension truncation and statistical error induced by finite-sample approximation in both policy evaluation and policy optimization. Our analysis includes two prominent kernel approximation methods: truncations onto random features and Nystrom features. We also empirically test the algorithm and compare the performance with Koopman-based, iLQR, and energy-based methods on a few benchmark problems. △ Less

Submitted 20 December, 2023; v1 submitted 8 April, 2023; originally announced April 2023.

Comments: Compared to v1, added analysis of Nystrom features, more streamlined proofs, and more extensive numerical studies; compared to v2, corrected a small error in ordering of author list

Simultaneous activity and attenuation estimation in TOF-PET with TV-constrained nonconvex optimization

Authors: Zhimei Ren, Emil Y. Sidky, Rina Foygel Barber, Chien-Min Kao, Xiaochuan Pan

Abstract: An alternating direction method of multipliers (ADMM) framework is developed for nonsmooth biconvex optimization for inverse problems in imaging. In particular, the simultaneous estimation of activity and attenuation (SAA) problem in time-of-flight positron emission tomography (TOF-PET) has such a structure when maximum likelihood estimation (MLE) is employed. The ADMM framework is applied to MLE… ▽ More An alternating direction method of multipliers (ADMM) framework is developed for nonsmooth biconvex optimization for inverse problems in imaging. In particular, the simultaneous estimation of activity and attenuation (SAA) problem in time-of-flight positron emission tomography (TOF-PET) has such a structure when maximum likelihood estimation (MLE) is employed. The ADMM framework is applied to MLE for SAA in TOF-PET, resulting in the ADMM-SAA algorithm. This algorithm is extended by imposing total variation (TV) constraints on both the activity and attenuation map, resulting in the ADMM-TVSAA algorithm. The performance of this algorithm is illustrated using the penalized maximum likelihood activity and attenuation estimation (P-MLAA) algorithm as a reference. Additional results on step-size tuning and on the use of unconstrained ADMM-SAA are presented in the previous arXiv submission: arXiv:2303.17042v1. △ Less

Submitted 9 February, 2024; v1 submitted 29 March, 2023; originally announced March 2023.

Comments: Manuscript accepted at IEEE transactions on medical imaging. This version contains the appendix for the ADMM-TVSAA pseudocode

Mean Field Optimization Problem Regularized by Fisher Information

Authors: Julien Claisse, Giovanni Conforti, Zhenjie Ren, Songbo Wang

Abstract: Recently there is a rising interest in the research of mean field optimization, in particular because of its role in analyzing the training of neural networks. In this paper by adding the Fisher Information as the regularizer, we relate the regularized mean field optimization problem to a so-called mean field Schrodinger dynamics. We develop an energy-dissipation method to show that the marginal d… ▽ More Recently there is a rising interest in the research of mean field optimization, in particular because of its role in analyzing the training of neural networks. In this paper by adding the Fisher Information as the regularizer, we relate the regularized mean field optimization problem to a so-called mean field Schrodinger dynamics. We develop an energy-dissipation method to show that the marginal distributions of the mean field Schrodinger dynamics converge exponentially quickly towards the unique minimizer of the regularized optimization problem. Remarkably, the mean field Schrodinger dynamics is proved to be a gradient flow on the probability measure space with respect to the relative entropy. Finally we propose a Monte Carlo method to sample the marginal distributions of the mean field Schrodinger dynamics. △ Less

Submitted 22 July, 2023; v1 submitted 12 February, 2023; originally announced February 2023.

Policy learning "without'' overlap: Pessimism and generalized empirical Bernstein's inequality

Authors: Ying **, Zhimei Ren, Zhuoran Yang, Zhaoran Wang

Abstract: This paper studies offline policy learning, which aims at utilizing observations collected a priori (from either fixed or adaptively evolving behavior policies) to learn the optimal individualized decision rule in a given class. Existing policy learning methods rely on a uniform overlap assumption, i.e., the propensities of exploring all actions for all individual characteristics are lower bounded… ▽ More This paper studies offline policy learning, which aims at utilizing observations collected a priori (from either fixed or adaptively evolving behavior policies) to learn the optimal individualized decision rule in a given class. Existing policy learning methods rely on a uniform overlap assumption, i.e., the propensities of exploring all actions for all individual characteristics are lower bounded in the offline dataset. In other words, the performance of these methods depends on the worst-case propensity in the offline dataset. As one has no control over the data collection process, this assumption can be unrealistic in many situations, especially when the behavior policies are allowed to evolve over time with diminishing propensities. In this paper, we propose a new algorithm that optimizes lower confidence bounds (LCBs) -- instead of point estimates -- of the policy values. The LCBs are constructed by quantifying the estimation uncertainty of the augmented inverse propensity weighted (AIPW)-type estimators using knowledge of the behavior policies for collecting the offline data. Without assuming any uniform overlap condition, we establish a data-dependent upper bound for the suboptimality of our algorithm, which depends only on (i) the overlap for the optimal policy, and (ii) the complexity of the policy class. As an implication, for adaptively collected data, we ensure efficient policy learning as long as the propensities for optimal actions are lower bounded over time, while those for suboptimal ones are allowed to diminish arbitrarily fast. In our theoretical analysis, we develop a new self-normalized concentration inequality for IPW estimators, generalizing the well-known empirical Bernstein's inequality to unbounded and non-i.i.d. data. △ Less

Submitted 14 March, 2023; v1 submitted 19 December, 2022; originally announced December 2022.

Enhanced Multi-Objective A* with Partial Expansion

Authors: Valmiki Kothare, Zhongqiang Ren, Sivakumar Rathinam, Howie Choset

Abstract: The Multi-Objective Shortest Path Problem (MO-SPP), typically posed on a graph, determines a set of paths from a start vertex to a destination vertex while optimizing multiple objectives. In general, there does not exist a single solution path that can simultaneously optimize all the objectives and the problem thus seeks to find a set of so-called Pareto-optimal solutions. To address this problem,… ▽ More The Multi-Objective Shortest Path Problem (MO-SPP), typically posed on a graph, determines a set of paths from a start vertex to a destination vertex while optimizing multiple objectives. In general, there does not exist a single solution path that can simultaneously optimize all the objectives and the problem thus seeks to find a set of so-called Pareto-optimal solutions. To address this problem, several Multi-Objective A* (MOA*) algorithms were recently developed to quickly compute solutions with quality guarantees. However, these MOA* algorithms often suffer from high memory usage, especially when the branching factor (i.e. the number of neighbors of any vertex) of the graph is large. This work thus aims at reducing the high memory consumption of MOA* with little increase in the runtime. By generalizing and unifying several single- and multi-objective search algorithms, we develop the Runtime and Memory Efficient MOA* (RME-MOA*) approach, which can balance between runtime and memory efficiency by tuning two user-defined hyper-parameters. △ Less

Submitted 8 July, 2023; v1 submitted 6 December, 2022; originally announced December 2022.

Comments: 8 pages, 4 figures

ACM Class: I.2.8

Uniform-in-time propagation of chaos for mean field Langevin dynamics

Authors: Fan Chen, Zhenjie Ren, Songbo Wang

Abstract: We study the mean field Langevin dynamics and the associated particle system. By assuming the functional convexity of the energy, we obtain the $L^p$-convergence of the marginal distributions towards the unique invariant measure for the mean field dynamics. Furthermore, we prove the uniform-in-time propagation of chaos in both the $L^2$-Wasserstein metric and relative entropy. We study the mean field Langevin dynamics and the associated particle system. By assuming the functional convexity of the energy, we obtain the $L^p$-convergence of the marginal distributions towards the unique invariant measure for the mean field dynamics. Furthermore, we prove the uniform-in-time propagation of chaos in both the $L^2$-Wasserstein metric and relative entropy. △ Less

Submitted 20 November, 2023; v1 submitted 6 December, 2022; originally announced December 2022.

Comments: 66 pages, 3 figures and 1 table. Contains corrections and enhancements to arXiv:2212.03050v2

MSC Class: 60J60; 60K35 (Primary) 35B40; 35Q83; 35Q84 (Secondary)

On Controller Reduction in Linear Quadratic Gaussian Control with Performance Bounds

Authors: Zhaolin Ren, Yang Zheng, Maryam Fazel, Na Li

Abstract: The problem of controller reduction has a rich history in control theory. Yet, many questions remain open. In particular, there exist very few results on the order reduction of general non-observer based controllers and the subsequent quantification of the closed-loop performance. Recent developments in model-free policy optimization for Linear Quadratic Gaussian (LQG) control have highlighted the… ▽ More The problem of controller reduction has a rich history in control theory. Yet, many questions remain open. In particular, there exist very few results on the order reduction of general non-observer based controllers and the subsequent quantification of the closed-loop performance. Recent developments in model-free policy optimization for Linear Quadratic Gaussian (LQG) control have highlighted the importance of this question. In this paper, we first propose a new set of sufficient conditions ensuring that a perturbed controller remains internally stabilizing. Based on this result, we illustrate how to perform order reduction of general non-observer based controllers using balanced truncation and modal truncation. We also provide explicit bounds on the LQG performance of the reduced-order controller. Furthermore, for single-input-single-output (SISO) systems, we introduce a new controller reduction technique by truncating unstable modes. We illustrate our theoretical results with numerical simulations. Our results will serve as valuable tools to design direct policy search algorithms for control problems with partial observations. △ Less

Submitted 29 November, 2022; originally announced November 2022.

Esca** saddle points in zeroth-order optimization: the power of two-point estimators

Authors: Zhaolin Ren, Yujie Tang, Na Li

Abstract: Two-point zeroth order methods are important in many applications of zeroth-order optimization, such as robotics, wind farms, power systems, online optimization, and adversarial robustness to black-box attacks in deep neural networks, where the problem may be high-dimensional and/or time-varying. Most problems in these applications are nonconvex and contain saddle points. While existing works have… ▽ More Two-point zeroth order methods are important in many applications of zeroth-order optimization, such as robotics, wind farms, power systems, online optimization, and adversarial robustness to black-box attacks in deep neural networks, where the problem may be high-dimensional and/or time-varying. Most problems in these applications are nonconvex and contain saddle points. While existing works have shown that zeroth-order methods utilizing $Ω(d)$ function valuations per iteration (with $d$ denoting the problem dimension) can escape saddle points efficiently, it remains an open question if zeroth-order methods based on two-point estimators can escape saddle points. In this paper, we show that by adding an appropriate isotropic perturbation at each iteration, a zeroth-order algorithm based on $2m$ (for any $1 \leq m \leq d$) function evaluations per iteration can not only find $ε$-second order stationary points polynomially fast, but do so using only $\tilde{O}\left(\frac{d}{mε^{2}\barψ}\right)$ function evaluations, where $\barψ \geq \tildeΩ\left(\sqrtε\right)$ is a parameter capturing the extent to which the function of interest exhibits the strict saddle property. △ Less

Submitted 8 May, 2023; v1 submitted 27 September, 2022; originally announced September 2022.

Comments: To appear at ICML 2023

Fast Bayesian Optimization of Needle-in-a-Haystack Problems using Zooming Memory-Based Initialization (ZoMBI)

Authors: Alexander E. Siemenn, Zekun Ren, Qianxiao Li, Tonio Buonassisi

Abstract: Needle-in-a-Haystack problems exist across a wide range of applications including rare disease prediction, ecological resource management, fraud detection, and material property optimization. A Needle-in-a-Haystack problem arises when there is an extreme imbalance of optimum conditions relative to the size of the dataset. For example, only $0.82\%$ out of $146$k total materials in the open-access… ▽ More Needle-in-a-Haystack problems exist across a wide range of applications including rare disease prediction, ecological resource management, fraud detection, and material property optimization. A Needle-in-a-Haystack problem arises when there is an extreme imbalance of optimum conditions relative to the size of the dataset. For example, only $0.82\%$ out of $146$k total materials in the open-access Materials Project database have a negative Poisson's ratio. However, current state-of-the-art optimization algorithms are not designed with the capabilities to find solutions to these challenging multidimensional Needle-in-a-Haystack problems, resulting in slow convergence to a global optimum or pigeonholing into a local minimum. In this paper, we present a Zooming Memory-Based Initialization algorithm, entitled ZoMBI. ZoMBI actively extracts knowledge from the previously best-performing evaluated experiments to iteratively zoom in the sampling search bounds towards the global optimum "needle" and then prunes the memory of low-performing historical experiments to accelerate compute times by reducing the algorithm time complexity from $O(n^3)$ to $O(φ^3)$ for $φ$ forward experiments per activation, which trends to a constant $O(1)$ over several activations. Additionally, ZoMBI implements two custom adaptive acquisition functions to further guide the sampling of new experiments toward the global optimum. We validate the algorithm's optimization performance on three real-world datasets exhibiting Needle-in-a-Haystack and further stress-test the algorithm's performance on an additional 174 analytical datasets. The ZoMBI algorithm demonstrates compute time speed-ups of 400x compared to traditional Bayesian optimization as well as efficiently discovering optima in under 100 experiments that are up to 3x more highly optimized than those discovered by similar methods MiP-EGO, TuRBO, and HEBO. △ Less

Submitted 2 February, 2023; v1 submitted 26 August, 2022; originally announced August 2022.

Comments: Paper 16 pages; SI 6 pages

Journal ref: npj Comput Mater 9, 79 (2023)

Penetration trajectory optimization for the hypersonic gliding vehicle encountering two interceptors

Authors: Zhipeng Shen, Jianglong Yu, Xiwang Dong, Yongzhao Hua, Zhang Ren

Abstract: The penetration trajectory optimization problem for the hypersonic gliding vehicle (HGV) encountering two interceptors is investigated. The HGV penetration trajectory optimization problem considering the terminal target area is formulated as a nonconvex optimal control problem. The nonconvex optimal control problem is transformed into a second-order cone programming (SOCP) problem, which can be so… ▽ More The penetration trajectory optimization problem for the hypersonic gliding vehicle (HGV) encountering two interceptors is investigated. The HGV penetration trajectory optimization problem considering the terminal target area is formulated as a nonconvex optimal control problem. The nonconvex optimal control problem is transformed into a second-order cone programming (SOCP) problem, which can be solved by state-of-the-art interior-point methods. In addition, a penetration strategy that only requires the initial line-of-sight angle information of the interceptors is proposed. The convergent trajectory obtained by the proposed method allows the HGV to evade two interceptors and reach the target area successfully. Furthermore, a successive SOCP method with a variable trust region is presented, which is critical to balancing the trade-off between time consumption and optimality. Finally, the effectiveness and performance of the proposed method are verified by numerical simulations. △ Less

Submitted 17 April, 2022; originally announced April 2022.

Comments: 34 pages, 18 figures

Journal ref: Aerospace Science and Technology, 2022, 121, 107363

Entropic optimal planning for path-dependent mean field games

Authors: Zhenjie Ren, Xiaolu Tan, Nizar Touzi, Junjian Yang

Abstract: In the context of mean field games, with possible control of the diffusion coefficient, we consider a path-dependent version of the planning problem introduced by P.L. Lions: given a pair of marginal distributions $(μ_0, μ_1)$, find a specification of the game problem starting from the initial distribution $μ_0$, and inducing the target distribution $μ_1$ at the mean field game equilibrium. Our ma… ▽ More In the context of mean field games, with possible control of the diffusion coefficient, we consider a path-dependent version of the planning problem introduced by P.L. Lions: given a pair of marginal distributions $(μ_0, μ_1)$, find a specification of the game problem starting from the initial distribution $μ_0$, and inducing the target distribution $μ_1$ at the mean field game equilibrium. Our main result reduces the path-dependent planning problem into an embedding problem, that is, constructing a McKean-Vlasov dynamics with given marginals $(μ_0,μ_1)$. Some sufficient conditions on $(μ_0,μ_1)$ are provided to guarantee the existence of solutions. We also characterize, up to integrability, the minimum entropy solution of the planning problem. In particular, as uniqueness does not hold anymore in our path-dependent setting, one can naturally introduce an optimal planning problem which would be reduced to an optimal transport problem along with controlled McKean-Vlasov dynamics. △ Less

Submitted 18 May, 2023; v1 submitted 14 March, 2022; originally announced March 2022.

MSC Class: 49N70; 91A13; 91B40; 93E20

Entropic Fictitious Play for Mean Field Optimization Problem

Authors: Fan Chen, Zhenjie Ren, Songbo Wang

Abstract: We study two-layer neural networks in the mean field limit, where the number of neurons tends to infinity. In this regime, the optimization over the neuron parameters becomes the optimization over the probability measures, and by adding an entropic regularizer, the minimizer of the problem is identified as a fixed point. We propose a novel training algorithm named entropic fictitious play, inspire… ▽ More We study two-layer neural networks in the mean field limit, where the number of neurons tends to infinity. In this regime, the optimization over the neuron parameters becomes the optimization over the probability measures, and by adding an entropic regularizer, the minimizer of the problem is identified as a fixed point. We propose a novel training algorithm named entropic fictitious play, inspired by the classical fictitious play in game theory for learning Nash equilibriums, to recover this fixed point, and the algorithm exhibits a two-loop iteration structure. Exponential convergence is proved in this paper and we also verify our theoretical results by simple numerical examples. △ Less

Submitted 21 July, 2023; v1 submitted 11 February, 2022; originally announced February 2022.

Comments: 36 pages, 2 figures; version accepted by Journal of Machine Learning Research

Journal ref: J.Mach.Learn.Res. 24(211) (2023) 1-36

On path-dependent multidimensional forward-backward SDEs

Authors: Kaitong Hu, Zhenjie Ren, Nizar Touzi

Abstract: This paper extends the results of Ma, Wu, Zhang, Zhang [11] to the context of path-dependent multidimensional forward-backward stochastic differential equations (FBSDE). By path-dependent we mean that the coefficients of the forward-backward SDE at time t can depend on the whole path of the forward process up to time t. Such a situation appears when solving path-dependent stochastic control proble… ▽ More This paper extends the results of Ma, Wu, Zhang, Zhang [11] to the context of path-dependent multidimensional forward-backward stochastic differential equations (FBSDE). By path-dependent we mean that the coefficients of the forward-backward SDE at time t can depend on the whole path of the forward process up to time t. Such a situation appears when solving path-dependent stochastic control problems by means of variational calculus. At the heart of our analysis is the construction of a decoupling random field on the path space. We first prove the existence and the uniqueness of decoupling field on small time interval. Then by introducing the characteristic BSDE, we show that a global decoupling field can be constructed by patching local solutions together as long as the solution of the characteristic BSDE remains bounded. Finally, we provide a stability result for path-dependent forward-backward SDEs. △ Less

Submitted 11 January, 2022; originally announced January 2022.

MSC Class: 60H07; 60H30; 35R60; 34F05

Entropic turnpike estimates for the kinetic Schrödinger problem

Authors: Alberto Chiarini, Giovanni Conforti, Giacomo Greco, Zhenjie Ren

Abstract: We investigate the kinetic Schrödinger problem, obtained considering Langevin dynamics instead of Brownian motion in Schrödinger's thought experiment. Under a quasilinearity assumption we establish exponential entropic turnpike estimates for the corresponding Schrödinger bridges and exponentially fast convergence of the entropic cost to the sum of the marginal entropies in the long-time regime, wh… ▽ More We investigate the kinetic Schrödinger problem, obtained considering Langevin dynamics instead of Brownian motion in Schrödinger's thought experiment. Under a quasilinearity assumption we establish exponential entropic turnpike estimates for the corresponding Schrödinger bridges and exponentially fast convergence of the entropic cost to the sum of the marginal entropies in the long-time regime, which provides as a corollary an entropic Talagrand inequality. In order to do so, we profit from recent advances in the understanding of classical Schrödinger bridges and adaptations of Bakry-Émery formalism to the kinetic setting. Our quantitative results are complemented by basic structural results such as dual representation of the entropic cost and the existence of Schrödinger potentials. △ Less

Submitted 8 September, 2022; v1 submitted 20 August, 2021; originally announced August 2021.

Comments: 33 pages

MSC Class: 93E20; 47D07; 60E15

Gradient play in stochastic games: stationary points, convergence, and sample complexity

Authors: Runyu Zhang, Zhaolin Ren, Na Li

Abstract: We study the performance of the gradient play algorithm for stochastic games (SGs), where each agent tries to maximize its own total discounted reward by making decisions independently based on current state information which is shared between agents. Policies are directly parameterized by the probability of choosing a certain action at a given state. We show that Nash equilibria (NEs) and first-o… ▽ More We study the performance of the gradient play algorithm for stochastic games (SGs), where each agent tries to maximize its own total discounted reward by making decisions independently based on current state information which is shared between agents. Policies are directly parameterized by the probability of choosing a certain action at a given state. We show that Nash equilibria (NEs) and first-order stationary policies are equivalent in this setting, and give a local convergence rate around strict NEs. Further, for a subclass of SGs called Markov potential games (which includes the setting with identical rewards as an important special case), we design a sample-based reinforcement learning algorithm and give a non-asymptotic global convergence rate analysis for both exact gradient play and our sample-based learning algorithm. Our result shows that the number of iterations to reach an $ε$-NE scales linearly, instead of exponentially, with the number of agents. Local geometry and local stability are also considered, where we prove that strict NEs are local maxima of the total potential function and fully-mixed NEs are saddle points. △ Less

Submitted 6 December, 2023; v1 submitted 31 May, 2021; originally announced June 2021.

Zeroth-Order Feedback Optimization for Cooperative Multi-Agent Systems

Authors: Yujie Tang, Zhaolin Ren, Na Li

Abstract: We study a class of cooperative multi-agent optimization problems, where each agent is associated with a local action vector and a local cost, and the goal is to cooperatively find the joint action profile that minimizes the average of the local costs. Such problems arise in many applications, such as distributed routing control, wind farm operation, etc. In many of these problems, gradient inform… ▽ More We study a class of cooperative multi-agent optimization problems, where each agent is associated with a local action vector and a local cost, and the goal is to cooperatively find the joint action profile that minimizes the average of the local costs. Such problems arise in many applications, such as distributed routing control, wind farm operation, etc. In many of these problems, gradient information may not be readily available, and the agents may only observe their local costs incurred by their actions as a feedback to determine their new actions. In this paper, we propose a zeroth-order feedback optimization scheme for the class of problems we consider, and provide explicit complexity bounds for both the convex and nonconvex settings with noiseless and noisy local cost observations. We also discuss briefly on the impacts of knowledge of local function dependence between agents. The algorithm's performance is justified by a numerical example of distributed routing control. △ Less

Submitted 22 February, 2021; v1 submitted 19 November, 2020; originally announced November 2020.

LQR with Tracking: A Zeroth-order Approach and Its Global Convergence

Authors: Zhaolin Ren, Aoxiao Zhong, Na Li

Abstract: There has been substantial recent progress on the theoretical understanding of model-free approaches to Linear Quadratic Regulator (LQR) problems. Much attention has been devoted to the special case when the goal is to drive the state close to a zero target. In this work, we consider the general case where the target is allowed to be arbitrary, which we refer to as the LQR tracking problem. We stu… ▽ More There has been substantial recent progress on the theoretical understanding of model-free approaches to Linear Quadratic Regulator (LQR) problems. Much attention has been devoted to the special case when the goal is to drive the state close to a zero target. In this work, we consider the general case where the target is allowed to be arbitrary, which we refer to as the LQR tracking problem. We study the optimization landscape of this problem, and show that similar to the zero-target LQR problem, the LQR tracking problem also satisfies gradient dominance and local smoothness properties. This allows us to develop a zeroth-order policy gradient algorithm that achieves global convergence. We support our arguments with numerical simulations on a linear system. △ Less

Submitted 12 April, 2021; v1 submitted 3 November, 2020; originally announced November 2020.

Comments: Modified parameterization for policy structure. Revised focus to be on optimization landscape and convergence for the LQR tracking problem

Generalization Guarantees for Imitation Learning

Authors: Allen Z. Ren, Sushant Veer, Anirudha Majumdar

Abstract: Control policies from imitation learning can often fail to generalize to novel environments due to imperfect demonstrations or the inability of imitation learning algorithms to accurately infer the expert's policies. In this paper, we present rigorous generalization guarantees for imitation learning by leveraging the Probably Approximately Correct (PAC)-Bayes framework to provide upper bounds on t… ▽ More Control policies from imitation learning can often fail to generalize to novel environments due to imperfect demonstrations or the inability of imitation learning algorithms to accurately infer the expert's policies. In this paper, we present rigorous generalization guarantees for imitation learning by leveraging the Probably Approximately Correct (PAC)-Bayes framework to provide upper bounds on the expected cost of policies in novel environments. We propose a two-stage training method where a latent policy distribution is first embedded with multi-modal expert behavior using a conditional variational autoencoder, and then "fine-tuned" in new training environments to explicitly optimize the generalization bound. We demonstrate strong generalization bounds and their tightness relative to empirical performance in simulation for (i) gras** diverse mugs, (ii) planar pushing with visual feedback, and (iii) vision-based indoor navigation, as well as through hardware experiments for the two manipulation tasks. △ Less

Submitted 3 December, 2020; v1 submitted 4 August, 2020; originally announced August 2020.

Comments: Presented at the Conference on Robot Learning (CoRL), 2020

Ergodicity of the underdamped mean-field Langevin dynamics

Authors: Anna Kazeykina, Zhenjie Ren, Xiaolu Tan, Junjian Yang

Abstract: We study the long time behavior of an underdamped mean-field Langevin (MFL) equation, and provide a general convergence as well as an exponential convergence rate result under different conditions. The results on the MFL equation can be applied to study the convergence of the Hamiltonian gradient descent algorithm for the overparametrized optimization. We then provide a numerical example of the al… ▽ More We study the long time behavior of an underdamped mean-field Langevin (MFL) equation, and provide a general convergence as well as an exponential convergence rate result under different conditions. The results on the MFL equation can be applied to study the convergence of the Hamiltonian gradient descent algorithm for the overparametrized optimization. We then provide a numerical example of the algorithm to train a generative adversarial networks (GAN). △ Less

Submitted 25 November, 2023; v1 submitted 29 July, 2020; originally announced July 2020.

Comments: 45 pages, 9 figures

MSC Class: 37M25; 60H10; 60H30; 60J60

Game on Random Environment, Mean-field Langevin System and Neural Networks

Authors: Giovanni Conforti, Anna Kazeykina, Zhenjie Ren

Abstract: In this paper we study a type of games regularized by the relative entropy, where the players' strategies are coupled through a random environment variable. Besides the existence and the uniqueness of equilibria of such games, we prove that the marginal laws of the corresponding mean-field Langevin systems can converge towards the games' equilibria in different settings. As applications, the dynam… ▽ More In this paper we study a type of games regularized by the relative entropy, where the players' strategies are coupled through a random environment variable. Besides the existence and the uniqueness of equilibria of such games, we prove that the marginal laws of the corresponding mean-field Langevin systems can converge towards the games' equilibria in different settings. As applications, the dynamic games can be treated as games on a random environment when one treats the time horizon as the environment. In practice, our results can be applied to analysing the stochastic gradient descent algorithm for deep neural networks in the context of supervised learning as well as for the generative adversarial networks. △ Less

Submitted 22 April, 2020; v1 submitted 6 April, 2020; originally announced April 2020.

Random horizon principal-agent problems

Authors: Yiqing Lin, Zhenjie Ren, Nizar Touzi, Junjian Yang

Abstract: We consider a general formulation of the random horizon Principal-Agent problem with a continuous payment and a lump-sum payment at termination. In the European version of the problem, the random horizon is chosen solely by the principal with no other possible action from the agent than exerting effort on the dynamics of the output process. We also consider the American version of the contract, wh… ▽ More We consider a general formulation of the random horizon Principal-Agent problem with a continuous payment and a lump-sum payment at termination. In the European version of the problem, the random horizon is chosen solely by the principal with no other possible action from the agent than exerting effort on the dynamics of the output process. We also consider the American version of the contract, which covers the seminal Sannikov's model, where the agent can also quit by optimally choosing the termination time of the contract. Our main result reduces such non-zero-sum stochastic differential games to appropriate stochastic control problems which may be solved by standard methods of stochastic control theory. This reduction is obtained by following Sannikov's approach, further developed by Cvitanic, Possamai, and Touzi. We first introduce an appropriate class of contracts for which the agent's optimal effort is immediately characterized by the standard verification argument in stochastic control theory. We then show that this class of contracts is dense in an appropriate sense so that the optimization over this restricted family of contracts represents no loss of generality. The result is obtained by using the recent well-posedness result of random horizon second-order backward SDE. △ Less

Submitted 10 February, 2022; v1 submitted 25 February, 2020; originally announced February 2020.

Knockoffs with Side Information

Authors: Zhimei Ren, Emmanuel Candès

Abstract: We consider the problem of assessing the importance of multiple variables or factors from a dataset when side information is available. In principle, using side information can allow the statistician to pay attention to variables with a greater potential, which in turn, may lead to more discoveries. We introduce an adaptive knockoff filter, which generalizes the knockoff procedure (Barber and Cand… ▽ More We consider the problem of assessing the importance of multiple variables or factors from a dataset when side information is available. In principle, using side information can allow the statistician to pay attention to variables with a greater potential, which in turn, may lead to more discoveries. We introduce an adaptive knockoff filter, which generalizes the knockoff procedure (Barber and Candès, 2015; Candès et al., 2018) in that it uses both the data at hand and side information to adaptively order the variables under study and focus on those that are most promising. Adaptive knockoffs controls the finite-sample false discovery rate (FDR) and we demonstrate its power by comparing it with other structured multiple testing methods. We also apply our methodology to real genetic data in order to find associations between genetic variants and various phenotypes such as Crohn's disease and lipid levels. Here, adaptive knockoffs makes more discoveries than reported in previous studies on the same datasets. △ Less

Submitted 21 January, 2020; originally announced January 2020.

Comments: 29 pages, 9 figures, 7 tables

Mean Field Games with Branching

Authors: Julien Claisse, Zhenjie Ren, Xiaolu Tan

Abstract: Mean field games are concerned with the limit of large-population stochastic differential games where the agents interact through their empirical distribution. In the classical setting, the number of players is large but fixed throughout the game. However, in various applications, such as population dynamics or economic growth, the number of players can vary across time which may lead to different… ▽ More Mean field games are concerned with the limit of large-population stochastic differential games where the agents interact through their empirical distribution. In the classical setting, the number of players is large but fixed throughout the game. However, in various applications, such as population dynamics or economic growth, the number of players can vary across time which may lead to different Nash equilibria. For this reason, we introduce a branching mechanism in the population of agents and obtain a variation on the mean field game problem. As a first step, we study a simple model using a PDE approach to illustrate the main differences with the classical setting. We prove existence of a solution and show that it provides an approximate Nash-equilibrium for large population games. We also present a numerical example for a linear--quadratic model. Then we study the problem in a general setting by a probabilistic approach. It is based upon the relaxed formulation of stochastic control problems which allows us to obtain a general existence result. △ Less

Submitted 26 December, 2019; originally announced December 2019.

Comments: 32 pages, 1 figure

MSC Class: 60J80; 91A13; 93E20

Continuous-Time Principal-Agent Problem in Degenerate Systems

Authors: Kaitong Hu, Zhenjie Ren, Nizar Touzi

Abstract: In this paper we present a variational calculus approach to Principal-Agent problem with a lump-sum payment on finite horizon in degenerate stochastic systems, such as filtered partially observed linear systems. Our work extends the existing methodologies in the Principal-Agent literature using dynamic programming and BSDE representation of the contracts in the non-degenerate controlled stochastic… ▽ More In this paper we present a variational calculus approach to Principal-Agent problem with a lump-sum payment on finite horizon in degenerate stochastic systems, such as filtered partially observed linear systems. Our work extends the existing methodologies in the Principal-Agent literature using dynamic programming and BSDE representation of the contracts in the non-degenerate controlled stochastic systems. We first solve the Principal's problem in an enlarged set of contracts defined by a forward-backward SDE system given by the first order condition of the Agent's problem using variational calculus. Then we use the sufficient condition of the Agent's problem to verify that the optimal contract that we obtain by solving the Principal's problem is indeed implementable (i.e. belonging to the admissible contract set). Importantly we consider the control problem in a weak formulation. Finally, we give explicit solution of the Principal-Agent problem in partially observed linear systems and extend our results to some mean field interacting Agents case. △ Less

Submitted 23 October, 2019; originally announced October 2019.

Comments: 25 pages

MSC Class: 60H30; 91A23; 91A35

Mean-field Langevin System, Optimal Control and Deep Neural Networks

Authors: Kaitong Hu, Anna Kazeykina, Zhenjie Ren

Abstract: In this paper, we study a regularised relaxed optimal control problem and, in particular, we are concerned with the case where the control variable is of large dimension. We introduce a system of mean-field Langevin equations, the invariant measure of which is shown to be the optimal control of the initial problem under mild conditions. Therefore, this system of processes can be viewed as a contin… ▽ More In this paper, we study a regularised relaxed optimal control problem and, in particular, we are concerned with the case where the control variable is of large dimension. We introduce a system of mean-field Langevin equations, the invariant measure of which is shown to be the optimal control of the initial problem under mild conditions. Therefore, this system of processes can be viewed as a continuous-time numerical algorithm for computing the optimal control. As an application, this result endorses the solvability of the stochastic gradient descent algorithm for a wide class of deep neural networks. △ Less

Submitted 3 October, 2019; v1 submitted 16 September, 2019; originally announced September 2019.

Comments: 25 pages

MSC Class: 60H30; 37M25

Consistency of semi-supervised learning algorithms on graphs: Probit and one-hot methods

Authors: Franca Hoffmann, Bamdad Hosseini, Zhi Ren, Andrew M. Stuart

Abstract: Graph-based semi-supervised learning is the problem of propagating labels from a small number of labelled data points to a larger set of unlabelled data. This paper is concerned with the consistency of optimization-based techniques for such problems, in the limit where the labels have small noise and the underlying unlabelled data is well clustered. We study graph-based probit for binary classific… ▽ More Graph-based semi-supervised learning is the problem of propagating labels from a small number of labelled data points to a larger set of unlabelled data. This paper is concerned with the consistency of optimization-based techniques for such problems, in the limit where the labels have small noise and the underlying unlabelled data is well clustered. We study graph-based probit for binary classification, and a natural generalization of this method to multi-class classification using one-hot encoding. The resulting objective function to be optimized comprises the sum of a quadratic form defined through a rational function of the graph Laplacian, involving only the unlabelled data, and a fidelity term involving only the labelled data. The consistency analysis sheds light on the choice of the rational function defining the optimization. △ Less

Submitted 9 March, 2020; v1 submitted 18 June, 2019; originally announced June 2019.

MSC Class: 62H30; 68T10; 68Q87; 91C20

Competitive Exclusion in a DAE Model for Microbial Electrolysis Cells

Authors: Harry J. Dudley, Zhiyong Jason Ren, David M. Bortz

Abstract: Microbial electrolysis cells (MECs) employ electroactive bacteria to perform extracellular electron transfer, enabling hydrogen generation from biodegradable substrates. In previous work, we developed and analyzed a differential-algebraic equation (DAE) model for MECs. The model resembles a chemostat with ordinary differential equations (ODEs) for concentrations of substrate, microorganisms, and a… ▽ More Microbial electrolysis cells (MECs) employ electroactive bacteria to perform extracellular electron transfer, enabling hydrogen generation from biodegradable substrates. In previous work, we developed and analyzed a differential-algebraic equation (DAE) model for MECs. The model resembles a chemostat with ordinary differential equations (ODEs) for concentrations of substrate, microorganisms, and an extracellular mediator involved in electron transfer. There is also an algebraic constraint for electric current and hydrogen production. Our goal is to determine the outcome of competition between methanogenic archaea and electroactive bacteria, because only the latter contribute to electric current and resulting hydrogen production. We investigate asymptotic stability in two industrially relevant versions of the model. An important aspect of chemostats models is the principle of competitive exclusion -- only microbes which grow at the lowest substrate concentration will survive as $t\to\infty$. We show that if methanogens grow at the lowest substrate concentration, then the equilibrium corresponding to competitive exclusion by methanogens is globally asymptotically stable. The analogous result for electroactive bacteria is not necessarily true. We show that local asymptotic stability of exclusion by electroactive bacteria is not guaranteed, even in a simplified version of the model. In this case, even if electroactive bacteria can grow at the lowest substrate concentration, a few additional conditions are required to guarantee local asymptotic stability. We also provide numerical simulations supporting these arguments. Our results suggest operating conditions that are most conducive to success of electroactive bacteria and the resulting current and hydrogen production in MECs. This will help identify when methane production or electricity and hydrogen production are favored. △ Less

Submitted 6 July, 2020; v1 submitted 5 June, 2019; originally announced June 2019.

A Fast Differential Grou** Algorithm for Large Scale Black-Box Optimization

Authors: Zhigang Ren, An Chen, Yaochu **, Wenhua Guo, Yongsheng Liang, Zuren Feng

Abstract: Decomposition plays a significant role in cooperative co-evolution which shows great potential in large scale black-box optimization. However, current popular decomposition algorithms generally require to sample and evaluate a large number of solutions for interdependency detection, which is very time-consuming. To address this issue, this study proposes a new decomposition algorithm named fast di… ▽ More Decomposition plays a significant role in cooperative co-evolution which shows great potential in large scale black-box optimization. However, current popular decomposition algorithms generally require to sample and evaluate a large number of solutions for interdependency detection, which is very time-consuming. To address this issue, this study proposes a new decomposition algorithm named fast differential grou** (FDG). FDG first identifies the type of an instance by detecting the interdependencies of a few pairs of variable subsets selected according to certain rules, and thus can rapidly complete the decomposition of a fully separable or nonseparable instance. For an identified partially separable instance, FDG converts the key decomposition process into a search process in a binary tree by taking corresponding variable subsets as tree nodes. This enables it to directly deduce the interdependency related to a child node by reutilizing the solutions sampled for corresponding parent and brother nodes. To support the above operations, this study designs a normalized variable-subset-oriented interdependency indicator, which can adaptively generate decomposition thresholds according to its distribution and thus enhances decomposition accuracy. Computational complexity analysis and experimental results verify that FDG outperforms popular decomposition algorithms. Further tests indicate that FDG embedded in a cooperative co-evolution framework can achieve highly competitive optimization results as compared with some state-of-the-art algorithms for large scale black-box optimization. △ Less

Submitted 28 May, 2019; originally announced May 2019.

Mean-Field Langevin Dynamics and Energy Landscape of Neural Networks

Authors: Kaitong Hu, Zhenjie Ren, David Siska, Lukasz Szpruch

Abstract: Our work is motivated by a desire to study the theoretical underpinning for the convergence of stochastic gradient type algorithms widely used for non-convex learning tasks such as training of neural networks. The key insight, already observed in the works of Mei, Montanari and Nguyen (2018), Chizat and Bach (2018) as well as Rotskoff and Vanden-Eijnden (2018), is that a certain class of the finit… ▽ More Our work is motivated by a desire to study the theoretical underpinning for the convergence of stochastic gradient type algorithms widely used for non-convex learning tasks such as training of neural networks. The key insight, already observed in the works of Mei, Montanari and Nguyen (2018), Chizat and Bach (2018) as well as Rotskoff and Vanden-Eijnden (2018), is that a certain class of the finite-dimensional non-convex problems becomes convex when lifted to infinite-dimensional space of measures. We leverage this observation and show that the corresponding energy functional defined on the space of probability measures has a unique minimiser which can be characterised by a first-order condition using the notion of linear functional derivative. Next, we study the corresponding gradient flow structure in 2-Wasserstein metric, which we call Mean-Field Langevin Dynamics (MFLD), and show that the flow of marginal laws induced by the gradient flow converges to a stationary distribution, which is exactly the minimiser of the energy functional. We observe that this convergence is exponential under conditions that are satisfied for highly regularised learning tasks. Our proof of convergence to stationary probability measure is novel and it relies on a generalisation of LaSalle's invariance principle combined with HWI inequality. Importantly, we assume neither that interaction potential of MFLD is of convolution type nor that it has any particular symmetric structure. Furthermore, we allow for the general convex objective function, unlike, most papers in the literature that focus on quadratic loss. Finally, we show that the error between finite-dimensional optimisation problem and its infinite-dimensional limit is of order one over the number of parameters. △ Less

Submitted 13 December, 2020; v1 submitted 19 May, 2019; originally announced May 2019.

Comments: 31 pages

MSC Class: 60H30; 37M25

Principal-agent problem with multiple principals

Authors: Kaitong Hu, Zhenjie Ren, Junjian Yang

Abstract: We consider a moral hazard problem with multiple principals in a continuous-time model. The agent can only work exclusively for one principal at a given time, so faces an optimal switching problem. Using a randomized formulation, we manage to represent the agent's value function and his optimal effort by an Itô process. This representation further helps to solve the principals' problem in case we… ▽ More We consider a moral hazard problem with multiple principals in a continuous-time model. The agent can only work exclusively for one principal at a given time, so faces an optimal switching problem. Using a randomized formulation, we manage to represent the agent's value function and his optimal effort by an Itô process. This representation further helps to solve the principals' problem in case we have infinite number of principals in the sense of mean field game. Finally the mean field formulation is justified by an argument of propagation of chaos. △ Less

Submitted 13 September, 2022; v1 submitted 30 March, 2019; originally announced April 2019.

MSC Class: 91B40; 93E20

User-Friendly Covariance Estimation for Heavy-Tailed Distributions

Authors: Yuan Ke, Stanislav Minsker, Zhao Ren, Qiang Sun, Wen-Xin Zhou

Abstract: We offer a survey of recent results on covariance estimation for heavy-tailed distributions. By unifying ideas scattered in the literature, we propose user-friendly methods that facilitate practical implementation. Specifically, we introduce element-wise and spectrum-wise truncation operators, as well as their $M$-estimator counterparts, to robustify the sample covariance matrix. Different from th… ▽ More We offer a survey of recent results on covariance estimation for heavy-tailed distributions. By unifying ideas scattered in the literature, we propose user-friendly methods that facilitate practical implementation. Specifically, we introduce element-wise and spectrum-wise truncation operators, as well as their $M$-estimator counterparts, to robustify the sample covariance matrix. Different from the classical notion of robustness that is characterized by the breakdown property, we focus on the tail robustness which is evidenced by the connection between nonasymptotic deviation and confidence level. The key observation is that the estimators needs to adapt to the sample size, dimensionality of the data and the noise level to achieve optimal tradeoff between bias and robustness. Furthermore, to facilitate their practical use, we propose data-driven procedures that automatically calibrate the tuning parameters. We demonstrate their applications to a series of structured models in high dimensions, including the bandable and low-rank covariance matrices and sparse precision matrices. Numerical studies lend strong support to the proposed methods. △ Less

Submitted 11 March, 2019; v1 submitted 5 November, 2018; originally announced November 2018.

Comments: 56 pages, 2 figures

Fitting a Graph to One-Dimensional Data

Authors: Siu-Wing Cheng, Otfried Cheong, Taegyoung Lee, Zhengtong Ren

Abstract: Given n data points in R^d, an appropriate edge-weighted graph connecting the data points finds application in solving clustering, classification, and regresssion problems. The graph proposed by Daitch, Kelner and Spielman (ICML~2009) can be computed by quadratic programming and hence in polynomial time. While a more efficient algorithm would be preferable, replacing quadratic programming is chall… ▽ More Given n data points in R^d, an appropriate edge-weighted graph connecting the data points finds application in solving clustering, classification, and regresssion problems. The graph proposed by Daitch, Kelner and Spielman (ICML~2009) can be computed by quadratic programming and hence in polynomial time. While a more efficient algorithm would be preferable, replacing quadratic programming is challenging even for the special case of points in one dimension. We develop a dynamic programming algorithm for this case that runs in O(n^2) time. △ Less

Submitted 30 September, 2020; v1 submitted 9 September, 2018; originally announced September 2018.

Nonlinear predictable representation and $L^1$-solutions of backward SDEs and second-order backward SDEs

Authors: Zhenjie Ren, Nizar Touzi, Junjian Yang

Abstract: The theory of backward SDEs extends the predictable representation property of Brownian motion to the nonlinear framework, thus providing a path-dependent analog of fully nonlinear parabolic PDEs. In this paper, we consider backward SDEs, their reflected version, and their second-order extension, in the context where the final data and the generator satisfy $L^1$-type of integrability condition. O… ▽ More The theory of backward SDEs extends the predictable representation property of Brownian motion to the nonlinear framework, thus providing a path-dependent analog of fully nonlinear parabolic PDEs. In this paper, we consider backward SDEs, their reflected version, and their second-order extension, in the context where the final data and the generator satisfy $L^1$-type of integrability condition. Our main objective is to provide the corresponding existence and uniqueness results for general Lipschitz generators. The uniqueness holds in the so-called Doob class of processes, simultaneously under an appropriate class of measures. We emphasize that the previous literature only deals with backward SDEs, and requires either that the generator is separable in $(y,z)$, see Peng [Pen97], or strictly sublinear in the gradient variable $z$, see [BDHPS03], or that the final data satisfies an $L\ln L$-integrability condition, see [HT18]. We by-pass these conditions by defining $L^1$-integrability under the nonlinear expectation operator induced by the previously mentioned class of measures. △ Less

Submitted 11 February, 2022; v1 submitted 17 August, 2018; originally announced August 2018.

MSC Class: 60H10

Viscosity solutions of path-dependent PDEs with randomized time

Authors: Zhenjie Ren, Mauro Rosestolato

Abstract: We introduce a new definition of viscosity solution to path-dependent partial differential equations, which is a slight modification of the definition introduced in [8]. With the new definition, we prove the two important results till now missing in the literature, namely, a general stability result and a comparison result for semicontinuous sub-/super-solutions. As an application, we prove the ex… ▽ More We introduce a new definition of viscosity solution to path-dependent partial differential equations, which is a slight modification of the definition introduced in [8]. With the new definition, we prove the two important results till now missing in the literature, namely, a general stability result and a comparison result for semicontinuous sub-/super-solutions. As an application, we prove the existence of viscosity solutions using the Perron method. Moreover, we connect viscosity solutions of path-dependent PDEs with viscosity solutions of partial differential equations on Hilbert spaces. △ Less

Submitted 20 June, 2018; originally announced June 2018.

MSC Class: 35K10; 35R15; 49L25; 60H30

Sensitivity and Bifurcation Analysis of a DAE Model for a Microbial Electrolysis Cell

Authors: Harry J. Dudley, Lu Lu, Zhiyong Jason Ren, David M. Bortz

Abstract: Microbial electrolysis cells (MECs) are a promising new technology for producing hydrogen cheaply, efficiently, and sustainably. However, to scale up this technology, we need a better understanding of the processes in the devices. In this effort, we present a differential-algebraic equation (DAE) model of a microbial electrolysis cell with an algebraic constraint on current. We then perform sensit… ▽ More Microbial electrolysis cells (MECs) are a promising new technology for producing hydrogen cheaply, efficiently, and sustainably. However, to scale up this technology, we need a better understanding of the processes in the devices. In this effort, we present a differential-algebraic equation (DAE) model of a microbial electrolysis cell with an algebraic constraint on current. We then perform sensitivity and bifurcation analysis for the DAE system. The model can be applied either to batch-cycle MECs or to continuous-flow MECs. We conduct differential-algebraic sensitivity analysis after fitting simulations to current density data for a batch-cycle MEC. The sensitivity analysis suggests which parameters have the greatest influence on the current density at particular times during the experiment. In particular, growth and consumption parameters for exoelectrogenic bacteria have a strong effect prior to the peak current density. An alternative strategy to maximizing peak current density is maintaining a long term stable equilibrium with non-zero current density in a continuous-flow MEC. We characterize the minimum dilution rate required for a stable nonzero current equilibrium and demonstrate transcritical bifurcations in the dilution rate parameter that exchange stability between several curves of equilibria. Specifically, increasing the dilution rate transitions the system through three regimes where the stable equilibrium exhibits (i) competitive exclusion by methanogens, (ii) coexistence, and (iii) competitive exclusion by exolectrogens. Positive long term current production is only feasible in the final two regimes. These results suggest how to modify system parameters to increase peak current density in a batch-cycle MEC or to increase the long term current density equilibrium value in a continuous-flow MEC. △ Less

Submitted 17 February, 2018; originally announced February 2018.

MSC Class: 37N25; 92C40

Second order backward SDE with random terminal time

Authors: Yiqing Lin, Zhenjie Ren, Nizar Touzi, Junjian Yang

Abstract: Backward stochastic differential equations extend the martingale representation theorem to the nonlinear setting. This can be seen as path-dependent counterpart of the extension from the heat equation to fully nonlinear parabolic equations in the Markov setting. This paper extends such a nonlinear representation to the context where the random variable of interest is measurable with respect to the… ▽ More Backward stochastic differential equations extend the martingale representation theorem to the nonlinear setting. This can be seen as path-dependent counterpart of the extension from the heat equation to fully nonlinear parabolic equations in the Markov setting. This paper extends such a nonlinear representation to the context where the random variable of interest is measurable with respect to the information at a finite stop** time. We provide a complete wellposedness theory which covers the semilinear case (backward SDE), the semilinear case with obstacle (reflected backward SDE), and the fully nonlinear case (second order backward SDE). △ Less

Submitted 11 February, 2022; v1 submitted 6 February, 2018; originally announced February 2018.

MSC Class: 60H10; 60H30

Minimax Estimation of Large Precision Matrices with Bandable Cholesky Factor

Authors: Yu Liu, Zhao Ren

Abstract: Last decade witnesses significant methodological and theoretical advances in estimating large precision matrices. In particular, there are scientific applications such as longitudinal data, meteorology and spectroscopy in which the ordering of the variables can be interpreted through a bandable structure on the Cholesky factor of the precision matrix. However, the minimax theory has still been lar… ▽ More Last decade witnesses significant methodological and theoretical advances in estimating large precision matrices. In particular, there are scientific applications such as longitudinal data, meteorology and spectroscopy in which the ordering of the variables can be interpreted through a bandable structure on the Cholesky factor of the precision matrix. However, the minimax theory has still been largely unknown, as opposed to the well established minimax results over the corresponding bandable covariance matrices. In this paper, we focus on two commonly used types of parameter spaces, and develop the optimal rates of convergence under both the operator norm and the Frobenius norm. A striking phenomenon is found: two types of parameter spaces are fundamentally different under the operator norm but enjoy the same rate optimality under the Frobenius norm, which is in sharp contrast to the equivalence of corresponding two types of bandable covariance matrices under both norms. This fundamental difference is established by carefully constructing the corresponding minimax lower bounds. Two new estimation procedures are developed: for the operator norm, our optimal procedure is based on a novel local crop** estimator targeting on all principle submatrices of the precision matrix while for the Frobenius norm, our optimal procedure relies on a delicate regression-based thresholding rule. Lepski's method is considered to achieve optimal adaptation. We further establish rate optimality in the nonparanormal model. Numerical studies are carried out to confirm our theoretical findings. △ Less

Submitted 18 August, 2019; v1 submitted 26 December, 2017; originally announced December 2017.

Variable screening with multiple studies

Authors: Tianzhou Ma, Zhao Ren, George C. Tseng

Abstract: Advancement in technology has generated abundant high-dimensional data that allows integration of multiple relevant studies. Due to their huge computational advantage, variable screening methods based on marginal correlation have become promising alternatives to the popular regularization methods for variable selection. However, all these screening methods are limited to single study so far. In th… ▽ More Advancement in technology has generated abundant high-dimensional data that allows integration of multiple relevant studies. Due to their huge computational advantage, variable screening methods based on marginal correlation have become promising alternatives to the popular regularization methods for variable selection. However, all these screening methods are limited to single study so far. In this paper, we consider a general framework for variable screening with multiple related studies, and further propose a novel two-step screening procedure using a self-normalized estimator for high-dimensional regression analysis in this framework. Compared to the one-step procedure and rank-based sure independence screening (SIS) procedure, our procedure greatly reduces false negative errors while kee** a low false positive rate. Theoretically, we show that our procedure possesses the sure screening property with weaker assumptions on signal strengths and allows the number of features to grow at an exponential rate of the sample size. In addition, we relax the commonly used normality assumption and allow sub-Gaussian distributions. Simulations and a real transcriptomic application illustrate the advantage of our method as compared to the rank-based SIS method. △ Less

Submitted 10 October, 2017; originally announced October 2017.

Comments: 25 pages, 1 figure

Principal-Agent Problem with Common Agency without Communication

Authors: Thibaut Mastrolia, Zhenjie Ren

Abstract: In this paper, we consider a problem of contract theory in which several Principals hire a common Agent and we study the model in the continuous time setting. We show that optimal contracts should satisfy some equilibrium conditions and we reduce the optimisation problem of the Principals to a system of coupled Hamilton-Jacobi-Bellman (HJB) equations. We provide conditions ensuring that for risk-n… ▽ More In this paper, we consider a problem of contract theory in which several Principals hire a common Agent and we study the model in the continuous time setting. We show that optimal contracts should satisfy some equilibrium conditions and we reduce the optimisation problem of the Principals to a system of coupled Hamilton-Jacobi-Bellman (HJB) equations. We provide conditions ensuring that for risk-neutral Principals, the system of coupled HJB equations admits a solution. Further, we apply our study in a more specific linear-quadratic model where two interacting Principals hire one common Agent. In this continuous time model, we extend the result of Bernheim and Whinston (1986) in which the authors compare the optimal effort of the Agent in a non-cooperative Principals model and that in the aggregate model, by showing that these two optimisations coincide only in the first best case. We also study the sensibility of the optimal effort and the optimal remunerations with respect to appetence parameters and the correlation between the projects. △ Less

Submitted 12 January, 2018; v1 submitted 9 June, 2017; originally announced June 2017.

Pairwise Difference Estimation of High Dimensional Partially Linear Model

Authors: Fang Han, Zhao Ren, Yuxin Zhu

Abstract: This paper proposes a regularized pairwise difference approach for estimating the linear component coefficient in a partially linear model, with consistency and exact rates of convergence obtained in high dimensions under mild scaling requirements. Our analysis reveals interesting features such as (i) the bandwidth parameter automatically adapts to the model and is actually tuning-insensitive; and… ▽ More This paper proposes a regularized pairwise difference approach for estimating the linear component coefficient in a partially linear model, with consistency and exact rates of convergence obtained in high dimensions under mild scaling requirements. Our analysis reveals interesting features such as (i) the bandwidth parameter automatically adapts to the model and is actually tuning-insensitive; and (ii) the procedure could even maintain fast rate of convergence for $α$-Hölder class of $α\leq1/2$. Simulation studies show the advantage of the proposed method, and application of our approach to a brain imaging data reveals some biological patterns which fail to be recovered using competing methods. △ Less

Submitted 11 January, 2018; v1 submitted 24 May, 2017; originally announced May 2017.

Comments: 28 pages

Dynamic Optimization of Trajectory for Ramp-up Current Profile in Tokamak Plasmas

Authors: Zhigang Ren, Chao Xu, Yongsheng Ou

Abstract: In this paper, we consider an open-loop, finite-time, optimal control problem of attaining a specific desired current profile during the ramp-up phase by finding the best open-loop actuator input trajectories. Average density, total power, and plasma current are used as control actuators to manipulate the profile shape in tokamak plasmas. Based on the control parameterization method, we propose a… ▽ More In this paper, we consider an open-loop, finite-time, optimal control problem of attaining a specific desired current profile during the ramp-up phase by finding the best open-loop actuator input trajectories. Average density, total power, and plasma current are used as control actuators to manipulate the profile shape in tokamak plasmas. Based on the control parameterization method, we propose a numerical solution procedure directly to solve the original PDE-constrained optimization problem using gradient-based optimization techniques such as sequential quadratic programming (SQP). This paper is aimed at proposing an effective framework for the solution of PDE-constrained optimization problem in tokamak plasmas. A more user-friendly and efficient graphical user interface (GUI) is designed in MATLAB and the numerical simulation results are verified to demonstrate its applicability. In addition, the proposed framework of combining existing PDE and numerical optimization solvers to solve PDE-constrained optimization problem has the prospective to target challenge advanced control problems arising in more general chemical engineering processes. △ Less

Submitted 10 August, 2016; originally announced August 2016.

Comments: 19 pages, 13 figures

Tuning-Free Heterogeneity Pursuit in Massive Networks

Authors: Zhao Ren, Yongjian Kang, Yingying Fan, **chi Lv

Abstract: Heterogeneity is often natural in many contemporary applications involving massive data. While posing new challenges to effective learning, it can play a crucial role in powering meaningful scientific discoveries through the understanding of important differences among subpopulations of interest. In this paper, we exploit multiple networks with Gaussian graphs to encode the connectivity patterns o… ▽ More Heterogeneity is often natural in many contemporary applications involving massive data. While posing new challenges to effective learning, it can play a crucial role in powering meaningful scientific discoveries through the understanding of important differences among subpopulations of interest. In this paper, we exploit multiple networks with Gaussian graphs to encode the connectivity patterns of a large number of features on the subpopulations. To uncover the heterogeneity of these structures across subpopulations, we suggest a new framework of tuning-free heterogeneity pursuit (THP) via large-scale inference, where the number of networks is allowed to diverge. In particular, two new tests, the chi-based test and the linear functional-based test, are introduced and their asymptotic null distributions are established. Under mild regularity conditions, we establish that both tests are optimal in achieving the testable region boundary and the sample size requirement for the latter test is minimal. Both theoretical guarantees and the tuning-free feature stem from efficient multiple-network estimation by our newly suggested approach of heterogeneous group square-root Lasso (HGSL) for high-dimensional multi-response regression with heterogeneous noises. To solve this convex program, we further introduce a tuning-free algorithm that is scalable and enjoys provable convergence to the global optimum. Both computational and theoretical advantages of our procedure are elucidated through simulation and real data examples. △ Less

Submitted 12 June, 2016; originally announced June 2016.

Comments: 29 pages for the main text including 1 figure and 7 tables, 28 pages for the Supplementary Material

A Gradient-based Kernel Optimization Approach for Parabolic Distributed Parameter Control Systems

Authors: Zhigang Ren, Chao Xu, Qun Lin, Ryan Loxton

Abstract: This paper proposes a new gradient-based optimization approach for designing optimal feedback kernels for parabolic distributed parameter systems with boundary control. Unlike traditional kernel optimization methods for parabolic systems, our new method does not require solving non-standard Riccati-type or Klein-Gorden-type partial differential equations (PDEs). Instead, the feedback kernel is par… ▽ More This paper proposes a new gradient-based optimization approach for designing optimal feedback kernels for parabolic distributed parameter systems with boundary control. Unlike traditional kernel optimization methods for parabolic systems, our new method does not require solving non-standard Riccati-type or Klein-Gorden-type partial differential equations (PDEs). Instead, the feedback kernel is parameterized as a second-order polynomial whose coefficients are decision variables to be tuned via gradient-based dynamic optimization, where the gradient of the system cost functional (which penalizes both kernel and output magnitude) is computed by solving a so-called costate PDE instandard form. Special constraints are imposed on the kernel coefficients to ensure that, under mild conditions, the optimized kernel yields closed-loop stability. Numerical simulations demonstrate the effectiveness of the proposed approach. △ Less

Submitted 15 March, 2016; originally announced March 2016.

Comments: 26 pages, 17 figures