-
Minimax-optimal estimation for sparse multi-reference alignment with collision-free signals
Authors:
Subhro Ghosh,
Soumendu Sundar Mukherjee,
**g Bin Pan
Abstract:
The Multi-Reference Alignment (MRA) problem aims at the recovery of an unknown signal from repeated observations under the latent action of a group of cyclic isometries, in the presence of additive noise of high intensity $σ$. It is a more tractable version of the celebrated cryo EM model. In the crucial high noise regime, it is known that its sample complexity scales as $σ^6$. Recent investigatio…
▽ More
The Multi-Reference Alignment (MRA) problem aims at the recovery of an unknown signal from repeated observations under the latent action of a group of cyclic isometries, in the presence of additive noise of high intensity $σ$. It is a more tractable version of the celebrated cryo EM model. In the crucial high noise regime, it is known that its sample complexity scales as $σ^6$. Recent investigations have shown that for the practically significant setting of sparse signals, the sample complexity of the maximum likelihood estimator asymptotically scales with the noise level as $σ^4$. In this work, we investigate minimax optimality for signal estimation under the MRA model for so-called collision-free signals. In particular, this signal class covers the setting of generic signals of dilute sparsity (wherein the support size $s=O(L^{1/3})$, where $L$ is the ambient dimension.
We demonstrate that the minimax optimal rate of estimation in for the sparse MRA problem in this setting is $σ^2/\sqrt{n}$, where $n$ is the sample size. In particular, this widely generalizes the sample complexity asymptotics for the restricted MLE in this setting, establishing it as the statistically optimal estimator. Finally, we demonstrate a concentration inequality for the restricted MLE on its deviations from the ground truth.
△ Less
Submitted 12 December, 2023;
originally announced December 2023.
-
Sample Average Approximation for Stochastic Optimization with Dependent Data: Performance Guarantees and Tractability
Authors:
Yafei Wang,
Bo Pan,
Wei Tu,
Peng Liu,
Bei Jiang,
Chao Gao,
Wei Lu,
Shangling Jui,
Linglong Kong
Abstract:
Sample average approximation (SAA), a popular method for tractably solving stochastic optimization problems, enjoys strong asymptotic performance guarantees in settings with independent training samples. However, these guarantees are not known to hold generally with dependent samples, such as in online learning with time series data or distributed computing with Markovian training samples. In this…
▽ More
Sample average approximation (SAA), a popular method for tractably solving stochastic optimization problems, enjoys strong asymptotic performance guarantees in settings with independent training samples. However, these guarantees are not known to hold generally with dependent samples, such as in online learning with time series data or distributed computing with Markovian training samples. In this paper, we show that SAA remains tractable when the distribution of unknown parameters is only observable through dependent instances and still enjoys asymptotic consistency and finite sample guarantees. Specifically, we provide a rigorous probability error analysis to derive $1 - β$ confidence bounds for the out-of-sample performance of SAA estimators and show that these estimators are asymptotically consistent. We then, using monotone operator theory, study the performance of a class of stochastic first-order algorithms trained on a dependent source of data. We show that approximation error for these algorithms is bounded and concentrates around zero, and establish deviation bounds for iterates when the underlying stochastic process is $φ$-mixing. The algorithms presented can be used to handle numerically inconvenient loss functions such as the sum of a smooth and non-smooth function or of non-smooth functions with constraints. To illustrate the usefulness of our results, we present several stochastic versions of popular algorithms such as stochastic proximal gradient descent (S-PGD), stochastic relaxed Peaceman--Rachford splitting algorithms (S-rPRS), and numerical experiment.
△ Less
Submitted 10 December, 2021;
originally announced December 2021.
-
Damped Anderson Mixing for Deep Reinforcement Learning: Acceleration, Convergence, and Stabilization
Authors:
Ke Sun,
Yafei Wang,
Yi Liu,
Yingnan Zhao,
Bo Pan,
Shangling Jui,
Bei Jiang,
Linglong Kong
Abstract:
Anderson mixing has been heuristically applied to reinforcement learning (RL) algorithms for accelerating convergence and improving the sampling efficiency of deep RL. Despite its heuristic improvement of convergence, a rigorous mathematical justification for the benefits of Anderson mixing in RL has not yet been put forward. In this paper, we provide deeper insights into a class of acceleration s…
▽ More
Anderson mixing has been heuristically applied to reinforcement learning (RL) algorithms for accelerating convergence and improving the sampling efficiency of deep RL. Despite its heuristic improvement of convergence, a rigorous mathematical justification for the benefits of Anderson mixing in RL has not yet been put forward. In this paper, we provide deeper insights into a class of acceleration schemes built on Anderson mixing that improve the convergence of deep RL algorithms. Our main results establish a connection between Anderson mixing and quasi-Newton methods and prove that Anderson mixing increases the convergence radius of policy iteration schemes by an extra contraction factor. The key focus of the analysis roots in the fixed-point iteration nature of RL. We further propose a stabilization strategy by introducing a stable regularization term in Anderson mixing and a differentiable, non-expansive MellowMax operator that can allow both faster convergence and more stable behavior. Extensive experiments demonstrate that our proposed method enhances the convergence, stability, and performance of RL algorithms.
△ Less
Submitted 20 October, 2021; v1 submitted 17 October, 2021;
originally announced October 2021.
-
Photoacoustic Reconstruction Using Sparsity in Curvelet Frame: Image versus Data Domain
Authors:
Bolin Pan,
Simon R. Arridge,
Felix Lucka,
Ben T. Cox,
Nam Huynh,
Paul C. Beard,
Edward Z. Zhang,
Marta M. Betcke
Abstract:
Curvelet frame is of special significance for photoacoustic tomography (PAT) due to its sparsifying and microlocalisation properties. We derive a one-to-one map between wavefront directions in image and data spaces in PAT which suggests near equivalence between the recovery of the initial pressure and PAT data from compressed/subsampled measurements when assuming sparsity in Curvelet frame. As the…
▽ More
Curvelet frame is of special significance for photoacoustic tomography (PAT) due to its sparsifying and microlocalisation properties. We derive a one-to-one map between wavefront directions in image and data spaces in PAT which suggests near equivalence between the recovery of the initial pressure and PAT data from compressed/subsampled measurements when assuming sparsity in Curvelet frame. As the latter is computationally more tractable, investigation to which extent this equivalence holds conducted in this paper is of immediate practical significance. To this end we formulate and compare DR, a two step approach based on the recovery of the complete volume of the photoacoustic data from the subsampled data followed by the acoustic inversion, and p0R, a one step approach where the photoacoustic image (the initial pressure, p0) is directly recovered from the subsampled data. Effective representation of the photoacoustic data requires basis defined on the range of the photoacoustic forward operator. To this end we propose a novel wedge-restriction of Curvelet transform which enables us to construct such basis. Both recovery problems are formulated in a variational framework. As the Curvelet frame is heavily overdetermined, we use reweighted l1 norm penalties to enhance the sparsity of the solution. The data reconstruction problem DR is a standard compressed sensing recovery problem, which we solve using an ADMMtype algorithm, SALSA. Subsequently, the initial pressure is recovered using time reversal as implemented in the k-Wave Toolbox. The p0 reconstruction problem, p0R, aims to recover the photoacoustic image directly via FISTA, or ADMM when in addition including a non-negativity constraint. We compare and discuss the relative merits of the two approaches and illustrate them on 2D simulated and 3D real data in a fair and rigorous manner.
△ Less
Submitted 6 August, 2021; v1 submitted 25 November, 2020;
originally announced November 2020.
-
Generic finiteness for a class of symmetric planar central configurations of the six-body problem and the six-vortex problem
Authors:
Thiago Dias,
Bo-Yu Pan
Abstract:
A symmetric planar central configuration of the Newtonian six-body problem $x$ is called cross central configuration if there are precisely four bodies on a symmetry line of $x$. We use complex algebraic geometry and Groebner basis theory to prove that for a generic choice of positive real masses $m_1,m_2,m_3,m_4,m_5=m_6$ there is a finite number of cross central configurations. We also show one e…
▽ More
A symmetric planar central configuration of the Newtonian six-body problem $x$ is called cross central configuration if there are precisely four bodies on a symmetry line of $x$. We use complex algebraic geometry and Groebner basis theory to prove that for a generic choice of positive real masses $m_1,m_2,m_3,m_4,m_5=m_6$ there is a finite number of cross central configurations. We also show one explicit example of a configuration in this class. A part of our approach is based on relaxing the output of the Groebner basis computations. This procedure allows us to obtain upper bounds for the dimension of an algebraic variety. We get the same results considering cross central configurations of the six-vortex problem.
△ Less
Submitted 21 November, 2018;
originally announced November 2018.
