Showing 1–6 of 6 results for author: Agarwal, M

Iterated Schrödinger bridge approximation to Wasserstein Gradient Flows

Authors: Medha Agarwal, Zaid Harchaoui, Garrett Mulcahy, Soumik Pal

Abstract: We introduce a novel discretization scheme for Wasserstein gradient flows that involves successively computing Schrödinger bridges with the same marginals. This is different from both the forward/geodesic approximation and the backward/Jordan-Kinderlehrer-Otto (JKO) approximations. The proposed scheme has two advantages: one, it avoids the use of the score function, and, two, it is amenable to par… ▽ More We introduce a novel discretization scheme for Wasserstein gradient flows that involves successively computing Schrödinger bridges with the same marginals. This is different from both the forward/geodesic approximation and the backward/Jordan-Kinderlehrer-Otto (JKO) approximations. The proposed scheme has two advantages: one, it avoids the use of the score function, and, two, it is amenable to particle-based approximations using the Sinkhorn algorithm. Our proof hinges upon showing that relative entropy between the Schrödinger bridge with the same marginals at temperature $ε$ and the joint distribution of a stationary Langevin diffusion at times zero and $ε$ is of the order $o(ε^2)$ with an explicit dependence given by Fisher information. Owing to this inequality, we can show, using a triangular approximation argument, that the interpolated iterated application of the Schrödinger bridge approximation converge to the Wasserstein gradient flow, for a class of gradient flows, including the heat flow. The results also provide a probabilistic and rigorous framework for the convergence of the self-attention mechanisms in transformer networks to the solutions of heat flows, first observed in the inspiring work SABP22 in machine learning research. △ Less

Submitted 16 June, 2024; originally announced June 2024.

Comments: 36 pages, 1 figure

MSC Class: 49N99; 49Q22; 60J60

Quasi-Newton acceleration of EM and MM algorithms via Broyden$'$s method

Authors: Medha Agarwal, Jason Xu

Abstract: The principle of majorization-minimization (MM) provides a general framework for eliciting effective algorithms to solve optimization problems. However, they often suffer from slow convergence, especially in large-scale and high-dimensional data settings. This has drawn attention to acceleration schemes designed exclusively for MM algorithms, but many existing designs are either problem-specific o… ▽ More The principle of majorization-minimization (MM) provides a general framework for eliciting effective algorithms to solve optimization problems. However, they often suffer from slow convergence, especially in large-scale and high-dimensional data settings. This has drawn attention to acceleration schemes designed exclusively for MM algorithms, but many existing designs are either problem-specific or rely on approximations and heuristics loosely inspired by the optimization literature. We propose a novel, rigorous quasi-Newton method for accelerating any valid MM algorithm, cast as seeking a fixed point of the MM \textit{algorithm map}. The method does not require specific information or computation from the objective function or its gradient and enjoys a limited-memory variant amenable to efficient computation in high-dimensional settings. By connecting our approach to Broyden's classical root-finding methods, we establish convergence guarantees and identify conditions for linear and super-linear convergence. These results are validated numerically and compared to peer methods in a thorough empirical study, showing that it achieves state-of-the-art performance across a diverse range of problems. △ Less

Submitted 15 January, 2022; originally announced January 2022.

Comments: 41 pages, 7 pages appendix, 4 figures, 7 tables; associated code for examples can be found at https://github.com/medhaaga/Quasi-Newton-accelerated-MM; submitted to Journal of Computational and Graphical Statistics; for R package (GitHub dev version) implementing the method, see https://github.com/medhaaga/quasiNewtonMM

Esca** Saddle Points for Zeroth-order Nonconvex Optimization using Estimated Gradient Descent

Authors: Qinbo Bai, Mridul Agarwal, Vaneet Aggarwal

Abstract: Gradient descent and its variants are widely used in machine learning. However, oracle access of gradient may not be available in many applications, limiting the direct use of gradient descent. This paper proposes a method of estimating gradient to perform gradient descent, that converges to a stationary point for general non-convex optimization problems. Beyond the first-order stationary properti… ▽ More Gradient descent and its variants are widely used in machine learning. However, oracle access of gradient may not be available in many applications, limiting the direct use of gradient descent. This paper proposes a method of estimating gradient to perform gradient descent, that converges to a stationary point for general non-convex optimization problems. Beyond the first-order stationary properties, the second-order stationary properties are important in machine learning applications to achieve better performance. We show that the proposed model-free non-convex optimization algorithm returns an $ε$-second-order stationary point with $\widetilde{O}(\frac{d^{2+\fracθ{2}}}{ε^{8+θ}})$ queries of the function for any arbitrary $θ>0$. △ Less

Submitted 2 October, 2019; originally announced October 2019.

Comments: arXiv admin note: text overlap with arXiv:1703.00887 by other authors

Composition formulas of Bessel-Struve kernel function

Authors: K. S Nisar, S. R. Mondal. P. Agarwal

Abstract: The generalized operators of fractional integration involving Appell's function $F_{3}(.) $ due to Marichev-Saigo-Maeda, is applied to the Bessel Struve kernel function $S_{α}\left( λz\right),λ,z\in \mathbb{C}$ to obtain the results in terms of generalized Wright functions.The pathway integral representations Bessel Struve kernel function and its relation between many other functions also derived… ▽ More The generalized operators of fractional integration involving Appell's function $F_{3}(.) $ due to Marichev-Saigo-Maeda, is applied to the Bessel Struve kernel function $S_{α}\left( λz\right),λ,z\in \mathbb{C}$ to obtain the results in terms of generalized Wright functions.The pathway integral representations Bessel Struve kernel function and its relation between many other functions also derived in this study. △ Less

Submitted 31 January, 2016; originally announced February 2016.

Comments: 13 pages

MSC Class: 05C38; 15A15; 05A15; 15A18

Optimal Release Time Decision from Fuzzy Mathematical Programming Perspective

Authors: Arvind Kumar, Adarsh Anand, Pankaj Kumar Garg, Mohini Agarwal

Abstract: Demand for high software reliability requires rigorous testing followed by requirement of robust modeling techniques for software quality prediction. On one side, firms have to steadily manage the reliability by testing it vigorously, the optimal release time determination is their biggest concern. In past many models have been developed and much research has been devoted towards assessment of rel… ▽ More Demand for high software reliability requires rigorous testing followed by requirement of robust modeling techniques for software quality prediction. On one side, firms have to steadily manage the reliability by testing it vigorously, the optimal release time determination is their biggest concern. In past many models have been developed and much research has been devoted towards assessment of release time of software. However, majority of the work deals in crisp study. This paper addresses the problem of release time prediction using fuzzy Logic. Here we have formulated a Fuzzy release time problem considering the cost of testing under the impact of warranty period. Results show that fuzzy model has good adaptability. △ Less

Submitted 27 September, 2015; originally announced September 2015.

Comments: 10 Pages. arXiv admin note: substantial overlap with text by other authors http://archive.org/stream/Software_Reliability_Assessment_with_OR_Applications/Software_Reliability_Assessment_with_OR_Applications_djvu.txt

MSC Class: 90C29 ? 90C70 ? 90C90

Journal ref: International Journal of Pure and Applied Mathematics, Volume 103 No. 2 2015, 359-376

On convex problems in chance-constrained stochastic model predictive control

Authors: Eugenio Cinquemani, Mayank Agarwal, Debasish Chatterjee, John Lygeros

Abstract: We investigate constrained optimal control problems for linear stochastic dynamical systems evolving in discrete time. We consider minimization of an expected value cost over a finite horizon. Hard constraints are introduced first, and then reformulated in terms of probabilistic constraints. It is shown that, for a suitable parametrization of the control policy, a wide class of the resulting opt… ▽ More We investigate constrained optimal control problems for linear stochastic dynamical systems evolving in discrete time. We consider minimization of an expected value cost over a finite horizon. Hard constraints are introduced first, and then reformulated in terms of probabilistic constraints. It is shown that, for a suitable parametrization of the control policy, a wide class of the resulting optimization problems are convex, or admit reasonable convex approximations. △ Less

Submitted 21 May, 2009; originally announced May 2009.

MSC Class: 93C05; 93E03