Showing 1–6 of 6 results for author: Ablaev, S

About some works of Boris Polyak on convergence of gradient methods and their development

Authors: Seydamet Ablaev, Aleksandr Beznosikov, Alexander Gasnikov, Darina Dvinskikh, Aleksandr Lobanov, Sergei Puchinin, Fedor Stonyakin

Abstract: The paper presents a review of the state-of-the-art of subgradient and accelerated methods of convex optimization, including in the presence of disturbances and access to various information about the objective function (function value, gradient, stochastic gradient, higher derivatives). For nonconvex problems, the Polak-Lojasiewicz condition is considered and a review of the main results is given… ▽ More The paper presents a review of the state-of-the-art of subgradient and accelerated methods of convex optimization, including in the presence of disturbances and access to various information about the objective function (function value, gradient, stochastic gradient, higher derivatives). For nonconvex problems, the Polak-Lojasiewicz condition is considered and a review of the main results is given. The behavior of numerical methods in the presence of sharp minima is considered. The purpose of this survey is to show the influence of the works of B.T. Polyak (1935 -- 2023) on gradient optimization methods and their neighborhoods on the modern development of numerical optimization methods. △ Less

Submitted 28 November, 2023; originally announced November 2023.

Comments: in Russian language

Adaptive Methods or Variational Inequalities with Relatively Smooth and Reletively Strongly Monotone Operators

Authors: S. S. Ablaev, F. S. Stonyakin, M. S. Alkousa, D. A. Pasechnyuk

Abstract: The article is devoted to some adaptive methods for variational inequalities with relatively smooth and relatively strongly monotone operators. Starting from the recently proposed proximal variant of the extragradient method for this class of problems, we investigate in detail the method with adaptively selected parameter values. An estimate of the convergence rate of this method is proved. The re… ▽ More The article is devoted to some adaptive methods for variational inequalities with relatively smooth and relatively strongly monotone operators. Starting from the recently proposed proximal variant of the extragradient method for this class of problems, we investigate in detail the method with adaptively selected parameter values. An estimate of the convergence rate of this method is proved. The result is generalized to a class of variational inequalities with relatively strongly monotone generalized smooth variational inequality operators. Numerical experiments have been performed for the problem of ridge regression and variational inequality associated with box-simplex games. △ Less

Submitted 1 August, 2023; originally announced August 2023.

Comments: in Russian language

Subgradient methods for non-smooth optimization problems with some relaxation of sharp minimum

Authors: S. S. Ablaev, D. V. Makarenko, F. S. Stonyakin, M. S. Alkousa, I. V. Baran

Abstract: In this paper we propose a generalized condition for a sharp minimum, somewhat similar to the inexact oracle proposed recently by Devolder-Glineur-Nesterov. The proposed approach makes it possible to extend the class of applicability of subgradient methods with the Polyak step-size, to the situation of inexact information about the value of the minimum, as well as the unknown Lipschitz constant of… ▽ More In this paper we propose a generalized condition for a sharp minimum, somewhat similar to the inexact oracle proposed recently by Devolder-Glineur-Nesterov. The proposed approach makes it possible to extend the class of applicability of subgradient methods with the Polyak step-size, to the situation of inexact information about the value of the minimum, as well as the unknown Lipschitz constant of the objective function. Moreover, the use of local analogs of the global characteristics of the objective function makes it possible to apply the results of this type to wider classes of problems. We show the possibility of applying the proposed approach to strongly convex non-smooth problems, also, we make an experimental comparison with the known optimal subgradient method for such a class of problems. Moreover, there were obtained some results connected to the applicability of the proposed technique to some types of problems with convexity relaxations: the recently proposed notion of weak $β$-quasi-convexity and ordinary quasi-convexity. Also in the paper, we study a generalization of the described technique to the situation with the assumption that the $δ$-subgradient of the objective function is available instead of the usual subgradient. For one of the considered methods, conditions are found under which, in practice, it is possible to escape the projection of the considered iterative sequence onto the feasible set of the problem. △ Less

Submitted 12 December, 2022; originally announced December 2022.

Comments: in Russian language

Some Adaptive First-order Methods for Variational Inequalities with Relatively Strongly Monotone Operators and Generalized Smoothness

Authors: A. A. Titov, S. S. Ablaev, M. S. Alkousa, F. S. Stonyakin, A. V. Gasnikov

Abstract: In this paper, we introduce some adaptive methods for solving variational inequalities with relatively strongly monotone operators. Firstly, we focus on the modification of the recently proposed, in smooth case [1], adaptive numerical method for generalized smooth (with Hölder condition) saddle point problem, which has convergence rate estimates similar to accelerated methods. We provide the motiv… ▽ More In this paper, we introduce some adaptive methods for solving variational inequalities with relatively strongly monotone operators. Firstly, we focus on the modification of the recently proposed, in smooth case [1], adaptive numerical method for generalized smooth (with Hölder condition) saddle point problem, which has convergence rate estimates similar to accelerated methods. We provide the motivation for such an approach and obtain theoretical results of the proposed method. Our second focus is the adaptation of widespread recently proposed methods for solving variational inequalities with relatively strongly monotone operators. The key idea in our approach is the refusal of the well-known restart technique, which in some cases causes difficulties in implementing such algorithms for applied problems. Nevertheless, our algorithms show a comparable rate of convergence with respect to algorithms based on the above-mentioned restart technique. Also, we present some numerical experiments, which demonstrate the effectiveness of the proposed methods. [1] **, Y., Sidford, A., & Tian, K. (2022). Sharper rates for separable minimax and finite sum optimization via primal-dual extragradient methods. arXiv preprint arXiv:2202.04640. △ Less

Submitted 28 October, 2022; v1 submitted 19 July, 2022; originally announced July 2022.

Adaptive Gradient-type Methods for Convex Optimization Problems with Relative Accuracy and Sharp Minimum

Authors: Fedor S. Stonyakin, Seydamet S. Ablaev, Inna V. Baran

Abstract: In this paper, we consider gradient-type methods for convex positively homogeneous optimization problems with relative accuracy. An analogue of the accelerated universal gradient-type method for positively homogeneous optimization problems with relative accuracy is investigated. The second approach is related to subgradient methods with B. T. Polyak stepsize. Result on the linear convergence rate… ▽ More In this paper, we consider gradient-type methods for convex positively homogeneous optimization problems with relative accuracy. An analogue of the accelerated universal gradient-type method for positively homogeneous optimization problems with relative accuracy is investigated. The second approach is related to subgradient methods with B. T. Polyak stepsize. Result on the linear convergence rate for some methods of this type with adaptive step adjustment is obtained for some class of non-smooth problems. Some generalization to a special class of non-convex non-smooth problems is also considered. △ Less

Submitted 12 December, 2021; v1 submitted 31 March, 2021; originally announced March 2021.

Comments: in Russian

Analogues of Switching Subgradient Schemes for Relatively Lipschitz-Continuous Convex Programming Problems

Authors: Alexander Titov, Fedor Stonyakin, Mohammad Alkousa, Seydamet Ablaev, Alexander Gasnikov

Abstract: Recently some specific classes of non-smooth and non-Lipschitz convex optimization problems were selected by Yu.~Nesterov along with H.~Lu. We consider convex programming problems with similar smoothness conditions for the objective function and functional constraints. We introduce a new concept of an inexact model and propose some analogues of switching subgradient schemes for convex programming… ▽ More Recently some specific classes of non-smooth and non-Lipschitz convex optimization problems were selected by Yu.~Nesterov along with H.~Lu. We consider convex programming problems with similar smoothness conditions for the objective function and functional constraints. We introduce a new concept of an inexact model and propose some analogues of switching subgradient schemes for convex programming problems for the relatively Lipschitz-continuous objective function and functional constraints. Some class of online convex optimization problems is considered. The proposed methods are optimal in the class of optimization problems with relatively Lipschitz-continuous objective and functional constraints. △ Less

Submitted 6 May, 2021; v1 submitted 20 March, 2020; originally announced March 2020.