Non-Smooth Setting of Stochastic Decentralized Convex Optimization Problem Over Time-Varying Graphs
Authors:
Aleksandr Lobanov,
Andrew Veprikov,
Georgiy Konin,
Aleksandr Beznosikov,
Alexander Gasnikov,
Dmitry Kovalev
Abstract:
Distributed optimization has a rich history. It has demonstrated its effectiveness in many machine learning applications, etc. In this paper we study a subclass of distributed optimization, namely decentralized optimization in a non-smooth setting. Decentralized means that $m$ agents (machines) working in parallel on one problem communicate only with the neighbors agents (machines), i.e. there is…
▽ More
Distributed optimization has a rich history. It has demonstrated its effectiveness in many machine learning applications, etc. In this paper we study a subclass of distributed optimization, namely decentralized optimization in a non-smooth setting. Decentralized means that $m$ agents (machines) working in parallel on one problem communicate only with the neighbors agents (machines), i.e. there is no (central) server through which agents communicate. And by non-smooth setting we mean that each agent has a convex stochastic non-smooth function, that is, agents can hold and communicate information only about the value of the objective function, which corresponds to a gradient-free oracle. In this paper, to minimize the global objective function, which consists of the sum of the functions of each agent, we create a gradient-free algorithm by applying a smoothing scheme via $l_2$ randomization. We also verify in experiments the obtained theoretical convergence results of the gradient-free algorithm proposed in this paper.
△ Less
Submitted 5 September, 2023; v1 submitted 1 July, 2023;
originally announced July 2023.
