-
New Douglas-Rashford Splitting Algorithms for Generalized DC Programming with Applications in Machine Learning
Authors:
Yonghong Yao,
Lateef O. Jolaoso,
Yekini Shehu,
Jen-Chih Yao
Abstract:
In this work, we propose some new Douglas-Rashford splitting algorithms for solving a class of generalized DC (difference of convex functions) in real Hilbert spaces. The proposed methods leverage the proximal properties of the nonsmooth component and a fasten control parameter which improves the convergence rate of the algorithms. We prove the convergence of these methods to the critical points o…
▽ More
In this work, we propose some new Douglas-Rashford splitting algorithms for solving a class of generalized DC (difference of convex functions) in real Hilbert spaces. The proposed methods leverage the proximal properties of the nonsmooth component and a fasten control parameter which improves the convergence rate of the algorithms. We prove the convergence of these methods to the critical points of nonconvex optimization under reasonable conditions. We evaluate the performance and effectiveness of our methods through experimentation with three practical examples in machine learning. Our findings demonstrated that our methods offer efficiency in problem-solving and outperform state-of-the-art techniques like the DCA (DC Algorithm) and ADMM.
△ Less
Submitted 23 April, 2024;
originally announced April 2024.
-
Linear Convergence Results for Inertial Type Projection Algorithm for Quasi-Variational Inequalities
Authors:
Yonghong Yao,
Lateef O. Jolaoso,
Yekini Shehu
Abstract:
Many recently proposed gradient projection algorithms with inertial extrapolation step for solving quasi-variational inequalities in Hilbert spaces are proven to be strongly convergent with no linear rate given when the cost operator is strongly monotone and Lipschitz continuous. In this paper, our aim is to design an inertial type gradient projection algorithm for quasi-variational inequalities a…
▽ More
Many recently proposed gradient projection algorithms with inertial extrapolation step for solving quasi-variational inequalities in Hilbert spaces are proven to be strongly convergent with no linear rate given when the cost operator is strongly monotone and Lipschitz continuous. In this paper, our aim is to design an inertial type gradient projection algorithm for quasi-variational inequalities and obtain its linear rate of convergence. Therefore, our results fill in the gap for linear convergence results for inertial type gradient projection algorithms for quasi variational inequalities in Hilbert spaces. We perform numerical implementations of our proposed algorithm and give numerical comparisons with other related inertial type gradient projection algorithms for quasi variational inequalities in the literature.
△ Less
Submitted 22 April, 2024;
originally announced April 2024.
-
A robust regularized extreme learning machine for regression problems based on self-adaptive accelerated extra-gradient algorithm
Authors:
Muideen Adegoke,
Lateef O. Jolaoso,
Mardiyyah Oduwole
Abstract:
The Extreme Learning Machine (ELM) technique is a machine learning approach for constructing feed-forward neural networks with a single hidden layer and their models. The ELM model can be constructed while being trained by concurrently reducing both the modeling errors and the norm of the output weights. Usually, the squared loss is widely utilized in the objective function of ELM, which can be tr…
▽ More
The Extreme Learning Machine (ELM) technique is a machine learning approach for constructing feed-forward neural networks with a single hidden layer and their models. The ELM model can be constructed while being trained by concurrently reducing both the modeling errors and the norm of the output weights. Usually, the squared loss is widely utilized in the objective function of ELM, which can be treated as a LASSO problem and hence applying the fast iterative shrinkage thresholding algorithm (FISTA). However, in this paper, the minimization problem is solved from a variational inequalities perspective giving rise to improved algorithms that are more efficient than the FISTA. A fast general extra-gradient algorithm which is a form of first-order algorithm is developed with inertial acceleration techniques and variable stepsize which is updated at every iteration. The strong convergence and linear rate of convergence of the proposed algorithm are established under mild conditions. In terms of experiments, two experiments were presented to illustrate the computational advantage of the proposed method to other algorithms.
△ Less
Submitted 27 January, 2024;
originally announced January 2024.
-
A fresh look at nonsmooth Levenberg--Marquardt methods with applications to bilevel optimization
Authors:
Lateef O. Jolaoso,
Patrick Mehlitz,
Alain B. Zemkoho
Abstract:
In this paper, we revisit the classical problem of solving over-determined systems of nonsmooth equations numerically. We suggest a nonsmooth Levenberg--Marquardt method for its solution which, in contrast to the existing literature, does not require local Lipschitzness of the data functions. This is possible when using Newton-differentiability instead of semismoothness as the underlying tool of g…
▽ More
In this paper, we revisit the classical problem of solving over-determined systems of nonsmooth equations numerically. We suggest a nonsmooth Levenberg--Marquardt method for its solution which, in contrast to the existing literature, does not require local Lipschitzness of the data functions. This is possible when using Newton-differentiability instead of semismoothness as the underlying tool of generalized differentiation. Conditions for fast local convergence of the method are given. Afterwards, in the context of over-determined mixed nonlinear complementarity systems, our findings are applied, and globalized solution methods, based on a residual induced by the maximum and the Fischer--Burmeister function, respectively, are constructed. The assumptions for fast local convergence are worked out and compared. Finally, these methods are applied for the numerical solution of bilevel optimization problems. We recall the derivation of a stationarity condition taking the shape of an over-determined mixed nonlinear complementarity system involving a penalty parameter, formulate assumptions for local fast convergence of our solution methods explicitly, and present results of numerical experiments. Particularly, we investigate whether the treatment of the appearing penalty parameter as an additional variable is beneficial or not.
△ Less
Submitted 9 November, 2023; v1 submitted 31 May, 2023;
originally announced May 2023.
-
Intercultural Science-Art Project 'Magritte Meet Science'
Authors:
Danilo Gregorin Afonso,
Mary Oluwabunmi Akinade,
Đorđe Baralić,
Demian Nahuel Goos,
Charles Gray,
Abdulganiyu Jimoh,
Lateef Olakunle Jolaoso,
Jie Li,
Sander Mack-Crane,
Aung Zaw Myint,
Kayode Oshinubi,
Matheus Pires Cardoso,
Dragana Radojičić
Abstract:
In this manuscript a joint project between young researchers from all around the world, the Intercultural Science-Art Project, is presented, in which all participants interpreted their own research with art. Art is meant in the most inclusive way possible, including drawings and digital works, but also songs and confectionary art. This project has two main goals. On the one hand it creates tools f…
▽ More
In this manuscript a joint project between young researchers from all around the world, the Intercultural Science-Art Project, is presented, in which all participants interpreted their own research with art. Art is meant in the most inclusive way possible, including drawings and digital works, but also songs and confectionary art. This project has two main goals. On the one hand it creates tools for young scientists to communicate their research to society by artistic means. On the other hand it tries to bring together different cultures and to strengthen ties between them.
△ Less
Submitted 29 September, 2020;
originally announced September 2020.
