-
Approaches to iterative algorithms for solving nonlinear equations with an application in tomographic absorption spectroscopy
Authors:
F. J. Aragón-Artacho,
W. Cai,
Y. Censor,
A. Gibali,
C. Shui,
D. Torregrosa-Belén
Abstract:
In this paper we propose an approach for solving systems of nonlinear equations without computing function derivatives. Motivated by the application area of tomographic absorption spectroscopy, which is a highly-nonlinear problem with variables coupling, we consider a situation where straightforward translation to a fixed point problem is not possible because the operators that represent the relev…
▽ More
In this paper we propose an approach for solving systems of nonlinear equations without computing function derivatives. Motivated by the application area of tomographic absorption spectroscopy, which is a highly-nonlinear problem with variables coupling, we consider a situation where straightforward translation to a fixed point problem is not possible because the operators that represent the relevant systems of nonlinear equations are not self-map**s, i.e., they operate between spaces of different dimensions. To overcome this difficulty we suggest an "alternating common fixed points algorithm" that acts alternatingly on the different vector variables. This approach translates the original problem to a common fixed point problem for which iterative algorithms are abound and exhibits a viable alternative to translation to an optimization problem, which usually requires derivatives information. However, to apply any of these iterative algorithms requires to ascertain the conditions that appear in their convergence theorems. To circumvent the need to verify conditions for convergence, we propose and motivate a derivative-free algorithm that better suits the tomographic absorption spectroscopy problem at hand and is even further improved by applying to it the superiorization approach. This is presented along with experimental results that demonstrate our approach.
△ Less
Submitted 14 May, 2024;
originally announced May 2024.
-
The $ω$-Condition Number for Optimal Preconditioning and Low Rank Generalized Jacobian Updating
Authors:
Woosuk L. Jung,
David Torregrosa-Belén,
Henry Wolkowicz
Abstract:
Preconditioning is essential in iterative methods for solving linear systems. It is also the implicit objective in updating approximations of Jacobians in optimization methods, e.g., in quasi-Newton methods. Motivated by the latter, we study a nonclassic matrix condition number, the $ω$-condition number. We do this in the context of optimal conditioning for: (i) our application to low rank updatin…
▽ More
Preconditioning is essential in iterative methods for solving linear systems. It is also the implicit objective in updating approximations of Jacobians in optimization methods, e.g., in quasi-Newton methods. Motivated by the latter, we study a nonclassic matrix condition number, the $ω$-condition number. We do this in the context of optimal conditioning for: (i) our application to low rank updating of generalized Jacobians; (ii) iterative methods for linear systems: (iia) clustering of eigenvalues and (iib) convergence rates.
For a positive definite matrix, the $ω$-condition measure is the ratio of the arithmetic and geometric means of the eigenvalues. In particular, our applications concentrate on linear systems with low rank updates of ill-conditioned positive definite matrices. These systems arise in the context of nonsmooth Newton methods using generalized Jacobians. We are able to use optimality conditions and derive explicit formulae for $ω$-optimal preconditioners and preconditioned updates. Connections to partial Cholesky sparse preconditioners are made.
Evaluating or estimating the classical condition number $κ$ can be expensive. We show that the $ω$-condition number can be evaluated explicitly following a Cholesky or LU factorization. Moreover, the simplicity of $ω$ allows for the derivation of formulae for optimal preconditioning in various scenarios, i.e., this avoids the need for expensive algorithmic calculations. Our empirics show that $ω$ estimates the actual condition of a linear system significantly better. Moreover, our empirical results show a significant decrease in the number of iterations required for a requested accuracy in the residual during an iterative method, i.e., these results confirm the efficacy of using the $ω$-condition number compared to the classical condition number.
△ Less
Submitted 23 June, 2024; v1 submitted 25 August, 2023;
originally announced August 2023.
-
The Boosted Double-Proximal Subgradient Algorithm for Nonconvex Optimization
Authors:
Francisco J. Aragón-Artacho,
Pedro Pérez-Aros,
David Torregrosa-Belén
Abstract:
In this paper we introduce the Boosted Double-proximal Subgradient Algorithm (BDSA), a novel splitting algorithm designed to address general structured nonsmooth and nonconvex mathematical programs expressed as sums and differences of composite functions. BDSA exploits the combined nature of subgradients from the data and proximal steps, and integrates a line-search procedure to enhance its perfor…
▽ More
In this paper we introduce the Boosted Double-proximal Subgradient Algorithm (BDSA), a novel splitting algorithm designed to address general structured nonsmooth and nonconvex mathematical programs expressed as sums and differences of composite functions. BDSA exploits the combined nature of subgradients from the data and proximal steps, and integrates a line-search procedure to enhance its performance. While BDSA encompasses existing schemes proposed in the literature, it extends its applicability to more diverse problem domains. We establish the convergence of BDSA under the Kurdyka--Lojasiewicz property and provide an analysis of its convergence rate. To evaluate the effectiveness of BDSA, we introduce a novel family of challenging test functions with an abundance of critical points. We conduct comparative evaluations demonstrating its ability to effectively escape non-optimal critical points. Additionally, we present two practical applications of BDSA for testing its efficacy, namely, a constrained minimum-sum-of-squares clustering problem and a nonconvex generalization of Heron's problem.
△ Less
Submitted 29 June, 2023;
originally announced June 2023.
-
The superiorization method with restarted perturbations for split minimization problems with an application to radiotherapy treatment planning
Authors:
Francisco J. Aragón-Artacho,
Yair Censor,
Aviv Gibali,
David Torregrosa-Belén
Abstract:
In this paper we study the split minimization problem that consists of two constrained minimization problems in two separate spaces that are connected via a linear operator that maps one space into the other. To handle the data of such a problem we develop a superiorization approach that can reach a feasible point with reduced (not necessarily minimal) objective function values. The superiorizatio…
▽ More
In this paper we study the split minimization problem that consists of two constrained minimization problems in two separate spaces that are connected via a linear operator that maps one space into the other. To handle the data of such a problem we develop a superiorization approach that can reach a feasible point with reduced (not necessarily minimal) objective function values. The superiorization methodology is based on interlacing the iterative steps of two separate and independent iterative processes by perturbing the iterates of one process according to the steps dictated by the other process. We include in our developed method two novel elements. The first one is the permission to restart the perturbations in the superiorized algorithm which results in a significant acceleration and increases the computational efficiency. The second element is the ability to independently superiorize subvectors. This caters to the needs of real-world applications, as demonstrated here for a problem in intensity-modulated radiation therapy treatment planning.
△ Less
Submitted 3 May, 2024; v1 submitted 12 July, 2022;
originally announced July 2022.
-
A primal-dual splitting algorithm for composite monotone inclusions with minimal lifting
Authors:
Francisco J. Aragón-Artacho,
Radu I. Boţ,
David Torregrosa-Belén
Abstract:
In this work, we study resolvent splitting algorithms for solving composite monotone inclusion problems. The objective of these general problems is finding a zero in the sum of maximally monotone operators composed with linear operators. Our main contribution is establishing the first primal-dual splitting algorithm for composite monotone inclusions with minimal lifting. Specifically, the proposed…
▽ More
In this work, we study resolvent splitting algorithms for solving composite monotone inclusion problems. The objective of these general problems is finding a zero in the sum of maximally monotone operators composed with linear operators. Our main contribution is establishing the first primal-dual splitting algorithm for composite monotone inclusions with minimal lifting. Specifically, the proposed scheme reduces the dimension of the product space where the underlying fixed point operator is defined, in comparison to other algorithms, without requiring additional evaluations of the resolvent operators. We prove the convergence of this new algorithm and analyze its performance in a problem arising in image deblurring and denoising. This work also contributes to the theory of resolvent splitting algorithms by extending the minimal lifting theorem recently proved by Malitsky and Tam to schemes with resolvent parameters.
△ Less
Submitted 19 February, 2022;
originally announced February 2022.
-
Distributed Forward-Backward Methods for Ring Networks
Authors:
Francisco J. Aragón-Artacho,
Yura Malitsky,
Matthew K. Tam,
David Torregrosa-Belén
Abstract:
In this work, we propose and analyse forward-backward-type algorithms for finding a zero of the sum of finitely many monotone operators, which are not based on reduction to a two operator inclusion in the product space. Each iteration of the studied algorithms requires one resolvent evaluation per set-valued operator, one forward evaluation per cocoercive operator, and two forward evaluations per…
▽ More
In this work, we propose and analyse forward-backward-type algorithms for finding a zero of the sum of finitely many monotone operators, which are not based on reduction to a two operator inclusion in the product space. Each iteration of the studied algorithms requires one resolvent evaluation per set-valued operator, one forward evaluation per cocoercive operator, and two forward evaluations per monotone operator. Unlike existing methods, the structure of the proposed algorithms are suitable for distributed, decentralised implementation in ring networks without needing global summation to enforce consensus between nodes.
△ Less
Submitted 12 July, 2022; v1 submitted 30 November, 2021;
originally announced December 2021.
-
A direct proof of convergence of Davis-Yin splitting algorithm allowing larger stepsizes
Authors:
Francisco J. Aragón-Artacho,
David Torregrosa-Belén
Abstract:
This note is devoted to the splitting algorithm proposed by Davis and Yin in 2017 for computing a zero of the sum of three maximally monotone operators, with one of them being cocoercive. We provide a direct proof that guarantees its convergence when the stepsizes are smaller than four times the cocoercivity constant, thus doubling the size of the interval established by Davis and Yin. As a by-pro…
▽ More
This note is devoted to the splitting algorithm proposed by Davis and Yin in 2017 for computing a zero of the sum of three maximally monotone operators, with one of them being cocoercive. We provide a direct proof that guarantees its convergence when the stepsizes are smaller than four times the cocoercivity constant, thus doubling the size of the interval established by Davis and Yin. As a by-product, the same conclusion applies to the forward-backward splitting algorithm. Further, we use the notion of "strengthening" of a set-valued operator to derive a new splitting algorithm for computing the resolvent of the sum. Last but not least, we provide some numerical experiments illustrating the importance of appropriately choosing the stepsize and relaxation parameters of the algorithms.
△ Less
Submitted 17 January, 2022; v1 submitted 3 August, 2021;
originally announced August 2021.