-
Comparing the Methods of Alternating and Simultaneous Projections for Two Subspaces
Authors:
Simeon Reich,
Rafał Zalas
Abstract:
We study the well-known methods of alternating and simultaneous projections when applied to two nonorthogonal linear subspaces of a real Euclidean space. Assuming that both of the methods have a common starting point chosen from either one of the subspaces, we show that the method of alternating projections converges significantly faster than the method of simultaneous projections. On the other ha…
▽ More
We study the well-known methods of alternating and simultaneous projections when applied to two nonorthogonal linear subspaces of a real Euclidean space. Assuming that both of the methods have a common starting point chosen from either one of the subspaces, we show that the method of alternating projections converges significantly faster than the method of simultaneous projections. On the other hand, we provide examples of subspaces and starting points, where the method of simultaneous projections outperforms the method of alternating projections.
△ Less
Submitted 14 November, 2023; v1 submitted 21 June, 2023;
originally announced June 2023.
-
Polynomial Estimates for the Method of Cyclic Projections in Hilbert Spaces
Authors:
Simeon Reich,
Rafał Zalas
Abstract:
We study the method of cyclic projections when applied to closed and linear subspaces $M_i$, $i=1,\ldots,m$, of a real Hilbert space $\mathcal H$. We show that the average distance to individual sets enjoys a polynomial behaviour $o(k^{-1/2})$ along the trajectory of the generated iterates. Surprisingly, when the starting points are chosen from the subspace $\sum_{i=1}^{m}M_i^\perp$, our result yi…
▽ More
We study the method of cyclic projections when applied to closed and linear subspaces $M_i$, $i=1,\ldots,m$, of a real Hilbert space $\mathcal H$. We show that the average distance to individual sets enjoys a polynomial behaviour $o(k^{-1/2})$ along the trajectory of the generated iterates. Surprisingly, when the starting points are chosen from the subspace $\sum_{i=1}^{m}M_i^\perp$, our result yields a polynomial rate of convergence $\mathcal O(k^{-1/2})$ for the method of cyclic projections itself. Moreover, if $\sum_{i=1}^{m} M_i^\perp$ is not closed, then both of the aforementioned rates are best possible in the sense that the corresponding polynomial $k^{1/2}$ cannot be replaced by $k^{1/2+\varepsilon}$ for any $\varepsilon >0$.
△ Less
Submitted 17 April, 2023; v1 submitted 27 May, 2022;
originally announced May 2022.
-
Finitely Convergent Iterative Methods with Overrelaxations Revisited
Authors:
Victor I. Kolobov,
Simeon Reich,
Rafał Zalas
Abstract:
We study the finite convergence of iterative methods for solving convex feasibility problems. Our key assumptions are that the interior of the solution set is nonempty and that certain overrelaxation parameters converge to zero, but with a rate slower than any geometric sequence. Unlike other works in this area, which require divergent series of overrelaxations, our approach allows us to consider…
▽ More
We study the finite convergence of iterative methods for solving convex feasibility problems. Our key assumptions are that the interior of the solution set is nonempty and that certain overrelaxation parameters converge to zero, but with a rate slower than any geometric sequence. Unlike other works in this area, which require divergent series of overrelaxations, our approach allows us to consider some summable series. By employing quasi-Fejérian analysis in the latter case, we obtain additional asymptotic convergence guarantees, even when the interior of the solution set is empty.
△ Less
Submitted 11 July, 2021; v1 submitted 31 January, 2021;
originally announced February 2021.
-
Error Bounds for the Method of Simultaneous Projections with Infinitely Many Subspaces
Authors:
Simeon Reich,
Rafał Zalas
Abstract:
We investigate the properties of the simultaneous projection method as applied to countably infinitely many closed and linear subspaces of a real Hilbert space. We establish the optimal error bound for linear convergence of this method, which we express in terms of the cosine of the Friedrichs angle computed in an infinite product space. In addition, we provide estimates and alternative expression…
▽ More
We investigate the properties of the simultaneous projection method as applied to countably infinitely many closed and linear subspaces of a real Hilbert space. We establish the optimal error bound for linear convergence of this method, which we express in terms of the cosine of the Friedrichs angle computed in an infinite product space. In addition, we provide estimates and alternative expressions for the above-mentioned number. Furthermore, we relate this number to the dichotomy theorem and to super-polynomially fast convergence. We also discuss polynomial convergence of the simultaneous projection method which takes place for particularly chosen starting points.
△ Less
Submitted 29 August, 2021; v1 submitted 21 July, 2020;
originally announced July 2020.
-
Outer Approximation Methods for Solving Variational Inequalities Defined over the Solution Set of a Split Convex Feasibility Problem
Authors:
Andrzej Cegielski,
Aviv Gibali,
Simeon Reich,
Rafał Zalas
Abstract:
We study variational inequalities which are governed by a strongly monotone and Lipschitz continuous operator $F$ over a closed and convex set $S$. We assume that $S=C\cap A^{-1}(Q)$ is the nonempty solution set of a (multiple-set) split convex feasibility problem, where $C$ and $Q$ are both closed and convex subsets of two real Hilbert spaces $\mathcal H_1$ and $\mathcal H_2$, respectively, and t…
▽ More
We study variational inequalities which are governed by a strongly monotone and Lipschitz continuous operator $F$ over a closed and convex set $S$. We assume that $S=C\cap A^{-1}(Q)$ is the nonempty solution set of a (multiple-set) split convex feasibility problem, where $C$ and $Q$ are both closed and convex subsets of two real Hilbert spaces $\mathcal H_1$ and $\mathcal H_2$, respectively, and the operator $A$ acting between them is linear. We consider a modification of the gradient projection method the main idea of which is to replace at each step the metric projection onto $S$ by another metric projection onto a half-space which contains $S$. We propose three variants of a method for constructing the above-mentioned half-spaces by employing the multiple-set and the split structure of the set $S$. For the split part we make use of the Landweber transform.
△ Less
Submitted 20 August, 2019;
originally announced August 2019.
-
Finitely Convergent Deterministic and Stochastic Iterative Methods for Solving Convex Feasibility Problems
Authors:
Victor I. Kolobov,
Simeon Reich,
Rafał Zalas
Abstract:
We propose finitely convergent methods for solving convex feasibility problems defined over a possibly infinite pool of constraints. Following other works in this area, we assume that the interior of the solution set is nonempty and that certain overrelaxation parameters form a divergent series. We combine our methods with a very general class of deterministic control sequences where, roughly spea…
▽ More
We propose finitely convergent methods for solving convex feasibility problems defined over a possibly infinite pool of constraints. Following other works in this area, we assume that the interior of the solution set is nonempty and that certain overrelaxation parameters form a divergent series. We combine our methods with a very general class of deterministic control sequences where, roughly speaking, we require that sooner or later we encounter a violated constraint if one exists. This requirement is satisfied, in particular, by the cyclic, repetitive and remotest set controls. Moreover, it is almost surely satisfied for random controls.
△ Less
Submitted 20 September, 2020; v1 submitted 14 May, 2019;
originally announced May 2019.
-
Weak, Strong and Linear Convergence of the CQ-Method Via the Regularity of Landweber Operators
Authors:
Andrzej Cegielski,
Simeon Reich,
Rafał Zalas
Abstract:
We consider the split convex feasibility problem in a fixed point setting. Motivated by the well-known CQ-method of Byrne (2002), we define an abstract andweber transform which applies to more general operators than the metric projection. We call the result of this transform a Landweber operator. It turns out that the Landweber transform preserves many interesting properties. For example, the Land…
▽ More
We consider the split convex feasibility problem in a fixed point setting. Motivated by the well-known CQ-method of Byrne (2002), we define an abstract andweber transform which applies to more general operators than the metric projection. We call the result of this transform a Landweber operator. It turns out that the Landweber transform preserves many interesting properties. For example, the Landweber transform of a (quasi/firmly) nonexpansive map** is again (quasi/firmly) nonexpansive. Moreover, the Landweber transform of a (weakly/linearly) regular map** is again (weakly/linearly) regular. The preservation of regularity is important because it leads to (weak/linear) convergence of many CQ-type methods.
△ Less
Submitted 18 December, 2018;
originally announced December 2018.
-
Linear Convergence Rates for Extrapolated Fixed Point Algorithms
Authors:
Christian Bargetz,
Victor I. Kolobov,
Simeon Reich,
Rafał Zalas
Abstract:
We establish linear convergence rates for a certain class of extrapolated fixed point algorithms which are based on dynamic string-averaging methods in a real Hilbert space. This applies, in particular, to the extrapolated simultaneous and cyclic cutter methods. Our analysis covers the cases of both metric and subgradient projections.
We establish linear convergence rates for a certain class of extrapolated fixed point algorithms which are based on dynamic string-averaging methods in a real Hilbert space. This applies, in particular, to the extrapolated simultaneous and cyclic cutter methods. Our analysis covers the cases of both metric and subgradient projections.
△ Less
Submitted 10 May, 2018;
originally announced May 2018.
-
Regular Sequences of Quasi-Nonexpansive Operators and Their Applications
Authors:
Andrzej Cegielski,
Simeon Reich,
Rafał Zalas
Abstract:
In this paper we present a systematic study of regular sequences of quasi-nonexpansive operators in Hilbert space. We are interested, in particular, in weakly, boundedly and linearly regular sequences of operators. We show that the type of the regularity is preserved under relaxations, convex combinations and products of operators. Moreover, in this connection, we show that weak, bounded and linea…
▽ More
In this paper we present a systematic study of regular sequences of quasi-nonexpansive operators in Hilbert space. We are interested, in particular, in weakly, boundedly and linearly regular sequences of operators. We show that the type of the regularity is preserved under relaxations, convex combinations and products of operators. Moreover, in this connection, we show that weak, bounded and linear regularity lead to weak, strong and linear convergence, respectively, of various iterative methods. This applies, in particular, to block iterative and string averaging projection methods, which, in principle, are based on the above-mentioned algebraic operations applied to projections. Finally, we show an application of regular sequences of operators to variational inequality problems.
△ Less
Submitted 10 February, 2018; v1 submitted 2 October, 2017;
originally announced October 2017.
-
The Optimal Error Bound for the Method of Simultaneous Projections
Authors:
Simeon Reich,
Rafał Zalas
Abstract:
In this paper we find the optimal error bound (smallest possible estimate, independent of the starting point) for the linear convergence rate of the simultaneous projection method applied to closed linear subspaces in a real Hilbert space. We achieve this by computing the norm of an error operator which we also express in terms of the Friedrichs number. We compare our estimate with the optimal one…
▽ More
In this paper we find the optimal error bound (smallest possible estimate, independent of the starting point) for the linear convergence rate of the simultaneous projection method applied to closed linear subspaces in a real Hilbert space. We achieve this by computing the norm of an error operator which we also express in terms of the Friedrichs number. We compare our estimate with the optimal one provided for the alternating projection method by Kayalar and Weinert (1988). Moreover, we relate our result to the alternating projection formalization of Pierra (1984) in a product space. Finally, we adjust our results to closed affine subspaces and put them in context with recent dichotomy theorems.
△ Less
Submitted 14 September, 2017; v1 submitted 2 April, 2017;
originally announced April 2017.
-
Weak, Strong and Linear Convergence of a Double-Layer Fixed Point Algorithm
Authors:
Victor I. Kolobov,
Simeon Reich,
Rafał Zalas
Abstract:
In this article we consider a consistent convex feasibility problem in a real Hilbert space defined by a finite family of sets $C_i$. We are interested, in particular, in the case where for each $i$, $C_i=Fix (U_i)=\{z\in \mathcal H\mid p_i(z)=0\}$, $U_i\colon\mathcal H\rightarrow \mathcal H$ is a cutter and $p_i\colon\mathcal H\rightarrow [0,\infty)$ is a proximity function. Moreover, we make the…
▽ More
In this article we consider a consistent convex feasibility problem in a real Hilbert space defined by a finite family of sets $C_i$. We are interested, in particular, in the case where for each $i$, $C_i=Fix (U_i)=\{z\in \mathcal H\mid p_i(z)=0\}$, $U_i\colon\mathcal H\rightarrow \mathcal H$ is a cutter and $p_i\colon\mathcal H\rightarrow [0,\infty)$ is a proximity function. Moreover, we make the following assumption: the computation of $p_i$ is at most as difficult as the evaluation of $U_i$ and this is at most as difficult as projecting onto $C_i$. We study a double-layer fixed point algorithm which applies two types of controls in every iteration step. The first one -- the outer control -- is assumed to be almost cyclic. The second one -- the inner control -- determines the most important sets from those offered by the first one. The selection is made in terms of proximity functions. The convergence results presented in this manuscript depend on the conditions which first, bind together the sets, the operators and the proximity functions and second, connect the inner and outer controls. In particular, weak regularity (demi-closedness principle), bounded regularity and bounded linear regularity imply weak, strong and linear convergence of our algorithm, respectively. The framework presented in this paper covers many known (subgradient) projection algorithms already existing in the literature; for example, those applied with (almost) cyclic, remotest-set, maximum displacement, most-violated constraint and simultaneous controls. In addition, we provide several new examples, where the double-layer approach indeed accelerates the convergence speed as we demonstrate numerically.
△ Less
Submitted 28 March, 2017;
originally announced March 2017.
-
Convergence Properties of Dynamic String Averaging Projection Methods in the Presence of Perturbations
Authors:
Christian Bargetz,
Simeon Reich,
Rafał Zalas
Abstract:
Assuming that the absence of perturbations guarantees weak or strong convergence to a common fixed point, we study the behavior of perturbed products of an infinite family of nonexpansive operators. Our main result indicates that the convergence rate of unperturbed products is essentially preserved in the presence of perturbations. This, in particular, applies to the linear convergence rate of dyn…
▽ More
Assuming that the absence of perturbations guarantees weak or strong convergence to a common fixed point, we study the behavior of perturbed products of an infinite family of nonexpansive operators. Our main result indicates that the convergence rate of unperturbed products is essentially preserved in the presence of perturbations. This, in particular, applies to the linear convergence rate of dynamic string averaging projection methods, which we establish here as well. Moreover, we show how this result can be applied to the superiorization methodology.
△ Less
Submitted 22 March, 2017;
originally announced March 2017.
-
Sparsity constrained split feasibility for dose-volume constraints in inverse planning of intensity-modulated photon or proton therapy
Authors:
S. Penfold,
R. Zalas,
M. Casiraghi,
M. Brooke,
Y. Censor,
R. Schulte
Abstract:
A split feasibility formulation for the inverse problem of intensity-modulated radiation therapy (IMRT) treatment planning with dose-volume constraints (DVCs) included in the planning algorithm is presented. It involves a new type of sparsity constraint that enables the inclusion of a percentage-violation constraint in the model problem and its handling by continuous (as opposed to integer) method…
▽ More
A split feasibility formulation for the inverse problem of intensity-modulated radiation therapy (IMRT) treatment planning with dose-volume constraints (DVCs) included in the planning algorithm is presented. It involves a new type of sparsity constraint that enables the inclusion of a percentage-violation constraint in the model problem and its handling by continuous (as opposed to integer) methods. We propose an iterative algorithmic framework for solving such a problem by applying the feasibility-seeking CQ-algorithm of Byrne combined with the automatic relaxation method (ARM) that uses cyclic projections. Detailed implementation instructions are furnished. Functionality of the algorithm was demonstrated through the creation of an intensity-modulated proton therapy plan for a simple 2D C-shaped geometry and also for a realistic base-of-skull chordoma treatment site. Monte Carlo simulations of proton pencil beams of varying energy were conducted to obtain dose distributions for the 2D test case. A research release of the Pinnacle3 proton treatment planning system was used to extract pencil beam doses for a clinical base-of-skull chordoma case. In both cases the beamlet doses were calculated to satisfy dose-volume constraints according to our new algorithm. Examination of the dose-volume histograms following inverse planning with our algorithm demonstrated that it performed as intended. The application of our proposed algorithm to dose-volume constraint inverse planning was successfully demonstrated. Comparison with optimized dose distributions from the research release of the Pinnacle3 treatment planning system showed the algorithm could achieve equivalent or superior results.
△ Less
Submitted 25 February, 2017;
originally announced February 2017.
-
Outer Approximation Methods for Solving Variational Inequalities in Hilbert Space
Authors:
Aviv Gibali,
Simeon Reich,
Rafal Zalas
Abstract:
In this paper we study variational inequalities in a real Hilbert space, which are governed by a strongly monotone and Lipschitz continuous operator $F$ over a closed and convex set $C$. We assume that the set $C$ can be outerly approximated by the fixed point sets of a sequence of certain quasi-nonexpansive operators called cutters. We propose an iterative method the main idea of which is to proj…
▽ More
In this paper we study variational inequalities in a real Hilbert space, which are governed by a strongly monotone and Lipschitz continuous operator $F$ over a closed and convex set $C$. We assume that the set $C$ can be outerly approximated by the fixed point sets of a sequence of certain quasi-nonexpansive operators called cutters. We propose an iterative method the main idea of which is to project at each step onto a particular half-space constructed by using the input data. Our approach is based on a method presented by Fukushima in 1986, which has recently been extended by several authors. In the present paper we establish strong convergence in Hilbert space. We emphasize that to the best of our knowledge, Fukushima's method has so far been considered only in the Euclidean setting with different conditions on $F$. We provide several examples for the case where $C$ is the common fixed point set of a finite number of cutters with numerical illustrations of our theoretical results.
△ Less
Submitted 2 February, 2017;
originally announced February 2017.
-
An algorithm for solving the variational inequality problem over the fixed point set of a quasi-nonexpansive operator in Euclidean space
Authors:
Andrzej Cegielski,
Aviv Gibali,
Simeon Reich,
Rafał Zalas
Abstract:
This paper is concerned with the variational inequality problem (VIP) over the fixed point set of a quasi-nonexpansive operator. We propose, in particular, an algorithm which entails, at each step, projecting onto a suitably chosen half-space, and prove that the sequences it generates converge to the unique solution of the VIP. We also present an application of our result to a hierarchical optimiz…
▽ More
This paper is concerned with the variational inequality problem (VIP) over the fixed point set of a quasi-nonexpansive operator. We propose, in particular, an algorithm which entails, at each step, projecting onto a suitably chosen half-space, and prove that the sequences it generates converge to the unique solution of the VIP. We also present an application of our result to a hierarchical optimization problem.
△ Less
Submitted 2 April, 2013;
originally announced April 2013.
