Showing 1–50 of 112 results for author: Bauschke, H H

On a result by Baillon, Bruck, and Reich

Authors: Heinz H. Bauschke, Yuan Gao

Abstract: It is well known that the iterates of an averaged nonexpansive map** may only converge weakly to fixed point. A celebrated result by Baillon, Bruck, and Reich from 1978 yields strong convergence in the presence of linearity. In this paper, we extend this result to allow for flexible relaxation parameters. Examples are also provided to illustrate the results. It is well known that the iterates of an averaged nonexpansive map** may only converge weakly to fixed point. A celebrated result by Baillon, Bruck, and Reich from 1978 yields strong convergence in the presence of linearity. In this paper, we extend this result to allow for flexible relaxation parameters. Examples are also provided to illustrate the results. △ Less

Submitted 5 April, 2024; originally announced April 2024.

MSC Class: 47H05; 47H09 (Primary) 47N10; 65K05; 90C25 (Secondary)

On the Bredies-Chenchene-Lorenz-Naldi algorithm

Authors: Heinz H. Bauschke, Walaa M. Moursi, Shambhavi Singh, Xianfu Wang

Abstract: Monotone inclusion problems occur in many areas of optimization and variational analysis. Splitting methods, which utilize resolvents or proximal map**s of the underlying operators, are often applied to solve these problems. In 2022, Bredies, Chenchene, Lorenz, and Naldi introduced a new elegant algorithmic framework that encompasses various well known algorithms including Douglas-Rachford and C… ▽ More Monotone inclusion problems occur in many areas of optimization and variational analysis. Splitting methods, which utilize resolvents or proximal map**s of the underlying operators, are often applied to solve these problems. In 2022, Bredies, Chenchene, Lorenz, and Naldi introduced a new elegant algorithmic framework that encompasses various well known algorithms including Douglas-Rachford and Chambolle-Pock. They obtained powerful weak and strong convergence results, where the latter type relies on additional strong monotonicity assumptions. In this paper, we complement the analysis by Bredies et al. by relating the projections of the fixed point sets of the underlying operators that generate the (reduced and original) preconditioned proximal point sequences. We also obtain strong convergence results in the case of linear relations. Various examples are provided to illustrate the applicability of our results. △ Less

Submitted 19 July, 2023; originally announced July 2023.

How averaged is the composition of two linear projections?

Authors: Heinz H. Bauschke, Theo Bendit, Walaa M. Moursi

Abstract: Projection operators are fundamental algorithmic operators in Analysis and Optimization. It is well known that these operators are firmly nonexpansive; however, their composition is generally only averaged and no longer firmly nonexpansive. In this note, we introduce the modulus of averagedness and provide an exact result for the composition of two linear projection operators. As a consequence, we… ▽ More Projection operators are fundamental algorithmic operators in Analysis and Optimization. It is well known that these operators are firmly nonexpansive; however, their composition is generally only averaged and no longer firmly nonexpansive. In this note, we introduce the modulus of averagedness and provide an exact result for the composition of two linear projection operators. As a consequence, we deduce that the Ogura-Yamada bound for the modulus of the composition is sharp. △ Less

Submitted 23 March, 2023; originally announced March 2023.

MSC Class: 47H09 (Primary) 65K05; 90C25 (Secondary)

Real roots of real cubics and optimization

Authors: Heinz H. Bauschke, Manish Krishan Lal, Xianfu Wang

Abstract: The solution of the cubic equation has a century-long history; however, the usual presentation is geared towards applications in algebra and is somewhat inconvenient to use in optimization where frequently the main interest lies in real roots. In this note, we present the roots of the cubic in a form that makes them convenient to use and we also focus on information on the location of the real roo… ▽ More The solution of the cubic equation has a century-long history; however, the usual presentation is geared towards applications in algebra and is somewhat inconvenient to use in optimization where frequently the main interest lies in real roots. In this note, we present the roots of the cubic in a form that makes them convenient to use and we also focus on information on the location of the real roots. Armed with this, we provide several applications in optimization where we compute Fenchel conjugates, proximal map**s and projections. △ Less

Submitted 21 February, 2023; originally announced February 2023.

Comments: 26 pages, 5 figures

MSC Class: Primary 90C25; Secondary 12D10; 26C10; 90C2

On Carlier's inequality

Authors: Heinz H. Bauschke, Shambhavi Singh, Xianfu Wang

Abstract: The Fenchel-Young inequality is fundamental in Convex Analysis and Optimization. It states that the difference between certain function values of two vectors and their inner product is nonnegative. Recently, Carlier introduced a very nice sharpening of this inequality, providing a lower bound that depends on a positive parameter. In this note, we expand on Carlier's inequality in three ways. Fir… ▽ More The Fenchel-Young inequality is fundamental in Convex Analysis and Optimization. It states that the difference between certain function values of two vectors and their inner product is nonnegative. Recently, Carlier introduced a very nice sharpening of this inequality, providing a lower bound that depends on a positive parameter. In this note, we expand on Carlier's inequality in three ways. First, a duality statement is provided. Secondly, we discuss asymptotic behaviour as the underlying parameter approaches zero or infinity. Thirdly, relying on cyclic monotonicity and associated Fitzpatrick functions, we present a lower bound that features an infinite series of squares of norms. Several examples illustrate our results. △ Less

Submitted 29 June, 2022; originally announced June 2022.

MSC Class: 26B25; 47H05 (Primary) 26D07; 90C25 (Secondary)

How to project onto the intersection of a closed affine subspace and a hyperplane

Authors: Heinz H. Bauschke, Dayou Mao, Walaa M. Moursi

Abstract: Let $A$ be a closed affine subspace and let $B$ be a hyperplane in a Hilbert space. Suppose we are given their associated nearest point map**s $P_A$ and $P_B$, respectively. We present a formula for the projection onto their intersection $A\cap B$. As a special case, we derive a formula for the projection onto the intersection of two hyperplanes. These formulas provides useful information even i… ▽ More Let $A$ be a closed affine subspace and let $B$ be a hyperplane in a Hilbert space. Suppose we are given their associated nearest point map**s $P_A$ and $P_B$, respectively. We present a formula for the projection onto their intersection $A\cap B$. As a special case, we derive a formula for the projection onto the intersection of two hyperplanes. These formulas provides useful information even if $A\cap B$ is empty. Examples and numerical experiments are also provided. △ Less

Submitted 22 June, 2022; originally announced June 2022.

MSC Class: 15A04; 41A50; 47A50 (Primary) 47H09; 90C25 (Secondary)

Projecting onto rectangular hyperbolic paraboloids in Hilbert space

Authors: Heinz H. Bauschke, Manish Krishan Lal, Xianfu Wang

Abstract: In $\mathbb{R}^3$, a hyperbolic paraboloid is a classical saddle-shaped quadric surface. Recently, Elser has modeled problems arising in Deep Learning using rectangular hyperbolic paraboloids in $\mathbb{R}^n$. Motivated by his work, we provide a rigorous analysis of the associated projection. In some cases, finding this projection amounts to finding a certain root of a quintic or cubic polynomial… ▽ More In $\mathbb{R}^3$, a hyperbolic paraboloid is a classical saddle-shaped quadric surface. Recently, Elser has modeled problems arising in Deep Learning using rectangular hyperbolic paraboloids in $\mathbb{R}^n$. Motivated by his work, we provide a rigorous analysis of the associated projection. In some cases, finding this projection amounts to finding a certain root of a quintic or cubic polynomial. We also observe when the projection is not a singleton and point out connections to graphical and set convergence. △ Less

Submitted 13 December, 2023; v1 submitted 10 June, 2022; originally announced June 2022.

Comments: 22 pages, 1 figure

MSC Class: Primary 41A50; 90C26; Secondary 15A63; 46C05

Journal ref: Applied Set-Valued Analysis and Optimization 5 (2023), No. 2, pp. 163-180

The Homogenization Cone: Polar Cone and Projection

Authors: Heinz H. Bauschke, Theo Bendit, Hansen Wang

Abstract: Let $C$ be a closed convex subset of a real Hilbert space containing the origin, and assume that $K$ is the homogenization cone of $C$, i.e., the smallest closed convex cone containing $C \times \{1\}$. Homogenization cones play an important role in optimization as they include, for instance, the second-order/Lorentz/"ice cream" cone. In this note, we discuss the polar cone of $K$ as well as an… ▽ More Let $C$ be a closed convex subset of a real Hilbert space containing the origin, and assume that $K$ is the homogenization cone of $C$, i.e., the smallest closed convex cone containing $C \times \{1\}$. Homogenization cones play an important role in optimization as they include, for instance, the second-order/Lorentz/"ice cream" cone. In this note, we discuss the polar cone of $K$ as well as an algorithm for finding the projection onto $K$ provided that the projection onto $C$ is available. Various examples illustrate our results. △ Less

Submitted 7 June, 2022; v1 submitted 6 June, 2022; originally announced June 2022.

Comments: Twenty-seven pages, four figures

MSC Class: 90C25 (Primary) 46A55; 52A07 (Secondary)

The splitting algorithms by Ryu, by Malitsky-Tam, and by Campoy applied to normal cones of linear subspaces converge strongly to the projection onto the intersection

Authors: Heinz H. Bauschke, Shambhavi Singh, Xianfu Wang

Abstract: Finding a zero of a sum of maximally monotone operators is a fundamental problem in modern optimization and nonsmooth analysis. Assuming that the resolvents of the operators are available, this problem can be tackled with the Douglas-Rachford algorithm. However, when dealing with three or more operators, one must work in a product space with as many factors as there are operators. In groundbreakin… ▽ More Finding a zero of a sum of maximally monotone operators is a fundamental problem in modern optimization and nonsmooth analysis. Assuming that the resolvents of the operators are available, this problem can be tackled with the Douglas-Rachford algorithm. However, when dealing with three or more operators, one must work in a product space with as many factors as there are operators. In groundbreaking recent work by Ryu and by Malitsky and Tam, it was shown that the number of factors can be reduced by one. A similar reduction was achieved recently by Campoy through a clever reformulation originally proposed by Kruger. All three splitting methods guarantee weak convergence to some solution of the underlying sum problem; strong convergence holds in the presence of uniform monotonicity. In this paper, we provide a case study when the operators involved are normal cone operators of subspaces and the solution set is thus the intersection of the subspaces. Even though these operators lack strict convexity, we show that striking conclusions are available in this case: strong (instead of weak) convergence and the solution obtained is (not arbitrary but) the projection onto the intersection. Numerical experiments to illustrate our results are also provided. △ Less

Submitted 19 December, 2022; v1 submitted 7 March, 2022; originally announced March 2022.

Comments: 25 Pages, 3 Figures. arXiv admin note: substantial text overlap with arXiv:2109.11072; final version that was submitted to SIOpt

MSC Class: 41A50; 49M27; 65K05; 47H05 (Primary) 15A10; 47H09; 49M37; 90C25 (Secondary)

Roots of the identity operator and proximal map**s: (classical and phantom) cycles and gap vectors

Authors: Heinz H. Bauschke, Xianfu Wang

Abstract: Recently, Simons provided a lemma for a support function of a closed convex set in a general Hilbert space and used it to prove the geometry conjecture on cycles of projections. In this paper, we extend Simons's lemma to closed convex functions, show its connections to Attouch-Thera duality, and use it to characterize (classical and phantom) cycles and gap vectors of proximal map**s. Recently, Simons provided a lemma for a support function of a closed convex set in a general Hilbert space and used it to prove the geometry conjecture on cycles of projections. In this paper, we extend Simons's lemma to closed convex functions, show its connections to Attouch-Thera duality, and use it to characterize (classical and phantom) cycles and gap vectors of proximal map**s. △ Less

Submitted 22 February, 2022; v1 submitted 13 January, 2022; originally announced January 2022.

MSC Class: 47H05; 52A41; 47H10 (Primary) 49J53; 46C05; 90C25 (Secondary)

Projections onto hyperbolas or bilinear constraint sets in Hilbert spaces

Authors: Heinz H. Bauschke, Manish Krishan Lal, Xianfu Wang

Abstract: Sets of bilinear constraints are important in various machine learning models. Mathematically, they are hyperbolas in a product space. In this paper, we give a complete formula for projections onto sets of bilinear constraints or hyperbolas in a general Hilbert space. Sets of bilinear constraints are important in various machine learning models. Mathematically, they are hyperbolas in a product space. In this paper, we give a complete formula for projections onto sets of bilinear constraints or hyperbolas in a general Hilbert space. △ Less

Submitted 3 December, 2021; originally announced December 2021.

Comments: 16 pages

MSC Class: Primary 41A50; 90C26; Secondary 90C20; 46C05

The splitting algorithms by Ryu and by Malitsky-Tam applied to normal cones of linear subspaces converge strongly to the projection onto the intersection

Authors: Heinz H. Bauschke, Shambhavi Singh, Xianfu Wang

Abstract: Finding a zero of a sum of maximally monotone operators is a fundamental problem in modern optimization and nonsmooth analysis. Assuming that resolvents of the operators are available, this problem can be tackled with the Douglas-Rachford algorithm. However, when dealing with three or more operators, one must work in a product space with as many factors as there are operators. In groundbreaking re… ▽ More Finding a zero of a sum of maximally monotone operators is a fundamental problem in modern optimization and nonsmooth analysis. Assuming that resolvents of the operators are available, this problem can be tackled with the Douglas-Rachford algorithm. However, when dealing with three or more operators, one must work in a product space with as many factors as there are operators. In groundbreaking recent work by Ryu and by Malitsky and Tam, it was shown that the number of factors can be reduced by one. These splitting methods guarantee weak convergence to some solution of the underlying sum problem; strong convergence holds in the presence of uniform monotonicity. In this paper, we provide a case study when the operators involved are normal cone operators of subspaces and the solution set is thus the intersection of the subspaces. Even though these operators lack strict convexity, we show that striking conclusions are available in this case: strong (instead of weak) convergence and the solution obtained is (not arbitrary but) the projection onto the intersection. Numerical experiments to illustrate our results are also provided △ Less

Submitted 22 September, 2021; originally announced September 2021.

Comments: 25 pages, 4 figures

MSC Class: 41A50; 49M27; 65K05; 47H05 (Primary) 15A10; 47H09; 49M37; 90C25 (Secondary)

The Bregman proximal average

Authors: Xianfu Wang, Heinz H. Bauschke

Abstract: We provide a proximal average with repect to a $1$-coercive Legendre function. In the sense of Bregman distance, the Bregman envelope of the proximal average is a convex combination of Bregman envelopes of individual functions. The Bregman proximal map** of the average is a convex combination of convexified proximal map**s of individual functions. Techniques from variational analysis provide t… ▽ More We provide a proximal average with repect to a $1$-coercive Legendre function. In the sense of Bregman distance, the Bregman envelope of the proximal average is a convex combination of Bregman envelopes of individual functions. The Bregman proximal map** of the average is a convex combination of convexified proximal map**s of individual functions. Techniques from variational analysis provide the keys for the Bregman proximal average. △ Less

Submitted 23 February, 2022; v1 submitted 25 August, 2021; originally announced August 2021.

MSC Class: 49J53; 49J52

The projection onto the cross

Authors: Heinz H. Bauschke, Manish Krishan Lal, Xianfu Wang

Abstract: We consider the set of pairs of orthogonal vectors in Hilbert space, which is also called the cross because it is the union of the horizontal and vertical axes in the Euclidean plane when the underlying space is the real line. Crosses, which are nonconvex sets, play a significant role in various branches of nonsmooth analysis such as feasibility problems and optimization problems. In this work,… ▽ More We consider the set of pairs of orthogonal vectors in Hilbert space, which is also called the cross because it is the union of the horizontal and vertical axes in the Euclidean plane when the underlying space is the real line. Crosses, which are nonconvex sets, play a significant role in various branches of nonsmooth analysis such as feasibility problems and optimization problems. In this work, we study crosses and show that in infinite-dimensional settings, they are never weakly (sequentially) closed. Nonetheless, crosses do turn out to be proximinal (i.e., they always admit projections) and we provide explicit formulas for the projection onto the cross in all cases. △ Less

Submitted 2 February, 2022; v1 submitted 9 August, 2021; originally announced August 2021.

MSC Class: 41A50; 90C26 (Primary) 46C05; 90C33 (Secondary)

Directional asymptotics of Fejér monotone sequences

Authors: Heinz H. Bauschke, Manish Krishan Lal, Xianfu Wang

Abstract: The notion of Fejér monotonicity is instrumental in unifying the convergence proofs of many iterative methods, such as the Krasnoselskii-Mann iteration, the proximal point method, the Douglas-Rachford splitting algorithm, and many others. In this paper, we present directionally asymptotical results of strongly convergent subsequences of Fejér monotone sequences. We also provide examples to show th… ▽ More The notion of Fejér monotonicity is instrumental in unifying the convergence proofs of many iterative methods, such as the Krasnoselskii-Mann iteration, the proximal point method, the Douglas-Rachford splitting algorithm, and many others. In this paper, we present directionally asymptotical results of strongly convergent subsequences of Fejér monotone sequences. We also provide examples to show that the sets of directionally asymptotic cluster points can be large and that weak convergence is needed in infinite-dimensional spaces. △ Less

Submitted 28 June, 2021; originally announced June 2021.

MSC Class: 47H09; 47J26; 90C25 (Primary) 47H05; 65K10 (Secondary)

On the Douglas-Rachford algorithm for solving possibly inconsistent optimization problems

Authors: Heinz H. Bauschke, Walaa M. Moursi

Abstract: More than 40 years ago, Lions and Mercier introduced in a seminal paper the Douglas-Rachford algorithm. Today, this method is well recognized as a classical and highly successful splitting method to find minimizers of the sum of two (not necessarily smooth) convex functions. While the underlying theory has matured, one case remains a mystery: the behaviour of the shadow sequence when the given fun… ▽ More More than 40 years ago, Lions and Mercier introduced in a seminal paper the Douglas-Rachford algorithm. Today, this method is well recognized as a classical and highly successful splitting method to find minimizers of the sum of two (not necessarily smooth) convex functions. While the underlying theory has matured, one case remains a mystery: the behaviour of the shadow sequence when the given functions have disjoint domains. Building on previous work, we establish for the first time weak and value convergence of the shadow sequence generated by the Douglas-Rachford algorithm in a setting of unprecedented generality. The weak limit point is shown to solve the associated normal problem which is a minimal perturbation of the original optimization problem. We also present new results on the geometry of the minimal displacement vector. △ Less

Submitted 22 June, 2021; originally announced June 2021.

MSC Class: 65K10; 90C25 (Primary); 47H05 (Secondary)

Projecting onto rectangular matrices with prescribed row and column sums

Authors: Heinz H. Bauschke, Shambhavi Singh, Xianfu Wang

Abstract: In 1990, Romero presented a beautiful formula for the projection onto the set of rectangular matrices with prescribed row and column sums. Variants of Romero's formula have been rediscovered by Khoury and by Glunt, Hayden, and Reams, for bistochastic (square) matrices in 1998. These results have found various generalizations and applications. In this paper, we provide a formula for the more gene… ▽ More In 1990, Romero presented a beautiful formula for the projection onto the set of rectangular matrices with prescribed row and column sums. Variants of Romero's formula have been rediscovered by Khoury and by Glunt, Hayden, and Reams, for bistochastic (square) matrices in 1998. These results have found various generalizations and applications. In this paper, we provide a formula for the more general problem of finding the projection onto the set of rectangular matrices with prescribed scaled row and column sums. Our approach is based on computing the Moore-Penrose inverse of a certain linear operator associated with the problem. In fact, our analysis holds even for Hilbert-Schmidt operators and we do not have to assume consistency. We also perform numerical experiments featuring the new projection operator. △ Less

Submitted 14 November, 2021; v1 submitted 25 May, 2021; originally announced May 2021.

MSC Class: 90C25 (Primary) 15A10; 15B51; 47L07 (Secondary)

Minimal angle spread in the probability simplex with respect to the uniform distribution

Authors: Heinz H. Bauschke, Peter A. V. DiBerardino

Abstract: We compute the minimal angle spread with respect to the uniform distribution in the probability simplex. The resulting optimization problem is analytically solved. The formula provided shows that the minimal angle spread approaches zero as the dimension tends to infinity. We also discuss an application in cognitive science. We compute the minimal angle spread with respect to the uniform distribution in the probability simplex. The resulting optimization problem is analytically solved. The formula provided shows that the minimal angle spread approaches zero as the dimension tends to infinity. We also discuss an application in cognitive science. △ Less

Submitted 25 April, 2022; v1 submitted 18 May, 2021; originally announced May 2021.

MSC Class: 52A20; 52B11 (Primary) 90C25 (Secondary)

Journal ref: Journal of Nonsmooth Analysis and Optimization, Volume 3, Original research articles (April 27, 2022) jnsao:7492

On angles between convex cones

Authors: Heinz H. Bauschke, Hui Ouyang, Xianfu Wang

Abstract: There are two basic angles associated with a pair of linear subspaces: the Diximier angle and the Friedrichs angle. The Dixmier angle of the pair of orthogonal complements is the same as the Dixmier angle of the original pair provided that the original pair gives rise to a direct (not necessarily orthogonal) sum of the underlying Hilbert space. The Friedrichs angles of the original pair and the pa… ▽ More There are two basic angles associated with a pair of linear subspaces: the Diximier angle and the Friedrichs angle. The Dixmier angle of the pair of orthogonal complements is the same as the Dixmier angle of the original pair provided that the original pair gives rise to a direct (not necessarily orthogonal) sum of the underlying Hilbert space. The Friedrichs angles of the original pair and the pair of the orthogonal complements always coincide. These two results are due to Krein, Krasnoselskii, and Milman and to Solmon, respectively. In 1995, Deutsch provided a very nice survey with complete proofs and interesting historical comments. One key result in Deutsch's survey was an inequality for Dixmier angles provided by Hundal. In this paper, we present extensions of these results to the case when the linear subspaces are only required to be convex cones. It turns out that Hundal's result has a nice conical extension while the situation is more technical for the results by Krein et al.\ and by Solmon. Our analysis is based on Deutsch's survey and our recent work on angles between convex sets. Throughout, we also provide examples illustrating the sharpness of our results. △ Less

Submitted 22 September, 2021; v1 submitted 12 May, 2021; originally announced May 2021.

Comments: arXiv admin note: text overlap with arXiv:2008.09313

MSC Class: 46C05; 52A05 (Primary) 90C25 (Secondary)

Finding best approximation pairs for two intersections of closed convex sets

Authors: Heinz H. Bauschke, Shambhavi Singh, Xianfu Wang

Abstract: The problem of finding a best approximation pair of two sets, which in turn generalizes the well known convex feasibility problem, has a long history that dates back to work by Cheney and Goldstein in 1959. In 2018, Aharoni, Censor, and Jiang revisited this problem and proposed an algorithm that can be used when the two sets are finite intersections of halfspaces. Motivated by their work, we pre… ▽ More The problem of finding a best approximation pair of two sets, which in turn generalizes the well known convex feasibility problem, has a long history that dates back to work by Cheney and Goldstein in 1959. In 2018, Aharoni, Censor, and Jiang revisited this problem and proposed an algorithm that can be used when the two sets are finite intersections of halfspaces. Motivated by their work, we present alternative algorithms that utilize projection and proximity operators. Numerical experiments indicate that these methods are competitive and sometimes superior to the one proposed by Aharoni et al. △ Less

Submitted 15 October, 2021; v1 submitted 25 February, 2021; originally announced February 2021.

MSC Class: 65K05 (Primary) 47H09; 90C25 (Secondary)

Attouch-Théra Duality, Generalized Cycles and Gap Vectors

Authors: Salihah Alwadani, Heinz H. Bauschke, Xianfu Wang

Abstract: Using the Attouch-Théra duality, we study the cycles, gap vectors and fixed point sets of compositions of proximal map**s. Sufficient conditions are given for the existence of cycles and gap vectors. A primal-dual framework provides an exact relationship between the cycles and gap vectors. We also introduce the generalized cycle and gap vectors to tackle the case when the classical ones do not e… ▽ More Using the Attouch-Théra duality, we study the cycles, gap vectors and fixed point sets of compositions of proximal map**s. Sufficient conditions are given for the existence of cycles and gap vectors. A primal-dual framework provides an exact relationship between the cycles and gap vectors. We also introduce the generalized cycle and gap vectors to tackle the case when the classical ones do not exist. Examples are given to illustrate our results. △ Less

Submitted 3 May, 2021; v1 submitted 14 January, 2021; originally announced January 2021.

Comments: 25 pages

MSC Class: (Primary) 47H05; 49J53; 47H09; 47N10; 49N15; 47A06; 47J25; 90C25 (Secondary)

The difference vectors for convex sets and a resolution of the geometry conjecture

Authors: Salihah Alwadani, Heinz H. Bauschke, Julian P. Revalski, Xianfu Wang

Abstract: The geometry conjecture, which was posed nearly a quarter of a century ago, states that the fixed point set of the composition of projectors onto nonempty closed convex sets in Hilbert space is actually equal to the intersection of certain translations of the underlying sets. In this paper, we provide a complete resolution of the geometry conjecture. Our proof relies on monotone operator theory.… ▽ More The geometry conjecture, which was posed nearly a quarter of a century ago, states that the fixed point set of the composition of projectors onto nonempty closed convex sets in Hilbert space is actually equal to the intersection of certain translations of the underlying sets. In this paper, we provide a complete resolution of the geometry conjecture. Our proof relies on monotone operator theory. We revisit previously known results and provide various illustrative examples. Comments on the numerical computation of the quantities involved are also presented. △ Less

Submitted 28 May, 2021; v1 submitted 8 December, 2020; originally announced December 2020.

MSC Class: 47H09 (Primary) 47H05; 47H10; 90C25 (Secondary)

Edelstein's Astonishing Affine Isometry

Authors: Heinz H. Bauschke, Sylvain Gretchko, Walaa M. Moursi, Matthew Saurette

Abstract: In 1964, Michael Edelstein presented an amazing affine isometry acting on the space of square-summable sequences. This operator has no fixed points, but a suborbit that converges to 0 while another escapes in norm to infinity! We revisit, extend and sharpen his construction. Moreover, we sketch a connection to modern optimization and monotone operator theory. In 1964, Michael Edelstein presented an amazing affine isometry acting on the space of square-summable sequences. This operator has no fixed points, but a suborbit that converges to 0 while another escapes in norm to infinity! We revisit, extend and sharpen his construction. Moreover, we sketch a connection to modern optimization and monotone operator theory. △ Less

Submitted 15 September, 2020; originally announced September 2020.

On angles between convex sets in Hilbert spaces

Authors: Heinz H. Bauschke, Hui Ouyang, Xianfu Wang

Abstract: The notion of the angle between two subspaces has a long history, dating back to Friedrichs's work in 1937 and Dixmier's work on the minimal angle in 1949. In 2006, Deutsch and Hundal studied extensions to convex sets in order to analyze convergence rates for the cyclic projections algorithm. In this work, we characterize the positivity of the minimal angle between two convex cones. We show the… ▽ More The notion of the angle between two subspaces has a long history, dating back to Friedrichs's work in 1937 and Dixmier's work on the minimal angle in 1949. In 2006, Deutsch and Hundal studied extensions to convex sets in order to analyze convergence rates for the cyclic projections algorithm. In this work, we characterize the positivity of the minimal angle between two convex cones. We show the existence of, and necessary conditions for, optimal solutions of minimal angle problems associated with two convex subsets as well. Moreover, we generalize a result by Deutsch on minimal angles from linear subspaces to cones. This generalization yields sufficient conditions for the closedness of the sum of two closed convex cones. This also relates to conditions proposed by Beutner and by Seeger and Sossa. Furthermore, we investigate the relation between the intersection of two cones (at least one of which is nonlinear) and the intersection of the polar and dual cones of the underlying cones. It turns out that the two angles involved cannot be positive simultaneously. Various examples illustrate the sharpness of our results. △ Less

Submitted 6 May, 2021; v1 submitted 21 August, 2020; originally announced August 2020.

Comments: 18 pages

MSC Class: Primary 46C05; 41A29; Secondary 41A65; 90C25; 90C26

Resolvents and Yosida approximations of displacement map**s of isometries

Authors: Salihah Alwadani, Heinz H. Bauschke, Julian P. Revalski, Xianfu Wang

Abstract: Maximally monotone operators are fundamental objects in modern optimization. The main classes of monotone operators are subdifferential operators and matrices with a positive semidefinite symmetric part. In this paper, we study a nice class of monotone operators: displacement map**s of isometries of finite order. We derive explicit formulas for resolvents, Yosida approximations, and (set-valued… ▽ More Maximally monotone operators are fundamental objects in modern optimization. The main classes of monotone operators are subdifferential operators and matrices with a positive semidefinite symmetric part. In this paper, we study a nice class of monotone operators: displacement map**s of isometries of finite order. We derive explicit formulas for resolvents, Yosida approximations, and (set-valued and MoorePenrose) inverses. We illustrate our results by considering certain rational rotators and circular shift operators. △ Less

Submitted 23 March, 2021; v1 submitted 8 June, 2020; originally announced June 2020.

MSC Class: 47H05; 47H09 (Primary) 47A06; 90C25 (Secondary)

Best approximation map**s in Hilbert spaces

Authors: Heinz H. Bauschke, Hui Ouyang, Xianfu Wang

Abstract: The notion of best approximation map** (BAM) with respect to a closed affine subspace in finite-dimensional space was introduced by Behling, Bello Cruz and Santos to show the linear convergence of the block-wise circumcentered-reflection method. The best approximation map** possesses two critical properties of the circumcenter map** for linear convergence. Because the iteration sequence of… ▽ More The notion of best approximation map** (BAM) with respect to a closed affine subspace in finite-dimensional space was introduced by Behling, Bello Cruz and Santos to show the linear convergence of the block-wise circumcentered-reflection method. The best approximation map** possesses two critical properties of the circumcenter map** for linear convergence. Because the iteration sequence of BAM linearly converges, the BAM is interesting in its own right. In this paper, we naturally extend the definition of BAM from closed affine subspace to nonempty closed convex set and from $\mathbb{R}^{n}$ to general Hilbert space. We discover that the convex set associated with the BAM must be the fixed point set of the BAM. Hence, the iteration sequence generated by a BAM linearly converges to the nearest fixed point of the BAM. Connections between BAMs and other map**s generating convergent iteration sequences are considered. Behling et al.\ proved that the finite composition of BAMs associated with closed affine subspaces is still a BAM in $\mathbb{R}^{n}$. We generalize their result from $\mathbb{R}^{n}$ to general Hilbert space and also construct a new constant associated with the composition of BAMs. This provides a new proof of the linear convergence of the method of alternating projections. Moreover, compositions of BAMs associated with general convex sets are investigated. In addition, we show that convex combinations of BAMs associated with affine subspaces are BAMs. Last but not least, we connect BAM with circumcenter map** in Hilbert spaces. △ Less

Submitted 4 June, 2020; originally announced June 2020.

Comments: 31 pages and 2 figures

MSC Class: Primary 90C25; 41A50; 65B99; Secondary 46B04; 41A65

Fixed points of compositions of nonexpansive map**s: finitely many linear reflectors

Authors: Salihah Alwadani, Heinz H. Bauschke, Xianfu Wang

Abstract: Nonexpansive map**s play a central role in modern optimization and monotone operator theory because their fixed points can describe solutions to optimization or critical point problems. It is known that when the map**s are sufficiently "nice", then the fixed point set of the composition coincides with the intersection of the individual fixed point sets. In this paper, we explore the situation… ▽ More Nonexpansive map**s play a central role in modern optimization and monotone operator theory because their fixed points can describe solutions to optimization or critical point problems. It is known that when the map**s are sufficiently "nice", then the fixed point set of the composition coincides with the intersection of the individual fixed point sets. In this paper, we explore the situation for compositions of linear reflectors. We provide positive results, upper bounds, and limiting examples. We also discuss classical reflectors in the Euclidean plane. △ Less

Submitted 27 April, 2020; originally announced April 2020.

MSC Class: 47H09

On Dykstra's algorithm: finite convergence, stalling, and the method of alternating projections

Authors: Heinz H. Bauschke, Regina S. Burachik, Daniel B. Herman, C. Yalcin Kaya

Abstract: A popular method for finding the projection onto the intersection of two closed convex subsets in Hilbert space is Dykstra's algorithm. In this paper, we provide sufficient conditions for Dykstra's algorithm to converge rapidly, in finitely many steps. We also analyze the behaviour of Dykstra's algorithm applied to a line and a square. This case study reveals stark similarities to the method of al… ▽ More A popular method for finding the projection onto the intersection of two closed convex subsets in Hilbert space is Dykstra's algorithm. In this paper, we provide sufficient conditions for Dykstra's algorithm to converge rapidly, in finitely many steps. We also analyze the behaviour of Dykstra's algorithm applied to a line and a square. This case study reveals stark similarities to the method of alternating projections. Moreover, we show that Dykstra's algorithm may stall for an arbitrarily long time. Finally, we present some open problems. △ Less

Submitted 18 January, 2020; originally announced January 2020.

MSC Class: 65K05; 90C25 (Primary) 47H09; 52A05 (Secondary)

On the linear convergence of circumcentered isometry methods

Authors: Heinz H. Bauschke, Hui Ouyang, Xianfu Wang

Abstract: The circumcentered Douglas--Rachford method (C--DRM), introduced by Behling, Bello Cruz and Santos, is an acceleration of the well-known Douglas-Rachford method (DRM) for finding the best approximation onto the intersection of finitely many affine subspaces. Inspired by the C--DRM, we introduced the more flexible circumcentered reflection method (CRM) and circumcentered isometry method (CIM). The… ▽ More The circumcentered Douglas--Rachford method (C--DRM), introduced by Behling, Bello Cruz and Santos, is an acceleration of the well-known Douglas-Rachford method (DRM) for finding the best approximation onto the intersection of finitely many affine subspaces. Inspired by the C--DRM, we introduced the more flexible circumcentered reflection method (CRM) and circumcentered isometry method (CIM). The CIM essentially chooses the closest point to the solution among all of the points in an associated affine hull as its iterate and is a generalization of the CRM. The circumcentered--reflection method introduced by Behling et al. to generalize the C--DRM is a special class of our CRM. We consider the CIM induced by a set of finitely many isometries for finding the best approximation onto the intersection of fixed point sets of the isometries which turns out to be an intersection of finitely many affine subspaces. We extend our previous linear convergence results on CRMs in finite-dimensional spaces from reflections to isometries. In order to better accelerate the symmetric method of alternating projections (MAP), the accelerated symmetric MAP first applies another operator to the initial point. (Similarly, to accelerate the DRM, the C--DRM first applies another operator to the initial point as well.) Motivated by these facts, we show results on the linear convergence of CIMs in Hilbert spaces with first applying another operator to the initial point. In particular, under some restrictions, our results imply that some CRMs attain the known linear convergence rate of the accelerated symmetric MAP in Hilbert spaces. We also exhibit a class of CRMs converging to the best approximation in Hilbert spaces with a convergence rate no worse than the sharp convergence rate of MAP. The fact that some CRMs attain the linear convergence rate of MAP or accelerated symmetric MAP is entirely new. △ Less

Submitted 12 June, 2020; v1 submitted 2 December, 2019; originally announced December 2019.

Comments: 29 pages

MSC Class: Primary 41A50; 47H30; 65B99; Secondary 46B04; 90C25

Circumcentered methods induced by isometries

Authors: Heinz H. Bauschke, Hui Ouyang, Xianfu Wang

Abstract: Motivated by the circumcentered Douglas--Rachford method recently introduced by Behling, Bello Cruz and Santos to accelerate the Douglas--Rachford method, we study the properness of the circumcenter map** and the circumcenter method induced by isometries. Applying the demiclosedness principle for circumcenter map**s, we present weak convergence results for circumcentered isometry methods, whic… ▽ More Motivated by the circumcentered Douglas--Rachford method recently introduced by Behling, Bello Cruz and Santos to accelerate the Douglas--Rachford method, we study the properness of the circumcenter map** and the circumcenter method induced by isometries. Applying the demiclosedness principle for circumcenter map**s, we present weak convergence results for circumcentered isometry methods, which include the Douglas--Rachford method (DRM) and circumcentered reflection methods as special instances. We provide sufficient conditions for the linear convergence of circumcentered isometry/reflection methods. We explore the convergence rate of circumcentered reflection methods by considering the required number of iterations and as well as run time as our performance measures. Performance profiles on circumcentered reflection methods, DRM and method of alternating projections for finding the best approximation to the intersection of linear subspaces are presented. △ Less

Submitted 28 February, 2020; v1 submitted 30 August, 2019; originally announced August 2019.

Comments: 31 pages, 8 figures

MSC Class: 47H09; 65K10 (Primary); 41A50; 65K05; 90C25 (Secondary)

On the behaviour of the Douglas-Rachford algorithm for minimizing a convex function subject to a linear constraint

Authors: Heinz H. Bauschke, Walaa M. Moursi

Abstract: The Douglas-Rachford algorithm (DRA) is a powerful optimization method for minimizing the sum of two convex (not necessarily smooth) functions. The vast majority of previous research dealt with the case when the sum has at least one minimizer. In the absence of minimizers, it was recently shown that for the case of two indicator functions, the DRA converges to a best approximation solution. In thi… ▽ More The Douglas-Rachford algorithm (DRA) is a powerful optimization method for minimizing the sum of two convex (not necessarily smooth) functions. The vast majority of previous research dealt with the case when the sum has at least one minimizer. In the absence of minimizers, it was recently shown that for the case of two indicator functions, the DRA converges to a best approximation solution. In this paper, we present a new convergence result on the the DRA applied to the problem of minimizing a convex function subject to a linear constraint. Indeed, a normal solution may be found even when the domain of the objective function and the linear subspace constraint have no point in common. As an important application, a new parallel splitting result is provided. We also illustrate our results through various examples. △ Less

Submitted 9 July, 2020; v1 submitted 14 August, 2019; originally announced August 2019.

MSC Class: 49M27; 65K10; 90C25; 47H14; 49M29

Maximally monotone operators with ranges whose closures are not convex and an answer to a recent question by Stephen Simons

Authors: Heinz H. Bauschke, Walaa M. Moursi, Xianfu Wang

Abstract: In his recent Proceedings of the AMS paper "Gossez's skew linear map and its pathological maximally monotone multifunctions", Stephen Simons proved that the closure of the range of the sum of the Gossez operator and a multiple of the duality map is nonconvex whenever the scalar is between 0 and 4. The problem of the convexity of that range when the scalar is equal to 4 was explicitly stated. In th… ▽ More In his recent Proceedings of the AMS paper "Gossez's skew linear map and its pathological maximally monotone multifunctions", Stephen Simons proved that the closure of the range of the sum of the Gossez operator and a multiple of the duality map is nonconvex whenever the scalar is between 0 and 4. The problem of the convexity of that range when the scalar is equal to 4 was explicitly stated. In this paper, we answer this question in the negative for any scalar greater than or equal to 4. We derive this result from an abstract framework that allows us to also obtain a corresponding result for the Fitzpatrick-Phelps integral operator. △ Less

Submitted 14 September, 2019; v1 submitted 29 April, 2019; originally announced April 2019.

MSC Class: Primary 47H05; Secondary 46B20; 47A05

Generalized monotone operators and their averaged resolvents

Authors: Heinz H. Bauschke, Walaa M. Moursi, Xianfu Wang

Abstract: The correspondence between the monotonicity of a (possibly) set-valued operator and the firm nonexpansiveness of its resolvent is a key ingredient in the convergence analysis of many optimization algorithms. Firmly nonexpansive operators form a proper subclass of the more general - but still pleasant from an algorithmic perspective - class of averaged operators. In this paper, we introduce the new… ▽ More The correspondence between the monotonicity of a (possibly) set-valued operator and the firm nonexpansiveness of its resolvent is a key ingredient in the convergence analysis of many optimization algorithms. Firmly nonexpansive operators form a proper subclass of the more general - but still pleasant from an algorithmic perspective - class of averaged operators. In this paper, we introduce the new notion of conically nonexpansive operators which generalize nonexpansive map**s. We characterize averaged operators as being resolvents of comonotone operators under appropriate scaling. As a consequence, we characterize the proximal point map**s associated with hypoconvex functions as cocoercive operators, or equivalently; as displacement map**s of conically nonexpansive operators. Several examples illustrate our analysis and demonstrate tightness of our results. △ Less

Submitted 26 February, 2019; originally announced February 2019.

MSC Class: 47H05; 47H09; 49N15; 90C25

Numerical explorations of feasibility algorithms for finding points in the intersection of finite sets

Authors: Heinz H. Bauschke, Sylvain Gretchko, Walaa M. Moursi

Abstract: Projection methods are popular algorithms for iteratively solving feasibility problems in Euclidean or even Hilbert spaces. They employ (selections of) nearest point map**s to generate sequences that are designed to approximate a point in the intersection of a collection of constraint sets. Theoretical properties of projection methods are fairly well understood when the underlying constraint set… ▽ More Projection methods are popular algorithms for iteratively solving feasibility problems in Euclidean or even Hilbert spaces. They employ (selections of) nearest point map**s to generate sequences that are designed to approximate a point in the intersection of a collection of constraint sets. Theoretical properties of projection methods are fairly well understood when the underlying constraint sets are convex; however, convergence results for the nonconvex case are more complicated and typically only local. In this paper, we explore the perhaps simplest instance of a feasibility algorithm, namely when each constraint set consists of only finitely many points. We numerically investigate four constellations: either few or many constraint sets, with either few or many points. Each constellation is tackled by four popular projection methods each of which features a tuning parameter. We examine the behaviour for a single and for a multitude of orbits, and we also consider local and global behaviour. Our findings demonstrate the importance of the choice of the algorithm and that of the tuning parameter. △ Less

Submitted 23 January, 2019; originally announced January 2019.

MSC Class: 49M20; 49M27; 49M37; 65K05; 65K10; 90C25; 90C26; 90C30

Multi-marginal maximal monotonicity and convex analysis

Authors: Sedi Bartz, Heinz H. Bauschke, Hung M. Phan, Xianfu Wang

Abstract: Monotonicity and convex analysis arise naturally in the framework of multi-marginal optimal transport theory. However, a comprehensive multi-marginal monotonicity and convex analysis theory is still missing. To this end we study extensions of classical monotone operator theory and convex analysis into the multi-marginal setting. We characterize multi-marginal c-monotonicity in terms of classical m… ▽ More Monotonicity and convex analysis arise naturally in the framework of multi-marginal optimal transport theory. However, a comprehensive multi-marginal monotonicity and convex analysis theory is still missing. To this end we study extensions of classical monotone operator theory and convex analysis into the multi-marginal setting. We characterize multi-marginal c-monotonicity in terms of classical monotonicity and firmly nonexpansive map**s. We provide Minty type, continuity and conjugacy criteria for multi-marginal maximal monotonicity. We extend the partition of the identity into a sum of firmly nonexpansive map**s and Moreau's decomposition of the quadratic function into envelopes and proximal map**s into the multi-marginal settings. We illustrate our discussion with examples and provide applications for the determination of multi-marginal maximal monotonicity and multi-marginal conjugacy. We also point out several open questions. △ Less

Submitted 18 September, 2019; v1 submitted 11 January, 2019; originally announced January 2019.

MSC Class: Primary 47H05; 26B25; Secondary 49N15; 49K30; 52A01; 91B68

On circumcenter map**s induced by nonexpansive operators

Authors: Heinz H. Bauschke, Hui Ouyang, Xianfu Wang

Abstract: We introduce the circumcenter map** induced by a set of (usually nonexpansive) operators. One prominent example of a circumcenter map** is the celebrated Douglas--Rachford splitting operator. Our study is motivated by the Circumcentered--Douglas--Rachford method recently introduced by Behling, Bello Cruz, and Santos in order to accelerate the Douglas--Rachford method for solving certain classe… ▽ More We introduce the circumcenter map** induced by a set of (usually nonexpansive) operators. One prominent example of a circumcenter map** is the celebrated Douglas--Rachford splitting operator. Our study is motivated by the Circumcentered--Douglas--Rachford method recently introduced by Behling, Bello Cruz, and Santos in order to accelerate the Douglas--Rachford method for solving certain classes of feasibility problems. We systematically explore the properness of the circumcenter map** induced by reflectors or projectors. Numerous examples are presented. We also present a version of Browder's demiclosedness principle for circumcenter map**s. △ Less

Submitted 28 November, 2018; originally announced November 2018.

Comments: There are 30 pages and 7 figures

MSC Class: 47H09; 47H04 (Primary); 41A50; 90C25 (Secondary)

Applying FISTA to optimization problems (with or) without minimizers

Authors: Heinz H. Bauschke, Minh N. Bui, Xianfu Wang

Abstract: Beck and Teboulle's FISTA method for finding a minimizer of the sum of two convex functions, one of which has a Lipschitz continuous gradient whereas the other may be nonsmooth, is arguably the most important optimization algorithm of the past decade. While research activity on FISTA has exploded ever since, the mathematically challenging case when the original optimization problem has no minimize… ▽ More Beck and Teboulle's FISTA method for finding a minimizer of the sum of two convex functions, one of which has a Lipschitz continuous gradient whereas the other may be nonsmooth, is arguably the most important optimization algorithm of the past decade. While research activity on FISTA has exploded ever since, the mathematically challenging case when the original optimization problem has no minimizer has found only limited attention. In this work, we systematically study FISTA and its variants. We present general results that are applicable, regardless of the existence of minimizers. △ Less

Submitted 3 July, 2019; v1 submitted 22 November, 2018; originally announced November 2018.

MSC Class: 90C25; 65K05 (Primary); 49M27 (Secondary)

On the minimal displacement vector of compositions and convex combinations of nonexpansive map**s

Authors: Heinz H. Bauschke, Walaa M. Moursi

Abstract: Monotone operators and (firmly) nonexpansive map**s are fundamental objects in modern analysis and computational optimization. Five years ago, it was shown that if finitely many firmly nonexpansive map**s have or "almost have" fixed points, then the same is true for compositions and convex combinations. More recently, sharp information about the minimal displacement vector of compositions and… ▽ More Monotone operators and (firmly) nonexpansive map**s are fundamental objects in modern analysis and computational optimization. Five years ago, it was shown that if finitely many firmly nonexpansive map**s have or "almost have" fixed points, then the same is true for compositions and convex combinations. More recently, sharp information about the minimal displacement vector of compositions and of convex combinations of firmly nonexpansive map**s was obtained in terms of the displacement vectors of the underlying operators. Using a new proof technique based on the Brezis-Haraux theorem and reflected resolvents, we extend these results from firmly nonexpansive to general averaged nonexpansive map**s. Various examples illustrate the tightness of our results. △ Less

Submitted 4 September, 2018; originally announced September 2018.

MSC Class: 47H05; 47H09 (Primary); 47H10; 90C25 (Secondary)

Proximal Averages for Minimization of Entropy Functionals

Authors: Heinz H. Bauschke, Scott B. Lindstrom

Abstract: In their 2016 article `Meetings with Lambert $\mathcal{W}$ and Other Special Functions in Optimization and Analysis', Borwein and Lindstrom considered the minimization of an entropy functional utilizing the weighted average of the negative Boltzmann--Shannon entropy $x\mapsto x\log x -x$ and the energy $x \mapsto x^2/2$; the solution employed the Fenchel-Moreau conjugate of the sum. However, in pl… ▽ More In their 2016 article `Meetings with Lambert $\mathcal{W}$ and Other Special Functions in Optimization and Analysis', Borwein and Lindstrom considered the minimization of an entropy functional utilizing the weighted average of the negative Boltzmann--Shannon entropy $x\mapsto x\log x -x$ and the energy $x \mapsto x^2/2$; the solution employed the Fenchel-Moreau conjugate of the sum. However, in place of a traditional arithmetic average of two convex functions, it is also, perhaps even more, natural to use their proximal average. We explain the advantages and illustrate them by computing the analogous proximal averages for the negative entropy and energy. We use the proximal averages to solve entropy functional minimization problems similar to those considered by Borwein and Lindstrom, illustrating the benefits of using a true homotopy. Through experimentation, we discover computational barriers to obtaining solutions when computing with proximal averages, and we demonstrate a method which appears to remedy them. △ Less

Submitted 23 July, 2018; originally announced July 2018.

MSC Class: 46N10 (Primary); 65D05 (Secondary)

On circumcenters of finite sets in Hilbert spaces

Authors: Heinz H. Bauschke, Hui Ouyang, Xianfu Wang

Abstract: A well-known object in classical Euclidean geometry is the circumcenter of a triangle, i.e., the point that is equidistant from all vertices. The purpose of this paper is to provide a systematic study of the circumcenter of sets containing finitely many points in Hilbert space. This is motivated by recent works of Behling, Bello Cruz, and Santos on accelerated versions of the Douglas--Rachford met… ▽ More A well-known object in classical Euclidean geometry is the circumcenter of a triangle, i.e., the point that is equidistant from all vertices. The purpose of this paper is to provide a systematic study of the circumcenter of sets containing finitely many points in Hilbert space. This is motivated by recent works of Behling, Bello Cruz, and Santos on accelerated versions of the Douglas--Rachford method. We present basic results and properties of the circumcenter. Several examples are provided to illustrate the tightness of various assumptions. △ Less

Submitted 5 July, 2018; originally announced July 2018.

Comments: 23 pages, 2 figures

MSC Class: Primary 51M05; Secondary 41A50; 51M05 (Primary) 41A50; 90C25 (Secondary)

On the asymptotic behaviour of the Aragon Artacho-Campoy algorithm

Authors: Salihah Alwadani, Heinz H. Bauschke, Walaa M. Moursi, X. Wang

Abstract: Aragón Artacho and Campoy recently proposed a new method for computing the projection onto the intersection of two closed convex sets in Hilbert space; moreover, they proposed in 2018 a generalization from normal cone operators to maximally monotone operators. In this paper, we complete this analysis by demonstrating that the underlying curve converges to the nearest zero of the sum of the two ope… ▽ More Aragón Artacho and Campoy recently proposed a new method for computing the projection onto the intersection of two closed convex sets in Hilbert space; moreover, they proposed in 2018 a generalization from normal cone operators to maximally monotone operators. In this paper, we complete this analysis by demonstrating that the underlying curve converges to the nearest zero of the sum of the two operators. We also provide a new interpretation of the underlying operators in terms of the resolvent and the proximal average. △ Less

Submitted 28 May, 2018; originally announced May 2018.

MSC Class: Primary 47H05; 90C25; Secondary 47H09; 49M27; 65K05; 65K10

Intriguing maximally monotone operators derived from nonsunny nonexpansive retractions

Authors: Heinz H. Bauschke, Levi Miller, Walaa M. Moursi

Abstract: Monotone operator theory and fixed point theory for nonexpansive map**s are central areas in modern nonlinear analysis and optimization. Although these areas are fairly well developed, almost all examples published are based on subdifferential operators, linear relations, or combinations thereof. In this paper, we construct an intriguing maximally monotone operator induced by a certain nonexpa… ▽ More Monotone operator theory and fixed point theory for nonexpansive map**s are central areas in modern nonlinear analysis and optimization. Although these areas are fairly well developed, almost all examples published are based on subdifferential operators, linear relations, or combinations thereof. In this paper, we construct an intriguing maximally monotone operator induced by a certain nonexpansive retraction. We analyze this operator, which does not appear to be assembled from subdifferential operators or linear relations, in some detail. Particular emphasis is placed on duality and strong monotonicity. △ Less

Submitted 23 May, 2018; originally announced May 2018.

MSC Class: Primary 47H05; 47H09; Secondary 90C25

The Douglas--Rachford algorithm for a hyperplane and a doubleton

Authors: Heinz H. Bauschke, Minh N. Dao, Scott B. Lindstrom

Abstract: The Douglas--Rachford algorithm is a popular algorithm for solving both convex and nonconvex feasibility problems. While its behaviour is settled in the convex inconsistent case, the general nonconvex inconsistent case is far from being fully understood. In this paper, we focus on the most simple nonconvex inconsistent case: when one set is a hyperplane and the other a doubleton (i.e., a two-point… ▽ More The Douglas--Rachford algorithm is a popular algorithm for solving both convex and nonconvex feasibility problems. While its behaviour is settled in the convex inconsistent case, the general nonconvex inconsistent case is far from being fully understood. In this paper, we focus on the most simple nonconvex inconsistent case: when one set is a hyperplane and the other a doubleton (i.e., a two-point set). We present a characterization of cycling in this case which --- somewhat surprisingly --- depends on whether the ratio of the distance of the points to the hyperplane is rational or not. Furthermore, we provide closed-form expressions as well as several concrete examples which illustrate the dynamical richness of this algorithm. △ Less

Submitted 24 April, 2018; originally announced April 2018.

MSC Class: 47H10; 49M27; 65K05; 65K10; 90C26

Journal ref: Journal of Global Optimization, 74(1):79--93, 2019

Constraint Splitting and Projection Methods for Optimal Control of Double Integrator

Authors: Heinz H. Bauschke, Regina S. Burachik, C. Yalçın Kaya

Abstract: We consider the minimum-energy control of a car, which is modelled as a point mass sliding on the ground in a fixed direction, and so it can be mathematically described as the double integrator. The control variable, representing the acceleration or the deceleration, is constrained by simple bounds from above and below. Despite the simplicity of the problem, it is not possible to find an analytica… ▽ More We consider the minimum-energy control of a car, which is modelled as a point mass sliding on the ground in a fixed direction, and so it can be mathematically described as the double integrator. The control variable, representing the acceleration or the deceleration, is constrained by simple bounds from above and below. Despite the simplicity of the problem, it is not possible to find an analytical solution to it because of the constrained control variable. To find a numerical solution to this problem we apply three different projection-type methods: (i)~Dykstra's algorithm, (ii)~the Douglas--Rachford (DR) method and (iii)~the Aragón Artacho--Campoy (AAC) algorithm. To the knowledge of the authors, these kinds of (projection) methods have not previously been applied to continuous-time optimal control problems, which are infinite-dimensional optimization problems. The problem we study in this article is posed in infinite-dimensional Hilbert spaces. Behaviour of the DR and AAC algorithms are explored via numerical experiments with respect to their parameters. An error analysis is also carried out numerically for a particular instance of the problem for each of the algorithms. △ Less

Submitted 10 April, 2018; originally announced April 2018.

MSC Class: 49M05; 47N70; 90C25

On sums and convex combinations of projectors onto convex sets

Authors: Heinz H. Bauschke, Minh N. Bui, Xianfu Wang

Abstract: The projector onto the Minkowski sum of closed convex sets is generally not equal to the sum of individual projectors. In this work, we provide a complete answer to the question of characterizing the instances where such an equality holds. Our results unify and extend the case of linear subspaces and Zarantonello's results for projectors onto cones. A detailed analysis in the case of convex combin… ▽ More The projector onto the Minkowski sum of closed convex sets is generally not equal to the sum of individual projectors. In this work, we provide a complete answer to the question of characterizing the instances where such an equality holds. Our results unify and extend the case of linear subspaces and Zarantonello's results for projectors onto cones. A detailed analysis in the case of convex combinations is also carried out. We establish the partial sum property for projectors onto convex cones, and we also present various examples as well as a detailed analysis in the univariate case. △ Less

Submitted 14 September, 2018; v1 submitted 6 February, 2018; originally announced February 2018.

Comments: 1 figure

MSC Class: 47H05; 47H09; 47L07 (Primary); 46A32; 52A05; 52A41; 90C25 (Secondary)

The magnitude of the minimal displacement vector for compositions and convex combinations of firmly nonexpansive map**s

Authors: Heinz H. Bauschke, Walaa M. Moursi

Abstract: Maximally monotone operators and firmly nonexpansive map**s play key roles in modern optimization and nonlinear analysis. Five years ago, it was shown that if finitely many firmly nonexpansive operators are all asymptotically regular (i.e., the have or "almost have" fixed points), then the same is true for compositions and convex combinations. In this paper, we derive bounds on the magnitude of… ▽ More Maximally monotone operators and firmly nonexpansive map**s play key roles in modern optimization and nonlinear analysis. Five years ago, it was shown that if finitely many firmly nonexpansive operators are all asymptotically regular (i.e., the have or "almost have" fixed points), then the same is true for compositions and convex combinations. In this paper, we derive bounds on the magnitude of the minimal displacement vectors of compositions and of convex combinations in terms of the displacement vectors of the underlying operators. Our results completely generalize earlier works. Moreover, we present various examples illustrating that our bounds are sharp. △ Less

Submitted 1 December, 2017; originally announced December 2017.

MSC Class: Primar 47H05; 47H09 Secondary 47H10; 90C25

Projecting onto the intersection of a cone and a sphere

Authors: Heinz H. Bauschke, Minh N. Bui, Xianfu Wang

Abstract: The projection onto the intersection of sets generally does not allow for a closed form even when the individual projection operators have explicit descriptions. In this work, we systematically analyze the projection onto the intersection of a cone with either a ball or a sphere. Several cases are provided where the projector is available in closed form. Various examples based on finitely generate… ▽ More The projection onto the intersection of sets generally does not allow for a closed form even when the individual projection operators have explicit descriptions. In this work, we systematically analyze the projection onto the intersection of a cone with either a ball or a sphere. Several cases are provided where the projector is available in closed form. Various examples based on finitely generated cones, the Lorentz cone, and the the cone of positive semidefinite matrices are presented. The usefulness of our formulae is illustrated by numerical experiments for determining copositivity of real symmetric matrices. △ Less

Submitted 12 April, 2018; v1 submitted 1 August, 2017; originally announced August 2017.

Comments: 1 figure

Regularizing with Bregman-Moreau envelopes

Authors: Heinz H. Bauschke, Minh N. Dao, Scott B. Lindstrom

Abstract: Moreau's seminal paper, introducing what is now called the Moreau envelope and the proximity operator (also known as the proximal map**), appeared in 1965. The Moreau envelope of a given convex function provides a regularized version which has additional desirable properties such as differentiability and full domain. Fifty years ago, Attouch proposed using the Moreau envelope for regularization.… ▽ More Moreau's seminal paper, introducing what is now called the Moreau envelope and the proximity operator (also known as the proximal map**), appeared in 1965. The Moreau envelope of a given convex function provides a regularized version which has additional desirable properties such as differentiability and full domain. Fifty years ago, Attouch proposed using the Moreau envelope for regularization. Since then, this branch of convex analysis has developed in many fruitful directions. In 1967, Bregman introduced what is nowadays known as the Bregman distance as a measure of discrepancy between two points generalizing the square of the Euclidean distance. Proximity operators based on the Bregman distance have become a topic of significant research as they are useful in the algorithmic solution of optimization problems. More recently, in 2012, Kan and Song studied regularization aspects of the left Bregman-Moreau envelope even for nonconvex functions. In this paper, we complement previous works by analyzing the left and right Bregman-Moreau envelopes and by providing additional asymptotic results. Several examples are provided. △ Less

Submitted 16 November, 2018; v1 submitted 17 May, 2017; originally announced May 2017.

Journal ref: SIAM Journal on Optimization, 28(4):3208--3228, 2018

A class of multi-marginal c-cyclically monotone sets with explicit c-splitting potentials

Authors: Sedi Bartz, Heinz H. Bauschke, Xianfu Wang

Abstract: Multi-marginal optimal transport plans are concentrated on c-splitting sets. It is known that, similar to the two-marginal case, c-splitting sets are c-cyclically monotone. Within a suitable framework, the converse implication was very recently established by Griessler. However, for an arbitrary cost c, given a multi-marginal c-cyclically monotone set, the question whether there exists an analogou… ▽ More Multi-marginal optimal transport plans are concentrated on c-splitting sets. It is known that, similar to the two-marginal case, c-splitting sets are c-cyclically monotone. Within a suitable framework, the converse implication was very recently established by Griessler. However, for an arbitrary cost c, given a multi-marginal c-cyclically monotone set, the question whether there exists an analogous explicit construction to the one from the two-marginal case of c-splitting potentials is still open. When the margins are one-dimensional and the cost belongs to a certain class, Carlier proved that the two-marginal projections of a c-splitting set are monotone. For arbitrary products of sets equipped with cost functions which are sums of two-marginal costs, we show that the two-marginal monotonicity condition is a sufficient condition which does give rise to an explicit construction of c-splitting potentials. Our condition is, in principle, easier to verify than the one of multi-marginal c-cyclic monotonicity. Various examples illustrate our results. We show that, in general, our condition is sufficient; however, it is not necessary. On the other hand, we conclude that when the margins are one-dimensional equipped with classical cost functions, our condition is a characterization of c-splitting sets and extends classical convex analysis. △ Less

Submitted 3 March, 2017; v1 submitted 16 August, 2016; originally announced August 2016.

MSC Class: 49K30; 49N15 (Primary) 26B25; 47H05; 52A01; 91B68 (Secondary)

The resolvent order: a unification of the orders by Zarantonello, by Loewner, and by Moreau

Authors: Sedi Bartz, Heinz H. Bauschke, Xianfu Wang

Abstract: We introduce and investigate the resolvent order, which is a binary relation on the set of firmly nonexpansive map**s. It unifies well-known orders introduced by Loewner (for positive semidefinite matrices) and by Zarantonello (for projectors onto convex cones). A connection with Moreau's order of convex functions is also presented. We also construct partial orders on (quotient sets of) proximal… ▽ More We introduce and investigate the resolvent order, which is a binary relation on the set of firmly nonexpansive map**s. It unifies well-known orders introduced by Loewner (for positive semidefinite matrices) and by Zarantonello (for projectors onto convex cones). A connection with Moreau's order of convex functions is also presented. We also construct partial orders on (quotient sets of) proximal map**s and convex functions. Various examples illustrate our results. △ Less

Submitted 28 June, 2016; originally announced June 2016.

MSC Class: 06A06; 26B25; 47H05; 47H09 (Primary); 15B57; 47L07; 52A41; 90C25 (Secondary)