Showing 1–34 of 34 results for author: Luke, D R

A minimalist approach to 3D photoemission orbital tomography: algorithms and data requirements

Authors: Thi Lan Dinh, G. S. Matthijs Jansen, D. Russell Luke, Wiebke Bennecke, Stefan Mathias

Abstract: Photoemission orbital tomography provides direct access from laboratory measurements to the real-space molecular orbitals of well-ordered organic semiconductor layers. Specifically, the application of phase retrieval algorithms to photon-energy- and angle-resolved photoemission data enables the direct reconstruction of full 3D molecular orbitals without the need for simulations using density funct… ▽ More Photoemission orbital tomography provides direct access from laboratory measurements to the real-space molecular orbitals of well-ordered organic semiconductor layers. Specifically, the application of phase retrieval algorithms to photon-energy- and angle-resolved photoemission data enables the direct reconstruction of full 3D molecular orbitals without the need for simulations using density functional theory or the like. A major limitation for the direct approach has been the need for densely-sampled, well-calibrated 3D photoemission patterns. Here, we present an iterative projection algorithm that completely eliminates this challenge: for the benchmark case of the pentacene frontier orbitals, we demonstrate the reconstruction of the full orbital based on a dataset containing only four simulated photoemission momentum measurements. We discuss the algorithm performance, sampling requirements with respect to the photon energy, optimal measurement strategies, and the accuracy of orbital images that can be achieved. △ Less

Submitted 31 January, 2024; originally announced February 2024.

Comments: 27 pages, 10 figures, 44 references

Stochastic Algorithms for Large-Scale Composite Optimization: the Case of Single-Shot X-FEL Imaging

Authors: D. Russell Luke, Steffen Schultze, Helmut Grubmüller

Abstract: We apply a recently developed framework for analyzing the convergence of stochastic algorithms to the general problem of large-scale nonconvex composite optimization more generally, and nonconvex likelihood maximization in particular. Our theory is demonstrated on a stochastic gradient descent algorithm for determining the electron density of a molecule from random samples of its scattering amplit… ▽ More We apply a recently developed framework for analyzing the convergence of stochastic algorithms to the general problem of large-scale nonconvex composite optimization more generally, and nonconvex likelihood maximization in particular. Our theory is demonstrated on a stochastic gradient descent algorithm for determining the electron density of a molecule from random samples of its scattering amplitude. Numerical results on an idealized synthetic example provide a proof of concept. This opens the door to a broad range of algorithmic possibilities and provides a basis for evaluating and comparing different strategies. While this case study is very specific, it shares a structure that transfers easily to many problems of current interest, particularly in machine learning. △ Less

Submitted 24 January, 2024; originally announced January 2024.

Comments: 20 pages, 10 figures

MSC Class: 65C40; 90C26; 90C06

Convergence in Distribution of Randomized Algorithms: The Case of Partially Separable Optimization

Authors: D. Russell Luke

Abstract: We present a Markov-chain analysis of blockwise-stochastic algorithms for solving partially block-separable optimization problems. Our main contributions to the extensive literature on these methods are statements about the Markov operators and distributions behind the iterates of stochastic algorithms, and in particular the regularity of Markov operators and rates of convergence of the distributi… ▽ More We present a Markov-chain analysis of blockwise-stochastic algorithms for solving partially block-separable optimization problems. Our main contributions to the extensive literature on these methods are statements about the Markov operators and distributions behind the iterates of stochastic algorithms, and in particular the regularity of Markov operators and rates of convergence of the distributions of the corresponding Markov chains. This provides a detailed characterization of the moments of the sequences beyond just the expected behavior. This also serves as a case study of how randomization restores favorable properties to algorithms that iterations of only partial information destroys. We demonstrate this on stochastic blockwise implementations of the forward-backward and Douglas-Rachford algorithms for nonconvex (and, as a special case, convex), nonsmooth optimization. △ Less

Submitted 31 October, 2023; originally announced October 2023.

Comments: 25 pages, 43 references

MSC Class: 65C40; 90C06; 90C26; 46N30; 60J05; 49M27; 65K05

Rates of Convergence for Chains of Expansive Markov Operators

Authors: Neal Hermer, D. Russell Luke, Anja Sturm

Abstract: We provide conditions that guarantee local rates of convergence in distribution of iterated random functions that are not nonexpansive map**s in locally compact Hadamard spaces. Our results are applied to stochastic instances of common algorithms in optimization, stochastic tomography for X-FEL imaging, and a stochastic algorithm for the computation of Fréchet means in model spaces for phylogene… ▽ More We provide conditions that guarantee local rates of convergence in distribution of iterated random functions that are not nonexpansive map**s in locally compact Hadamard spaces. Our results are applied to stochastic instances of common algorithms in optimization, stochastic tomography for X-FEL imaging, and a stochastic algorithm for the computation of Fréchet means in model spaces for phylogenetic trees. △ Less

Submitted 31 October, 2023; v1 submitted 10 June, 2022; originally announced June 2022.

Comments: 29 pages, 75 references. Accepted to Transactions of Mathematics and its Applications. This is a more narrowly focused version of arXiv:2007.06479. arXiv admin note: text overlap with arXiv:2205.15897

MSC Class: 60J05; 46N10; 46N30; 65C40; 49J55

Nonexpansive Markov Operators and Random Function Iterations for Stochastic Fixed Point Problems

Authors: Neal Hermer, D. Russell Luke, Anja Sturm

Abstract: We study the convergence of random function iterations for finding an invariant measure of the corresponding Markov operator. We call the problem of finding such an invariant measure the stochastic fixed point problem. This generalizes earlier work studying the stochastic feasibility problem, namely, to find points that are, with probability 1, fixed points of the random functions. When no such po… ▽ More We study the convergence of random function iterations for finding an invariant measure of the corresponding Markov operator. We call the problem of finding such an invariant measure the stochastic fixed point problem. This generalizes earlier work studying the stochastic feasibility problem, namely, to find points that are, with probability 1, fixed points of the random functions. When no such points exist, the stochastic feasibility problem is called inconsistent, but still under certain assumptions, the more general stochastic fixed point problem has a solution and the random function iterations converge to an invariant measure for the corresponding Markov operator. We show how common structures in deterministic fixed point theory can be exploited to establish existence of invariant measures and convergence in distribution of the Markov chain. This framework specializes to many applications of current interest including, for instance, stochastic algorithms for large-scale distributed computation, and deterministic iterative procedures with computational error. The theory developed in this study provides a solid basis for describing the convergence of simple computational methods without the assumption of infinite precision arithmetic or vanishing computational errors. △ Less

Submitted 9 March, 2023; v1 submitted 31 May, 2022; originally announced May 2022.

Comments: 40 pages, 61 references, 2 figures. This is the revision of a narrowed version of arXiv:2007.06479 which could not be published in its entirety

MSC Class: 60J05; 46N10; 46N30; 65C40; 49J55

Journal ref: Journal of Convex Analysis 30 (2023), No. 4, 1073--1114

Formation of moiré interlayer excitons in space and time

Authors: David Schmitt, Jan Philipp Bange, Wiebke Bennecke, AbdulAziz AlMutairi, Kenji Watanabe, Takashi Taniguchi, Daniel Steil, D. Russell Luke, R. Thomas Weitz, Sabine Steil, G. S. Matthijs Jansen, Stephan Hofmann, Marcel Reutzel, Stefan Mathias

Abstract: Moiré superlattices in atomically thin van-der-Waals heterostructures hold great promise for an extended control of electronic and valleytronic lifetimes, the confinement of excitons in artificial moiré lattices, and the formation of novel exotic quantum phases. Such moiré-induced emergent phenomena are particularly strong for interlayer excitons, where the hole and the electron are localized in d… ▽ More Moiré superlattices in atomically thin van-der-Waals heterostructures hold great promise for an extended control of electronic and valleytronic lifetimes, the confinement of excitons in artificial moiré lattices, and the formation of novel exotic quantum phases. Such moiré-induced emergent phenomena are particularly strong for interlayer excitons, where the hole and the electron are localized in different layers of the heterostructure. In order to exploit the full potential of correlated moiré and exciton physics, a thorough understanding of the ultrafast interlayer exciton formation process and the real-space wavefunction confinement in the moiré potential is indispensable. However, direct experimental access to these parameters is limited since most excitonic quasiparticles are optically dark. Here we show that femtosecond photoemission momentum microscopy provides quantitative access to these key properties of the moiré interlayer excitons. We find that interlayer excitons are dominantly formed on the sub-50~fs timescale via interlayer tunneling at the K valleys of the Brillouin zones. In addition, we directly measure energy-momentum fingerprints of the moiré interlayer excitons by map** their spectral signatures within the mini Brillouin zone that is built up by the twisted heterostructure. From these momentum-fingerprints, we gain quantitative access to the modulation of the exciton wavefunction within the moiré potential in real-space. Our work provides the first direct access to the interlayer moiré exciton formation dynamics in space and time and reveals new opportunities to study correlated moiré and exciton physics for the future realization of exotic quantum phases of matter. △ Less

Submitted 9 December, 2021; originally announced December 2021.

Comments: 7 pages main text, 11 pages SI

Convergence of Proximal Splitting Algorithms in CAT(k) Spaces and Beyond

Authors: Florian Lauster, D. Russell Luke

Abstract: In the setting of CAT(k) spaces, common fixed point iterations built from prox map**s (e.g. prox-prox, Krasnoselsky-Mann relaxations, nonlinear projected-gradients) converge locally linearly under the assumption of linear metric subregularity. Linear metric subregularity is in any case necessary for linearly convergent fixed point sequences, so the result is tight. To show this, we develop a the… ▽ More In the setting of CAT(k) spaces, common fixed point iterations built from prox map**s (e.g. prox-prox, Krasnoselsky-Mann relaxations, nonlinear projected-gradients) converge locally linearly under the assumption of linear metric subregularity. Linear metric subregularity is in any case necessary for linearly convergent fixed point sequences, so the result is tight. To show this, we develop a theory of fixed point map**s that violate the usual assumptions of nonexpansiveness and firm nonexpansiveness in p-uniformly convex spaces. △ Less

Submitted 12 May, 2021; originally announced May 2021.

Comments: 18 pages, 13 citations. arXiv admin note: text overlap with arXiv:2104.11302

MSC Class: 47H09; 47H10; 53C22; 53C21

Journal ref: Fixed Point Theory Algorithms Sci Eng 2021, 13 (2021)

$α$-Firmly Nonexpansive Operators on Metric Spaces

Authors: Arian Bërdëllima, Florian Lauster, D. Russell Luke

Abstract: We extend to $p$-uniformly convex spaces tools from the analysis of fixed point iterations in linear spaces. This study is restricted to an appropriate generalization of single-valued, pointwise $α$-averaged map**s. Our main contribution is establishing a calculus for these map**s in p-uniformly convex spaces, showing in particular how the property is preserved under compositions and convex co… ▽ More We extend to $p$-uniformly convex spaces tools from the analysis of fixed point iterations in linear spaces. This study is restricted to an appropriate generalization of single-valued, pointwise $α$-averaged map**s. Our main contribution is establishing a calculus for these map**s in p-uniformly convex spaces, showing in particular how the property is preserved under compositions and convex combinations. This is of central importance to splitting algorithms that are built by such convex combinations and compositions, and reduces the convergence analysis to simply verifying $α$-firm nonexpansiveness of the individual components at fixed points of the splitting algorithms. Our convergence analysis differs from what can be found in the previous literature in that only $α$-firm nonexpansiveness with respect to fixed points is required. Indeed we show that, if the fixed point map** is pointwise nonexpansive at all cluster points, then these cluster points are in fact fixed points, and convergence of the sequence follows. Additionally, we provide a quantitative convergence analysis built on the notion of gauge metric subregularity, which we show is necessary for quantifiable convergence estimates. This allows one for the first time to prove convergence of a tremendous variety of splitting algorithms in spaces with curvature bounded from above. △ Less

Submitted 22 April, 2021; originally announced April 2021.

Comments: 26 pages, 38 references

MSC Class: 47H09; 47H10; 53C22; 53C21

Random Function Iterations for Stochastic Fixed Point Problems

Authors: Neal Hermer, D. Russell Luke, Anja Sturm

Abstract: We study the convergence of random function iterations for finding an invariant measure of the corresponding Markov operator. We call the problem of finding such an invariant measure the stochastic fixed point problem. This generalizes earlier work studying the stochastic feasibility problem}, namely, to find points that are, with probability 1, fixed points of the random functions [Hermer, Luke,… ▽ More We study the convergence of random function iterations for finding an invariant measure of the corresponding Markov operator. We call the problem of finding such an invariant measure the stochastic fixed point problem. This generalizes earlier work studying the stochastic feasibility problem}, namely, to find points that are, with probability 1, fixed points of the random functions [Hermer, Luke, Sturm, 2019]. When no such points exist, the stochastic feasibility problem is called inconsistent, but still under certain assumptions, the more general stochastic fixed point problem has a solution and the random function iterations converge to an invariant measure for the corresponding Markov operator. There are two major types of convergence: almost sure convergence of the iterates to a fixed point in the case of stochastic feasibility, and convergence in distribution more generally. We show how common structures in deterministic fixed point theory can be exploited to establish existence of invariant measures and convergence of the Markov chain. We show that weaker assumptions than are usually encountered in the analysis of Markov chains guarantee linear/geometric convergence. This framework specializes to many applications of current interest including, for instance, stochastic algorithms for large-scale distributed computation, and deterministic iterative procedures with computational error. The theory developed in this study provides a solid basis for describing the convergence of simple computational methods without the assumption of infinite precision arithmetic or vanishing computational errors. △ Less

Submitted 23 March, 2022; v1 submitted 13 July, 2020; originally announced July 2020.

Comments: 54 pages, 96 references

MSC Class: 60J05; 46N10; 46N30; 65C40; 49J55

Efficient orbital imaging based on ultrafast momentum microscopy and sparsity-driven phase retrieval

Authors: G. S. M. Jansen, M. Keunecke, M. Düvel, C. Möller, D. Schmitt, W. Bennecke, F. J. S. Kappert, D. Steil, D. R. Luke, S. Steil, S. Mathias

Abstract: We present energy-resolved photoelectron momentum maps for orbital tomography that have been collected with a novel and efficient time-of-flight momentum microscopy setup. This setup is combined with a 0.5 MHz table-top femtosecond extreme-ultraviolet light source, which enables unprecedented speed in data collection and paves the way towards time-resolved orbital imaging experiments in the future… ▽ More We present energy-resolved photoelectron momentum maps for orbital tomography that have been collected with a novel and efficient time-of-flight momentum microscopy setup. This setup is combined with a 0.5 MHz table-top femtosecond extreme-ultraviolet light source, which enables unprecedented speed in data collection and paves the way towards time-resolved orbital imaging experiments in the future. Moreover, we take a significant step forward in the data analysis procedure for orbital imaging, and present a sparsity-driven approach to the required phase retrieval problem, which uses only the number of non-zero pixels in the orbital. Here, no knowledge of the object support is required, and the sparsity number can easily be determined from the measured data. Used in the relaxed averaged alternating reflections algorithm, this sparsity constraint enables fast and reliable phase retrieval for our experimental as well as noise-free and noisy simulated photoelectron momentum map data. △ Less

Submitted 23 March, 2020; v1 submitted 29 January, 2020; originally announced January 2020.

Journal ref: New Journal of Physics, vol. 22, 2020

Convergence Analysis of the Relaxed Douglas-Rachford Algorithm

Authors: D. Russell Luke, Anna-Lena Martins

Abstract: Motivated by nonconvex, inconsistent feasibility problems in imaging, the relaxed alternating averaged reflections algorithm, or relaxed Douglas-Rachford algorithm (DR$λ$), was first proposed over a decade ago. Convergence results for this algorithm are limited either to convex feasibility or consistent nonconvex feasibility with strong assumptions on the regularity of the underlying sets. Using a… ▽ More Motivated by nonconvex, inconsistent feasibility problems in imaging, the relaxed alternating averaged reflections algorithm, or relaxed Douglas-Rachford algorithm (DR$λ$), was first proposed over a decade ago. Convergence results for this algorithm are limited either to convex feasibility or consistent nonconvex feasibility with strong assumptions on the regularity of the underlying sets. Using an analytical framework depending only on metric subregularity and pointwise almost averagedness, we analyze the convergence behavior of DR$λ$ for feasibility problems that are both nonconvex and inconsistent. We introduce a new type of regularity of sets, called super-regular at a distance, to establish sufficient conditions for local linear convergence of the corresponding sequence. These results subsume and extend existing results for this algorithm. △ Less

Submitted 23 May, 2019; v1 submitted 28 November, 2018; originally announced November 2018.

Comments: 36 pages, 10 figures, 25 citations

MSC Class: 65K10; 49K40; 49M05; 65K05; 90C26; 49M20; 49J53

Journal ref: SIAM J. on Optimization, 30(1):542--584(2020)

Optimization on Spheres: Models and Proximal Algorithms with Computational Performance Comparisons

Authors: D. Russell Luke, Shoham Sabach, Marc Teboulle

Abstract: We present a unified treatment of the abstract problem of finding the best approximation between a cone and spheres in the image of affine transformations. Prominent instances of this problem are phase retrieval and source localization. The common geometry binding these problems permits a generic application of algorithmic ideas and abstract convergence results for nonconvex optimization. We organ… ▽ More We present a unified treatment of the abstract problem of finding the best approximation between a cone and spheres in the image of affine transformations. Prominent instances of this problem are phase retrieval and source localization. The common geometry binding these problems permits a generic application of algorithmic ideas and abstract convergence results for nonconvex optimization. We organize variational models for this problem into three different classes and derive the main algorithmic approaches within these classes (13 in all). We identify the central ideas underlying these methods and provide thorough numerical benchmarks comparing their performance on synthetic and laboratory data. The software and data of our experiments are all publicly accessible. We also introduce one new algorithm, a cyclic relaxed Douglas-Rachford algorithm, which outperforms all other algorithms by every measure: speed, stability and accuracy. The analysis of this algorithm remains open. △ Less

Submitted 5 October, 2018; originally announced October 2018.

Comments: 23 pages, 11 benchmarking studies

Journal ref: SIAM J. Mathematics of Data Science, 1(3), 408-445 (2019)

Random Function Iterations for Consistent Stochastic Feasibility

Authors: Neal Hermer, D. Russell Luke, Anja Sturm

Abstract: We study the convergence of stochastic fixed point iterations in the consistent case (in the sense of Butnariu and Flåm (1995)) in several different settings, under decreasingly restrictive regularity assumptions of the fixed point map**s. The iterations are Markov chains and, for the purposes of this study, convergence is understood in very restrictive terms. We show that sufficient conditions… ▽ More We study the convergence of stochastic fixed point iterations in the consistent case (in the sense of Butnariu and Flåm (1995)) in several different settings, under decreasingly restrictive regularity assumptions of the fixed point map**s. The iterations are Markov chains and, for the purposes of this study, convergence is understood in very restrictive terms. We show that sufficient conditions for geometric (linear) convergence in expectation of stochastic projection algorithms presented in Nedić (2011), are in fact necessary for geometric (linear) convergence in expectation more generally of iterated random functions. △ Less

Submitted 21 September, 2018; v1 submitted 16 August, 2018; originally announced August 2018.

Comments: 29 pages, 4 figures

MSC Class: 60J05; 52A22; 49J55 (Primary) 49J53; 65K05 (Secondary)

Journal ref: Numerical Functional Analysis and Optimization 40(4):386--420 (2019)

Characterizations of Super-regularity and its Variants

Authors: Aris Daniilidis, D. Russell Luke, Matthew K. Tam

Abstract: Convergence of projection-based methods for nonconvex set feasibility problems has been established for sets with ever weaker regularity assumptions. What has not kept pace with these developments is analogous results for convergence of optimization problems with correspondingly weak assumptions on the value functions. Indeed, one of the earliest classes of nonconvex sets for which convergence res… ▽ More Convergence of projection-based methods for nonconvex set feasibility problems has been established for sets with ever weaker regularity assumptions. What has not kept pace with these developments is analogous results for convergence of optimization problems with correspondingly weak assumptions on the value functions. Indeed, one of the earliest classes of nonconvex sets for which convergence results were obtainable, the class of so-called super-regular sets introduced by Lewis, Luke and Malick (2009), has no functional counterpart. In this work, we amend this gap in the theory by establishing the equivalence between a property slightly stronger than super-regularity, which we call Clarke super-regularity, and subsmootheness of sets as introduced by Aussel, Daniilidis and Thibault (2004). The bridge to functions shows that approximately convex functions studied by Ngai, Luc and Théra (2000) are those which have Clarke super-regular epigraphs. Further classes of regularity of functions based on the corresponding regularity of their epigraph are also discussed. △ Less

Submitted 15 August, 2018; originally announced August 2018.

Comments: 15 pages, 2 figures

Journal ref: Splitting Algorithms, Modern Operator Theory and Applications (2019)

Phase retrieval with sparse phase constraint

Authors: Hieu Thao Nguyen, D. Russell Luke, Oleg Soloviev, Michel Verhaegen

Abstract: For the first time, this paper investigates the phase retrieval problem with the assumption that the phase (of the complex signal) is sparse in contrast to the sparsity assumption on the signal itself as considered in the literature of sparse signal processing. The intended application of this new problem model, which will be conducted in a follow-up paper, is to practical phase retrieval problems… ▽ More For the first time, this paper investigates the phase retrieval problem with the assumption that the phase (of the complex signal) is sparse in contrast to the sparsity assumption on the signal itself as considered in the literature of sparse signal processing. The intended application of this new problem model, which will be conducted in a follow-up paper, is to practical phase retrieval problems where the aberration phase is sparse with respect to the orthogonal basis of Zernike polynomials. Such a problem is called sparse phase retrieval (SPR) problem in this paper. When the amplitude modulation at the exit pupil is uniform, a new scheme of sparsity regularization on phase is proposed to capture the sparsity property of the SPR problem. Based on this regularization scheme, we design and analyze an efficient solution method, named SROP algorithm, for solving SPR given only a single intensity point-spread-function image. The algorithm is a combination of the Gerchberg-Saxton algorithm with the newly proposed sparsity regularization on the phase. The latter regularization step is mathematically a rotation but with direction varying in iterations. Surprisingly, this rotation is shown to be a metric projection on an auxiliary set which is independent of iterations. As a consequence, SROP algorithm is proved to be the cyclic projections algorithm for solving a feasibility problem involving three auxiliary sets. Analyzing regularity properties of the latter auxiliary sets, we obtain convergence results for SROP algorithm based on recent convergence theory for the cyclic projections algorithm. Numerical results show clear effectiveness of the new regularization scheme for solving the SPR problem. △ Less

Submitted 26 January, 2019; v1 submitted 4 April, 2018; originally announced April 2018.

MSC Class: 65K15; 65Z05; 65T50; 78A45; 78A46; 90C26

Block-coordinate primal-dual method for the nonsmooth minimization over linear constraints

Authors: D. Russell Luke, Yura Malitsky

Abstract: We consider the problem of minimizing a convex, separable, nonsmooth function subject to linear constraints. The numerical method we propose is a block-coordinate extension of the Chambolle-Pock primal-dual algorithm. We prove convergence of the method without resorting to assumptions like smoothness or strong convexity of the objective, full-rank condition on the matrix, strong duality or even co… ▽ More We consider the problem of minimizing a convex, separable, nonsmooth function subject to linear constraints. The numerical method we propose is a block-coordinate extension of the Chambolle-Pock primal-dual algorithm. We prove convergence of the method without resorting to assumptions like smoothness or strong convexity of the objective, full-rank condition on the matrix, strong duality or even consistency of the linear system. Freedom from imposing the latter assumption permits convergence guarantees for misspecified or noisy systems. △ Less

Submitted 15 January, 2018; originally announced January 2018.

Comments: 25 pages 46 references, 3 tables and 3 figures

MSC Class: 49M29 65K10 65Y20 90C25

Journal ref: Distributed and Large-Scale Optimization (2018)

Necessary conditions for linear convergence of iterated expansive, set-valued map**s with application to alternating projections

Authors: D. Russell Luke, Marc Teboulle, Nguyen H. Thao

Abstract: We present necessary conditions for monotonicity, in one form or another, of fixed point iterations of map**s that violate the usual nonexpansive property. We show that most reasonable notions of linear-type monotonicity of fixed point sequences imply {\em metric subregularity}. This is specialized to the alternating projections iteration where the metric subregularity property takes on a distin… ▽ More We present necessary conditions for monotonicity, in one form or another, of fixed point iterations of map**s that violate the usual nonexpansive property. We show that most reasonable notions of linear-type monotonicity of fixed point sequences imply {\em metric subregularity}. This is specialized to the alternating projections iteration where the metric subregularity property takes on a distinct geometric characterization of sets at points of intersection called {\em subtransversality}. Our more general results for fixed point iterations are specialized to establish the necessity of subtransversality for consistent feasibility with a number of reasonable types of sequential monotonicity, under varying degrees of assumptions on the regularity of the sets. △ Less

Submitted 25 May, 2018; v1 submitted 28 April, 2017; originally announced April 2017.

Comments: 22 pages, 35 references

MSC Class: 49J53; 65K10

Journal ref: Mathematical Programming A 180(1), 1-31 (2020)

Symbolic computation with monotone operators

Authors: Florian Lauster, D. Russell Luke, Matthew K. Tam

Abstract: We consider a class of monotone operators which are appropriate for symbolic representation and manipulation within a computer algebra system. Various structural properties of the class (e.g., closure under taking inverses, resolvents) are investigated as well as the role played by maximal monotonicity within the class. In particular, we show that there is a natural correspondence between our clas… ▽ More We consider a class of monotone operators which are appropriate for symbolic representation and manipulation within a computer algebra system. Various structural properties of the class (e.g., closure under taking inverses, resolvents) are investigated as well as the role played by maximal monotonicity within the class. In particular, we show that there is a natural correspondence between our class of monotone operators and the subdifferentials of convex functions belonging to a class of convex functions deemed suitable for symbolic computation of Fenchel conjugates which were previously studied by Bauschke & von Mohrenschildt and by Borwein & Hamilton. A number of illustrative examples utilizing the introduced class of operators are provided including computation of proximity operators, recovery of a convex penalty function associated with the hard thresholding operator, and computation of superexpectations, superdistributions and superquantiles with specialization to risk measures. △ Less

Submitted 17 March, 2017; originally announced March 2017.

Comments: 17 pages, 2 figures

MSC Class: 47H05; 47N10; 68W30

Journal ref: Set-Valued and Variational Analysis 26, 2018

A Globally Linearly Convergent Method for Pointwise Quadratically Supportable Convex-Concave Saddle Point Problems

Authors: D. Russell Luke, Ron Shefi

Abstract: We study the \emph{Proximal Alternating Predictor-Corrector} (PAPC) algorithm introduced recently by Drori, Sabach and Teboulle to solve nonsmooth structured convex-concave saddle point problems consisting of the sum of a smooth convex function, a finite collection of nonsmooth convex functions and bilinear terms. We introduce the notion of pointwise quadratic supportability, which is a relaxation… ▽ More We study the \emph{Proximal Alternating Predictor-Corrector} (PAPC) algorithm introduced recently by Drori, Sabach and Teboulle to solve nonsmooth structured convex-concave saddle point problems consisting of the sum of a smooth convex function, a finite collection of nonsmooth convex functions and bilinear terms. We introduce the notion of pointwise quadratic supportability, which is a relaxation of a standard strong convexity assumption and allows us to show that the primal sequence is R-linearly convergent to an optimal solution and the primal-dual sequence is globally Q-linearly convergent. We illustrate the proposed method on total variation denoising problems and on locally adaptive estimation in signal/image deconvolution and denoising with multiresolution statistical constraints. △ Less

Submitted 28 February, 2017; originally announced February 2017.

Comments: 34 pages, 18 figures

MSC Class: 49J52; 49M20; 90C26

Journal ref: J. Math. Anal. Appl., 457(2):1568--1590, 2018

About subtransversality of collections of sets

Authors: Alexander Y. Kruger, D. Russell Luke, Nguyen H. Thao

Abstract: We provide dual sufficient conditions for subtransversality of collections of sets in an Asplund space setting. For the convex case, we formulate a necessary and sufficient dual criterion of subtransversality in general Banach spaces. Our more general results suggest an intermediate notion of subtransversality, what we call weak intrinsic subtransversality, which lies between intrinsic transversal… ▽ More We provide dual sufficient conditions for subtransversality of collections of sets in an Asplund space setting. For the convex case, we formulate a necessary and sufficient dual criterion of subtransversality in general Banach spaces. Our more general results suggest an intermediate notion of subtransversality, what we call weak intrinsic subtransversality, which lies between intrinsic transversality and subtransversality in Asplund spaces. △ Less

Submitted 1 July, 2017; v1 submitted 15 November, 2016; originally announced November 2016.

Comments: 23 pages

MSC Class: 49J53; 65K10 (Primary); 49K40; 49M05; 49M37; 65K05; 90C30 (Secondary)

Journal ref: Set-Valued and Variational Analysis 25(4), 701-729 (2017)

Quantitative convergence analysis of iterated expansive, set-valued map**s

Authors: D. Russell Luke, Nguyen H. Thao, Matthew K. Tam

Abstract: We develop a framework for quantitative convergence analysis of Picard iterations of expansive set-valued fixed point map**s. There are two key components of the analysis. The first is a natural generalization of single-valued averaged map**s to expansive, set-valued map**s that characterizes a type of strong calmness of the fixed point map**. The second component to this analysis is an ex… ▽ More We develop a framework for quantitative convergence analysis of Picard iterations of expansive set-valued fixed point map**s. There are two key components of the analysis. The first is a natural generalization of single-valued averaged map**s to expansive, set-valued map**s that characterizes a type of strong calmness of the fixed point map**. The second component to this analysis is an extension of the well-established notion of metric subregularity -- or inverse calmness -- of the map** at fixed points. Convergence of expansive fixed point iterations is proved using these two properties, and quantitative estimates are a natural byproduct of the framework. To demonstrate the application of the theory, we prove for the first time a number of results showing local linear convergence of nonconvex cyclic projections for inconsistent (and consistent) feasibility problems, local linear convergence of the forward-backward algorithm for structured optimization without convexity, strong or otherwise, and local linear convergence of the Douglas--Rachford algorithm for structured nonconvex minimization. This theory includes earlier approaches for known results, convex and nonconvex, as special cases. △ Less

Submitted 25 March, 2017; v1 submitted 18 May, 2016; originally announced May 2016.

Comments: 41 pages 70 references. Detailed examples added and more historical commentary (with citations)

MSC Class: 49J53; 65K10; 49K40; 49M05; 49M27; 65K05; 90C26

Journal ref: Mathematics of Operations Research, online 2018

Set Regularities and Feasibility Problems

Authors: Alexander Y. Kruger, D. Russell Luke, Nguyen H. Thao

Abstract: We synthesize and unify notions of regularity, both of individual sets and of collections of sets, as they appear in the convergence theory of projection methods for consistent feasibility problems. Several new characterizations of regularities are presented which shed light on the relations between seemingly different ideas and point to possible necessary conditions for local linear convergence o… ▽ More We synthesize and unify notions of regularity, both of individual sets and of collections of sets, as they appear in the convergence theory of projection methods for consistent feasibility problems. Several new characterizations of regularities are presented which shed light on the relations between seemingly different ideas and point to possible necessary conditions for local linear convergence of fundamental algorithms. △ Less

Submitted 16 February, 2016; originally announced February 2016.

MSC Class: 49J53; 65K10; 49K40; 49M05; 49M37; 65K05; 90C30

Journal ref: Math. Program., Ser. B (2018) 168:279-311

Local Linear Convergence of the ADMM/Douglas--Rachford Algorithms without Strong Convexity and Application to Statistical Imaging

Authors: Timo Aspelmeier, C. Charitha, D. Russell Luke

Abstract: We consider the problem of minimizing the sum of a convex function and a convex function composed with an injective linear map**. For such problems, subject to a coercivity condition at fixed points of the corresponding Picard iteration, iterates of the alternating directions method of multipliers converge locally linearly to points from which the solution to the original problem can be computed… ▽ More We consider the problem of minimizing the sum of a convex function and a convex function composed with an injective linear map**. For such problems, subject to a coercivity condition at fixed points of the corresponding Picard iteration, iterates of the alternating directions method of multipliers converge locally linearly to points from which the solution to the original problem can be computed. Our proof strategy uses duality and strong metric subregularity of the Douglas--Rachford fixed point map**. Our analysis does not require strong convexity and yields error bounds to the set of model solutions. We show in particular that convex piecewise linear-quadratic functions naturally satisfy the requirements of the theory, guaranteeing eventual linear convergence of both the Douglas--Rachford algorithm and the alternating directions method of multipliers for this class of objectives under mild assumptions on the set of fixed points. We demonstrate this result on quantitative image deconvolution and denoising with multiresolution statistical constraints. △ Less

Submitted 15 March, 2016; v1 submitted 18 August, 2015; originally announced August 2015.

Comments: Revised manuscript: 30 pages including 9 figures, one appendix and 57 references. Difference from version 2: title and abstract changed, one new figure added, and a posteriori error estimates in numerical experiments reported

MSC Class: 49J52; 49M20; 90C26

Journal ref: SIAM J. Imaging Sciences 9(2) pp. 842-868(2016)

Lagrange Multipliers, (Exact) Regularization and Error Bounds for Monotone Variational Inequalities

Authors: C. Charitha, Joydeep Dutta, D. Russell Luke

Abstract: We examine two central regularization strategies for monotone variational inequalities, the first a direct regularization of the operative monotone map**, and the second via regularization of the associated dual gap function. A key link in the relationship between the solution sets to these various regularized problems is the idea of exact regularization, which, in turn, is fundamentally associa… ▽ More We examine two central regularization strategies for monotone variational inequalities, the first a direct regularization of the operative monotone map**, and the second via regularization of the associated dual gap function. A key link in the relationship between the solution sets to these various regularized problems is the idea of exact regularization, which, in turn, is fundamentally associated with the existence of Lagrange multipliers for the regularized variational inequality. A regularization is said to be exact if a solution to the regularized problem is a solution to the unregularized problem for all parameters beyond a certain value. The Lagrange multipliers corresponding to a particular regularization of a variational inequality, on the other hand, are defined via the dual gap function. Our analysis suggests various conceptual, iteratively regularized numerical schemes, for which we provide error bounds, and hence stop** criteria, under the additional assumption that the solution set to the unregularized problem is what we call weakly sharp of order greater than one. △ Less

Submitted 2 November, 2015; v1 submitted 14 October, 2014; originally announced October 2014.

Comments: Updated version after referee comments. 34 pages, 1 table, 20 references

MSC Class: 49J40; 47J20

Journal ref: Mathematical Programming A(2017) 161: 519--549

Proximal Heterogeneous Block Input-Output Method and application to Blind Ptychographic Diffraction Imaging

Authors: Robert Hesse, D. Russell Luke, Shoham Sabach, Matthew K. Tam

Abstract: We propose a general alternating minimization algorithm for nonconvex optimization problems with separable structure and nonconvex coupling between blocks of variables. To fix our ideas, we apply the methodology to the problem of blind ptychographic imaging. Compared to other schemes in the literature, our approach differs in two ways: (i) it is posed within a clear mathematical framework with pra… ▽ More We propose a general alternating minimization algorithm for nonconvex optimization problems with separable structure and nonconvex coupling between blocks of variables. To fix our ideas, we apply the methodology to the problem of blind ptychographic imaging. Compared to other schemes in the literature, our approach differs in two ways: (i) it is posed within a clear mathematical framework with practically verifiable assumptions, and (ii) under the given assumptions, it is provably convergent to critical points. A numerical comparison of our proposed algorithm with the current state-of-the-art on simulated and experimental data validates our approach and points toward directions for further improvement. △ Less

Submitted 8 August, 2014; originally announced August 2014.

Comments: 32 pages, 3 tables, 5 figures

MSC Class: 90C05; 90C25; 90C30; 90C52; 65K05

Journal ref: SIAM J. on Imaging Sciences, 8(1):426--457 (2015)

Alternating Projections and Douglas-Rachford for Sparse Affine Feasibility

Authors: Robert Hesse, D. Russell Luke, Patrick Neumann

Abstract: The problem of finding a vector with the fewest nonzero elements that satisfies an underdetermined system of linear equations is an NP-complete problem that is typically solved numerically via convex heuristics or nicely-behaved nonconvex relaxations. In this work we consider elementary methods based on projections for solving a sparse feasibility problem without employing convex heuristics. In a… ▽ More The problem of finding a vector with the fewest nonzero elements that satisfies an underdetermined system of linear equations is an NP-complete problem that is typically solved numerically via convex heuristics or nicely-behaved nonconvex relaxations. In this work we consider elementary methods based on projections for solving a sparse feasibility problem without employing convex heuristics. In a recent paper Bauschke, Luke, Phan and Wang (2014) showed that, locally, the fundamental method of alternating projections must converge linearly to a solution to the sparse feasibility problem with an affine constraint. In this paper we apply different analytical tools that allow us to show global linear convergence of alternating projections under familiar constraint qualifications. These analytical tools can also be applied to other algorithms. This is demonstrated with the prominent Douglas-Rachford algorithm where we establish local linear convergence of this method applied to the sparse affine feasibility problem. △ Less

Submitted 14 March, 2014; v1 submitted 8 July, 2013; originally announced July 2013.

Comments: 29 pages, 2 figures, 37 references. Much expanded version from last submission. Title changed to reflect new developments

MSC Class: Primary 49J52; 49M20; 90C30; Secondary 15A29; 47H09; 65K05; 65K10; 94A08

Journal ref: IEEE Transactions on Signal Processing, 62(18):4868--4881, 2014

Real-Time Phase Masks for Interactive Stimulation of Optogenetic Neurons

Authors: Stephan C. Kramer, Johannes Hagemann, D. Russell Luke

Abstract: Experiments with networks of optogenetically altered neurons require stimulation with high spatio-temporal selectivity. Computer-assisted holography is an energy-efficient method for robust and reliable addressing of single neurons on the millisecond-timescale inherent to biologial information processing. We show that real-time control of neurons can be achieved by a CUDA-based hologram computatio… ▽ More Experiments with networks of optogenetically altered neurons require stimulation with high spatio-temporal selectivity. Computer-assisted holography is an energy-efficient method for robust and reliable addressing of single neurons on the millisecond-timescale inherent to biologial information processing. We show that real-time control of neurons can be achieved by a CUDA-based hologram computation. △ Less

Submitted 1 February, 2013; originally announced February 2013.

Comments: 8 pages, 5 figures, 20 references

MSC Class: 68N25; 68N19; 68W10; 68U10; 49M27; 90-04; 90C26; 78M50; 92C20; 85-08

Nonconvex notions of regularity and convergence of fundamental algorithms for feasibility problems

Authors: Robert Hesse, D. Russell Luke

Abstract: We consider projection algorithms for solving (nonconvex) feasibility problems in Euclidean spaces. Of special interest are the Method of Alternating Projections (MAP) and the Douglas-Rachford or Averaged Alternating Reflection Algorithm (AAR). In the case of convex feasibility, firm nonexpansiveness of projection map**s is a global property that yields global convergence of MAP and for consiste… ▽ More We consider projection algorithms for solving (nonconvex) feasibility problems in Euclidean spaces. Of special interest are the Method of Alternating Projections (MAP) and the Douglas-Rachford or Averaged Alternating Reflection Algorithm (AAR). In the case of convex feasibility, firm nonexpansiveness of projection map**s is a global property that yields global convergence of MAP and for consistent problems AAR. Based on (ε, δ)-regularity of sets developed by Bauschke, Luke, Phan and Wang in 2012, a relaxed local version of firm nonexpansiveness with respect to the intersection is introduced for consistent feasibility problems. Together with a coercivity condition that relates to the regularity of the intersection, this yields local linear convergence of MAP for a wide class of nonconvex problems, △ Less

Submitted 24 June, 2013; v1 submitted 13 December, 2012; originally announced December 2012.

Comments: 22 pages, no figures, 30 references

MSC Class: 49J52; 49M20 (Primary) 47H09; 65K05; 65K10; 90C26 (Secondary)

Journal ref: SIAM J. Optim. 23(4), 2397-2419, 2013

Restricted normal cones and sparsity optimization with affine constraints

Authors: Heinz H. Bauschke, D. Russell Luke, Hung M. Phan, Xianfu Wang

Abstract: The problem of finding a vector with the fewest nonzero elements that satisfies an underdetermined system of linear equations is an NP-complete problem that is typically solved numerically via convex heuristics or nicely-behaved non convex relaxations. In this paper we consider the elementary method of alternating projections (MAP) for solving the sparsity optimization problem without employing co… ▽ More The problem of finding a vector with the fewest nonzero elements that satisfies an underdetermined system of linear equations is an NP-complete problem that is typically solved numerically via convex heuristics or nicely-behaved non convex relaxations. In this paper we consider the elementary method of alternating projections (MAP) for solving the sparsity optimization problem without employing convex heuristics. In a parallel paper we recently introduced the restricted normal cone which generalizes the classical Mordukhovich normal cone and reconciles some fundamental gaps in the theory of sufficient conditions for local linear convergence of the MAP algorithm. We use the restricted normal cone together with the notion of superregularity, which is naturally satisfied for the affine sparse optimization problem, to obtain local linear convergence results with estimates for the radius of convergence of the MAP algorithm applied to sparsity optimization with an affine constraint. △ Less

Submitted 2 May, 2012; originally announced May 2012.

MSC Class: 49J52; 49M20; 90C26 (Primary) 15A29; 47H09; 65K05; 65K10; 94A08 (Secondary)

Restricted normal cones and the method of alternating projections

Authors: Heinz H. Bauschke, D. Russell Luke, Hung M. Phan, Xianfu Wang

Abstract: The method of alternating projections (MAP) is a common method for solving feasibility problems. While employed traditionally to subspaces or to convex sets, little was known about the behavior of the MAP in the nonconvex case until 2009, when Lewis, Luke, and Malick derived local linear convergence results provided that a condition involving normal cones holds and at least one of the sets is supe… ▽ More The method of alternating projections (MAP) is a common method for solving feasibility problems. While employed traditionally to subspaces or to convex sets, little was known about the behavior of the MAP in the nonconvex case until 2009, when Lewis, Luke, and Malick derived local linear convergence results provided that a condition involving normal cones holds and at least one of the sets is superregular (a property less restrictive than convexity). However, their results failed to capture very simple classical convex instances such as two lines in three-dimensional space. In this paper, we extend and develop the Lewis-Luke-Malick framework so that not only any two linear subspaces but also any two closed convex sets whose relative interiors meet are covered. We also allow for sets that are more structured such as unions of convex sets. The key tool required is the restricted normal cone, which is a generalization of the classical Mordukhovich normal cone. We thoroughly study restricted normal cones from the viewpoint of constraint qualifications and regularity. Numerous examples are provided to illustrate the theory. △ Less

Submitted 2 May, 2012; originally announced May 2012.

MSC Class: 49J52; 49M20 (Primary) 47H09; 65K05; 65K10; 90C26 (Secondary)

Prox-regularity of rank constraint sets and implications for algorithms

Authors: D. Russell Luke

Abstract: We present an analysis of sets of matrices with rank less than or equal to a specified number $s$. We provide a simple formula for the normal cone to such sets, and use this to show that these sets are prox-regular at all points with rank exactly equal to $s$. The normal cone formula appears to be new. This allows for easy application of prior results guaranteeing local linear convergence of the f… ▽ More We present an analysis of sets of matrices with rank less than or equal to a specified number $s$. We provide a simple formula for the normal cone to such sets, and use this to show that these sets are prox-regular at all points with rank exactly equal to $s$. The normal cone formula appears to be new. This allows for easy application of prior results guaranteeing local linear convergence of the fundamental alternating projection algorithm between sets, one of which is a rank constraint set. We apply this to show local linear convergence of another fundamental algorithm, approximate steepest descent. Our results apply not only to linear systems with rank constraints, as has been treated extensively in the literature, but also nonconvex systems with rank constraints. △ Less

Submitted 3 December, 2012; v1 submitted 2 December, 2011; originally announced December 2011.

Comments: 12 pages, 24 references. Revised manuscript to appear in the Journal of Mathematical Imaging and Vision

MSC Class: 49M20; 65K10; 90C30

Journal ref: J. Mathematical Imaging and Vision, 47(3):231--238. 2013

Local Linear Convergence of Approximate Projections onto Regularized Sets

Authors: D. Russell Luke

Abstract: The numerical properties of algorithms for finding the intersection of sets depend to some extent on the regularity of the sets, but even more importantly on the regularity of the intersection. The alternating projection algorithm of von Neumann has been shown to converge locally at a linear rate dependent on the regularity modulus of the intersection. In many applications, however, the sets in qu… ▽ More The numerical properties of algorithms for finding the intersection of sets depend to some extent on the regularity of the sets, but even more importantly on the regularity of the intersection. The alternating projection algorithm of von Neumann has been shown to converge locally at a linear rate dependent on the regularity modulus of the intersection. In many applications, however, the sets in question come from inexact measurements that are matched to idealized models. It is unlikely that any such problems in applications will enjoy metrically regular intersection, let alone set intersection. We explore a regularization strategy that generates an intersection with the desired regularity properties. The regularization, however, can lead to a significant increase in computational complexity. In a further refinement, we investigate and prove linear convergence of an approximate alternating projection algorithm. The analysis provides a regularization strategy that fits naturally with many ill-posed inverse problems, and a mathematically sound stop** criterion for extrapolated, approximate algorithms. The theory is demonstrated on the phase retrieval problem with experimental data. The conventional early termination applied in practice to unregularized, consistent problems in diffraction imaging can be justified fully in the framework of this analysis providing, for the first time, proof of convergence of alternating approximate projections for finite dimensional, consistent phase retrieval problems. △ Less

Submitted 14 September, 2011; v1 submitted 10 August, 2011; originally announced August 2011.

Comments: 23 pages, 5 figures

MSC Class: 49M20; 65K10; 90C30

Journal ref: Nonlinear Analysis, 75:1531--1546, 2012

Robust Mixing for Ab-Initio Quantum Mechanical Calculations

Authors: L. D. Marks, D. R. Luke

Abstract: We study the general problem of mixing for ab-initio quantum-mechanical problems. Guided by general mathematical principles and the underlying physics, we propose a multisecant form of Broydens second method for solving the self-consistent field equations of Kohn-Sham density functional theory. The algorithm is robust, requires relatively little finetuning and appears to outperform the current s… ▽ More We study the general problem of mixing for ab-initio quantum-mechanical problems. Guided by general mathematical principles and the underlying physics, we propose a multisecant form of Broydens second method for solving the self-consistent field equations of Kohn-Sham density functional theory. The algorithm is robust, requires relatively little finetuning and appears to outperform the current state of the art, converging for cases that defeat many other methods. We compare our technique to the conventional methods for problems ranging from simple to nearly pathological. △ Less

Submitted 20 January, 2008; originally announced January 2008.

Comments: 32 Pages, 4 figures

Relaxed Averaged Alternating Reflections for Diffraction Imaging

Authors: D. Russell Luke

Abstract: We report on progress in algorithms for iterative phase retrieval. The theory of convex optimization is used to develop and to gain insight into counterparts for the nonconvex problem of phase retrieval. We propose a relaxation of averaged alternating reflectors and determine the fixed point set of the related operator in the convex case. A numerical study supports our theoretical observations a… ▽ More We report on progress in algorithms for iterative phase retrieval. The theory of convex optimization is used to develop and to gain insight into counterparts for the nonconvex problem of phase retrieval. We propose a relaxation of averaged alternating reflectors and determine the fixed point set of the related operator in the convex case. A numerical study supports our theoretical observations and demonstrates the effectiveness of the algorithm compared to the current state of the art. △ Less

Submitted 11 May, 2004; originally announced May 2004.

Comments: 13 pages, 19 figures

MSC Class: 30E10; 49M27; 49M45; 78M50

Journal ref: Inverse Problems 21:37-50(2005)