Showing 1–50 of 123 results for author: Lorenz, D

On inertial Levenberg-Marquardt type methods for solving nonlinear ill-posed operator equations

Authors: Antonio Leitão, Joel C. Rabelo, Dirk A. Lorenz, Maximilian Winkler

Abstract: In these notes we propose and analyze an inertial type method for obtaining stable approximate solutions to nonlinear ill-posed operator equations. The method is based on the Levenberg-Marquardt (LM) iteration. The main obtained results are: monotonicity and convergence for exact data, stability and semi-convergence for noisy data. Regarding numerical experiments we consider: i) a parameter identi… ▽ More In these notes we propose and analyze an inertial type method for obtaining stable approximate solutions to nonlinear ill-posed operator equations. The method is based on the Levenberg-Marquardt (LM) iteration. The main obtained results are: monotonicity and convergence for exact data, stability and semi-convergence for noisy data. Regarding numerical experiments we consider: i) a parameter identification problem in elliptic PDEs, ii) a parameter identification problem in machine learning; the computational efficiency of the proposed method is compared with canonical implementations of the LM method. △ Less

Submitted 11 June, 2024; originally announced June 2024.

MSC Class: 65J20; 47J06

Scaling Rectified Flow Transformers for High-Resolution Image Synthesis

Authors: Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, Robin Rombach

Abstract: Diffusion models create data from noise by inverting the forward paths of data towards noise and have emerged as a powerful generative modeling technique for high-dimensional, perceptual data such as images and videos. Rectified flow is a recent generative model formulation that connects data and noise in a straight line. Despite its better theoretical properties and conceptual simplicity, it is n… ▽ More Diffusion models create data from noise by inverting the forward paths of data towards noise and have emerged as a powerful generative modeling technique for high-dimensional, perceptual data such as images and videos. Rectified flow is a recent generative model formulation that connects data and noise in a straight line. Despite its better theoretical properties and conceptual simplicity, it is not yet decisively established as standard practice. In this work, we improve existing noise sampling techniques for training rectified flow models by biasing them towards perceptually relevant scales. Through a large-scale study, we demonstrate the superior performance of this approach compared to established diffusion formulations for high-resolution text-to-image synthesis. Additionally, we present a novel transformer-based architecture for text-to-image generation that uses separate weights for the two modalities and enables a bidirectional flow of information between image and text tokens, improving text comprehension, typography, and human preference ratings. We demonstrate that this architecture follows predictable scaling trends and correlates lower validation loss to improved text-to-image synthesis as measured by various metrics and human evaluations. Our largest models outperform state-of-the-art models, and we will make our experimental data, code, and model weights publicly available. △ Less

Submitted 5 March, 2024; originally announced March 2024.

Proximal Algorithms for a class of abstract convex functions

Authors: Ewa Bednarczuk, Dirk Lorenz, The Hung Tran

Abstract: In this paper we analyze a class of nonconvex optimization problem from the viewpoint of abstract convexity. Using the respective generalizations of the subgradient we propose an abstract notion proximal operator and derive a number of algorithms, namely an abstract proximal point method, an abstract forward-backward method and an abstract projected subgradient method. Global convergence results f… ▽ More In this paper we analyze a class of nonconvex optimization problem from the viewpoint of abstract convexity. Using the respective generalizations of the subgradient we propose an abstract notion proximal operator and derive a number of algorithms, namely an abstract proximal point method, an abstract forward-backward method and an abstract projected subgradient method. Global convergence results for all algorithms are discussed and numerical examples are given △ Less

Submitted 28 February, 2024; v1 submitted 26 February, 2024; originally announced February 2024.

MSC Class: 49J45; 65K05; 74P10; 90C26;

A Practical Near Optimal Deployment of Service Function Chains in Edge-to-Cloud Networks

Authors: Rasoul Behravesh, David Breitgand, Dean H. Lorenz, Danny Raz

Abstract: Mobile edge computing offers a myriad of opportunities to innovate and introduce novel applications, thereby enhancing user experiences considerably. A critical issue extensively investigated in this domain is efficient deployment of Service Function Chains (SFCs) across the physical network, spanning from the edge to the cloud. This problem is known to be NP-hard. As a result of its practical imp… ▽ More Mobile edge computing offers a myriad of opportunities to innovate and introduce novel applications, thereby enhancing user experiences considerably. A critical issue extensively investigated in this domain is efficient deployment of Service Function Chains (SFCs) across the physical network, spanning from the edge to the cloud. This problem is known to be NP-hard. As a result of its practical importance, there is significant interest in the development of high-quality sub-optimal solutions. In this paper, we consider this problem and propose a novel near-optimal heuristic that is extremely efficient and scalable. We compare our solution to the state-of-the-art heuristic and to the theoretical optimum. In our large-scale evaluations, we use realistic topologies which were previously reported in the literature. We demonstrate that the execution time offered by our solution grows slowly as the number of Virtual Network Function (VNF) forwarding graph embedding requests grows, and it handles one million requests in slightly more than 20 seconds for 100 nodes and 150 edges physical topology. △ Less

Submitted 15 January, 2024; originally announced January 2024.

Comments: Accepted to IEEE INFOCOM 2024

Adversarial Diffusion Distillation

Authors: Axel Sauer, Dominik Lorenz, Andreas Blattmann, Robin Rombach

Abstract: We introduce Adversarial Diffusion Distillation (ADD), a novel training approach that efficiently samples large-scale foundational image diffusion models in just 1-4 steps while maintaining high image quality. We use score distillation to leverage large-scale off-the-shelf image diffusion models as a teacher signal in combination with an adversarial loss to ensure high image fidelity even in the l… ▽ More We introduce Adversarial Diffusion Distillation (ADD), a novel training approach that efficiently samples large-scale foundational image diffusion models in just 1-4 steps while maintaining high image quality. We use score distillation to leverage large-scale off-the-shelf image diffusion models as a teacher signal in combination with an adversarial loss to ensure high image fidelity even in the low-step regime of one or two sampling steps. Our analyses show that our model clearly outperforms existing few-step methods (GANs, Latent Consistency Models) in a single step and reaches the performance of state-of-the-art diffusion models (SDXL) in only four steps. ADD is the first method to unlock single-step, real-time image synthesis with foundation models. Code and weights available under https://github.com/Stability-AI/generative-models and https://huggingface.co/stabilityai/ . △ Less

Submitted 28 November, 2023; originally announced November 2023.

Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets

Authors: Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, Varun Jampani, Robin Rombach

Abstract: We present Stable Video Diffusion - a latent video diffusion model for high-resolution, state-of-the-art text-to-video and image-to-video generation. Recently, latent diffusion models trained for 2D image synthesis have been turned into generative video models by inserting temporal layers and finetuning them on small, high-quality video datasets. However, training methods in the literature vary wi… ▽ More We present Stable Video Diffusion - a latent video diffusion model for high-resolution, state-of-the-art text-to-video and image-to-video generation. Recently, latent diffusion models trained for 2D image synthesis have been turned into generative video models by inserting temporal layers and finetuning them on small, high-quality video datasets. However, training methods in the literature vary widely, and the field has yet to agree on a unified strategy for curating video data. In this paper, we identify and evaluate three different stages for successful training of video LDMs: text-to-image pretraining, video pretraining, and high-quality video finetuning. Furthermore, we demonstrate the necessity of a well-curated pretraining dataset for generating high-quality videos and present a systematic curation process to train a strong base model, including captioning and filtering strategies. We then explore the impact of finetuning our base model on high-quality data and train a text-to-video model that is competitive with closed-source video generation. We also show that our base model provides a powerful motion representation for downstream tasks such as image-to-video generation and adaptability to camera motion-specific LoRA modules. Finally, we demonstrate that our model provides a strong multi-view 3D-prior and can serve as a base to finetune a multi-view diffusion model that jointly generates multiple views of objects in a feedforward fashion, outperforming image-based methods at a fraction of their compute budget. We release code and model weights at https://github.com/Stability-AI/generative-models . △ Less

Submitted 25 November, 2023; originally announced November 2023.

Acceleration and restart for the randomized Bregman-Kaczmarz method

Authors: Lionel Tondji, Ion Necoara, Dirk A. Lorenz

Abstract: Optimizing strongly convex functions subject to linear constraints is a fundamental problem with numerous applications. In this work, we propose a block (accelerated) randomized Bregman-Kaczmarz method that only uses a block of constraints in each iteration to tackle this problem. We consider a dual formulation of this problem in order to deal in an efficient way with the linear constraints. Using… ▽ More Optimizing strongly convex functions subject to linear constraints is a fundamental problem with numerous applications. In this work, we propose a block (accelerated) randomized Bregman-Kaczmarz method that only uses a block of constraints in each iteration to tackle this problem. We consider a dual formulation of this problem in order to deal in an efficient way with the linear constraints. Using convex tools, we show that the corresponding dual function satisfies the Polyak-Lojasiewicz (PL) property, provided that the primal objective function is strongly convex and verifies additionally some other mild assumptions. However, adapting the existing theory on coordinate descent methods to our dual formulation can only give us sublinear convergence results in the dual space. In order to obtain convergence results in some criterion corresponding to the primal (original) problem, we transfer our algorithm to the primal space, which combined with the PL property allows us to get linear convergence rates. More specifically, we provide a theoretical analysis of the convergence of our proposed method under different assumptions on the objective and demonstrate in the numerical experiments its superior efficiency and speed up compared to existing methods for the same problem. △ Less

Submitted 3 April, 2024; v1 submitted 26 October, 2023; originally announced October 2023.

MSC Class: 65F10; 68W20; 90C25

Adaptive Bregman-Kaczmarz: An Approach to Solve Linear Inverse Problems with Independent Noise Exactly

Authors: Lionel Tondji, Idriss Tondji, Dirk A. Lorenz

Abstract: We consider the block Bregman-Kaczmarz method for finite dimensional linear inverse problems. The block Bregman-Kaczmarz method uses blocks of the linear system and performs iterative steps with these blocks only. We assume a noise model that we call independent noise, i.e. each time the method performs a step for some block, one obtains a noisy sample of the respective part of the right-hand side… ▽ More We consider the block Bregman-Kaczmarz method for finite dimensional linear inverse problems. The block Bregman-Kaczmarz method uses blocks of the linear system and performs iterative steps with these blocks only. We assume a noise model that we call independent noise, i.e. each time the method performs a step for some block, one obtains a noisy sample of the respective part of the right-hand side which is contaminated with new noise that is independent of all previous steps of the method. One can view these noise models as making a fresh noisy measurement of the respective block each time it is used. In this framework, we are able to show that a well-chosen adaptive stepsize of the block Bergman-Kaczmarz method is able to converge to the exact solution of the linear inverse problem. The plain form of this adaptive stepsize relies on unknown quantities (like the Bregman distance to the solution), but we show a way how these quantities can be estimated purely from given data. We illustrate the finding in numerical experiments and confirm that these heuristic estimates lead to effective stepsizes. △ Less

Submitted 9 May, 2024; v1 submitted 12 September, 2023; originally announced September 2023.

MSC Class: 65F10; 15A29; 65F20

Minimal error momentum Bregman-Kaczmarz

Authors: Dirk A. Lorenz, Maximilian Winkler

Abstract: The Bregman-Kaczmarz method is an iterative method which can solve strongly convex problems with linear constraints and uses only one or a selected number of rows of the system matrix in each iteration, thereby making it amenable for large-scale systems. To speed up convergence, we investigate acceleration by heavy ball momentum in the so-called dual update. Heavy ball acceleration of the Kaczmarz… ▽ More The Bregman-Kaczmarz method is an iterative method which can solve strongly convex problems with linear constraints and uses only one or a selected number of rows of the system matrix in each iteration, thereby making it amenable for large-scale systems. To speed up convergence, we investigate acceleration by heavy ball momentum in the so-called dual update. Heavy ball acceleration of the Kaczmarz method with constant parameters has turned out to be difficult to analyze, in particular no accelerated convergence for the L2-error of the iterates has been proven to the best of our knowledge. Here we propose a way to adaptively choose the momentum parameter by a minimal-error principle similar to a recently proposed method for the standard randomized Kaczmarz method. The momentum parameter can be chosen to exactly minimize the error in the next iterate or to minimize a relaxed version of the minimal error principle. The former choice leads to a theoretically optimal step while the latter is cheaper to compute. We prove improved convergence results compared to the non-accelerated method. Numerical experiments show that the proposed methods can accelerate convergence in practice, also for matrices which arise from applications such as computational tomography. △ Less

Submitted 28 July, 2023; originally announced July 2023.

MSC Class: 65B05; 65F10; 90C06; 90C25 ACM Class: G.1.3; G.1.6; G.3

On the Interplay of Subset Selection and Informed Graph Neural Networks

Authors: Niklas Breustedt, Paolo Climaco, Jochen Garcke, Jan Hamaekers, Gitta Kutyniok, Dirk A. Lorenz, Rick Oerder, Chirag Varun Shukla

Abstract: Machine learning techniques paired with the availability of massive datasets dramatically enhance our ability to explore the chemical compound space by providing fast and accurate predictions of molecular properties. However, learning on large datasets is strongly limited by the availability of computational resources and can be infeasible in some scenarios. Moreover, the instances in the datasets… ▽ More Machine learning techniques paired with the availability of massive datasets dramatically enhance our ability to explore the chemical compound space by providing fast and accurate predictions of molecular properties. However, learning on large datasets is strongly limited by the availability of computational resources and can be infeasible in some scenarios. Moreover, the instances in the datasets may not yet be labelled and generating the labels can be costly, as in the case of quantum chemistry computations. Thus, there is a need to select small training subsets from large pools of unlabelled data points and to develop reliable ML methods that can effectively learn from small training sets. This work focuses on predicting the molecules atomization energy in the QM9 dataset. We investigate the advantages of employing domain knowledge-based data sampling methods for an efficient training set selection combined with informed ML techniques. In particular, we show how maximizing molecular diversity in the training set selection process increases the robustness of linear and nonlinear regression techniques such as kernel methods and graph neural networks. We also check the reliability of the predictions made by the graph neural network with a model-agnostic explainer based on the rate distortion explanation framework. △ Less

Submitted 15 June, 2023; originally announced June 2023.

Linearly convergent adjoint free solution of least squares problems by random descent

Authors: Dirk A. Lorenz, Felix Schneppe, Lionel Tondji

Abstract: We consider the problem of solving linear least squares problems in a framework where only evaluations of the linear map are possible. We derive randomized methods that do not need any other matrix operations than forward evaluations, especially no evaluation of the adjoint map is needed. Our method is motivated by the simple observation that one can get an unbiased estimate of the application of… ▽ More We consider the problem of solving linear least squares problems in a framework where only evaluations of the linear map are possible. We derive randomized methods that do not need any other matrix operations than forward evaluations, especially no evaluation of the adjoint map is needed. Our method is motivated by the simple observation that one can get an unbiased estimate of the application of the adjoint. We show convergence of the method and then derive a more efficient method that uses an exact linesearch. This method, called random descent, resembles known methods in other context and has the randomized coordinate descent method as special case. We provide convergence analysis of the random descent method emphasizing the dependence on the underlying distribution of the random vectors. Furthermore we investigate the applicability of the method in the context of ill-posed inverse problems and show that the method can have beneficial properties when the unknown solution is rough. We illustrate the theoretical findings in numerical examples. One particular result is that the random descent method actually outperforms established transposed-free methods (TFQMR and CGS) in examples. △ Less

Submitted 14 September, 2023; v1 submitted 2 June, 2023; originally announced June 2023.

MSC Class: 65F10; 68W20; 15A06

A Bregman-Kaczmarz method for nonlinear systems of equations

Authors: Robert Gower, Dirk A. Lorenz, Maximilian Winkler

Abstract: We propose a new randomized method for solving systems of nonlinear equations, which can find sparse solutions or solutions under certain simple constraints. The scheme only takes gradients of component functions and uses Bregman projections onto the solution space of a Newton equation. In the special case of euclidean projections, the method is known as nonlinear Kaczmarz method. Furthermore, if… ▽ More We propose a new randomized method for solving systems of nonlinear equations, which can find sparse solutions or solutions under certain simple constraints. The scheme only takes gradients of component functions and uses Bregman projections onto the solution space of a Newton equation. In the special case of euclidean projections, the method is known as nonlinear Kaczmarz method. Furthermore, if the component functions are nonnegative, we are in the setting of optimization under the interpolation assumption and the method reduces to SGD with the recently proposed stochastic Polyak step size. For general Bregman projections, our method is a stochastic mirror descent with a novel adaptive step size. We prove that in the convex setting each iteration of our method results in a smaller Bregman distance to exact solutions as compared to the standard Polyak step. Our generalization to Bregman projections comes with the price that a convex one-dimensional optimization problem needs to be solved in each iteration. This can typically be done with globalized Newton iterations. Convergence is proved in two classical settings of nonlinearity: for convex nonnegative functions and locally for functions which fulfill the tangential cone condition. Finally, we show examples in which the proposed method outperforms similar methods with the same memory requirements. △ Less

Submitted 23 February, 2024; v1 submitted 15 March, 2023; originally announced March 2023.

MSC Class: 49M15; 90C53; 65Y20

The Degenerate Variable Metric Proximal Point Algorithm and Adaptive Stepsizes for Primal-Dual Douglas-Rachford

Authors: Dirk A. Lorenz, Jannis Marquardt, Emanuele Naldi

Abstract: In this paper the degenerate preconditioned proximal point algorithm will be combined with the idea of varying preconditioners leading to the degenerate variable metric proximal point algorithm. The weak convergence of the resulting iteration will be proven. From the perspective of the degenerate variable metric proximal point algorithm, a version of the primal-dual Douglas-Rachford method with va… ▽ More In this paper the degenerate preconditioned proximal point algorithm will be combined with the idea of varying preconditioners leading to the degenerate variable metric proximal point algorithm. The weak convergence of the resulting iteration will be proven. From the perspective of the degenerate variable metric proximal point algorithm, a version of the primal-dual Douglas-Rachford method with varying preconditioners will be derived and a proof of its weak convergence which is based on the previous results for the proximal point algorithm, is provided, too. After that, we derive a heuristic on how to choose those varying preconditioners in order to increase the convergence speed of the method. △ Less

Submitted 25 February, 2023; originally announced February 2023.

MSC Class: 47H05; 65K05; 90C25

Detection and characterization of wind-blown charged sand grains on Titan with the DraGMet/EFIELD experiment on Dragonfly

Authors: Audrey Chatain, Alice Le Gall, Jean-Jacques Berthelier, Ralph D. Lorenz, Rafik Hassen-Khodja, Jean-Pierre Lebreton, Tom Joly-Jehenne, Grégoire Déprez

Abstract: The EFIELD instrument is part of the geophysics and meteorology sensor package DraGMet on the Dragonfly mission, which will explore the surface of Titan in the mid-2030s. EFIELD consists of two electrodes designed to passively record the AC electric field at each landing site. The exploration zone of Dragonfly will mostly consist of dune fields, covered with sand grains. Little is known on the p… ▽ More The EFIELD instrument is part of the geophysics and meteorology sensor package DraGMet on the Dragonfly mission, which will explore the surface of Titan in the mid-2030s. EFIELD consists of two electrodes designed to passively record the AC electric field at each landing site. The exploration zone of Dragonfly will mostly consist of dune fields, covered with sand grains. Little is known on the properties of these grains, although Cassini-Huygens observations suggest they are mostly made of organic material produced by Titan's atmospheric photochemistry and evolved at the surface. Little is known also about dune formation and in general about the transport of sediments by winds. The latter much depends on inter-particle forces and therefore on how grains are charged by friction. We demonstrate here that the EFIELD experiment can bring new insights on these questions. We have developed a hydrodynamic-electrostatic model to simulate the trajectory of a wind-blown charged sand grain in the vicinity of an idealized EFIELD probe and to predict how such a grain flying close to the probe would affect its potential. We show that, in some conditions, the resulting perturbation will be strong enough to be detected by the EFIELD probe. More specifically, we find that the detection of typical charged wind-blown grains (200 microns) on Titan requires an instrument standard deviation noise inferior to 1mV, though occasional larger grains flying close to one electrode could be detected with a higher noise level. Furthermore, we propose a method to retrieve information on the charge and velocity of wind-blown charged grains detected by the EFIELD experiment. This method well applies for cases where the particle trajectory can be regarded as quasi-linear. We validate our inversion approach on both synthetic and experimental data obtained with a laboratory prototype of the EFIELD experiment. △ Less

Submitted 15 May, 2023; v1 submitted 2 November, 2022; originally announced November 2022.

Comments: Accepted for publication in Icarus (2 Nov. 2022). Supporting Information, datasets and codes available on Zenodo (DOI: 10.5281/zenodo.7274939)

Journal ref: Icarus 391 (2023) 115345

Learning Variational Models with Unrolling and Bilevel Optimization

Authors: Christoph Brauer, Niklas Breustedt, Timo de Wolff, Dirk A. Lorenz

Abstract: In this paper we consider the problem of learning variational models in the context of supervised learning via risk minimization. Our goal is to provide a deeper understanding of the two approaches of learning of variational models via bilevel optimization and via algorithm unrolling. The former considers the variational model as a lower level optimization problem below the risk minimization probl… ▽ More In this paper we consider the problem of learning variational models in the context of supervised learning via risk minimization. Our goal is to provide a deeper understanding of the two approaches of learning of variational models via bilevel optimization and via algorithm unrolling. The former considers the variational model as a lower level optimization problem below the risk minimization problem, while the latter replaces the lower level optimization problem by an algorithm that solves said problem approximately. Both approaches are used in practice, but unrolling is much simpler from a computational point of view. To analyze and compare the two approaches, we consider a simple toy model, and compute all risks and the respective estimators explicitly. We show that unrolling can be better than the bilevel optimization approach, but also that the performance of unrolling can depend significantly on further parameters, sometimes in unexpected ways: While the stepsize of the unrolled algorithm matters a lot (and learning the stepsize gives a significant improvement), the number of unrolled iterations plays a minor role. △ Less

Submitted 6 September, 2023; v1 submitted 26 September, 2022; originally announced September 2022.

MSC Class: 68T07; 68T05

Damage Identification in Fiber Metal Laminates using Bayesian Analysis with Model Order Reduction

Authors: Nanda Kishore Bellam Muralidhar, Carmen Gräßle, Natalie Rauter, Andrey Mikhaylenko, Rolf Lammering, Dirk A. Lorenz

Abstract: Fiber metal laminates (FML) are composite structures consisting of metals and fiber reinforced plastics (FRP) which have experienced an increasing interest as the choice of materials in aerospace and automobile industries. Due to a sophisticated built up of the material, not only the design and production of such structures is challenging but also its damage detection. This research work focuses o… ▽ More Fiber metal laminates (FML) are composite structures consisting of metals and fiber reinforced plastics (FRP) which have experienced an increasing interest as the choice of materials in aerospace and automobile industries. Due to a sophisticated built up of the material, not only the design and production of such structures is challenging but also its damage detection. This research work focuses on damage identification in FML with guided ultrasonic waves (GUW) through an inverse approach based on the Bayesian paradigm. As the Bayesian inference approach involves multiple queries of the underlying system, a parameterized reduced-order model (ROM) is used to closely approximate the solution with considerably less computational cost. The signals measured by the embedded sensors and the ROM forecasts are employed for the localization and characterization of damage in FML. In this paper, a Markov Chain Monte-Carlo (MCMC) based Metropolis-Hastings (MH) algorithm and an Ensemble Kalman filtering (EnKF) technique are deployed to identify the damage. Numerical tests illustrate the approaches and the results are compared in regard to accuracy and efficiency. It is found that both methods are successful in multivariate characterization of the damage with a high accuracy and were also able to quantify their associated uncertainties. The EnKF distinguishes itself with the MCMC-MH algorithm in the matter of computational efficiency. In this application of identifying the damage, the EnKF is approximately thrice faster than the MCMC-MH. △ Less

Submitted 21 April, 2023; v1 submitted 9 June, 2022; originally announced June 2022.

Faster Randomized Block Sparse Kaczmarz by Averaging

Authors: Lionel Tondji, Dirk A Lorenz

Abstract: The standard randomized sparse Kaczmarz (RSK) method is an algorithm to compute sparse solutions of linear systems of equations and uses sequential updates, and thus, does not take advantage of parallel computations. In this work, we introduce a parallel (mini batch) version of RSK based on averaging several Kaczmarz steps. Naturally, this method allows for parallelization and we show that it can… ▽ More The standard randomized sparse Kaczmarz (RSK) method is an algorithm to compute sparse solutions of linear systems of equations and uses sequential updates, and thus, does not take advantage of parallel computations. In this work, we introduce a parallel (mini batch) version of RSK based on averaging several Kaczmarz steps. Naturally, this method allows for parallelization and we show that it can also leverage large over-relaxation. We prove linear expected convergence and show that, given that parallel computations can be exploited, the method provably provides faster convergence than the standard method. This method can also be viewed as a variant of the linearized Bregman algorithm, a randomized dual block coordinate descent update, a stochastic mirror descent update, or a relaxed version of RSK and we recover the standard RSK method when the batch size is equal to one. We also provide estimates for inconsistent systems and show that the iterates convergence to an error in the order of the noise level. Finally, numerical examples illustrate the benefits of the new algorithm. △ Less

Submitted 17 October, 2022; v1 submitted 21 March, 2022; originally announced March 2022.

MSC Class: 65F10; 68W20; 68W10; 90C25

Extended Randomized Kaczmarz Method for Sparse Least Squares and Impulsive Noise Problems

Authors: Frank Schöpfer, Dirk A Lorenz, Lionel Tondji, Maximilian Winkler

Abstract: The Extended Randomized Kaczmarz method is a well known iterative scheme which can find the Moore-Penrose inverse solution of a possibly inconsistent linear system and requires only one additional column of the system matrix in each iteration in comparison with the standard randomized Kaczmarz method. Also, the Sparse Randomized Kaczmarz method has been shown to converge linearly to a sparse solut… ▽ More The Extended Randomized Kaczmarz method is a well known iterative scheme which can find the Moore-Penrose inverse solution of a possibly inconsistent linear system and requires only one additional column of the system matrix in each iteration in comparison with the standard randomized Kaczmarz method. Also, the Sparse Randomized Kaczmarz method has been shown to converge linearly to a sparse solution of a consistent linear system. Here, we combine both ideas and propose an Extended Sparse Randomized Kaczmarz method. We show linear expected convergence to a sparse least squares solution in the sense that an extended variant of the regularized basis pursuit problem is solved. Moreover, we generalize the additional step in the method and prove convergence to a more abstract optimization problem. We demonstrate numerically that our method can find sparse least squares solutions of real and complex systems if the noise is concentrated in the complement of the range of the system matrix and that our generalization can handle impulsive noise. △ Less

Submitted 20 July, 2022; v1 submitted 21 January, 2022; originally announced January 2022.

MSC Class: 65F10; 68W20; 90C25

$L^α$-Regularization of the Beckmann Problem

Authors: Dirk Lorenz, Hinrich Mahler, Christian Meyer

Abstract: We investigate the problem of optimal transport in the so-called Beckmann form, i.e. given two Radon measures on a compact set, we seek an optimal flow field which is a vector valued Radon measure on the same set that describes a flow between these two measures and minimizes a certain linear cost function. We consider $L^α$ regularization of the problem, which guarantees uniqueness and forces th… ▽ More We investigate the problem of optimal transport in the so-called Beckmann form, i.e. given two Radon measures on a compact set, we seek an optimal flow field which is a vector valued Radon measure on the same set that describes a flow between these two measures and minimizes a certain linear cost function. We consider $L^α$ regularization of the problem, which guarantees uniqueness and forces the solution to be an integrable function rather than a Radon measure. This regularization naturally gives rise to a semi-smooth Newton scheme that can be used to solve the problem numerically. Besides motivating and develo** the numerical scheme, we also include approximation results for vanishing regularization in the continuous setting. △ Less

Submitted 18 January, 2022; originally announced January 2022.

Chambolle-Pock's Primal-Dual Method with Mismatched Adjoint

Authors: Dirk A. Lorenz, Felix Schneppe

Abstract: The primal-dual method of Chambolle and Pock is a widely used algorithm to solve various optimization problems written as convex-concave saddle point problems. Each update step involves the application of both the forward linear operator and its adjoint. However, in practical applications like computerized tomography, it is often computationally favourable to replace the adjoint operator by a comp… ▽ More The primal-dual method of Chambolle and Pock is a widely used algorithm to solve various optimization problems written as convex-concave saddle point problems. Each update step involves the application of both the forward linear operator and its adjoint. However, in practical applications like computerized tomography, it is often computationally favourable to replace the adjoint operator by a computationally more efficient approximation. This leads to an adjoint mismatch in the algorithm. In this paper, we analyze the convergence of Chambolle-Pock's primal-dual method under the presence of a mismatched adjoint in the strongly convex setting. We present an upper bound on the error of the primal solution and derive stepsizes and mild conditions under which convergence to a fixed point is still guaranteed. Furthermore we show linear convergence similar to the result of Chambolle-Pock's primal-dual method without the adjoint mismatch. Moreover, we illustrate our results both for an academic and a real-world inspired application. △ Less

Submitted 12 October, 2022; v1 submitted 13 January, 2022; originally announced January 2022.

MSC Class: 49M29; 90C25; 65K10

High-Resolution Image Synthesis with Latent Diffusion Models

Authors: Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, Björn Ommer

Abstract: By decomposing the image formation process into a sequential application of denoising autoencoders, diffusion models (DMs) achieve state-of-the-art synthesis results on image data and beyond. Additionally, their formulation allows for a guiding mechanism to control the image generation process without retraining. However, since these models typically operate directly in pixel space, optimization o… ▽ More By decomposing the image formation process into a sequential application of denoising autoencoders, diffusion models (DMs) achieve state-of-the-art synthesis results on image data and beyond. Additionally, their formulation allows for a guiding mechanism to control the image generation process without retraining. However, since these models typically operate directly in pixel space, optimization of powerful DMs often consumes hundreds of GPU days and inference is expensive due to sequential evaluations. To enable DM training on limited computational resources while retaining their quality and flexibility, we apply them in the latent space of powerful pretrained autoencoders. In contrast to previous work, training diffusion models on such a representation allows for the first time to reach a near-optimal point between complexity reduction and detail preservation, greatly boosting visual fidelity. By introducing cross-attention layers into the model architecture, we turn diffusion models into powerful and flexible generators for general conditioning inputs such as text or bounding boxes and high-resolution synthesis becomes possible in a convolutional manner. Our latent diffusion models (LDMs) achieve a new state of the art for image inpainting and highly competitive performance on various tasks, including unconditional image generation, semantic scene synthesis, and super-resolution, while significantly reducing computational requirements compared to pixel-based DMs. Code is available at https://github.com/CompVis/latent-diffusion . △ Less

Submitted 13 April, 2022; v1 submitted 20 December, 2021; originally announced December 2021.

Comments: CVPR 2022

The InSight HP$^3$ mole on Mars: Lessons learned from attempts to penetrate to depth in the Martian soil

Authors: T. Spohn, T. L. Hudson, L. Witte, T. Wippermann, L. Wisniewski, B. Kediziora, C. Vrettos, R. D. Lorenz, M. Golombek, R. Lichtenfeld, M. Grott, J. Knollenberg, C. Krause, C. Fantinati, S. Nagihara, J. Grygorczuk

Abstract: The NASA InSight mission payload includes the Heat Flow and Physical Properties Package HP$^3$ to measure the surface heat flow. The package was designed to use a small penetrator -- nicknamed the mole -- to implement a string of temperature sensors in the soil to a depth of 5m. The mole itself is equipped with sensors to measure a thermal conductivity as it proceeds to depth. The heat flow would… ▽ More The NASA InSight mission payload includes the Heat Flow and Physical Properties Package HP$^3$ to measure the surface heat flow. The package was designed to use a small penetrator -- nicknamed the mole -- to implement a string of temperature sensors in the soil to a depth of 5m. The mole itself is equipped with sensors to measure a thermal conductivity as it proceeds to depth. The heat flow would be calculated from the product of the temperature gradient and the thermal conductivity. To avoid the perturbation caused by annual surface temperature variations, the measurements would be taken at a depth between 3 m and 5 m. The mole was designed to penetrate cohesionless soil similar to Quartz sand which was expected to provide a good analogue material for Martian sand. The sand would provide friction to the buried mole hull to balance the remaining recoil of the mole hammer mechanism that drives the mole forward. Unfortunately, the mole did not penetrate more than a mole length of 40 cm. The failure to penetrate deeper was largely due to a few tens of centimeter thick cohesive duricrust that failed to provide the required friction. Although a suppressor mass and spring in the hammer mechanism absorbed much of the recoil, the available mass did not allow a system that would have eliminated the recoil. The mole penetrated to 40 cm depth benefiting from friction provided by springs in the support structure from which it was deployed. It was found in addition that the Martian soil provided unexpected levels of penetration resistance that would have motivated to designing a more powerful mole. It is concluded that more mass would have allowed to design a more robust system with little or no recoil, more energy of the mole hammer mechanism and a more massive support structure. △ Less

Submitted 6 December, 2021; originally announced December 2021.

Comments: 34 pages, 15 figures, submitted to Adnaves in Space Research

Report number: InSight Contribution Number 234

Journal ref: Advances in Space Research, 69 (2022) 3140-3163

Nonconvex flexible sparsity regularization: theory and monotone numerical schemes

Authors: Daria Ghilli, Dirk A. Lorenz, Elena Resmerita

Abstract: Flexible sparsity regularization means stably approximating sparse solutions of operator equations by using coefficient-dependent penalizations. We propose and analyse a general nonconvex approach in this respect, from both theoretical and numerical perspectives. Namely, we show convergence of the regularization method and establish convergence properties of a couple of majorization approaches for… ▽ More Flexible sparsity regularization means stably approximating sparse solutions of operator equations by using coefficient-dependent penalizations. We propose and analyse a general nonconvex approach in this respect, from both theoretical and numerical perspectives. Namely, we show convergence of the regularization method and establish convergence properties of a couple of majorization approaches for the associated nonconvex problems. We also test a monotone algorithm for an academic example where the operator is an $M$ matrix, and on a time-dependent optimal control problem, pointing out the advantages of employing variable penalties over a fixed penalty. △ Less

Submitted 11 November, 2021; originally announced November 2021.

MSC Class: 49XX; 65KXX; 90CXX

Gravitational atmospheric tides as a probe of Titan's interior: Application to Dragonfly

Authors: Benjamin Charnay, Gabriel Tobie, Sébastien Lebonnois, Ralph D. Lorenz

Abstract: Context: Saturn's massive gravity is expected to causes a tide in Titan's atmosphere, producing a surface pressure variation through the orbit of Titan and tidal winds in the troposphere. The future Dragonfly mission could analyse this exotic meteorological phenomenon. Aims: We analyse the effect of Saturn's tides on Titan's atmosphere and interior to determine how pressure measurements by Drago… ▽ More Context: Saturn's massive gravity is expected to causes a tide in Titan's atmosphere, producing a surface pressure variation through the orbit of Titan and tidal winds in the troposphere. The future Dragonfly mission could analyse this exotic meteorological phenomenon. Aims: We analyse the effect of Saturn's tides on Titan's atmosphere and interior to determine how pressure measurements by Dragonfly could constrain Titan's interior. Methods: We model atmospheric tides with analytical calculations and with a 3D Global Climate Model (the IPSL-Titan GCM), including the tidal response of the interior. Results: We predict that the Love numbers of Titan's interior should verify 1 + Re(k2 - h2) ~ 0.02-0.1 and Im(k2 - h2) < 0.04. The deformation of Titan's interior should therefore strongly weaken gravitational atmospheric tides, yielding a residual surface pressure amplitude of only ~ 5 Pa, with a phase shift of 5-20 hours. Tidal winds are very weak, of the order of 3*10^-4 m/s in the lower troposphere. Finally, constraints from Dragonfly data may permit the real and the imaginary parts of k2 - h2 to be estimated with a precision of ~0.01-0.03. Conclusions: Measurements of pressure variations by Dragonfly over the whole mission could give valuable constraints on the thickness of Titan's ice shell, and via geophysical models, its heat flux and the density of Titan's internal ocean. △ Less

Submitted 3 November, 2021; originally announced November 2021.

Comments: Accepted for publication in A&A. 10 pages, 8 figures, 1 table

Journal ref: A&A 658, A108 (2022)

Science goals and new mission concepts for future exploration of Titan's atmosphere geology and habitability: Titan POlar Scout/orbitEr and In situ lake lander and DrONe explorer (POSEIDON)

Authors: Sébastien Rodriguez, Sandrine Vinatier, Daniel Cordier, Gabriel Tobie, Richard K. Achterberg, Carrie M. Anderson, Sarah V. Badman, Jason W. Barnes, Erika L. Barth, Bruno Bézard, Nathalie Carrasco, Benjamin Charnay, Roger N. Clark, Patrice Coll, Thomas Cornet, Athena Coustenis, Isabelle Couturier-Tamburelli, Michel Dobrijevic, F. Michael Flasar, Remco de Kok, Caroline Freissinet, Marina Galand, Thomas Gautier, Wolf D. Geppert, Caitlin A. Griffith , et al. (39 additional authors not shown)

Abstract: In response to ESA Voyage 2050 announcement of opportunity, we propose an ambitious L-class mission to explore one of the most exciting bodies in the Solar System, Saturn largest moon Titan. Titan, a "world with two oceans", is an organic-rich body with interior-surface-atmosphere interactions that are comparable in complexity to the Earth. Titan is also one of the few places in the Solar System w… ▽ More In response to ESA Voyage 2050 announcement of opportunity, we propose an ambitious L-class mission to explore one of the most exciting bodies in the Solar System, Saturn largest moon Titan. Titan, a "world with two oceans", is an organic-rich body with interior-surface-atmosphere interactions that are comparable in complexity to the Earth. Titan is also one of the few places in the Solar System with habitability potential. Titan remarkable nature was only partly revealed by the Cassini-Huygens mission and still holds mysteries requiring a complete exploration using a variety of vehicles and instruments. The proposed mission concept POSEIDON (Titan POlar Scout/orbitEr and In situ lake lander DrONe explorer) would perform joint orbital and in situ investigations of Titan. It is designed to build on and exceed the scope and scientific/technological accomplishments of Cassini-Huygens, exploring Titan in ways that were not previously possible, in particular through full close-up and in situ coverage over long periods of time. In the proposed mission architecture, POSEIDON consists of two major elements: a spacecraft with a large set of instruments that would orbit Titan, preferably in a low-eccentricity polar orbit, and a suite of in situ investigation components, i.e. a lake lander, a "heavy" drone (possibly amphibious) and/or a fleet of mini-drones, dedicated to the exploration of the polar regions. The ideal arrival time at Titan would be slightly before the next northern Spring equinox (2039), as equinoxes are the most active periods to monitor still largely unknown atmospheric and surface seasonal changes. The exploration of Titan northern latitudes with an orbiter and in situ element(s) would be highly complementary with the upcoming NASA New Frontiers Dragonfly mission that will provide in situ exploration of Titan equatorial regions in the mid-2030s. △ Less

Submitted 20 October, 2021; originally announced October 2021.

Comments: arXiv admin note: substantial text overlap with arXiv:1908.01374

Degenerate Preconditioned Proximal Point algorithms

Authors: Kristian Bredies, Enis Chenchene, Dirk A. Lorenz, Emanuele Naldi

Abstract: In this paper we describe a systematic procedure to analyze the convergence of degenerate preconditioned proximal point algorithms. We establish weak convergence results under mild assumptions that can be easily employed in the context of splitting methods for monotone inclusion and convex minimization problems. Moreover, we show that the degeneracy of the preconditioner allows for a reduction of… ▽ More In this paper we describe a systematic procedure to analyze the convergence of degenerate preconditioned proximal point algorithms. We establish weak convergence results under mild assumptions that can be easily employed in the context of splitting methods for monotone inclusion and convex minimization problems. Moreover, we show that the degeneracy of the preconditioner allows for a reduction of the variables involved in the iteration updates. We show the strength of the proposed framework in the context of splitting algorithms, providing new simplified proofs of convergence and highlighting the link between existing schemes, such as Chambolle-Pock, Forward Douglas-Rachford and Peaceman-Rachford, that we study from a preconditioned proximal point perspective. The proposed framework allows to devise new flexible schemes and provides new ways to generalize existing splitting schemes to the case of the sum of many terms. As an example, we present a new sequential generalization of Forward Douglas-Rachford along with numerical experiments that demonstrate its interest in the context of nonsmooth convex optimization. △ Less

Submitted 23 September, 2021; originally announced September 2021.

MSC Class: 47H05; 47H09; 47N10; 90C25

Optimal Software Architecture From Initial Requirements: An End-to-End Approach

Authors: Ofir T. Erlich, David H. Lorenz

Abstract: A software architect turns system requirements into a suitable software architecture through an architecture optimization process. However, how should the architect decide which quality improvement to prioritize, e.g., security or reliability? In software product line, should a small improvement in multiple products be preferred over a large improvement in a single product? Existing architecture o… ▽ More A software architect turns system requirements into a suitable software architecture through an architecture optimization process. However, how should the architect decide which quality improvement to prioritize, e.g., security or reliability? In software product line, should a small improvement in multiple products be preferred over a large improvement in a single product? Existing architecture optimization methods handle various steps in the process, but none of them systematically guides the architect in generating an optimal architecture from the initial requirements. In this work we present an end-to-end approach for generating an optimal software architecture for a single software product and an optimal family of architectures for a family of products. We report on a case-study of applying our approach to optimize five industry-grade products in a real-life product line architecture, where 359 possible combinations of ten different quality efforts were prioritized. △ Less

Submitted 31 December, 2020; originally announced December 2020.

Managed Information: A New Abstraction Mechanism for Handling Information in Software-as-a-Service

Authors: David H. Lorenz, Boaz Rosenan

Abstract: Management of information is an important aspect of every application. This includes, for example, protecting user data against breaches (like the one reported in the news about 50 million Facebook profiles being harvested for Cambridge Analytica), complying with data protection laws and regulations (like EU's new General Data Protection Regulation), co** with large databases, and retaining user… ▽ More Management of information is an important aspect of every application. This includes, for example, protecting user data against breaches (like the one reported in the news about 50 million Facebook profiles being harvested for Cambridge Analytica), complying with data protection laws and regulations (like EU's new General Data Protection Regulation), co** with large databases, and retaining user data across software versions. Today, every application needs to cope with such concerns by itself and on its own. In this paper we introduce Managed Information (MI), an abstraction mechanism for managing extra-functional data related concerns, similar to how managed memory today abstracts away many memory related concerns. MI limits the access applications have to user data, which, in return, relieves them from responsibility over it. This is achieved by hosting them on a Managed Information Platform (MIP), and implementing their logic in a language that supports MI. As evidence for the feasibility of MI we describe the design and implementation of such a platform. For demonstration of MI, we describe a simple social network application built with it. The implementation is open source. △ Less

Submitted 30 December, 2020; originally announced December 2020.

Modeling transmission windows in Titan's lower troposphere: Implications for infrared spectrometers aboard future aerial and surface missions

Authors: Paul Corlies, George D. McDonald, Alexander G. Hayes, James J. Wray, Mate Adamkovics, Michael J. Malaska, Morgan L. Cable, Jason D. Hofgartner, Sarah M. Horst, Lucas R. Liuzzo, Jacob J. Buffo, Ralph D. Lorenz, Elizabeth P. Turtle

Abstract: From orbit, the visibility of Titan's surface is limited to a handful of narrow spectral windows in the near-infrared (near-IR), primarily from the absorption of methane gas. This has limited the ability to identify specific compounds on the surface -- to date Titan's bulk surface composition remains unknown. Further, understanding of the surface composition would provide insight into geologic pro… ▽ More From orbit, the visibility of Titan's surface is limited to a handful of narrow spectral windows in the near-infrared (near-IR), primarily from the absorption of methane gas. This has limited the ability to identify specific compounds on the surface -- to date Titan's bulk surface composition remains unknown. Further, understanding of the surface composition would provide insight into geologic processes, photochemical production and evolution, and the biological potential of Titan's surface. One approach to obtain wider spectral coverage with which to study Titan's surface is by decreasing the integrated column of absorbers (primarily methane) and scatterers between the observer and the surface. This is only possible if future missions operate at lower altitudes in Titan's atmosphere. Herein, we use a radiative transfer model to measure in detail the absorption through Titan's atmosphere from different mission altitudes, and consider the impacts this would have for interpreting reflectance measurements of Titan's surface. Over our modeled spectral range of 0.4 - 10 micron, we find that increases in the width of the transmission windows as large as 317% can be obtained for missions performing remote observations at the surface. However, any appreciable widening of the windows requires onboard illumination. Further, we make note of possible surface compounds that are not currently observable from orbit, but could be identified using the wider windows at low altitudes. These range from simple nitriles such as cyanoacetylene, to building blocks of amino acids such as urea. Finally, we discuss the implications that the identifications of these compounds would have for Titan science. △ Less

Submitted 3 December, 2020; originally announced December 2020.

Comments: 52 pages, 6 figures, Icarus 2020

Regularization of Inverse Problems by Filtered Diagonal Frame Decomposition

Authors: Andrea Ebner, Jürgen Frikel, Dirk Lorenz, Johannes Schwab, Markus Haltmeier

Abstract: The characteristic feature of inverse problems is their instability with respect to data perturbations. In order to stabilize the inversion process, regularization methods have to be developed and applied. In this work we introduce and analyze the concept of filtered diagonal frame decomposition which extends the standard filtered singular value decomposition to the frame case. Frames as generaliz… ▽ More The characteristic feature of inverse problems is their instability with respect to data perturbations. In order to stabilize the inversion process, regularization methods have to be developed and applied. In this work we introduce and analyze the concept of filtered diagonal frame decomposition which extends the standard filtered singular value decomposition to the frame case. Frames as generalized singular system allows to better adapt to a given class of potential solutions. In this paper, we show that filtered diagonal frame decomposition yield a convergent regularization method. Moreover, we derive convergence rates under source type conditions and prove order optimality under the assumption that the considered frame is a Riesz-basis. △ Less

Submitted 19 August, 2022; v1 submitted 14 August, 2020; originally announced August 2020.

MSC Class: 65J20; 65J22; 45F05

The Science Case for a Titan Flagship-class Orbiter with Probes

Authors: Conor A. Nixon, James Abshire, Andrew Ashton, Jason W. Barnes, Nathalie Carrasco, Mathieu Choukroun, Athena Coustenis, Louis-Alexandre Couston, Niklas Edberg, Alexander Gagnon, Jason D. Hofgartner, Luciano Iess, Stéphane Le Mouélic, Rosaly Lopes, Juan Lora, Ralph D. Lorenz, Adrienn Luspay-Kuti, Michael Malaska, Kathleen Mandt, Marco Mastrogiuseppe, Erwan Mazarico, Marc Neveu, Taylor Perron, Jani Radebaugh, Sébastien Rodriguez , et al. (14 additional authors not shown)

Abstract: We outline a flagship-class mission concept focused on studying Titan as a global system, with particular emphasis on the polar regions. Investigating Titan from the unique standpoint of a polar orbit would enable comprehensive global maps to uncover the physics and chemistry of the atmosphere, and the topography and geophysical environment of the surface and subsurface. The mission includes two k… ▽ More We outline a flagship-class mission concept focused on studying Titan as a global system, with particular emphasis on the polar regions. Investigating Titan from the unique standpoint of a polar orbit would enable comprehensive global maps to uncover the physics and chemistry of the atmosphere, and the topography and geophysical environment of the surface and subsurface. The mission includes two key elements: (1) an orbiter spacecraft, which also acts as a data relay, and (2) one or more small probes to directly investigate Titan's seas and make the first direct measurements of their liquid composition and physical environment. The orbiter would carry a sophisticated remote sensing payload, including a novel topographic lidar, a long-wavelength surface-penetrating radar, a sub-millimeter sounder for winds and for mesospheric/thermospheric composition, and a camera and near-infrared spectrometer. An instrument suite to analyze particles and fields would include a mass spectrometer to focus on the interactions between Titan's esca** upper atmosphere and the solar wind and Saturnian magnetosphere. The orbiter would enter a stable polar orbit around 1500 to 1800 km, from which vantage point it would make global maps of the atmosphere and surface. One or more probes, released from the orbiter, would investigate Titan's seas in situ, including possible differences in composition between higher and lower latitude seas, as well as the atmosphere during the parachute descent. The number of probes, as well as the instrument complement on the orbiter and probe, remain to be finalized during a mission study that we recommend to NASA as part of the NRC Decadal Survey for Planetary Science now underway, with the goal of an overall mission cost in the "small flagship" category of ~$2 bn. International partnerships, similar to Cassini-Huygens, may also be included for consideration. △ Less

Submitted 13 August, 2020; originally announced August 2020.

Comments: 13 pages, white paper submitted to the NRC Decadal Survey for Planetary Science and Astrobiology

Integrated Methodology to Cognitive Network Slice Management in Virtualized 5G Networks

Authors: Xenofon Vasilakos, Navid Nikaein, Dean H Lorenz, Berkay Koksal, Nasim Ferdosian

Abstract: Fifth Generation (5G) networks are envisioned to be fully autonomous in accordance to the ETSI-defined Zero touch network and Service Management (ZSM) concept. To this end, purpose-specific Machine Learning (ML) models can be used to manage and control physical as well as virtual network resources in a way that is fully compliant to slice Service Level Agreements (SLAs), while also boosting the re… ▽ More Fifth Generation (5G) networks are envisioned to be fully autonomous in accordance to the ETSI-defined Zero touch network and Service Management (ZSM) concept. To this end, purpose-specific Machine Learning (ML) models can be used to manage and control physical as well as virtual network resources in a way that is fully compliant to slice Service Level Agreements (SLAs), while also boosting the revenue of the underlying physical network operator(s). This is because specially designed and trained ML models can be both proactive and very effective against slice management issues that can induce significant SLA penalties or runtime costs. However, reaching that point is very challenging. 5G networks will be highly dynamic and complex, offering a large scale of heterogeneous, sophisticated and resource-demanding 5G services as network slices. This raises a need for a well-defined, generic and step-wise roadmap to designing, building and deploying efficient ML models as collaborative components of what can be defined as Cognitive Network and Slice Management (CNSM) 5G systems. To address this need, we take a use case-driven approach to design and present a novel Integrated Methodology for CNSM in virtualized 5G networks based on a concrete eHealth use case, and elaborate on it to derive a generic approach for 5G slice management use cases. The three fundamental components that comprise our proposed methodology include (i) a 5G Cognitive Workflow model that conditions everything from the design up to the final deployment of ML models; (ii) a Four-stage approach to Cognitive Slice Management with an emphasis on anomaly detection; and (iii) a Proactive Control Scheme for the collaboration of different ML models targeting different slice life-cycle management problems. △ Less

Submitted 10 May, 2020; originally announced May 2020.

Comments: 22 pages, 4 figures, 5 author bios and photos

AIOps for a Cloud Object Storage Service

Authors: Anna Levin, Shelly Garion, Elliot K. Kolodner, Dean H. Lorenz, Katherine Barabash, Mike Kugler, Niall McShane

Abstract: With the growing reliance on the ubiquitous availability of IT systems and services, these systems become more global, scaled, and complex to operate. To maintain business viability, IT service providers must put in place reliable and cost efficient operations support. Artificial Intelligence for IT Operations (AIOps) is a promising technology for alleviating operational complexity of IT systems a… ▽ More With the growing reliance on the ubiquitous availability of IT systems and services, these systems become more global, scaled, and complex to operate. To maintain business viability, IT service providers must put in place reliable and cost efficient operations support. Artificial Intelligence for IT Operations (AIOps) is a promising technology for alleviating operational complexity of IT systems and services. AIOps platforms utilize big data, machine learning and other advanced analytics technologies to enhance IT operations with proactive actionable dynamic insight. In this paper we share our experience applying the AIOps approach to a production cloud object storage service to get actionable insights into system's behavior and health. We describe a real-life production cloud scale service and its operational data, present the AIOps platform we have created, and show how it has helped us resolving operational pain points. △ Less

Submitted 6 May, 2020; originally announced May 2020.

Comments: 5 pages

Journal ref: 2019 IEEE International Congress on Big Data (BigDataCongress)

Orlicz space regularization of continuous optimal transport problems

Authors: Dirk Lorenz, Hinrich Mahler

Abstract: In this work we analyze regularized optimal transport problems in the so-called Kantorovich form, i.e. given two Radon measures on two compact sets, the aim is to find a transport plan, which is another Radon measure on the product of the sets, that has these two measures as marginals and minimizes the sum of a certain linear cost function and a regularization term. We focus on regularization term… ▽ More In this work we analyze regularized optimal transport problems in the so-called Kantorovich form, i.e. given two Radon measures on two compact sets, the aim is to find a transport plan, which is another Radon measure on the product of the sets, that has these two measures as marginals and minimizes the sum of a certain linear cost function and a regularization term. We focus on regularization terms where a Young's function applied to the (density of the) transport plan is integrated against a product measure. This forces the transport plan to belong to a certain Orlicz space. The predual problem is derived and proofs for strong duality and existence of primal solutions of the regularized problem are presented. Existence of (pre-)dual solutions is shown for the special case of $L^p$ regularization for $p\geq 2$. Moreover, two results regarding $Γ$-convergence are stated: The first is concerned with marginals that do not lie in the appropriate Orlicz space and guarantees $Γ$-convergence to the original Kantorovich problem, when smoothing the marginals. The second results gives convergence of a regularized and discretized problem to the unregularized, continuous problem. △ Less

Submitted 8 September, 2021; v1 submitted 24 April, 2020; originally announced April 2020.

Comments: Update paper in response to reviewers' comments

MSC Class: 49Q20; 49J45; 90C25

On-deck seismology: Lessons from InSight for future planetary seismology

Authors: Mark P. Panning, W. Tom Pike, Philippe Lognonné, W. Bruce Banerdt, Naomi Murdoch, Don Banfield, Constantinos Charalambous, Sharon Kedar, Ralph D. Lorenz, Angela G. Marusiak, John B. McClean, Ceri Nunn, Simon C. Stähler, Alexander E. Stott, Tristram Warren

Abstract: Before deploying to the surface of Mars, the short-period (SP) seismometer of the InSight mission operated on deck for a total of 48 hours. This dataset can be used to understand how deck-mounted seismometers can be used in future landed missions to Mars, Europa, and other planetary bodies. While operating on deck, the SP seismometer showed signals comparable to the Viking-2 seismometer near 3 Hz… ▽ More Before deploying to the surface of Mars, the short-period (SP) seismometer of the InSight mission operated on deck for a total of 48 hours. This dataset can be used to understand how deck-mounted seismometers can be used in future landed missions to Mars, Europa, and other planetary bodies. While operating on deck, the SP seismometer showed signals comparable to the Viking-2 seismometer near 3 Hz where the sensitivity of the Viking instrument peaked. Wind sensitivity showed similar patterns to the Viking instrument, although amplitudes on InSight were ~80% larger for a given wind velocity. However, during the low wind evening hours the instrument noise levels at frequencies between 0.1 and 1 Hz were comparable to quiet stations on Earth, although deployment to the surface below the Wind and Thermal Shield lowered installation noise by roughly 40 dB in acceleration power. With the observed noise levels and estimated seismicity rates for Mars, detection probability for quakes for a deck-mounted instrument are low enough that up to years of on-deck recordings may be necessary to observe an event. Because the noise is dominated by wind acting on the lander, though, deck-mounted seismometers may be more practical for deployment on airless bodies, and it is important to evaluate the seismicity of the target body and the specific design of the lander. Detection probabilities for operation on Europa reach over 99% for some proposed seismicity models for a similar duration of operation if noise levels are comparable to low-wind time periods on Mars. △ Less

Submitted 19 March, 2020; originally announced March 2020.

Comments: 20 pages, 7 figures, accepted to Journal of Geophysical Research: Planets

Dust Devils on Titan

Authors: Brian Jackson, Ralph D. Lorenz, Jason W. Barnes, Michelle Szurgot

Abstract: Conditions on Saturn's moon Titan suggest dust devils, which are convective, dust-laden plumes, may be active. Although the exact nature of dust on Titan is unclear, previous observations confirm an active aeolian cycle, and dust devils may play an important role in Titan's aeolian cycle, possibly contributing to regional transport of dust and even production of sand grains. The Dragonfly mission… ▽ More Conditions on Saturn's moon Titan suggest dust devils, which are convective, dust-laden plumes, may be active. Although the exact nature of dust on Titan is unclear, previous observations confirm an active aeolian cycle, and dust devils may play an important role in Titan's aeolian cycle, possibly contributing to regional transport of dust and even production of sand grains. The Dragonfly mission to Titan will document dust devil and convective vortex activity and thereby provide a new window into these features, and our analysis shows that associated winds are likely to be modest and pose no hazard to the mission. △ Less

Submitted 13 February, 2020; originally announced February 2020.

Comments: Accepted for publication in JGR: Planets; data and code available here - https://zenodo.org/record/3470280

Orlicz-space regularization for optimal transport and algorithms for quadratic regularization

Authors: Dirk A. Lorenz, Hinrich Mahler

Abstract: We investigate the continuous optimal transport problem in the so-called Kantorovich form, i.e. given two Radon measures on two compact sets, we seek an optimal transport plan which is another Radon measure on the product of the sets that has these two measures as marginals and minimizes a certain cost function. We consider regularization of the problem with so-called Young's functions, which fo… ▽ More We investigate the continuous optimal transport problem in the so-called Kantorovich form, i.e. given two Radon measures on two compact sets, we seek an optimal transport plan which is another Radon measure on the product of the sets that has these two measures as marginals and minimizes a certain cost function. We consider regularization of the problem with so-called Young's functions, which forces the optimal transport plan to be a function in the corresponding Orlicz space rather than a Radon measure. We derive the predual problem and show strong duality and existence of primal solutions to the regularized problem. Existence of (pre-)dual solutions will be shown for the special case of $L^p$ regularization for $p\geq2$. Then we derive four algorithms to solve the dual problem of the quadratically regularized problem: A cyclic projection method, a dual gradient decent, a simple fixed point method, and Nesterov's accelerated gradient, all of which have a very low cost per iteration. △ Less

Submitted 13 September, 2019; originally announced September 2019.

Entropic regularization of continuous optimal transport problems

Authors: Christian Clason, Dirk A. Lorenz, Hinrich Mahler, Benedikt Wirth

Abstract: We analyze continuous optimal transport problems in the so-called Kantorovich form, where we seek a transport plan between two marginals that are probability measures on compact subsets of Euclidean space. We consider the case of regularization with the negative entropy with respect to the Lebesgue measure, which has attracted attention because it can be solved by the very simple Sinkhorn algorith… ▽ More We analyze continuous optimal transport problems in the so-called Kantorovich form, where we seek a transport plan between two marginals that are probability measures on compact subsets of Euclidean space. We consider the case of regularization with the negative entropy with respect to the Lebesgue measure, which has attracted attention because it can be solved by the very simple Sinkhorn algorithm. We first analyze the regularized problem in the context of classical Fenchel duality and derive a strong duality result for a predual problem in the space of continuous functions. However, this problem may not admit a minimizer, which prevents obtaining primal-dual optimality conditions. We then show that the primal problem is naturally analyzed in the Orlicz space of functions with finite entropy in the sense that the entropically regularized problem admits a minimizer if and only if the marginals have finite entropy. We then derive a dual problem in the corresponding dual space, for which existence can be shown by purely variational arguments and primal-dual optimality conditions can be derived. For marginals that do not have finite entropy, we finally show Gamma-convergence of the regularized problem with smoothed marginals to the original Kantorovich problem. △ Less

Submitted 15 June, 2020; v1 submitted 4 June, 2019; originally announced June 2019.

Journal ref: Journal of Mathematical Analysis and Applications 494 (2021), 124432

Unsupervised Part-Based Disentangling of Object Shape and Appearance

Authors: Dominik Lorenz, Leonard Bereska, Timo Milbich, Björn Ommer

Abstract: Large intra-class variation is the result of changes in multiple object characteristics. Images, however, only show the superposition of different variable factors such as appearance or shape. Therefore, learning to disentangle and represent these different characteristics poses a great challenge, especially in the unsupervised case. Moreover, large object articulation calls for a flexible part-ba… ▽ More Large intra-class variation is the result of changes in multiple object characteristics. Images, however, only show the superposition of different variable factors such as appearance or shape. Therefore, learning to disentangle and represent these different characteristics poses a great challenge, especially in the unsupervised case. Moreover, large object articulation calls for a flexible part-based model. We present an unsupervised approach for disentangling appearance and shape by learning parts consistently over all instances of a category. Our model for learning an object representation is trained by simultaneously exploiting invariance and equivariance constraints between synthetically transformed images. Since no part annotation or prior information on an object class is required, the approach is applicable to arbitrary classes. We evaluate our approach on a wide range of object categories and diverse tasks including pose prediction, disentangled image synthesis, and video-to-video translation. The approach outperforms the state-of-the-art on unsupervised keypoint prediction and compares favorably even against supervised approaches on the task of shape and appearance transfer. △ Less

Submitted 17 June, 2019; v1 submitted 16 March, 2019; originally announced March 2019.

Comments: CVPR 2019 Oral

Quadratically regularized optimal transport

Authors: Dirk A. Lorenz, Paul Manns, Christian Meyer

Abstract: We investigate the problem of optimal transport in the so-called Kantorovich form, i.e. given two Radon measures on two compact sets, we seek an optimal transport plan which is another Radon measure on the product of the sets that has these two measures as marginals and minimizes a certain cost function. We consider quadratic regularization of the problem, which forces the optimal transport plan t… ▽ More We investigate the problem of optimal transport in the so-called Kantorovich form, i.e. given two Radon measures on two compact sets, we seek an optimal transport plan which is another Radon measure on the product of the sets that has these two measures as marginals and minimizes a certain cost function. We consider quadratic regularization of the problem, which forces the optimal transport plan to be a square integrable function rather than a Radon measure. We derive the dual problem and show strong duality and existence of primal and dual solutions to the regularized problem. Then we derive two algorithms to solve the dual problem of the regularized problem: A Gauss-Seidel method and a semismooth quasi-Newton method and investigate both methods numerically. Our experiments show that the methods perform well even for small regularization parameters. Quadratic regularization is of interest since the resulting optimal transport plans are sparse, i.e. they have a small support (which is not the case for the often used entropic regularization where the optimal transport plan always has full measure). △ Less

Submitted 9 September, 2019; v1 submitted 4 March, 2019; originally announced March 2019.

MSC Class: 49Q20; 65D99; 90C25

AnchorHash: A Scalable Consistent Hash

Authors: Gal Mendelson, Shay Vargaftik, Katherine Barabash, Dean Lorenz, Isaac Keslassy, Ariel Orda

Abstract: Consistent hashing (CH) is a central building block in many networking applications, from datacenter load-balancing to distributed storage. Unfortunately, state-of-the-art CH solutions cannot ensure full consistency under arbitrary changes and/or cannot scale while maintaining reasonable memory footprints and update times. We present AnchorHash, a scalable and fully-consistent hashing algorithm. A… ▽ More Consistent hashing (CH) is a central building block in many networking applications, from datacenter load-balancing to distributed storage. Unfortunately, state-of-the-art CH solutions cannot ensure full consistency under arbitrary changes and/or cannot scale while maintaining reasonable memory footprints and update times. We present AnchorHash, a scalable and fully-consistent hashing algorithm. AnchorHash achieves high key lookup rates, a low memory footprint, and low update times. We formally establish its strong theoretical guarantees, and present advanced implementations with a memory footprint of only a few bytes per resource. Moreover, extensive evaluations indicate that it outperforms state-of-the-art algorithms, and that it can scale on a single core to 100 million resources while still achieving a key lookup rate of more than 15 million keys per second. △ Less

Submitted 22 November, 2020; v1 submitted 23 December, 2018; originally announced December 2018.

Sarrus rules and dihedral groups

Authors: Dirk A. Lorenz, Karl-Joachim Wirths

Abstract: This paper is devoted to the analysis of a false generalization of the rule of Sarrus and its properties that can be derived with the help of dihedral groups. Further, we discuss a Sarrus-like scheme that could be helpful for students to memorize the calculation of a $4\times 4$ determinant. This paper is devoted to the analysis of a false generalization of the rule of Sarrus and its properties that can be derived with the help of dihedral groups. Further, we discuss a Sarrus-like scheme that could be helpful for students to memorize the calculation of a $4\times 4$ determinant. △ Less

Submitted 26 September, 2018; v1 submitted 21 September, 2018; originally announced September 2018.

Comments: To appear in "The College Mathematics Journal"

MSC Class: 15A15

Strictly hyperbolic Cauchy problems with coefficients low-regular in time and space

Authors: Daniel Lorenz

Abstract: We consider the strictly hyperbolic Cauchy problem \begin{align*} &D_t^m u - \sum\limits_{j = 0}^{m-1} \sum\limits_{|γ|+j = m} a_{m-j,\,γ}(t,\,x) D_x^γD_t^j u = 0, \newline &D_t^{k-1}u(0,\,x) = g_k(x),\,k = 1,\,\ldots,\,m, \end{align*} for $(t,\,x) \in [0,\,T]\times \mathbb{R}^n$ with coefficients belonging to the Zygmund class $C^s_\ast$ in $x$ and having a modulus of continuity below L… ▽ More We consider the strictly hyperbolic Cauchy problem \begin{align*} &D_t^m u - \sum\limits_{j = 0}^{m-1} \sum\limits_{|γ|+j = m} a_{m-j,\,γ}(t,\,x) D_x^γD_t^j u = 0, \newline &D_t^{k-1}u(0,\,x) = g_k(x),\,k = 1,\,\ldots,\,m, \end{align*} for $(t,\,x) \in [0,\,T]\times \mathbb{R}^n$ with coefficients belonging to the Zygmund class $C^s_\ast$ in $x$ and having a modulus of continuity below Lipschitz in $t$. Imposing additional conditions to control oscillations, we obtain a global (on $[0,\,T]$) $L^2$ energy estimate without loss of derivatives for $s \geq \{1+\varepsilon,\,\frac{2m_0}{2-m_0}\}$, where $m_0$ is linked to the modulus of continuity of the coefficients in time. △ Less

Submitted 16 July, 2018; originally announced July 2018.

MSC Class: 35S05; 35L30; 47G30

Primal-dual residual networks

Authors: Christoph Brauer, Dirk Lorenz

Abstract: In this work, we propose a deep neural network architecture motivated by primal-dual splitting methods from convex optimization. We show theoretically that there exists a close relation between the derived architecture and residual networks, and further investigate this connection in numerical experiments. Moreover, we demonstrate how our approach can be used to unroll optimization algorithms for… ▽ More In this work, we propose a deep neural network architecture motivated by primal-dual splitting methods from convex optimization. We show theoretically that there exists a close relation between the derived architecture and residual networks, and further investigate this connection in numerical experiments. Moreover, we demonstrate how our approach can be used to unroll optimization algorithms for certain problems with hard constraints. Using the example of speech dequantization, we show that our method can outperform classical splitting methods when both are applied to the same task. △ Less

Submitted 15 June, 2018; originally announced June 2018.

Gaia Data Release 2: The first Gaia catalogue of long-period variable candidates

Authors: N. Mowlavi, I. Lecoeur-Taïbi, T. Lebzelter, L. Rimoldini, D. Lorenz, M. Audard, J. De Ridder, L. Eyer, L. P. Guy, B. Holl, G. Jevardat de Fombelle, O. Marchal, K. Nienartowicz, S. Regibo, M. Roelens, L. M. Sarro

Abstract: Gaia DR2 provides a unique all-sky catalogue of 550'737 variable stars, of which 151'761 are long-period variable (LPV) candidates with G variability amplitudes larger than 0.2 mag (5-95% quantile range). About one-fifth of the LPV candidates are Mira candidates, the majority of the rest are semi-regular variable candidates. For each source, G, BP , and RP photometric time-series are published, to… ▽ More Gaia DR2 provides a unique all-sky catalogue of 550'737 variable stars, of which 151'761 are long-period variable (LPV) candidates with G variability amplitudes larger than 0.2 mag (5-95% quantile range). About one-fifth of the LPV candidates are Mira candidates, the majority of the rest are semi-regular variable candidates. For each source, G, BP , and RP photometric time-series are published, together with some LPV-specific attributes for the subset of 89'617 candidates with periods in G longer than 60 days. We describe this first Gaia catalogue of LPV candidates, and present various validation checks. Various samples of LPVs were used to validate the catalogue: a sample of well-studied very bright LPVs with light curves from the AAVSO that are partly contemporaneous with Gaia light curves, a sample of Gaia LPV candidates with good parallaxes, the ASAS_SN catalogue of LPVs, and the OGLE catalogues of LPVs towards the Magellanic Clouds and the Galactic bulge. The analyses of these samples show a good agreement between Gaia DR2 and literature periods. The same is globally true for bolometric corrections of M-type stars. The main contaminant of our DR2 catalogue comes from young stellar objects (YSOs) in the solar vicinity (within ~1 kpc), although their number in the whole catalogue is only at the percent level. A cautionary note is provided about parallax-dependent LPV attributes published in the catalogue. This first Gaia catalogue of LPVs approximately doubles the number of known LPVs with amplitudes larger than 0.2 mag, despite the conservative candidate selection criteria that prioritise low contamination over high completeness, and despite the limited DR2 time coverage compared to the long periods characteristic of LPVs. It also contains a small set of YSO candidates, which offers the serendipitous opportunity to study these objects at an early stage of the Gaia data releases. △ Less

Submitted 27 July, 2018; v1 submitted 5 May, 2018; originally announced May 2018.

Comments: Accepted for publication in A&A. 29 pages, 52 figures

Journal ref: A&A 618, A58 (2018)

Gaia Data Release 2: Summary of the variability processing & analysis results

Authors: B. Holl, M. Audard, K. Nienartowicz, G. Jevardat de Fombelle, O. Marchal, N. Mowlavi, G. Clementini, J. De Ridder, D. W. Evans, L. P. Guy, A. C. Lanzafame, T. Lebzelter, L. Rimoldini, M. Roelens, S. Zucker, E. Distefano, A. Garofalo, I. Lecoeur-Taïbi, M. Lopez, R. Molinaro, T. Muraveva, A. Panahi, S. Regibo, V. Ripepi, L. M. Sarro , et al. (38 additional authors not shown)

Abstract: The Gaia Data Release 2 (DR2): we summarise the processing and results of the identification of variable source candidates of RR Lyrae stars, Cepheids, long period variables (LPVs), rotation modulation (BY Dra-type) stars, delta Scuti & SX Phoenicis stars, and short-timescale variables. In this release we aim to provide useful but not necessarily complete samples of candidates. The processed Gai… ▽ More The Gaia Data Release 2 (DR2): we summarise the processing and results of the identification of variable source candidates of RR Lyrae stars, Cepheids, long period variables (LPVs), rotation modulation (BY Dra-type) stars, delta Scuti & SX Phoenicis stars, and short-timescale variables. In this release we aim to provide useful but not necessarily complete samples of candidates. The processed Gaia data consist of the G, BP, and RP photometry during the first 22 months of operations as well as positions and parallaxes. Various methods from classical statistics, data mining and time series analysis were applied and tailored to the specific properties of Gaia data, as well as various visualisation tools. The DR2 variability release contains: 228'904 RR Lyrae stars, 11'438 Cepheids, 151'761 LPVs, 147'535 stars with rotation modulation, 8'882 delta Scuti & SX Phoenicis stars, and 3'018 short-timescale variables. These results are distributed over a classification and various Specific Object Studies (SOS) tables in the Gaia archive, along with the three-band time series and associated statistics for the underlying 550'737 unique sources. We estimate that about half of them are newly identified variables. The variability type completeness varies strongly as function of sky position due to the non-uniform sky coverage and intermediate calibration level of this data. The probabilistic and automated nature of this work implies certain completeness and contamination rates which are quantified so that users can anticipate their effects. This means that even well-known variable sources can be missed or misidentified in the published data. The DR2 variability release only represents a small subset of the processed data. Future releases will include more variable sources and data products; however, DR2 shows the (already) very high quality of the data and great promise for variability studies. △ Less

Submitted 6 July, 2018; v1 submitted 25 April, 2018; originally announced April 2018.

Comments: 21 pages, 10 figures, 5 tables, accepted by Astronomy & Astrophysics, added several language corrections, and expanded Gaia archive query examples

Journal ref: A&A 618, A30 (2018)

The Randomized Kaczmarz Method with Mismatched Adjoint

Authors: Dirk A. Lorenz, Sean Rose, Frank Schöpfer

Abstract: This paper investigates the randomized version of the Kaczmarz method to solve linear systems in the case where the adjoint of the system matrix is not exact---a situation we refer to as "mismatched adjoint". We show that the method may still converge both in the over- and underdetermined consistent case under appropriate conditions, and we calculate the expected asymptotic rate of linear converge… ▽ More This paper investigates the randomized version of the Kaczmarz method to solve linear systems in the case where the adjoint of the system matrix is not exact---a situation we refer to as "mismatched adjoint". We show that the method may still converge both in the over- and underdetermined consistent case under appropriate conditions, and we calculate the expected asymptotic rate of linear convergence. Moreover, we analyze the inconsistent case and obtain results for the method with mismatched adjoint as for the standard method. Finally, we derive a method to compute optimized probabilities for the choice of the rows and illustrate our findings with numerical example. △ Less

Submitted 7 March, 2018; originally announced March 2018.

MSC Class: 65F10; 68W20; 15A24

Non-stationary Douglas-Rachford and alternating direction method of multipliers: adaptive stepsizes and convergence

Authors: Dirk A. Lorenz, Quoc Tran-Dinh

Abstract: We revisit the classical Douglas-Rachford (DR) method for finding a zero of the sum of two maximal monotone operators. Since the practical performance of the DR method crucially depends on the stepsizes, we aim at develo** an adaptive stepsize rule. To that end, we take a closer look at a linear case of the problem and use our findings to develop a stepsize strategy that eliminates the need for… ▽ More We revisit the classical Douglas-Rachford (DR) method for finding a zero of the sum of two maximal monotone operators. Since the practical performance of the DR method crucially depends on the stepsizes, we aim at develo** an adaptive stepsize rule. To that end, we take a closer look at a linear case of the problem and use our findings to develop a stepsize strategy that eliminates the need for stepsize tuning. We analyze a general non-stationary DR scheme and prove its convergence for a convergent sequence of stepsizes with summable increments. This, in turn, proves the convergence of the method with the new adaptive stepsize rule. We also derive the related non-stationary alternating direction method of multipliers (ADMM) from such a non-stationary DR method. We illustrate the efficiency of the proposed methods on several numerical examples. △ Less

Submitted 27 September, 2018; v1 submitted 11 January, 2018; originally announced January 2018.

MSC Class: 90C25; 65K05; 65J15; 47H05

Denoising of image gradients and total generalized variation denoising

Authors: Birgit Komander, Dirk A. Lorenz, Lena Vestweber

Abstract: We revisit total variation denoising and study an augmented model where we assume that an estimate of the image gradient is available. We show that this increases the image reconstruction quality and derive that the resulting model resembles the total generalized variation denoising method, thus providing a new motivation for this model. Further, we propose to use a constraint denoising model and… ▽ More We revisit total variation denoising and study an augmented model where we assume that an estimate of the image gradient is available. We show that this increases the image reconstruction quality and derive that the resulting model resembles the total generalized variation denoising method, thus providing a new motivation for this model. Further, we propose to use a constraint denoising model and develop a variational denoising model that is basically parameter free, i.e. all model parameters are estimated directly from the noisy image. Moreover, we use Chambolle-Pock's primal dual method as well as the Douglas-Rachford method for the new models. For the latter one has to solve large discretizations of partial differential equations. We propose to do this in an inexact manner using the preconditioned conjugate gradients method and derive preconditioners for this. Numerical experiments show that the resulting method has good denoising properties and also that preconditioning does increase convergence speed significantly. Finally we analyze the duality gap of different formulations of the TGV denoising problem and derive a simple stop** criterion. △ Less

Submitted 4 April, 2018; v1 submitted 22 December, 2017; originally announced December 2017.

Driving Spin-Boson Models From Equilibrium Using Exact Quantum Dynamics

Authors: G. M. G. McCaul, C. D. Lorenz, L. Kantorovich

Abstract: We present an application of the Extended Stochastic Liouville Equation (ESLE) Phys. Rev. B 95, 125124, which gives an exact solution for the reduced density matrix of an open system surrounded by a harmonic heat bath. This method considers the extended system (the open system and the bath) being thermally equilibrated prior to the action of a time dependent perturbation, as opposed to the usual a… ▽ More We present an application of the Extended Stochastic Liouville Equation (ESLE) Phys. Rev. B 95, 125124, which gives an exact solution for the reduced density matrix of an open system surrounded by a harmonic heat bath. This method considers the extended system (the open system and the bath) being thermally equilibrated prior to the action of a time dependent perturbation, as opposed to the usual assumption that system and bath are initially partitioned. This is an exact technique capable of accounting for arbitrary parameter regimes of the model. Here we present our first numerical implementation of the method in the simplest case of a Caldeira-Leggett representation of the bath Hamiltonian, and apply it to a spin-Boson system driven from coupled equilibrium. We observe significant behaviours in both the transient dynamics and asymptotic states of the reduced density matrix not present in the usual approximation. △ Less

Submitted 18 December, 2017; originally announced December 2017.

Comments: 24 pages, 5 figures

Journal ref: Phys. Rev. B 97, 224310 (2018)