-
Learning Variational Models with Unrolling and Bilevel Optimization
Authors:
Christoph Brauer,
Niklas Breustedt,
Timo de Wolff,
Dirk A. Lorenz
Abstract:
In this paper we consider the problem of learning variational models in the context of supervised learning via risk minimization. Our goal is to provide a deeper understanding of the two approaches of learning of variational models via bilevel optimization and via algorithm unrolling. The former considers the variational model as a lower level optimization problem below the risk minimization probl…
▽ More
In this paper we consider the problem of learning variational models in the context of supervised learning via risk minimization. Our goal is to provide a deeper understanding of the two approaches of learning of variational models via bilevel optimization and via algorithm unrolling. The former considers the variational model as a lower level optimization problem below the risk minimization problem, while the latter replaces the lower level optimization problem by an algorithm that solves said problem approximately. Both approaches are used in practice, but unrolling is much simpler from a computational point of view. To analyze and compare the two approaches, we consider a simple toy model, and compute all risks and the respective estimators explicitly. We show that unrolling can be better than the bilevel optimization approach, but also that the performance of unrolling can depend significantly on further parameters, sometimes in unexpected ways: While the stepsize of the unrolled algorithm matters a lot (and learning the stepsize gives a significant improvement), the number of unrolled iterations plays a minor role.
△ Less
Submitted 6 September, 2023; v1 submitted 26 September, 2022;
originally announced September 2022.
-
Damage Identification in Fiber Metal Laminates using Bayesian Analysis with Model Order Reduction
Authors:
Nanda Kishore Bellam Muralidhar,
Carmen Gräßle,
Natalie Rauter,
Andrey Mikhaylenko,
Rolf Lammering,
Dirk A. Lorenz
Abstract:
Fiber metal laminates (FML) are composite structures consisting of metals and fiber reinforced plastics (FRP) which have experienced an increasing interest as the choice of materials in aerospace and automobile industries. Due to a sophisticated built up of the material, not only the design and production of such structures is challenging but also its damage detection. This research work focuses o…
▽ More
Fiber metal laminates (FML) are composite structures consisting of metals and fiber reinforced plastics (FRP) which have experienced an increasing interest as the choice of materials in aerospace and automobile industries. Due to a sophisticated built up of the material, not only the design and production of such structures is challenging but also its damage detection. This research work focuses on damage identification in FML with guided ultrasonic waves (GUW) through an inverse approach based on the Bayesian paradigm. As the Bayesian inference approach involves multiple queries of the underlying system, a parameterized reduced-order model (ROM) is used to closely approximate the solution with considerably less computational cost. The signals measured by the embedded sensors and the ROM forecasts are employed for the localization and characterization of damage in FML. In this paper, a Markov Chain Monte-Carlo (MCMC) based Metropolis-Hastings (MH) algorithm and an Ensemble Kalman filtering (EnKF) technique are deployed to identify the damage. Numerical tests illustrate the approaches and the results are compared in regard to accuracy and efficiency. It is found that both methods are successful in multivariate characterization of the damage with a high accuracy and were also able to quantify their associated uncertainties. The EnKF distinguishes itself with the MCMC-MH algorithm in the matter of computational efficiency. In this application of identifying the damage, the EnKF is approximately thrice faster than the MCMC-MH.
△ Less
Submitted 21 April, 2023; v1 submitted 9 June, 2022;
originally announced June 2022.
-
Non-stationary Douglas-Rachford and alternating direction method of multipliers: adaptive stepsizes and convergence
Authors:
Dirk A. Lorenz,
Quoc Tran-Dinh
Abstract:
We revisit the classical Douglas-Rachford (DR) method for finding a zero of the sum of two maximal monotone operators. Since the practical performance of the DR method crucially depends on the stepsizes, we aim at develo** an adaptive stepsize rule. To that end, we take a closer look at a linear case of the problem and use our findings to develop a stepsize strategy that eliminates the need for…
▽ More
We revisit the classical Douglas-Rachford (DR) method for finding a zero of the sum of two maximal monotone operators. Since the practical performance of the DR method crucially depends on the stepsizes, we aim at develo** an adaptive stepsize rule. To that end, we take a closer look at a linear case of the problem and use our findings to develop a stepsize strategy that eliminates the need for stepsize tuning. We analyze a general non-stationary DR scheme and prove its convergence for a convergent sequence of stepsizes with summable increments. This, in turn, proves the convergence of the method with the new adaptive stepsize rule. We also derive the related non-stationary alternating direction method of multipliers (ADMM) from such a non-stationary DR method. We illustrate the efficiency of the proposed methods on several numerical examples.
△ Less
Submitted 27 September, 2018; v1 submitted 11 January, 2018;
originally announced January 2018.
-
An extended Perona-Malik model based on probabilistic models
Authors:
Lars M. Mescheder,
Dirk A. Lorenz
Abstract:
The Perona-Malik model has been very successful at restoring images from noisy input. In this paper, we reinterpret the Perona-Malik model in the language of Gaussian scale mixtures and derive some extensions of the model. Specifically, we show that the expectation-maximization (EM) algorithm applied to Gaussian scale mixtures leads to the lagged-diffusivity algorithm for computing stationary poin…
▽ More
The Perona-Malik model has been very successful at restoring images from noisy input. In this paper, we reinterpret the Perona-Malik model in the language of Gaussian scale mixtures and derive some extensions of the model. Specifically, we show that the expectation-maximization (EM) algorithm applied to Gaussian scale mixtures leads to the lagged-diffusivity algorithm for computing stationary points of the Perona-Malik diffusion equations. Moreover, we show how mean field approximations to these Gaussian scale mixtures lead to a modification of the lagged-diffusivity algorithm that better captures the uncertainties in the restoration. Since this modification can be hard to compute in practice we propose relaxations to the mean field objective to make the algorithm computationally feasible. Our numerical experiments show that this modified lagged-diffusivity algorithm often performs better at restoring textured areas and fuzzy edges than the unmodified algorithm. As a second application of the Gaussian scale mixture framework, we show how an efficient sampling procedure can be obtained for the probabilistic model, making the computation of the conditional mean and other expectations algorithmically feasible. Again, the resulting algorithm has a strong resemblance to the lagged-diffusivity algorithm. Finally, we show that a probabilistic version of the Mumford-Shah segementation model can be obtained in the same framework with a discrete edge-prior.
△ Less
Submitted 19 December, 2016;
originally announced December 2016.
-
Rank-optimal weighting or "How to be best in the OECD Better Life Index?"
Authors:
Jan Lorenz,
Christoph Brauer,
Dirk A. Lorenz
Abstract:
We present a method of rank-optimal weighting which can be used to explore the best possible position of a subject in a ranking based on a composite indicator by means of a mathematical optimization problem. As an example, we explore the dataset of the OECD Better Life Index and compute for each country a weight vector which brings it as far up in the ranking as possible with the greatest advance…
▽ More
We present a method of rank-optimal weighting which can be used to explore the best possible position of a subject in a ranking based on a composite indicator by means of a mathematical optimization problem. As an example, we explore the dataset of the OECD Better Life Index and compute for each country a weight vector which brings it as far up in the ranking as possible with the greatest advance of the immediate rivals. The method is able to answer the question "What is the best possible rank a country can achieve with a given set of weighted indicators?" Typically, weights in composite indicators are justified normatively and not empirically. Our approach helps to give bounds on what is achievable by such normative judgments from a purely output-oriented and strongly competitive perspective. The method can serve as a basis for exact bounds in sensitivity analysis focused on ranking positions.
In the OECD Better Life Index data we find that 19 out the 36 countries in the OECD Better Life Index 2014 can be brought to the top of the ranking by specific weights. We give a table of weights for each country which brings it to its highest possible position. Many countries achieve their best rank by focusing on their strong dimensions and setting the weights of many others to zero. Although setting dimensions to zero is possible in the OECD's online tool, this contradicts the idea of better life being multidimensional in essence. We discuss modifications of the optimization problem which could take this into account, e.g. by allowing only a minimal weight of one.
Methods to find rank-optimal weights can be useful for various multidimensional datasets like the ones used to rank universities or employers.
△ Less
Submitted 16 August, 2016;
originally announced August 2016.
