-
Riemannian geometry for efficient analysis of protein dynamics data
Authors:
Willem Diepeveen,
Carlos Esteve-Yagüe,
Jan Lellmann,
Ozan Öktem,
Carola-Bibiane Schönlieb
Abstract:
An increasingly common viewpoint is that protein dynamics data sets reside in a non-linear subspace of low conformational energy. Ideal data analysis tools for such data sets should therefore account for such non-linear geometry. The Riemannian geometry setting can be suitable for a variety of reasons. First, it comes with a rich structure to account for a wide range of geometries that can be mode…
▽ More
An increasingly common viewpoint is that protein dynamics data sets reside in a non-linear subspace of low conformational energy. Ideal data analysis tools for such data sets should therefore account for such non-linear geometry. The Riemannian geometry setting can be suitable for a variety of reasons. First, it comes with a rich structure to account for a wide range of geometries that can be modelled after an energy landscape. Second, many standard data analysis tools initially developed for data in Euclidean space can also be generalised to data on a Riemannian manifold. In the context of protein dynamics, a conceptual challenge comes from the lack of a suitable smooth manifold and the lack of guidelines for constructing a smooth Riemannian structure based on an energy landscape. In addition, computational feasibility in computing geodesics and related map**s poses a major challenge. This work considers these challenges. The first part of the paper develops a novel local approximation technique for computing geodesics and related map**s on Riemannian manifolds in a computationally feasible manner. The second part constructs a smooth manifold of point clouds modulo rigid body group actions and a Riemannian structure that is based on an energy landscape for protein conformations. The resulting Riemannian geometry is tested on several data analysis tasks relevant for protein dynamics data. It performs exceptionally well on coarse-grained molecular dynamics simulated data. In particular, the geodesics with given start- and end-points approximately recover corresponding molecular dynamics trajectories for proteins that undergo relatively ordered transitions with medium sized deformations. The Riemannian protein geometry also gives physically realistic summary statistics and retrieves the underlying dimension even for large-sized deformations within seconds on a laptop.
△ Less
Submitted 26 October, 2023; v1 submitted 15 August, 2023;
originally announced August 2023.
-
A Universal Quantum Algorithm for Weighted Maximum Cut and Ising Problems
Authors:
Natacha Kuete Meli,
Florian Mannel,
Jan Lellmann
Abstract:
We propose a hybrid quantum-classical algorithm to compute approximate solutions of binary combinatorial problems. We employ a shallow-depth quantum circuit to implement a unitary and Hermitian operator that block-encodes the weighted maximum cut or the Ising Hamiltonian. Measuring the expectation of this operator on a variational quantum state yields the variational energy of the quantum system.…
▽ More
We propose a hybrid quantum-classical algorithm to compute approximate solutions of binary combinatorial problems. We employ a shallow-depth quantum circuit to implement a unitary and Hermitian operator that block-encodes the weighted maximum cut or the Ising Hamiltonian. Measuring the expectation of this operator on a variational quantum state yields the variational energy of the quantum system. The system is enforced to evolve towards the ground state of the problem Hamiltonian by optimizing a set of angles using normalized gradient descent. Experimentally, our algorithm outperforms the state-of-the-art quantum approximate optimization algorithm on random fully connected graphs and challenges D-Wave quantum annealers by producing good approximate solutions. Source code and data files are publicly available.
△ Less
Submitted 15 June, 2023; v1 submitted 10 June, 2023;
originally announced June 2023.
-
Regularising orientation estimation in Cryo-EM 3D map refinement through measure-based lifting over Riemannian manifolds
Authors:
Willem Diepeveen,
Jan Lellmann,
Ozan Öktem,
Carola-Bibiane Schönlieb
Abstract:
Motivated by the trade-off between noise-robustness and data-consistency for joint 3D map reconstruction and rotation estimation in single particle cryogenic-electron microscopy (Cryo-EM), we propose ellipsoidal support lifting (ESL), a measure-based lifting scheme for regularising and approximating the global minimiser of a smooth function over a Riemannian manifold. Under a uniqueness assumption…
▽ More
Motivated by the trade-off between noise-robustness and data-consistency for joint 3D map reconstruction and rotation estimation in single particle cryogenic-electron microscopy (Cryo-EM), we propose ellipsoidal support lifting (ESL), a measure-based lifting scheme for regularising and approximating the global minimiser of a smooth function over a Riemannian manifold. Under a uniqueness assumption on the minimiser we show several theoretical results, in particular well-posedness of the method and an error bound due to the induced bias with respect to the global minimiser. Additionally, we use the developed theory to integrate the measure-based lifting scheme into an alternating update method for joint homogeneous 3D map reconstruction and rotation estimation, where typically tens of thousands of manifold-valued minimisation problems have to be solved and where regularisation is necessary because of the high noise levels in the data. The joint recovery method is used to test both the theoretical predictions and algorithmic performance through numerical experiments with Cryo-EM data. In particular, the induced bias due to the regularising effect of ESL empirically estimates better rotations, i.e., rotations closer to the ground truth, than global optimisation would.
△ Less
Submitted 31 January, 2023; v1 submitted 7 September, 2022;
originally announced September 2022.
-
Inverse Scale Space Iterations for Non-Convex Variational Problems: The Continuous and Discrete Case
Authors:
Danielle Bednarski,
Jan Lellmann
Abstract:
Non-linear filtering approaches allow to obtain decompositions of images with respect to a non-classical notion of scale, induced by the choice of a convex, absolutely one-homogeneous regularizer. The associated inverse scale space flow can be obtained using the classical Bregman iteration with quadratic data term. We apply the Bregman iteration to lifted, i.e. higher-dimensional and convex, funct…
▽ More
Non-linear filtering approaches allow to obtain decompositions of images with respect to a non-classical notion of scale, induced by the choice of a convex, absolutely one-homogeneous regularizer. The associated inverse scale space flow can be obtained using the classical Bregman iteration with quadratic data term. We apply the Bregman iteration to lifted, i.e. higher-dimensional and convex, functionals in order to extend the scope of these approaches to functionals with arbitrary data term. We provide conditions for the subgradients of the regularizer -- in the continuous and discrete setting -- under which this lifted iteration reduces to the standard Bregman iteration. We show experimental results for the convex and non-convex case.
△ Less
Submitted 21 March, 2022;
originally announced March 2022.
-
Inverse Scale Space Iterations for Non-Convex Variational Problems Using Functional Lifting
Authors:
Danielle Bednarski,
Jan Lellmann
Abstract:
Non-linear filtering approaches allow to obtain decompositions of images with respect to a non-classical notion of scale. The associated inverse scale space flow can be obtained using the classical Bregman iteration applied to a convex, absolutely one-homogeneous regularizer. In order to extend these approaches to general energies with non-convex data term, we apply the Bregman iteration to a lift…
▽ More
Non-linear filtering approaches allow to obtain decompositions of images with respect to a non-classical notion of scale. The associated inverse scale space flow can be obtained using the classical Bregman iteration applied to a convex, absolutely one-homogeneous regularizer. In order to extend these approaches to general energies with non-convex data term, we apply the Bregman iteration to a lifted version of the functional with sublabel-accurate discretization. We provide a condition for the subgradients of the regularizer under which this lifted iteration reduces to the standard Bregman iteration. We show experimental results for the convex and non-convex case.
△ Less
Submitted 6 May, 2021;
originally announced May 2021.
-
An Inexact Semi-smooth Newton Method on Riemannian Manifolds with Application to Duality-based Total Variation Denoising
Authors:
Willem Diepeveen,
Jan Lellmann
Abstract:
We propose a higher-order method for solving non-smooth optimization problems on manifolds. In order to obtain superlinear convergence, we apply a Riemannian Semi-smooth Newton method to a non-smooth non-linear primal-dual optimality system based on a recent extension of Fenchel duality theory to Riemannian manifolds. We also propose an inexact version of the Riemannian Semi-smooth Newton method a…
▽ More
We propose a higher-order method for solving non-smooth optimization problems on manifolds. In order to obtain superlinear convergence, we apply a Riemannian Semi-smooth Newton method to a non-smooth non-linear primal-dual optimality system based on a recent extension of Fenchel duality theory to Riemannian manifolds. We also propose an inexact version of the Riemannian Semi-smooth Newton method and prove conditions for local linear and superlinear convergence that hold independent of the sign of the curvature. Numerical experiments on l2-TV-like problems with dual regularization confirm superlinear convergence on manifolds with positive and negative curvature.
△ Less
Submitted 16 August, 2021; v1 submitted 20 February, 2021;
originally announced February 2021.
-
On the Connection between Dynamical Optimal Transport and Functional Lifting
Authors:
Thomas Vogt,
Roland Haase,
Danielle Bednarski,
Jan Lellmann
Abstract:
Functional lifting methods provide a tool for approximating solutions of difficult non-convex problems by embedding them into a larger space. In this work, we investigate a mathematically rigorous formulation based on embedding into the space of pointwise probability measures over a fixed range $Γ$. Interestingly, this approach can be derived as a generalization of the theory of dynamical optimal…
▽ More
Functional lifting methods provide a tool for approximating solutions of difficult non-convex problems by embedding them into a larger space. In this work, we investigate a mathematically rigorous formulation based on embedding into the space of pointwise probability measures over a fixed range $Γ$. Interestingly, this approach can be derived as a generalization of the theory of dynamical optimal transport. Imposing the established continuity equation as a constraint corresponds to variational models with first-order regularization. By modifying the continuity equation, the approach can also be extended to models with higher-order regularization.
△ Less
Submitted 6 July, 2020;
originally announced July 2020.
-
Deformable Groupwise Image Registration using Low-Rank and Sparse Decomposition
Authors:
Roland Haase,
Stefan Heldmann,
Jan Lellmann
Abstract:
Low-rank and sparse decompositions and robust PCA (RPCA) are highly successful techniques in image processing and have recently found use in groupwise image registration. In this paper, we investigate the drawbacks of the most common RPCA-dissimi\-larity metric in image registration and derive an improved version. In particular, this new metric models low-rank requirements through explicit constra…
▽ More
Low-rank and sparse decompositions and robust PCA (RPCA) are highly successful techniques in image processing and have recently found use in groupwise image registration. In this paper, we investigate the drawbacks of the most common RPCA-dissimi\-larity metric in image registration and derive an improved version. In particular, this new metric models low-rank requirements through explicit constraints instead of penalties and thus avoids the pitfalls of the established metric. Equipped with total variation regularization, we present a theoretically justified multilevel scheme based on first-order primal-dual optimization to solve the resulting non-parametric registration problem. As confirmed by numerical experiments, our metric especially lends itself to data involving recurring changes in object appearance and potential sparse perturbations. We numerically compare its peformance to a number of related approaches.
△ Less
Submitted 10 January, 2020;
originally announced January 2020.
-
Lifting methods for manifold-valued variational problems
Authors:
Thomas Vogt,
Evgeny Strekalovskiy,
Daniel Cremers,
Jan Lellmann
Abstract:
Lifting methods allow to transform hard variational problems such as segmentation and optical flow estimation into convex problems in a suitable higher-dimensional space. The lifted models can then be efficiently solved to a global optimum, which allows to find approximate global minimizers of the original problem. Recently, these techniques have also been applied to problems with values in a mani…
▽ More
Lifting methods allow to transform hard variational problems such as segmentation and optical flow estimation into convex problems in a suitable higher-dimensional space. The lifted models can then be efficiently solved to a global optimum, which allows to find approximate global minimizers of the original problem. Recently, these techniques have also been applied to problems with values in a manifold. We provide a review of such methods in a refined framework based on a finite element discretization of the range, which extends the concept of sublabel-accurate lifting to manifolds. We also generalize existing methods for total variation regularization to support general convex regularization.
△ Less
Submitted 10 August, 2019;
originally announced August 2019.
-
Functional Liftings of Vectorial Variational Problems with Laplacian Regularization
Authors:
Thomas Vogt,
Jan Lellmann
Abstract:
We propose a functional lifting-based convex relaxation of variational problems with Laplacian-based second-order regularization. The approach rests on ideas from the calibration method as well as from sublabel-accurate continuous multilabeling approaches, and makes these approaches amenable for variational problems with vectorial data and higher-order regularization, as is common in image process…
▽ More
We propose a functional lifting-based convex relaxation of variational problems with Laplacian-based second-order regularization. The approach rests on ideas from the calibration method as well as from sublabel-accurate continuous multilabeling approaches, and makes these approaches amenable for variational problems with vectorial data and higher-order regularization, as is common in image processing applications. We motivate the approach in the function space setting and prove that, in the special case of absolute Laplacian regularization, it encompasses the discretization-first sublabel-accurate continuous multilabeling approach as a special case. We present a mathematical connection between the lifted and original functional and discuss possible interpretations of minimizers in the lifted function space. Finally, we exemplarily apply the proposed approach to 2D image registration problems.
△ Less
Submitted 1 April, 2019;
originally announced April 2019.
-
Fully-deformable 3D image registration in two seconds
Authors:
Daniel Budelmann,
Lars König,
Nils Papenberg,
Jan Lellmann
Abstract:
We present a highly parallel method for accurate and efficient variational deformable 3D image registration on a consumer-grade graphics processing unit (GPU). We build on recent matrix-free variational approaches and specialize the concepts to the massively-parallel manycore architecture provided by the GPU. Compared to a parallel and optimized CPU implementation, this allows us to achieve an ave…
▽ More
We present a highly parallel method for accurate and efficient variational deformable 3D image registration on a consumer-grade graphics processing unit (GPU). We build on recent matrix-free variational approaches and specialize the concepts to the massively-parallel manycore architecture provided by the GPU. Compared to a parallel and optimized CPU implementation, this allows us to achieve an average speedup of 32.53 on 986 real-world CT thorax-abdomen follow-up scans. At a resolution of approximately $256^3$ voxels, the average runtime is 1.99 seconds for the full registration. On the publicly available DIR-lab benchmark, our method ranks third with respect to average landmark error at an average runtime of 0.32 seconds.
△ Less
Submitted 17 December, 2018;
originally announced December 2018.
-
Higher-Order Total Directional Variation: Imaging Applications
Authors:
Simone Parisotto,
Jan Lellmann,
Simon Masnou,
Carola-Bibiane Schönlieb
Abstract:
We introduce a class of higher-order anisotropic total variation regularisers, which are defined for possibly inhomogeneous, smooth elliptic anisotropies, that extends the Total Generalized Variation (TGV) regulariser and its variants. We propose a primal-dual hybrid gradient approach to approximate numerically the associated gradient flow. This choice of regularisers allows to preserve and enhanc…
▽ More
We introduce a class of higher-order anisotropic total variation regularisers, which are defined for possibly inhomogeneous, smooth elliptic anisotropies, that extends the Total Generalized Variation (TGV) regulariser and its variants. We propose a primal-dual hybrid gradient approach to approximate numerically the associated gradient flow. This choice of regularisers allows to preserve and enhance intrinsic anisotropic features in images. This is illustrated on various examples from different imaging applications: image denoising, wavelet-based image zooming, and reconstruction of surfaces from scattered height measurements.
△ Less
Submitted 6 July, 2020; v1 submitted 12 December, 2018;
originally announced December 2018.
-
A multi-contrast MRI approach to thalamus segmentation
Authors:
Veronica Corona,
Jan Lellmann,
Peter Nestor,
Carola-Bibiane Schoenlieb,
Julio Acosta-Cabronero
Abstract:
Thalamic alterations are relevant to many neurological disorders including Alzheimer's disease, Parkinson's disease and multiple sclerosis. Routine interventions to improve symptom severity in movement disorders, for example, often consist of surgery or deep brain stimulation to diencephalic nuclei. Therefore, accurate delineation of grey matter thalamic subregions is of the upmost clinical import…
▽ More
Thalamic alterations are relevant to many neurological disorders including Alzheimer's disease, Parkinson's disease and multiple sclerosis. Routine interventions to improve symptom severity in movement disorders, for example, often consist of surgery or deep brain stimulation to diencephalic nuclei. Therefore, accurate delineation of grey matter thalamic subregions is of the upmost clinical importance. MRI is highly appropriate for structural segmentation as it provides different views of the anatomy from a single scanning session. Though with several contrasts potentially available, it is also of increasing importance to develop new image segmentation techniques that can operate multi-spectrally. We hereby propose a new segmentation method for use with multi-modality data, which we evaluated for automated segmentation of major thalamic subnuclear groups using T1-, T2*-weighted and quantitative susceptibility map** (QSM) information. The proposed method consists of four steps: highly iterative image co-registration, manual segmentation on the average training-data template, supervised learning for pattern recognition, and a final convex optimisation step imposing further spatial constraints to refine the solution. This led to solutions in greater agreement with manual segmentation than the standard Morel atlas based approach. Furthermore, we show that the multi-contrast approach boosts segmentation performances. We then investigated whether prior knowledge using the training-template contours could further improve convex segmentation accuracy and robustness, which led to highly precise multi-contrast segmentations in single subjects. This approach can be extended to most 3D imaging data types and any region of interest discernible in single scans or multi-subject templates.
△ Less
Submitted 27 July, 2018;
originally announced July 2018.
-
A matrix-free approach to parallel and memory-efficient deformable image registration
Authors:
Lars König,
Jan Rühaak,
Alexander Derksen,
Jan Lellmann
Abstract:
We present a novel computational approach to fast and memory-efficient deformable image registration. In the variational registration model, the computation of the objective function derivatives is the computationally most expensive operation, both in terms of runtime and memory requirements. In order to target this bottleneck, we analyze the matrix structure of gradient and Hessian computations f…
▽ More
We present a novel computational approach to fast and memory-efficient deformable image registration. In the variational registration model, the computation of the objective function derivatives is the computationally most expensive operation, both in terms of runtime and memory requirements. In order to target this bottleneck, we analyze the matrix structure of gradient and Hessian computations for the case of the normalized gradient fields distance measure and curvature regularization. Based on this analysis, we derive equivalent matrix-free closed-form expressions for derivative computations, eliminating the need for storing intermediate results and the costs of sparse matrix arithmetic. This has further benefits: (1) matrix computations can be fully parallelized, (2) memory complexity for derivative computation is reduced from linear to constant, and (3) overall computation times are substantially reduced.
In comparison with an optimized matrix-based reference implementation, the CPU implementation achieves speedup factors between 3.1 and 9.7, and we are able to handle substantially higher resolutions. Using a GPU implementation, we achieve an additional speedup factor of up to 9.2.
Furthermore, we evaluated the approach on real-world medical datasets. On ten publicly available lung CT images from the DIR-Lab 4DCT dataset, we achieve the best mean landmark error of 0.93 mm compared to other submissions on the DIR-Lab website, with an average runtime of only 9.23 s. Complete non-rigid registration of full-size 3D thorax-abdomen CT volumes from oncological follow-up is achieved in 12.6 s. The experimental results show that the proposed matrix-free algorithm enables the use of variational registration models also in applications which were previously impractical due to memory or runtime restrictions.
△ Less
Submitted 27 April, 2018;
originally announced April 2018.
-
Measure-Valued Variational Models with Applications to Diffusion-Weighted Imaging
Authors:
Thomas Vogt,
Jan Lellmann
Abstract:
We develop a general mathematical framework for variational problems where the unknown function assumes values in the space of probability measures on some metric space. We study weak and strong topologies and define a total variation seminorm for functions taking values in a Banach space. The seminorm penalizes jumps and is rotationally invariant under certain conditions. We prove existence of a…
▽ More
We develop a general mathematical framework for variational problems where the unknown function assumes values in the space of probability measures on some metric space. We study weak and strong topologies and define a total variation seminorm for functions taking values in a Banach space. The seminorm penalizes jumps and is rotationally invariant under certain conditions. We prove existence of a minimizer for a class of variational problems based on this formulation of total variation, and provide an example where uniqueness fails to hold. Employing the Kan\-torovich-Rubinstein transport norm from the theory of optimal transport, we propose a variational approach for the restoration of orientation distribution function (ODF)-valued images, as commonly used in Diffusion MRI. We demonstrate that the approach is numerically feasible on several data sets.
△ Less
Submitted 31 May, 2018; v1 submitted 2 October, 2017;
originally announced October 2017.
-
Image reconstruction with imperfect forward models and applications in deblurring
Authors:
Yury Korolev,
Jan Lellmann
Abstract:
We present and analyse an approach to image reconstruction problems with imperfect forward models based on partially ordered spaces - Banach lattices. In this approach, errors in the data and in the forward models are described using order intervals. The method can be characterised as the lattice analogue of the residual method, where the feasible set is defined by linear inequality constraints. T…
▽ More
We present and analyse an approach to image reconstruction problems with imperfect forward models based on partially ordered spaces - Banach lattices. In this approach, errors in the data and in the forward models are described using order intervals. The method can be characterised as the lattice analogue of the residual method, where the feasible set is defined by linear inequality constraints. The study of this feasible set is the main contribution of this paper. Convexity of this feasible set is examined in several settings and modifications for introducing additional information about the forward operator are considered. Numerical examples demonstrate the performance of the method in deblurring with errors in the blurring kernel.
△ Less
Submitted 23 October, 2017; v1 submitted 3 August, 2017;
originally announced August 2017.
-
A graph cut approach to 3D tree delineation, using integrated airborne LiDAR and hyperspectral imagery
Authors:
Juheon Lee,
David Coomes,
Carola-Bibiane Schonlieb,
Xiaohao Cai,
Jan Lellmann,
Michele Dalponte,
Yadvinder Malhi,
Nathalie Butt,
Mike Morecroft
Abstract:
Recognising individual trees within remotely sensed imagery has important applications in forest ecology and management. Several algorithms for tree delineation have been suggested, mostly based on locating local maxima or inverted basins in raster canopy height models (CHMs) derived from Light Detection And Ranging (LiDAR) data or photographs. However, these algorithms often lead to inaccurate es…
▽ More
Recognising individual trees within remotely sensed imagery has important applications in forest ecology and management. Several algorithms for tree delineation have been suggested, mostly based on locating local maxima or inverted basins in raster canopy height models (CHMs) derived from Light Detection And Ranging (LiDAR) data or photographs. However, these algorithms often lead to inaccurate estimates of forest stand characteristics due to the limited information content of raster CHMs. Here we develop a 3D tree delineation method which uses graph cut to delineate trees from the full 3D LiDAR point cloud, and also makes use of any optical imagery available (hyperspectral imagery in our case). First, conventional methods are used to locate local maxima in the CHM and generate an initial map of trees. Second, a graph is built from the LiDAR point cloud, fused with the hyperspectral data. For computational efficiency, the feature space of hyperspectral imagery is reduced using robust PCA. Third, a multi-class normalised cut is applied to the graph, using the initial map of trees to constrain the number of clusters and their locations. Finally, recursive normalised cut is used to subdivide, if necessary, each of the clusters identified by the initial analysis. We call this approach Multiclass Cut followed by Recursive Cut (MCRC). The effectiveness of MCRC was tested using three datasets: i) NewFor, ii) a coniferous forest in the Italian Alps, and iii) a deciduous woodland in the UK. The performance of MCRC was usually superior to that of other delineation methods, and was further improved by including high-resolution optical imagery. Since MCRC delineates the entire LiDAR point cloud in 3D, it allows individual crown characteristics to be measured. By making full use of the data available, graph cut has the potential to considerably improve the accuracy of tree delineation.
△ Less
Submitted 23 January, 2017;
originally announced January 2017.
-
Sublabel-Accurate Convex Relaxation of Vectorial Multilabel Energies
Authors:
Emanuel Laude,
Thomas Möllenhoff,
Michael Moeller,
Jan Lellmann,
Daniel Cremers
Abstract:
Convex relaxations of nonconvex multilabel problems have been demonstrated to produce superior (provably optimal or near-optimal) solutions to a variety of classical computer vision problems. Yet, they are of limited practical use as they require a fine discretization of the label space, entailing a huge demand in memory and runtime. In this work, we propose the first sublabel accurate convex rela…
▽ More
Convex relaxations of nonconvex multilabel problems have been demonstrated to produce superior (provably optimal or near-optimal) solutions to a variety of classical computer vision problems. Yet, they are of limited practical use as they require a fine discretization of the label space, entailing a huge demand in memory and runtime. In this work, we propose the first sublabel accurate convex relaxation for vectorial multilabel problems. The key idea is that we approximate the dataterm of the vectorial labeling problem in a piecewise convex (rather than piecewise linear) manner. As a result we have a more faithful approximation of the original cost function that provides a meaningful interpretation for the fractional solutions of the relaxed convex problem. In numerous experiments on large-displacement optical flow estimation and on color image denoising we demonstrate that the computed solutions have superior quality while requiring much lower memory and runtime.
△ Less
Submitted 10 October, 2016; v1 submitted 7 April, 2016;
originally announced April 2016.
-
Sublabel-Accurate Relaxation of Nonconvex Energies
Authors:
Thomas Möllenhoff,
Emanuel Laude,
Michael Moeller,
Jan Lellmann,
Daniel Cremers
Abstract:
We propose a novel spatially continuous framework for convex relaxations based on functional lifting. Our method can be interpreted as a sublabel-accurate solution to multilabel problems. We show that previously proposed functional lifting methods optimize an energy which is linear between two labels and hence require (often infinitely) many labels for a faithful approximation. In contrast, the pr…
▽ More
We propose a novel spatially continuous framework for convex relaxations based on functional lifting. Our method can be interpreted as a sublabel-accurate solution to multilabel problems. We show that previously proposed functional lifting methods optimize an energy which is linear between two labels and hence require (often infinitely) many labels for a faithful approximation. In contrast, the proposed formulation is based on a piecewise convex approximation and therefore needs far fewer labels. In comparison to recent MRF-based approaches, our method is formulated in a spatially continuous setting and shows less grid bias. Moreover, in a local sense, our formulation is the tightest possible convex relaxation. It is easy to implement and allows an efficient primal-dual optimization on GPUs. We show the effectiveness of our approach on several computer vision problems.
△ Less
Submitted 4 December, 2015;
originally announced December 2015.
-
Analysis and Application of a non-local Hessian
Authors:
Jan Lellmann,
Konstantinos Papafitsoros,
Carola Schoenlieb,
Daniel Spector
Abstract:
In this work we introduce a formulation for a non-local Hessian that combines the ideas of higher-order and non-local regularization for image restoration, extending the idea of non-local gradients to higher-order derivatives. By carefully choosing the weights, the model allows to improve on the current state of the art higher-order method, Total Generalized Variation, with respect to overall qual…
▽ More
In this work we introduce a formulation for a non-local Hessian that combines the ideas of higher-order and non-local regularization for image restoration, extending the idea of non-local gradients to higher-order derivatives. By carefully choosing the weights, the model allows to improve on the current state of the art higher-order method, Total Generalized Variation, with respect to overall quality and particularly close to jumps in the data. In the spirit of recent work by Brezis et al., our formulation also has analytic implications: for a suitable choice of weights, it can be shown to converge to classical second-order regularizers, and in fact allows a novel characterization of higher-order Sobolev and BV spaces
△ Less
Submitted 31 October, 2014;
originally announced October 2014.
-
Solving QVIs for Image Restoration with Adaptive Constraint Sets
Authors:
Frank Lenzen,
Jan Lellmann,
Florian Becker,
Christoph Schnörr
Abstract:
We consider a class of quasi-variational inequalities (QVIs) for adaptive image restoration, where the adaptivity is described via solution-dependent constraint sets. In previous work we studied both theoretical and numerical issues. While we were able to show the existence of solutions for a relatively broad class of problems, we encountered problems concerning uniqueness of the solution as well…
▽ More
We consider a class of quasi-variational inequalities (QVIs) for adaptive image restoration, where the adaptivity is described via solution-dependent constraint sets. In previous work we studied both theoretical and numerical issues. While we were able to show the existence of solutions for a relatively broad class of problems, we encountered problems concerning uniqueness of the solution as well as convergence of existing algorithms for solving QVIs. In particular, it seemed that with increasing image size the growing condition number of the involved differential operator poses severe problems. In the present paper we prove uniqueness for a larger class of problems and in particular independent of the image size. Moreover, we provide a numerical algorithm with proved convergence. Experimental results support our theoretical findings.
△ Less
Submitted 3 July, 2014;
originally announced July 2014.
-
Imaging with Kantorovich-Rubinstein discrepancy
Authors:
Jan Lellmann,
Dirk A. Lorenz,
Carola Schönlieb,
Tuomo Valkonen
Abstract:
We propose the use of the Kantorovich-Rubinstein norm from optimal transport in imaging problems. In particular, we discuss a variational regularisation model endowed with a Kantorovich-Rubinstein discrepancy term and total variation regularization in the context of image denoising and cartoon-texture decomposition. We point out connections of this approach to several other recently proposed metho…
▽ More
We propose the use of the Kantorovich-Rubinstein norm from optimal transport in imaging problems. In particular, we discuss a variational regularisation model endowed with a Kantorovich-Rubinstein discrepancy term and total variation regularization in the context of image denoising and cartoon-texture decomposition. We point out connections of this approach to several other recently proposed methods such as total generalized variation and norms capturing oscillating patterns. We also show that the respective optimization problem can be turned into a convex-concave saddle point problem with simple constraints and hence, can be solved by standard tools. Numerical examples exhibit interesting features and favourable performance for denoising and cartoon-texture decomposition.
△ Less
Submitted 1 July, 2014;
originally announced July 2014.
-
A Comparative Study of Modern Inference Techniques for Structured Discrete Energy Minimization Problems
Authors:
Jörg H. Kappes,
Bjoern Andres,
Fred A. Hamprecht,
Christoph Schnörr,
Sebastian Nowozin,
Dhruv Batra,
Sungwoong Kim,
Bernhard X. Kausler,
Thorben Kröger,
Jan Lellmann,
Nikos Komodakis,
Bogdan Savchynskyy,
Carsten Rother
Abstract:
Szeliski et al. published an influential study in 2006 on energy minimization methods for Markov Random Fields (MRF). This study provided valuable insights in choosing the best optimization technique for certain classes of problems. While these insights remain generally useful today, the phenomenal success of random field models means that the kinds of inference problems that have to be solved cha…
▽ More
Szeliski et al. published an influential study in 2006 on energy minimization methods for Markov Random Fields (MRF). This study provided valuable insights in choosing the best optimization technique for certain classes of problems. While these insights remain generally useful today, the phenomenal success of random field models means that the kinds of inference problems that have to be solved changed significantly. Specifically, the models today often include higher order interactions, flexible connectivity structures, large la\-bel-spaces of different cardinalities, or learned energy tables. To reflect these changes, we provide a modernized and enlarged study. We present an empirical comparison of 32 state-of-the-art optimization techniques on a corpus of 2,453 energy minimization instances from diverse applications in computer vision. To ensure reproducibility, we evaluate all methods in the OpenGM 2 framework and report extensive results regarding runtime and solution quality. Key insights from our study agree with the results of Szeliski et al. for the types of models they studied. However, on new and challenging types of models our findings disagree and suggest that polyhedral methods and integer programming solvers are competitive in terms of runtime and solution quality over a large range of model types.
△ Less
Submitted 2 April, 2014;
originally announced April 2014.
-
Optimality Bounds for a Variational Relaxation of the Image Partitioning Problem
Authors:
Jan Lellmann,
Frank Lenzen,
Christoph Schnörr
Abstract:
We consider a variational convex relaxation of a class of optimal partitioning and multiclass labeling problems, which has recently proven quite successful and can be seen as a continuous analogue of Linear Programming (LP) relaxation methods for finite-dimensional problems. While for the latter case several optimality bounds are known, to our knowledge no such bounds exist in the continuous setti…
▽ More
We consider a variational convex relaxation of a class of optimal partitioning and multiclass labeling problems, which has recently proven quite successful and can be seen as a continuous analogue of Linear Programming (LP) relaxation methods for finite-dimensional problems. While for the latter case several optimality bounds are known, to our knowledge no such bounds exist in the continuous setting. We provide such a bound by analyzing a probabilistic rounding method, showing that it is possible to obtain an integral solution of the original partitioning problem from a solution of the relaxed problem with an a priori upper bound on the objective, ensuring the quality of the result from the viewpoint of optimization. The approach has a natural interpretation as an approximate, multiclass variant of the celebrated coarea formula.
△ Less
Submitted 5 December, 2011;
originally announced December 2011.
-
Continuous Multiclass Labeling Approaches and Algorithms
Authors:
Jan Lellmann,
Christoph Schnörr
Abstract:
We study convex relaxations of the image labeling problem on a continuous domain with regularizers based on metric interaction potentials. The generic framework ensures existence of minimizers and covers a wide range of relaxations of the originally combinatorial problem. We focus on two specific relaxations that differ in flexibility and simplicity -- one can be used to tightly relax any metric i…
▽ More
We study convex relaxations of the image labeling problem on a continuous domain with regularizers based on metric interaction potentials. The generic framework ensures existence of minimizers and covers a wide range of relaxations of the originally combinatorial problem. We focus on two specific relaxations that differ in flexibility and simplicity -- one can be used to tightly relax any metric interaction potential, while the other one only covers Euclidean metrics but requires less computational effort. For solving the nonsmooth discretized problem, we propose a globally convergent Douglas-Rachford scheme, and show that a sequence of dual iterates can be recovered in order to provide a posteriori optimality bounds. In a quantitative comparison to two other first-order methods, the approach shows competitive performance on synthetical and real-world images. By combining the method with an improved binarization technique for nonstandard potentials, we were able to routinely recover discrete solutions within 1%--5% of the global optimum for the combinatorial image labeling problem.
△ Less
Submitted 28 February, 2011; v1 submitted 26 February, 2011;
originally announced February 2011.
