Learning Preconditioners for Inverse Problems
Authors:
Matthias J. Ehrhardt,
Patrick Fahy,
Mohammad Golbabaee
Abstract:
We explore the application of preconditioning in optimisation algorithms, specifically those appearing in Inverse Problems in imaging. Such problems often contain an ill-posed forward operator and are large-scale. Therefore, computationally efficient algorithms which converge quickly are desirable. To remedy these issues, learning-to-optimise leverages training data to accelerate solving particula…
▽ More
We explore the application of preconditioning in optimisation algorithms, specifically those appearing in Inverse Problems in imaging. Such problems often contain an ill-posed forward operator and are large-scale. Therefore, computationally efficient algorithms which converge quickly are desirable. To remedy these issues, learning-to-optimise leverages training data to accelerate solving particular optimisation problems. Many traditional optimisation methods use scalar hyperparameters, significantly limiting their convergence speed when applied to ill-conditioned problems. In contrast, we propose a novel approach that replaces these scalar quantities with matrices learned using data. Often, preconditioning considers only symmetric positive-definite preconditioners. However, we consider multiple parametrisations of the preconditioner, which do not require symmetry or positive-definiteness. These parametrisations include using full matrices, diagonal matrices, and convolutions. We analyse the convergence properties of these methods and compare their performance against classical optimisation algorithms. Generalisation performance of these methods is also considered, both for in-distribution and out-of-distribution data.
△ Less
Submitted 31 May, 2024;
originally announced June 2024.
