-
Adaptive scaling of the learning rate by second order automatic differentiation
Authors:
Frédéric de Gournay,
Alban Gossard
Abstract:
In the context of the optimization of Deep Neural Networks, we propose to rescale the learning rate using a new technique of automatic differentiation. This technique relies on the computation of the {\em curvature}, a second order information whose computational complexity is in between the computation of the gradient and the one of the Hessian-vector product. If (1C,1M) represents respectively…
▽ More
In the context of the optimization of Deep Neural Networks, we propose to rescale the learning rate using a new technique of automatic differentiation. This technique relies on the computation of the {\em curvature}, a second order information whose computational complexity is in between the computation of the gradient and the one of the Hessian-vector product. If (1C,1M) represents respectively the computational time and memory footprint of the gradient method, the new technique increase the overall cost to either (1.5C,2M) or (2C,1M). This rescaling has the appealing characteristic of having a natural interpretation, it allows the practitioner to choose between exploration of the parameters set and convergence of the algorithm. The rescaling is adaptive, it depends on the data and on the direction of descent. The numerical experiments highlight the different exploration/convergence regimes.
△ Less
Submitted 26 October, 2022;
originally announced October 2022.
-
CorticalFlow: A Diffeomorphic Mesh Deformation Module for Cortical Surface Reconstruction
Authors:
Léo Lebrat,
Rodrigo Santa Cruz,
Frédéric de Gournay,
Darren Fu,
Pierrick Bourgeat,
Jurgen Fripp,
Clinton Fookes,
Olivier Salvado
Abstract:
In this paper we introduce CorticalFlow, a new geometric deep-learning model that, given a 3-dimensional image, learns to deform a reference template towards a targeted object. To conserve the template mesh's topological properties, we train our model over a set of diffeomorphic transformations. This new implementation of a flow Ordinary Differential Equation (ODE) framework benefits from a small…
▽ More
In this paper we introduce CorticalFlow, a new geometric deep-learning model that, given a 3-dimensional image, learns to deform a reference template towards a targeted object. To conserve the template mesh's topological properties, we train our model over a set of diffeomorphic transformations. This new implementation of a flow Ordinary Differential Equation (ODE) framework benefits from a small GPU memory footprint, allowing the generation of surfaces with several hundred thousand vertices. To reduce topological errors introduced by its discrete resolution, we derive numeric conditions which improve the manifoldness of the predicted triangle mesh. To exhibit the utility of CorticalFlow, we demonstrate its performance for the challenging task of brain cortical surface reconstruction. In contrast to current state-of-the-art, CorticalFlow produces superior surfaces while reducing the computation time from nine and a half minutes to one second. More significantly, CorticalFlow enforces the generation of anatomically plausible surfaces; the absence of which has been a major impediment restricting the clinical relevance of such surface reconstruction methods.
△ Less
Submitted 6 June, 2022;
originally announced June 2022.
-
Off-the-grid data-driven optimization of sampling schemes in MRI
Authors:
Alban Gossard,
Frédéric de Gournay,
Pierre Weiss
Abstract:
We propose a novel learning based algorithm to generate efficient and physically plausible sampling patterns in MRI. This method has a few advantages compared to recent learning based approaches: i) it works off-the-grid and ii) allows to handle arbitrary physical constraints. These two features allow for much more versatility in the sampling patterns that can take advantage of all the degrees of…
▽ More
We propose a novel learning based algorithm to generate efficient and physically plausible sampling patterns in MRI. This method has a few advantages compared to recent learning based approaches: i) it works off-the-grid and ii) allows to handle arbitrary physical constraints. These two features allow for much more versatility in the sampling patterns that can take advantage of all the degrees of freedom offered by an MRI scanner. The method consists in a high dimensional optimization of a cost function defined implicitly by an algorithm. We propose various numerical tools to address this numerical challenge.
△ Less
Submitted 5 October, 2020;
originally announced October 2020.
-
Convex Regularization and Representer Theorems
Authors:
Claire Boyer,
Antonin Chambolle,
Yohann de Castro,
Vincent Duval,
Frédéric de Gournay,
Pierre Weiss
Abstract:
We establish a result which states that regularizing an inverse problem with the gauge of a convex set $C$ yields solutions which are linear combinations of a few extreme points or elements of the extreme rays of $C$. These can be understood as the \textit{atoms} of the regularizer. We then explicit that general principle by using a few popular applications. In particular, we relate it to the comm…
▽ More
We establish a result which states that regularizing an inverse problem with the gauge of a convex set $C$ yields solutions which are linear combinations of a few extreme points or elements of the extreme rays of $C$. These can be understood as the \textit{atoms} of the regularizer. We then explicit that general principle by using a few popular applications. In particular, we relate it to the common wisdom that total gradient variation minimization favors the reconstruction of piecewise constant images.
△ Less
Submitted 11 December, 2018;
originally announced December 2018.
-
On Representer Theorems and Convex Regularization
Authors:
Claire Boyer,
Antonin Chambolle,
Yohann De Castro,
Vincent Duval,
Frédéric De Gournay,
Pierre Weiss
Abstract:
We establish a general principle which states that regularizing an inverse problem with a convex function yields solutions which are convex combinations of a small number of atoms. These atoms are identified with the extreme points and elements of the extreme rays of the regularizer level sets. An extension to a broader class of quasi-convex regularizers is also discussed. As a side result, we cha…
▽ More
We establish a general principle which states that regularizing an inverse problem with a convex function yields solutions which are convex combinations of a small number of atoms. These atoms are identified with the extreme points and elements of the extreme rays of the regularizer level sets. An extension to a broader class of quasi-convex regularizers is also discussed. As a side result, we characterize the minimizers of the total gradient variation, which was still an unresolved problem.
△ Less
Submitted 26 November, 2018; v1 submitted 26 June, 2018;
originally announced June 2018.
