-
Adversarial Boot Camp: label free certified robustness in one epoch
Authors:
Ryan Campbell,
Chris Finlay,
Adam M Oberman
Abstract:
Machine learning models are vulnerable to adversarial attacks. One approach to addressing this vulnerability is certification, which focuses on models that are guaranteed to be robust for a given perturbation size. A drawback of recent certified models is that they are stochastic: they require multiple computationally expensive model evaluations with random noise added to a given input. In our wor…
▽ More
Machine learning models are vulnerable to adversarial attacks. One approach to addressing this vulnerability is certification, which focuses on models that are guaranteed to be robust for a given perturbation size. A drawback of recent certified models is that they are stochastic: they require multiple computationally expensive model evaluations with random noise added to a given input. In our work, we present a deterministic certification approach which results in a certifiably robust model. This approach is based on an equivalence between training with a particular regularized loss, and the expected values of Gaussian averages. We achieve certified models on ImageNet-1k by retraining a model with this loss for one epoch without the use of label information.
△ Less
Submitted 5 October, 2020;
originally announced October 2020.
-
Deterministic Gaussian Averaged Neural Networks
Authors:
Ryan Campbell,
Chris Finlay,
Adam M Oberman
Abstract:
We present a deterministic method to compute the Gaussian average of neural networks used in regression and classification. Our method is based on an equivalence between training with a particular regularized loss, and the expected values of Gaussian averages. We use this equivalence to certify models which perform well on clean data but are not robust to adversarial perturbations. In terms of cer…
▽ More
We present a deterministic method to compute the Gaussian average of neural networks used in regression and classification. Our method is based on an equivalence between training with a particular regularized loss, and the expected values of Gaussian averages. We use this equivalence to certify models which perform well on clean data but are not robust to adversarial perturbations. In terms of certified accuracy and adversarial robustness, our method is comparable to known stochastic methods such as randomized smoothing, but requires only a single model evaluation during inference.
△ Less
Submitted 10 June, 2020;
originally announced June 2020.
-
Learning normalizing flows from Entropy-Kantorovich potentials
Authors:
Chris Finlay,
Augusto Gerolin,
Adam M Oberman,
Aram-Alexandre Pooladian
Abstract:
We approach the problem of learning continuous normalizing flows from a dual perspective motivated by entropy-regularized optimal transport, in which continuous normalizing flows are cast as gradients of scalar potential functions. This formulation allows us to train a dual objective comprised only of the scalar potential functions, and removes the burden of explicitly computing normalizing flows…
▽ More
We approach the problem of learning continuous normalizing flows from a dual perspective motivated by entropy-regularized optimal transport, in which continuous normalizing flows are cast as gradients of scalar potential functions. This formulation allows us to train a dual objective comprised only of the scalar potential functions, and removes the burden of explicitly computing normalizing flows during training. After training, the normalizing flow is easily recovered from the potential functions.
△ Less
Submitted 10 June, 2020;
originally announced June 2020.
-
How to train your neural ODE: the world of Jacobian and kinetic regularization
Authors:
Chris Finlay,
Jörn-Henrik Jacobsen,
Levon Nurbekyan,
Adam M Oberman
Abstract:
Training neural ODEs on large datasets has not been tractable due to the necessity of allowing the adaptive numerical ODE solver to refine its step size to very small values. In practice this leads to dynamics equivalent to many hundreds or even thousands of layers. In this paper, we overcome this apparent difficulty by introducing a theoretically-grounded combination of both optimal transport and…
▽ More
Training neural ODEs on large datasets has not been tractable due to the necessity of allowing the adaptive numerical ODE solver to refine its step size to very small values. In practice this leads to dynamics equivalent to many hundreds or even thousands of layers. In this paper, we overcome this apparent difficulty by introducing a theoretically-grounded combination of both optimal transport and stability regularizations which encourage neural ODEs to prefer simpler dynamics out of all the dynamics that solve a problem well. Simpler dynamics lead to faster convergence and to fewer discretizations of the solver, considerably decreasing wall-clock time without loss in performance. Our approach allows us to train neural ODE-based generative models to the same performance as the unregularized dynamics, with significant reductions in training time. This brings neural ODEs closer to practical relevance in large-scale applications.
△ Less
Submitted 23 June, 2020; v1 submitted 7 February, 2020;
originally announced February 2020.
-
No-collision Transportation Maps
Authors:
Levon Nurbekyan,
Alexander Iannantuono,
Adam M. Oberman
Abstract:
Transportation maps between probability measures are critical objects in numerous areas of mathematics and applications such as PDE, fluid mechanics, geometry, machine learning, computer science, and economics. Given a pair of source and target measures, one searches for a map that has suitable properties and transports the source measure to the target one. Here, we study maps that possess the \te…
▽ More
Transportation maps between probability measures are critical objects in numerous areas of mathematics and applications such as PDE, fluid mechanics, geometry, machine learning, computer science, and economics. Given a pair of source and target measures, one searches for a map that has suitable properties and transports the source measure to the target one. Here, we study maps that possess the \textit{no-collision} property; that is, particles simultaneously traveling from sources to targets in a unit time with uniform velocities do not collide. These maps are particularly relevant for applications in swarm control problems. We characterize these no-collision maps in terms of \textit{half-space preserving} property and establish a direct connection between these maps and \textit{binary-space-partitioning (BSP) tree} structures. Based on this characterization, we provide explicit BSP algorithms, of cost $O(n \log n)$, to construct no-collision maps. Moreover, interpreting these maps as approximations of optimal transportation maps, we find that they succeed in computing nearly optimal maps for $q$-Wasserstein metric ($q=1,2$). In some cases, our maps yield costs that are just a few percent off from being optimal.
△ Less
Submitted 5 April, 2020; v1 submitted 4 December, 2019;
originally announced December 2019.
-
Farkas layers: don't shift the data, fix the geometry
Authors:
Aram-Alexandre Pooladian,
Chris Finlay,
Adam M Oberman
Abstract:
Successfully training deep neural networks often requires either batch normalization, appropriate weight initialization, both of which come with their own challenges. We propose an alternative, geometrically motivated method for training. Using elementary results from linear programming, we introduce Farkas layers: a method that ensures at least one neuron is active at a given layer. Focusing on r…
▽ More
Successfully training deep neural networks often requires either batch normalization, appropriate weight initialization, both of which come with their own challenges. We propose an alternative, geometrically motivated method for training. Using elementary results from linear programming, we introduce Farkas layers: a method that ensures at least one neuron is active at a given layer. Focusing on residual networks with ReLU activation, we empirically demonstrate a significant improvement in training capacity in the absence of batch normalization or methods of initialization across a broad range of network sizes on benchmark datasets.
△ Less
Submitted 4 October, 2019;
originally announced October 2019.
-
Partial differential equation regularization for supervised machine learning
Authors:
Adam M Oberman
Abstract:
This article is an overview of supervised machine learning problems for regression and classification. Topics include: kernel methods, training by stochastic gradient descent, deep learning architecture, losses for classification, statistical learning theory, and dimension independent generalization bounds. Implicit regularization in deep learning examples are presented, including data augmentatio…
▽ More
This article is an overview of supervised machine learning problems for regression and classification. Topics include: kernel methods, training by stochastic gradient descent, deep learning architecture, losses for classification, statistical learning theory, and dimension independent generalization bounds. Implicit regularization in deep learning examples are presented, including data augmentation, adversarial training, and additive noise. These methods are reframed as explicit gradient regularization.
△ Less
Submitted 3 October, 2019;
originally announced October 2019.
-
Nesterov's method with decreasing learning rate leads to accelerated stochastic gradient descent
Authors:
Maxime Laborde,
Adam M. Oberman
Abstract:
We present a coupled system of ODEs which, when discretized with a constant time step/learning rate, recovers Nesterov's accelerated gradient descent algorithm. The same ODEs, when discretized with a decreasing learning rate, leads to novel stochastic gradient descent (SGD) algorithms, one in the convex and a second in the strongly convex case. In the strongly convex case, we obtain an algorithm s…
▽ More
We present a coupled system of ODEs which, when discretized with a constant time step/learning rate, recovers Nesterov's accelerated gradient descent algorithm. The same ODEs, when discretized with a decreasing learning rate, leads to novel stochastic gradient descent (SGD) algorithms, one in the convex and a second in the strongly convex case. In the strongly convex case, we obtain an algorithm superficially similar to momentum SGD, but with additional terms. In the convex case, we obtain an algorithm with a novel order $k^{3/4}$ learning rate. We prove, extending the Lyapunov function approach from the full gradient case to the stochastic case, that the algorithms converge at the optimal rate for the last iterate of SGD, with rate constants which are better than previously available.
△ Less
Submitted 1 September, 2020; v1 submitted 21 August, 2019;
originally announced August 2019.
-
Scaleable input gradient regularization for adversarial robustness
Authors:
Chris Finlay,
Adam M Oberman
Abstract:
In this work we revisit gradient regularization for adversarial robustness with some new ingredients. First, we derive new per-image theoretical robustness bounds based on local gradient information. These bounds strongly motivate input gradient regularization. Second, we implement a scaleable version of input gradient regularization which avoids double backpropagation: adversarially robust ImageN…
▽ More
In this work we revisit gradient regularization for adversarial robustness with some new ingredients. First, we derive new per-image theoretical robustness bounds based on local gradient information. These bounds strongly motivate input gradient regularization. Second, we implement a scaleable version of input gradient regularization which avoids double backpropagation: adversarially robust ImageNet models are trained in 33 hours on four consumer grade GPUs. Finally, we show experimentally and through theoretical certification that input gradient regularization is competitive with adversarial training. Moreover we demonstrate that gradient regularization does not lead to gradient obfuscation or gradient masking.
△ Less
Submitted 4 October, 2019; v1 submitted 27 May, 2019;
originally announced May 2019.
-
The LogBarrier adversarial attack: making effective use of decision boundary information
Authors:
Chris Finlay,
Aram-Alexandre Pooladian,
Adam M. Oberman
Abstract:
Adversarial attacks for image classification are small perturbations to images that are designed to cause misclassification by a model. Adversarial attacks formally correspond to an optimization problem: find a minimum norm image perturbation, constrained to cause misclassification. A number of effective attacks have been developed. However, to date, no gradient-based attacks have used best practi…
▽ More
Adversarial attacks for image classification are small perturbations to images that are designed to cause misclassification by a model. Adversarial attacks formally correspond to an optimization problem: find a minimum norm image perturbation, constrained to cause misclassification. A number of effective attacks have been developed. However, to date, no gradient-based attacks have used best practices from the optimization literature to solve this constrained minimization problem. We design a new untargeted attack, based on these best practices, using the established logarithmic barrier method. On average, our attack distance is similar or better than all state-of-the-art attacks on benchmark datasets (MNIST, CIFAR10, ImageNet-1K). In addition, our method performs significantly better on the most challenging images, those which normally require larger perturbations for misclassification. We employ the LogBarrier attack on several adversarially defended models, and show that it adversarially perturbs all images more efficiently than other attacks: the distance needed to perturb all images is significantly smaller with the LogBarrier attack than with other state-of-the-art attacks.
△ Less
Submitted 25 March, 2019;
originally announced March 2019.
-
Calibrated Top-1 Uncertainty estimates for classification by score based models
Authors:
Adam M. Oberman,
Chris Finlay,
Alexander Iannantuono,
Tiago Salvador
Abstract:
While the accuracy of modern deep learning models has significantly improved in recent years, the ability of these models to generate uncertainty estimates has not progressed to the same degree. Uncertainty methods are designed to provide an estimate of class probabilities when predicting class assignment.
While there are a number of proposed methods for estimating uncertainty, they all suffer f…
▽ More
While the accuracy of modern deep learning models has significantly improved in recent years, the ability of these models to generate uncertainty estimates has not progressed to the same degree. Uncertainty methods are designed to provide an estimate of class probabilities when predicting class assignment.
While there are a number of proposed methods for estimating uncertainty, they all suffer from a lack of calibration: predicted probabilities can be off from empirical ones by a few percent or more. By restricting the scope of our predictions to only the probability of Top-1 error, we can decrease the calibration error of existing methods to less than one percent. As a result, the scores of the methods also improve significantly over benchmarks.
△ Less
Submitted 16 June, 2020; v1 submitted 21 March, 2019;
originally announced March 2019.
-
Stochastic Gradient Descent with Polyak's Learning Rate
Authors:
Adam M. Oberman,
Mariana Prazeres
Abstract:
Stochastic gradient descent (SGD) for strongly convex functions converges at the rate $\bO(1/k)$. However, achieving good results in practice requires tuning the parameters (for example the learning rate) of the algorithm. In this paper we propose a generalization of the Polyak step size, used for subgradient methods, to Stochastic gradient descent. We prove a non-asymptotic convergence at the rat…
▽ More
Stochastic gradient descent (SGD) for strongly convex functions converges at the rate $\bO(1/k)$. However, achieving good results in practice requires tuning the parameters (for example the learning rate) of the algorithm. In this paper we propose a generalization of the Polyak step size, used for subgradient methods, to Stochastic gradient descent. We prove a non-asymptotic convergence at the rate $\bO(1/k)$ with a rate constant which can be better than the corresponding rate constant for optimally scheduled SGD. We demonstrate that the method is effective in practice, and on convex optimization problems and on training deep neural networks, and compare to the theoretical rate.
△ Less
Submitted 11 July, 2019; v1 submitted 20 March, 2019;
originally announced March 2019.
-
Approximate homogenization of fully nonlinear elliptic PDEs: estimates and numerical results for Pucci type equations
Authors:
Chris Finlay,
Adam M. Oberman
Abstract:
We are interested in the shape of the homogenized operator $\overline F(Q)$ for PDEs which have the structure of a nonlinear Pucci operator. A typical operator is $H^{a_1,a_2}(Q,x) = a_1(x) λ_{\min}(Q) + a_2(x)λ_{\max}(Q)$. Linearization of the operator leads to a non-divergence form homogenization problem, which can be solved by averaging against the invariant measure. We estimate the error obtai…
▽ More
We are interested in the shape of the homogenized operator $\overline F(Q)$ for PDEs which have the structure of a nonlinear Pucci operator. A typical operator is $H^{a_1,a_2}(Q,x) = a_1(x) λ_{\min}(Q) + a_2(x)λ_{\max}(Q)$. Linearization of the operator leads to a non-divergence form homogenization problem, which can be solved by averaging against the invariant measure. We estimate the error obtained by linearization based on semi-concavity estimates on the nonlinear operator. These estimates show that away from high curvature regions, the linearization can be accurate. Numerical results show that for many values of $Q$, the linearization is highly accurate, and that even near corners, the error can be small (a few percent) even for relatively wide ranges of the coefficients.
△ Less
Submitted 14 May, 2018; v1 submitted 27 October, 2017;
originally announced October 2017.
-
Approximate homogenization of convex nonlinear elliptic PDEs
Authors:
Chris Finlay,
Adam M. Oberman
Abstract:
We approximate the homogenization of fully nonlinear, convex, uniformly elliptic Partial Differential Equations in the periodic setting, using a variational formula for the optimal invariant measure, which may be derived via Legendre-Fenchel duality. The variational formula expresses $\bar H$ as an average of the operator against the optimal invariant measure, generalizing the linear case. Several…
▽ More
We approximate the homogenization of fully nonlinear, convex, uniformly elliptic Partial Differential Equations in the periodic setting, using a variational formula for the optimal invariant measure, which may be derived via Legendre-Fenchel duality. The variational formula expresses $\bar H$ as an average of the operator against the optimal invariant measure, generalizing the linear case. Several nontrivial analytic formulas for $\bar H$ are obtained. These formulas are compared to numerical simulations, using both PDE and variational methods. We also perform a numerical study of convergence rates for homogenization in the periodic and random setting and compare these to theoretical results.
△ Less
Submitted 27 October, 2017;
originally announced October 2017.
-
Approximate Convex Hulls: sketching the convex hull using curvature
Authors:
Robert Graham,
Adam M. Oberman
Abstract:
Convex hulls are fundamental objects in computational geometry. In moderate dimensions or for large numbers of vertices, computing the convex hull can be impractical due to the computational complexity of convex hull algorithms. In this article we approximate the convex hull in using a scalable algorithm which finds high curvature vertices with high probability. The algorithm is particularly effec…
▽ More
Convex hulls are fundamental objects in computational geometry. In moderate dimensions or for large numbers of vertices, computing the convex hull can be impractical due to the computational complexity of convex hull algorithms. In this article we approximate the convex hull in using a scalable algorithm which finds high curvature vertices with high probability. The algorithm is particularly effective for approximating convex hulls which have a relatively small number of extreme points.
△ Less
Submitted 14 June, 2017; v1 submitted 27 February, 2017;
originally announced March 2017.
-
A partial differential equation for the strictly quasiconvex envelope
Authors:
Bilal Abbasi,
Adam M. Oberman
Abstract:
In a series of papers Barron, Goebel, and Jensen studied Partial Differential Equations (PDE)s for quasiconvex (QC) functions \cite{barron2012functions, barron2012quasiconvex,barron2013quasiconvex,barron2013uniqueness}. To overcome the lack of uniqueness for the QC PDE, they introduced a regularization: a PDE for $\e$-robust QC functions, which is well-posed. Building on this work, we introduce a…
▽ More
In a series of papers Barron, Goebel, and Jensen studied Partial Differential Equations (PDE)s for quasiconvex (QC) functions \cite{barron2012functions, barron2012quasiconvex,barron2013quasiconvex,barron2013uniqueness}. To overcome the lack of uniqueness for the QC PDE, they introduced a regularization: a PDE for $\e$-robust QC functions, which is well-posed. Building on this work, we introduce a stronger regularization which is amenable to numerical approximation. We build convergent finite difference approximations, comparing the QC envelope and the two regularization. Solutions of this PDE are strictly convex, and smoother than the robust-QC functions.
△ Less
Submitted 20 December, 2016;
originally announced December 2016.
-
Computing the quasiconvex envelope using a nonlocal line solver
Authors:
Bilal Abbasi,
Adam M. Oberman
Abstract:
Recently in a series of articles, Barron, Goebel, and Jensen \cite{barron2012functions} \cite{barron2012quasiconvex} \cite{barron2013quasiconvex} \cite{barron2013uniqueness} have studied second order degenerate elliptic PDE and first order nonlocal PDEs for the quasiconvex envelope. Quasiconvex functions are functions whose level sets are convex. The PDE is difficult to solve. In this article we p…
▽ More
Recently in a series of articles, Barron, Goebel, and Jensen \cite{barron2012functions} \cite{barron2012quasiconvex} \cite{barron2013quasiconvex} \cite{barron2013uniqueness} have studied second order degenerate elliptic PDE and first order nonlocal PDEs for the quasiconvex envelope. Quasiconvex functions are functions whose level sets are convex. The PDE is difficult to solve. In this article we present an algorithm for computing the quasiconvex envelope (QCE) of a given function. The QCE operator is a level set operator, so this algorithm gives a method to compute convex hull of sets represented by a level set functions. We present a nonlocal line solver for the quasiconvex envelope (QCE), based on solving the one dimensional problem on lines. We find an explicit formula for the QCE of a function defined on a line.
△ Less
Submitted 16 December, 2016;
originally announced December 2016.
-
Numerical methods for motion of level sets by affine curvature
Authors:
Adam M. Oberman,
Tiago Salvador
Abstract:
We study numerical methods for the nonlinear partial differential equation that governs the motion of level sets by affine curvature. We show that standard finite difference schemes are nonlinearly unstable. We build convergent finite difference schemes, using the theory of viscosity solutions. We demonstrate that our approximate solutions capture the affine invariance and morphological properties…
▽ More
We study numerical methods for the nonlinear partial differential equation that governs the motion of level sets by affine curvature. We show that standard finite difference schemes are nonlinearly unstable. We build convergent finite difference schemes, using the theory of viscosity solutions. We demonstrate that our approximate solutions capture the affine invariance and morphological properties of the evolution. Numerical experiments demonstrate the accuracy and stability of the discretization.
△ Less
Submitted 28 October, 2016; v1 submitted 27 October, 2016;
originally announced October 2016.
-
Anomaly detection and classification for streaming data using PDEs
Authors:
Bilal Abbasi,
Jeff Calder,
Adam M. Oberman
Abstract:
Nondominated sorting, also called Pareto Depth Analysis (PDA), is widely used in multi-objective optimization and has recently found important applications in multi-criteria anomaly detection. Recently, a partial differential equation (PDE) continuum limit was discovered for nondominated sorting leading to a very fast approximate sorting algorithm called PDE-based ranking. We propose in this paper…
▽ More
Nondominated sorting, also called Pareto Depth Analysis (PDA), is widely used in multi-objective optimization and has recently found important applications in multi-criteria anomaly detection. Recently, a partial differential equation (PDE) continuum limit was discovered for nondominated sorting leading to a very fast approximate sorting algorithm called PDE-based ranking. We propose in this paper a fast real-time streaming version of the PDA algorithm for anomaly detection that exploits the computational advantages of PDE continuum limits. Furthermore, we derive new PDE continuum limits for sorting points within their nondominated layers and show how the new PDEs can be used to classify anomalies based on which criterion was more significantly violated. We also prove statistical convergence rates for PDE-based ranking, and present the results of numerical experiments with both synthetic and real data.
△ Less
Submitted 15 March, 2017; v1 submitted 15 August, 2016;
originally announced August 2016.
-
A partial differential equation for the rank one convex envelope
Authors:
Adam M. Oberman,
Yuanlong Ruan
Abstract:
In this article we introduce a Partial Differential Equation (PDE) for the rank one convex envelope. Rank one convex envelopes arise in non-convex vector valued variational problems \cite{BallElasticity, kohn1986optimal1, BallJames87, chipot1988equilibrium}. More generally, we study a PDE for directional convex envelopes, which includes the usual convex envelope \cite{ObermanConvexEnvelope} and th…
▽ More
In this article we introduce a Partial Differential Equation (PDE) for the rank one convex envelope. Rank one convex envelopes arise in non-convex vector valued variational problems \cite{BallElasticity, kohn1986optimal1, BallJames87, chipot1988equilibrium}. More generally, we study a PDE for directional convex envelopes, which includes the usual convex envelope \cite{ObermanConvexEnvelope} and the rank one convex envelope as special cases. Existence and uniqueness of viscosity solutions to the PDE is established. Wide stencil elliptic finite difference schemes are built. Convergence of finite difference solutions to the viscosity solution of the PDE is proven. Numerical examples of rank one and other directional convex envelopes are presented. Additionally, laminates are computed from the rank one convex envelope.
△ Less
Submitted 12 January, 2017; v1 submitted 10 May, 2016;
originally announced May 2016.
-
An efficient linear programming method for Optimal Transportation
Authors:
Adam M. Oberman,
Yuanlong Ruan
Abstract:
An efficient method for computing solutions to the Optimal Transportation (OT) problem with a wide class of cost functions is presented. The standard linear programming (LP) discretization of the continuous problem becomes intractible for moderate grid sizes. A grid refinement method results in a linear cost algorithm. Weak convergence of solutions is stablished. Barycentric projection of transfer…
▽ More
An efficient method for computing solutions to the Optimal Transportation (OT) problem with a wide class of cost functions is presented. The standard linear programming (LP) discretization of the continuous problem becomes intractible for moderate grid sizes. A grid refinement method results in a linear cost algorithm. Weak convergence of solutions is stablished. Barycentric projection of transference plans is used to improve the accuracy of solutions. The method is applied to more general problems, including partial optimal transportation, and barycenter problems. Computational examples validate the accuracy and efficiency of the method. Optimal maps between nonconvex domains, partial OT free boundaries, and high accuracy barycenters are presented.
△ Less
Submitted 11 September, 2015;
originally announced September 2015.
-
Numerical Methods for the 2-Hessian Elliptic Partial Differential Equation
Authors:
Brittany D. Froese,
Adam M. Oberman,
Tiago Salvador
Abstract:
The elliptic 2-Hessian equation is a fully nonlinear partial differential equation (PDE) that is related to intrinsic curvature for three dimensional manifolds. We introduce two numerical methods for this PDE: the first is provably convergent to the viscosity solution, and the second is more accurate, and convergent in practice but lacks a proof. The PDE is elliptic on a restricted set of function…
▽ More
The elliptic 2-Hessian equation is a fully nonlinear partial differential equation (PDE) that is related to intrinsic curvature for three dimensional manifolds. We introduce two numerical methods for this PDE: the first is provably convergent to the viscosity solution, and the second is more accurate, and convergent in practice but lacks a proof. The PDE is elliptic on a restricted set of functions: a convexity type constraint is needed for the ellipticity of the PDE operator. Solutions with both discretizations are obtained using Newton's method. Computational results are presented on a number of exact solutions which range in regularity from smooth to nondifferentiable and in shape from convex to non convex.
△ Less
Submitted 10 February, 2016; v1 submitted 17 February, 2015;
originally announced February 2015.
-
Adaptive finite difference methods for nonlinear elliptic and parabolic partial differential equations with free boundaries
Authors:
Adam M. Oberman,
Ian Zwiers
Abstract:
Monotone finite difference methods provide stable convergent discretizations of a class of degenerate elliptic and parabolic Partial Differential Equations (PDEs). These methods are best suited to regular rectangular grids, which leads to low accuracy near curved boundaries or singularities of solutions. In this article we combine monotone finite difference methods with an adaptive grid refinement…
▽ More
Monotone finite difference methods provide stable convergent discretizations of a class of degenerate elliptic and parabolic Partial Differential Equations (PDEs). These methods are best suited to regular rectangular grids, which leads to low accuracy near curved boundaries or singularities of solutions. In this article we combine monotone finite difference methods with an adaptive grid refinement technique to produce a PDE discretization and solver which is applied to a broad class of equations, in curved or unbounded domains which include free boundaries. The grid refinement is flexible and adaptive. The discretization is combined with a fast solution method, which incorporates asynchronous time step** adapted to the spatial scale. The framework is validated on linear problems in curved and unbounded domains. Key applications include the obstacle problem and the one-phase Stefan free boundary problem.
△ Less
Submitted 18 November, 2015; v1 submitted 9 December, 2014;
originally announced December 2014.
-
A multigrid scheme for 3D Monge-Ampère equations
Authors:
Jun Liu,
Brittany D. Froese,
Adam M. Oberman,
Mingqing Xiao
Abstract:
The elliptic Monge-Ampère equation is a fully nonlinear partial differential equation which has been the focus of increasing attention from the scientific computing community. Fast three dimensional solvers are needed, for example in medical image registration but are not yet available. We build fast solvers for smooth solutions in three dimensions using a nonlinear full-approximation storage mult…
▽ More
The elliptic Monge-Ampère equation is a fully nonlinear partial differential equation which has been the focus of increasing attention from the scientific computing community. Fast three dimensional solvers are needed, for example in medical image registration but are not yet available. We build fast solvers for smooth solutions in three dimensions using a nonlinear full-approximation storage multigrid method. Starting from a second-order accurate centered finite difference approximation, we present a nonlinear Gauss-Seidel iterative method which has a mechanism for selecting the convex solution of the equation. The iterative method is used as an effective smoother, combined with the full-approximation storage multigrid method. Numerical experiments are provided to validate the accuracy of the finite difference scheme and illustrate the computational efficiency of the proposed multigrid solver.
△ Less
Submitted 29 December, 2016; v1 submitted 25 November, 2014;
originally announced November 2014.
-
Filtered schemes for Hamilton-Jacobi equations: a simple construction of convergent accurate difference schemes
Authors:
Adam M. Oberman,
Tiago Salvador
Abstract:
We build a simple and general class of finite difference schemes for first order Hamilton-Jacobi (HJ) Partial Differential Equations. These filtered schemes are convergent to the unique viscosity solution of the equation. The schemes are accurate: we implement second, third and fourth order accurate schemes in one dimension and second order accurate schemes in two dimensions, indicating how to bui…
▽ More
We build a simple and general class of finite difference schemes for first order Hamilton-Jacobi (HJ) Partial Differential Equations. These filtered schemes are convergent to the unique viscosity solution of the equation. The schemes are accurate: we implement second, third and fourth order accurate schemes in one dimension and second order accurate schemes in two dimensions, indicating how to build higher order ones. They are also explicit, which means they can be solved using the fast swee** method or the fast marching method.The accuracy of the method is validated with computational results for the eikonal equation in one and two dimensions, using filtered schemes made from standard centered differences, higher order upwinding and ENO interpolation.
△ Less
Submitted 12 November, 2014;
originally announced November 2014.
-
Nonlinear elliptic Partial Differential Equations and p-harmonic functions on graphs
Authors:
Juan J. Manfredi,
Adam M. Oberman,
Alex P. Svirodov
Abstract:
In this article we study the well-posedness (uniqueness and existence of solutions) of nonlinear elliptic Partial Differential Equations (PDEs) on a finite graph. These results are obtained using the discrete comparison principle and connectivity properties of the graph. This work is in the spirit of the theory of viscosity solutions for PDEs.
The equations include the graph Laplacian, the $p$-L…
▽ More
In this article we study the well-posedness (uniqueness and existence of solutions) of nonlinear elliptic Partial Differential Equations (PDEs) on a finite graph. These results are obtained using the discrete comparison principle and connectivity properties of the graph. This work is in the spirit of the theory of viscosity solutions for PDEs.
The equations include the graph Laplacian, the $p$-Laplacian, the Infinity Laplacian, the Mean Curvature equation, and the Eikonal operator on the graph.
△ Less
Submitted 28 February, 2013; v1 submitted 4 December, 2012;
originally announced December 2012.
-
A viscosity solution approach to the Monge-Ampere formulation of the Optimal Transportation Problem
Authors:
Jean-David Benamou,
Brittany D. Froese,
Adam M. Oberman
Abstract:
In this work we present a numerical method for the Optimal Mass Transportation problem. Optimal Mass Transportation (OT) is an active research field in mathematics.It has recently led to significant theoretical results as well as applications in diverse areas. Numerical solution techniques for the OT problem remain underdeveloped. The solution is obtained by solving the second boundary value probl…
▽ More
In this work we present a numerical method for the Optimal Mass Transportation problem. Optimal Mass Transportation (OT) is an active research field in mathematics.It has recently led to significant theoretical results as well as applications in diverse areas. Numerical solution techniques for the OT problem remain underdeveloped. The solution is obtained by solving the second boundary value problem for the MA equation, a fully nonlinear elliptic partial differential equation (PDE). Instead of standard boundary conditions the problem has global state constraints. These are reformulated as a tractable local PDE. We give a proof of convergence of the numerical method, using the theory of viscosity solutions. Details of the implementation and a fast solution method are provided in the companion paper arXiv:1208.4870.
△ Less
Submitted 2 August, 2013; v1 submitted 23 August, 2012;
originally announced August 2012.
-
Numerical solution of the Optimal Transportation problem using the Monge-Ampere equation
Authors:
Jean-David Benamou,
Brittany D. Froese,
Adam M. Oberman
Abstract:
A numerical method for the solution of the elliptic Monge-Ampere Partial Differential Equation, with boundary conditions corresponding to the Optimal Transportation (OT) problem is presented. A local representation of the OT boundary conditions is combined with a finite difference scheme for the Monge-Ampere equation. Newton's method is implemented leading to a fast solver, comparable to solving t…
▽ More
A numerical method for the solution of the elliptic Monge-Ampere Partial Differential Equation, with boundary conditions corresponding to the Optimal Transportation (OT) problem is presented. A local representation of the OT boundary conditions is combined with a finite difference scheme for the Monge-Ampere equation. Newton's method is implemented leading to a fast solver, comparable to solving the Laplace equation on the same grid several times. Theoretical justification for the method is given by a convergence proof in the companion paper (Benamou et al., 2012). In this paper, the algorithm is modified to a simpler compact stencil implementation and details of the implementation are given. Solutions are computed with densities supported on non-convex and disconnected domains. Computational examples demonstrate robust performance on singular solutions and fast computational times.
△ Less
Submitted 23 August, 2012;
originally announced August 2012.
-
Convergent filtered schemes for the Monge-Ampère partial differential equation
Authors:
Brittany D. Froese,
Adam M. Oberman
Abstract:
The theory of viscosity solutions has been effective for representing and approximating weak solutions to fully nonlinear Partial Differential Equations (PDEs) such as the elliptic Monge-Ampère equation. The approximation theory of Barles-Souganidis [Barles and Souganidis, Asymptotic Anal., 4 (1999) 271-283] requires that numerical schemes be monotone (or elliptic in the sense of [Oberman, SIAM J.…
▽ More
The theory of viscosity solutions has been effective for representing and approximating weak solutions to fully nonlinear Partial Differential Equations (PDEs) such as the elliptic Monge-Ampère equation. The approximation theory of Barles-Souganidis [Barles and Souganidis, Asymptotic Anal., 4 (1999) 271-283] requires that numerical schemes be monotone (or elliptic in the sense of [Oberman, SIAM J. Numer. Anal, 44 (2006) 879-895]. But such schemes have limited accuracy. In this article, we establish a convergence result for nearly monotone schemes. This allows us to construct finite difference discretizations of arbitrarily high-order. We demonstrate that the higher accuracy is achieved when solutions are sufficiently smooth. In addition, the filtered scheme provides a natural detection principle for singularities. We employ this framework to construct a formally second-order scheme for the Monge-Ampère equation and present computational results on smooth and singular solutions.
△ Less
Submitted 3 December, 2012; v1 submitted 25 April, 2012;
originally announced April 2012.
-
A numerical method for variational problems with convexity constraints
Authors:
Adam M. Oberman
Abstract:
We consider the problem of approximating the solution of variational problems subject to the constraint that the admissible functions must be convex. This problem is at the interface between convex analysis, convex optimization, variational problems, and partial differential equation techniques.
The approach is to approximate the (non-polyhedral) cone of convex functions by a polyhedral cone whi…
▽ More
We consider the problem of approximating the solution of variational problems subject to the constraint that the admissible functions must be convex. This problem is at the interface between convex analysis, convex optimization, variational problems, and partial differential equation techniques.
The approach is to approximate the (non-polyhedral) cone of convex functions by a polyhedral cone which can be represented by linear inequalities. This approach leads to an optimization problem with linear constraints which can be computed efficiently, hundreds of times faster than existing methods.
△ Less
Submitted 23 August, 2012; v1 submitted 26 July, 2011;
originally announced July 2011.
-
Finite difference methods for the Infinity Laplace and p-Laplace equations
Authors:
Adam M. Oberman
Abstract:
We build convergent discretizations and semi-implicit solvers for the Infinity Laplacian and the game theoretical $p$-Laplacian. The discretizations simplify and generalize earlier ones. We prove convergence of the solution of the Wide Stencil finite difference schemes to the unique viscosity solution of the underlying equation. We build a semi-implicit solver, which solves the Laplace equation as…
▽ More
We build convergent discretizations and semi-implicit solvers for the Infinity Laplacian and the game theoretical $p$-Laplacian. The discretizations simplify and generalize earlier ones. We prove convergence of the solution of the Wide Stencil finite difference schemes to the unique viscosity solution of the underlying equation. We build a semi-implicit solver, which solves the Laplace equation as each step. It is fast in the sense that the number of iterations is independent of the problem size. This is an improvement over previous explicit solvers, which are slow due to the CFL-condition.
△ Less
Submitted 5 December, 2012; v1 submitted 26 July, 2011;
originally announced July 2011.
-
The Dirichlet problem for the convex envelope
Authors:
Luis Silvestre,
Adam M. Oberman
Abstract:
The Convex Envelope of a given function was recently characterized as the solution of a fully nonlinear Partial Differential Equation (PDE). In this article we study a modified problem: the Dirichlet problem for the underlying PDE. The main result is an optimal regularity result. Differentiability ($C^{1,α}$ regularity) of the boundary data implies the corresponding result for the solution in the…
▽ More
The Convex Envelope of a given function was recently characterized as the solution of a fully nonlinear Partial Differential Equation (PDE). In this article we study a modified problem: the Dirichlet problem for the underlying PDE. The main result is an optimal regularity result. Differentiability ($C^{1,α}$ regularity) of the boundary data implies the corresponding result for the solution in the interior, despite the fact that the solution need not be continuous up to the boundary. Secondary results are the characterization of the convex envelope as: (i) the value function of a stochastic control problem, and (ii) the optimal underestimator for a class of nonlinear elliptic PDEs.
△ Less
Submitted 5 July, 2010;
originally announced July 2010.
-
Convergent finite difference solvers for viscosity solutions of the elliptic Monge-Ampère equation in dimensions two and higher
Authors:
Brittany D. Froese,
Adam M. Oberman
Abstract:
The elliptic Monge-Ampère equation is a fully nonlinear Partial Differential Equation that originated in geometric surface theory and has been applied in dynamic meteorology, elasticity, geometric optics, image processing and image registration. Solutions can be singular, in which case standard numerical approaches fail. Novel solution methods are required for stability and convergence to the weak…
▽ More
The elliptic Monge-Ampère equation is a fully nonlinear Partial Differential Equation that originated in geometric surface theory and has been applied in dynamic meteorology, elasticity, geometric optics, image processing and image registration. Solutions can be singular, in which case standard numerical approaches fail. Novel solution methods are required for stability and convergence to the weak (viscosity) solution.
In this article we build a wide stencil finite difference discretization for the \MA equation. The scheme is monotone, so the Barles-Souganidis theory allows us to prove that the solution of the scheme converges to the unique viscosity solution of the equation.
Solutions of the scheme are found using a damped Newton's method. We prove convergence of Newton's method and provide a systematic method to determine a starting point for the Newton iteration.
Computational results are presented in two and three dimensions, which demonstrates the speed and accuracy of the method on a number of exact solutions, which range in regularity from smooth to non-differentiable.
△ Less
Submitted 3 June, 2011; v1 submitted 5 July, 2010;
originally announced July 2010.
-
Fast finite difference solvers for singular solutions of the elliptic Monge-Ampère equation
Authors:
Brittany D. Froese,
Adam M. Oberman
Abstract:
The elliptic Monge-Ampere equation is a fully nonlinear Partial Differential Equation which originated in geometric surface theory, and has been applied in dynamic meteorology, elasticity, geometric optics, image processing and image registration. Solutions can be singular, in which case standard numerical approaches fail. In this article we build a finite difference solver for the Monge-Ampere eq…
▽ More
The elliptic Monge-Ampere equation is a fully nonlinear Partial Differential Equation which originated in geometric surface theory, and has been applied in dynamic meteorology, elasticity, geometric optics, image processing and image registration. Solutions can be singular, in which case standard numerical approaches fail. In this article we build a finite difference solver for the Monge-Ampere equation, which converges even for singular solutions. Regularity results are used to select a priori between a stable, provably convergent monotone discretization and an accurate finite difference discretization in different regions of the computational domain. This allows singular solutions to be computed using a stable method, and regular solutions to be computed more accurately. The resulting nonlinear equations are then solved by Newton's method. Computational results in two and three dimensions validate the claims of accuracy and solution speed. A computational example is presented which demonstrates the necessity of the use of the monotone scheme near singularities.
△ Less
Submitted 8 July, 2010; v1 submitted 29 June, 2010;
originally announced June 2010.
