-
Event-Based Eye Tracking. AIS 2024 Challenge Survey
Authors:
Zuowen Wang,
Chang Gao,
Zongwei Wu,
Marcos V. Conde,
Radu Timofte,
Shih-Chii Liu,
Qinyu Chen,
Zheng-jun Zha,
Wei Zhai,
Han Han,
Bohao Liao,
Yuliang Wu,
Zengyu Wan,
Zhong Wang,
Yang Cao,
Ganchao Tan,
**ze Chen,
Yan Ru Pei,
Sasskia Brüers,
Sébastien Crouzet,
Douglas McLelland,
Oliver Coenen,
Baoheng Zhang,
Yizhao Gao,
**gyuan Li
, et al. (14 additional authors not shown)
Abstract:
This survey reviews the AIS 2024 Event-Based Eye Tracking (EET) Challenge. The task of the challenge focuses on processing eye movement recorded with event cameras and predicting the pupil center of the eye. The challenge emphasizes efficient eye tracking with event cameras to achieve good task accuracy and efficiency trade-off. During the challenge period, 38 participants registered for the Kaggl…
▽ More
This survey reviews the AIS 2024 Event-Based Eye Tracking (EET) Challenge. The task of the challenge focuses on processing eye movement recorded with event cameras and predicting the pupil center of the eye. The challenge emphasizes efficient eye tracking with event cameras to achieve good task accuracy and efficiency trade-off. During the challenge period, 38 participants registered for the Kaggle competition, and 8 teams submitted a challenge factsheet. The novel and diverse methods from the submitted factsheets are reviewed and analyzed in this survey to advance future event-based eye tracking research.
△ Less
Submitted 17 April, 2024;
originally announced April 2024.
-
StEik: Stabilizing the Optimization of Neural Signed Distance Functions and Finer Shape Representation
Authors:
Huizong Yang,
Yuxin Sun,
Ganesh Sundaramoorthi,
Anthony Yezzi
Abstract:
We present new insights and a novel paradigm (StEik) for learning implicit neural representations (INR) of shapes. In particular, we shed light on the popular eikonal loss used for imposing a signed distance function constraint in INR. We show analytically that as the representation power of the network increases, the optimization approaches a partial differential equation (PDE) in the continuum l…
▽ More
We present new insights and a novel paradigm (StEik) for learning implicit neural representations (INR) of shapes. In particular, we shed light on the popular eikonal loss used for imposing a signed distance function constraint in INR. We show analytically that as the representation power of the network increases, the optimization approaches a partial differential equation (PDE) in the continuum limit that is unstable. We show that this instability can manifest in existing network optimization, leading to irregularities in the reconstructed surface and/or convergence to sub-optimal local minima, and thus fails to capture fine geometric and topological structure. We show analytically how other terms added to the loss, currently used in the literature for other purposes, can actually eliminate these instabilities. However, such terms can over-regularize the surface, preventing the representation of fine shape detail. Based on a similar PDE theory for the continuum limit, we introduce a new regularization term that still counteracts the eikonal instability but without over-regularizing. Furthermore, since stability is now guaranteed in the continuum limit, this stabilization also allows for considering new network structures that are able to represent finer shape detail. We introduce such a structure based on quadratic layers. Experiments on multiple benchmark data sets show that our new regularization and network are able to capture more precise shape details and more accurate topology than existing state-of-the-art.
△ Less
Submitted 11 November, 2023; v1 submitted 28 May, 2023;
originally announced May 2023.
-
A PDE-based Explanation of Extreme Numerical Sensitivities and Edge of Stability in Training Neural Networks
Authors:
Yuxin Sun,
Dong Lao,
Ganesh Sundaramoorthi,
Anthony Yezzi
Abstract:
We discover restrained numerical instabilities in current training practices of deep networks with stochastic gradient descent (SGD), and its variants. We show numerical error (on the order of the smallest floating point bit and thus the most extreme or limiting numerical perturbations induced from floating point arithmetic in training deep nets can be amplified significantly and result in signifi…
▽ More
We discover restrained numerical instabilities in current training practices of deep networks with stochastic gradient descent (SGD), and its variants. We show numerical error (on the order of the smallest floating point bit and thus the most extreme or limiting numerical perturbations induced from floating point arithmetic in training deep nets can be amplified significantly and result in significant test accuracy variance (sensitivities), comparable to the test accuracy variance due to stochasticity in SGD. We show how this is likely traced to instabilities of the optimization dynamics that are restrained, i.e., localized over iterations and regions of the weight tensor space. We do this by presenting a theoretical framework using numerical analysis of partial differential equations (PDE), and analyzing the gradient descent PDE of convolutional neural networks (CNNs). We show that it is stable only under certain conditions on the learning rate and weight decay. We show that rather than blowing up when the conditions are violated, the instability can be restrained. We show this is a consequence of the non-linear PDE associated with the gradient descent of the CNN, whose local linearization changes when over-driving the step size of the discretization, resulting in a stabilizing effect. We link restrained instabilities to the recently discovered Edge of Stability (EoS) phenomena, in which the stable step size predicted by classical theory is exceeded while continuing to optimize the loss and still converging. Because restrained instabilities occur at the EoS, our theory provides new insights and predictions about the EoS, in particular, the role of regularization and the dependence on the network complexity.
△ Less
Submitted 12 June, 2024; v1 submitted 4 June, 2022;
originally announced June 2022.
-
Formulating Event-based Image Reconstruction as a Linear Inverse Problem with Deep Regularization using Optical Flow
Authors:
Zelin Zhang,
Anthony Yezzi,
Guillermo Gallego
Abstract:
Event cameras are novel bio-inspired sensors that measure per-pixel brightness differences asynchronously. Recovering brightness from events is appealing since the reconstructed images inherit the high dynamic range (HDR) and high-speed properties of events; hence they can be used in many robotic vision applications and to generate slow-motion HDR videos. However, state-of-the-art methods tackle t…
▽ More
Event cameras are novel bio-inspired sensors that measure per-pixel brightness differences asynchronously. Recovering brightness from events is appealing since the reconstructed images inherit the high dynamic range (HDR) and high-speed properties of events; hence they can be used in many robotic vision applications and to generate slow-motion HDR videos. However, state-of-the-art methods tackle this problem by training an event-to-image Recurrent Neural Network (RNN), which lacks explainability and is difficult to tune. In this work we show, for the first time, how tackling the combined problem of motion and brightness estimation leads us to formulate event-based image reconstruction as a linear inverse problem that can be solved without training an image reconstruction RNN. Instead, classical and learning-based regularizers are used to solve the problem and remove artifacts from the reconstructed images. The experiments show that the proposed approach generates images with visual quality on par with state-of-the-art methods despite only using data from a short time interval. State-of-the-art results are achieved using an image denoising Convolutional Neural Network (CNN) as the regularization function. The proposed regularized formulation and solvers have a unifying character because they can be applied also to reconstruct brightness from the second derivative. Additionally, the formulation is attractive because it can be naturally combined with super-resolution, motion-segmentation and color demosaicing. Code is available at https://github.com/tub-rip/event_based_image_rec_inverse_problem
△ Less
Submitted 12 December, 2022; v1 submitted 12 December, 2021;
originally announced December 2021.
-
Deep Learning 3D Dose Prediction for Conventional Lung IMRT Using Consistent/Unbiased Automated Plans
Authors:
Navdeep Dahiya,
Gourav Jhanwar,
Anthony Yezzi,
Masoud Zarepisheh,
Saad Nadeem
Abstract:
Deep learning (DL) 3D dose prediction has recently gained a lot of attention. However, the variability of plan quality in the training dataset, generated manually by planners with wide range of expertise, can dramatically effect the quality of the final predictions. Moreover, any changes in the clinical criteria requires a new set of manually generated plans by planners to build a new prediction m…
▽ More
Deep learning (DL) 3D dose prediction has recently gained a lot of attention. However, the variability of plan quality in the training dataset, generated manually by planners with wide range of expertise, can dramatically effect the quality of the final predictions. Moreover, any changes in the clinical criteria requires a new set of manually generated plans by planners to build a new prediction model. In this work, we instead use consistent plans generated by our in-house automated planning system (named ``ECHO'') to train the DL model. ECHO (expedited constrained hierarchical optimization) generates consistent/unbiased plans by solving large-scale constrained optimization problems sequentially. If the clinical criteria changes, a new training data set can be easily generated offline using ECHO, with no or limited human intervention, making the DL-based prediction model easily adaptable to the changes in the clinical practice. We used 120 conventional lung patients (100 for training, 20 for testing) with different beam configurations and trained our DL-model using manually-generated as well as automated ECHO plans. We evaluated different inputs: (1) CT+(PTV/OAR)contours, and (2) CT+contours+beam configurations, and different loss functions: (1) MAE (mean absolute error), and (2) MAE+DVH (dose volume histograms). The quality of the predictions was compared using different DVH metrics as well as dose-score and DVH-score, recently introduced by the AAPM knowledge-based planning grand challenge. The best results were obtained using automated ECHO plans and CT+contours+beam as training inputs and MAE+DVH as loss function.
△ Less
Submitted 7 June, 2021;
originally announced June 2021.
-
A Variational Surface-Evolution Perspective for Optimal Transport between Densities with Differing Compact Support
Authors:
Anthony Yezzi
Abstract:
We examine the optimal mass transport problem in $\mathbb{R}^{n}$ between densities having independent compact support by considering the geometry of a continuous interpolating support boundary in space-time within which the mass density evolves according to the fluid dynamical framework of Benamou and Brenier. We treat the geometry of this space--time embedding in terms of points, vectors, and se…
▽ More
We examine the optimal mass transport problem in $\mathbb{R}^{n}$ between densities having independent compact support by considering the geometry of a continuous interpolating support boundary in space-time within which the mass density evolves according to the fluid dynamical framework of Benamou and Brenier. We treat the geometry of this space--time embedding in terms of points, vectors, and sets in $\mathbb{R}^{n+1}\!=\mathbb{R}\times\mathbb{R}^{n}$ and blend the mass density and velocity as well into a space-time solenoidal vector field ${\bf W}\;|\;{\bf Ω\subset}\mathbb{R}^{n+1}\!\to\mathbb{R}^{n+1}$ over compact sets ${\bf Ω}$ . We then formulate a coupled gradient descent approach containing separate evolution steps for $\partial{\bf Ω}$ and ${\bf W}$.
△ Less
Submitted 21 June, 2021; v1 submitted 25 May, 2021;
originally announced May 2021.
-
Intrinsic Integration
Authors:
Navdeep Dahiya,
Martin Mueller,
Anthony Yezzi
Abstract:
If we wish to integrate a function $h|Ω\subset\Re^{n}\to\Re$ along a single $T$-level surface of a function $ψ|Ω\subset\Re^{n}\to\Re$, then a number of different methods for extracting finite elements appropriate to the dimension of the level surface may be employed to obtain an explicit representation over which the integration may be performed using standard numerical quadrature techniques along…
▽ More
If we wish to integrate a function $h|Ω\subset\Re^{n}\to\Re$ along a single $T$-level surface of a function $ψ|Ω\subset\Re^{n}\to\Re$, then a number of different methods for extracting finite elements appropriate to the dimension of the level surface may be employed to obtain an explicit representation over which the integration may be performed using standard numerical quadrature techniques along each element. However, when the goal is to compute an entire continuous family $m(T)$ of integrals over all the $T$-level surfaces of $ψ$, then this method of explicit level set extraction is no longer practical. We introduce a novel method to perform this type of numerical integration efficiently by making use of the coarea formula. We present the technique for discretization of the coarea formula and present the algorithms to compute the integrals over families of T-level surfaces. While validation of our method in the special case of a single level surface demonstrates accuracies close to more explicit isosurface integration methods, we show a sizable boost in computational efficiency in the case of multiple T-level surfaces, where our coupled integration algorithms significantly outperform sequential one-at-a-time application of explicit methods.
△ Less
Submitted 30 March, 2021; v1 submitted 27 March, 2021;
originally announced March 2021.
-
An Efficiently Coupled Shape and Appearance Prior for Active Contour Segmentation
Authors:
Martin Mueller,
Navdeep Dahiya,
Anthony Yezzi
Abstract:
This paper proposes a novel training model based on shape and appearance features for object segmentation in images and videos. Whereas most such models rely on two-dimensional appearance templates or a finite set of descriptors, our appearance-based feature is a one-dimensional function, which is efficiently coupled with the object's shape by integrating intensities along the object's iso-contour…
▽ More
This paper proposes a novel training model based on shape and appearance features for object segmentation in images and videos. Whereas most such models rely on two-dimensional appearance templates or a finite set of descriptors, our appearance-based feature is a one-dimensional function, which is efficiently coupled with the object's shape by integrating intensities along the object's iso-contours. Joint PCA training on these shape and appearance features further exploits shape-appearance correlations and the resulting training model is incorporated in an active-contour-type energy functional for recognition-segmentation tasks. Experiments on synthetic and infrared images demonstrate how this shape and appearance training model improves accuracy compared to methods based on the Chan-Vese energy.
△ Less
Submitted 30 March, 2021; v1 submitted 27 March, 2021;
originally announced March 2021.
-
Multitask 3D CBCT-to-CT Translation and Organs-at-Risk Segmentation Using Physics-Based Data Augmentation
Authors:
Navdeep Dahiya,
Sadegh R Alam,
Pengpeng Zhang,
Si-Yuan Zhang,
Anthony Yezzi,
Saad Nadeem
Abstract:
In current clinical practice, noisy and artifact-ridden weekly cone-beam computed tomography (CBCT) images are only used for patient setup during radiotherapy. Treatment planning is done once at the beginning of the treatment using high-quality planning CT (pCT) images and manual contours for organs-at-risk (OARs) structures. If the quality of the weekly CBCT images can be improved while simultane…
▽ More
In current clinical practice, noisy and artifact-ridden weekly cone-beam computed tomography (CBCT) images are only used for patient setup during radiotherapy. Treatment planning is done once at the beginning of the treatment using high-quality planning CT (pCT) images and manual contours for organs-at-risk (OARs) structures. If the quality of the weekly CBCT images can be improved while simultaneously segmenting OAR structures, this can provide critical information for adapting radiotherapy mid-treatment as well as for deriving biomarkers for treatment response. Using a novel physics-based data augmentation strategy, we synthesize a large dataset of perfectly/inherently registered planning CT and synthetic-CBCT pairs for locally advanced lung cancer patient cohort, which are then used in a multitask 3D deep learning framework to simultaneously segment and translate real weekly CBCT images to high-quality planning CT-like images. We compared the synthetic CT and OAR segmentations generated by the model to real planning CT and manual OAR segmentations and showed promising results. The real week 1 (baseline) CBCT images which had an average MAE of 162.77 HU compared to pCT images are translated to synthetic CT images that exhibit a drastically improved average MAE of 29.31 HU and average structural similarity of 92% with the pCT images. The average DICE scores of the 3D organs-at-risk segmentations are: lungs 0.96, heart 0.88, spinal cord 0.83 and esophagus 0.66. This approach could allow clinicians to adjust treatment plans using only the routine low-quality CBCT images, potentially improving patient outcomes. Our code, data, and pre-trained models will be made available via our physics-based data augmentation library, Physics-ArX, at https://github.com/nadeemlab/Physics-ArX.
△ Less
Submitted 30 August, 2021; v1 submitted 9 March, 2021;
originally announced March 2021.
-
Verifying the Causes of Adversarial Examples
Authors:
Honglin Li,
Yifei Fan,
Frieder Ganz,
Anthony Yezzi,
Payam Barnaghi
Abstract:
The robustness of neural networks is challenged by adversarial examples that contain almost imperceptible perturbations to inputs, which mislead a classifier to incorrect outputs in high confidence. Limited by the extreme difficulty in examining a high-dimensional image space thoroughly, research on explaining and justifying the causes of adversarial examples falls behind studies on attacks and de…
▽ More
The robustness of neural networks is challenged by adversarial examples that contain almost imperceptible perturbations to inputs, which mislead a classifier to incorrect outputs in high confidence. Limited by the extreme difficulty in examining a high-dimensional image space thoroughly, research on explaining and justifying the causes of adversarial examples falls behind studies on attacks and defenses. In this paper, we present a collection of potential causes of adversarial examples and verify (or partially verify) them through carefully-designed controlled experiments. The major causes of adversarial examples include model linearity, one-sum constraint, and geometry of the categories. To control the effect of those causes, multiple techniques are applied such as $L_2$ normalization, replacement of loss functions, construction of reference datasets, and novel models using multi-layer perceptron probabilistic neural networks (MLP-PNN) and density estimation (DE). Our experiment results show that geometric factors tend to be more direct causes and statistical factors magnify the phenomenon, especially for assigning high prediction confidence. We believe this paper will inspire more studies to rigorously investigate the root causes of adversarial examples, which in turn provide useful guidance on designing more robust models.
△ Less
Submitted 19 October, 2020;
originally announced October 2020.
-
An Adaptive View of Adversarial Robustness from Test-time Smoothing Defense
Authors:
Chao Tang,
Yifei Fan,
Anthony Yezzi
Abstract:
The safety and robustness of learning-based decision-making systems are under threats from adversarial examples, as imperceptible perturbations can mislead neural networks to completely different outputs. In this paper, we present an adaptive view of the issue via evaluating various test-time smoothing defense against white-box untargeted adversarial examples. Through controlled experiments with p…
▽ More
The safety and robustness of learning-based decision-making systems are under threats from adversarial examples, as imperceptible perturbations can mislead neural networks to completely different outputs. In this paper, we present an adaptive view of the issue via evaluating various test-time smoothing defense against white-box untargeted adversarial examples. Through controlled experiments with pretrained ResNet-152 on ImageNet, we first illustrate the non-monotonic relation between adversarial attacks and smoothing defenses. Then at the dataset level, we observe large variance among samples and show that it is easy to inflate accuracy (even to 100%) or build large-scale (i.e., with size ~10^4) subsets on which a designated method outperforms others by a large margin. Finally at the sample level, as different adversarial examples require different degrees of defense, the potential advantages of iterative methods are also discussed. We hope this paper reveal useful behaviors of test-time defenses, which could help improve the evaluation process for adversarial robustness in the future.
△ Less
Submitted 26 November, 2019;
originally announced November 2019.
-
An Interactive Control Approach to 3D Shape Reconstruction
Authors:
Bipul Islam,
Ji Liu,
Anthony Yezzi,
Romeil Sandhu
Abstract:
The ability to accurately reconstruct the 3D facets of a scene is one of the key problems in robotic vision. However, even with recent advances with machine learning, there is no high-fidelity universal 3D reconstruction method for this optimization problem as schemes often cater to specific image modalities and are often biased by scene abnormalities. Simply put, there always remains an informati…
▽ More
The ability to accurately reconstruct the 3D facets of a scene is one of the key problems in robotic vision. However, even with recent advances with machine learning, there is no high-fidelity universal 3D reconstruction method for this optimization problem as schemes often cater to specific image modalities and are often biased by scene abnormalities. Simply put, there always remains an information gap due to the dynamic nature of real-world scenarios. To this end, we demonstrate a feedback control framework which invokes operator inputs (also prone to errors) in order to augment existing reconstruction schemes. For proof-of-concept, we choose a classical region-based stereoscopic reconstruction approach and show how an ill-posed model can be augmented with operator input to be much more robust to scene artifacts. We provide necessary conditions for stability via Lyapunov analysis and perhaps more importantly, we show that the stability depends on a notion of absolute curvature. Mathematically, this aligns with previous work that has shown Ricci curvature as proxy for functional robustness of dynamical networked systems. We conclude with results that show how our method can improve standalone reconstruction schemes.
△ Less
Submitted 7 October, 2019;
originally announced October 2019.
-
PDE Acceleration: A convergence rate analysis and applications to obstacle problems
Authors:
Jeff Calder,
Anthony Yezzi
Abstract:
This paper provides a rigorous convergence rate and complexity analysis for a recently introduced framework, called PDE acceleration, for solving problems in the calculus of variations, and explores applications to obstacle problems. PDE acceleration grew out of a variational interpretation of momentum methods, such as Nesterov's accelerated gradient method and Polyak's heavy ball method, that vie…
▽ More
This paper provides a rigorous convergence rate and complexity analysis for a recently introduced framework, called PDE acceleration, for solving problems in the calculus of variations, and explores applications to obstacle problems. PDE acceleration grew out of a variational interpretation of momentum methods, such as Nesterov's accelerated gradient method and Polyak's heavy ball method, that views acceleration methods as equations of motion for a generalized Lagrangian action. Its application to convex variational problems yields equations of motion in the form of a damped nonlinear wave equation rather than nonlinear diffusion arising from gradient descent. These accelerated PDE's can be efficiently solved with simple explicit finite difference schemes where acceleration is realized by an improvement in the CFL condition from $dt\sim dx^2$ for diffusion equations to $dt\sim dx$ for wave equations. In this paper, we prove a linear convergence rate for PDE acceleration for strongly convex problems, provide a complexity analysis of the discrete scheme, and show how to optimally select the dam** parameter for linear problems. We then apply PDE acceleration to solve minimal surface obstacle problems, including double obstacles with forcing, and stochastic homogenization problems with obstacles, obtaining state of the art computational results.
△ Less
Submitted 29 July, 2019; v1 submitted 2 October, 2018;
originally announced October 2018.
-
Accelerated PDE's for efficient solution of regularized inversion problems
Authors:
Minas Benyamin,
Jeff Calder,
Ganesh Sundaramoorthi,
Anthony Yezzi
Abstract:
We further develop a new framework, called PDE Acceleration, by applying it to calculus of variations problems defined for general functions on $\mathbb{R}^n$, obtaining efficient numerical algorithms to solve the resulting class of optimization problems based on simple discretizations of their corresponding accelerated PDE's. While the resulting family of PDE's and numerical schemes are quite gen…
▽ More
We further develop a new framework, called PDE Acceleration, by applying it to calculus of variations problems defined for general functions on $\mathbb{R}^n$, obtaining efficient numerical algorithms to solve the resulting class of optimization problems based on simple discretizations of their corresponding accelerated PDE's. While the resulting family of PDE's and numerical schemes are quite general, we give special attention to their application for regularized inversion problems, with particular illustrative examples on some popular image processing applications. The method is a generalization of momentum, or accelerated, gradient descent to the PDE setting. For elliptic problems, the descent equations are a nonlinear damped wave equation, instead of a diffusion equation, and the acceleration is realized as an improvement in the CFL condition from $Δt\sim Δx^{2}$ (for diffusion) to $Δt\sim Δx$ (for wave equations). We work out several explicit as well as a semi-implicit numerical schemes, together with their necessary stability constraints, and include recursive update formulations which allow minimal-effort adaptation of existing gradient descent PDE codes into the accelerated PDE framework. We explore these schemes more carefully for a broad class of regularized inversion applications, with special attention to quadratic, Beltrami, and Total Variation regularization, where the accelerated PDE takes the form of a nonlinear wave equation. Experimental examples demonstrate the application of these schemes for image denoising, deblurring, and inpainting, including comparisons against Primal Dual, Split Bregman, and ADMM algorithms.
△ Less
Submitted 30 September, 2018;
originally announced October 2018.
-
Accelerated Optimization in the PDE Framework: Formulations for the Manifold of Diffeomorphisms
Authors:
Ganesh Sundaramoorthi,
Anthony Yezzi
Abstract:
We consider the problem of optimization of cost functionals on the infinite-dimensional manifold of diffeomorphisms. We present a new class of optimization methods, valid for any optimization problem setup on the space of diffeomorphisms by generalizing Nesterov accelerated optimization to the manifold of diffeomorphisms. While our framework is general for infinite dimensional manifolds, we specif…
▽ More
We consider the problem of optimization of cost functionals on the infinite-dimensional manifold of diffeomorphisms. We present a new class of optimization methods, valid for any optimization problem setup on the space of diffeomorphisms by generalizing Nesterov accelerated optimization to the manifold of diffeomorphisms. While our framework is general for infinite dimensional manifolds, we specifically treat the case of diffeomorphisms, motivated by optical flow problems in computer vision. This is accomplished by building on a recent variational approach to a general class of accelerated optimization methods by Wibisono, Wilson and Jordan, which applies in finite dimensions. We generalize that approach to infinite dimensional manifolds. We derive the surprisingly simple continuum evolution equations, which are partial differential equations, for accelerated gradient descent, and relate it to simple mechanical principles from fluid mechanics. Our approach has natural connections to the optimal mass transport problem. This is because one can think of our approach as an evolution of an infinite number of particles endowed with mass (represented with a mass density) that moves in an energy landscape. The mass evolves with the optimization variable, and endows the particles with dynamics. This is different than the finite dimensional case where only a single particle moves and hence the dynamics does not depend on the mass. We derive the theory, compute the PDEs for accelerated optimization, and illustrate the behavior of these new accelerated optimization schemes.
△ Less
Submitted 23 May, 2018; v1 submitted 4 April, 2018;
originally announced April 2018.
-
Towards an Understanding of Neural Networks in Natural-Image Spaces
Authors:
Yifei Fan,
Anthony Yezzi
Abstract:
Two major uncertainties, dataset bias and adversarial examples, prevail in state-of-the-art AI algorithms with deep neural networks. In this paper, we present an intuitive explanation for these issues as well as an interpretation of the performance of deep networks in a natural-image space. The explanation consists of two parts: the philosophy of neural networks and a hypothetical model of natural…
▽ More
Two major uncertainties, dataset bias and adversarial examples, prevail in state-of-the-art AI algorithms with deep neural networks. In this paper, we present an intuitive explanation for these issues as well as an interpretation of the performance of deep networks in a natural-image space. The explanation consists of two parts: the philosophy of neural networks and a hypothetical model of natural-image spaces. Following the explanation, we 1) demonstrate that the values of training samples differ, 2) provide incremental boost to the accuracy of a CIFAR-10 classifier by introducing an additional "random-noise" category during training, 3) alleviate over-fitting thereby enhancing the robustness against adversarial examples by detecting and excluding illusive training samples that are consistently misclassified. Our overall contribution is therefore twofold. First, while most existing algorithms treat data equally and have a strong appetite for more data, we demonstrate in contrast that an individual datum can sometimes have disproportionate and counterproductive influence and that it is not always better to train neural networks with more data. Next, we consider more thoughtful strategies by taking into account the geometric and topological properties of natural-image spaces to which deep networks are applied.
△ Less
Submitted 11 February, 2019; v1 submitted 27 January, 2018;
originally announced January 2018.
-
Accelerated Optimization in the PDE Framework: Formulations for the Active Contour Case
Authors:
Anthony Yezzi,
Ganesh Sundaramoorthi
Abstract:
Following the seminal work of Nesterov, accelerated optimization methods have been used to powerfully boost the performance of first-order, gradient-based parameter estimation in scenarios where second-order optimization strategies are either inapplicable or impractical. Not only does accelerated gradient descent converge considerably faster than traditional gradient descent, but it also performs…
▽ More
Following the seminal work of Nesterov, accelerated optimization methods have been used to powerfully boost the performance of first-order, gradient-based parameter estimation in scenarios where second-order optimization strategies are either inapplicable or impractical. Not only does accelerated gradient descent converge considerably faster than traditional gradient descent, but it also performs a more robust local search of the parameter space by initially overshooting and then oscillating back as it settles into a final configuration, thereby selecting only local minimizers with a basis of attraction large enough to contain the initial overshoot. This behavior has made accelerated and stochastic gradient search methods particularly popular within the machine learning community. In their recent PNAS 2016 paper, Wibisono, Wilson, and Jordan demonstrate how a broad class of accelerated schemes can be cast in a variational framework formulated around the Bregman divergence, leading to continuum limit ODE's. We show how their formulation may be further extended to infinite dimension manifolds (starting here with the geometric space of curves and surfaces) by substituting the Bregman divergence with inner products on the tangent space and explicitly introducing a distributed mass model which evolves in conjunction with the object of interest during the optimization process. The co-evolving mass model, which is introduced purely for the sake of endowing the optimization with helpful dynamics, also links the resulting class of accelerated PDE based optimization schemes to fluid dynamical formulations of optimal mass transport.
△ Less
Submitted 27 November, 2017;
originally announced November 2017.
-
Tracking via Motion Estimation with Physically Motivated Inter-Region Constraints
Authors:
Omar Arif,
Ganesh Sundaramoorthi,
Byung-Woo Hong,
Anthony Yezzi
Abstract:
In this paper, we propose a method for tracking structures (e.g., ventricles and myocardium) in cardiac images (e.g., magnetic resonance) by propagating forward in time a previous estimate of the structures via a new deformation estimation scheme that is motivated by physical constraints of fluid motion. The method employs within structure motion estimation (so that differing motions among differe…
▽ More
In this paper, we propose a method for tracking structures (e.g., ventricles and myocardium) in cardiac images (e.g., magnetic resonance) by propagating forward in time a previous estimate of the structures via a new deformation estimation scheme that is motivated by physical constraints of fluid motion. The method employs within structure motion estimation (so that differing motions among different structures are not mixed) while simultaneously satisfying the physical constraint in fluid motion that at the interface between a fluid and a medium, the normal component of the fluid's motion must match the normal component of the motion of the medium. We show how to estimate the motion according to the previous considerations in a variational framework, and in particular, show that these conditions lead to PDEs with boundary conditions at the interface that resemble Robin boundary conditions and induce coupling between structures. We illustrate the use of this motion estimation scheme in propagating a segmentation across frames and show that it leads to more accurate segmentation than traditional motion estimation that does not make use of physical constraints. Further, the method is naturally suited to interactive segmentation methods, which are prominently used in practice in commercial applications for cardiac analysis, where typically a segmentation from the previous frame is used to predict a segmentation in the next frame. We show that our propagation scheme reduces the amount of user interaction by predicting more accurate segmentations than commonly used and recent interactive commercial techniques.
△ Less
Submitted 6 February, 2014;
originally announced February 2014.
-
A compact formula for the derivative of a 3-D rotation in exponential coordinates
Authors:
Guillermo Gallego,
Anthony Yezzi
Abstract:
We present a compact formula for the derivative of a 3-D rotation matrix with respect to its exponential coordinates. A geometric interpretation of the resulting expression is provided, as well as its agreement with other less-compact but better-known formulas. To the best of our knowledge, this simpler formula does not appear anywhere in the literature. We hope by providing this more compact expr…
▽ More
We present a compact formula for the derivative of a 3-D rotation matrix with respect to its exponential coordinates. A geometric interpretation of the resulting expression is provided, as well as its agreement with other less-compact but better-known formulas. To the best of our knowledge, this simpler formula does not appear anywhere in the literature. We hope by providing this more compact expression to alleviate the common pressure to reluctantly resort to alternative representations in various computational applications simply as a means to avoid the complexity of differential analysis in exponential coordinates.
△ Less
Submitted 8 August, 2014; v1 submitted 3 December, 2013;
originally announced December 2013.
-
Sobolev--type metrics in the space of curves
Authors:
A. C. G. Mennucci,
A. Yezzi,
G. Sundaramoorthi
Abstract:
We define a manifold $M$ where objects $c\in M$ are curves, which we parameterize as $c:S^1\to R^n$ ($n\ge 2$, $S^1$ is the circle). Given a curve $c$, we define the tangent space $T_cM$ of $M$ at $c$ including in it all deformations $h:S^1\to R^n$ of $c$. In this paper we study geometries on the manifold of curves, provided by Sobolev--type metrics $H^j$. We study $H^j$ type metrics for the cas…
▽ More
We define a manifold $M$ where objects $c\in M$ are curves, which we parameterize as $c:S^1\to R^n$ ($n\ge 2$, $S^1$ is the circle). Given a curve $c$, we define the tangent space $T_cM$ of $M$ at $c$ including in it all deformations $h:S^1\to R^n$ of $c$. In this paper we study geometries on the manifold of curves, provided by Sobolev--type metrics $H^j$. We study $H^j$ type metrics for the cases $j=1,2$; we prove estimates, and characterize the completion of the space of smooth curves. As a bonus, we prove that the Fréchet distance of curves (see arXiv:math.DG/0312384) coincides with the distance induced by the ``Finsler $L^\infinity$ metric'' defined in \S2.2 in arXiv:math.DG/0412454.
△ Less
Submitted 30 April, 2006;
originally announced May 2006.
-
Metrics in the space of curves
Authors:
A. Yezzi,
A. Mennucci
Abstract:
In this paper we study geometries on the manifold of curves.
We define a manifold $M$ where objects $c\in M$ are curves, which we parameterize as $c:S^1\to \real^n$ ($n\ge 2$, $S^1$ is the circle). Given a curve $c$, we define the tangent space $T_cM$ of $M$ at $c$ including in it all deformations $h:S^1\to\real^n$ of $c$.
We discuss Riemannian and Finsler metrics $F(c,h)$ on this manifold…
▽ More
In this paper we study geometries on the manifold of curves.
We define a manifold $M$ where objects $c\in M$ are curves, which we parameterize as $c:S^1\to \real^n$ ($n\ge 2$, $S^1$ is the circle). Given a curve $c$, we define the tangent space $T_cM$ of $M$ at $c$ including in it all deformations $h:S^1\to\real^n$ of $c$.
We discuss Riemannian and Finsler metrics $F(c,h)$ on this manifold $M$, and in particular the case of the geometric $H^0$ metric $F(c,h)=\int |h|^2ds$ of normal deformations $h$ of $c$; we study the existence of minimal geodesics of $H^0$ under constraints; we moreover propose a conformal version of the $H^0$ metric.
△ Less
Submitted 25 May, 2005; v1 submitted 22 December, 2004;
originally announced December 2004.