Showing 1–35 of 35 results for author: Duits, R

Semiring Activation in Neural Networks

Authors: Bart M. N. Smets, Peter D. Donker, Jim W. Portegies, Remco Duits

Abstract: We introduce a class of trainable nonlinear operators based on semirings that are suitable for use in neural networks. These operators generalize the traditional alternation of linear operators with activation functions in neural networks. Semirings are algebraic structures that describe a generalised notation of linearity, greatly expanding the range of trainable operators that can be included in… ▽ More We introduce a class of trainable nonlinear operators based on semirings that are suitable for use in neural networks. These operators generalize the traditional alternation of linear operators with activation functions in neural networks. Semirings are algebraic structures that describe a generalised notation of linearity, greatly expanding the range of trainable operators that can be included in neural networks. In fact, max- or min-pooling operations are convolutions in the tropical semiring with a fixed kernel. We perform experiments where we replace the activation functions for trainable semiring-based operators to show that these are viable operations to include in fully connected as well as convolutional neural networks (ConvNeXt). We discuss some of the challenges of replacing traditional activation functions with trainable semiring activations and the trade-offs of doing so. △ Less

Submitted 5 July, 2024; v1 submitted 29 May, 2024; originally announced May 2024.

Segmentation tool for images of cracks

Authors: Andrii Kompanets, Remco Duits, Davide Leonetti, Nicky van den Berg, H. H., Snijder

Abstract: Safety-critical infrastructures, such as bridges, are periodically inspected to check for existing damage, such as fatigue cracks and corrosion, and to guarantee the safe use of the infrastructure. Visual inspection is the most frequent type of general inspection, despite the fact that its detection capability is rather limited, especially for fatigue cracks. Machine learning algorithms can be use… ▽ More Safety-critical infrastructures, such as bridges, are periodically inspected to check for existing damage, such as fatigue cracks and corrosion, and to guarantee the safe use of the infrastructure. Visual inspection is the most frequent type of general inspection, despite the fact that its detection capability is rather limited, especially for fatigue cracks. Machine learning algorithms can be used for augmenting the capability of classical visual inspection of bridge structures, however, the implementation of such an algorithm requires a massive annotated training dataset, which is time-consuming to produce. This paper proposes a semi-automatic crack segmentation tool that eases the manual segmentation of cracks on images needed to create a training dataset for a machine learning algorithm. Also, it can be used to measure the geometry of the crack. This tool makes use of an image processing algorithm, which was initially developed for the analysis of vascular systems on retinal images. The algorithm relies on a multi-orientation wavelet transform, which is applied to the image to construct the so-called "orientation scores", i.e. a modified version of the image. Afterwards, the filtered orientation scores are used to formulate an optimal path problem that identifies the crack. The globally optimal path between manually selected crack endpoints is computed, using a state-of-the-art geometric tracking method. The pixel-wise segmentation is done afterwards using the obtained crack path. The proposed method outperforms fully automatic methods and shows potential to be an adequate alternative to the manual data annotation. △ Less

Submitted 28 March, 2024; originally announced March 2024.

Deep Learning for Segmentation of Cracks in High-Resolution Images of Steel Bridges

Authors: Andrii Kompanets, Gautam Pai, Remco Duits, Davide Leonetti, Bert Snijder

Abstract: Automating the current bridge visual inspection practices using drones and image processing techniques is a prominent way to make these inspections more effective, robust, and less expensive. In this paper, we investigate the development of a novel deep-learning method for the detection of fatigue cracks in high-resolution images of steel bridges. First, we present a novel and challenging dataset… ▽ More Automating the current bridge visual inspection practices using drones and image processing techniques is a prominent way to make these inspections more effective, robust, and less expensive. In this paper, we investigate the development of a novel deep-learning method for the detection of fatigue cracks in high-resolution images of steel bridges. First, we present a novel and challenging dataset comprising of images of cracks in steel bridges. Secondly, we integrate the ConvNext neural network with a previous state-of-the-art encoder-decoder network for crack segmentation. We study and report, the effects of the use of background patches on the network performance when applied to high-resolution images of cracks in steel bridges. Finally, we introduce a loss function that allows the use of more background patches for the training process, which yields a significant reduction in false positive rates. △ Less

Submitted 26 March, 2024; originally announced March 2024.

PDE-CNNs: Axiomatic Derivations and Applications

Authors: Gijs Bellaard, Sei Sakata, Bart M. N. Smets, Remco Duits

Abstract: PDE-based Group Convolutional Neural Networks (PDE-G-CNNs) utilize solvers of geometrically meaningful evolution PDEs as substitutes for the conventional components in G-CNNs. PDE-G-CNNs offer several key benefits all at once: fewer parameters, inherent equivariance, better performance, data efficiency, and geometric interpretability. In this article we focus on Euclidean equivariant PDE-G-CNNs… ▽ More PDE-based Group Convolutional Neural Networks (PDE-G-CNNs) utilize solvers of geometrically meaningful evolution PDEs as substitutes for the conventional components in G-CNNs. PDE-G-CNNs offer several key benefits all at once: fewer parameters, inherent equivariance, better performance, data efficiency, and geometric interpretability. In this article we focus on Euclidean equivariant PDE-G-CNNs where the feature maps are two dimensional throughout. We call this variant of the framework a PDE-CNN. From a machine learning perspective, we list several practically desirable axioms and derive from these which PDEs should be used in a PDE-CNN. Here our approach to geometric learning via PDEs is inspired by the axioms of classical linear and morphological scale-space theory, which we generalize by introducing semifield-valued signals. Furthermore, we experimentally confirm for small networks that PDE-CNNs offer fewer parameters, increased performance, and better data efficiency when compared to CNNs. We also investigate what effect the use of different semifields has on the performance of the models. △ Less

Submitted 18 April, 2024; v1 submitted 22 March, 2024; originally announced March 2024.

Optimal Transport on the Lie Group of Roto-translations

Authors: Daan Bon, Gautam Pai, Gijs Bellaard, Olga Mula, Remco Duits

Abstract: The roto-translation group SE2 has been of active interest in image analysis due to methods that lift the image data to multi-orientation representations defined on this Lie group. This has led to impactful applications of crossing-preserving flows for image de-noising, geodesic tracking, and roto-translation equivariant deep learning. In this paper, we develop a computational framework for optima… ▽ More The roto-translation group SE2 has been of active interest in image analysis due to methods that lift the image data to multi-orientation representations defined on this Lie group. This has led to impactful applications of crossing-preserving flows for image de-noising, geodesic tracking, and roto-translation equivariant deep learning. In this paper, we develop a computational framework for optimal transportation over Lie groups, with a special focus on SE2. We make several theoretical contributions (generalizable to matrix Lie groups) such as the non-optimality of group actions as transport maps, invariance and equivariance of optimal transport, and the quality of the entropic-regularized optimal transport plan using geodesic distance approximations. We develop a Sinkhorn like algorithm that can be efficiently implemented using fast and accurate distance approximations of the Lie group and GPU-friendly group convolutions. We report valuable advancements in the experiments on 1) image barycentric interpolation, 2) interpolation of planar orientation fields, and 3) Wasserstein gradient flows on SE2. We observe that our framework of lifting images to SE2 and optimal transport with left-invariant anisotropic metrics leads to equivariant transport along dominant contours and salient line structures in the image. This yields sharper and more meaningful interpolations compared to their counterparts on R^2 △ Less

Submitted 5 March, 2024; v1 submitted 23 February, 2024; originally announced February 2024.

MSC Class: 62H35; 68U10; 90B06; 68T45; 68U99

Analysis of (sub-)Riemannian PDE-G-CNNs

Authors: Gijs Bellaard, Daan L. J. Bon, Gautam Pai, Bart M. N. Smets, Remco Duits

Abstract: Group equivariant convolutional neural networks (G-CNNs) have been successfully applied in geometric deep learning. Typically, G-CNNs have the advantage over CNNs that they do not waste network capacity on training symmetries that should have been hard-coded in the network. The recently introduced framework of PDE-based G-CNNs (PDE-G-CNNs) generalises G-CNNs. PDE-G-CNNs have the core advantages th… ▽ More Group equivariant convolutional neural networks (G-CNNs) have been successfully applied in geometric deep learning. Typically, G-CNNs have the advantage over CNNs that they do not waste network capacity on training symmetries that should have been hard-coded in the network. The recently introduced framework of PDE-based G-CNNs (PDE-G-CNNs) generalises G-CNNs. PDE-G-CNNs have the core advantages that they simultaneously 1) reduce network complexity, 2) increase classification performance, and 3) provide geometric interpretability. Their implementations primarily consist of linear and morphological convolutions with kernels. In this paper we show that the previously suggested approximative morphological kernels do not always accurately approximate the exact kernels accurately. More specifically, depending on the spatial anisotropy of the Riemannian metric, we argue that one must resort to sub-Riemannian approximations. We solve this problem by providing a new approximative kernel that works regardless of the anisotropy. We provide new theorems with better error estimates of the approximative kernels, and prove that they all carry the same reflectional symmetries as the exact ones. We test the effectiveness of multiple approximative kernels within the PDE-G-CNN framework on two datasets, and observe an improvement with the new approximative kernels. We report that the PDE-G-CNNs again allow for a considerable reduction of network complexity while having comparable or better performance than G-CNNs and CNNs on the two datasets. Moreover, PDE-G-CNNs have the advantage of better geometric interpretability over G-CNNs, as the morphological kernels are related to association fields from neurogeometry. △ Less

Submitted 3 April, 2023; v1 submitted 3 October, 2022; originally announced October 2022.

Comments: 29 pages, 21 figures

Geodesic Tracking via New Data-driven Connections of Cartan Type for Vascular Tree Tracking

Authors: Nicky van den Berg, Bart Smets, Gautam Pai, Jean-Marie Mirebeau, Remco Duits

Abstract: We introduce a data-driven version of the plus Cartan connection on the homogeneous space $\mathbb{M}_2$ of 2D positions and orientations. We formulate a theorem that describes all shortest and straight curves (parallel velocity and parallel momentum, respectively) with respect to this new data-driven connection and corresponding Riemannian manifold. Then we use these shortest curves for geodesic… ▽ More We introduce a data-driven version of the plus Cartan connection on the homogeneous space $\mathbb{M}_2$ of 2D positions and orientations. We formulate a theorem that describes all shortest and straight curves (parallel velocity and parallel momentum, respectively) with respect to this new data-driven connection and corresponding Riemannian manifold. Then we use these shortest curves for geodesic tracking of complex vasculature in multi-orientation image representations defined on $\mathbb{M}_{2}$. The data-driven Cartan connection characterizes the Hamiltonian flow of all geodesics. It also allows for improved adaptation to curvature and misalignment of the (lifted) vessel structure that we track via globally optimal geodesics. We compute these geodesics numerically via steepest descent on distance maps on $\mathbb{M}_2$ that we compute by a new modified anisotropic fast-marching method. Our experiments range from tracking single blood vessels with fixed endpoints to tracking complete vascular trees in retinal images. Single vessel tracking is performed in a single run in the multi-orientation image representation, where we project the resulting geodesics back onto the underlying image. The complete vascular tree tracking requires only two runs and avoids prior segmentation, placement of extra anchor points, and dynamic switching between geodesic models. Altogether we provide a geodesic tracking method using a single, flexible, transparent, data-driven geodesic model providing globally optimal curves which correctly follow highly complex vascular structures in retinal images. All experiments in this article can be reproduced via documented Mathematica notebooks available at GitHub (https://github.com/NickyvdBerg/DataDrivenTracking). △ Less

Submitted 8 December, 2023; v1 submitted 23 August, 2022; originally announced August 2022.

Roto-Translation Equivariant Convolutional Networks: Application to Histopathology Image Analysis

Authors: Maxime W. Lafarge, Erik J. Bekkers, Josien P. W. Pluim, Remco Duits, Mitko Veta

Abstract: Rotation-invariance is a desired property of machine-learning models for medical image analysis and in particular for computational pathology applications. We propose a framework to encode the geometric structure of the special Euclidean motion group SE(2) in convolutional networks to yield translation and rotation equivariance via the introduction of SE(2)-group convolution layers. This structure… ▽ More Rotation-invariance is a desired property of machine-learning models for medical image analysis and in particular for computational pathology applications. We propose a framework to encode the geometric structure of the special Euclidean motion group SE(2) in convolutional networks to yield translation and rotation equivariance via the introduction of SE(2)-group convolution layers. This structure enables models to learn feature representations with a discretized orientation dimension that guarantees that their outputs are invariant under a discrete set of rotations. Conventional approaches for rotation invariance rely mostly on data augmentation, but this does not guarantee the robustness of the output when the input is rotated. At that, trained conventional CNNs may require test-time rotation augmentation to reach their full capability. This study is focused on histopathology image analysis applications for which it is desirable that the arbitrary global orientation information of the imaged tissues is not captured by the machine learning models. The proposed framework is evaluated on three different histopathology image analysis tasks (mitosis detection, nuclei segmentation and tumor classification). We present a comparative analysis for each problem and show that consistent increase of performances can be achieved when using the proposed framework. △ Less

Submitted 20 February, 2020; originally announced February 2020.

PDE-based Group Equivariant Convolutional Neural Networks

Authors: Bart Smets, Jim Portegies, Erik Bekkers, Remco Duits

Abstract: We present a PDE-based framework that generalizes Group equivariant Convolutional Neural Networks (G-CNNs). In this framework, a network layer is seen as a set of PDE-solvers where geometrically meaningful PDE-coefficients become the layer's trainable weights. Formulating our PDEs on homogeneous spaces allows these networks to be designed with built-in symmetries such as rotation in addition to th… ▽ More We present a PDE-based framework that generalizes Group equivariant Convolutional Neural Networks (G-CNNs). In this framework, a network layer is seen as a set of PDE-solvers where geometrically meaningful PDE-coefficients become the layer's trainable weights. Formulating our PDEs on homogeneous spaces allows these networks to be designed with built-in symmetries such as rotation in addition to the standard translation equivariance of CNNs. Having all the desired symmetries included in the design obviates the need to include them by means of costly techniques such as data augmentation. We will discuss our PDE-based G-CNNs (PDE-G-CNNs) in a general homogeneous space setting while also going into the specifics of our primary case of interest: roto-translation equivariance. We solve the PDE of interest by a combination of linear group convolutions and non-linear morphological group convolutions with analytic kernel approximations that we underpin with formal theorems. Our kernel approximations allow for fast GPU-implementation of the PDE-solvers, we release our implementation with this article in the form of the LieTorch extension to PyTorch, available at https://gitlab.com/bsmetsjr/lietorch . Just like for linear convolution a morphological convolution is specified by a kernel that we train in our PDE-G-CNNs. In PDE-G-CNNs we do not use non-linearities such as max/min-pooling and ReLUs as they are already subsumed by morphological convolutions. We present a set of experiments to demonstrate the strength of the proposed PDE-G-CNNs in increasing the performance of deep learning based imaging applications with far fewer parameters than traditional CNNs. △ Less

Submitted 30 May, 2022; v1 submitted 24 January, 2020; originally announced January 2020.

Comments: 27 pages, 18 figures. v2 changes: - mentioned KerCNNs - added section Generalization of G-CNNs - clarification that the experiments utilized automatic differentiation and SGD. v3 changes: - streamlined theoretical framework - formulation and proof Thm.1 & 2 - expanded experiments. v4 changes: typos in Prop.5 and (20) v5/6 changes: minor revision

Lesion Segmentation in Ultrasound Using Semi-pixel-wise Cycle Generative Adversarial Nets

Authors: Jie Xing, Zheren Li, Biyuan Wang, Yuji Qi, Bingbin Yu, Farhad G. Zanjani, Aiwen Zheng, Remco Duits, Tao Tan

Abstract: Breast cancer is the most common invasive cancer with the highest cancer occurrence in females. Handheld ultrasound is one of the most efficient ways to identify and diagnose the breast cancer. The area and the shape information of a lesion is very helpful for clinicians to make diagnostic decisions. In this study we propose a new deep-learning scheme, semi-pixel-wise cycle generative adversarial… ▽ More Breast cancer is the most common invasive cancer with the highest cancer occurrence in females. Handheld ultrasound is one of the most efficient ways to identify and diagnose the breast cancer. The area and the shape information of a lesion is very helpful for clinicians to make diagnostic decisions. In this study we propose a new deep-learning scheme, semi-pixel-wise cycle generative adversarial net (SPCGAN) for segmenting the lesion in 2D ultrasound. The method takes the advantage of a fully convolutional neural network (FCN) and a generative adversarial net to segment a lesion by using prior knowledge. We compared the proposed method to a fully connected neural network and the level set segmentation method on a test dataset consisting of 32 malignant lesions and 109 benign lesions. Our proposed method achieved a Dice similarity coefficient (DSC) of 0.92 while FCN and the level set achieved 0.90 and 0.79 respectively. Particularly, for malignant lesions, our method increases the DSC (0.90) of the fully connected neural network to 0.93 significantly (p$<$0.001). The results show that our SPCGAN can obtain robust segmentation results. The framework of SPCGAN is particularly effective when sufficient training samples are not available compared to FCN. Our proposed method may be used to relieve the radiologists' burden for annotation. △ Less

Submitted 17 October, 2020; v1 submitted 6 May, 2019; originally announced May 2019.

Journal ref: IEEE/ACM Transactions on Computational Biology and Bioinformatics, 04 March 2020, pp.1-1

Total Variation and Mean Curvature PDEs on $\mathbb{R}^d \rtimes S^{d-1}$

Authors: Remco Duits, Etienne St-Onge, Jim Portegies, Bart Smets

Abstract: Total variation regularization and total variation flows (TVF) have been widely applied for image enhancement and denoising. To include a generic preservation of crossing curvilinear structures in TVF we lift images to the homogeneous space $M = \mathbb{R}^d \rtimes S^{d-1}$ of positions and orientations as a Lie group quotient in SE(d). For d = 2 this is called 'total roto-translation variation'… ▽ More Total variation regularization and total variation flows (TVF) have been widely applied for image enhancement and denoising. To include a generic preservation of crossing curvilinear structures in TVF we lift images to the homogeneous space $M = \mathbb{R}^d \rtimes S^{d-1}$ of positions and orientations as a Lie group quotient in SE(d). For d = 2 this is called 'total roto-translation variation' by Chambolle & Pock. We extend this to d = 3, by a PDE-approach with a limiting procedure for which we prove convergence. We also include a Mean Curvature Flow (MCF) in our PDE model on M. This was first proposed for d = 2 by Citti et al. and we extend this to d = 3. Furthermore, for d = 2 we take advantage of locally optimal differential frames in invertible orientation scores (OS). We apply our TVF and MCF in the denoising/enhancement of crossing fiber bundles in DW-MRI. In comparison to data-driven diffusions, we see a better preservation of bundle boundaries and angular sharpness in fiber orientation densities at crossings. We support this by error comparisons on a noisy DW-MRI phantom. We also apply our TVF and MCF in enhancement of crossing elongated structures in 2D images via OS, and compare the results to nonlinear diffusions (CED-OS) via OS. △ Less

Submitted 29 June, 2019; v1 submitted 21 February, 2019; originally announced February 2019.

Comments: Submission to the Seventh International Conference on Scale Space and Variational Methods in Computer Vision (SSVM 2019). (v2) Typo correction in lemma 1. (v3) Typo correction last paragraph page 9

Fourier Transform on the Homogeneous Space of 3D Positions and Orientations for Exact Solutions to Linear Parabolic and (Hypo-)Elliptic PDEs

Authors: Remco Duits, Erik J. Bekkers, Alexey Mashtakov

Abstract: Fokker-Planck PDEs (incl. diffusions) for stable Lévy processes (incl. Wiener processes) on the joint space of positions and orientations play a major role in mechanics, robotics, image analysis, directional statistics and probability theory. Exact analytic designs and solutions are known in the 2D case, where they have been obtained using Fourier transform on $SE(2)$. Here we extend these approac… ▽ More Fokker-Planck PDEs (incl. diffusions) for stable Lévy processes (incl. Wiener processes) on the joint space of positions and orientations play a major role in mechanics, robotics, image analysis, directional statistics and probability theory. Exact analytic designs and solutions are known in the 2D case, where they have been obtained using Fourier transform on $SE(2)$. Here we extend these approaches to 3D using Fourier transform on the Lie group $SE(3)$ of rigid body motions. More precisely, we define the homogeneous space of 3D positions and orientations $\mathbb{R}^{3}\rtimes S^{2}:=SE(3)/(\{\mathbf{0}\} \times SO(2))$ as the quotient in $SE(3)$. In our construction, two group elements are equivalent if they are equal up to a rotation around the reference axis. On this quotient we design a specific Fourier transform. We apply this Fourier transform to derive new exact solutions to Fokker-Planck PDEs of $α$-stable Lévy processes on $\mathbb{R}^{3}\rtimes S^{2}$. This reduces classical analysis computations and provides an explicit algebraic spectral decomposition of the solutions. We compare the exact probability kernel for $α= 1$ (the diffusion kernel) to the kernel for $α=\frac12$ (the Poisson kernel). We set up SDEs for the Lévy processes on the quotient and derive corresponding Monte-Carlo methods. We verify that the exact probability kernels arise as the limit of the Monte-Carlo approximations. △ Less

Submitted 13 December, 2018; v1 submitted 1 November, 2018; originally announced November 2018.

Comments: 37 pages; 5 figures

MSC Class: 35A22; 60B15; 42B10

Roto-Translation Covariant Convolutional Networks for Medical Image Analysis

Authors: Erik J Bekkers, Maxime W Lafarge, Mitko Veta, Koen AJ Eppenhof, Josien PW Pluim, Remco Duits

Abstract: We propose a framework for rotation and translation covariant deep learning using $SE(2)$ group convolutions. The group product of the special Euclidean motion group $SE(2)$ describes how a concatenation of two roto-translations results in a net roto-translation. We encode this geometric structure into convolutional neural networks (CNNs) via $SE(2)$ group convolutional layers, which fit into the… ▽ More We propose a framework for rotation and translation covariant deep learning using $SE(2)$ group convolutions. The group product of the special Euclidean motion group $SE(2)$ describes how a concatenation of two roto-translations results in a net roto-translation. We encode this geometric structure into convolutional neural networks (CNNs) via $SE(2)$ group convolutional layers, which fit into the standard 2D CNN framework, and which allow to generically deal with rotated input samples without the need for data augmentation. We introduce three layers: a lifting layer which lifts a 2D (vector valued) image to an $SE(2)$-image, i.e., 3D (vector valued) data whose domain is $SE(2)$; a group convolution layer from and to an $SE(2)$-image; and a projection layer from an $SE(2)$-image to a 2D image. The lifting and group convolution layers are $SE(2)$ covariant (the output roto-translates with the input). The final projection layer, a maximum intensity projection over rotations, makes the full CNN rotation invariant. We show with three different problems in histopathology, retinal imaging, and electron microscopy that with the proposed group CNNs, state-of-the-art performance can be achieved, without the need for data augmentation by rotation and with increased performance compared to standard CNNs that do rely on augmentation. △ Less

Submitted 11 June, 2018; v1 submitted 10 April, 2018; originally announced April 2018.

Comments: 8 pages, 2 figures, 1 table, accepted at MICCAI 2018

Design and Processing of Invertible Orientation Scores of 3D Images for Enhancement of Complex Vasculature

Authors: M. H. J. Janssen, A. J. E. M. Janssen, E. J. Bekkers, J. Olivan Bescos, R. Duits

Abstract: The enhancement and detection of elongated structures in noisy image data is relevant for many biomedical imaging applications. To handle complex crossing structures in 2D images, 2D orientation scores $U: \mathbb{R} ^ 2\times S ^ 1 \rightarrow \mathbb{C}$ were introduced, which already showed their use in a variety of applications. Here we extend this work to 3D orientation scores… ▽ More The enhancement and detection of elongated structures in noisy image data is relevant for many biomedical imaging applications. To handle complex crossing structures in 2D images, 2D orientation scores $U: \mathbb{R} ^ 2\times S ^ 1 \rightarrow \mathbb{C}$ were introduced, which already showed their use in a variety of applications. Here we extend this work to 3D orientation scores $U: \mathbb{R} ^ 3 \times S ^ 2\rightarrow \mathbb{C}$. First, we construct the orientation score from a given dataset, which is achieved by an invertible coherent state type of transform. For this transformation we introduce 3D versions of the 2D cake-wavelets, which are complex wavelets that can simultaneously detect oriented structures and oriented edges. Here we introduce two types of cake-wavelets, the first uses a discrete Fourier transform, the second is designed in the 3D generalized Zernike basis, allowing us to calculate analytical expressions for the spatial filters. Finally, we show two applications of the orientation score transformation. In the first application we propose an extension of crossing-preserving coherence enhancing diffusion via our invertible orientation scores of 3D images which we apply to real medical image data. In the second one we develop a new tubularity measure using 3D orientation scores and apply the tubularity measure to both artificial and real medical data. △ Less

Submitted 27 November, 2017; v1 submitted 7 July, 2017; originally announced July 2017.

MSC Class: 62H35; 65T60; 58J65; 37L05

Vessel Tracking via Sub-Riemannian Geodesics on $\mathbb{R}^2 \times P^{1}$

Authors: E. J. Bekkers, R. Duits, A. Mashtakov, Yu. Sachkov

Abstract: We study a data-driven sub-Riemannian (SR) curve optimization model for connecting local orientations in orientation lifts of images. Our model lives on the projective line bundle $\mathbb{R}^{2} \times P^{1}$, with $P^{1}=S^{1}/_{\sim}$ with identification of antipodal points. It extends previous cortical models for contour perception on $\mathbb{R}^{2} \times P^{1}$ to the data-driven case. We p… ▽ More We study a data-driven sub-Riemannian (SR) curve optimization model for connecting local orientations in orientation lifts of images. Our model lives on the projective line bundle $\mathbb{R}^{2} \times P^{1}$, with $P^{1}=S^{1}/_{\sim}$ with identification of antipodal points. It extends previous cortical models for contour perception on $\mathbb{R}^{2} \times P^{1}$ to the data-driven case. We provide a complete (mainly numerical) analysis of the dynamics of the 1st Maxwell-set with growing radii of SR-spheres, revealing the cut-locus. Furthermore, a comparison of the cusp-surface in $\mathbb{R}^{2} \times P^{1}$ to its counterpart in $\mathbb{R}^{2} \times S^{1}$ of a previous model, reveals a general and strong reduction of cusps in spatial projections of geodesics. Numerical solutions of the model are obtained by a single wavefront propagation method relying on a simple extension of existing anisotropic fast-marching or iterative morphological scale space methods. Experiments show that the projective line bundle structure greatly reduces the presence of cusps. Another advantage of including $\mathbb{R}^2 \times P^{1}$ instead of $\mathbb{R}^{2} \times S^{1}$ in the wavefront propagation is reduction of computational time. △ Less

Submitted 13 April, 2017; originally announced April 2017.

Comments: 8 pages, 4 figures, Submitted to GSI2017

Optimal Paths for Variants of the 2D and 3D Reeds-Shepp Car with Applications in Image Analysis

Authors: Remco Duits, Stephan P. L. Meesters, Jean-Marie Mirebeau, Jorg M. Portegies

Abstract: We present a PDE-based approach for finding optimal paths for the Reeds-Shepp car. In our model we minimize a (data-driven) functional involving both curvature and length penalization, with several generalizations. Our approach encompasses the two and three dimensional variants of this model, state dependent costs, and moreover, the possibility of removing the reverse gear of the vehicle. We prove… ▽ More We present a PDE-based approach for finding optimal paths for the Reeds-Shepp car. In our model we minimize a (data-driven) functional involving both curvature and length penalization, with several generalizations. Our approach encompasses the two and three dimensional variants of this model, state dependent costs, and moreover, the possibility of removing the reverse gear of the vehicle. We prove both global and local controllability results of the models. Via eikonal equations on the manifold $\mathbb{R}^d \times \mathbb{S}^{d-1}$ we compute distance maps w.r.t. highly anisotropic Finsler metrics, which approximate the singular (quasi)-distances underlying the model. This is achieved using a Fast-Marching (FM) method, building on work by Mirebeau. The FM method is based on specific discretization stencils which are adapted to the preferred directions of the Finsler metric and obey a generalized acuteness property. The shortest paths can be found with a gradient descent method on the distance map, which we formalize in a theorem. We justify the use of our approximating metrics by proving convergence results. Our curve optimization model in $\mathbb{R}^{d} \times \mathbb{S}^{d-1}$ with data-driven cost allows to extract complex tubular structures from medical images, e.g. crossings, and incomplete data due to occlusions or low contrast. Our work extends the results of Sanguinetti et al. on numerical sub-Riemannian eikonal equations and the Reeds-Shepp Car to 3D, with comparisons to exact solutions by Duits et al. Numerical experiments show the high potential of our method in two applications: vessel tracking in retinal images for the case $d=2$, and brain connectivity measures from diffusion weighted MRI-data for the case $d=3$, extending the work of Bekkers et al. We demonstrate how the new model without reverse gear better handles bifurcations. △ Less

Submitted 15 December, 2017; v1 submitted 19 December, 2016; originally announced December 2016.

Comments: To appear in JMIV Special Issue

Retrieving challenging vessel connections in retinal images by line co-occurrence statistics

Authors: Samaneh Abbasi-Sureshjani, Jiong Zhang, Remco Duits, Bart ter Haar Romeny

Abstract: Natural images contain often curvilinear structures, which might be disconnected, or partly occluded. Recovering the missing connection of disconnected structures is an open issue and needs appropriate geometric reasoning. We propose to find line co-occurrence statistics from the centerlines of blood vessels in retinal images and show its remarkable similarity to a well-known probabilistic model f… ▽ More Natural images contain often curvilinear structures, which might be disconnected, or partly occluded. Recovering the missing connection of disconnected structures is an open issue and needs appropriate geometric reasoning. We propose to find line co-occurrence statistics from the centerlines of blood vessels in retinal images and show its remarkable similarity to a well-known probabilistic model for the connectivity pattern in the primary visual cortex. Furthermore, the probabilistic model is trained from the data via statistics and used for automated grou** of interrupted vessels in a spectral clustering based approach. Several challenging image patches are investigated around junction points, where successful results indicate the perfect match of the trained model to the profiles of blood vessels in retinal images. Also, comparisons among several statistical models obtained from different datasets reveals their high similarity i.e., they are independent of the dataset. On top of that, the best approximation of the statistical model with the symmetrized extension of the probabilistic model on the projective line bundle is found with a least square error smaller than 2%. Apparently, the direction process on the projective line bundle is a good continuation model for vessels in retinal images. △ Less

Submitted 20 October, 2016; originally announced October 2016.

New Exact and Numerical Solutions of the (Convection-)Diffusion Kernels on SE(3)

Authors: J. M. Portegies, R. Duits

Abstract: We consider hypo-elliptic diffusion and convection-diffusion on $\mathbb{R}^3 \rtimes S^2$, the quotient of the Lie group of rigid body motions SE(3) in which group elements are equivalent if they are equal up to a rotation around the reference axis. We show that we can derive expressions for the convolution kernels in terms of eigenfunctions of the PDE, by extending the approach for the SE(2) cas… ▽ More We consider hypo-elliptic diffusion and convection-diffusion on $\mathbb{R}^3 \rtimes S^2$, the quotient of the Lie group of rigid body motions SE(3) in which group elements are equivalent if they are equal up to a rotation around the reference axis. We show that we can derive expressions for the convolution kernels in terms of eigenfunctions of the PDE, by extending the approach for the SE(2) case. This goes via application of the Fourier transform of the PDE in the spatial variables, yielding a second order differential operator. We show that the eigenfunctions of this operator can be expressed as (generalized) spheroidal wave functions. The same exact formulas are derived via the Fourier transform on SE(3). We solve both the evolution itself, as well as the time-integrated process that corresponds to the resolvent operator. Furthermore, we have extended a standard numerical procedure from SE(2) to SE(3) for the computation of the solution kernels that is directly related to the exact solutions. Finally, we provide a novel analytic approximation of the kernels that we briefly compare to the exact kernels. △ Less

Submitted 12 May, 2017; v1 submitted 13 April, 2016; originally announced April 2016.

Comments: Revised and restructured

Tracking of Lines in Spherical Images via Sub-Riemannian Geodesics on SO(3)

Authors: A. Mashtakov, R. Duits, Yu. Sachkov, E. J. Bekkers, I. Beschastnyi

Abstract: In order to detect salient lines in spherical images, we consider the problem of minimizing the functional $\int \limits_0^l C(γ(s)) \sqrt{ξ^2 + k_g^2(s)} \, {\rm d}s$ for a curve $γ$ on a sphere with fixed boundary points and directions. The total length $l$ is free, $s$ denotes the spherical arclength, and $k_g$ denotes the geodesic curvature of $γ$. Here the smooth external cost $C\geq δ>0$ is… ▽ More In order to detect salient lines in spherical images, we consider the problem of minimizing the functional $\int \limits_0^l C(γ(s)) \sqrt{ξ^2 + k_g^2(s)} \, {\rm d}s$ for a curve $γ$ on a sphere with fixed boundary points and directions. The total length $l$ is free, $s$ denotes the spherical arclength, and $k_g$ denotes the geodesic curvature of $γ$. Here the smooth external cost $C\geq δ>0$ is obtained from spherical data. We lift this problem to the sub-Riemannian (SR) problem in Lie group $SO(3)$ and show that the spherical projection of certain SR geodesics provides a solution to our curve optimization problem. In fact, this holds only for the geodesics whose spherical projection does not exhibit a cusp. The problem is a spherical extension of a well-known contour perception model, where we extend the model by Boscain and Rossi to the general case $ξ> 0$, $C \neq 1$. For $C=1$, we derive SR geodesics and evaluate the first cusp time. We show that these curves have a simpler expression when they are parameterized by spherical arclength rather than by sub-Riemannian arclength. For case $C \neq 1$ (data-driven SR geodesics), we solve via a SR Fast Marching method. Finally, we show an experiment of vessel tracking in a spherical image of the retina and study the effect of including the spherical geometry in analysis of vessels curvature. △ Less

Submitted 24 March, 2017; v1 submitted 13 April, 2016; originally announced April 2016.

Comments: 25 pages, 17 figures; Corrected typos, improved images, updated Appendix D. E-link: http://link.springer.com/article/10.1007/s10851-017-0705-9

Template Matching via Densities on the Roto-Translation Group

Authors: Erik J. Bekkers, Marco Loog, Bart M. ter Haar Romeny, Remco Duits

Abstract: We propose a template matching method for the detection of 2D image objects that are characterized by orientation patterns. Our method is based on data representations via orientation scores, which are functions on the space of positions and orientations, and which are obtained via a wavelet-type transform. This new representation allows us to detect orientation patterns in an intuitive and direct… ▽ More We propose a template matching method for the detection of 2D image objects that are characterized by orientation patterns. Our method is based on data representations via orientation scores, which are functions on the space of positions and orientations, and which are obtained via a wavelet-type transform. This new representation allows us to detect orientation patterns in an intuitive and direct way, namely via cross-correlations. Additionally, we propose a generalized linear regression framework for the construction of suitable templates using smoothing splines. Here, it is important to recognize a curved geometry on the position-orientation domain, which we identify with the Lie group SE(2): the roto-translation group. Templates are then optimized in a B-spline basis, and smoothness is defined with respect to the curved geometry. We achieve state-of-the-art results on three different applications: detection of the optic nerve head in the retina (99.83% success rate on 1737 images), of the fovea in the retina (99.32% success rate on 1616 images), and of the pupil in regular camera images (95.86% on 1521 images). The high performance is due to inclusion of both intensity and orientation features with effective geometric priors in the template matching. Moreover, our method is fast due to a cross-correlation based matching approach. △ Less

Submitted 9 March, 2017; v1 submitted 10 March, 2016; originally announced March 2016.

Comments: 28 pages, 11 figures, 5 appendices. Paper accepted at ieee transaction on Pattern Analysis and Machine Inteligence (ieee tPAMI)

Sub-Riemannian Fast Marching in SE(2)

Authors: Gonzalo Sanguinetti, Erik Bekkers, Remco Duits, Michiel Janssen, Alexey Mashtakov, Jean-Marie Mirebeau

Abstract: We propose a Fast Marching based implementation for computing sub-Riemanninan (SR) geodesics in the roto-translation group SE(2), with a metric depending on a cost induced by the image data. The key ingredient is a Riemannian approximation of the SR-metric. Then, a state of the art Fast Marching solver that is able to deal with extreme anisotropies is used to compute a SR-distance map as the solut… ▽ More We propose a Fast Marching based implementation for computing sub-Riemanninan (SR) geodesics in the roto-translation group SE(2), with a metric depending on a cost induced by the image data. The key ingredient is a Riemannian approximation of the SR-metric. Then, a state of the art Fast Marching solver that is able to deal with extreme anisotropies is used to compute a SR-distance map as the solution of a corresponding eikonal equation. Subsequent backtracking on the distance map gives the geodesics. To validate the method, we consider the uniform cost case in which exact formulas for SR-geodesics are known and we show remarkable accuracy of the numerically computed SR-spheres. We also show a dramatic decrease in computational time with respect to a previous PDE-based iterative approach. Regarding image analysis applications, we show the potential of considering these data adaptive geodesics for a fully automated retinal vessel tree segmentation. △ Less

Submitted 11 August, 2015; originally announced August 2015.

Comments: CIARP 2015

Improving Fiber Alignment in HARDI by Combining Contextual PDE Flow with Constrained Spherical Deconvolution

Authors: J. M. Portegies, R. H. J. Fick, G. R. Sanguinetti, S. P. L. Meesters, G. Girard, R. Duits

Abstract: We propose two strategies to improve the quality of tractography results computed from diffusion weighted magnetic resonance imaging (DW-MRI) data. Both methods are based on the same PDE framework, defined in the coupled space of positions and orientations, associated with a stochastic process describing the enhancement of elongated structures while preserving crossing structures. In the first met… ▽ More We propose two strategies to improve the quality of tractography results computed from diffusion weighted magnetic resonance imaging (DW-MRI) data. Both methods are based on the same PDE framework, defined in the coupled space of positions and orientations, associated with a stochastic process describing the enhancement of elongated structures while preserving crossing structures. In the first method we use the enhancement PDE for contextual regularization of a fiber orientation distribution (FOD) that is obtained on individual voxels from high angular resolution diffusion imaging (HARDI) data via constrained spherical deconvolution (CSD). Thereby we improve the FOD as input for subsequent tractography. Secondly, we introduce the fiber to bundle coherence (FBC), a measure for quantification of fiber alignment. The FBC is computed from a tractography result using the same PDE framework and provides a criterion for removing the spurious fibers. We validate the proposed combination of CSD and enhancement on phantom data and on human data, acquired with different scanning protocols. On the phantom data we find that PDE enhancements improve both local metrics and global metrics of tractography results, compared to CSD without enhancements. On the human data we show that the enhancements allow for a better reconstruction of crossing fiber bundles and they reduce the variability of the tractography output with respect to the acquisition parameters. Finally, we show that both the enhancement of the FODs and the use of the FBC measure on the tractography improve the stability with respect to different stochastic realizations of probabilistic tractography. This is shown in a clinical application: the reconstruction of the optic radiation for epilepsy surgery planning. △ Less

Submitted 23 June, 2015; originally announced June 2015.

New Approximation of a Scale Space Kernel on SE(3) and Applications in Neuroimaging

Authors: J. M. Portegies, G. R. Sanguinetti, S. P. L Meesters, R. Duits

Abstract: We provide a new, analytic kernel for scale space filtering of dMRI data. The kernel is an approximation for the Green's function of a hypo-elliptic diffusion on the 3D rigid body motion group SE(3), for fiber enhancement in dMRI. The enhancements are described by linear scale space PDEs in the coupled space of positions and orientations embedded in SE(3). As initial condition for the evolution we… ▽ More We provide a new, analytic kernel for scale space filtering of dMRI data. The kernel is an approximation for the Green's function of a hypo-elliptic diffusion on the 3D rigid body motion group SE(3), for fiber enhancement in dMRI. The enhancements are described by linear scale space PDEs in the coupled space of positions and orientations embedded in SE(3). As initial condition for the evolution we use either a Fiber Orientation Distribution (FOD) or an Orientation Density Function (ODF). Explicit formulas for the exact kernel do not exist. Although approximations well-suited for fast implementation have been proposed in literature, they lack important symmetries of the exact kernel. We introduce techniques to include these symmetries in approximations based on the logarithm on SE(3), resulting in an improved kernel. Regarding neuroimaging applications, we apply our enhancement kernel (a) to improve dMRI tractography results and (b) to quantify coherence of obtained streamline bundles. △ Less

Submitted 8 June, 2015; originally announced June 2015.

Comments: 12 pages, 4 figures, SSVM 2015

Invertible Orientation Scores of 3D Images

Authors: Michiel Janssen, Remco Duits, Marcel Breeuwer

Abstract: The enhancement and detection of elongated structures in noisy image data is relevant for many biomedical applications. To handle complex crossing structures in 2D images, 2D orientation scores were introduced, which already showed their use in a variety of applications. Here we extend this work to 3D orientation scores. First, we construct the orientation score from a given dataset, which is achi… ▽ More The enhancement and detection of elongated structures in noisy image data is relevant for many biomedical applications. To handle complex crossing structures in 2D images, 2D orientation scores were introduced, which already showed their use in a variety of applications. Here we extend this work to 3D orientation scores. First, we construct the orientation score from a given dataset, which is achieved by an invertible coherent state type of transform. For this transformation we introduce 3D versions of the 2D cake-wavelets, which are complex wavelets that can simultaneously detect oriented structures and oriented edges. For efficient implementation of the different steps in the wavelet creation we use a spherical harmonic transform. Finally, we show some first results of practical applications of 3D orientation scores. △ Less

Submitted 28 May, 2015; originally announced May 2015.

Comments: ssvm 2015 published version in LNCS contains a mistake (a switch notation spherical angles) that is corrected in this arxiv version

A PDE Approach to Data-driven Sub-Riemannian Geodesics in SE(2)

Authors: Erik J. Bekkers, Remco Duits, Alexey Mashtakov, Gonzalo R. Sanguinetti

Abstract: We present a new flexible wavefront propagation algorithm for the boundary value problem for sub-Riemannian (SR) geodesics in the roto-translation group $SE(2) = \mathbb{R}^2 \rtimes S^1$ with a metric tensor depending on a smooth external cost $\mathcal{C}:SE(2) \to [δ,1]$, $δ>0$, computed from image data. The method consists of a first step where a SR-distance map is computed as a viscosity solu… ▽ More We present a new flexible wavefront propagation algorithm for the boundary value problem for sub-Riemannian (SR) geodesics in the roto-translation group $SE(2) = \mathbb{R}^2 \rtimes S^1$ with a metric tensor depending on a smooth external cost $\mathcal{C}:SE(2) \to [δ,1]$, $δ>0$, computed from image data. The method consists of a first step where a SR-distance map is computed as a viscosity solution of a Hamilton-Jacobi-Bellman (HJB) system derived via Pontryagin's Maximum Principle (PMP). Subsequent backward integration, again relying on PMP, gives the SR-geodesics. For $\mathcal{C}=1$ we show that our method produces the global minimizers. Comparison with exact solutions shows a remarkable accuracy of the SR-spheres and the SR-geodesics. We present numerical computations of Maxwell points and cusp points, which we again verify for the uniform cost case $\mathcal{C}=1$. Regarding image analysis applications, tracking of elongated structures in retinal and synthetic images show that our line tracking generically deals with crossings. We show the benefits of including the sub-Riemannian geometry. △ Less

Submitted 20 April, 2015; v1 submitted 4 March, 2015; originally announced March 2015.

Comments: Extended version of SSVM 2015 conference article "Data-driven Sub-Riemannian Geodesics in SE(2)"

Journal ref: SIAM J Imaging Sci, 2015, 8(4), 2740-2770. (31 pages)

Locally Adaptive Frames in the Roto-Translation Group and their Applications in Medical Imaging

Authors: R. Duits, M. H. J. Janssen, J. Hannink, G. R. Sanguinetti

Abstract: Locally adaptive differential frames (gauge frames) are a well-known effective tool in image analysis, used in differential invariants and PDE-flows. However, at complex structures such as crossings or junctions, these frames are not well-defined. Therefore, we generalize the notion of gauge frames on images to gauge frames on data representations $U:\mathbb{R}^{d} \rtimes S^{d-1} \to \mathbb{R}$… ▽ More Locally adaptive differential frames (gauge frames) are a well-known effective tool in image analysis, used in differential invariants and PDE-flows. However, at complex structures such as crossings or junctions, these frames are not well-defined. Therefore, we generalize the notion of gauge frames on images to gauge frames on data representations $U:\mathbb{R}^{d} \rtimes S^{d-1} \to \mathbb{R}$ defined on the extended space of positions and orientations, which we relate to data on the roto-translation group $SE(d)$, $d=2,3$. This allows to define multiple frames per position, one per orientation. We compute these frames via exponential curve fits in the extended data representations in $SE(d)$. These curve fits minimize first or second order variational problems which are solved by spectral decomposition of, respectively, a structure tensor or Hessian of data on $SE(d)$. We include these gauge frames in differential invariants and crossing preserving PDE-flows acting on extended data representation $U$ and we show their advantage compared to the standard left-invariant frame on $SE(d)$. Applications include crossing-preserving filtering and improved segmentations of the vascular tree in retinal images, and new 3D extensions of coherence-enhancing diffusion via invertible orientation scores. △ Less

Submitted 12 January, 2017; v1 submitted 27 February, 2015; originally announced February 2015.

MSC Class: 58J65; 49Q20; 22E30

Numerical Approaches for Linear Left-invariant Diffusions on SE(2), their Comparison to Exact Solutions, and their Applications in Retinal Imaging

Authors: Jiong Zhang, Remco Duits, Gonzalo Sanguinetti, Bart M. ter Haar Romeny

Abstract: Left-invariant PDE-evolutions on the roto-translation group $SE(2)$ (and their resolvent equations) have been widely studied in the fields of cortical modeling and image analysis. They include hypo-elliptic diffusion (for contour enhancement) proposed by Citti & Sarti, and Petitot, and they include the direction process (for contour completion) proposed by Mumford. This paper presents a thorough s… ▽ More Left-invariant PDE-evolutions on the roto-translation group $SE(2)$ (and their resolvent equations) have been widely studied in the fields of cortical modeling and image analysis. They include hypo-elliptic diffusion (for contour enhancement) proposed by Citti & Sarti, and Petitot, and they include the direction process (for contour completion) proposed by Mumford. This paper presents a thorough study and comparison of the many numerical approaches, which, remarkably, is missing in the literature. Existing numerical approaches can be classified into 3 categories: Finite difference methods, Fourier based methods (equivalent to $SE(2)$-Fourier methods), and stochastic methods (Monte Carlo simulations). There are also 3 types of exact solutions to the PDE-evolutions that were derived explicitly (in the spatial Fourier domain) in previous works by Duits and van Almsick in 2005. Here we provide an overview of these 3 types of exact solutions and explain how they relate to each of the 3 numerical approaches. We compute relative errors of all numerical approaches to the exact solutions, and the Fourier based methods show us the best performance with smallest relative errors. We also provide an improvement of Mathematica algorithms for evaluating Mathieu-functions, crucial in implementations of the exact solutions. Furthermore, we include an asymptotical analysis of the singularities within the kernels and we propose a probabilistic extension of underlying stochastic processes that overcomes the singular behavior in the origin of time-integrated kernels. Finally, we show retinal imaging applications of combining left-invariant PDE-evolutions with invertible orientation scores. △ Less

Submitted 1 March, 2016; v1 submitted 13 March, 2014; originally announced March 2014.

Comments: A final and corrected version of the manuscript is Published in Numerical Mathematics: Theory, Methods and Applications (NM-TMA), vol. (9), p.1-50, 2016

Journal ref: Numerical Mathematics: Theory, Methods and Applications (NM-TMA), vol. (9), p.1-50, 2016

Vesselness via Multiple Scale Orientation Scores

Authors: Julius Hannink, Remco Duits, Erik Bekkers

Abstract: The multi-scale Frangi vesselness filter is an established tool in (retinal) vascular imaging. However, it cannot cope with crossings or bifurcations, since it only looks for elongated structures. Therefore, we disentangle crossing structures in the image via (multiple scale) invertible orientation scores. The described vesselness filter via scale-orientation scores performs considerably better at… ▽ More The multi-scale Frangi vesselness filter is an established tool in (retinal) vascular imaging. However, it cannot cope with crossings or bifurcations, since it only looks for elongated structures. Therefore, we disentangle crossing structures in the image via (multiple scale) invertible orientation scores. The described vesselness filter via scale-orientation scores performs considerably better at enhancing vessels throughout crossings and bifurcations than the Frangi version. Both methods are evaluated on a public dataset. Performance is measured by comparing ground truth data to the segmentation results obtained by basic thresholding and morphological component analysis of the filtered images. △ Less

Submitted 19 May, 2014; v1 submitted 20 February, 2014; originally announced February 2014.

Comments: 9 pages, 8 figures

Left-invariant evolutions of wavelet transforms on the Similitude Group

Authors: U. Sharma, R. Duits

Abstract: Enhancement of multiple-scale elongated structures in noisy image data is relevant for many biomedical applications but commonly used PDE-based enhancement techniques often fail at crossings in an image. To get an overview of how an image is composed of local multiple-scale elongated structures we construct a multiple scale orientation score, which is a continuous wavelet transform on the similitu… ▽ More Enhancement of multiple-scale elongated structures in noisy image data is relevant for many biomedical applications but commonly used PDE-based enhancement techniques often fail at crossings in an image. To get an overview of how an image is composed of local multiple-scale elongated structures we construct a multiple scale orientation score, which is a continuous wavelet transform on the similitude group, SIM(2). Our unitary transform maps the space of images onto a reproducing kernel space defined on SIM(2), allowing us to robustly relate Euclidean (and scaling) invariant operators on images to left-invariant operators on the corresponding continuous wavelet transform. Rather than often used wavelet (soft-)thresholding techniques, we employ the group structure in the wavelet domain to arrive at left-invariant evolutions and flows (diffusion), for contextual crossing preserving enhancement of multiple scale elongated structures in noisy images. We present experiments that display benefits of our work compared to recent PDE techniques acting directly on the images and to our previous work on left-invariant diffusions on orientation scores defined on Euclidean motion group. △ Less

Submitted 25 February, 2014; v1 submitted 7 June, 2013; originally announced June 2013.

Comments: 40 pages

MSC Class: 35R03; 22E25; 35Q68

On sub-Riemannian geodesics in $SE(3)$ whose spatial projections do not have cusps

Authors: Remco Duits, Arpan Ghosh, Tom Dela Haije, Alexey Mashtakov

Abstract: We consider the problem $\mathbf{P_{curve}}$ of minimizing $\int \limits_0^L \sqrt{ξ^2 + κ^2(s)} \, {\rm d}s$ for a curve $\mathbf{x}$ on $\mathbb R$ with fixed boundary points and directions. Here the total length $L\geq 0$ is free, $s$ denotes the arclength parameter, $κ$ denotes the absolute curvature of $\mathbf{x}$, and $ξ>0$ is constant. We lift problem $\mathbf{P_{curve}}$ on $\mathbb R^3$… ▽ More We consider the problem $\mathbf{P_{curve}}$ of minimizing $\int \limits_0^L \sqrt{ξ^2 + κ^2(s)} \, {\rm d}s$ for a curve $\mathbf{x}$ on $\mathbb R$ with fixed boundary points and directions. Here the total length $L\geq 0$ is free, $s$ denotes the arclength parameter, $κ$ denotes the absolute curvature of $\mathbf{x}$, and $ξ>0$ is constant. We lift problem $\mathbf{P_{curve}}$ on $\mathbb R^3$ to a sub-Riemannian problem $\mathbf{P_{mec}}$ on $\operatorname{SE(3)}\nolimits/(\{\mathbf{0}\}\times \operatorname{SO(2)}\nolimits)$. Here, for admissible boundary conditions, the spatial projections of sub-Riemannian geodesics do not exhibit cusps and they solve problem $\mathbf{P_{curve}}$. We apply the Pontryagin Maximum Principle (PMP) and prove Liouville integrability of the Hamiltonian system. We derive explicit analytic formulas for such sub-Riemannian geodesics, relying on the co-adjoint orbit structure, an underlying Cartan connection, and the matrix representation of $\operatorname{SE(3)}\nolimits$ arising in the Cartan-matrix. These formulas allow us to extract geometrical properties of the sub-Riemannian geodesics with cuspless projection, such as planarity conditions, explicit bounds on their torsion, and their symmetries. Furthermore, they allow us to parameterize all admissible boundary conditions reachable by geodesics with cuspless spatial projection. Such projections lay in the upper half space. We prove this for most cases, and the rest is checked numerically. Finally, we employ the formulas to numerically solve the boundary value problem, and visualize the set of admissible boundary conditions. △ Less

Submitted 8 April, 2016; v1 submitted 26 May, 2013; originally announced May 2013.

Comments: 28 pages, 9 figures

A Multi-Orientation Analysis Approach to Retinal Vessel Tracking

Authors: Erik Bekkers, Remco Duits, Tos Berendschot, Bart ter Haar Romeny

Abstract: This paper presents a method for retinal vasculature extraction based on biologically inspired multi-orientation analysis. We apply multi-orientation analysis via so-called invertible orientation scores, modeling the cortical columns in the visual system of higher mammals. This allows us to generically deal with many hitherto complex problems inherent to vessel tracking, such as crossings, bifurca… ▽ More This paper presents a method for retinal vasculature extraction based on biologically inspired multi-orientation analysis. We apply multi-orientation analysis via so-called invertible orientation scores, modeling the cortical columns in the visual system of higher mammals. This allows us to generically deal with many hitherto complex problems inherent to vessel tracking, such as crossings, bifurcations, parallel vessels, vessels of varying widths and vessels with high curvature. Our approach applies tracking in invertible orientation scores via a novel geometrical principle for curve optimization in the Euclidean motion group SE(2). The method runs fully automatically and provides a detailed model of the retinal vasculature, which is crucial as a sound basis for further quantitative analysis of the retina, especially in screening applications. △ Less

Submitted 30 December, 2013; v1 submitted 14 December, 2012; originally announced December 2012.

Comments: Accepted at JMIV. The final publication will become available at springerlink.com

Journal ref: Journal of Mathematical Imaging and Vision 49(3) (2014) 583-610

Curve cuspless reconstruction via sub-Riemannian geometry

Authors: Ugo Boscain, Remco Duits, Francesco Rossi, Yuri Sachkov

Abstract: We consider the problem of minimizing $\int_{0}^L \sqrt{ξ^2 +K^2(s)}\, ds $ for a planar curve having fixed initial and final positions and directions. The total length $L$ is free. Here $s$ is the variable of arclength parametrization, $K(s)$ is the curvature of the curve and $ξ>0$ a parameter. This problem comes from a model of geometry of vision due to Petitot, Citti and Sarti. We study exist… ▽ More We consider the problem of minimizing $\int_{0}^L \sqrt{ξ^2 +K^2(s)}\, ds $ for a planar curve having fixed initial and final positions and directions. The total length $L$ is free. Here $s$ is the variable of arclength parametrization, $K(s)$ is the curvature of the curve and $ξ>0$ a parameter. This problem comes from a model of geometry of vision due to Petitot, Citti and Sarti. We study existence of local and global minimizers for this problem. We prove that if for a certain choice of boundary conditions there is no global minimizer, then there is neither a local minimizer nor a geodesic. We finally give properties of the set of boundary conditions for which there exists a solution to the problem. △ Less

Submitted 25 April, 2013; v1 submitted 14 March, 2012; originally announced March 2012.

Evolution Equations on Gabor Transforms and their Applications

Authors: Remco Duits, Hartmut Führ, Bart Janssen, Mark Bruurmijn, Luc Florack, Hans van Assen

Abstract: We introduce a systematic approach to the design, implementation and analysis of left-invariant evolution schemes acting on Gabor transform, primarily for applications in signal and image analysis. Within this approach we relate operators on signals to operators on Gabor transforms. In order to obtain a translation and modulation invariant operator on the space of signals, the corresponding operat… ▽ More We introduce a systematic approach to the design, implementation and analysis of left-invariant evolution schemes acting on Gabor transform, primarily for applications in signal and image analysis. Within this approach we relate operators on signals to operators on Gabor transforms. In order to obtain a translation and modulation invariant operator on the space of signals, the corresponding operator on the reproducing kernel space of Gabor transforms must be left invariant, i.e. it should commute with the left regular action of the reduced Heisenberg group H_r. By using the left-invariant vector fields on H_r in the generators of our evolution equations on Gabor transforms, we naturally employ the essential group structure on the domain of a Gabor transform. Here we distinguish between two tasks. Firstly, we consider non-linear adaptive left-invariant convection (reassignment) to sharpen Gabor transforms, while maintaining the original signal. Secondly, we consider signal enhancement via left-invariant diffusion on the corresponding Gabor transform. We provide numerical experiments and analytical evidence for our methods and we consider an explicit medical imaging application. △ Less

Submitted 27 October, 2011; originally announced October 2011.

MSC Class: 35R03; 22E25; 35Q68

Diffusion, Convection and Erosion on SE(3)/({0} \times SO(2)) and their Application to the Enhancement of Crossing Fibers

Authors: Remco Duits, Eric Creusen, Arpan Ghosh, Tom Dela Haije

Abstract: In this article we study both left-invariant (convection-)diffusions and left-invariant Hamilton-Jacobi equations on the space SE(3)/({0} \times SO(2)) of 3D-positions and orientations naturally embedded in the group SE(3) of 3D-rigid body movements. The general motivation for these (convection-)diffusions and erosions is to obtain crossing-preserving fiber enhancement on probability densities def… ▽ More In this article we study both left-invariant (convection-)diffusions and left-invariant Hamilton-Jacobi equations on the space SE(3)/({0} \times SO(2)) of 3D-positions and orientations naturally embedded in the group SE(3) of 3D-rigid body movements. The general motivation for these (convection-)diffusions and erosions is to obtain crossing-preserving fiber enhancement on probability densities defined on the space of positions and orientations. The linear left-invariant (convection-)diffusions are forward Kolmogorov equations of Brownian motions on SE(3)/({0}\timesSO(2)) and can be solved by convolution with the corresponding Green's functions or by a finite difference scheme. The left-invariant Hamilton-Jacobi equations are Bellman equations of cost processes on SE(3)/({0}\timesSO(2)) and they are solved by a morphological convolution with the corresponding Green's functions. Furthermore, we consider pseudo-linear scale spaces on the space of positions and orientations that combines dilation and diffusion in a single evolution. In our design and analysis for appropriate linear, non-linear, morphological and pseudo-linear scale spaces on SE(3)/({0}\timesSO(2)) we employ the underlying differential geometry on SE(3), where the frame of left-invariant vector fields serves as a moving frame of reference. Furthermore, we will present new and simpler finite difference schemes for our diffusions, which are clear improvements of our previous finite difference schemes. We apply our theory to the enhancement of fibres in magnetic resonance imaging (MRI) techniques for imaging water diffusion processes in brain white matter. We provide experiments of our crossing-preserving evolutions on neural images of a human brain containing crossing fibers. △ Less

Submitted 14 November, 2011; v1 submitted 3 March, 2011; originally announced March 2011.

Report number: CASA-report March 2011 MSC Class: 53D10; 49L25; 65D18; 58J65; 49Q20; 22E30; 34B30; 37L05; 47D06

Left-invariant Stochastic Evolution Equations on SE(2) and its Applications to Contour Enhancement and Contour Completion via Invertible Orientation Scores

Authors: Remco Duits, Erik Franken

Abstract: We provide the explicit solutions of linear, left-invariant, (convection)-diffusion equations and the corresponding resolvent equations on the 2D-Euclidean motion group SE(2). These diffusion equations are forward Kolmogorov equations for stochastic processes for contour enhancement and completion. The solutions are group-convolutions with the corresponding Green's function, which we derive in e… ▽ More We provide the explicit solutions of linear, left-invariant, (convection)-diffusion equations and the corresponding resolvent equations on the 2D-Euclidean motion group SE(2). These diffusion equations are forward Kolmogorov equations for stochastic processes for contour enhancement and completion. The solutions are group-convolutions with the corresponding Green's function, which we derive in explicit form. We mainly focus on the Kolmogorov equations for contour enhancement processes which, in contrast to the Kolmogorov equations for contour completion, do not include convection. The Green's functions of these left-invariant partial differential equations coincide with the heat-kernels on SE(2), which we explicitly derive. Then we compute completion distributions on SE(2) which are the product of a forward and a backward resolvent evolved from resp. source and sink distribution on SE(2). On the one hand, the modes of Mumford's direction process for contour completion coincide with elastica curves minimizing $\int κ^{2} + εds$, related to zero-crossings of 2 left-invariant derivatives of the completion distribution. On the other hand, the completion measure for the contour enhancement concentrates on geodesics minimizing $\int \sqrt{κ^{2} + ε} ds$. This motivates a comparison between geodesics and elastica, which are quite similar. However, we derive more practical analytic solutions for the geodesics. The theory is motivated by medical image analysis applications where enhancement of elongated structures in noisy images is required. We use left-invariant (non)-linear evolution processes for automated contour enhancement on invertible orientation scores, obtained from an image by means of a special type of unitary wavelet transform. △ Less

Submitted 7 January, 2008; v1 submitted 6 November, 2007; originally announced November 2007.

Report number: CASA-report 07-35 Math. Dep. TUE Eindhoven University of Technology MSC Class: 58J65 ; 49Q20; 22E30