-
Using Atom-Like Local Image Features to Study Human Genetics and Neuroanatomy in Large Sets of 3D Medical Image Volumes
Authors:
Laurent Chauvin
Abstract:
The contributions of this thesis stem from technology developed to analyse large sets of volumetric images in terms of atom-like features extracted in 3D image space, following SIFT algorithm in 2D image space. New feature properties are introduced including a binary feature sign, analogous to an electrical charge, and a discrete set of symmetric feature orientation states in 3D space. These new p…
▽ More
The contributions of this thesis stem from technology developed to analyse large sets of volumetric images in terms of atom-like features extracted in 3D image space, following SIFT algorithm in 2D image space. New feature properties are introduced including a binary feature sign, analogous to an electrical charge, and a discrete set of symmetric feature orientation states in 3D space. These new properties are leveraged to extend feature invariance to include the sign inversion and parity (SP) transform, analogous to the charge conjugation and parity (CP) transform between a particle and its antiparticle in quantum mechanics, thereby accounting for local intensity contrast inversion between imaging modalities and axis reflections due to shape symmetry. A novel exponential kernel is proposed to quantify the similarity of a pair of features extracted in different images from their properties including location, scale, orientation, sign and appearance. A novel measure entitled the soft Jaccard is proposed to quantify the similarity of a pair of feature sets based on their overlap or intersection-over-union, where a kernel establishes non-binary or soft equivalence between a pair of feature elements. The soft Jaccard may be used to identify pairs of feature sets extracted from the same individuals or families with high accuracy, and a simple distance threshold led to the surprising discovery of previously unknown individual and family labeling errors in major public neuroimage datasets. A new algorithm is proposed to register or spatially align a pair of feature sets, entitled SIFT Coherent Point Drift (SIFT-CPD), by identifying a transform that maximizes the soft Jaccard between a fixed feature set and a transformed set. SIFT-CPD achieves faster and more accurate registration than the original CPD algorithm based on feature location information alone, in a variety of challenging.
△ Less
Submitted 25 August, 2022;
originally announced August 2022.
-
Registering Image Volumes using 3D SIFT and Discrete SP-Symmetry
Authors:
Laurent Chauvin,
William Wells III,
Matthew Toews
Abstract:
This paper proposes to extend local image features in 3D to include invariance to discrete symmetry including inversion of spatial axes and image contrast. A binary feature sign $s \in \{-1,+1\}$ is defined as the sign of the Laplacian operator $\nabla^2$, and used to obtain a descriptor that is invariant to image sign inversion $s \rightarrow -s$ and 3D parity transforms…
▽ More
This paper proposes to extend local image features in 3D to include invariance to discrete symmetry including inversion of spatial axes and image contrast. A binary feature sign $s \in \{-1,+1\}$ is defined as the sign of the Laplacian operator $\nabla^2$, and used to obtain a descriptor that is invariant to image sign inversion $s \rightarrow -s$ and 3D parity transforms $(x,y,z)\rightarrow(-x,-y,-z)$, i.e. SP-invariant or SP-symmetric. SP-symmetry applies to arbitrary scalar image fields $I: R^3 \rightarrow R^1$ map** 3D coordinates $(x,y,z) \in R^3$ to scalar intensity $I(x,y,z) \in R^1$, generalizing the well-known charge conjugation and parity symmetry (CP-symmetry) applying to elementary charged particles. Feature orientation is modeled as a set of discrete states corresponding to potential axis reflections, independently of image contrast inversion. Two primary axis vectors are derived from image observations and potentially subject to reflection, and a third axis is an axial vector defined by the right-hand rule. Augmenting local feature properties with sign in addition to standard (location, scale, orientation) geometry leads to descriptors that are invariant to coordinate reflections and intensity contrast inversion. Feature properties are factored in to probabilistic point-based registration as symmetric kernels, based on a model of binary feature correspondence. Experiments using the well-known coherent point drift (CPD) algorithm demonstrate that SIFT-CPD kernels achieve the most accurate and rapid registration of the human brain and CT chest, including multiple MRI modalities of differing intensity contrast, and abnormal local variations such as tumors or occlusions. SIFT-CPD image registration is invariant to global scaling, rotation and translation and image intensity inversions of the input data.
△ Less
Submitted 30 May, 2022;
originally announced May 2022.
-
GPU optimization of the 3D Scale-invariant Feature Transform Algorithm and a Novel BRIEF-inspired 3D Fast Descriptor
Authors:
Jean-Baptiste Carluer,
Laurent Chauvin,
Jie Luo,
William M. Wells III,
Ines Machado,
Rola Harmouche,
Matthew Toews
Abstract:
This work details a highly efficient implementation of the 3D scale-invariant feature transform (SIFT) algorithm, for the purpose of machine learning from large sets of volumetric medical image data. The primary operations of the 3D SIFT code are implemented on a graphics processing unit (GPU), including convolution, sub-sampling, and 4D peak detection from scale-space pyramids. The performance im…
▽ More
This work details a highly efficient implementation of the 3D scale-invariant feature transform (SIFT) algorithm, for the purpose of machine learning from large sets of volumetric medical image data. The primary operations of the 3D SIFT code are implemented on a graphics processing unit (GPU), including convolution, sub-sampling, and 4D peak detection from scale-space pyramids. The performance improvements are quantified in keypoint detection and image-to-image matching experiments, using 3D MRI human brain volumes of different people. Computationally efficient 3D keypoint descriptors are proposed based on the Binary Robust Independent Elementary Feature (BRIEF) code, including a novel descriptor we call Ranked Robust Independent Elementary Features (RRIEF), and compared to the original 3D SIFT-Rank method\citep{toews2013efficient}. The GPU implementation affords a speedup of approximately 7X beyond an optimised CPU implementation, where computation time is reduced from 1.4 seconds to 0.2 seconds for 3D volumes of size (145, 174, 145) voxels with approximately 3000 keypoints. Notable speedups include the convolution operation (20X), 4D peak detection (3X), sub-sampling (3X), and difference-of-Gaussian pyramid construction (2X). Efficient descriptors offer a speedup of 2X and a memory savings of 6X compared to standard SIFT-Rank descriptors, at a cost of reduced numbers of keypoint correspondences, revealing a trade-off between computational efficiency and algorithmic performance. The speedups gained by our implementation will allow for a more efficient analysis on larger data sets. Our optimized GPU implementation of the 3D SIFT-Rank extractor is available at https://github.com/CarluerJB/3D_SIFT_CUDA.
△ Less
Submitted 19 December, 2021;
originally announced December 2021.
-
Curating Subject ID Labels using Keypoint Signatures
Authors:
Laurent Chauvin,
Matthew Toews
Abstract:
Subject ID labels are unique, anonymized codes that can be used to group all images of a subject while maintaining anonymity. ID errors may be inadvertently introduced manually error during enrollment and may lead to systematic error into machine learning evaluation (e.g. due to double-dip**) or potential patient misdiagnosis in clinical contexts. Here we describe a highly efficient system for c…
▽ More
Subject ID labels are unique, anonymized codes that can be used to group all images of a subject while maintaining anonymity. ID errors may be inadvertently introduced manually error during enrollment and may lead to systematic error into machine learning evaluation (e.g. due to double-dip**) or potential patient misdiagnosis in clinical contexts. Here we describe a highly efficient system for curating subject ID labels in large generic medical image datasets, based on the 3D image keypoint representation, which recently led to the discovery of previously unknown labeling errors in widely-used public brain MRI datasets
△ Less
Submitted 7 October, 2021;
originally announced October 2021.
-
Efficient Pairwise Neuroimage Analysis using the Soft Jaccard Index and 3D Keypoint Sets
Authors:
Laurent Chauvin,
Kuldeep Kumar,
Christian Desrosiers,
William Wells III,
Matthew Toews
Abstract:
We propose a novel pairwise distance measure between image keypoint sets, for the purpose of large-scale medical image indexing. Our measure generalizes the Jaccard index to account for soft set equivalence (SSE) between keypoint elements, via an adaptive kernel framework modeling uncertainty in keypoint appearance and geometry. A new kernel is proposed to quantify the variability of keypoint geom…
▽ More
We propose a novel pairwise distance measure between image keypoint sets, for the purpose of large-scale medical image indexing. Our measure generalizes the Jaccard index to account for soft set equivalence (SSE) between keypoint elements, via an adaptive kernel framework modeling uncertainty in keypoint appearance and geometry. A new kernel is proposed to quantify the variability of keypoint geometry in location and scale. Our distance measure may be estimated between $O(N^2)$ image pairs in $O(N~\log~N)$ operations via keypoint indexing. Experiments report the first results for the task of predicting family relationships from medical images, using 1010 T1-weighted MRI brain volumes of 434 families including monozygotic and dizygotic twins, siblings and half-siblings sharing 100%-25% of their polymorphic genes. Soft set equivalence and the keypoint geometry kernel improve upon standard hard set equivalence (HSE) and appearance kernels alone in predicting family relationships. Monozygotic twin identification is near 100%, and three subjects with uncertain genoty** are automatically paired with their self-reported families, the first reported practical application of image-based family identification. Our distance measure can also be used to predict group categories, sex is predicted with an AUC=0.97. Software is provided for efficient fine-grained curation of large, generic image datasets.
△ Less
Submitted 16 September, 2021; v1 submitted 11 March, 2021;
originally announced March 2021.
-
Large Scale Indexing of Generic Medical Image Data using Unbiased Shallow Keypoints and Deep CNN Features
Authors:
L. Chauvin,
M. Ben Lazreg,
J. B. Carluer,
W. Wells,
M. Toews
Abstract:
We propose a unified appearance model accounting for traditional shallow (i.e. 3D SIFT keypoints) and deep (i.e. CNN output layers) image feature representations, encoding respectively specific, localized neuroanatomical patterns and rich global information into a single indexing and classification framework. A novel Bayesian model combines shallow and deep features based on an assumption of condi…
▽ More
We propose a unified appearance model accounting for traditional shallow (i.e. 3D SIFT keypoints) and deep (i.e. CNN output layers) image feature representations, encoding respectively specific, localized neuroanatomical patterns and rich global information into a single indexing and classification framework. A novel Bayesian model combines shallow and deep features based on an assumption of conditional independence and validated by experiments indexing specific family members and general group categories in 3D MRI neuroimage data of 1010 subjects from the Human Connectome Project, including twins and non-twin siblings. A novel domain adaptation strategy is presented, transforming deep CNN vectors elements into binary class-informative descriptors. A GPU-based implementation of all processing is provided. State-of-the-art performance is achieved in large-scale neuroimage indexing, both in terms of computational complexity, accuracy in identifying family members and sex classification.
△ Less
Submitted 20 October, 2020; v1 submitted 8 October, 2020;
originally announced October 2020.
-
Multi-modal analysis of genetically-related subjects using SIFT descriptors in brain MRI
Authors:
Kuldeep Kumar,
Laurent Chauvin,
Mathew Toews,
Olivier Colliot,
Christian Desrosiers
Abstract:
So far, fingerprinting studies have focused on identifying features from single-modality MRI data, which capture individual characteristics in terms of brain structure, function, or white matter microstructure. However, due to the lack of a framework for comparing across multiple modalities, studies based on multi-modal data remain elusive. This paper presents a multi-modal analysis of genetically…
▽ More
So far, fingerprinting studies have focused on identifying features from single-modality MRI data, which capture individual characteristics in terms of brain structure, function, or white matter microstructure. However, due to the lack of a framework for comparing across multiple modalities, studies based on multi-modal data remain elusive. This paper presents a multi-modal analysis of genetically-related subjects to compare and contrast the information provided by various MRI modalities. The proposed framework represents MRI scans as bags of SIFT features, and uses these features in a nearest-neighbor graph to measure subject similarity. Experiments using the T1/T2-weighted MRI and diffusion MRI data of 861 Human Connectome Project subjects demonstrate strong links between the proposed similarity measure and genetic proximity.
△ Less
Submitted 18 September, 2017;
originally announced September 2017.
-
Integration of the OpenIGTLink Network Protocol for Image-Guided Therapy with the Medical Platform MeVisLab
Authors:
Jan Egger,
Junichi Tokuda,
Laurent Chauvin,
Bernd Freisleben,
Christopher Nimsky,
Tina Kapur,
William M. Wells III
Abstract:
We present the integration of the OpenIGTLink network protocol for image-guided therapy (IGT) with the medical prototy** platform MeVisLab. OpenIGTLink is a new, open, simple and extensible network communication protocol for IGT. The protocol provides a standardized mechanism to connect hardware and software by the transfer of coordinate transforms, images, and status messages. MeVisLab is a fra…
▽ More
We present the integration of the OpenIGTLink network protocol for image-guided therapy (IGT) with the medical prototy** platform MeVisLab. OpenIGTLink is a new, open, simple and extensible network communication protocol for IGT. The protocol provides a standardized mechanism to connect hardware and software by the transfer of coordinate transforms, images, and status messages. MeVisLab is a framework for the development of image processing algorithms and visualization and interaction methods, with a focus on medical imaging. The integration of OpenIGTLink into MeVisLab has been realized by develo** a software module using the C++ programming language. As a result, researchers using MeVisLab can interface their software to hardware devices that already support the OpenIGTLink protocol, such as the NDI Aurora magnetic tracking system. In addition, the OpenIGTLink module can also be used to communicate directly with Slicer, a free, open source software package for visualization and image analysis. The integration has been tested with tracker clients available online and a real tracking system.
△ Less
Submitted 7 September, 2013;
originally announced September 2013.
