-
X-Diffusion: Generating Detailed 3D MRI Volumes From a Single Image Using Cross-Sectional Diffusion Models
Authors:
Emmanuelle Bourigault,
Abdullah Hamdi,
Amir Jamaludin
Abstract:
In this work, we present X-Diffusion, a cross-sectional diffusion model tailored for Magnetic Resonance Imaging (MRI) data. X-Diffusion is capable of generating the entire MRI volume from just a single MRI slice or optionally from few multiple slices, setting new benchmarks in the precision of synthesized MRIs from extremely sparse observations. The uniqueness lies in the novel view-conditional tr…
▽ More
In this work, we present X-Diffusion, a cross-sectional diffusion model tailored for Magnetic Resonance Imaging (MRI) data. X-Diffusion is capable of generating the entire MRI volume from just a single MRI slice or optionally from few multiple slices, setting new benchmarks in the precision of synthesized MRIs from extremely sparse observations. The uniqueness lies in the novel view-conditional training and inference of X-Diffusion on MRI volumes, allowing for generalized MRI learning. Our evaluations span both brain tumour MRIs from the BRATS dataset and full-body MRIs from the UK Biobank dataset. Utilizing the paired pre-registered Dual-energy X-ray Absorptiometry (DXA) and MRI modalities in the UK Biobank dataset, X-Diffusion is able to generate detailed 3D MRI volume from a single full-body DXA. Remarkably, the resultant MRIs not only stand out in precision on unseen examples (surpassing state-of-the-art results by large margins) but also flawlessly retain essential features of the original MRI, including tumour profiles, spine curvature, brain volume, and beyond. Furthermore, the trained X-Diffusion model on the MRI datasets attains a generalization capacity out-of-domain (e.g. generating knee MRIs even though it is trained on brains). The code is available on the project website https://emmanuelleb985.github.io/XDiffusion/ .
△ Less
Submitted 30 April, 2024;
originally announced April 2024.
-
Predicting Spine Geometry and Scoliosis from DXA Scans
Authors:
Amir Jamaludin,
Timor Kadir,
Emma Clark,
Andrew Zisserman
Abstract:
Our objective in this paper is to estimate spine curvature in DXA scans. To this end we first train a neural network to predict the middle spine curve in the scan, and then use an integral-based method to determine the curvature along the spine curve. We use the curvature to compare to the standard angle scoliosis measure obtained using the DXA Scoliosis Method (DSM). The performance improves over…
▽ More
Our objective in this paper is to estimate spine curvature in DXA scans. To this end we first train a neural network to predict the middle spine curve in the scan, and then use an integral-based method to determine the curvature along the spine curve. We use the curvature to compare to the standard angle scoliosis measure obtained using the DXA Scoliosis Method (DSM). The performance improves over the prior work of Jamaludin et al. 2018. We show that the maximum curvature can be used as a scoring function for ordering the severity of spinal deformation.
△ Less
Submitted 15 November, 2023;
originally announced November 2023.
-
Contouring by Unit Vector Field Regression
Authors:
Amir Jamaludin,
Sarim Ather,
Timor Kadir,
Rhydian Windsor
Abstract:
This work introduces a simple deep-learning based method to delineate contours by `walking' along learnt unit vector fields. We demonstrate the effectiveness of our pipeline on the unique case of open contours on the task of delineating the sacroiliac joints (SIJs) in spinal MRIs. We show that: (i) 95% of the time the average root mean square error of the predicted contour against the original gro…
▽ More
This work introduces a simple deep-learning based method to delineate contours by `walking' along learnt unit vector fields. We demonstrate the effectiveness of our pipeline on the unique case of open contours on the task of delineating the sacroiliac joints (SIJs) in spinal MRIs. We show that: (i) 95% of the time the average root mean square error of the predicted contour against the original ground truth is below 4.5 pixels (2.5mm for a standard T1-weighted SIJ MRI), and (ii) the proposed method is better than the baseline of regressing vertices or landmarks of contours.
△ Less
Submitted 26 May, 2023;
originally announced May 2023.
-
Vision-Language Modelling For Radiological Imaging and Reports In The Low Data Regime
Authors:
Rhydian Windsor,
Amir Jamaludin,
Timor Kadir,
Andrew Zisserman
Abstract:
This paper explores training medical vision-language models (VLMs) -- where the visual and language inputs are embedded into a common space -- with a particular focus on scenarios where training data is limited, as is often the case in clinical datasets. We explore several candidate methods to improve low-data performance, including: (i) adapting generic pre-trained models to novel image and text…
▽ More
This paper explores training medical vision-language models (VLMs) -- where the visual and language inputs are embedded into a common space -- with a particular focus on scenarios where training data is limited, as is often the case in clinical datasets. We explore several candidate methods to improve low-data performance, including: (i) adapting generic pre-trained models to novel image and text domains (i.e. medical imaging and reports) via unimodal self-supervision; (ii) using local (e.g. GLoRIA) & global (e.g. InfoNCE) contrastive loss functions as well as a combination of the two; (iii) extra supervision during VLM training, via: (a) image- and text-only self-supervision, and (b) creating additional positive image-text pairs for training through augmentation and nearest-neighbour search.
Using text-to-image retrieval as a benchmark, we evaluate the performance of these methods with variable sized training datasets of paired chest X-rays and radiological reports. Combined, they significantly improve retrieval compared to fine-tuning CLIP, roughly equivalent to training with the data. A similar pattern is found in the downstream task classification of CXR-related conditions with our method outperforming CLIP and also BioVIL, a strong CXR VLM benchmark, in the zero-shot and linear probing settings. We conclude with a set of recommendations for researchers aiming to train vision-language models on other medical imaging modalities when training data is scarce. To facilitate further research, we will make our code and models publicly available.
△ Less
Submitted 30 March, 2023;
originally announced March 2023.
-
Low progress math in a high performing system
Authors:
A. Jamaludin,
A. I. Jabir,
F. J. Wang,
A. L. Tan
Abstract:
Math anxiety negatively relates to math performance. This negative relationship may be exacerbated in low-progress math learners. However, there are limited studies on math anxiety among low progress learners in a paradoxically high performing education system like Singapore. To fill this research gap, this research analysed the anxiety profiles of 151 students who were in the math learning suppor…
▽ More
Math anxiety negatively relates to math performance. This negative relationship may be exacerbated in low-progress math learners. However, there are limited studies on math anxiety among low progress learners in a paradoxically high performing education system like Singapore. To fill this research gap, this research analysed the anxiety profiles of 151 students who were in the math learning support intervention program administered by the Ministry of Education, Singapore (MOE). We examined the complex relationship centred in math anxiety with relevant variables such as demographic characteristics, working memory and math performance. Limitations and future directions are discussed.
△ Less
Submitted 25 January, 2023;
originally announced February 2023.
-
Behavioural predictors of math anxiety
Authors:
M. Y. K. Chen,
A. Jamaludin,
A. L. Tan
Abstract:
Math anxiety is a highly prevalent problem in education that has consistently shown to lead to poor math performance. This study sought to investigate whether certain behaviours are predictive of math anxiety among students. This study involved elementary school students who were low-progressing in math, and is part of an educational intervention program. Ten classifications types of behavioural i…
▽ More
Math anxiety is a highly prevalent problem in education that has consistently shown to lead to poor math performance. This study sought to investigate whether certain behaviours are predictive of math anxiety among students. This study involved elementary school students who were low-progressing in math, and is part of an educational intervention program. Ten classifications types of behavioural indicators were identified, such as counting out loud. A multiple linear regression was conducted, identifying three behavioural observations that were positively and significantly associated with their math anxiety. Implications and limitations are discussed.
△ Less
Submitted 29 January, 2023;
originally announced January 2023.
-
Context-Aware Transformers For Spinal Cancer Detection and Radiological Grading
Authors:
Rhydian Windsor,
Amir Jamaludin,
Timor Kadir,
Andrew Zisserman
Abstract:
This paper proposes a novel transformer-based model architecture for medical imaging problems involving analysis of vertebrae. It considers two applications of such models in MR images: (a) detection of spinal metastases and the related conditions of vertebral fractures and metastatic cord compression, (b) radiological grading of common degenerative changes in intervertebral discs. Our contributio…
▽ More
This paper proposes a novel transformer-based model architecture for medical imaging problems involving analysis of vertebrae. It considers two applications of such models in MR images: (a) detection of spinal metastases and the related conditions of vertebral fractures and metastatic cord compression, (b) radiological grading of common degenerative changes in intervertebral discs. Our contributions are as follows: (i) We propose a Spinal Context Transformer (SCT), a deep-learning architecture suited for the analysis of repeated anatomical structures in medical imaging such as vertebral bodies (VBs). Unlike previous related methods, SCT considers all VBs as viewed in all available image modalities together, making predictions for each based on context from the rest of the spinal column and all available imaging modalities. (ii) We apply the architecture to a novel and important task: detecting spinal metastases and the related conditions of cord compression and vertebral fractures/collapse from multi-series spinal MR scans. This is done using annotations extracted from free-text radiological reports as opposed to bespoke annotation. However, the resulting model shows strong agreement with vertebral-level bespoke radiologist annotations on the test set. (iii) We also apply SCT to an existing problem: radiological grading of inter-vertebral discs (IVDs) in lumbar MR scans for common degenerative changes.We show that by considering the context of vertebral bodies in the image, SCT improves the accuracy for several gradings compared to previously published model.
△ Less
Submitted 27 June, 2022;
originally announced June 2022.
-
SpineNetV2: Automated Detection, Labelling and Radiological Grading Of Clinical MR Scans
Authors:
Rhydian Windsor,
Amir Jamaludin,
Timor Kadir,
Andrew Zisserman
Abstract:
This technical report presents SpineNetV2, an automated tool which: (i) detects and labels vertebral bodies in clinical spinal magnetic resonance (MR) scans across a range of commonly used sequences; and (ii) performs radiological grading of lumbar intervertebral discs in T2-weighted scans for a range of common degenerative changes. SpineNetV2 improves over the original SpineNet software in two wa…
▽ More
This technical report presents SpineNetV2, an automated tool which: (i) detects and labels vertebral bodies in clinical spinal magnetic resonance (MR) scans across a range of commonly used sequences; and (ii) performs radiological grading of lumbar intervertebral discs in T2-weighted scans for a range of common degenerative changes. SpineNetV2 improves over the original SpineNet software in two ways: (1) The vertebral body detection stage is significantly faster, more accurate and works across a range of fields-of-view (as opposed to just lumbar scans). (2) Radiological grading adopts a more powerful architecture, adding several new grading schemes without loss in performance. A demo of the software is available at the project website: http://zeus.robots.ox.ac.uk/spinenet2/.
△ Less
Submitted 3 May, 2022;
originally announced May 2022.
-
Self-Supervised Multi-Modal Alignment for Whole Body Medical Imaging
Authors:
Rhydian Windsor,
Amir Jamaludin,
Timor Kadir,
Andrew Zisserman
Abstract:
This paper explores the use of self-supervised deep learning in medical imaging in cases where two scan modalities are available for the same subject. Specifically, we use a large publicly-available dataset of over 20,000 subjects from the UK Biobank with both whole body Dixon technique magnetic resonance (MR) scans and also dual-energy x-ray absorptiometry (DXA) scans. We make three contributions…
▽ More
This paper explores the use of self-supervised deep learning in medical imaging in cases where two scan modalities are available for the same subject. Specifically, we use a large publicly-available dataset of over 20,000 subjects from the UK Biobank with both whole body Dixon technique magnetic resonance (MR) scans and also dual-energy x-ray absorptiometry (DXA) scans. We make three contributions: (i) We introduce a multi-modal image-matching contrastive framework, that is able to learn to match different-modality scans of the same subject with high accuracy. (ii) Without any adaption, we show that the correspondences learnt during this contrastive training step can be used to perform automatic cross-modal scan registration in a completely unsupervised manner. (iii) Finally, we use these registrations to transfer segmentation maps from the DXA scans to the MR scans where they are used to train a network to segment anatomical regions without requiring ground-truth MR examples. To aid further research, our code will be made publicly available.
△ Less
Submitted 6 August, 2021; v1 submitted 14 July, 2021;
originally announced July 2021.
-
A Deep Learning Approach to Private Data Sharing of Medical Images Using Conditional GANs
Authors:
Hanxi Sun,
Jason Plawinski,
Sajanth Subramaniam,
Amir Jamaludin,
Timor Kadir,
Aimee Readie,
Gregory Ligozio,
David Ohlssen,
Mark Baillie,
Thibaud Coroller
Abstract:
Sharing data from clinical studies can facilitate innovative data-driven research and ultimately lead to better public health. However, sharing biomedical data can put sensitive personal information at risk. This is usually solved by anonymization, which is a slow and expensive process. An alternative to anonymization is sharing a synthetic dataset that bears a behaviour similar to the real data b…
▽ More
Sharing data from clinical studies can facilitate innovative data-driven research and ultimately lead to better public health. However, sharing biomedical data can put sensitive personal information at risk. This is usually solved by anonymization, which is a slow and expensive process. An alternative to anonymization is sharing a synthetic dataset that bears a behaviour similar to the real data but preserves privacy. As part of the collaboration between Novartis and the Oxford Big Data Institute, we generate a synthetic dataset based on COSENTYX (secukinumab) Ankylosing Spondylitis clinical study. We apply an Auxiliary Classifier GAN to generate synthetic MRIs of vertebral units. The images are conditioned on the VU location (cervical, thoracic and lumbar). In this paper, we present a method for generating a synthetic dataset and conduct an in-depth analysis on its properties along three key metrics: image fidelity, sample diversity and dataset privacy.
△ Less
Submitted 19 August, 2021; v1 submitted 24 June, 2021;
originally announced June 2021.
-
A Convolutional Approach to Vertebrae Detection and Labelling in Whole Spine MRI
Authors:
Rhydian Windsor,
Amir Jamaludin,
Timor Kadir,
Andrew Zisserman
Abstract:
We propose a novel convolutional method for the detection and identification of vertebrae in whole spine MRIs. This involves using a learnt vector field to group detected vertebrae corners together into individual vertebral bodies and convolutional image-to-image translation followed by beam search to label vertebral levels in a self-consistent manner. The method can be applied without modificatio…
▽ More
We propose a novel convolutional method for the detection and identification of vertebrae in whole spine MRIs. This involves using a learnt vector field to group detected vertebrae corners together into individual vertebral bodies and convolutional image-to-image translation followed by beam search to label vertebral levels in a self-consistent manner. The method can be applied without modification to lumbar, cervical and thoracic-only scans across a range of different MR sequences. The resulting system achieves 98.1% detection rate and 96.5% identification rate on a challenging clinical dataset of whole spine scans and matches or exceeds the performance of previous systems on lumbar-only scans. Finally, we demonstrate the clinical applicability of this method, using it for automated scoliosis detection in both lumbar and whole spine MR scans.
△ Less
Submitted 13 July, 2020; v1 submitted 6 July, 2020;
originally announced July 2020.
-
The Ladder Algorithm: Finding Repetitive Structures in Medical Images by Induction
Authors:
Rhydian Windsor,
Amir Jamaludin
Abstract:
In this paper we introduce the Ladder Algorithm; a novel recurrent algorithm to detect repetitive structures in natural images with high accuracy using little training data.
We then demonstrate the algorithm on the task of extracting vertebrae from whole spine magnetic resonance scans with only lumbar MR scans for training data. It is shown to achieve high perforamance with 99.8% precision and r…
▽ More
In this paper we introduce the Ladder Algorithm; a novel recurrent algorithm to detect repetitive structures in natural images with high accuracy using little training data.
We then demonstrate the algorithm on the task of extracting vertebrae from whole spine magnetic resonance scans with only lumbar MR scans for training data. It is shown to achieve high perforamance with 99.8% precision and recall, exceeding current state of the art approaches for lumbar vertebrae detection in T1 and T2 weighted scans. It also generalises without retraining to whole spine images with minimal drop in accuracy, achieving 99.4% detection rate.
△ Less
Submitted 8 April, 2020; v1 submitted 30 January, 2020;
originally announced January 2020.
-
Self-Supervised Learning for Spinal MRIs
Authors:
Amir Jamaludin,
Timor Kadir,
Andrew Zisserman
Abstract:
A significant proportion of patients scanned in a clinical setting have follow-up scans. We show in this work that such longitudinal scans alone can be used as a form of 'free' self-supervision for training a deep network. We demonstrate this self-supervised learning for the case of T2-weighted sagittal lumbar Magnetic Resonance Images (MRIs). A Siamese convolutional neural network (CNN) is traine…
▽ More
A significant proportion of patients scanned in a clinical setting have follow-up scans. We show in this work that such longitudinal scans alone can be used as a form of 'free' self-supervision for training a deep network. We demonstrate this self-supervised learning for the case of T2-weighted sagittal lumbar Magnetic Resonance Images (MRIs). A Siamese convolutional neural network (CNN) is trained using two losses: (i) a contrastive loss on whether the scan is of the same person (i.e. longitudinal) or not, together with (ii) a classification loss on predicting the level of vertebral bodies. The performance of this pre-trained network is then assessed on a grading classification task. We experiment on a dataset of 1016 subjects, 423 possessing follow-up scans, with the end goal of learning the disc degeneration radiological gradings attached to the intervertebral discs. We show that the performance of the pre-trained CNN on the supervised classification task is (i) superior to that of a network trained from scratch; and (ii) requires far fewer annotated training samples to reach an equivalent performance to that of the network trained from scratch.
△ Less
Submitted 1 August, 2017;
originally announced August 2017.
-
You said that?
Authors:
Joon Son Chung,
Amir Jamaludin,
Andrew Zisserman
Abstract:
We present a method for generating a video of a talking face. The method takes as inputs: (i) still images of the target face, and (ii) an audio speech segment; and outputs a video of the target face lip synched with the audio. The method runs in real time and is applicable to faces and audio not seen at training time.
To achieve this we propose an encoder-decoder CNN model that uses a joint emb…
▽ More
We present a method for generating a video of a talking face. The method takes as inputs: (i) still images of the target face, and (ii) an audio speech segment; and outputs a video of the target face lip synched with the audio. The method runs in real time and is applicable to faces and audio not seen at training time.
To achieve this we propose an encoder-decoder CNN model that uses a joint embedding of the face and audio to generate synthesised talking face video frames. The model is trained on tens of hours of unlabelled videos.
We also show results of re-dubbing videos using speech from a different person.
△ Less
Submitted 18 July, 2017; v1 submitted 8 May, 2017;
originally announced May 2017.
-
Using Mathematica & Matlab for CAGD/CAD research and education
Authors:
R. U. Gobithaasan,
M. A Jamaludin
Abstract:
In CAGD/CAD research and education, users are involved with development of mathematical algorithms and followed by the analysis of the resultant algorithm. This process involves geometric display which can only be carried out with high end graphics display. There are many approaches practiced and one of the so-called easiest approaches is by using C/C++ programming language and OpenGL application…
▽ More
In CAGD/CAD research and education, users are involved with development of mathematical algorithms and followed by the analysis of the resultant algorithm. This process involves geometric display which can only be carried out with high end graphics display. There are many approaches practiced and one of the so-called easiest approaches is by using C/C++ programming language and OpenGL application program interface, API. There are practitioners uses C/C++ programming language to develop the algorithms and finally utilize AutoCAD for graphics display. On the other hand, high end CAD users manage to use Auto Lisp as their programming language in AutoCAD. Nevertheless, these traditional ways are definitely time consuming. This paper introduces an alternative method whereby the practitioners may maximize scientific computation programs, SCPs: Mathematica and MATLAB in the context of CAGD/CAD for research and education.
△ Less
Submitted 13 March, 2013; v1 submitted 12 March, 2013;
originally announced March 2013.
-
Bright solitary waves of atomic Bose-Einstein condensates under rotation
Authors:
N. A. Jamaludin,
N. G. Parker,
A. M. Martin
Abstract:
We analyse the rotation of bright solitary waves formed of atomic Bose-Einstein condensates with attractive atomic interactions. By employing a variational technique and assuming an irrotational quadrupolar flow field, we map out the variational solutions in the rotating frame. In particular, we show that rotation has a considerable stabilising effect on the system, significantly raising the cri…
▽ More
We analyse the rotation of bright solitary waves formed of atomic Bose-Einstein condensates with attractive atomic interactions. By employing a variational technique and assuming an irrotational quadrupolar flow field, we map out the variational solutions in the rotating frame. In particular, we show that rotation has a considerable stabilising effect on the system, significantly raising the critical threshold for collapse of the bright solitary waves.
△ Less
Submitted 28 February, 2008;
originally announced February 2008.
