-
Multimodal MRI-based Detection of Amyloid Status in Alzheimer's Disease Continuum
Authors:
Giorgio Dolci,
Charles A. Ellis,
Federica Cruciani,
Lorenza Brusini,
Anees Abrol,
Ilaria Boscolo Galazzo,
Gloria Menegaz,
Vince D. Calhoun
Abstract:
Amyloid-$β$ (A$β$) plaques in conjunction with hyperphosphorylated tau proteins in the form of neurofibrillary tangles are the two neuropathological hallmarks of Alzheimer's disease (AD). In particular, the accumulation of A$β$ plaques, as evinced by the A/T/N (amyloid/tau/neurodegeneration) framework, marks the initial stage. Thus, the identification of individuals with A$β$ positivity could enab…
▽ More
Amyloid-$β$ (A$β$) plaques in conjunction with hyperphosphorylated tau proteins in the form of neurofibrillary tangles are the two neuropathological hallmarks of Alzheimer's disease (AD). In particular, the accumulation of A$β$ plaques, as evinced by the A/T/N (amyloid/tau/neurodegeneration) framework, marks the initial stage. Thus, the identification of individuals with A$β$ positivity could enable early diagnosis and potentially lead to more effective interventions. Deep learning methods relying mainly on amyloid PET images have been employed to this end. However, PET imaging has some disadvantages, including the need of radiotracers and expensive acquisitions. Hence, in this work, we propose a novel multimodal approach that integrates information from structural, functional, and diffusion MRI data to discriminate A$β$ status in the AD continuum. Our method achieved an accuracy of $0.762\pm0.04$. Furthermore, a \textit{post-hoc} explainability analysis (guided backpropagation) was performed to retrieve the brain regions that most influenced the model predictions. This analysis identified some key regions that were common across modalities, some of which were well-established AD-discriminative biomarkers and related to A$β$ deposition, such as the hippocampus, thalamus, precuneus, and cingulate gyrus. Hence, our study demonstrates the potential viability of MRI-based characterization of A$β$ status, paving the way for further research in this domain.
△ Less
Submitted 19 June, 2024;
originally announced June 2024.
-
Improving age prediction: Utilizing LSTM-based dynamic forecasting for data augmentation in multivariate time series analysis
Authors:
Yutong Gao,
Charles A. Ellis,
Vince D. Calhoun,
Robyn L. Miller
Abstract:
The high dimensionality and complexity of neuroimaging data necessitate large datasets to develop robust and high-performing deep learning models. However, the neuroimaging field is notably hampered by the scarcity of such datasets. In this work, we proposed a data augmentation and validation framework that utilizes dynamic forecasting with Long Short-Term Memory (LSTM) networks to enrich datasets…
▽ More
The high dimensionality and complexity of neuroimaging data necessitate large datasets to develop robust and high-performing deep learning models. However, the neuroimaging field is notably hampered by the scarcity of such datasets. In this work, we proposed a data augmentation and validation framework that utilizes dynamic forecasting with Long Short-Term Memory (LSTM) networks to enrich datasets. We extended multivariate time series data by predicting the time courses of independent component networks (ICNs) in both one-step and recursive configurations. The effectiveness of these augmented datasets was then compared with the original data using various deep learning models designed for chronological age prediction tasks. The results suggest that our approach improves model performance, providing a robust solution to overcome the challenges presented by the limited size of neuroimaging datasets.
△ Less
Submitted 11 December, 2023;
originally announced December 2023.
-
Machine learning using magnetic stochastic synapses
Authors:
Matthew O. A. Ellis,
Alex Welbourne,
Stephan J. Kyle,
Paul W. Fry,
Dan A. Allwood,
Thomas J. Hayward,
Eleni Vasilaki
Abstract:
The impressive performance of artificial neural networks has come at the cost of high energy usage and CO$_2$ emissions. Unconventional computing architectures, with magnetic systems as a candidate, have potential as alternative energy-efficient hardware, but, still face challenges, such as stochastic behaviour, in implementation. Here, we present a methodology for exploiting the traditionally det…
▽ More
The impressive performance of artificial neural networks has come at the cost of high energy usage and CO$_2$ emissions. Unconventional computing architectures, with magnetic systems as a candidate, have potential as alternative energy-efficient hardware, but, still face challenges, such as stochastic behaviour, in implementation. Here, we present a methodology for exploiting the traditionally detrimental stochastic effects in magnetic domain-wall motion in nanowires. We demonstrate functional binary stochastic synapses alongside a gradient learning rule that allows their training with applicability to a range of stochastic systems. The rule, utilising the mean and variance of the neuronal output distribution, finds a trade-off between synaptic stochasticity and energy efficiency depending on the number of measurements of each synapse. For single measurements, the rule results in binary synapses with minimal stochasticity, sacrificing potential performance for robustness. For multiple measurements, synaptic distributions are broad, approximating better-performing continuous synapses. This observation allows us to choose design principles depending on the desired performance and the device's operational speed and energy cost. We verify performance on physical hardware, showing it is comparable to a standard neural network.
△ Less
Submitted 3 March, 2023;
originally announced March 2023.
-
A perspective on physical reservoir computing with nanomagnetic devices
Authors:
Dan A Allwood,
Matthew O A Ellis,
David Griffin,
Thomas J Hayward,
Luca Manneschi,
Mohammad F KH Musameh,
Simon O'Keefe,
Susan Stepney,
Charles Swindells,
Martin A Trefzer,
Eleni Vasilaki,
Guru Venkat,
Ian Vidamour,
Chester Wringe
Abstract:
Neural networks have revolutionized the area of artificial intelligence and introduced transformative applications to almost every scientific field and industry. However, this success comes at a great price; the energy requirements for training advanced models are unsustainable. One promising way to address this pressing issue is by develo** low-energy neuromorphic hardware that directly support…
▽ More
Neural networks have revolutionized the area of artificial intelligence and introduced transformative applications to almost every scientific field and industry. However, this success comes at a great price; the energy requirements for training advanced models are unsustainable. One promising way to address this pressing issue is by develo** low-energy neuromorphic hardware that directly supports the algorithm's requirements. The intrinsic non-volatility, non-linearity, and memory of spintronic devices make them appealing candidates for neuromorphic devices. Here we focus on the reservoir computing paradigm, a recurrent network with a simple training algorithm suitable for computation with spintronic devices since they can provide the properties of non-linearity and memory. We review technologies and methods for develo** neuromorphic spintronic devices and conclude with critical open issues to address before such devices become widely used.
△ Less
Submitted 9 December, 2022;
originally announced December 2022.
-
Environmental Sensor Placement with Convolutional Gaussian Neural Processes
Authors:
Tom R. Andersson,
Wessel P. Bruinsma,
Stratis Markou,
James Requeima,
Alejandro Coca-Castro,
Anna Vaughan,
Anna-Louise Ellis,
Matthew A. Lazzara,
Dani Jones,
J. Scott Hosking,
Richard E. Turner
Abstract:
Environmental sensors are crucial for monitoring weather conditions and the impacts of climate change. However, it is challenging to place sensors in a way that maximises the informativeness of their measurements, particularly in remote regions like Antarctica. Probabilistic machine learning models can suggest informative sensor placements by finding sites that maximally reduce prediction uncertai…
▽ More
Environmental sensors are crucial for monitoring weather conditions and the impacts of climate change. However, it is challenging to place sensors in a way that maximises the informativeness of their measurements, particularly in remote regions like Antarctica. Probabilistic machine learning models can suggest informative sensor placements by finding sites that maximally reduce prediction uncertainty. Gaussian process (GP) models are widely used for this purpose, but they struggle with capturing complex non-stationary behaviour and scaling to large datasets. This paper proposes using a convolutional Gaussian neural process (ConvGNP) to address these issues. A ConvGNP uses neural networks to parameterise a joint Gaussian distribution at arbitrary target locations, enabling flexibility and scalability. Using simulated surface air temperature anomaly over Antarctica as training data, the ConvGNP learns spatial and seasonal non-stationarities, outperforming a non-stationary GP baseline. In a simulated sensor placement experiment, the ConvGNP better predicts the performance boost obtained from new observations than GP baselines, leading to more informative sensor placements. We contrast our approach with physics-based sensor placement methods and propose future steps towards an operational sensor placement recommendation system. Our work could help to realise environmental digital twins that actively direct measurement sampling to improve the digital representation of reality.
△ Less
Submitted 15 May, 2023; v1 submitted 18 November, 2022;
originally announced November 2022.
-
Domain Adaptation: the Key Enabler of Neural Network Equalizers in Coherent Optical Systems
Authors:
Pedro J. Freire,
Bernhard Spinnler,
Daniel Abode,
Jaroslaw E. Prilepsky,
Abdallah A. I. Ali,
Nelson Costa,
Wolfgang Schairer,
Antonio Napoli,
Andrew D. Ellis,
Sergei K. Turitsyn
Abstract:
We introduce the domain adaptation and randomization approach for calibrating neural network-based equalizers for real transmissions, using synthetic data. The approach renders up to 99\% training process reduction, which we demonstrate in three experimental setups.
We introduce the domain adaptation and randomization approach for calibrating neural network-based equalizers for real transmissions, using synthetic data. The approach renders up to 99\% training process reduction, which we demonstrate in three experimental setups.
△ Less
Submitted 25 February, 2022;
originally announced February 2022.
-
Quantifying the Computational Capability of a Nanomagnetic Reservoir Computing Platform with Emergent Magnetization Dynamics
Authors:
Ian T Vidamour,
Matthew O A Ellis,
David Griffin,
Guru Venkat,
Charles Swindells,
Richard W S Dawidek,
Thomas J Broomhall,
Nina-Juliane Steinke,
Joshaniel F K Cooper,
Francisco Maccherozzi,
Sarnjeet S Dhesi,
Susan Stepney,
Eleni Vasilaki,
Dan A Allwood,
Thomas J Hayward
Abstract:
Devices based on arrays of interconnected magnetic nano-rings with emergent magnetization dynamics have recently been proposed for use in reservoir computing applications, but for them to be computationally useful it must be possible to optimise their dynamical responses. Here, we use a phenomenological model to demonstrate that such reservoirs can be optimised for classification tasks by tuning h…
▽ More
Devices based on arrays of interconnected magnetic nano-rings with emergent magnetization dynamics have recently been proposed for use in reservoir computing applications, but for them to be computationally useful it must be possible to optimise their dynamical responses. Here, we use a phenomenological model to demonstrate that such reservoirs can be optimised for classification tasks by tuning hyperparameters that control the scaling and input rate of data into the system using rotating magnetic fields. We use task-independent metrics to assess the rings' computational capabilities at each set of these hyperparameters and show how these metrics correlate directly to performance in spoken and written digit recognition tasks. We then show that these metrics, and performance in tasks, can be further improved by expanding the reservoir's output to include multiple, concurrent measures of the ring arrays magnetic states.
△ Less
Submitted 31 January, 2022; v1 submitted 29 November, 2021;
originally announced November 2021.
-
Algorithm-Agnostic Explainability for Unsupervised Clustering
Authors:
Charles A. Ellis,
Mohammad S. E. Sendi,
Eloy P. T. Geenjaar,
Sergey M. Plis,
Robyn L. Miller,
Vince D. Calhoun
Abstract:
Supervised machine learning explainability has developed rapidly in recent years. However, clustering explainability has lagged behind. Here, we demonstrate the first adaptation of model-agnostic explainability methods to explain unsupervised clustering. We present two novel "algorithm-agnostic" explainability methods - global permutation percent change (G2PC) and local perturbation percent change…
▽ More
Supervised machine learning explainability has developed rapidly in recent years. However, clustering explainability has lagged behind. Here, we demonstrate the first adaptation of model-agnostic explainability methods to explain unsupervised clustering. We present two novel "algorithm-agnostic" explainability methods - global permutation percent change (G2PC) and local perturbation percent change (L2PC) - that identify feature importance globally to a clustering algorithm and locally to the clustering of individual samples. The methods are (1) easy to implement and (2) broadly applicable across clustering algorithms, which could make them highly impactful. We demonstrate the utility of the methods for explaining five popular clustering methods on low-dimensional synthetic datasets and on high-dimensional functional network connectivity data extracted from a resting-state functional magnetic resonance imaging dataset of 151 individuals with schizophrenia and 160 controls. Our results are consistent with existing literature while also shedding new light on how changes in brain connectivity may lead to schizophrenia symptoms. We further compare the explanations from our methods to an interpretable classifier and find them to be highly similar. Our proposed methods robustly explain multiple clustering algorithms and could facilitate new insights into many applications. We hope this study will greatly accelerate the development of the field of clustering explainability.
△ Less
Submitted 28 August, 2021; v1 submitted 17 May, 2021;
originally announced May 2021.
-
Exploiting Multiple Timescales in Hierarchical Echo State Networks
Authors:
Luca Manneschi,
Matthew O. A. Ellis,
Guido Gigante,
Andrew C. Lin,
Paolo Del Giudice,
Eleni Vasilaki
Abstract:
Echo state networks (ESNs) are a powerful form of reservoir computing that only require training of linear output weights whilst the internal reservoir is formed of fixed randomly connected neurons. With a correctly scaled connectivity matrix, the neurons' activity exhibits the echo-state property and responds to the input dynamics with certain timescales. Tuning the timescales of the network can…
▽ More
Echo state networks (ESNs) are a powerful form of reservoir computing that only require training of linear output weights whilst the internal reservoir is formed of fixed randomly connected neurons. With a correctly scaled connectivity matrix, the neurons' activity exhibits the echo-state property and responds to the input dynamics with certain timescales. Tuning the timescales of the network can be necessary for treating certain tasks, and some environments require multiple timescales for an efficient representation. Here we explore the timescales in hierarchical ESNs, where the reservoir is partitioned into two smaller linked reservoirs with distinct properties. Over three different tasks (NARMA10, a reconstruction task in a volatile environment, and psMNIST), we show that by selecting the hyper-parameters of each partition such that they focus on different timescales, we achieve a significant performance improvement over a single ESN. Through a linear analysis, and under the assumption that the timescales of the first partition are much shorter than the second's (typically corresponding to optimal operating conditions), we interpret the feedforward coupling of the partitions in terms of an effective representation of the input signal, provided by the first partition to the second, whereby the instantaneous input signal is expanded into a weighted combination of its time derivatives. Furthermore, we propose a data-driven approach to optimise the hyper-parameters through a gradient descent optimisation method that is an online approximation of backpropagation through time. We demonstrate the application of the online learning rule across all the tasks considered.
△ Less
Submitted 2 August, 2021; v1 submitted 11 January, 2021;
originally announced January 2021.
-
Accelerating Finite-temperature Kohn-Sham Density Functional Theory with Deep Neural Networks
Authors:
J. Austin Ellis,
Lenz Fiedler,
Gabriel A. Popoola,
Normand A. Modine,
J. Adam Stephens,
Aidan P. Thompson,
Attila Cangi,
Sivasankaran Rajamanickam
Abstract:
We present a numerical modeling workflow based on machine learning (ML) which reproduces the the total energies produced by Kohn-Sham density functional theory (DFT) at finite electronic temperature to within chemical accuracy at negligible computational cost. Based on deep neural networks, our workflow yields the local density of states (LDOS) for a given atomic configuration. From the LDOS, spat…
▽ More
We present a numerical modeling workflow based on machine learning (ML) which reproduces the the total energies produced by Kohn-Sham density functional theory (DFT) at finite electronic temperature to within chemical accuracy at negligible computational cost. Based on deep neural networks, our workflow yields the local density of states (LDOS) for a given atomic configuration. From the LDOS, spatially-resolved, energy-resolved, and integrated quantities can be calculated, including the DFT total free energy, which serves as the Born-Oppenheimer potential energy surface for the atoms. We demonstrate the efficacy of this approach for both solid and liquid metals and compare results between independent and unified machine-learning models for solid and liquid aluminum. Our machine-learning density functional theory framework opens up the path towards multiscale materials modeling for matter under ambient and extreme conditions at a computational scale and cost that is unattainable with current algorithms.
△ Less
Submitted 9 July, 2021; v1 submitted 10 October, 2020;
originally announced October 2020.
-
Impact of Integrated Circuit Packaging on Synaptic Dynamics of Memristive Devices
Authors:
Aidana Irmanova,
Grant A. Ellis,
Alex Pappachen James
Abstract:
The memristor can be used as non volatile memory (NVM) and for emulating neuron behavior. It has the ability to switch between low resistance $R_{on}$ and high resistance values $R_{off}$, and exhibit the synaptic dynamic behaviour such as potentiation and depression. This paper presents a study on potentiation and depression of memristors in Quad Flat Pack. A comparison is drawn between the memri…
▽ More
The memristor can be used as non volatile memory (NVM) and for emulating neuron behavior. It has the ability to switch between low resistance $R_{on}$ and high resistance values $R_{off}$, and exhibit the synaptic dynamic behaviour such as potentiation and depression. This paper presents a study on potentiation and depression of memristors in Quad Flat Pack. A comparison is drawn between the memristors with and without the impact of parasitics of packaging, using measured data and equivalent circuit models. The parameters in memristor and packaging models for the SPICE simulations were determined using measured data to reflect the memristor parasitics in Quad Flat Packs.
△ Less
Submitted 27 September, 2018;
originally announced September 2018.
