-
Neuroformer: Multimodal and Multitask Generative Pretraining for Brain Data
Authors:
Antonis Antoniades,
Yiyi Yu,
Joseph Canzano,
William Wang,
Spencer LaVere Smith
Abstract:
State-of-the-art systems neuroscience experiments yield large-scale multimodal data, and these data sets require new tools for analysis. Inspired by the success of large pretrained models in vision and language domains, we reframe the analysis of large-scale, cellular-resolution neuronal spiking data into an autoregressive spatiotemporal generation problem. Neuroformer is a multimodal, multitask g…
▽ More
State-of-the-art systems neuroscience experiments yield large-scale multimodal data, and these data sets require new tools for analysis. Inspired by the success of large pretrained models in vision and language domains, we reframe the analysis of large-scale, cellular-resolution neuronal spiking data into an autoregressive spatiotemporal generation problem. Neuroformer is a multimodal, multitask generative pretrained transformer (GPT) model that is specifically designed to handle the intricacies of data in systems neuroscience. It scales linearly with feature size, can process an arbitrary number of modalities, and is adaptable to downstream tasks, such as predicting behavior. We first trained Neuroformer on simulated datasets, and found that it both accurately predicted simulated neuronal circuit activity, and also intrinsically inferred the underlying neural circuit connectivity, including direction. When pretrained to decode neural responses, the model predicted the behavior of a mouse with only few-shot fine-tuning, suggesting that the model begins learning how to do so directly from the neural representations themselves, without any explicit supervision. We used an ablation study to show that joint training on neuronal responses and behavior boosted performance, highlighting the model's ability to associate behavioral and neural representations in an unsupervised manner. These findings show that Neuroformer can analyze neural datasets and their emergent properties, informing the development of models and hypotheses associated with the brain.
△ Less
Submitted 15 March, 2024; v1 submitted 31 October, 2023;
originally announced November 2023.
-
Towards Monitoring Parkinson's Disease Following Drug Treatment: CGP Classification of rs-MRI Data
Authors:
Amir Dehsarvi,
Jennifer Kay South Palomares,
Stephen Leslie Smith
Abstract:
Background and Objective: It is commonly accepted that accurate monitoring of neurodegenerative diseases is crucial for effective disease management and delivery of medication and treatment. This research develops automatic clinical monitoring techniques for PD, following treatment, using the novel application of EAs. Specifically, the research question addressed was: Can accurate monitoring of PD…
▽ More
Background and Objective: It is commonly accepted that accurate monitoring of neurodegenerative diseases is crucial for effective disease management and delivery of medication and treatment. This research develops automatic clinical monitoring techniques for PD, following treatment, using the novel application of EAs. Specifically, the research question addressed was: Can accurate monitoring of PD be achieved using EAs on rs-fMRI data for patients prescribed Modafinil (typically prescribed for PD patients to relieve physical fatigue)? Methods: This research develops novel clinical monitoring tools using data from a controlled experiment where participants were administered Modafinil versus placebo, examining the novel application of EAs to both map and predict the functional connectivity in participants using rs-fMRI data. Specifically, CGP was used to classify DCM analysis and timeseries data. Results were validated with two other commonly used classification methods (ANN and SVM) and via k-fold cross-validation. Results: Findings revealed a maximum accuracy of 74.57% for CGP. Furthermore, CGP provided comparable performance accuracy relative to ANN and SVM. Nevertheless, EAs enable us to decode the classifier, in terms of understanding the data inputs that are used, more easily than in ANN and SVM. Conclusions: These findings underscore the applicability of both DCM analyses for classification and CGP as a novel classification technique for brain imaging data with medical implications for medication monitoring. Furthermore, classification of fMRI data for research typically involves statistical modelling techniques being often hypothesis driven, whereas EAs use data-driven explanatory modelling methods resulting in numerous benefits. DCM analysis is novel for classification and advantageous as it provides information on the causal links between different brain regions.
△ Less
Submitted 6 November, 2019;
originally announced November 2019.
-
Classification of Resting-State fMRI using Evolutionary Algorithms: Towards a Brain Imaging Biomarker for Parkinson's Disease
Authors:
Amir Dehsarvi,
Stephen L. Smith
Abstract:
Accurate early diagnosis and monitoring of neurodegenerative conditions is essential for effective disease management and delivery of medication and treatment. This research develops automatic methods for detecting brain imaging preclinical biomarkers for Parkinson's disease (PD) by considering the novel application of evolutionary algorithms. A fundamental novel element of this work is the use of…
▽ More
Accurate early diagnosis and monitoring of neurodegenerative conditions is essential for effective disease management and delivery of medication and treatment. This research develops automatic methods for detecting brain imaging preclinical biomarkers for Parkinson's disease (PD) by considering the novel application of evolutionary algorithms. A fundamental novel element of this work is the use of evolutionary algorithms to both map and predict the functional connectivity in patients using resting state functional MRI data taken from the PPMI to identify PD progression biomarkers. Specifically, Cartesian Genetic Programming was used to classify DCM data as well as time-series data. The findings were validated using two other commonly used classification methods (Artificial Neural Networks and Support Vector Machines) and by employing k-fold cross-validation. Across DCM and time-series analyses, findings revealed maximum accuracies of 75.21% for early stage (prodromal) PD patients versus healthy controls, 85.87% for PD patients versus prodromal PD patients, and 92.09% for PD patients versus healthy controls. Prodromal PD patients were classified from healthy controls with high accuracy - this is notable and represents the key finding of this research since current methods of diagnosing prodromal PD have both low reliability and low accuracy. Furthermore, Cartesian Genetic Programming provided comparable performance accuracy relative to ANN and SVM. Evolutionary algorithms enable us to decode the classifier in terms of understanding the data inputs that are used, more easily than in ANN and SVM. Hence, these findings underscore the relevance of both DCM analyses for classification and CGP as a novel classification tool for brain imaging data with medical implications for disease diagnosis, particularly in early and asymptomatic stages.
△ Less
Submitted 11 October, 2019;
originally announced October 2019.
