-
The Temporal Structure of Language Processing in the Human Brain Corresponds to The Layered Hierarchy of Deep Language Models
Authors:
Ariel Goldstein,
Eric Ham,
Mariano Schain,
Samuel Nastase,
Zaid Zada,
Avigail Dabush,
Bobbi Aubrey,
Harshvardhan Gazula,
Amir Feder,
Werner K Doyle,
Sasha Devore,
Patricia Dugan,
Daniel Friedman,
Roi Reichart,
Michael Brenner,
Avinatan Hassidim,
Orrin Devinsky,
Adeen Flinker,
Omer Levy,
Uri Hasson
Abstract:
Deep Language Models (DLMs) provide a novel computational paradigm for understanding the mechanisms of natural language processing in the human brain. Unlike traditional psycholinguistic models, DLMs use layered sequences of continuous numerical vectors to represent words and context, allowing a plethora of emerging applications such as human-like text generation. In this paper we show evidence th…
▽ More
Deep Language Models (DLMs) provide a novel computational paradigm for understanding the mechanisms of natural language processing in the human brain. Unlike traditional psycholinguistic models, DLMs use layered sequences of continuous numerical vectors to represent words and context, allowing a plethora of emerging applications such as human-like text generation. In this paper we show evidence that the layered hierarchy of DLMs may be used to model the temporal dynamics of language comprehension in the brain by demonstrating a strong correlation between DLM layer depth and the time at which layers are most predictive of the human brain. Our ability to temporally resolve individual layers benefits from our use of electrocorticography (ECoG) data, which has a much higher temporal resolution than noninvasive methods like fMRI. Using ECoG, we record neural activity from participants listening to a 30-minute narrative while also feeding the same narrative to a high-performing DLM (GPT2-XL). We then extract contextual embeddings from the different layers of the DLM and use linear encoding models to predict neural activity. We first focus on the Inferior Frontal Gyrus (IFG, or Broca's area) and then extend our model to track the increasing temporal receptive window along the linguistic processing hierarchy from auditory to syntactic and semantic areas. Our results reveal a connection between human language processing and DLMs, with the DLM's layer-by-layer accumulation of contextual information mirroring the timing of neural activity in high-order language areas.
△ Less
Submitted 10 October, 2023;
originally announced October 2023.
-
The Past, Present, and Future of the Brain Imaging Data Structure (BIDS)
Authors:
Russell A. Poldrack,
Christopher J. Markiewicz,
Stefan Appelhoff,
Yoni K. Ashar,
Tibor Auer,
Sylvain Baillet,
Shashank Bansal,
Leandro Beltrachini,
Christian G. Benar,
Giacomo Bertazzoli,
Suyash Bhogawar,
Ross W. Blair,
Marta Bortoletto,
Mathieu Boudreau,
Teon L. Brooks,
Vince D. Calhoun,
Filippo Maria Castelli,
Patricia Clement,
Alexander L Cohen,
Julien Cohen-Adad,
Sasha D'Ambrosio,
Gilles de Hollander,
María de la iglesia-Vayá,
Alejandro de la Vega,
Arnaud Delorme
, et al. (89 additional authors not shown)
Abstract:
The Brain Imaging Data Structure (BIDS) is a community-driven standard for the organization of data and metadata from a growing range of neuroscience modalities. This paper is meant as a history of how the standard has developed and grown over time. We outline the principles behind the project, the mechanisms by which it has been extended, and some of the challenges being addressed as it evolves.…
▽ More
The Brain Imaging Data Structure (BIDS) is a community-driven standard for the organization of data and metadata from a growing range of neuroscience modalities. This paper is meant as a history of how the standard has developed and grown over time. We outline the principles behind the project, the mechanisms by which it has been extended, and some of the challenges being addressed as it evolves. We also discuss the lessons learned through the project, with the aim of enabling researchers in other domains to learn from the success of BIDS.
△ Less
Submitted 8 January, 2024; v1 submitted 11 September, 2023;
originally announced September 2023.
-
Evolutionary mismatch and the role of GxE interactions in human disease
Authors:
Amanda J. Lea,
Andrew G. Clark,
Andrew W. Dahl,
Orrin Devinsky,
Angela R. Garcia,
Christopher D. Golden,
Joseph Kamau,
Thomas S. Kraft,
Yvonne A. L. Lim,
Dino Martins,
Donald Mogoi,
Paivi Pajukanta,
George Perry,
Herman Pontzer,
Benjamin C. Trumble,
Samuel S. Urlacher,
Vivek V. Venkataraman,
Ian J. Wallace,
Michael Gurven,
Daniel Lieberman,
Julien F. Ayroles
Abstract:
Globally, we are witnessing the rise of complex, non-communicable diseases (NCDs) related to changes in our daily environments. Obesity, asthma, cardiovascular disease, and type 2 diabetes are part of a long list of "lifestyle" diseases that were rare throughout human history but are now common. A key idea from anthropology and evolutionary biology--the evolutionary mismatch hypothesis--seeks to e…
▽ More
Globally, we are witnessing the rise of complex, non-communicable diseases (NCDs) related to changes in our daily environments. Obesity, asthma, cardiovascular disease, and type 2 diabetes are part of a long list of "lifestyle" diseases that were rare throughout human history but are now common. A key idea from anthropology and evolutionary biology--the evolutionary mismatch hypothesis--seeks to explain this phenomenon. It posits that humans evolved in environments that radically differ from the ones experienced by most people today, and thus traits that were advantageous in past environments may now be "mismatched" and disease-causing. This hypothesis is, at its core, a genetic one: it predicts that loci with a history of selection will exhibit "genotype by environment" (GxE) interactions and have differential health effects in ancestral versus modern environments. Here, we discuss how this concept could be leveraged to uncover the genetic architecture of NCDs in a principled way. Specifically, we advocate for partnering with small-scale, subsistence-level groups that are currently transitioning from environments that are arguably more "matched" with their recent evolutionary history to those that are more "mismatched". These populations provide diverse genetic backgrounds as well as the needed levels and types of environmental variation necessary for map** GxE interactions in an explicit mismatch framework. Such work would make important contributions to our understanding of environmental and genetic risk factors for NCDs across diverse ancestries and sociocultural contexts.
△ Less
Submitted 13 February, 2023; v1 submitted 12 January, 2023;
originally announced January 2023.
-
Localized Motion Artifact Reduction on Brain MRI Using Deep Learning with Effective Data Augmentation Techniques
Authors:
Yijun Zhao,
Jacek Ossowski,
Xuming Wang,
Shang** Li,
Orrin Devinsky,
Samantha P. Martin,
Heath R. Pardoe
Abstract:
In-scanner motion degrades the quality of magnetic resonance imaging (MRI) thereby reducing its utility in the detection of clinically relevant abnormalities. We introduce a deep learning-based MRI artifact reduction model (DMAR) to localize and correct head motion artifacts in brain MRI scans. Our approach integrates the latest advances in object detection and noise reduction in Computer Vision.…
▽ More
In-scanner motion degrades the quality of magnetic resonance imaging (MRI) thereby reducing its utility in the detection of clinically relevant abnormalities. We introduce a deep learning-based MRI artifact reduction model (DMAR) to localize and correct head motion artifacts in brain MRI scans. Our approach integrates the latest advances in object detection and noise reduction in Computer Vision. Specifically, DMAR employs a two-stage approach: in the first, degraded regions are detected using the Single Shot Multibox Detector (SSD), and in the second, the artifacts within the found regions are reduced using a convolutional autoencoder (CAE). We further introduce a set of novel data augmentation techniques to address the high dimensionality of MRI images and the scarcity of available data. As a result, our model was trained on a large synthetic dataset of 225,000 images generated from 375 whole brain T1-weighted MRI scans. DMAR visibly reduces image artifacts when applied to both synthetic test images and 55 real-world motion-affected slices from 18 subjects from the multi-center Autism Brain Imaging Data Exchange (ABIDE) study. Quantitatively, depending on the level of degradation, our model achieves a 27.8%-48.1% reduction in RMSE and a 2.88--5.79 dB gain in PSNR on a 5000-sample set of synthetic images. For real-world artifact-affected scans from ABIDE, our model reduced the variance of image voxel intensity within artifact-affected brain regions (p = 0.014).
△ Less
Submitted 30 October, 2020; v1 submitted 9 July, 2020;
originally announced July 2020.
-
Multi-task Learning with Weak Class Labels: Leveraging iEEG to Detect Cortical Lesions in Cryptogenic Epilepsy
Authors:
Bilal Ahmed,
Thomas Thesen,
Karen E. Blackmon,
Ruben Kuzniecky,
Orrin Devinsky,
Jennifer G. Dy,
Carla E. Brodley
Abstract:
Multi-task learning (MTL) is useful for domains in which data originates from multiple sources that are individually under-sampled. MTL methods are able to learn classification models that have higher performance as compared to learning a single model by aggregating all the data together or learning a separate model for each data source. The performance of these methods relies on label accuracy. W…
▽ More
Multi-task learning (MTL) is useful for domains in which data originates from multiple sources that are individually under-sampled. MTL methods are able to learn classification models that have higher performance as compared to learning a single model by aggregating all the data together or learning a separate model for each data source. The performance of these methods relies on label accuracy. We address the problem of simultaneously learning multiple classifiers in the MTL framework when the training data has imprecise labels. We assume that there is an additional source of information that provides a score for each instance which reflects the certainty about its label. Modeling this score as being generated by an underlying ranking function, we augment the MTL framework with an added layer of supervision. This results in new MTL methods that are able to learn accurate classifiers while preserving the domain structure provided through the rank information. We apply these methods to the task of detecting abnormal cortical regions in the MRIs of patients suffering from focal epilepsy whose MRI were read as normal by expert neuroradiologists. In addition to the noisy labels provided by the results of surgical resection, we employ the results of an invasive intracranial-EEG exam as an additional source of label information. Our proposed methods are able to successfully detect abnormal regions for all patients in our dataset and achieve a higher performance as compared to baseline methods.
△ Less
Submitted 30 July, 2016;
originally announced August 2016.
