-
Semantic reconstruction of continuous language from MEG signals
Authors:
Bo Wang,
Xiran Xu,
Longxiang Zhang,
Boda Xiao,
Xihong Wu,
**g Chen
Abstract:
Decoding language from neural signals holds considerable theoretical and practical importance. Previous research has indicated the feasibility of decoding text or speech from invasive neural signals. However, when using non-invasive neural signals, significant challenges are encountered due to their low quality. In this study, we proposed a data-driven approach for decoding semantic of language fr…
▽ More
Decoding language from neural signals holds considerable theoretical and practical importance. Previous research has indicated the feasibility of decoding text or speech from invasive neural signals. However, when using non-invasive neural signals, significant challenges are encountered due to their low quality. In this study, we proposed a data-driven approach for decoding semantic of language from Magnetoencephalography (MEG) signals recorded while subjects were listening to continuous speech. First, a multi-subject decoding model was trained using contrastive learning to reconstruct continuous word embeddings from MEG data. Subsequently, a beam search algorithm was adopted to generate text sequences based on the reconstructed word embeddings. Given a candidate sentence in the beam, a language model was used to predict the subsequent words. The word embeddings of the subsequent words were correlated with the reconstructed word embedding. These correlations were then used as a measure of the probability for the next word. The results showed that the proposed continuous word embedding model can effectively leverage both subject-specific and subject-shared information. Additionally, the decoded text exhibited significant similarity to the target text, with an average BERTScore of 0.816, a score comparable to that in the previous fMRI study.
△ Less
Submitted 14 September, 2023;
originally announced September 2023.
-
Recurrent Feedback Improves Feedforward Representations in Deep Neural Networks
Authors:
Siming Yan,
Xuyang Fang,
Bowen Xiao,
Harold Rockwell,
Yimeng Zhang,
Tai Sing Lee
Abstract:
The abundant recurrent horizontal and feedback connections in the primate visual cortex are thought to play an important role in bringing global and semantic contextual information to early visual areas during perceptual inference, hel** to resolve local ambiguity and fill in missing details. In this study, we find that introducing feedback loops and horizontal recurrent connections to a deep co…
▽ More
The abundant recurrent horizontal and feedback connections in the primate visual cortex are thought to play an important role in bringing global and semantic contextual information to early visual areas during perceptual inference, hel** to resolve local ambiguity and fill in missing details. In this study, we find that introducing feedback loops and horizontal recurrent connections to a deep convolution neural network (VGG16) allows the network to become more robust against noise and occlusion during inference, even in the initial feedforward pass. This suggests that recurrent feedback and contextual modulation transform the feedforward representations of the network in a meaningful and interesting way. We study the population codes of neurons in the network, before and after learning with feedback, and find that learning with feedback yielded an increase in discriminability (measured by d-prime) between the different object classes in the population codes of the neurons in the feedforward path, even at the earliest layer that receives feedback. We find that recurrent feedback, by injecting top-down semantic meaning to the population activities, helps the network learn better feedforward paths to robustly map noisy image patches to the latent representations corresponding to important visual concepts of each object class, resulting in greater robustness of the network against noises and occlusion as well as better fine-grained recognition.
△ Less
Submitted 22 December, 2019;
originally announced December 2019.
-
Protein-protein docking by generalized Fourier transforms on 5D rotational manifolds
Authors:
Dmitry Padhorny,
Andrey Kazennov,
Brandon S. Zerbe,
Kathryn Porter,
Bing Xia,
Scott E. Mortadella,
Yaroslav Kholodov,
David W. Ritchie,
Sandor Vajda,
Dima Kozakov
Abstract:
Energy evaluation using fast Fourier transforms enables sampling billions of putative complex structures and hence revolutionized rigid protein-protein docking. However, in current methods efficient acceleration is achieved only in either the translational or the rotational subspace. Develo** an efficient and accurate docking method that expands FFT based sampling to 5 rotational coordinates is…
▽ More
Energy evaluation using fast Fourier transforms enables sampling billions of putative complex structures and hence revolutionized rigid protein-protein docking. However, in current methods efficient acceleration is achieved only in either the translational or the rotational subspace. Develo** an efficient and accurate docking method that expands FFT based sampling to 5 rotational coordinates is an extensively studied but still unsolved problem. The algorithm presented here retains the accuracy of earlier methods but yields at least tenfold speedup. The improvement is due to two innovations. First, the search space is treated as the product manifold $\mathbf{SO(3)x(SO(3)\setminus S^1)}$, where $\mathbf{SO(3)}$ is the rotation group representing the space of the rotating ligand, and $\mathbf{(SO(3)\setminus S^1)}$ is the space spanned by the two Euler angles that define the orientation of the vector from the center of the fixed receptor toward the center of the ligand. This representation enables the use of efficient FFT methods developed for $\mathbf{SO(3)}$. Second, we select the centers of highly populated clusters of docked structures, rather than the lowest energy conformations, as predictions of the complex, and hence there is no need for very high accuracy in energy evaluation. Therefore it is sufficient to use a limited number of spherical basis functions in the Fourier space, which increases the efficiency of sampling while retaining the accuracy of docking results. A major advantage of the method is that, in contrast to classical approaches, increasing the number of correlation function terms is computationally inexpensive, which enables using complex energy functions for scoring.
△ Less
Submitted 21 September, 2016;
originally announced September 2016.
-
Computational Fluid Dynamic Approach for Biological System Modeling
Authors:
Weidong Huang,
Chundu Wu,
Bingjia Xiao,
Weidong Xia
Abstract:
Various biological system models have been proposed in systems biology, which are based on the complex biological reactions kinetic of various components. These models are not practical because we lack of kinetic information. In this paper, it is found that the enzymatic reaction and multi-order reaction rate is often controlled by the transport of the reactants in biological systems. A Computat…
▽ More
Various biological system models have been proposed in systems biology, which are based on the complex biological reactions kinetic of various components. These models are not practical because we lack of kinetic information. In this paper, it is found that the enzymatic reaction and multi-order reaction rate is often controlled by the transport of the reactants in biological systems. A Computational Fluid Dynamic (CFD) approach, which is based on transport of the components and kinetics of biological reactions, is introduced for biological system modeling. We apply this approach to a biological wastewater treatment system for the study of metabolism of organic carbon substrates and the population of microbial. The results show that CFD model coupled with reaction kinetics is more accurate and more feasible than kinetic models for biological system modeling.
△ Less
Submitted 2 August, 2005;
originally announced August 2005.
