-
Advancing EEG/MEG Source Imaging with Geometric-Informed Basis Functions
Authors:
Song Wang,
Chen Wei,
Kexin Lou,
Dongfeng Gu,
Quanying Liu
Abstract:
Electroencephalography (EEG) and Magnetoencephalography (MEG) are pivotal in understanding brain activity but are limited by their poor spatial resolution. EEG/MEG source imaging (ESI) infers the high-resolution electric field distribution in the brain based on the low-resolution scalp EEG/MEG observations. However, the ESI problem is ill-posed, and how to bring neuroscience priors into ESI method…
▽ More
Electroencephalography (EEG) and Magnetoencephalography (MEG) are pivotal in understanding brain activity but are limited by their poor spatial resolution. EEG/MEG source imaging (ESI) infers the high-resolution electric field distribution in the brain based on the low-resolution scalp EEG/MEG observations. However, the ESI problem is ill-posed, and how to bring neuroscience priors into ESI method is the key. Here, we present a novel method which utilizes the Brain Geometric-informed Basis Functions (GBFs) as priors to enhance EEG/MEG source imaging. Through comprehensive experiments on both synthetic data and real task EEG data, we demonstrate the superiority of GBFs over traditional spatial basis functions (e.g., Harmonic and MSP), as well as existing ESI methods (e.g., dSPM, MNE, sLORETA, eLORETA). GBFs provide robust ESI results under different noise levels, and result in biologically interpretable EEG sources. We believe the high-resolution EEG source imaging from GBFs will greatly advance neuroscience research.
△ Less
Submitted 31 January, 2024;
originally announced January 2024.
-
Cross-Modal ASR Post-Processing System for Error Correction and Utterance Rejection
Authors:
**g Du,
Shiliang Pu,
Qinbo Dong,
Chao **,
Xin Qi,
Dian Gu,
Ru Wu,
Hongwei Zhou
Abstract:
Although modern automatic speech recognition (ASR) systems can achieve high performance, they may produce errors that weaken readers' experience and do harm to downstream tasks. To improve the accuracy and reliability of ASR hypotheses, we propose a cross-modal post-processing system for speech recognizers, which 1) fuses acoustic features and textual features from different modalities, 2) joints…
▽ More
Although modern automatic speech recognition (ASR) systems can achieve high performance, they may produce errors that weaken readers' experience and do harm to downstream tasks. To improve the accuracy and reliability of ASR hypotheses, we propose a cross-modal post-processing system for speech recognizers, which 1) fuses acoustic features and textual features from different modalities, 2) joints a confidence estimator and an error corrector in multi-task learning fashion and 3) unifies error correction and utterance rejection modules. Compared with single-modal or single-task models, our proposed system is proved to be more effective and efficient. Experiment result shows that our post-processing system leads to more than 10% relative reduction of character error rate (CER) for both single-speaker and multi-speaker speech on our industrial ASR system, with about 1.7ms latency for each token, which ensures that extra latency introduced by post-processing is acceptable in streaming speech recognition.
△ Less
Submitted 10 January, 2022;
originally announced January 2022.
-
SGANVO: Unsupervised Deep Visual Odometry and Depth Estimation with Stacked Generative Adversarial Networks
Authors:
Tuo Feng,
Dongbing Gu
Abstract:
Recently end-to-end unsupervised deep learning methods have achieved an effect beyond geometric methods for visual depth and ego-motion estimation tasks. These data-based learning methods perform more robustly and accurately in some of the challenging scenes. The encoder-decoder network has been widely used in the depth estimation and the RCNN has brought significant improvements in the ego-motion…
▽ More
Recently end-to-end unsupervised deep learning methods have achieved an effect beyond geometric methods for visual depth and ego-motion estimation tasks. These data-based learning methods perform more robustly and accurately in some of the challenging scenes. The encoder-decoder network has been widely used in the depth estimation and the RCNN has brought significant improvements in the ego-motion estimation. Furthermore, the latest use of Generative Adversarial Nets(GANs) in depth and ego-motion estimation has demonstrated that the estimation could be further improved by generating pictures in the game learning process. This paper proposes a novel unsupervised network system for visual depth and ego-motion estimation: Stacked Generative Adversarial Network(SGANVO). It consists of a stack of GAN layers, of which the lowest layer estimates the depth and ego-motion while the higher layers estimate the spatial features. It can also capture the temporal dynamic due to the use of a recurrent representation across the layers. See Fig.1 for details. We select the most commonly used KITTI [1] data set for evaluation. The evaluation results show that our proposed method can produce better or comparable results in depth and ego-motion estimation.
△ Less
Submitted 20 June, 2019;
originally announced June 2019.
