-
EEG-DBNet: A Dual-Branch Network for Temporal-Spectral Decoding in Motor-Imagery Brain-Computer Interfaces
Authors:
Xicheng Lou,
Xinwei Li,
Hongying Meng,
Jun Hu,
Meili Xu,
Yue Zhao,
Jiazhang Yang,
Zhangyong Li
Abstract:
Motor imagery electroencephalogram (EEG)-based brain-computer interfaces (BCIs) offer significant advantages for individuals with restricted limb mobility. However, challenges such as low signal-to-noise ratio and limited spatial resolution impede accurate feature extraction from EEG signals, thereby affecting the classification accuracy of different actions. To address these challenges, this stud…
▽ More
Motor imagery electroencephalogram (EEG)-based brain-computer interfaces (BCIs) offer significant advantages for individuals with restricted limb mobility. However, challenges such as low signal-to-noise ratio and limited spatial resolution impede accurate feature extraction from EEG signals, thereby affecting the classification accuracy of different actions. To address these challenges, this study proposes an end-to-end dual-branch network (EEG-DBNet) that decodes the temporal and spectral sequences of EEG signals in parallel through two distinct network branches. Each branch comprises a local convolutional block and a global convolutional block. The local convolutional block transforms the source signal from the temporal-spatial domain to the temporal-spectral domain. By varying the number of filters and convolution kernel sizes, the local convolutional blocks in different branches adjust the length of their respective dimension sequences. Different types of pooling layers are then employed to emphasize the features of various dimension sequences, setting the stage for subsequent global feature extraction. The global convolution block splits and reconstructs the feature of the signal sequence processed by the local convolution block in the same branch and further extracts features through the dilated causal convolutional neural networks. Finally, the outputs from the two branches are concatenated, and signal classification is completed via a fully connected layer. Our proposed method achieves classification accuracies of 85.84% and 91.60% on the BCI Competition 4-2a and BCI Competition 4-2b datasets, respectively, surpassing existing state-of-the-art models. The source code is available at https://github.com/xicheng105/EEG-DBNet.
△ Less
Submitted 19 June, 2024; v1 submitted 25 May, 2024;
originally announced May 2024.
-
Power Optimization for Integrated Active and Passive Sensing in DFRC Systems
Authors:
Xingliang Lou,
Wenchao Xia,
Kai-Kit Wong,
Haitao Zhao,
Tony Q. S. Quek,
Hongbo Zhu
Abstract:
Most existing works on dual-function radar-communication (DFRC) systems mainly focus on active sensing, but ignore passive sensing. To leverage multi-static sensing capability, we explore integrated active and passive sensing (IAPS) in DFRC systems to remedy sensing performance. The multi-antenna base station (BS) is responsible for communication and active sensing by transmitting signals to user…
▽ More
Most existing works on dual-function radar-communication (DFRC) systems mainly focus on active sensing, but ignore passive sensing. To leverage multi-static sensing capability, we explore integrated active and passive sensing (IAPS) in DFRC systems to remedy sensing performance. The multi-antenna base station (BS) is responsible for communication and active sensing by transmitting signals to user equipments while detecting a target according to echo signals. In contrast, passive sensing is performed at the receive access points (RAPs). We consider both the cases where the capacity of the backhaul links between the RAPs and BS is unlimited or limited and adopt different fusion strategies. Specifically, when the backhaul capacity is unlimited, the BS and RAPs transfer sensing signals they have received to the central controller (CC) for signal fusion. The CC processes the signals and leverages the generalized likelihood ratio test detector to determine the present of a target. However, when the backhaul capacity is limited, each RAP, as well as the BS, makes decisions independently and sends its binary inference results to the CC for result fusion via voting aggregation. Then, aiming at maximize the target detection probability under communication quality of service constraints, two power optimization algorithms are proposed. Finally, numerical simulations demonstrate that the sensing performance in case of unlimited backhaul capacity is much better than that in case of limited backhaul capacity. Moreover, it implied that the proposed IAPS scheme outperforms only-passive and only-active sensing schemes, especially in unlimited capacity case.
△ Less
Submitted 17 February, 2024;
originally announced February 2024.
-
Strong Structural Controllability of Structured Networks with MIMO node systems
Authors:
Yanting Ni,
Xuyang Lou,
Junjie Jiao,
Jiajia Jia
Abstract:
The article addresses the problem of strong structural controllability of structured networks with multi-input multi-output (MIMO) node systems. The authors first present necessary and sufficient conditions for strong structural controllability, which involve both algebraic and graph-theoretic aspects. These conditions are computationally expensive, especially for large-scale networks with high-di…
▽ More
The article addresses the problem of strong structural controllability of structured networks with multi-input multi-output (MIMO) node systems. The authors first present necessary and sufficient conditions for strong structural controllability, which involve both algebraic and graph-theoretic aspects. These conditions are computationally expensive, especially for large-scale networks with high-dimensional state spaces. To overcome this computational complexity, we propose a necessary algebraic condition from a node system's perspective and a graph-theoretic condition from a network topology's perspective. The latter condition is derived from the structured interconnection laws and employs a new color change rule, namely weakly color change rule introduced in this paper. Overall, this article contributes to the study of strong structural controllability in structured networks with MIMO node systems, providing both theoretical and practical insights for their analysis and design.
△ Less
Submitted 18 May, 2023;
originally announced May 2023.
-
Learned Smartphone ISP on Mobile GPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report
Authors:
Andrey Ignatov,
Radu Timofte,
Shuai Liu,
Chaoyu Feng,
Furui Bai,
Xiaotao Wang,
Lei Lei,
Ziyao Yi,
Yan Xiang,
Zibin Liu,
Shaoqing Li,
Keming Shi,
Dehui Kong,
Ke Xu,
Minsu Kwon,
Yaqi Wu,
Jiesi Zheng,
Zhihao Fan,
Xun Wu,
Feng Zhang,
Albert No,
Minhyeok Cho,
Zewen Chen,
Xiaze Zhang,
Ran Li
, et al. (13 additional authors not shown)
Abstract:
The role of mobile cameras increased dramatically over the past few years, leading to more and more research in automatic image quality enhancement and RAW photo processing. In this Mobile AI challenge, the target was to develop an efficient end-to-end AI-based image signal processing (ISP) pipeline replacing the standard mobile ISPs that can run on modern smartphone GPUs using TensorFlow Lite. Th…
▽ More
The role of mobile cameras increased dramatically over the past few years, leading to more and more research in automatic image quality enhancement and RAW photo processing. In this Mobile AI challenge, the target was to develop an efficient end-to-end AI-based image signal processing (ISP) pipeline replacing the standard mobile ISPs that can run on modern smartphone GPUs using TensorFlow Lite. The participants were provided with a large-scale Fujifilm UltraISP dataset consisting of thousands of paired photos captured with a normal mobile camera sensor and a professional 102MP medium-format FujiFilm GFX100 camera. The runtime of the resulting models was evaluated on the Snapdragon's 8 Gen 1 GPU that provides excellent acceleration results for the majority of common deep learning ops. The proposed solutions are compatible with all recent mobile GPUs, being able to process Full HD photos in less than 20-50 milliseconds while achieving high fidelity results. A detailed description of all models developed in this challenge is provided in this paper.
△ Less
Submitted 7 November, 2022;
originally announced November 2022.
-
Notions, Stability, Existence, and Robustness of Limit Cycles in Hybrid Dynamical Systems
Authors:
Xuyang Lou,
Yuchun Li,
Ricardo G. Sanfelice
Abstract:
This paper deals with existence and robust stability of hybrid limit cycles for a class of hybrid systems given by the combination of continuous dynamics on a flow set and discrete dynamics on a jump set. For this purpose, the notion of Zhukovskii stability, typically stated for continuous-time systems, is extended to the hybrid systems. Necessary conditions, particularly, a condition using a forw…
▽ More
This paper deals with existence and robust stability of hybrid limit cycles for a class of hybrid systems given by the combination of continuous dynamics on a flow set and discrete dynamics on a jump set. For this purpose, the notion of Zhukovskii stability, typically stated for continuous-time systems, is extended to the hybrid systems. Necessary conditions, particularly, a condition using a forward invariance notion, for existence of hybrid limit cycles are first presented. In addition, a sufficient condition, related to Zhukovskii stability, for the existence of (or lack of) hybrid limit cycles is established. Furthermore, under mild assumptions, we show that asymptotic stability of such hybrid limit cycles is not only equivalent to asymptotic stability of a fixed point of the associated Poincaré map but also robust to perturbations. Specifically, robustness to generic perturbations, which capture state noise and unmodeled dynamics, and to inflations of the flow and jump sets are established in terms of $\mathcal{KL}$ bounds. Furthermore, results establishing relationships between the properties of a computed Poincaré map, which is necessarily affected by computational error, and the actual asymptotic stability properties of a hybrid limit cycle are proposed. In particular, it is shown that asymptotic stability of the exact Poincaré map is preserved when computed with enough precision. Several examples, including a congestion control system and spiking neurons, are presented to illustrate the notions and results throughout the paper.
△ Less
Submitted 18 December, 2023; v1 submitted 4 August, 2022;
originally announced August 2022.
-
Raw Bayer Pattern Image Synthesis for Computer Vision-oriented Image Signal Processing Pipeline Design
Authors:
Wei Zhou,
Xiangyu Zhang,
Hongyu Wang,
Shenghua Gao,
Xin Lou
Abstract:
In this paper, we propose a method to add constraints that are un-formulatable in generative adversarial networks (GAN)-based arbitrary size RAW Bayer image generation. It is shown theoretically that by using the transformed data in GAN training, it is able to improve the learning of the original data distribution, owing to the invariant of Jensen-Shannon (JS) divergence between two distributions…
▽ More
In this paper, we propose a method to add constraints that are un-formulatable in generative adversarial networks (GAN)-based arbitrary size RAW Bayer image generation. It is shown theoretically that by using the transformed data in GAN training, it is able to improve the learning of the original data distribution, owing to the invariant of Jensen-Shannon (JS) divergence between two distributions under invertible and differentiable transformation. Benefiting from the proposed method, RAW Bayer pattern images can be generated by configuring the transformation as demosaicing. It is shown that by adding another transformation, the proposed method is able to synthesize high-quality RAW Bayer images with arbitrary size. Experimental results show that images generated by the proposed method outperform the existing methods in the Fréchet inception distance (FID) score, peak signal to noise ratio (PSNR), and mean structural similarity (MSSIM), and the training process is more stable. To the best knowledge of the authors, there is no open-source, large-scale image dataset in the RAW Bayer domain, which is crucial for research works aiming to explore the image signal processing (ISP) pipeline design for computer vision tasks. Converting the existing commonly used color image datasets to their corresponding RAW Bayer versions, the proposed method can be a promising solution to the RAW image dataset problem. We also show in the experiments that, by training object detection frameworks using the synthesized RAW Bayer images, they can be used in an end-to-end manner (from RAW images to vision tasks) with negligible performance degradation.
△ Less
Submitted 15 December, 2021; v1 submitted 25 October, 2021;
originally announced October 2021.
-
Gradient-based Feature Extraction From Raw Bayer Pattern Images
Authors:
Wei Zhou,
Ling Zhang,
Shengyu Gao,
Xin Lou
Abstract:
In this paper, the impact of demosaicing on gradient extraction is studied and a gradient-based feature extraction pipeline based on raw Bayer pattern images is proposed. It is shown both theoretically and experimentally that the Bayer pattern images are applicable to the central difference gradient-based feature extraction algorithms without performance degradation, or even with superior performa…
▽ More
In this paper, the impact of demosaicing on gradient extraction is studied and a gradient-based feature extraction pipeline based on raw Bayer pattern images is proposed. It is shown both theoretically and experimentally that the Bayer pattern images are applicable to the central difference gradient-based feature extraction algorithms without performance degradation, or even with superior performance in some cases. The color difference constancy assumption, which is widely used in various demosaicing algorithms, is applied in the proposed Bayer pattern image-based gradient extraction pipeline. Experimental results show that the gradients extracted from Bayer pattern images are robust enough to be used in histogram of oriented gradients (HOG)-based pedestrian detection algorithms and shift-invariant feature transform (SIFT)-based matching algorithms.
△ Less
Submitted 28 October, 2021; v1 submitted 6 April, 2020;
originally announced April 2020.
