Showing 1–13 of 13 results for author: Paquelet, S

TocBERT: Medical Document Structure Extraction Using Bidirectional Transformers

Authors: Majd Saleh, Sarra Baghdadi, Stéphane Paquelet

Abstract: Text segmentation holds paramount importance in the field of Natural Language Processing (NLP). It plays an important role in several NLP downstream tasks like information retrieval and document summarization. In this work, we propose a new solution, namely TocBERT, for segmenting texts using bidirectional transformers. TocBERT represents a supervised solution trained on the detection of titles an… ▽ More Text segmentation holds paramount importance in the field of Natural Language Processing (NLP). It plays an important role in several NLP downstream tasks like information retrieval and document summarization. In this work, we propose a new solution, namely TocBERT, for segmenting texts using bidirectional transformers. TocBERT represents a supervised solution trained on the detection of titles and sub-titles from their semantic representations. This task was formulated as a named entity recognition (NER) problem. The solution has been applied on a medical text segmentation use-case where the Bio-ClinicalBERT model is fine-tuned to segment discharge summaries of the MIMIC-III dataset. The performance of TocBERT has been evaluated on a human-labeled ground truth corpus of 250 notes. It achieved an F1-score of 84.6% when evaluated on a linear text segmentation problem and 72.8% on a hierarchical text segmentation problem. It outperformed a carefully designed rule-based solution, particularly in distinguishing titles from subtitles. △ Less

Submitted 27 June, 2024; originally announced June 2024.

Comments: 6 pages, 6 figures

Anatomy of Neural Language Models

Authors: Majd Saleh, Stéphane Paquelet

Abstract: The fields of generative AI and transfer learning have experienced remarkable advancements in recent years especially in the domain of Natural Language Processing (NLP). Transformers have been at the heart of these advancements where the cutting-edge transformer-based Language Models (LMs) have led to new state-of-the-art results in a wide spectrum of applications. While the number of research wor… ▽ More The fields of generative AI and transfer learning have experienced remarkable advancements in recent years especially in the domain of Natural Language Processing (NLP). Transformers have been at the heart of these advancements where the cutting-edge transformer-based Language Models (LMs) have led to new state-of-the-art results in a wide spectrum of applications. While the number of research works involving neural LMs is exponentially increasing, their vast majority are high-level and far from self-contained. Consequently, a deep understanding of the literature in this area is a tough task especially in the absence of a unified mathematical framework explaining the main types of neural LMs. We address the aforementioned problem in this tutorial where the objective is to explain neural LMs in a detailed, simplified and unambiguous mathematical framework accompanied by clear graphical illustrations. Concrete examples on widely used models like BERT and GPT2 are explored. Finally, since transformers pretrained on language-modeling-like tasks have been widely adopted in computer vision and time series applications, we briefly explore some examples of such solutions in order to enable readers to understand how transformers work in the aforementioned domains and compare this use with the original one in NLP. △ Less

Submitted 27 February, 2024; v1 submitted 8 January, 2024; originally announced January 2024.

Comments: 36 Pages; 25 Figures; some typos and notation errors are corrected in this version

Model-based Deep Learning for Beam Prediction based on a Channel Chart

Authors: Taha Yassine, Baptiste Chatelier, Vincent Corlay, Matthieu Crussière, Stephane Paquelet, Olav Tirkkonen, Luc Le Magoarou

Abstract: Channel charting builds a map of the radio environment in an unsupervised way. The obtained chart locations can be seen as low-dimensional compressed versions of channel state information that can be used for a wide variety of applications, including beam prediction. In non-standalone or cell-free systems, chart locations computed at a given base station can be transmitted to several other base st… ▽ More Channel charting builds a map of the radio environment in an unsupervised way. The obtained chart locations can be seen as low-dimensional compressed versions of channel state information that can be used for a wide variety of applications, including beam prediction. In non-standalone or cell-free systems, chart locations computed at a given base station can be transmitted to several other base stations (possibly operating at different frequency bands) for them to predict which beams to use. This potentially yields a dramatic reduction of the overhead due to channel estimation or beam management, since only the base station performing charting requires channel state information, the others directly predicting the beam from the chart location. In this paper, advanced model-based neural network architectures are proposed for both channel charting and beam prediction. The proposed methods are assessed on realistic synthetic channels, yielding promising results. △ Less

Submitted 4 December, 2023; originally announced December 2023.

Optimizing Multicarrier Multiantenna Systems for LoS Channel Charting

Authors: Taha Yassine, Luc Le Magoarou, Matthieu Crussière, Stephane Paquelet

Abstract: Channel charting (CC) consists in learning a map** between the space of raw channel observations, made available from pilot-based channel estimation in multicarrier multiantenna system, and a low-dimensional space where close points correspond to channels of user equipments (UEs) close spatially. Among the different methods of learning this map**, some rely on a distance measure between channe… ▽ More Channel charting (CC) consists in learning a map** between the space of raw channel observations, made available from pilot-based channel estimation in multicarrier multiantenna system, and a low-dimensional space where close points correspond to channels of user equipments (UEs) close spatially. Among the different methods of learning this map**, some rely on a distance measure between channel vectors. Such a distance should reliably reflect the local spatial neighborhoods of the UEs. The recently proposed phase-insensitive (PI) distance exhibits good properties in this regards, but suffers from ambiguities due to both its periodic and oscillatory aspects, making users far away from each other appear closer in some cases. In this paper, a thorough theoretical analysis of the said distance and its limitations is provided, giving insights on how they can be mitigated. Guidelines for designing systems capable of learning quality charts are consequently derived. Experimental validation is then conducted on synthetic and realistic data in different scenarios. △ Less

Submitted 28 September, 2023; originally announced October 2023.

LatentForensics: Towards frugal deepfake detection in the StyleGAN latent space

Authors: Matthieu Delmas, Amine Kacete, Stephane Paquelet, Simon Leglaive, Renaud Seguier

Abstract: The classification of forged videos has been a challenge for the past few years. Deepfake classifiers can now reliably predict whether or not video frames have been tampered with. However, their performance is tied to both the dataset used for training and the analyst's computational power. We propose a deepfake detection method that operates in the latent space of a state-of-the-art generative ad… ▽ More The classification of forged videos has been a challenge for the past few years. Deepfake classifiers can now reliably predict whether or not video frames have been tampered with. However, their performance is tied to both the dataset used for training and the analyst's computational power. We propose a deepfake detection method that operates in the latent space of a state-of-the-art generative adversarial network (GAN) trained on high-quality face images. The proposed method leverages the structure of the latent space of StyleGAN to learn a lightweight binary classification model. Experimental results on standard datasets reveal that the proposed approach outperforms other state-of-the-art deepfake classification methods, especially in contexts where the data available to train the models is rare, such as when a new manipulation method is introduced. To the best of our knowledge, this is the first study showing the interest of the latent space of StyleGAN for deepfake classification. Combined with other recent studies on the interpretation and manipulation of this latent space, we believe that the proposed approach can further help in develo** frugal deepfake classification methods based on interpretable high-level properties of face images. △ Less

Submitted 6 May, 2024; v1 submitted 30 March, 2023; originally announced March 2023.

Comments: 7 pages, 3 figures, 5 tables

MSC Class: 68T45 ACM Class: I.2.10

Channel charting based beamforming

Authors: Luc Le Magoarou, Taha Yassine, Stephane Paquelet, Matthieu Crussière

Abstract: Channel charting (CC) is an unsupervised learning method allowing to locate users relative to each other without reference. From a broader perspective, it can be viewed as a way to discover a low-dimensional latent space charting the channel manifold. In this paper, this latent modeling vision is leveraged together with a recently proposed location-based beamforming (LBB) method to show that chann… ▽ More Channel charting (CC) is an unsupervised learning method allowing to locate users relative to each other without reference. From a broader perspective, it can be viewed as a way to discover a low-dimensional latent space charting the channel manifold. In this paper, this latent modeling vision is leveraged together with a recently proposed location-based beamforming (LBB) method to show that channel charting can be used for map** channels in space or frequency. Combining CC and LBB yields a neural network resembling an autoencoder. The proposed method is empirically assessed on a channel map** task whose objective is to predict downlink channels from uplink channels. △ Less

Submitted 27 December, 2022; v1 submitted 6 December, 2022; originally announced December 2022.

Comments: Asilomar Conference on Signals, Systems, and Computers, Oct 2022, Pacific Grove, United States

Leveraging triplet loss and nonlinear dimensionality reduction for on-the-fly channel charting

Authors: Taha Yassine, Luc Le Magoarou, Stéphane Paquelet, Matthieu Crussière

Abstract: Channel charting is an unsupervised learning method that aims at map** wireless channels to a so-called chart, preserving as much as possible spatial neighborhoods. In this paper, a model-based deep learning approach to this problem is proposed. It builds on a physically motivated distance measure to structure and initialize a neural network that is subsequently trained using a triplet loss func… ▽ More Channel charting is an unsupervised learning method that aims at map** wireless channels to a so-called chart, preserving as much as possible spatial neighborhoods. In this paper, a model-based deep learning approach to this problem is proposed. It builds on a physically motivated distance measure to structure and initialize a neural network that is subsequently trained using a triplet loss function. The proposed structure exhibits a low number of parameters and clever initialization leads to fast training. These two features make the proposed approach amenable to on-the-fly channel charting. The method is empirically assessed on realistic synthetic channels, yielding encouraging results. △ Less

Submitted 4 April, 2022; originally announced April 2022.

Deep learning for location based beamforming with NLOS channels

Authors: Luc Le Magoarou, Taha Yassine, Stéphane Paquelet, Matthieu Crussière

Abstract: Massive MIMO systems are highly efficient but critically rely on accurate channel state information (CSI) at the base station in order to determine appropriate precoders. CSI acquisition requires sending pilot symbols which induce an important overhead. In this paper, a method whose objective is to determine an appropriate precoder from the knowledge of the user's location only is proposed. Such a… ▽ More Massive MIMO systems are highly efficient but critically rely on accurate channel state information (CSI) at the base station in order to determine appropriate precoders. CSI acquisition requires sending pilot symbols which induce an important overhead. In this paper, a method whose objective is to determine an appropriate precoder from the knowledge of the user's location only is proposed. Such a way to determine precoders is known as location based beamforming. It allows to reduce or even eliminate the need for pilot symbols, depending on how the location is obtained. the proposed method learns a direct map** from location to precoder in a supervised way. It involves a neural network with a specific structure based on random Fourier features allowing to learn functions containing high spatial frequencies. It is assessed empirically and yields promising results on realistic synthetic channels. As opposed to previously proposed methods, it allows to handle both line-of-sight (LOS) and non-line-of-sight (NLOS) channels. △ Less

Submitted 29 December, 2021; originally announced January 2022.

Performance of MIMO channel estimation with a physical model

Authors: Luc Le Magoarou, Stéphane Paquelet

Abstract: Channel estimation is challenging in multi-antenna communication systems, because of the large number of parameters to estimate. One way of facilitating this task is to use a physical model describing the multiple paths constituting the channel, in the hope of reducing the number of unknowns in the problem. The achievable performance of estimation using this kind of physical model is studied theor… ▽ More Channel estimation is challenging in multi-antenna communication systems, because of the large number of parameters to estimate. One way of facilitating this task is to use a physical model describing the multiple paths constituting the channel, in the hope of reducing the number of unknowns in the problem. The achievable performance of estimation using this kind of physical model is studied theoretically. It is found that adjusting the number of estimated paths leads to a bias-variance tradeoff which is characterized. Moreover, computing the Fisher information matrix of the model allows to identify orthogonal parameters, ultimately leading to fast and asymptotically optimal algorithms as a byproduct. △ Less

Submitted 19 February, 2019; originally announced February 2019.

Massive MIMO channel estimation taking into account spherical waves

Authors: Antoine Le Calvez, Luc Le Magoarou, Stéphane Paquelet

Abstract: Together with millimiter waves (mmWaves), massive multiple-input multiple-output (MIMO) systems are key technological components of fifth generation (5G) wireless communication systems. In such a context, geometric considerations show that the largely adopted plane wave model (PWM) of the channel potentially loses its validity. An alternative is to consider the more accurate but more complex spher… ▽ More Together with millimiter waves (mmWaves), massive multiple-input multiple-output (MIMO) systems are key technological components of fifth generation (5G) wireless communication systems. In such a context, geometric considerations show that the largely adopted plane wave model (PWM) of the channel potentially loses its validity. An alternative is to consider the more accurate but more complex spherical wave model (SWM). This paper introduces an intermediate parabolic wave model (ParWM), more accurate than the PWM while less complex than the SWM. The validity domains of those three physical models are assessed in a novel way. Finally, estimation algorithms for the SWM and ParWM are proposed and compared with classical algorithms, showing a promising performance complexity trade-off. △ Less

Submitted 14 November, 2018; originally announced November 2018.

Bias-variance tradeoff in MIMO channel estimation

Authors: Luc Le Magoarou, Stéphane Paquelet

Abstract: Channel estimation is challenging in multi-antenna communication systems, because of the large number of parameters to estimate. It is possible to facilitate this task by using a physical model describing the multiple paths constituting the channel, in the hope of reducing the number of unknowns in the problem. Adjusting the number of estimated paths leads to a bias-variance tradeoff. This paper e… ▽ More Channel estimation is challenging in multi-antenna communication systems, because of the large number of parameters to estimate. It is possible to facilitate this task by using a physical model describing the multiple paths constituting the channel, in the hope of reducing the number of unknowns in the problem. Adjusting the number of estimated paths leads to a bias-variance tradeoff. This paper explores this tradeoff, aiming to find the optimal number of paths to estimate. Moreover, the approach based on a physical model is compared to the classical least squares and Bayesian techniques. Finally, the impact of channel estimation error on the system data rate is assessed. △ Less

Submitted 26 April, 2018; v1 submitted 20 April, 2018; originally announced April 2018.

MIMO Channel Hardening: A Physical Model based Analysis

Authors: Matthieu Roy, Stéphane Paquelet, Luc Le Magoarou, Matthieu Crussière

Abstract: In a multiple-input-multiple-output (MIMO) communication system, the multipath fading is averaged over radio links. This well-known channel hardening phenomenon plays a central role in the design of massive MIMO systems. The aim of this paper is to study channel hardening using a physical channel model in which the influences of propagation rays and antenna array topologies are highlighted. A meas… ▽ More In a multiple-input-multiple-output (MIMO) communication system, the multipath fading is averaged over radio links. This well-known channel hardening phenomenon plays a central role in the design of massive MIMO systems. The aim of this paper is to study channel hardening using a physical channel model in which the influences of propagation rays and antenna array topologies are highlighted. A measure of channel hardening is derived through the coefficient of variation of the channel gain. Our analyses and closed form results based on the used physical model are consistent with those of the literature relying on more abstract Rayleigh fading models, but offer further insights on the relationship with channel characteristics. △ Less

Submitted 20 April, 2018; originally announced April 2018.

Parametric channel estimation for massive MIMO

Authors: Luc Le Magoarou, Stéphane Paquelet

Abstract: Channel state information is crucial to achieving the capacity of multi-antenna (MIMO) wireless communication systems. It requires estimating the channel matrix. This estimation task is studied, considering a sparse channel model particularly suited to millimeter wave propagation, as well as a general measurement model taking into account hybrid architectures. The contribution is twofold. First, t… ▽ More Channel state information is crucial to achieving the capacity of multi-antenna (MIMO) wireless communication systems. It requires estimating the channel matrix. This estimation task is studied, considering a sparse channel model particularly suited to millimeter wave propagation, as well as a general measurement model taking into account hybrid architectures. The contribution is twofold. First, the Cram{é}r-Rao bound in this context is derived. Second, interpretation of the Fisher Information Matrix structure allows to assess the role of system parameters, as well as to propose asymptotically optimal and computationally efficient estimation algorithms. △ Less

Submitted 5 April, 2018; v1 submitted 23 October, 2017; originally announced October 2017.