Showing 1–18 of 18 results for author: Latre, S

Designing, Develo**, and Validating Network Intelligence for Scaling in Service-Based Architectures based on Deep Reinforcement Learning

Authors: Paola Soto, Miguel Camelo, Danny De Vleeschauwer, Yorick De Bock, Nina Slamnik-Kriještorac, Chia-Yu Chang, Natalia Gaviria, Erik Mannens, Juan F. Botero, Steven Latré

Abstract: Automating network processes without human intervention is crucial for the complex 6G environment. This requires zero-touch management and orchestration, the integration of Network Intelligence (NI) into the network architecture, and the efficient lifecycle management of intelligent functions. Reinforcement Learning (RL) plays a key role in this context, offering intelligent decision-making capabi… ▽ More Automating network processes without human intervention is crucial for the complex 6G environment. This requires zero-touch management and orchestration, the integration of Network Intelligence (NI) into the network architecture, and the efficient lifecycle management of intelligent functions. Reinforcement Learning (RL) plays a key role in this context, offering intelligent decision-making capabilities suited to networks' dynamic nature. Despite its potential, integrating RL poses challenges in model development and application. To tackle those issues, we delve into designing, develo**, and validating RL algorithms for scaling network functions in service-based network architectures such as Open Radio Access Network (O-RAN). It builds upon and expands previous research on RL lifecycle management by proposing several RL algorithms and Reward Functions (RFns). Our proposed methodology is anchored on a dual approach: firstly, it evaluates the training performance of these algorithms under varying RFns, and secondly, it validates their performance after being trained to discern the practical applicability in real-world settings. We show that, despite significant progress, the development stage of RL techniques for networking applications, particularly in scaling scenarios, still leaves room for significant improvements. This study underscores the importance of ongoing research and development to enhance the practicality and resilience of RL techniques in real-world networking environments. △ Less

Submitted 7 May, 2024; originally announced May 2024.

Inferring the relationship between soil temperature and the normalized difference vegetation index with machine learning

Authors: Steven Mortier, Amir Hamedpour, Bart Bussmann, Ruth Phoebe Tchana Wandji, Steven Latré, Bjarni D. Sigurdsson, Tom De Schepper, Tim Verdonck

Abstract: Changes in climate can greatly affect the phenology of plants, which can have important feedback effects, such as altering the carbon cycle. These phenological feedback effects are often induced by a shift in the start or end dates of the growing season of plants. The normalized difference vegetation index (NDVI) serves as a straightforward indicator for assessing the presence of green vegetation… ▽ More Changes in climate can greatly affect the phenology of plants, which can have important feedback effects, such as altering the carbon cycle. These phenological feedback effects are often induced by a shift in the start or end dates of the growing season of plants. The normalized difference vegetation index (NDVI) serves as a straightforward indicator for assessing the presence of green vegetation and can also provide an estimation of the plants' growing season. In this study, we investigated the effect of soil temperature on the timing of the start of the season (SOS), timing of the peak of the season (POS), and the maximum annual NDVI value (PEAK) in subarctic grassland ecosystems between 2014 and 2019. We also explored the impact of other meteorological variables, including air temperature, precipitation, and irradiance, on the inter-annual variation in vegetation phenology. Using machine learning (ML) techniques and SHapley Additive exPlanations (SHAP) values, we analyzed the relative importance and contribution of each variable to the phenological predictions. Our results reveal a significant relationship between soil temperature and SOS and POS, indicating that higher soil temperatures lead to an earlier start and peak of the growing season. However, the Peak NDVI values showed just a slight increase with higher soil temperatures. The analysis of other meteorological variables demonstrated their impacts on the inter-annual variation of the vegetation phenology. Ultimately, this study contributes to our knowledge of the relationships between soil temperature, meteorological variables, and vegetation phenology, providing valuable insights for predicting vegetation phenology characteristics and managing subarctic grasslands in the face of climate change. Additionally, this work provides a solid foundation for future ML-based vegetation phenology studies. △ Less

Submitted 19 December, 2023; originally announced December 2023.

Comments: 31 pages, 7 figures, 5 tables

An Encoding Framework for Binarized Images using HyperDimensional Computing

Authors: Laura Smets, Werner Van Leekwijck, Ing Jyh Tsang, Steven Latré

Abstract: Hyperdimensional Computing (HDC) is a brain-inspired and light-weight machine learning method. It has received significant attention in the literature as a candidate to be applied in the wearable internet of things, near-sensor artificial intelligence applications and on-device processing. HDC is computationally less complex than traditional deep learning algorithms and typically achieves moderate… ▽ More Hyperdimensional Computing (HDC) is a brain-inspired and light-weight machine learning method. It has received significant attention in the literature as a candidate to be applied in the wearable internet of things, near-sensor artificial intelligence applications and on-device processing. HDC is computationally less complex than traditional deep learning algorithms and typically achieves moderate to good classification performance. A key aspect that determines the performance of HDC is the encoding of the input data to the hyperdimensional (HD) space. This article proposes a novel light-weight approach relying only on native HD arithmetic vector operations to encode binarized images that preserves similarity of patterns at nearby locations by using point of interest selection and local linear map**. The method reaches an accuracy of 97.35% on the test set for the MNIST data set and 84.12% for the Fashion-MNIST data set. These results outperform other studies using baseline HDC with different encoding approaches and are on par with more complex hybrid HDC models. The proposed encoding approach also demonstrates a higher robustness to noise and blur compared to the baseline encoding. △ Less

Submitted 1 December, 2023; originally announced December 2023.

Co-learning synaptic delays, weights and adaptation in spiking neural networks

Authors: Lucas Deckers, Laurens Van Damme, Ing Jyh Tsang, Werner Van Leekwijck, Steven Latré

Abstract: Spiking neural networks (SNN) distinguish themselves from artificial neural networks (ANN) because of their inherent temporal processing and spike-based computations, enabling a power-efficient implementation in neuromorphic hardware. In this paper, we demonstrate that data processing with spiking neurons can be enhanced by co-learning the connection weights with two other biologically inspired ne… ▽ More Spiking neural networks (SNN) distinguish themselves from artificial neural networks (ANN) because of their inherent temporal processing and spike-based computations, enabling a power-efficient implementation in neuromorphic hardware. In this paper, we demonstrate that data processing with spiking neurons can be enhanced by co-learning the connection weights with two other biologically inspired neuronal features: 1) a set of parameters describing neuronal adaptation processes and 2) synaptic propagation delays. The former allows the spiking neuron to learn how to specifically react to incoming spikes based on its past. The trained adaptation parameters result in neuronal heterogeneity, which is found in the brain and also leads to a greater variety in available spike patterns. The latter enables to learn to explicitly correlate patterns that are temporally distanced. Synaptic delays reflect the time an action potential requires to travel from one neuron to another. We show that each of the co-learned features separately leads to an improvement over the baseline SNN and that the combination of both leads to state-of-the-art SNN results on all speech recognition datasets investigated with a simple 2-hidden layer feed-forward network. Our SNN outperforms the ANN on the neuromorpic datasets (Spiking Heidelberg Digits and Spiking Speech Commands), even with fewer trainable parameters. On the 35-class Google Speech Commands dataset, our SNN also outperforms a GRU of similar size. Our work presents brain-inspired improvements to SNN that enable them to excel over an equivalent ANN of similar size on tasks with rich temporal dynamics. △ Less

Submitted 12 September, 2023; originally announced November 2023.

Comments: 15 pages, 8 figures

Training a HyperDimensional Computing Classifier using a Threshold on its Confidence

Authors: Laura Smets, Werner Van Leekwijck, Ing Jyh Tsang, Steven Latre

Abstract: Hyperdimensional computing (HDC) has become popular for light-weight and energy-efficient machine learning, suitable for wearable Internet-of-Things (IoT) devices and near-sensor or on-device processing. HDC is computationally less complex than traditional deep learning algorithms and achieves moderate to good classification performance. This article proposes to extend the training procedure in HD… ▽ More Hyperdimensional computing (HDC) has become popular for light-weight and energy-efficient machine learning, suitable for wearable Internet-of-Things (IoT) devices and near-sensor or on-device processing. HDC is computationally less complex than traditional deep learning algorithms and achieves moderate to good classification performance. This article proposes to extend the training procedure in HDC by taking into account not only wrongly classified samples, but also samples that are correctly classified by the HDC model but with low confidence. As such, a confidence threshold is introduced that can be tuned for each dataset to achieve the best classification accuracy. The proposed training procedure is tested on UCIHAR, CTG, ISOLET and HAND dataset for which the performance consistently improves compared to the baseline across a range of confidence threshold values. The extended training procedure also results in a shift towards higher confidence values of the correctly classified samples making the classifier not only more accurate but also more confident about its predictions. △ Less

Submitted 30 November, 2023; v1 submitted 30 May, 2023; originally announced May 2023.

Journal ref: Neural Computation, 35(12), 2006-2023 (2023)

Deep set conditioned latent representations for action recognition

Authors: Akash Singh, Tom De Schepper, Kevin Mets, Peter Hellinckx, Jose Oramas, Steven Latre

Abstract: In recent years multi-label, multi-class video action recognition has gained significant popularity. While reasoning over temporally connected atomic actions is mundane for intelligent species, standard artificial neural networks (ANN) still struggle to classify them. In the real world, atomic actions often temporally connect to form more complex composite actions. The challenge lies in recognisin… ▽ More In recent years multi-label, multi-class video action recognition has gained significant popularity. While reasoning over temporally connected atomic actions is mundane for intelligent species, standard artificial neural networks (ANN) still struggle to classify them. In the real world, atomic actions often temporally connect to form more complex composite actions. The challenge lies in recognising composite action of varying durations while other distinct composite or atomic actions occur in the background. Drawing upon the success of relational networks, we propose methods that learn to reason over the semantic concept of objects and actions. We empirically show how ANNs benefit from pretraining, relational inductive biases and unordered set-based latent representations. In this paper we propose deep set conditioned I3D (SCI3D), a two stream relational network that employs latent representation of state and visual representation for reasoning over events and actions. They learn to reason about temporally connected actions in order to identify all of them in the video. The proposed method achieves an improvement of around 1.49% mAP in atomic action recognition and 17.57% mAP in composite action recognition, over a I3D-NL baseline, on the CATER dataset. △ Less

Submitted 21 December, 2022; originally announced December 2022.

Comments: Conference VISAPP 2022, 11 pages,5 figures, 2 Tables, 6 plots

Journal ref: In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 5: VISAPP, ISBN 978-989-758-555-5; ISSN 2184-4321, year 2022, pages 456-466

Structured Exploration Through Instruction Enhancement for Object Navigation

Authors: Matthias Hutsebaut-Buysse, Kevin Mets, Tom De Schepper, Steven Latré

Abstract: Finding an object of a specific class in an unseen environment remains an unsolved navigation problem. Hence, we propose a hierarchical learning-based method for object navigation. The top-level is capable of high-level planning, and building a memory on a floorplan-level (e.g., which room makes the most sense for the agent to visit next, where has the agent already been?). While the lower-level i… ▽ More Finding an object of a specific class in an unseen environment remains an unsolved navigation problem. Hence, we propose a hierarchical learning-based method for object navigation. The top-level is capable of high-level planning, and building a memory on a floorplan-level (e.g., which room makes the most sense for the agent to visit next, where has the agent already been?). While the lower-level is tasked with efficiently navigating between rooms and looking for objects in them. Instructions can be provided to the agent using a simple synthetic language. The top-level intelligently enhances the instructions in order to make the overall task more tractable. Language grounding, map** instructions to visual observations, is performed by utilizing an additional separate supervised trained goal assessment module. We demonstrate the effectiveness of our method on a dynamic configurable domestic environment. △ Less

Submitted 15 November, 2022; originally announced November 2022.

Comments: Paper accepted to the BNAIC/BeNeLearn 2022 conference

An Analysis of Discretization Methods for Communication Learning with Multi-Agent Reinforcement Learning

Authors: Astrid Vanneste, Simon Vanneste, Kevin Mets, Tom De Schepper, Siegfried Mercelis, Steven Latré, Peter Hellinckx

Abstract: Communication is crucial in multi-agent reinforcement learning when agents are not able to observe the full state of the environment. The most common approach to allow learned communication between agents is the use of a differentiable communication channel that allows gradients to flow between agents as a form of feedback. However, this is challenging when we want to use discrete messages to redu… ▽ More Communication is crucial in multi-agent reinforcement learning when agents are not able to observe the full state of the environment. The most common approach to allow learned communication between agents is the use of a differentiable communication channel that allows gradients to flow between agents as a form of feedback. However, this is challenging when we want to use discrete messages to reduce the message size since gradients cannot flow through a discrete communication channel. Previous work proposed methods to deal with this problem. However, these methods are tested in different communication learning architectures and environments, making it hard to compare them. In this paper, we compare several state-of-the-art discretization methods as well as two methods that have not been used for communication learning before. We do this comparison in the context of communication learning using gradients from other agents and perform tests on several environments. Our results show that none of the methods is best in all environments. The best choice in discretization method greatly depends on the environment. However, the discretize regularize unit (DRU), straight through DRU and the straight through gumbel softmax show the most consistent results across all the tested environments. Therefore, these methods prove to be the best choice for general use while the straight through estimator and the gumbel softmax may provide better results in specific environments but fail completely in others. △ Less

Submitted 12 April, 2022; originally announced April 2022.

Comments: Accepted at Adaptive and Learning Agents Workshop (ALA 2022) https://ala2022.github.io/

Learning to Communicate with Reinforcement Learning for an Adaptive Traffic Control System

Authors: Simon Vanneste, Gauthier de Borrekens, Stig Bosmans, Astrid Vanneste, Kevin Mets, Siegfried Mercelis, Steven Latré, Peter Hellinckx

Abstract: Recent work in multi-agent reinforcement learning has investigated inter agent communication which is learned simultaneously with the action policy in order to improve the team reward. In this paper, we investigate independent Q-learning (IQL) without communication and differentiable inter-agent learning (DIAL) with learned communication on an adaptive traffic control system (ATCS). In real world… ▽ More Recent work in multi-agent reinforcement learning has investigated inter agent communication which is learned simultaneously with the action policy in order to improve the team reward. In this paper, we investigate independent Q-learning (IQL) without communication and differentiable inter-agent learning (DIAL) with learned communication on an adaptive traffic control system (ATCS). In real world ATCS, it is impossible to present the full state of the environment to every agent so in our simulation, the individual agents will only have a limited observation of the full state of the environment. The ATCS will be simulated using the Simulation of Urban MObility (SUMO) traffic simulator in which two connected intersections are simulated. Every intersection is controlled by an agent which has the ability to change the direction of the traffic flow. Our results show that a DIAL agent outperforms an independent Q-learner on both training time and on maximum achieved reward as it is able to share relevant information with the other agents. △ Less

Submitted 29 October, 2021; originally announced October 2021.

Mixed Cooperative-Competitive Communication Using Multi-Agent Reinforcement Learning

Authors: Astrid Vanneste, Wesley Van Wijnsberghe, Simon Vanneste, Kevin Mets, Siegfried Mercelis, Steven Latré, Peter Hellinckx

Abstract: By using communication between multiple agents in multi-agent environments, one can reduce the effects of partial observability by combining one agent's observation with that of others in the same dynamic environment. While a lot of successful research has been done towards communication learning in cooperative settings, communication learning in mixed cooperative-competitive settings is also impo… ▽ More By using communication between multiple agents in multi-agent environments, one can reduce the effects of partial observability by combining one agent's observation with that of others in the same dynamic environment. While a lot of successful research has been done towards communication learning in cooperative settings, communication learning in mixed cooperative-competitive settings is also important and brings its own complexities such as the opposing team overhearing the communication. In this paper, we apply differentiable inter-agent learning (DIAL), designed for cooperative settings, to a mixed cooperative-competitive setting. We look at the difference in performance between communication that is private for a team and communication that can be overheard by the other team. Our research shows that communicating agents are able to achieve similar performance to fully observable agents after a given training period in our chosen environment. Overall, we find that sharing communication across teams results in decreased performance for the communicating team in comparison to results achieved with private communication. △ Less

Submitted 29 October, 2021; originally announced October 2021.

A Review of the Deep Sea Treasure problem as a Multi-Objective Reinforcement Learning Benchmark

Authors: Amber Cassimon, Reinout Eyckerman, Siegfried Mercelis, Steven Latré, Peter Hellinckx

Abstract: In this paper, the authors investigate the Deep Sea Treasure (DST) problem as proposed by Vamplew et al. Through a number of proofs, the authors show the original DST problem to be quite basic, and not always representative of practical Multi-Objective Optimization problems. In an attempt to bring theory closer to practice, the authors propose an alternative, improved version of the DST problem, a… ▽ More In this paper, the authors investigate the Deep Sea Treasure (DST) problem as proposed by Vamplew et al. Through a number of proofs, the authors show the original DST problem to be quite basic, and not always representative of practical Multi-Objective Optimization problems. In an attempt to bring theory closer to practice, the authors propose an alternative, improved version of the DST problem, and prove that some of the properties that simplify the original DST problem no longer hold. The authors also provide a reference implementation and perform a comparison between their implementation, and other existing open-source implementations of the problem. Finally, the authors also provide a complete Pareto-front for their new DST problem. △ Less

Submitted 21 May, 2024; v1 submitted 13 October, 2021; originally announced October 2021.

Comments: 10 pages, 4 figures; Fixed Supplementary Materials PDF

An optimized application-context relocation approach for Connected and Automated Mobility (CAM)

Authors: Nina Slamnik-Kriještorac, Steven Latré, Johann M. Marquez-Barja

Abstract: In this paper, we study and present a management and orchestration framework for vehicular communications, which enables service continuity for the vehicle via an optimized application-context relocation approach. To optimize the transfer of the application-context for Connected and Automated Mobility (CAM) services, our MEC orchestrator performs prediction of resource availability in the edge inf… ▽ More In this paper, we study and present a management and orchestration framework for vehicular communications, which enables service continuity for the vehicle via an optimized application-context relocation approach. To optimize the transfer of the application-context for Connected and Automated Mobility (CAM) services, our MEC orchestrator performs prediction of resource availability in the edge infrastructure based on the Long Short-Term Memory (LSTM) model, and it makes a final decision on relocation by calculating the outcome of a Multi-Criteria Decision-Making (MCDM) algorithm, taking into account the i) resource prediction, ii) latency and bandwidth on the communication links, and iii) geographical locations of the vehicle and edge hosts in the network infrastructure. Furthermore, we have built a proof-of-concept for the orchestration framework in a real-life distributed testbed environment, to showcase the efficiency in optimizing the edge host selection and application context relocation towards achieving continuity of a service that informs vehicle about the driving conditions on the road. △ Less

Submitted 23 September, 2021; originally announced September 2021.

Journal ref: IEEE 5G for Connected and Automated Mobility (CAM), 2021

Neural Additive Vector Autoregression Models for Causal Discovery in Time Series

Authors: Bart Bussmann, Jannes Nys, Steven Latré

Abstract: Causal structure discovery in complex dynamical systems is an important challenge for many scientific domains. Although data from (interventional) experiments is usually limited, large amounts of observational time series data sets are usually available. Current methods that learn causal structure from time series often assume linear relationships. Hence, they may fail in realistic settings that c… ▽ More Causal structure discovery in complex dynamical systems is an important challenge for many scientific domains. Although data from (interventional) experiments is usually limited, large amounts of observational time series data sets are usually available. Current methods that learn causal structure from time series often assume linear relationships. Hence, they may fail in realistic settings that contain nonlinear relations between the variables. We propose Neural Additive Vector Autoregression (NAVAR) models, a neural approach to causal structure learning that can discover nonlinear relationships. We train deep neural networks that extract the (additive) Granger causal influences from the time evolution in multi-variate time series. The method achieves state-of-the-art results on various benchmark data sets for causal discovery, while providing clear interpretations of the mapped causal relations. △ Less

Submitted 18 October, 2021; v1 submitted 19 October, 2020; originally announced October 2020.

Comments: 11 pages, 5 figures

Journal ref: Discovery Science. DS 2021. Lecture Notes in Computer Science, vol 12986. Springer, Cham

HTMRL: Biologically Plausible Reinforcement Learning with Hierarchical Temporal Memory

Authors: Jakob Struye, Kevin Mets, Steven Latré

Abstract: Building Reinforcement Learning (RL) algorithms which are able to adapt to continuously evolving tasks is an open research challenge. One technology that is known to inherently handle such non-stationary input patterns well is Hierarchical Temporal Memory (HTM), a general and biologically plausible computational model for the human neocortex. As the RL paradigm is inspired by human learning, HTM i… ▽ More Building Reinforcement Learning (RL) algorithms which are able to adapt to continuously evolving tasks is an open research challenge. One technology that is known to inherently handle such non-stationary input patterns well is Hierarchical Temporal Memory (HTM), a general and biologically plausible computational model for the human neocortex. As the RL paradigm is inspired by human learning, HTM is a natural framework for an RL algorithm supporting non-stationary environments. In this paper, we present HTMRL, the first strictly HTM-based RL algorithm. We empirically and statistically show that HTMRL scales to many states and actions, and demonstrate that HTM's ability for adapting to changing patterns extends to RL. Specifically, HTMRL performs well on a 10-armed bandit after 750 steps, but only needs a third of that to adapt to the bandit suddenly shuffling its arms. HTMRL is the first iteration of a novel RL approach, with the potential of extending to a capable algorithm for Meta-RL. △ Less

Submitted 18 September, 2020; originally announced September 2020.

Pre-trained Word Embeddings for Goal-conditional Transfer Learning in Reinforcement Learning

Authors: Matthias Hutsebaut-Buysse, Kevin Mets, Steven Latré

Abstract: Reinforcement learning (RL) algorithms typically start tabula rasa, without any prior knowledge of the environment, and without any prior skills. This however often leads to low sample efficiency, requiring a large amount of interaction with the environment. This is especially true in a lifelong learning setting, in which the agent needs to continually extend its capabilities. In this paper, we ex… ▽ More Reinforcement learning (RL) algorithms typically start tabula rasa, without any prior knowledge of the environment, and without any prior skills. This however often leads to low sample efficiency, requiring a large amount of interaction with the environment. This is especially true in a lifelong learning setting, in which the agent needs to continually extend its capabilities. In this paper, we examine how a pre-trained task-independent language model can make a goal-conditional RL agent more sample efficient. We do this by facilitating transfer learning between different related tasks. We experimentally demonstrate our approach on a set of object navigation tasks. △ Less

Submitted 10 July, 2020; originally announced July 2020.

Comments: Paper accepted to the ICML 2020 Language in Reinforcement Learning (LaReL) Workshop

Learning to Communicate Using Counterfactual Reasoning

Authors: Simon Vanneste, Astrid Vanneste, Kevin Mets, Tom De Schepper, Ali Anwar, Siegfried Mercelis, Steven Latré, Peter Hellinckx

Abstract: Learning to communicate in order to share state information is an active problem in the area of multi-agent reinforcement learning (MARL). The credit assignment problem, the non-stationarity of the communication environment and the creation of influenceable agents are major challenges within this research field which need to be overcome in order to learn a valid communication protocol. This paper… ▽ More Learning to communicate in order to share state information is an active problem in the area of multi-agent reinforcement learning (MARL). The credit assignment problem, the non-stationarity of the communication environment and the creation of influenceable agents are major challenges within this research field which need to be overcome in order to learn a valid communication protocol. This paper introduces the novel multi-agent counterfactual communication learning (MACC) method which adapts counterfactual reasoning in order to overcome the credit assignment problem for communicating agents. Secondly, the non-stationarity of the communication environment while learning the communication Q-function is overcome by creating the communication Q-function using the action policy of the other agents and the Q-function of the action environment. Additionally, a social loss function is introduced in order to create influenceable agents which is required to learn a valid communication protocol. Our experiments show that MACC is able to outperform the state-of-the-art baselines in four different scenarios in the Particle environment. △ Less

Submitted 26 April, 2022; v1 submitted 12 June, 2020; originally announced June 2020.

Comments: Accepted at Adaptive and Learning Agents Workshop (ALA 2022) https://ala2022.github.io/

Neurosciences and 6G: Lessons from and Needs of Communicative Brains

Authors: Renan C. Moioli, Pedro H. J. Nardelli, Michael Taynnan Barros, Walid Saad, Amin Hekmatmanesh, Pedro Gória, Arthur S. de Sena, Merim Dzaferagic, Harun Siljak, Werner van Leekwijck, Dick Carrillo, Steven Latré

Abstract: This paper presents the first comprehensive tutorial on a promising research field located at the frontier of two well-established domains: Neurosciences and wireless communications, motivated by the ongoing efforts to define how the sixth generation of mobile networks (6G) will be. In particular, this tutorial first provides a novel integrative approach that bridges the gap between these two, see… ▽ More This paper presents the first comprehensive tutorial on a promising research field located at the frontier of two well-established domains: Neurosciences and wireless communications, motivated by the ongoing efforts to define how the sixth generation of mobile networks (6G) will be. In particular, this tutorial first provides a novel integrative approach that bridges the gap between these two, seemingly disparate fields. Then, we present the state-of-the-art and key challenges of these two topics. In particular, we propose a novel systematization that divides the contributions into two groups, one focused on what neurosciences will offer to 6G in terms of new applications and systems architecture (Neurosciences for Wireless), and the other focused on how wireless communication theory and 6G systems can provide new ways to study the brain (Wireless for Neurosciences). For the first group, we concretely explain how current scientific understanding of the brain would enable new application for 6G within the context of a new type of service that we dub braintype communications and that has more stringent requirements than human- and machine-type communication. In this regard, we expose the key requirements of brain-type communication services and we discuss how future wireless networks can be equipped to deal with such services. Meanwhile, for the second group, we thoroughly explore modern communication system paradigms, including Internet of Bio-nano Things and chaosbased communications, in addition to highlighting how complex systems tools can help bridging 6G and neuroscience applications. Brain-controlled vehicles are then presented as our case study. All in all, this tutorial is expected to provide a largely missing articulation between these two emerging fields while delineating concrete ways to move forward in such an interdisciplinary endeavor. △ Less

Submitted 3 April, 2020; originally announced April 2020.

Fast Task-Adaptation for Tasks Labeled Using Natural Language in Reinforcement Learning

Authors: Matthias Hutsebaut-Buysse, Kevin Mets, Steven Latré

Abstract: Over its lifetime, a reinforcement learning agent is often tasked with different tasks. How to efficiently adapt a previously learned control policy from one task to another, remains an open research question. In this paper, we investigate how instructions formulated in natural language can enable faster and more effective task adaptation. This can serve as the basis for develo** language instru… ▽ More Over its lifetime, a reinforcement learning agent is often tasked with different tasks. How to efficiently adapt a previously learned control policy from one task to another, remains an open research question. In this paper, we investigate how instructions formulated in natural language can enable faster and more effective task adaptation. This can serve as the basis for develo** language instructed skills, which can be used in a lifelong learning setting. Our method is capable of assessing, given a set of developed base control policies, which policy will adapt best to a new unseen task. △ Less

Submitted 9 October, 2019; originally announced October 2019.