-
Balancing Performance and Cost for Two-Hop Cooperative Communications: Stackelberg Game and Distributed Multi-Agent Reinforcement Learning
Authors:
Yuanzhe Geng,
Erwu Liu,
Wei Ni,
Rui Wang,
Yan Liu,
Hao Xu,
Chen Cai,
Abbas Jamalipour
Abstract:
This paper aims to balance performance and cost in a two-hop wireless cooperative communication network where the source and relays have contradictory optimization goals and make decisions in a distributed manner. This differs from most existing works that have typically assumed that source and relay nodes follow a schedule created implicitly by a central controller. We propose that the relays for…
▽ More
This paper aims to balance performance and cost in a two-hop wireless cooperative communication network where the source and relays have contradictory optimization goals and make decisions in a distributed manner. This differs from most existing works that have typically assumed that source and relay nodes follow a schedule created implicitly by a central controller. We propose that the relays form an alliance in an attempt to maximize the benefit of relaying while the source aims to increase the channel capacity cost-effectively. To this end, we establish the trade problem as a Stackelberg game, and prove the existence of its equilibrium. Another important aspect is that we use multi-agent reinforcement learning (MARL) to approach the equilibrium in a situation where the instantaneous channel state information (CSI) is unavailable, and the source and relays do not have knowledge of each other's goal. A multi-agent deep deterministic policy gradient-based framework is designed, where the relay alliance and the source act as agents. Experiments demonstrate that the proposed method can obtain an acceptable performance that is close to the game-theoretic equilibrium for all players under time-invariant environments, which considerably outperforms its potential alternatives and is only about 2.9% away from the optimal solution.
△ Less
Submitted 17 June, 2024;
originally announced June 2024.
-
A Material Sensing-Assisted Initial Beam Establishment Method for JCAS Systems
Authors:
Yi Geng
Abstract:
Communication systems operating at high frequency bands must use narrow beams to compensate the high path loss. However, it is incredibly time-consuming to achieve beam alignment between the transmitter and receiver due to the large volume of beam space with narrow beams. The high latency of initial beam establishment will challenge the implementation of future 6G networks at high frequency bands.…
▽ More
Communication systems operating at high frequency bands must use narrow beams to compensate the high path loss. However, it is incredibly time-consuming to achieve beam alignment between the transmitter and receiver due to the large volume of beam space with narrow beams. The high latency of initial beam establishment will challenge the implementation of future 6G networks at high frequency bands. To tackle this problem, this paper proposes an initial beam establishment method using the material sensing results from joint communications and sensing (JCAS) systems. The reflection loss (RL) induced by each reflector can be predicted by exploiting the pre-identified material information of reflectors in the environment. The base station (BS) first scans the beam directions with low RL and establishes the connection immediately without swee** the rest of the beam directions. In this way, the latency of initial beam establishment is significantly reduced.
△ Less
Submitted 16 February, 2024;
originally announced February 2024.
-
Diagonal Waveform and Algorithm to Estimate Range and Velocity in Multi-Object Scenarios
Authors:
Yi Geng
Abstract:
Waveform design for joint communication and sensing (JCAS) is an important research direction, focusing on providing an optimal tradeoff between communication and sensing performance. In this paper, we first describe the conventional grid-type waveform structure and the corresponding two-dimension (2D)-discrete Fourier transform (DFT) algorithm. We then introduce an emerging diagonal scheme, inclu…
▽ More
Waveform design for joint communication and sensing (JCAS) is an important research direction, focusing on providing an optimal tradeoff between communication and sensing performance. In this paper, we first describe the conventional grid-type waveform structure and the corresponding two-dimension (2D)-discrete Fourier transform (DFT) algorithm. We then introduce an emerging diagonal scheme, including a diagonal waveform structure and corresponding 1D-DFT diagonal algorithm. The diagonal scheme substantially reduces the signaling overhead and computational complexity compared to the conventional 2D-DFT algorithm while still achieving the same radar performance. But the previous study of diagonal waveform used a single target to evaluate the performance of the diagonal scheme. This paper verifies the diagonal waveform with simulations demonstrating its feasibility in a traffic monitoring scenario with multiple vehicles.
△ Less
Submitted 19 June, 2023;
originally announced June 2023.
-
Phase regeneration of QPSK signals based on Kerr soliton combs in a highly nonlinear optical fiber
Authors:
Xinjie Han,
Yong Geng,
Haocheng Ke,
Kun Qiu
Abstract:
We demonstrate an all-optical phase regeneration technique based on Kerr soliton combs, which can realize degraded quaternary phase shift keying (QPSK) signal regeneration through phase-sensitive amplification. A Kerr soliton comb is generated at the receiver side of optical communication systems based on a carrier recovery scheme and is used as coherent dual pumps to achieve phase regeneration. O…
▽ More
We demonstrate an all-optical phase regeneration technique based on Kerr soliton combs, which can realize degraded quaternary phase shift keying (QPSK) signal regeneration through phase-sensitive amplification. A Kerr soliton comb is generated at the receiver side of optical communication systems based on a carrier recovery scheme and is used as coherent dual pumps to achieve phase regeneration. Our study will enhance the relay and reception performance of all-optical communication systems.
△ Less
Submitted 26 March, 2023; v1 submitted 22 March, 2023;
originally announced March 2023.
-
A Novel Waveform Design for OFDM-Based Joint Sensing and Communication System
Authors:
Yi Geng
Abstract:
The dominating waveform in 5G is orthogonal frequency division multiplexing (OFDM). OFDM will remain a promising waveform candidate for joint communication and sensing (JCAS) in 6G since OFDM can provide excellent data transmission capability and accurate sensing information. This paper proposes a novel OFDM-based diagonal waveform structure and corresponding signal processing algorithm. This appr…
▽ More
The dominating waveform in 5G is orthogonal frequency division multiplexing (OFDM). OFDM will remain a promising waveform candidate for joint communication and sensing (JCAS) in 6G since OFDM can provide excellent data transmission capability and accurate sensing information. This paper proposes a novel OFDM-based diagonal waveform structure and corresponding signal processing algorithm. This approach allocates the sensing signals along the diagonal of the time-frequency resource block. Therefore, the sensing signals in a linear structure span both the frequency and time domains. The range and velocity of the object can be estimated simultaneously by applying 1D-discrete Fourier transform (DFT) to the diagonal sensing signals. Compared to the conventional 2D-DFT OFDM radar algorithm, the computational complexity of the proposed algorithm is low. In addition, the sensing overhead can be substantially reduced. The performance of the proposed waveform is evaluated using simulation and analysis of results.
△ Less
Submitted 9 January, 2023;
originally announced January 2023.
-
Reinforcement Learning Based Robust Policy Design for Relay and Power Optimization in DF Relaying Networks
Authors:
Yuanzhe Geng,
Erwu Liu,
Rui Wang,
Pengcheng Sun,
Binyu Lu
Abstract:
In this paper, we study the outage minimization problem in a decode-and-forward cooperative network with relay uncertainty. To reduce the outage probability and improve the quality of service, existing researches usually rely on the assumption of both exact instantaneous channel state information (CSI) and environmental uncertainty. However, it is difficult to obtain perfect instantaneous CSI imme…
▽ More
In this paper, we study the outage minimization problem in a decode-and-forward cooperative network with relay uncertainty. To reduce the outage probability and improve the quality of service, existing researches usually rely on the assumption of both exact instantaneous channel state information (CSI) and environmental uncertainty. However, it is difficult to obtain perfect instantaneous CSI immediately under practical situations where channel states change rapidly, and the uncertainty in communication environments may not be observed, which makes traditional methods not applicable. Therefore, we turn to reinforcement learning (RL) methods for solutions, which do not need any prior knowledge of underlying channel or assumptions of environmental uncertainty. RL method is to learn from the interaction with communication environment, optimize its action policy, and then propose relay selection and power allocation schemes. We first analyse the robustness of RL action policy by giving the lower bound of the worst-case performance, when RL methods are applied to communication scenarios with environment uncertainty. Then, we propose a robust algorithm for outage probability minimization based on RL. Simulation results reveal that compared with traditional RL methods, our approach has better generalization ability and can improve the worst-case performance by about 6% when evaluated in unseen environments.
△ Less
Submitted 8 May, 2022;
originally announced May 2022.
-
Map-Assisted Material Identification at 100 GHz and Above Using Radio Access Technology
Authors:
Yi Geng
Abstract:
The inclusion of material identification in wireless communication system is an emerging area that offers many opportunities for 6G systems. By using reflected radio wave to determine the material of reflecting surface, not only the performance of 6G networks can be improved, but also some exciting applications can be developed. In this paper, we recap a few prior methods for material identificati…
▽ More
The inclusion of material identification in wireless communication system is an emerging area that offers many opportunities for 6G systems. By using reflected radio wave to determine the material of reflecting surface, not only the performance of 6G networks can be improved, but also some exciting applications can be developed. In this paper, we recap a few prior methods for material identification, then analyze the impact of thickness of reflecting surface on reflection coefficient and present a new concept "settling thickness", which indicates the minimum thickness of reflecting surface to induce steady reflection coefficient. Finally, we propose a novel material identification method based on ray-tracing and 3D-map. Compared to some prior methods that can be implemented in single-bounce-reflection scenario only, we extend the capability of the method to multiple-bounce-reflection scenarios.
△ Less
Submitted 23 January, 2022;
originally announced January 2022.
-
Linking Physical Objects to Their Digital Twins via Fiducial Markers Designed for Invisibility to Humans
Authors:
Mathew Schwartz,
Yong Geng,
Hakam Agha,
Rijeesh Kizhakidathazhath,
Danqing Liu,
Gabriele Lenzini,
Jan PF Lagerwall
Abstract:
The ability to label and track physical objects that are assets in digital representations of the world is foundational to many complex systems. Simple, yet powerful methods such as bar- and QR-codes have been highly successful, e.g. in the retail space, but the lack of security, limited information content and impossibility of seamless integration with the environment have prevented a large-scale…
▽ More
The ability to label and track physical objects that are assets in digital representations of the world is foundational to many complex systems. Simple, yet powerful methods such as bar- and QR-codes have been highly successful, e.g. in the retail space, but the lack of security, limited information content and impossibility of seamless integration with the environment have prevented a large-scale linking of physical objects to their digital twins. This paper proposes to link digital assets created through BIM with their physical counterparts using fiducial markers with patterns defined by Cholesteric Spherical Reflectors (CSRs), selective retroreflectors produced using liquid crystal self-assembly. The markers leverage the ability of CSRs to encode information that is easily detected and read with computer vision while remaining practically invisible to the human eye. We analyze the potential of a CSR-based infrastructure from the perspective of BIM, critically reviewing the outstanding challenges in applying this new class of functional materials, and we discuss extended opportunities arising in assisting autonomous mobile robots to reliably navigate human-populated environments, as well as in augmented reality.
△ Less
Submitted 12 May, 2021;
originally announced May 2021.
-
Coherent optical communications using coherence-cloned Kerr soliton microcombs
Authors:
Yong Geng,
Heng Zhou,
Wenwen Cui,
Xinjie Han,
Qiang Zhang,
Boyuan Liu,
Guangwei Deng,
Qiang Zhou,
Kun Qiu
Abstract:
Dissipative Kerr soliton microcomb has been recognized as a promising on-chip multi-wavelength laser source for fiber optical communications, as its comb lines possess frequency and phase stability far beyond independent lasers. In the scenarios of coherent optical transmission and interconnect, a highly beneficial but rarely explored target is to re-generate a Kerr soliton microcomb at the receiv…
▽ More
Dissipative Kerr soliton microcomb has been recognized as a promising on-chip multi-wavelength laser source for fiber optical communications, as its comb lines possess frequency and phase stability far beyond independent lasers. In the scenarios of coherent optical transmission and interconnect, a highly beneficial but rarely explored target is to re-generate a Kerr soliton microcomb at the receiver side as local oscillators that conserve the frequency and phase property of the incoming data carriers, so that to enable coherent detection with minimized optical and electrical compensations. Here, by using the techniques of pump laser conveying and two-point locking, we implement re-generation of a Kerr soliton microcomb that faithfully clones the frequency and phase coherence of another microcomb sent from 50 km away. Moreover, leveraging the coherence-cloned soliton microcombs as carriers and local oscillators, we demonstrate terabit coherent data interconnect, wherein traditional digital processes for frequency offset estimation is totally dispensed with, and carrier phase estimation is substantially simplified via slowed-down phase estimation rate per channel and joint phase estimation among multiple channels. Our work reveals that, in addition to providing a multitude of laser tones, regulating the frequency and phase of Kerr soliton microcombs among transmitters and receivers can significantly improve coherent communication in terms of performance, power consumption, and simplicity.
△ Less
Submitted 31 December, 2020;
originally announced January 2021.
-
Deep Deterministic Policy Gradient for Relay Selection and Power Allocation in Cooperative Communication Network
Authors:
Yuanzhe Geng,
Erwu Liu,
Rui Wang,
Yiming Liu,
Jie Wang,
Gang Shen,
Zhao Dong
Abstract:
Perfect channel state information (CSI) is usually required when considering relay selection and power allocation in cooperative communication. However, it is difficult to get an accurate CSI in practical situations. In this letter, we study the outage probability minimizing problem based on optimizing relay selection and transmission power. We propose a prioritized experience replay aided deep de…
▽ More
Perfect channel state information (CSI) is usually required when considering relay selection and power allocation in cooperative communication. However, it is difficult to get an accurate CSI in practical situations. In this letter, we study the outage probability minimizing problem based on optimizing relay selection and transmission power. We propose a prioritized experience replay aided deep deterministic policy gradient learning framework, which can find an optimal solution by dealing with continuous action space, without any prior knowledge of CSI. Simulation results reveal that our approach outperforms reinforcement learning based methods in existing literatures, and improves the communication success rate by about 4%.
△ Less
Submitted 14 March, 2021; v1 submitted 11 December, 2020;
originally announced December 2020.
-
Hierarchical Reinforcement Learning for Relay Selection and Power Optimization in Two-Hop Cooperative Relay Network
Authors:
Yuanzhe Geng,
Erwu Liu,
Rui Wang,
Yiming Liu
Abstract:
Cooperative communication is an effective approach to improve spectrum utilization. In order to reduce outage probability of communication system, most studies propose various schemes for relay selection and power allocation, which are based on the assumption of channel state information (CSI). However, it is difficult to get an accurate CSI in practice. In this paper, we study the outage probabil…
▽ More
Cooperative communication is an effective approach to improve spectrum utilization. In order to reduce outage probability of communication system, most studies propose various schemes for relay selection and power allocation, which are based on the assumption of channel state information (CSI). However, it is difficult to get an accurate CSI in practice. In this paper, we study the outage probability minimizing problem subjected to a total transmission power constraint in a two-hop cooperative relay network. We use reinforcement learning (RL) methods to learn strategies for relay selection and power allocation, which do not need any prior knowledge of CSI but simply rely on the interaction with communication environment. It is noted that conventional RL methods, including most deep reinforcement learning (DRL) methods, cannot perform well when the search space is too large. Therefore, we first propose a DRL framework with an outage-based reward function, which is then used as a baseline. Then, we further propose a hierarchical reinforcement learning (HRL) framework and training algorithm. A key difference from other RL-based methods in existing literatures is that, our proposed HRL approach decomposes relay selection and power allocation into two hierarchical optimization objectives, which are trained in different levels. With the simplification of search space, the HRL approach can solve the problem of sparse reward, while the conventional RL method fails. Simulation results reveal that compared with traditional DRL method, the HRL training algorithm can reach convergence 30 training iterations earlier and reduce the outage probability by 5% in two-hop relay network with the same outage threshold.
△ Less
Submitted 28 January, 2021; v1 submitted 9 November, 2020;
originally announced November 2020.
-
Deep Reinforcement Learning Based Dynamic Route Planning for Minimizing Travel Time
Authors:
Yuanzhe Geng,
Erwu Liu,
Rui Wang,
Yiming Liu
Abstract:
Route planning is important in transportation. Existing works focus on finding the shortest path solution or using metrics such as safety and energy consumption to determine the planning. It is noted that most of these studies rely on prior knowledge of road network, which may be not available in certain situations. In this paper, we design a route planning algorithm based on deep reinforcement le…
▽ More
Route planning is important in transportation. Existing works focus on finding the shortest path solution or using metrics such as safety and energy consumption to determine the planning. It is noted that most of these studies rely on prior knowledge of road network, which may be not available in certain situations. In this paper, we design a route planning algorithm based on deep reinforcement learning (DRL) for pedestrians. We use travel time consumption as the metric, and plan the route by predicting pedestrian flow in the road network. We put an agent, which is an intelligent robot, on a virtual map. Different from previous studies, our approach assumes that the agent does not need any prior information about road network, but simply relies on the interaction with the environment. We propose a dynamically adjustable route planning (DARP) algorithm, where the agent learns strategies through a dueling deep Q network to avoid congested roads. Simulation results show that the DARP algorithm saves 52% of the time under congestion condition when compared with traditional shortest path planning algorithms.
△ Less
Submitted 3 November, 2020;
originally announced November 2020.
-
Information-Theoretic Bounds for Performance of Resource-Constrained Communication Systems
Authors:
Albert Y. S. Lam,
Yanhui Geng,
Victor O. K. Li
Abstract:
Resource-constrained systems are prevalent in communications. Such a system is composed of many components but only some of them can be allocated with resources such as time slots. According to the amount of information about the system, algorithms are employed to allocate resources and the overall system performance depends on the result of resource allocation. We do not always have complete info…
▽ More
Resource-constrained systems are prevalent in communications. Such a system is composed of many components but only some of them can be allocated with resources such as time slots. According to the amount of information about the system, algorithms are employed to allocate resources and the overall system performance depends on the result of resource allocation. We do not always have complete information, and thus, the system performance may not be satisfactory. In this work, we propose a general model for the resource-constrained communication systems. We draw the relationship between system information and performance and derive the performance bounds for the optimal algorithm for the system. This gives the expected performance corresponding to the available information, and we can determine if we should put more efforts to collect more accurate information before actually constructing an algorithm for the system. Several examples of applications in communications to the model are also given.
△ Less
Submitted 1 April, 2014;
originally announced April 2014.
-
Quasi-dynamic Traffic Light Control for a Single Intersection
Authors:
Yanfeng Geng,
Christos G. Cassandras
Abstract:
We address the traffic light control problem for a single intersection by viewing it as a stochastic hybrid system and develo** a Stochastic Flow Model (SFM) for it. We adopt a quasi-dynamic control policy based on partial state information defined by detecting whether vehicle backlog is above or below a certain threshold, without the need to observe an exact vehicle count. The policy is paramet…
▽ More
We address the traffic light control problem for a single intersection by viewing it as a stochastic hybrid system and develo** a Stochastic Flow Model (SFM) for it. We adopt a quasi-dynamic control policy based on partial state information defined by detecting whether vehicle backlog is above or below a certain threshold, without the need to observe an exact vehicle count. The policy is parameterized by green and red cycle lengths which depend on this partial state information. Using Infinitesimal Perturbation Analysis (IPA), we derive online gradient estimators of an average traffic congestion metric with respect to these controllable green and red cycle lengths when the vehicle backlog is above or below the threshold. The estimators are used to iteratively adjust light cycle lengths so as to improve performance and, in conjunction with a standard gradient-based algorithm, to seek optimal values which adapt to changing traffic conditions. Simulation results are included to illustrate the approach and quantify the benefits of quasidynamic traffic light control over earlier static approaches.
△ Less
Submitted 4 August, 2013;
originally announced August 2013.
-
Multi-intersection Traffic Light Control Using Infinitesimal Perturbation Analysis
Authors:
Yanfeng Geng,
Christos G. Cassandras
Abstract:
We address the traffic light control problem for multiple intersections in tandem by viewing it as a stochastic hybrid system and develo** a Stochastic Flow Model (SFM) for it. Using Infinitesimal Perturbation Analysis (IPA), we derive on-line gradient estimates of a cost metric with respect to the controllable green and red cycle lengths. The IPA estimators obtained require counting traffic lig…
▽ More
We address the traffic light control problem for multiple intersections in tandem by viewing it as a stochastic hybrid system and develo** a Stochastic Flow Model (SFM) for it. Using Infinitesimal Perturbation Analysis (IPA), we derive on-line gradient estimates of a cost metric with respect to the controllable green and red cycle lengths. The IPA estimators obtained require counting traffic light switchings and estimating car flow rates only when specific events occur. The estimators are used to iteratively adjust light cycle lengths to improve performance and, in conjunction with a standard gradient-based algorithm, to obtain optimal values which adapt to changing traffic conditions. Simulation results are included to illustrate the approach.
△ Less
Submitted 18 April, 2012; v1 submitted 9 April, 2012;
originally announced April 2012.
