-
DeepMPR: Enhancing Opportunistic Routing in Wireless Networks through Multi-Agent Deep Reinforcement Learning
Authors:
Saeed Kaviani,
Bo Ryu,
Ejaz Ahmed,
Deokseong Kim,
Jae Kim,
Carrie Spiker,
Blake Harnden
Abstract:
Opportunistic routing relies on the broadcast capability of wireless networks. It brings higher reliability and robustness in highly dynamic and/or severe environments such as mobile or vehicular ad-hoc networks (MANETs/VANETs). To reduce the cost of broadcast, multicast routing schemes use the connected dominating set (CDS) or multi-point relaying (MPR) set to decrease the network overhead and he…
▽ More
Opportunistic routing relies on the broadcast capability of wireless networks. It brings higher reliability and robustness in highly dynamic and/or severe environments such as mobile or vehicular ad-hoc networks (MANETs/VANETs). To reduce the cost of broadcast, multicast routing schemes use the connected dominating set (CDS) or multi-point relaying (MPR) set to decrease the network overhead and hence, their selection algorithms are critical. Common MPR selection algorithms are heuristic, rely on coordination between nodes, need high computational power for large networks, and are difficult to tune for network uncertainties. In this paper, we use multi-agent deep reinforcement learning to design a novel MPR multicast routing technique, DeepMPR, which is outperforming the OLSR MPR selection algorithm while it does not require MPR announcement messages from the neighbors. Our evaluation results demonstrate the performance gains of our trained DeepMPR multicast forwarding policy compared to other popular techniques.
△ Less
Submitted 16 June, 2023;
originally announced June 2023.
-
DeepADMR: A Deep Learning based Anomaly Detection for MANET Routing
Authors:
Alex Yahja,
Saeed Kaviani,
Bo Ryu,
Jae H. Kim,
Kevin A. Larson
Abstract:
We developed DeepADMR, a novel neural anomaly detector for the deep reinforcement learning (DRL)-based DeepCQ+ MANET routing policy. The performance of DRL-based algorithms such as DeepCQ+ is only verified within the trained and tested environments, hence their deployment in the tactical domain induces high risks. DeepADMR monitors unexpected behavior of the DeepCQ+ policy based on the temporal di…
▽ More
We developed DeepADMR, a novel neural anomaly detector for the deep reinforcement learning (DRL)-based DeepCQ+ MANET routing policy. The performance of DRL-based algorithms such as DeepCQ+ is only verified within the trained and tested environments, hence their deployment in the tactical domain induces high risks. DeepADMR monitors unexpected behavior of the DeepCQ+ policy based on the temporal difference errors (TD-errors) in real-time and detects anomaly scenarios with empirical and non-parametric cumulative-sum statistics. The DeepCQ+ design via multi-agent weight-sharing proximal policy optimization (PPO) is slightly modified to enable the real-time estimation of the TD-errors. We report the DeepADMR performance in the presence of channel disruptions, high mobility levels, and network sizes beyond the training environments, which shows its effectiveness.
△ Less
Submitted 24 January, 2023;
originally announced February 2023.
-
Semi-Supervised 3D Hand Shape and Pose Estimation with Label Propagation
Authors:
Samira Kaviani,
Amir Rahimi,
Richard Hartley
Abstract:
To obtain 3D annotations, we are restricted to controlled environments or synthetic datasets, leading us to 3D datasets with less generalizability to real-world scenarios. To tackle this issue in the context of semi-supervised 3D hand shape and pose estimation, we propose the Pose Alignment network to propagate 3D annotations from labelled frames to nearby unlabelled frames in sparsely annotated v…
▽ More
To obtain 3D annotations, we are restricted to controlled environments or synthetic datasets, leading us to 3D datasets with less generalizability to real-world scenarios. To tackle this issue in the context of semi-supervised 3D hand shape and pose estimation, we propose the Pose Alignment network to propagate 3D annotations from labelled frames to nearby unlabelled frames in sparsely annotated videos. We show that incorporating the alignment supervision on pairs of labelled-unlabelled frames allows us to improve the pose estimation accuracy. Besides, we show that the proposed Pose Alignment network can effectively propagate annotations on unseen sparsely labelled videos without fine-tuning.
△ Less
Submitted 30 November, 2021;
originally announced November 2021.
-
DeepCQ+: Robust and Scalable Routing with Multi-Agent Deep Reinforcement Learning for Highly Dynamic Networks
Authors:
Saeed Kaviani,
Bo Ryu,
Ejaz Ahmed,
Kevin Larson,
Anh Le,
Alex Yahja,
Jae H. Kim
Abstract:
Highly dynamic mobile ad-hoc networks (MANETs) remain as one of the most challenging environments to develop and deploy robust, efficient, and scalable routing protocols. In this paper, we present DeepCQ+ routing protocol which, in a novel manner integrates emerging multi-agent deep reinforcement learning (MADRL) techniques into existing Q-learning-based routing protocols and their variants and ac…
▽ More
Highly dynamic mobile ad-hoc networks (MANETs) remain as one of the most challenging environments to develop and deploy robust, efficient, and scalable routing protocols. In this paper, we present DeepCQ+ routing protocol which, in a novel manner integrates emerging multi-agent deep reinforcement learning (MADRL) techniques into existing Q-learning-based routing protocols and their variants and achieves persistently higher performance across a wide range of topology and mobility configurations. While kee** the overall protocol structure of the Q-learning-based routing protocols, DeepCQ+ replaces statically configured parameterized thresholds and hand-written rules with carefully designed MADRL agents such that no configuration of such parameters is required a priori. Extensive simulation shows that DeepCQ+ yields significantly increased end-to-end throughput with lower overhead and no apparent degradation of end-to-end delays (hop counts) compared to its Q-learning based counterparts. Qualitatively, and perhaps more significantly, DeepCQ+ maintains remarkably similar performance gains under many scenarios that it was not trained for in terms of network sizes, mobility conditions, and traffic dynamics. To the best of our knowledge, this is the first successful application of the MADRL framework for the MANET routing problem that demonstrates a high degree of scalability and robustness even under environments that are outside the trained range of scenarios. This implies that our MARL-based DeepCQ+ design solution significantly improves the performance of Q-learning based CQ+ baseline approach for comparison and increases its practicality and explainability because the real-world MANET environment will likely vary outside the trained range of MANET scenarios. Additional techniques to further increase the gains in performance and scalability are discussed.
△ Less
Submitted 29 November, 2021;
originally announced November 2021.
-
Robust and Scalable Routing with Multi-Agent Deep Reinforcement Learning for MANETs
Authors:
Saeed Kaviani,
Bo Ryu,
Ejaz Ahmed,
Kevin A. Larson,
Anh Le,
Alex Yahja,
Jae H. Kim
Abstract:
Highly dynamic mobile ad-hoc networks (MANETs) are continuing to serve as one of the most challenging environments to develop and deploy robust, efficient, and scalable routing protocols. In this paper, we present DeepCQ+ routing which, in a novel manner, integrates emerging multi-agent deep reinforcement learning (MADRL) techniques into existing Q-learning-based routing protocols and their varian…
▽ More
Highly dynamic mobile ad-hoc networks (MANETs) are continuing to serve as one of the most challenging environments to develop and deploy robust, efficient, and scalable routing protocols. In this paper, we present DeepCQ+ routing which, in a novel manner, integrates emerging multi-agent deep reinforcement learning (MADRL) techniques into existing Q-learning-based routing protocols and their variants, and achieves persistently higher performance across a wide range of MANET configurations while training only on a limited range of network parameters and conditions. Quantitatively, DeepCQ+ shows consistently higher end-to-end throughput with lower overhead compared to its Q-learning-based counterparts with the overall gain of 10-15% in its efficiency. Qualitatively and more significantly, DeepCQ+ maintains remarkably similar performance gains under many scenarios that it was not trained for in terms of network sizes, mobility conditions, and traffic dynamics. To the best of our knowledge, this is the first successful demonstration of MADRL for the MANET routing problem that achieves and maintains a high degree of scalability and robustness even in the environments that are outside the trained range of scenarios. This implies that the proposed hybrid design approach of DeepCQ+ that combines MADRL and Q-learning significantly increases its practicality and explainability because the real-world MANET environment will likely vary outside the trained range of MANET scenarios.
△ Less
Submitted 28 March, 2021; v1 submitted 8 January, 2021;
originally announced January 2021.
-
Robust Joint Precoder and Equalizer Design in MIMO Communication Systems
Authors:
Saeed Kaviani,
Witold A. Krzymien
Abstract:
We address joint design of robust precoder and equalizer in a MIMO communication system using the minimization of weighted sum of mean square errors. In addition to imperfect knowledge of channel state information, we also account for inaccurate awareness of interference plus noise covariance matrix and power sha** matrix. We follow the worst-case model for imperfect knowledge of these matrices.…
▽ More
We address joint design of robust precoder and equalizer in a MIMO communication system using the minimization of weighted sum of mean square errors. In addition to imperfect knowledge of channel state information, we also account for inaccurate awareness of interference plus noise covariance matrix and power sha** matrix. We follow the worst-case model for imperfect knowledge of these matrices. First, we derive the worst-case values of these matrices. Then, we transform the joint precoder and equalizer optimization problem into a convex scalar optimization problem. Further, the solution to this problem will be simplified to a depressed quartic equation, the closed-form expressions for roots of which are known. Finally, we propose an iterative algorithm to obtain the worst-case robust transceivers.
△ Less
Submitted 16 April, 2013;
originally announced April 2013.
-
Optimal Multiuser Zero-Forcing with Per-Antenna Power Constraints for Network MIMO Coordination
Authors:
Saeed Kaviani,
Witold A. Krzymien
Abstract:
We consider a multi-cell multiple-input multiple-output (MIMO) coordinated downlink transmission, also known as network MIMO, under per-antenna power constraints. We investigate a simple multiuser zero-forcing (ZF) linear precoding technique known as block diagonalization (BD) for network MIMO. The optimal form of BD with per-antenna power constraints is proposed. It involves a novel approach of o…
▽ More
We consider a multi-cell multiple-input multiple-output (MIMO) coordinated downlink transmission, also known as network MIMO, under per-antenna power constraints. We investigate a simple multiuser zero-forcing (ZF) linear precoding technique known as block diagonalization (BD) for network MIMO. The optimal form of BD with per-antenna power constraints is proposed. It involves a novel approach of optimizing the precoding matrices over the entire null space of other users' transmissions.
An iterative gradient descent method is derived by solving the dual of the throughput maximization problem, which finds the optimal precoding matrices globally and efficiently. The comprehensive simulations illustrate several network MIMO coordination advantages when the optimal BD scheme is used. Its achievable throughput is compared with the capacity region obtained through the recently established duality concept under per-antenna power constraints.
△ Less
Submitted 16 April, 2013;
originally announced April 2013.
-
Linear Precoding and Equalization for Network MIMO with Partial Cooperation
Authors:
Saeed Kaviani,
Osvaldo Simeone,
Witold A Krzymień,
Shlomo Shamai
Abstract:
A cellular multiple-input multiple-output (MIMO) downlink system is studied in which each base station (BS) transmits to some of the users, so that each user receives its intended signal from a subset of the BSs. This scenario is referred to as network MIMO with partial cooperation, since only a subset of the BSs are able to coordinate their transmission towards any user. The focus of this paper i…
▽ More
A cellular multiple-input multiple-output (MIMO) downlink system is studied in which each base station (BS) transmits to some of the users, so that each user receives its intended signal from a subset of the BSs. This scenario is referred to as network MIMO with partial cooperation, since only a subset of the BSs are able to coordinate their transmission towards any user. The focus of this paper is on the optimization of linear beamforming strategies at the BSs and at the users for network MIMO with partial cooperation. Individual power constraints at the BSs are enforced, along with constraints on the number of streams per user. It is first shown that the system is equivalent to a MIMO interference channel with generalized linear constraints (MIMO-IFC-GC). The problems of maximizing the sum-rate(SR) and minimizing the weighted sum mean square error (WSMSE) of the data estimates are non-convex, and suboptimal solutions with reasonable complexity need to be devised. Based on this, suboptimal techniques that aim at maximizing the sum-rate for the MIMO-IFC-GC are reviewed from recent literature and extended to the MIMO-IFC-GC where necessary. Novel designs that aim at minimizing the WSMSE are then proposed. Extensive numerical simulations are provided to compare the performance of the considered schemes for realistic cellular systems.
△ Less
Submitted 8 February, 2013;
originally announced February 2013.
