-
FinGPT-HPC: Efficient Pretraining and Finetuning Large Language Models for Financial Applications with High-Performance Computing
Authors:
Xiao-Yang Liu,
Jie Zhang,
Guoxuan Wang,
Weiqing Tong,
Anwar Walid
Abstract:
Large language models (LLMs) are computationally intensive. The computation workload and the memory footprint grow quadratically with the dimension (layer width). Most of LLMs' parameters come from the linear layers of the transformer structure and are highly redundant. These linear layers contribute more than 80% of the computation workload and 99% of the model size. To pretrain and finetune LLMs…
▽ More
Large language models (LLMs) are computationally intensive. The computation workload and the memory footprint grow quadratically with the dimension (layer width). Most of LLMs' parameters come from the linear layers of the transformer structure and are highly redundant. These linear layers contribute more than 80% of the computation workload and 99% of the model size. To pretrain and finetune LLMs efficiently, there are three major challenges to address: 1) reducing redundancy of the linear layers; 2) reducing GPU memory footprint; 3) improving GPU utilization when using distributed training. Prior methods, such as LoRA and QLoRA, utilized low-rank matrices and quantization to reduce the number of trainable parameters and model size, respectively. However, the resulting model still consumes a large amount of GPU memory. In this paper, we present high-performance GPU-based methods that exploit low-rank structures to pretrain and finetune LLMs for financial applications. We replace one conventional linear layer of the transformer structure with two narrower linear layers, which allows us to reduce the number of parameters by several orders of magnitude. By quantizing the parameters into low precision (8-bit and 4-bit), the memory consumption of the resulting model is further reduced. Compared with existing LLMs, our methods achieve a speedup of 1.3X and a model compression ratio of 2.64X for pretaining without accuracy drop. For finetuning, our methods achieve an average accuracy increase of 6.3% and 24.0% in general tasks and financial tasks, respectively, and GPU memory consumption ratio of 6.3X. The sizes of our models are smaller than 0.59 GB, allowing inference on a smartphone.
△ Less
Submitted 21 February, 2024;
originally announced February 2024.
-
Deep Reinforcement Learning for Traffic Light Control in Intelligent Transportation Systems
Authors:
Xiao-Yang Liu,
Ming Zhu,
Sem Borst,
Anwar Walid
Abstract:
Smart traffic lights in intelligent transportation systems (ITSs) are envisioned to greatly increase traffic efficiency and reduce congestion. Deep reinforcement learning (DRL) is a promising approach to adaptively control traffic lights based on the real-time traffic situation in a road network. However, conventional methods may suffer from poor scalability. In this paper, we investigate deep rei…
▽ More
Smart traffic lights in intelligent transportation systems (ITSs) are envisioned to greatly increase traffic efficiency and reduce congestion. Deep reinforcement learning (DRL) is a promising approach to adaptively control traffic lights based on the real-time traffic situation in a road network. However, conventional methods may suffer from poor scalability. In this paper, we investigate deep reinforcement learning to control traffic lights, and both theoretical analysis and numerical experiments show that the intelligent behavior ``greenwave" (i.e., a vehicle will see a progressive cascade of green lights, and not have to brake at any intersection) emerges naturally a grid road network, which is proved to be the optimal policy in an avenue with multiple cross streets. As a first step, we use two DRL algorithms for the traffic light control problems in two scenarios. In a single road intersection, we verify that the deep Q-network (DQN) algorithm delivers a thresholding policy; and in a grid road network, we adopt the deep deterministic policy gradient (DDPG) algorithm. Secondly, numerical experiments show that the DQN algorithm delivers the optimal control, and the DDPG algorithm with passive observations has the capability to produce on its own a high-level intelligent behavior in a grid road network, namely, the ``greenwave" policy emerges. We also verify the ``greenwave" patterns in a $5 \times 10$ grid road network. Thirdly, the ``greenwave" patterns demonstrate that DRL algorithms produce favorable solutions since the ``greenwave" policy shown in experiment results is proved to be optimal in a specified traffic model (an avenue with multiple cross streets). The delivered policies both in a single road intersection and a grid road network demonstrate the scalability of DRL algorithms.
△ Less
Submitted 5 March, 2023; v1 submitted 3 February, 2023;
originally announced February 2023.
-
Fair and Efficient Distributed Edge Learning with Hybrid Multipath TCP
Authors:
Shiva Raj Pokhrel,
**ho Choi,
Anwar Walid
Abstract:
The bottleneck of distributed edge learning (DEL) over wireless has shifted from computing to communication, primarily the aggregation-averaging (Agg-Avg) process of DEL. The existing transmission control protocol (TCP)-based data networking schemes for DEL are application-agnostic and fail to deliver adjustments according to application layer requirements. As a result, they introduce massive exce…
▽ More
The bottleneck of distributed edge learning (DEL) over wireless has shifted from computing to communication, primarily the aggregation-averaging (Agg-Avg) process of DEL. The existing transmission control protocol (TCP)-based data networking schemes for DEL are application-agnostic and fail to deliver adjustments according to application layer requirements. As a result, they introduce massive excess time and undesired issues such as unfairness and stragglers. Other prior mitigation solutions have significant limitations as they balance data flow rates from workers across paths but often incur imbalanced backlogs when the paths exhibit variance, causing stragglers. To facilitate a more productive DEL, we develop a hybrid multipath TCP (MPTCP) by combining model-based and deep reinforcement learning (DRL) based MPTCP for DEL that strives to realize quicker iteration of DEL and better fairness (by ameliorating stragglers). Hybrid MPTCP essentially integrates two radical TCP developments: i) successful existing model-based MPTCP control strategies and ii) advanced emerging DRL-based techniques, and introduces a novel hybrid MPTCP data transport for easing the communication of the Agg-Avg process. Extensive emulation results demonstrate that the proposed hybrid MPTCP can overcome excess time consumption and ameliorate the application layer unfairness of DEL effectively without injecting additional inconstancy and stragglers.
△ Less
Submitted 3 November, 2022;
originally announced November 2022.
-
Regenerative Particle Thompson Sampling
Authors:
Zeyu Zhou,
Bruce Hajek,
Nakjung Choi,
Anwar Walid
Abstract:
This paper proposes regenerative particle Thompson sampling (RPTS), a flexible variation of Thompson sampling. Thompson sampling itself is a Bayesian heuristic for solving stochastic bandit problems, but it is hard to implement in practice due to the intractability of maintaining a continuous posterior distribution. Particle Thompson sampling (PTS) is an approximation of Thompson sampling obtained…
▽ More
This paper proposes regenerative particle Thompson sampling (RPTS), a flexible variation of Thompson sampling. Thompson sampling itself is a Bayesian heuristic for solving stochastic bandit problems, but it is hard to implement in practice due to the intractability of maintaining a continuous posterior distribution. Particle Thompson sampling (PTS) is an approximation of Thompson sampling obtained by simply replacing the continuous distribution by a discrete distribution supported at a set of weighted static particles. We observe that in PTS, the weights of all but a few fit particles converge to zero. RPTS is based on the heuristic: delete the decaying unfit particles and regenerate new particles in the vicinity of fit surviving particles. Empirical evidence shows uniform improvement from PTS to RPTS and flexibility and efficacy of RPTS across a set of representative bandit problems, including an application to 5G network slicing.
△ Less
Submitted 22 January, 2024; v1 submitted 15 March, 2022;
originally announced March 2022.
-
ElegantRL-Podracer: Scalable and Elastic Library for Cloud-Native Deep Reinforcement Learning
Authors:
Xiao-Yang Liu,
Zechu Li,
Zhuoran Yang,
Jiahao Zheng,
Zhaoran Wang,
Anwar Walid,
Jian Guo,
Michael I. Jordan
Abstract:
Deep reinforcement learning (DRL) has revolutionized learning and actuation in applications such as game playing and robotic control. The cost of data collection, i.e., generating transitions from agent-environment interactions, remains a major challenge for wider DRL adoption in complex real-world problems. Following a cloud-native paradigm to train DRL agents on a GPU cloud platform is a promisi…
▽ More
Deep reinforcement learning (DRL) has revolutionized learning and actuation in applications such as game playing and robotic control. The cost of data collection, i.e., generating transitions from agent-environment interactions, remains a major challenge for wider DRL adoption in complex real-world problems. Following a cloud-native paradigm to train DRL agents on a GPU cloud platform is a promising solution. In this paper, we present a scalable and elastic library ElegantRL-podracer for cloud-native deep reinforcement learning, which efficiently supports millions of GPU cores to carry out massively parallel training at multiple levels. At a high-level, ElegantRL-podracer employs a tournament-based ensemble scheme to orchestrate the training process on hundreds or even thousands of GPUs, scheduling the interactions between a leaderboard and a training pool with hundreds of pods. At a low-level, each pod simulates agent-environment interactions in parallel by fully utilizing nearly 7,000 GPU CUDA cores in a single GPU. Our ElegantRL-podracer library features high scalability, elasticity and accessibility by following the development principles of containerization, microservices and MLOps. Using an NVIDIA DGX SuperPOD cloud, we conduct extensive experiments on various tasks in locomotion and stock trading and show that ElegantRL-podracer substantially outperforms RLlib. Our codes are available on GitHub.
△ Less
Submitted 3 April, 2022; v1 submitted 11 December, 2021;
originally announced December 2021.
-
FinRL-Podracer: High Performance and Scalable Deep Reinforcement Learning for Quantitative Finance
Authors:
Zechu Li,
Xiao-Yang Liu,
Jiahao Zheng,
Zhaoran Wang,
Anwar Walid,
Jian Guo
Abstract:
Machine learning techniques are playing more and more important roles in finance market investment. However, finance quantitative modeling with conventional supervised learning approaches has a number of limitations. The development of deep reinforcement learning techniques is partially addressing these issues. Unfortunately, the steep learning curve and the difficulty in quick modeling and agile…
▽ More
Machine learning techniques are playing more and more important roles in finance market investment. However, finance quantitative modeling with conventional supervised learning approaches has a number of limitations. The development of deep reinforcement learning techniques is partially addressing these issues. Unfortunately, the steep learning curve and the difficulty in quick modeling and agile development are impeding finance researchers from using deep reinforcement learning in quantitative trading. In this paper, we propose an RLOps in finance paradigm and present a FinRL-Podracer framework to accelerate the development pipeline of deep reinforcement learning (DRL)-driven trading strategy and to improve both trading performance and training efficiency. FinRL-Podracer is a cloud solution that features high performance and high scalability and promises continuous training, continuous integration, and continuous delivery of DRL-driven trading strategies, facilitating a rapid transformation from algorithmic innovations into a profitable trading strategy. First, we propose a generational evolution mechanism with an ensemble strategy to improve the trading performance of a DRL agent, and schedule the training of a DRL algorithm onto a GPU cloud via multi-level map**. Then, we carry out the training of DRL components with high-performance optimizations on GPUs. Finally, we evaluate the FinRL-Podracer framework for a stock trend prediction task on an NVIDIA DGX SuperPOD cloud. FinRL-Podracer outperforms three popular DRL libraries Ray RLlib, Stable Baseline 3 and FinRL, i.e., 12% \sim 35% improvements in annual return, 0.1 \sim 0.6 improvements in Sharpe ratio and 3 times \sim 7 times speed-up in training time. We show the high scalability by training a trading agent in 10 minutes with $80$ A100 GPUs, on NASDAQ-100 constituent stocks with minute-level data over 10 years.
△ Less
Submitted 6 November, 2021;
originally announced November 2021.
-
Learning to Harness Bandwidth with Multipath Congestion Control and Scheduling
Authors:
Shiva Raj Pokhrel,
Anwar Walid
Abstract:
Multipath TCP (MPTCP) has emerged as a facilitator for harnessing and pooling available bandwidth in wireless/wireline communication networks and in data centers. Existing implementations of MPTCP such as, Linked Increase Algorithm (LIA), Opportunistic LIA (OLIA) and BAlanced LInked Adaptation (BALIA) include separate algorithms for congestion control and packet scheduling, with pre-selected contr…
▽ More
Multipath TCP (MPTCP) has emerged as a facilitator for harnessing and pooling available bandwidth in wireless/wireline communication networks and in data centers. Existing implementations of MPTCP such as, Linked Increase Algorithm (LIA), Opportunistic LIA (OLIA) and BAlanced LInked Adaptation (BALIA) include separate algorithms for congestion control and packet scheduling, with pre-selected control parameters. We propose a Deep Q-Learning (DQL) based framework for joint congestion control and packet scheduling for MPTCP. At the heart of the solution is an intelligent agent for interface, learning and actuation, which learns from experience optimal congestion control and scheduling mechanism using DQL techniques with policy gradients. We provide a rigorous stability analysis of system dynamics which provides important practical design insights. In addition, the proposed DQL-MPTCP algorithm utilizes the `recurrent neural network' and integrates it with `long short-term memory' for continuously i) learning dynamic behavior of subflows (paths) and ii) responding promptly to their behavior using prioritized experience replay. With extensive emulations, we show that the proposed DQL-based MPTCP algorithm outperforms MPTCP LIA, OLIA and BALIA algorithms. Moreover, the DQL-MPTCP algorithm is robust to time-varying network characteristics, and provides dynamic exploration and exploitation of paths.
△ Less
Submitted 29 May, 2021;
originally announced May 2021.
-
Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning
Authors:
Shaoxiong Ji,
Yue Tan,
Teemu Saravirta,
Zhiqin Yang,
Yixin Liu,
Lauri Vasankari,
Shirui Pan,
Guodong Long,
Anwar Walid
Abstract:
Federated learning is a new learning paradigm that decouples data collection and model training via multi-party computation and model aggregation. As a flexible learning setting, federated learning has the potential to integrate with other learning frameworks. We conduct a focused survey of federated learning in conjunction with other learning algorithms. Specifically, we explore various learning…
▽ More
Federated learning is a new learning paradigm that decouples data collection and model training via multi-party computation and model aggregation. As a flexible learning setting, federated learning has the potential to integrate with other learning frameworks. We conduct a focused survey of federated learning in conjunction with other learning algorithms. Specifically, we explore various learning algorithms to improve the vanilla federated averaging algorithm and review model fusion methods such as adaptive aggregation, regularization, clustered methods, and Bayesian methods. Following the emerging trends, we also discuss federated learning in the intersection with other learning paradigms, termed federated X learning, where X includes multitask learning, meta-learning, transfer learning, unsupervised learning, and reinforcement learning. In addition to reviewing state-of-the-art studies, this paper also identifies key challenges and applications in this field, while also highlighting promising future directions.
△ Less
Submitted 27 March, 2024; v1 submitted 25 February, 2021;
originally announced February 2021.
-
Privacy-preserving Decentralized Aggregation for Federated Learning
Authors:
Beomyeol Jeon,
S. M. Ferdous,
Muntasir Raihan Rahman,
Anwar Walid
Abstract:
Federated learning is a promising framework for learning over decentralized data spanning multiple regions. This approach avoids expensive central training data aggregation cost and can improve privacy because distributed sites do not have to reveal privacy-sensitive data. In this paper, we develop a privacy-preserving decentralized aggregation protocol for federated learning. We formulate the dis…
▽ More
Federated learning is a promising framework for learning over decentralized data spanning multiple regions. This approach avoids expensive central training data aggregation cost and can improve privacy because distributed sites do not have to reveal privacy-sensitive data. In this paper, we develop a privacy-preserving decentralized aggregation protocol for federated learning. We formulate the distributed aggregation protocol with the Alternating Direction Method of Multiplier (ADMM) and examine its privacy weakness. Unlike prior work that use Differential Privacy or homomorphic encryption for privacy, we develop a protocol that controls communication among participants in each round of aggregation to minimize privacy leakage. We establish its privacy guarantee against an honest-but-curious adversary. We also propose an efficient algorithm to construct such a communication pattern, inspired by combinatorial block design theory. Our secure aggregation protocol based on this novel group communication pattern design leads to an efficient algorithm for federated training with privacy guarantees. We evaluate our federated training algorithm on image classification and next-word prediction applications over benchmark datasets with 9 and 15 distributed sites. Evaluation results show that our algorithm performs comparably to the standard centralized federated learning method while preserving privacy; the degradation in test accuracy is only up to 0.73%.
△ Less
Submitted 28 December, 2020; v1 submitted 13 December, 2020;
originally announced December 2020.
-
Dynamic Sampling and Selective Masking for Communication-Efficient Federated Learning
Authors:
Shaoxiong Ji,
Wenqi Jiang,
Anwar Walid,
Xue Li
Abstract:
Federated learning (FL) is a novel machine learning setting that enables on-device intelligence via decentralized training and federated optimization. Deep neural networks' rapid development facilitates the learning techniques for modeling complex problems and emerges into federated deep learning under the federated setting. However, the tremendous amount of model parameters burdens the communicat…
▽ More
Federated learning (FL) is a novel machine learning setting that enables on-device intelligence via decentralized training and federated optimization. Deep neural networks' rapid development facilitates the learning techniques for modeling complex problems and emerges into federated deep learning under the federated setting. However, the tremendous amount of model parameters burdens the communication network with a high load of transportation. This paper introduces two approaches for improving communication efficiency by dynamic sampling and top-$k$ selective masking. The former controls the fraction of selected client models dynamically, while the latter selects parameters with top-$k$ largest values of difference for federated updating. Experiments on convolutional image classification and recurrent language modeling are conducted on three public datasets to show our proposed methods' effectiveness.
△ Less
Submitted 20 September, 2021; v1 submitted 21 March, 2020;
originally announced March 2020.
-
Stability and Scalability of Blockchain Systems
Authors:
Aditya Gopalan,
Abishek Sankararaman,
Anwar Walid,
Sriram Vishwanath
Abstract:
The blockchain paradigm provides a mechanism for content dissemination and distributed consensus on Peer-to-Peer (P2P) networks. While this paradigm has been widely adopted in industry, it has not been carefully analyzed in terms of its network scaling with respect to the number of peers. Applications for blockchain systems, such as cryptocurrencies and IoT, require this form of network scaling.…
▽ More
The blockchain paradigm provides a mechanism for content dissemination and distributed consensus on Peer-to-Peer (P2P) networks. While this paradigm has been widely adopted in industry, it has not been carefully analyzed in terms of its network scaling with respect to the number of peers. Applications for blockchain systems, such as cryptocurrencies and IoT, require this form of network scaling.
In this paper, we propose a new stochastic network model for a blockchain system. We identify a structural property called \emph{one-endedness}, which we show to be desirable in any blockchain system as it is directly related to distributed consensus among the peers. We show that the stochastic stability of the network is sufficient for the one-endedness of a blockchain. We further establish that our model belongs to a class of network models, called monotone separable models. This allows us to establish upper and lower bounds on the stability region. The bounds on stability depend on the connectivity of the P2P network through its conductance and allow us to analyze the scalability of blockchain systems on large P2P networks. We verify our theoretical insights using both synthetic data and real data from the Bitcoin network.
△ Less
Submitted 18 December, 2020; v1 submitted 6 February, 2020;
originally announced February 2020.
-
Deep Reinforcement Learning for Network Slicing with Heterogeneous Resource Requirements and Time Varying Traffic Dynamics
Authors:
Jaehoon Koo,
Veena B. Mendiratta,
Muntasir Raihan Rahman,
Anwar Walid
Abstract:
Efficient network slicing is vital to deal with the highly variable and dynamic characteristics of network traffic generated by a varied range of applications. The problem is made more challenging with the advent of new technologies such as 5G and new architectures such as SDN and NFV. Network slicing addresses a challenging dynamic network resource allocation problem where a single network infras…
▽ More
Efficient network slicing is vital to deal with the highly variable and dynamic characteristics of network traffic generated by a varied range of applications. The problem is made more challenging with the advent of new technologies such as 5G and new architectures such as SDN and NFV. Network slicing addresses a challenging dynamic network resource allocation problem where a single network infrastructure is divided into (virtual) multiple slices to meet the demands of different users with varying requirements, the main challenges being --- the traffic arrival characteristics and the job resource requirements (e.g., compute, memory and bandwidth resources) for each slice can be highly dynamic. Traditional model-based optimization or queueing theoretic modeling becomes intractable with the high reliability, and stringent bandwidth and latency requirements imposed by 5G technologies. In addition these approaches lack adaptivity in dynamic environments. We propose a deep reinforcement learning approach to address this dynamic coupled resource allocation problem. Model evaluation using both synthetic simulation data and real workload driven traces demonstrates that our deep reinforcement learning solution improves overall resource utilization, latency performance, and demands satisfied as compared to a baseline equal-slicing strategy.
△ Less
Submitted 8 August, 2019;
originally announced August 2019.
-
Deep Reinforcement Learning for Unmanned Aerial Vehicle-Assisted Vehicular Networks
Authors:
Ming Zhu,
Xiao-Yang Liu,
Anwar Walid
Abstract:
Unmanned aerial vehicles (UAVs) are envisioned to complement the 5G communication infrastructure in future smart cities. Hot spots easily appear in road intersections, where effective communication among vehicles is challenging. UAVs may serve as relays with the advantages of low price, easy deployment, line-of-sight links, and flexible mobility. In this paper, we study a UAV-assisted vehicular ne…
▽ More
Unmanned aerial vehicles (UAVs) are envisioned to complement the 5G communication infrastructure in future smart cities. Hot spots easily appear in road intersections, where effective communication among vehicles is challenging. UAVs may serve as relays with the advantages of low price, easy deployment, line-of-sight links, and flexible mobility. In this paper, we study a UAV-assisted vehicular network where the UAV jointly adjusts its transmission control (power and channel) and 3D flight to maximize the total throughput. First, we formulate a Markov decision process (MDP) problem by modeling the mobility of the UAV/vehicles and the state transitions. Secondly, we solve the target problem using a deep reinforcement learning method, namely, the deep deterministic policy gradient (DDPG), and propose three solutions with different control objectives. Deep reinforcement learning methods obtain the optimal policy through the interactions with the environment without knowing the environment variables. Considering that environment variables in our problem are unknown and unmeasurable, we choose a deep reinforcement learning method to solve it. Moreover, considering the energy consumption of 3D flight, we extend the proposed solutions to maximize the total throughput per unit energy. To encourage or discourage the UAV's mobility according to its prediction, the DDPG framework is modified, where the UAV adjusts its learning rate automatically. Thirdly, in a simplified model with small state space and action space, we verify the optimality of proposed algorithms. Comparing with two baseline schemes, we demonstrate the effectiveness of proposed algorithms in a realistic model.
△ Less
Submitted 14 February, 2023; v1 submitted 12 June, 2019;
originally announced June 2019.
-
Deep Reinforcement Learning for Intelligent Transportation Systems
Authors:
Xiao-Yang Liu,
Zihan Ding,
Sem Borst,
Anwar Walid
Abstract:
Intelligent Transportation Systems (ITSs) are envisioned to play a critical role in improving traffic flow and reducing congestion, which is a pervasive issue impacting urban areas around the globe. Rapidly advancing vehicular communication and edge cloud computation technologies provide key enablers for smart traffic management. However, operating viable real-time actuation mechanisms on a practi…
▽ More
Intelligent Transportation Systems (ITSs) are envisioned to play a critical role in improving traffic flow and reducing congestion, which is a pervasive issue impacting urban areas around the globe. Rapidly advancing vehicular communication and edge cloud computation technologies provide key enablers for smart traffic management. However, operating viable real-time actuation mechanisms on a practically relevant scale involves formidable challenges, e.g., policy iteration and conventional Reinforcement Learning (RL) techniques suffer from poor scalability due to state space explosion. Motivated by these issues, we explore the potential for Deep Q-Networks (DQN) to optimize traffic light control policies. As an initial benchmark, we establish that the DQN algorithms yield the "thresholding" policy in a single-intersection. Next, we examine the scalability properties of DQN algorithms and their performance in a linear network topology with several intersections along a main artery. We demonstrate that DQN algorithms produce intelligent behavior, such as the emergence of "greenwave" patterns, reflecting their ability to learn favorable traffic light actuations.
△ Less
Submitted 3 December, 2018;
originally announced December 2018.
-
Practical Deep Reinforcement Learning Approach for Stock Trading
Authors:
Xiao-Yang Liu,
Zhuoran Xiong,
Shan Zhong,
Hongyang Yang,
Anwar Walid
Abstract:
Stock trading strategy plays a crucial role in investment companies. However, it is challenging to obtain optimal strategy in the complex and dynamic stock market. We explore the potential of deep reinforcement learning to optimize stock trading strategy and thus maximize investment return. 30 stocks are selected as our trading stocks and their daily prices are used as the training and trading mar…
▽ More
Stock trading strategy plays a crucial role in investment companies. However, it is challenging to obtain optimal strategy in the complex and dynamic stock market. We explore the potential of deep reinforcement learning to optimize stock trading strategy and thus maximize investment return. 30 stocks are selected as our trading stocks and their daily prices are used as the training and trading market environment. We train a deep reinforcement learning agent and obtain an adaptive trading strategy. The agent's performance is evaluated and compared with Dow Jones Industrial Average and the traditional min-variance portfolio allocation strategy. The proposed deep reinforcement learning approach is shown to outperform the two baselines in terms of both the Sharpe ratio and cumulative returns.
△ Less
Submitted 30 July, 2022; v1 submitted 19 November, 2018;
originally announced November 2018.
-
Transform-Based Multilinear Dynamical System for Tensor Time Series Analysis
Authors:
Weijun Lu,
Xiao-Yang Liu,
Qingwei Wu,
Yue Sun,
Anwar Walid
Abstract:
We propose a novel multilinear dynamical system (MLDS) in a transform domain, named $\mathcal{L}$-MLDS, to model tensor time series. With transformations applied to a tensor data, the latent multidimensional correlations among the frontal slices are built, and thus resulting in the computational independence in the transform domain. This allows the exact separability of the multi-dimensional probl…
▽ More
We propose a novel multilinear dynamical system (MLDS) in a transform domain, named $\mathcal{L}$-MLDS, to model tensor time series. With transformations applied to a tensor data, the latent multidimensional correlations among the frontal slices are built, and thus resulting in the computational independence in the transform domain. This allows the exact separability of the multi-dimensional problem into multiple smaller LDS problems. To estimate the system parameters, we utilize the expectation-maximization (EM) algorithm to determine the parameters of each LDS. Further, $\mathcal{L}$-MLDSs significantly reduce the model parameters and allows parallel processing. Our general $\mathcal{L}$-MLDS model is implemented based on different transforms: discrete Fourier transform, discrete cosine transform and discrete wavelet transform. Due to the nonlinearity of these transformations, $\mathcal{L}$-MLDS is able to capture the nonlinear correlations within the data unlike the MLDS \cite{rogers2013multilinear} which assumes multi-way linear correlations. Using four real datasets, the proposed $\mathcal{L}$-MLDS is shown to achieve much higher prediction accuracy than the state-of-the-art MLDS and LDS with an equal number of parameters under different noise models. In particular, the relative errors are reduced by $50\% \sim 99\%$. Simultaneously, $\mathcal{L}$-MLDS achieves an exponential improvement in the model's training time than MLDS.
△ Less
Submitted 18 November, 2018;
originally announced November 2018.
-
Multidimensional Data Tensor Sensing for RF Tomographic Imaging
Authors:
Tao Deng,
Xiao-Yang Liu,
Feng Qian,
Anwar Walid
Abstract:
Radio-frequency (RF) tomographic imaging is a promising technique for inferring multi-dimensional physical space by processing RF signals traversed across a region of interest. However, conventional RF tomography schemes are generally based on vector compressed sensing, which ignores the geometric structures of the target spaces and leads to low recovery precision. The recently proposed transform-…
▽ More
Radio-frequency (RF) tomographic imaging is a promising technique for inferring multi-dimensional physical space by processing RF signals traversed across a region of interest. However, conventional RF tomography schemes are generally based on vector compressed sensing, which ignores the geometric structures of the target spaces and leads to low recovery precision. The recently proposed transform-based tensor model is more appropriate for sensory data processing, as it helps exploit the geometric structures of the three-dimensional target and improve the recovery precision. In this paper, we propose a novel tensor sensing approach that achieves highly accurate estimation for real-world three-dimensional spaces. First, we use the transform-based tensor model to formulate a tensor sensing problem, and propose a fast alternating minimization algorithm called Alt-Min. Secondly, we drive an algorithm which is optimized to reduce memory and computation requirements. Finally, we present evaluation of our Alt-Min approach using IKEA 3D data and demonstrate significant improvement in recovery error and convergence speed compared to prior tensor-based compressed sensing.
△ Less
Submitted 16 December, 2017; v1 submitted 13 December, 2017;
originally announced December 2017.
-
Scalable Routing in SDN-enabled Networks with Consolidated Middleboxes
Authors:
Andrey Gushchin,
Anwar Walid,
Ao Tang
Abstract:
Middleboxes are special network devices that perform various functions such as enabling security and efficiency. SDN-based routing approaches in networks with middleboxes need to address resource constraints, such as memory in the switches and processing power of middleboxes, and traversal constraint where a flow must visit the required middleboxes in a specific order. In this work we propose a so…
▽ More
Middleboxes are special network devices that perform various functions such as enabling security and efficiency. SDN-based routing approaches in networks with middleboxes need to address resource constraints, such as memory in the switches and processing power of middleboxes, and traversal constraint where a flow must visit the required middleboxes in a specific order. In this work we propose a solution based on MultiPoint-To-Point Trees (MPTPT) for routing traffic in SDN-enabled networks with consolidated middleboxes. We show both theoretically and via simulations that our solution significantly reduces the number of routing rules in the switches, while guaranteeing optimum throughput and meeting processing requirements. Additionally, the underlying algorithm has low complexity making it suitable in dynamic network environment.
△ Less
Submitted 7 April, 2015;
originally announced April 2015.
-
Multipath TCP: Analysis, Design and Implementation
Authors:
Qiuyu Peng,
Anwar Walid,
Jaehyun Hwang,
Steven H. Low
Abstract:
Multi-path TCP (MP-TCP) has the potential to greatly improve application performance by using multiple paths transparently. We propose a fluid model for a large class of MP-TCP algorithms and identify design criteria that guarantee the existence, uniqueness, and stability of system equilibrium. We clarify how algorithm parameters impact TCP-friendliness, responsiveness, and window oscillation and…
▽ More
Multi-path TCP (MP-TCP) has the potential to greatly improve application performance by using multiple paths transparently. We propose a fluid model for a large class of MP-TCP algorithms and identify design criteria that guarantee the existence, uniqueness, and stability of system equilibrium. We clarify how algorithm parameters impact TCP-friendliness, responsiveness, and window oscillation and demonstrate an inevitable tradeoff among these properties. We discuss the implications of these properties on the behavior of existing algorithms and motivate a new design that generalizes existing algorithms and strikes a good balance among TCP-friendliness, responsiveness, and window oscillation. We have implemented our algorithm in the Linux kernel. We use our prototype to compare the new algorithm with existing MP-TCP algorithms.
△ Less
Submitted 7 December, 2014; v1 submitted 14 August, 2013;
originally announced August 2013.
-
Protection Over Asymmetric Channels, S-MATE: Secure Multipath Adaptive Traffic Engineering
Authors:
Salah A. Aly,
Nirwan Ansari,
H. Vincent Poor,
Anwar I. Walid
Abstract:
Several approaches have been proposed to the problem of provisioning traffic engineering between core network nodes in Internet Service Provider (ISP) networks. Such approaches aim to minimize network delay, increase capacity, and enhance security services between two core (relay) network nodes, an ingress node and an egress node. MATE (Multipath Adaptive Traffic Engineering) has been proposed for…
▽ More
Several approaches have been proposed to the problem of provisioning traffic engineering between core network nodes in Internet Service Provider (ISP) networks. Such approaches aim to minimize network delay, increase capacity, and enhance security services between two core (relay) network nodes, an ingress node and an egress node. MATE (Multipath Adaptive Traffic Engineering) has been proposed for multipath adaptive traffic engineering between an ingress node (source) and an egress node (destination) to distribute the network flow among multiple disjoint paths. Its novel idea is to avoid network congestion and attacks that might exist in edge and node disjoint paths between two core network nodes.
This paper proposes protection schemes over asymmetric channels. Precisely, the paper aims to develop an adaptive, robust, and reliable traffic engineering scheme to improve performance and reliability of communication networks. This scheme will also provision Quality of Server (QoS) and protection of traffic engineering to maximize network efficiency. Specifically, S-MATE (secure MATE) is proposed to protect the network traffic between two core nodes (routers, switches, etc.) in a cloud network. S-MATE secures against a single link attack/failure by adding redundancy in one of the operational redundant paths between the sender and receiver nodes. It is also extended to secure against multiple attacked links. The proposed scheme can be applied to secure core networks such as optical and IP networks.
△ Less
Submitted 29 December, 2010;
originally announced December 2010.
-
S-MATE: Secure Coding-based Multipath Adaptive Traffic Engineering
Authors:
Salah A. Aly,
Nirwan Ansari,
Anwar I. Walid,
H. Vincent Poor
Abstract:
There have been several approaches to provisioning traffic between core network nodes in Internet Service Provider (ISP) networks. Such approaches aim to minimize network delay, increase network capacity, and enhance network security services. MATE (Multipath Adaptive Traffic Engineering) protocol has been proposed for multipath adaptive traffic engineering between an ingress node (source) and an…
▽ More
There have been several approaches to provisioning traffic between core network nodes in Internet Service Provider (ISP) networks. Such approaches aim to minimize network delay, increase network capacity, and enhance network security services. MATE (Multipath Adaptive Traffic Engineering) protocol has been proposed for multipath adaptive traffic engineering between an ingress node (source) and an egress node (destination). Its novel idea is to avoid network congestion and attacks that might exist in edge and node disjoint paths between two core network nodes.
This paper builds an adaptive, robust, and reliable traffic engineering scheme for better performance of communication network operations. This will also provision quality of service (QoS) and protection of traffic engineering to maximize network efficiency. Specifically, we present a new approach, S-MATE (secure MATE) is developed to protect the network traffic between two core nodes (routers or switches) in a cloud network. S-MATE secures against a single link attack/failure by adding redundancy in one of the operational paths between the sender and receiver. The proposed scheme can be built to secure core networks such as optical and IP networks.
△ Less
Submitted 23 October, 2010;
originally announced October 2010.
-
Network Protection Design Using Network Coding
Authors:
Salah A. Aly,
Ahmed E. Kamal,
Anwar I. Walid
Abstract:
Link and node failures are two common fundamental problems that affect operational networks. Protection of communication networks against such failures is essential for maintaining network reliability and performance. Network protection codes (NPC) are proposed to protect operational networks against link and node failures. Furthermore, encoding and decoding operations of such codes are well devel…
▽ More
Link and node failures are two common fundamental problems that affect operational networks. Protection of communication networks against such failures is essential for maintaining network reliability and performance. Network protection codes (NPC) are proposed to protect operational networks against link and node failures. Furthermore, encoding and decoding operations of such codes are well developed over binary and finite fields. Finding network topologies, practical scenarios, and limits on graphs applicable for NPC are of interest. In this paper, we establish limits on network protection design. We investigate several network graphs where NPC can be deployed using network coding. Furthermore, we construct graphs with minimum number of edges suitable for network protection codes deployment.
△ Less
Submitted 25 August, 2010;
originally announced August 2010.
