Showing 1–4 of 4 results for author: Vo, P L

Reinforcement Learning -based Adaptation and Scheduling Methods for Multi-source DASH

Authors: Nghia T. Nguyen, Long Luu, Phuong L. Vo, Thi Thanh Sang Nguyen, Cuong T. Do, Ngoc-thanh Nguyen

Abstract: Dynamic adaptive streaming over HTTP (DASH) has been widely used in video streaming recently. In DASH, the client downloads video chunks in order from a server. The rate adaptation function at the video client enhances the user's quality-of-experience (QoE) by choosing a suitable quality level for each video chunk to download based on the network condition. Today networks such as content delivery… ▽ More Dynamic adaptive streaming over HTTP (DASH) has been widely used in video streaming recently. In DASH, the client downloads video chunks in order from a server. The rate adaptation function at the video client enhances the user's quality-of-experience (QoE) by choosing a suitable quality level for each video chunk to download based on the network condition. Today networks such as content delivery networks, edge caching networks, content-centric networks,... usually replicate video contents on multiple cache nodes. We study video streaming from multiple sources in this work. In multi-source streaming, video chunks may arrive out of order due to different conditions of the network paths. Hence, to guarantee a high QoE, the video client needs not only rate adaptation but also chunk scheduling. Reinforcement learning (RL) has emerged as the state-of-the-art control method in various fields in recent years. This paper proposes two algorithms for streaming from multiple sources: RL-based adaptation with greedy scheduling (RLAGS) and RL-based adaptation and scheduling (RLAS). We also build a simulation environment for training and evaluating. The efficiency of the proposed algorithms is proved via extensive simulations with real-trace data. △ Less

Submitted 25 July, 2023; originally announced August 2023.

Comments: 19 pages

MSC Class: 14J60 (Primary) 14F05; 14J26 (Secondary) 14J60 (Primary) 14F05; 14J26 (Secondary) ACM Class: C.2.4; I.2.11

Federated Deep Reinforcement Learning-based Bitrate Adaptation for Dynamic Adaptive Streaming over HTTP

Authors: Phuong L. Vo, Nghia T. Nguyen, Long Luu, Canh T. Dinh, Nguyen H. Tran, Tuan-Anh Le

Abstract: In video streaming over HTTP, the bitrate adaptation selects the quality of video chunks depending on the current network condition. Some previous works have applied deep reinforcement learning (DRL) algorithms to determine the chunk's bitrate from the observed states to maximize the quality-of-experience (QoE). However, to build an intelligent model that can predict in various environments, such… ▽ More In video streaming over HTTP, the bitrate adaptation selects the quality of video chunks depending on the current network condition. Some previous works have applied deep reinforcement learning (DRL) algorithms to determine the chunk's bitrate from the observed states to maximize the quality-of-experience (QoE). However, to build an intelligent model that can predict in various environments, such as 3G, 4G, Wifi, \textit{etc.}, the states observed from these environments must be sent to a server for training centrally. In this work, we integrate federated learning (FL) to DRL-based rate adaptation to train a model appropriate for different environments. The clients in the proposed framework train their model locally and only update the weights to the server. The simulations show that our federated DRL-based rate adaptations, called FDRLABR with different DRL algorithms, such as deep Q-learning, advantage actor-critic, and proximal policy optimization, yield better performance than the traditional bitrate adaptation methods in various environments. △ Less

Submitted 27 June, 2023; originally announced June 2023.

Comments: 13 pages, 1 column

The Multi-path Utility Maximization and Multi-path TCP Design

Authors: Phuong L. Vo, Anh T. Le, Choong S. Hong

Abstract: The network utility maximization problem (NUM) for multi-path is a problem which is non-strictly convex and non-separable. Using Jensen's inequality, we approximate the NUM to a strictly convex and separable problem which can be solved efficiently by the dual decomposition method. After a series of approximations, the result of the approximation problem converges to the globally optimal solution o… ▽ More The network utility maximization problem (NUM) for multi-path is a problem which is non-strictly convex and non-separable. Using Jensen's inequality, we approximate the NUM to a strictly convex and separable problem which can be solved efficiently by the dual decomposition method. After a series of approximations, the result of the approximation problem converges to the globally optimal solution of the original problem. Moreover, because of the separable and dual-based natures of the proposed algorithm, we utilize the reverse engineering frameworks of the current TCPs to develop a series of multi-path TCPs which are totally compatible with current TCPs. The multi-path users using our protocols can run simultaneously with the single-path users using the current TCPs. The simulations of our Multi-path Reno on ns-2 show the compatibility and the fairness among multi-path and single-path users. △ Less

Submitted 4 January, 2013; v1 submitted 19 August, 2011; originally announced August 2011.

Comments: This paper has been withdrawn by the author due to a mistake in the convergence proof

The Successive Approximation Approach for NUM Frameworks with Elastic and Inelastic Traffic

Authors: Phuong L. Vo, Nguyen H. Tran, Choong Seon Hong

Abstract: The concave utility in the Network Utility Maximization (NUM) problem is only suitable for elastic flows. However, the networks with the multiclass traffic, the utility of inelastic traffic is usually represented by the sigmoidal function which is a nonconcave function. Hence, the basic NUM problem becomes a nonconvex optimization problem. Solving the nonconvex NUM distributively is a difficult pr… ▽ More The concave utility in the Network Utility Maximization (NUM) problem is only suitable for elastic flows. However, the networks with the multiclass traffic, the utility of inelastic traffic is usually represented by the sigmoidal function which is a nonconcave function. Hence, the basic NUM problem becomes a nonconvex optimization problem. Solving the nonconvex NUM distributively is a difficult problem. The current works utilize the standard dual-based algorithm for the convex NUM and find the criteria for the global optimal convergence of the algorithm. It turns out that the link capacity must higher than a certain value to achieve the global optimum. We propose a new distributed algorithm that converges to the suboptimal solution of the nonconvex NUM for all of link capacity. We approximate the logarithm of the original problem to the convex problem which is solved efficiently by the standard dual-base distributed algorithm. After a sequence of approximations, the solutions converge to the KKT solution of the original problem. In many of our experiments, it also converges to the global optimal solution of the NUM. Moreover, we extend our work to solve the joint rate and power NUM problem with elastic and inelastic traffic in a wireless network. Our techniques can be applied to any log-concave utilities. △ Less

Submitted 15 April, 2012; v1 submitted 18 August, 2011; originally announced August 2011.

Comments: The authors would like to withdraw the paper because we must change the statements Theorem 1 and Theorem 2 more correctly. We also want to change the structure of the paper (the NUM problems) to be more readable