Showing 1–22 of 22 results for author: Tsang, D H K

Bayesian Federated Model Compression for Communication and Computation Efficiency

Authors: Chengyu Xia, Danny H. K. Tsang, Vincent K. N. Lau

Abstract: In this paper, we investigate Bayesian model compression in federated learning (FL) to construct sparse models that can achieve both communication and computation efficiencies. We propose a decentralized Turbo variational Bayesian inference (D-Turbo-VBI) FL framework where we firstly propose a hierarchical sparse prior to promote a clustered sparse structure in the weight matrix. Then, by carefull… ▽ More In this paper, we investigate Bayesian model compression in federated learning (FL) to construct sparse models that can achieve both communication and computation efficiencies. We propose a decentralized Turbo variational Bayesian inference (D-Turbo-VBI) FL framework where we firstly propose a hierarchical sparse prior to promote a clustered sparse structure in the weight matrix. Then, by carefully integrating message passing and VBI with a decentralized turbo framework, we propose the D-Turbo-VBI algorithm which can (i) reduce both upstream and downstream communication overhead during federated training, and (ii) reduce the computational complexity during local inference. Additionally, we establish the convergence property for thr proposed D-Turbo-VBI algorithm. Simulation results show the significant gain of our proposed algorithm over the baselines in reducing communication overhead during federated training and computational complexity of final model. △ Less

Submitted 11 April, 2024; originally announced April 2024.

Federated Prompt-based Decision Transformer for Customized VR Services in Mobile Edge Computing System

Authors: Tailin Zhou, Jiadong Yu, Jun Zhang, Danny H. K. Tsang

Abstract: This paper investigates resource allocation to provide heterogeneous users with customized virtual reality (VR) services in a mobile edge computing (MEC) system. We first introduce a quality of experience (QoE) metric to measure user experience, which considers the MEC system's latency, user attention levels, and preferred resolutions. Then, a QoE maximization problem is formulated for resource al… ▽ More This paper investigates resource allocation to provide heterogeneous users with customized virtual reality (VR) services in a mobile edge computing (MEC) system. We first introduce a quality of experience (QoE) metric to measure user experience, which considers the MEC system's latency, user attention levels, and preferred resolutions. Then, a QoE maximization problem is formulated for resource allocation to ensure the highest possible user experience,which is cast as a reinforcement learning problem, aiming to learn a generalized policy applicable across diverse user environments for all MEC servers. To learn the generalized policy, we propose a framework that employs federated learning (FL) and prompt-based sequence modeling to pre-train a common decision model across MEC servers, which is named FedPromptDT. Using FL solves the problem of insufficient local MEC data while protecting user privacy during offline training. The design of prompts integrating user-environment cues and user-preferred allocation improves the model's adaptability to various user environments during online execution. △ Less

Submitted 15 February, 2024; originally announced February 2024.

Mode Connectivity and Data Heterogeneity of Federated Learning

Authors: Tailin Zhou, Jun Zhang, Danny H. K. Tsang

Abstract: Federated learning (FL) enables multiple clients to train a model while kee** their data private collaboratively. Previous studies have shown that data heterogeneity between clients leads to drifts across client updates. However, there are few studies on the relationship between client and global modes, making it unclear where these updates end up drifting. We perform empirical and theoretical s… ▽ More Federated learning (FL) enables multiple clients to train a model while kee** their data private collaboratively. Previous studies have shown that data heterogeneity between clients leads to drifts across client updates. However, there are few studies on the relationship between client and global modes, making it unclear where these updates end up drifting. We perform empirical and theoretical studies on this relationship by utilizing mode connectivity, which measures performance change (i.e., connectivity) along parametric paths between different modes. Empirically, reducing data heterogeneity makes the connectivity on different paths more similar, forming more low-error overlaps between client and global modes. We also find that a barrier to connectivity occurs when linearly connecting two global modes, while it disappears with considering non-linear mode connectivity. Theoretically, we establish a quantitative bound on the global-mode connectivity using mean-field theory or dropout stability. The bound demonstrates that the connectivity improves when reducing data heterogeneity and widening trained models. Numerical results further corroborate our analytical findings. △ Less

Submitted 28 September, 2023; originally announced September 2023.

Understanding and Improving Model Averaging in Federated Learning on Heterogeneous Data

Authors: Tailin Zhou, Zehong Lin, Jun Zhang, Danny H. K. Tsang

Abstract: Model averaging is a widely adopted technique in federated learning (FL) that aggregates multiple client models to obtain a global model. Remarkably, model averaging in FL yields a superior global model, even when client models are trained with non-convex objective functions and on heterogeneous local datasets. However, the rationale behind its success remains poorly understood. To shed light on t… ▽ More Model averaging is a widely adopted technique in federated learning (FL) that aggregates multiple client models to obtain a global model. Remarkably, model averaging in FL yields a superior global model, even when client models are trained with non-convex objective functions and on heterogeneous local datasets. However, the rationale behind its success remains poorly understood. To shed light on this issue, we first visualize the loss landscape of FL over client and global models to illustrate their geometric properties. The visualization shows that the client models encompass the global model within a common basin, and interestingly, the global model may deviate from the basin's center while still outperforming the client models. To gain further insights into model averaging in FL, we decompose the expected loss of the global model into five factors related to the client models. Specifically, our analysis reveals that the global model loss after early training mainly arises from \textit{i)} the client model's loss on non-overlap** data between client datasets and the global dataset and \textit{ii)} the maximum distance between the global and client models. Based on the findings from our loss landscape visualization and loss decomposition, we propose utilizing iterative moving averaging (IMA) on the global model at the late training phase to reduce its deviation from the expected minimum, while constraining client exploration to limit the maximum distance between the global and client models. Our experiments demonstrate that incorporating IMA into existing FL methods significantly improves their accuracy and training speed on various heterogeneous data setups of benchmark datasets. Code is available at \url{https://github.com/TailinZhou/FedIMA}. △ Less

Submitted 30 May, 2024; v1 submitted 13 May, 2023; originally announced May 2023.

Comments: To appear in IEEE Transactions on Mobile Computing. Code is available at https://github.com/TailinZhou/FedIMA

Energy-Latency Aware Intelligent Reflecting Surface Aided Multi-cell Mobile Edge Computing

Authors: Wenhan Xu, Jiadong Yu, Yuan Wu, Danny H. K. Tsang

Abstract: The explosive development of the Internet of Things (IoT) has led to increased interest in mobile edge computing (MEC), which provides computational resources at network edges to accommodate computation-intensive and latency-sensitive applications. Intelligent reflecting surfaces (IRSs) have gained attention as a solution to overcome blockage problems during the offloading uplink transmission in M… ▽ More The explosive development of the Internet of Things (IoT) has led to increased interest in mobile edge computing (MEC), which provides computational resources at network edges to accommodate computation-intensive and latency-sensitive applications. Intelligent reflecting surfaces (IRSs) have gained attention as a solution to overcome blockage problems during the offloading uplink transmission in MEC systems. This paper explores IRS-aided multi-cell networks that enable servers to serve neighboring cells and cooperate to handle resource exhaustion. We aim to minimize the joint energy and latency cost, by jointly optimizing computation tasks, edge computing resources, user beamforming, and IRS phase shifts. The problem is decomposed into two subproblems--the MEC subproblem and the IRS communication subproblem--using the block coordinate descent (BCD) technique. The MEC subproblem is reformulated as a nonconvex quadratic constrained problem (QCP), while the IRS communication subproblem is transformed into a weight-sum-rate problem with auxiliary variables. We propose an efficient algorithm to iteratively optimize MEC resources and IRS communication until convergence. Numerical results show that our algorithm outperforms benchmarks and that multi-cell MEC systems achieve additional performance gains when supported by IRS. △ Less

Submitted 5 May, 2023; originally announced May 2023.

Comments: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Journal ref: IEEE Transactions on Green Communications and Networking, vol. 8, no. 1, pp. 362-374, March 2024

Structured Bayesian Compression for Deep Neural Networks Based on The Turbo-VBI Approach

Authors: Chengyu Xia, Danny H. K. Tsang, Vincent K. N. Lau

Abstract: With the growth of neural network size, model compression has attracted increasing interest in recent research. As one of the most common techniques, pruning has been studied for a long time. By exploiting the structured sparsity of the neural network, existing methods can prune neurons instead of individual weights. However, in most existing pruning methods, surviving neurons are randomly connect… ▽ More With the growth of neural network size, model compression has attracted increasing interest in recent research. As one of the most common techniques, pruning has been studied for a long time. By exploiting the structured sparsity of the neural network, existing methods can prune neurons instead of individual weights. However, in most existing pruning methods, surviving neurons are randomly connected in the neural network without any structure, and the non-zero weights within each neuron are also randomly distributed. Such irregular sparse structure can cause very high control overhead and irregular memory access for the hardware and even increase the neural network computational complexity. In this paper, we propose a three-layer hierarchical prior to promote a more regular sparse structure during pruning. The proposed three-layer hierarchical prior can achieve per-neuron weight-level structured sparsity and neuron-level structured sparsity. We derive an efficient Turbo-variational Bayesian inferencing (Turbo-VBI) algorithm to solve the resulting model compression problem with the proposed prior. The proposed Turbo-VBI algorithm has low complexity and can support more general priors than existing model compression algorithms. Simulation results show that our proposed algorithm can promote a more regular structure in the pruned neural networks while achieving even better performance in terms of compression rate and inferencing accuracy compared with the baselines. △ Less

Submitted 21 February, 2023; originally announced February 2023.

FedFA: Federated Learning with Feature Anchors to Align Features and Classifiers for Heterogeneous Data

Authors: Tailin Zhou, Jun Zhang, Danny H. K. Tsang

Abstract: Federated learning allows multiple clients to collaboratively train a model without exchanging their data, thus preserving data privacy. Unfortunately, it suffers significant performance degradation due to heterogeneous data at clients. Common solutions involve designing an auxiliary loss to regularize weight divergence or feature inconsistency during local training. However, we discover that thes… ▽ More Federated learning allows multiple clients to collaboratively train a model without exchanging their data, thus preserving data privacy. Unfortunately, it suffers significant performance degradation due to heterogeneous data at clients. Common solutions involve designing an auxiliary loss to regularize weight divergence or feature inconsistency during local training. However, we discover that these approaches fall short of the expected performance because they ignore the existence of a vicious cycle between feature inconsistency and classifier divergence across clients. This vicious cycle causes client models to be updated in inconsistent feature spaces with more diverged classifiers. To break the vicious cycle, we propose a novel framework named Federated learning with Feature Anchors (FedFA). FedFA utilizes feature anchors to align features and calibrate classifiers across clients simultaneously. This enables client models to be updated in a shared feature space with consistent classifiers during local training. Theoretically, we analyze the non-convex convergence rate of FedFA. We also demonstrate that the integration of feature alignment and classifier calibration in FedFA brings a virtuous cycle between feature and classifier updates, which breaks the vicious cycle existing in current approaches. Extensive experiments show that FedFA significantly outperforms existing approaches on various classification datasets under label distribution skew and feature distribution skew. △ Less

Submitted 17 October, 2023; v1 submitted 16 November, 2022; originally announced November 2022.

Comments: Accepted by IEEE Transactions on Mobile Computing

An Efficient Ratio Detector for Ambient Backscatter Communication

Authors: Wen**g Liu, Shanpu Shen, Danny H. K. Tsang, Ranjan K. Mallik, Ross Murch

Abstract: Ambient backscatter communication (AmBC) leverages the existing ambient radio frequency (RF) environment to implement communication with battery-free devices. One critical challenge of AmBC systems is signal recovery because the transmitted information bits are embedded in the ambient RF signals and these are unknown and uncontrollable. To address this problem, most existing approaches use averagi… ▽ More Ambient backscatter communication (AmBC) leverages the existing ambient radio frequency (RF) environment to implement communication with battery-free devices. One critical challenge of AmBC systems is signal recovery because the transmitted information bits are embedded in the ambient RF signals and these are unknown and uncontrollable. To address this problem, most existing approaches use averaging-based energy detectors and consequently the data rate is low and there is an error floor. Here we propose a new detection strategy based on the ratio between signals received from a multiple-antenna Reader. The advantage of using the ratio is that ambient RF signals are removed directly from the embedded signals without averaging and hence it can increase data rates and avoid the error floor. Different from original ratio detectors that use the magnitude ratio of the signals between two Reader antennas, in our proposed approach, we utilize the complex ratio so that phase information is preserved and propose an accurate linear channel model approximation. This allows the application of existing linear detection techniques from which we can obtain a minimum distance detector and closed-form expressions for bit error rate (BER). In addition, averaging, coding and interleaving can also be included to further enhance the BER. The results are also general, allowing any number of Reader antennas to be utilized in the approach. Numerical results demonstrate that the proposed approach performs better than approaches based on energy detection and original ratio detectors. △ Less

Submitted 18 October, 2022; originally announced October 2022.

The Online Knapsack Problem with Departures

Authors: Bo Sun, Lin Yang, Mohammad Hajiesmaili, Adam Wierman, John C. S. Lui, Don Towsley, Danny H. K. Tsang

Abstract: The online knapsack problem is a classic online resource allocation problem in networking and operations research. Its basic version studies how to pack online arriving items of different sizes and values into a capacity-limited knapsack. In this paper, we study a general version that includes item departures, while also considering multiple knapsacks and multi-dimensional item sizes. We design a… ▽ More The online knapsack problem is a classic online resource allocation problem in networking and operations research. Its basic version studies how to pack online arriving items of different sizes and values into a capacity-limited knapsack. In this paper, we study a general version that includes item departures, while also considering multiple knapsacks and multi-dimensional item sizes. We design a threshold-based online algorithm and prove that the algorithm can achieve order-optimal competitive ratios. Beyond worst-case performance guarantees, we also aim to achieve near-optimal average performance under typical instances. Towards this goal, we propose a data-driven online algorithm that learns within a policy-class that guarantees a worst-case performance bound. In trace-driven experiments, we show that our data-driven algorithm outperforms other benchmark algorithms in an application of online knapsack to job scheduling for cloud computing. △ Less

Submitted 15 March, 2023; v1 submitted 24 September, 2022; originally announced September 2022.

Online Combinatorial Auctions for Resource Allocation with Supply Costs and Capacity Limits

Authors: Xiaoqi Tan, Alberto Leon-Garcia, Yuan Wu, Danny H. K. Tsang

Abstract: We study a general online combinatorial auction problem in algorithmic mechanism design. A provider allocates multiple types of capacity-limited resources to customers that arrive in a sequential and arbitrary manner. Each customer has a private valuation function on bundles of resources that she can purchase (e.g., a combination of different resources such as CPU and RAM in cloud computing). The… ▽ More We study a general online combinatorial auction problem in algorithmic mechanism design. A provider allocates multiple types of capacity-limited resources to customers that arrive in a sequential and arbitrary manner. Each customer has a private valuation function on bundles of resources that she can purchase (e.g., a combination of different resources such as CPU and RAM in cloud computing). The provider charges payment from customers who purchase a bundle of resources and incurs an increasing supply cost with respect to the totality of resources allocated. The goal is to maximize the social welfare, namely, the total valuation of customers for their purchased bundles, minus the total supply cost of the provider for all the resources that have been allocated. We adopt the competitive analysis framework and provide posted-price mechanisms with optimal competitive ratios. Our pricing mechanism is optimal in the sense that no other online algorithms can achieve a better competitive ratio. We validate the theoretic results via empirical studies of online resource allocation in cloud computing. Our numerical results demonstrate that the proposed pricing mechanism is competitive and robust against system uncertainties and outperforms existing benchmarks. △ Less

Submitted 8 October, 2023; v1 submitted 15 September, 2022; originally announced September 2022.

Comments: arXiv admin note: text overlap with arXiv:2004.09640

Optimal Regularized Online Allocation by Adaptive Re-Solving

Authors: Wanteng Ma, Ying Cao, Danny H. K. Tsang, Dong Xia

Abstract: This paper introduces a dual-based algorithm framework for solving the regularized online resource allocation problems, which have potentially non-concave cumulative rewards, hard resource constraints, and a non-separable regularizer. Under a strategy of adaptively updating the resource constraints, the proposed framework only requests approximate solutions to the empirical dual problems up to a c… ▽ More This paper introduces a dual-based algorithm framework for solving the regularized online resource allocation problems, which have potentially non-concave cumulative rewards, hard resource constraints, and a non-separable regularizer. Under a strategy of adaptively updating the resource constraints, the proposed framework only requests approximate solutions to the empirical dual problems up to a certain accuracy and yet delivers an optimal logarithmic regret under a locally second-order growth condition. Surprisingly, a delicate analysis of the dual objective function enables us to eliminate the notorious log-log factor in regret bound. The flexible framework renders renowned and computationally fast algorithms immediately applicable, e.g., dual stochastic gradient descent. Additionally, an infrequent re-solving scheme is proposed, which significantly reduces computational demands without compromising the optimal regret performance. A worst-case square-root regret lower bound is established if the resource constraints are not adaptively updated during dual optimization, which underscores the critical role of adaptive dual variable update. Comprehensive numerical experiments demonstrate the merits of the proposed algorithm framework. △ Less

Submitted 15 July, 2023; v1 submitted 1 September, 2022; originally announced September 2022.

Distributed Intelligence in Wireless Networks

Authors: Xiaolan Liu, Jiadong Yu, Yuanwei Liu, Yue Gao, Toktam Mahmoodi, Sangarapillai Lambotharan, Danny H. K. Tsang

Abstract: The cloud-based solutions are becoming inefficient due to considerably large time delays, high power consumption, security and privacy concerns caused by billions of connected wireless devices and typically zillions bytes of data they produce at the network edge. A blend of edge computing and Artificial Intelligence (AI) techniques could optimally shift the resourceful computation servers closer t… ▽ More The cloud-based solutions are becoming inefficient due to considerably large time delays, high power consumption, security and privacy concerns caused by billions of connected wireless devices and typically zillions bytes of data they produce at the network edge. A blend of edge computing and Artificial Intelligence (AI) techniques could optimally shift the resourceful computation servers closer to the network edge, which provides the support for advanced AI applications (e.g., video/audio surveillance and personal recommendation system) by enabling intelligent decision making on computing at the point of data generation as and when it is needed, and distributed Machine Learning (ML) with its potential to avoid the transmission of large dataset and possible compromise of privacy that may exist in cloud-based centralized learning. Therefore, AI is envisioned to become native and ubiquitous in future communication and networking systems. In this paper, we conduct a comprehensive overview of recent advances in distributed intelligence in wireless networks under the umbrella of native-AI wireless networks, with a focus on the basic concepts of native-AI wireless networks, on the AI-enabled edge computing, on the design of distributed learning architectures for heterogeneous networks, on the communication-efficient technologies to support distributed learning, and on the AI-empowered end-to-end communications. We highlight the advantages of hybrid distributed learning architectures compared to the state-of-art distributed learning techniques. We summarize the challenges of existing research contributions in distributed intelligence in wireless networks and identify the potential future opportunities. △ Less

Submitted 31 July, 2022; originally announced August 2022.

Comments: submitted to IEEE Communications Surveys & Tutorials

Pareto-Optimal Learning-Augmented Algorithms for Online Conversion Problems

Authors: Bo Sun, Russell Lee, Mohammad Hajiesmaili, Adam Wierman, Danny H. K. Tsang

Abstract: This paper leverages machine-learned predictions to design competitive algorithms for online conversion problems with the goal of improving the competitive ratio when predictions are accurate (i.e., consistency), while also guaranteeing a worst-case competitive ratio regardless of the prediction quality (i.e., robustness). We unify the algorithmic design of both integral and fractional conversion… ▽ More This paper leverages machine-learned predictions to design competitive algorithms for online conversion problems with the goal of improving the competitive ratio when predictions are accurate (i.e., consistency), while also guaranteeing a worst-case competitive ratio regardless of the prediction quality (i.e., robustness). We unify the algorithmic design of both integral and fractional conversion problems, which are also known as the 1-max-search and one-way trading problems, into a class of online threshold-based algorithms (OTA). By incorporating predictions into design of OTA, we achieve the Pareto-optimal trade-off of consistency and robustness, i.e., no online algorithm can achieve a better consistency guarantee given for a robustness guarantee. We demonstrate the performance of OTA using numerical experiments on Bitcoin conversion. △ Less

Submitted 3 September, 2021; originally announced September 2021.

Online Network Utility Maximization: Algorithm, Competitive Analysis, and Applications

Authors: Ying Cao, Bo Sun, Danny H. K. Tsang

Abstract: We consider an online version of the well-studied network utility maximization problem, where users arrive one by one and an operator makes irrevocable decisions for each user without knowing the details of future arrivals. We propose a threshold-based algorithm and analyze its worst-case performance. We prove that the competitive ratio of the proposed algorithm is linearly increasing in the numbe… ▽ More We consider an online version of the well-studied network utility maximization problem, where users arrive one by one and an operator makes irrevocable decisions for each user without knowing the details of future arrivals. We propose a threshold-based algorithm and analyze its worst-case performance. We prove that the competitive ratio of the proposed algorithm is linearly increasing in the number of links in a network and show this competitive analysis is tight. Extensive trace-driven simulations are conducted to demonstrate the performance of our proposed algorithm. In addition, since worst-case scenarios rarely occur in practice, we devise an adaptive implementation of our algorithm to improve its average-case performance and validate its effectiveness via simulations. △ Less

Submitted 26 January, 2021; originally announced January 2021.

Competitive Algorithms for the Online Multiple Knapsack Problem with Application to Electric Vehicle Charging

Authors: Bo Sun, Ali Zeynali, Tongxin Li, Mohammad Hajiesmaili, Adam Wierman, Danny H. K. Tsang

Abstract: We introduce and study a general version of the fractional online knapsack problem with multiple knapsacks, heterogeneous constraints on which items can be assigned to which knapsack, and rate-limiting constraints on the assignment of items to knapsacks. This problem generalizes variations of the knapsack problem and of the one-way trading problem that have previously been treated separately, and… ▽ More We introduce and study a general version of the fractional online knapsack problem with multiple knapsacks, heterogeneous constraints on which items can be assigned to which knapsack, and rate-limiting constraints on the assignment of items to knapsacks. This problem generalizes variations of the knapsack problem and of the one-way trading problem that have previously been treated separately, and additionally finds application to the real-time control of electric vehicle (EV) charging. We introduce a new algorithm that achieves a competitive ratio within an additive factor of one of the best achievable competitive ratios for the general problem and matches or improves upon the best-known competitive ratio for special cases in the knapsack and one-way trading literatures. Moreover, our analysis provides a novel approach to online algorithm design based on an instance-dependent primal-dual analysis that connects the identification of worst-case instances to the design of algorithms. Finally, we illustrate the proposed algorithm via trace-based experiments of EV charging. △ Less

Submitted 17 October, 2020; v1 submitted 1 October, 2020; originally announced October 2020.

Delay-Aware Scheduling over mmWave/Sub-6 Dual Interfaces: A Reinforcement Learning Approach

Authors: Ying Cao, Bo Sun, Danny H. K. Tsang

Abstract: We consider a transmitter with mmWave/sub6 dual interfaces. Due to the intermittency of mmWave channel, the transmitter must schedule packets wisely across the interfaces to minimize the average delay by observing the system state. We usethe well-known dynamic programming methods and Q-learning to find the optimal scheduling policy and investigate the influenceof observing CSI on the optimal polic… ▽ More We consider a transmitter with mmWave/sub6 dual interfaces. Due to the intermittency of mmWave channel, the transmitter must schedule packets wisely across the interfaces to minimize the average delay by observing the system state. We usethe well-known dynamic programming methods and Q-learning to find the optimal scheduling policy and investigate the influenceof observing CSI on the optimal policy under different levels of knowledge of the environment. We find that only when the channel state transition model is not available, the instantaneousCSI can help in reducing system delay △ Less

Submitted 18 May, 2020; originally announced May 2020.

Comments: To be published in Workshop on 5G Long Term Evolution and Intelligent Communication, ICC 2020

Optimal Online Algorithms for One-Way Trading and Online Knapsack Problems: A Unified Competitive Analysis

Authors: Ying Cao, Bo Sun, Danny H. K. Tsang

Abstract: We study two canonical online optimization problems under capacity/budget constraints: the fractional one-way trading problem (OTP) and the integral online knapsack problem (OKP) under an infinitesimal assumption. Under the competitive analysis framework, it is well-known that both problems have the same optimal competitive ratio. However, these two problems are investigated by distinct approaches… ▽ More We study two canonical online optimization problems under capacity/budget constraints: the fractional one-way trading problem (OTP) and the integral online knapsack problem (OKP) under an infinitesimal assumption. Under the competitive analysis framework, it is well-known that both problems have the same optimal competitive ratio. However, these two problems are investigated by distinct approaches under separate contexts in the literature. There is a gap in understanding the connection between these two problems and the nature of their online algorithm design. This paper provides a unified framework for the online algorithm design, analysis and optimality proof for both problems. We find that the infinitesimal assumption of the OKP is the key that connects the OTP in the analysis of online algorithms and the construction of worst-case instances. With this unified understanding, our framework shows its potential for analyzing other extensions of OKP and OTP in a more systematic manner. △ Less

Submitted 22 September, 2020; v1 submitted 21 April, 2020; originally announced April 2020.

Mechanism Design for Online Resource Allocation: A Unified Approach

Authors: Xiaoqi Tan, Bo Sun, Alberto Leon-Garcia, Yuan Wu, Danny H. K. Tsang

Abstract: This paper concerns the mechanism design for online resource allocation in a strategic setting. In this setting, a single supplier allocates capacity-limited resources to requests that arrive in a sequential and arbitrary manner. Each request is associated with an agent who may act selfishly to misreport the requirement and valuation of her request. The supplier charges payment from agents whose r… ▽ More This paper concerns the mechanism design for online resource allocation in a strategic setting. In this setting, a single supplier allocates capacity-limited resources to requests that arrive in a sequential and arbitrary manner. Each request is associated with an agent who may act selfishly to misreport the requirement and valuation of her request. The supplier charges payment from agents whose requests are satisfied, but incurs a load-dependent supply cost. The goal is to design an incentive compatible online mechanism, which determines not only the resource allocation of each request, but also the payment of each agent, so as to (approximately) maximize the social welfare (i.e., aggregate valuations minus supply cost). We study this problem under the framework of competitive analysis. The major contribution of this paper is the development of a unified approach that achieves the best-possible competitive ratios for setups with different supply costs. Specifically, we show that when there is no supply cost or the supply cost function is linear, our model is essentially a standard 0-1 knapsack problem, for which our approach achieves logarithmic competitive ratios that match the state-of-the-art (which is optimal). For the more challenging setup when the supply cost is strictly-convex, we provide online mechanisms, for the first time, that lead to the optimal competitive ratios as well. To the best of our knowledge, this is the first approach that unifies the characterization of optimal competitive ratios in online resource allocation for different setups including zero, linear and strictly-convex supply costs. △ Less

Submitted 8 October, 2023; v1 submitted 20 April, 2020; originally announced April 2020.

Parallel Stochastic Optimization Framework for Large-Scale Non-Convex Stochastic Problems

Authors: Naeimeh Omidvar, An Liu, Vincent Lau, Danny H. K. Tsang, Mohammad Reza Pakravan

Abstract: In this paper, we consider the problem of stochastic optimization, where the objective function is in terms of the expectation of a (possibly non-convex) cost function that is parametrized by a random variable. While the convergence speed is critical for many emerging applications, most existing stochastic optimization methods suffer from slow convergence. Furthermore, the emerging technology of p… ▽ More In this paper, we consider the problem of stochastic optimization, where the objective function is in terms of the expectation of a (possibly non-convex) cost function that is parametrized by a random variable. While the convergence speed is critical for many emerging applications, most existing stochastic optimization methods suffer from slow convergence. Furthermore, the emerging technology of parallel computing has motivated an increasing demand for designing new stochastic optimization schemes that can handle parallel optimization for implementation in distributed systems. We propose a fast parallel stochastic optimization framework that can solve a large class of possibly non-convex stochastic optimization problems that may arise in applications with multi-agent systems. In the proposed method, each agent updates its control variable in parallel, by solving a convex quadratic subproblem independently. The convergence of the proposed method to the optimal solution for convex problems and to a stationary point for general non-convex problems is established. The proposed algorithm can be applied to solve a large class of optimization problems arising in important applications from various fields, such as machine learning and wireless networks. As a representative application of our proposed stochastic optimization framework, we focus on large-scale support vector machines and demonstrate how our algorithm can efficiently solve this problem, especially in modern applications with huge datasets. Using popular real-world datasets, we present experimental results to demonstrate the merits of our proposed framework by comparing its performance to the state-of-the-art in the literature. Numerical results show that the proposed method can significantly outperform the state-of-the-art methods in terms of the convergence speed while having the same or lower complexity and storage requirement. △ Less

Submitted 22 October, 2019; originally announced October 2019.

Asymptotic Performance Evaluation of Battery Swap** and Charging Station for Electric Vehicles

Authors: Xiaoqi Tan, Bo Sun, Yuan Wu, Danny H. K. Tsang

Abstract: A battery swap** and charging station (BSCS) is an energy refueling station, where i) electric vehicles (EVs) with depleted batteries (DBs) can swap their DBs for fully-charged ones, and ii) the swapped DBs are then charged until they are fully-charged. Successful deployment of a BSCS system necessitates a careful planning of swap**- and charging-related infrastructures, and thus a comprehensi… ▽ More A battery swap** and charging station (BSCS) is an energy refueling station, where i) electric vehicles (EVs) with depleted batteries (DBs) can swap their DBs for fully-charged ones, and ii) the swapped DBs are then charged until they are fully-charged. Successful deployment of a BSCS system necessitates a careful planning of swap**- and charging-related infrastructures, and thus a comprehensive performance evaluation of the BSCS is becoming crucial. This paper studies such a performance evaluation problem with a novel mixed queueing network (MQN) model and validates this model with extensive numerical simulation. We adopt the EVs' blocking probability as our quality-of-service measure and focus on studying the impact of the key parameters of the BSCS (e.g., the numbers of parking spaces, swap** islands, chargers, and batteries) on the blocking probability. We prove a necessary and sufficient condition for showing the ergodicity of the MQN when the number of batteries approaches infinity, and further prove that the blocking probability has two different types of asymptotic behaviors. Meanwhile, for each type of asymptotic behavior, we analytically derive the asymptotic lower bound of the blocking probability. △ Less

Submitted 26 July, 2017; v1 submitted 22 July, 2017; originally announced July 2017.

Comments: 35 pages, 11 figures

Optimal Hierarchical Radio Resource Management for HetNets with Flexible Backhaul

Authors: Naeimeh Omidvar, An Liu, Vincent Lau, Fan Zhang, Danny H. K. Tsang, Mohammad Reza Pakravan

Abstract: Providing backhaul connectivity for macro and pico base stations (BSs) constitutes a significant share of infrastructure costs in future heterogeneous networks (HetNets). To address this issue, the emerging idea of flexible backhaul is proposed. Under this architecture, not all the pico BSs are connected to the backhaul, resulting in a significant reduction in the infrastructure costs. In this reg… ▽ More Providing backhaul connectivity for macro and pico base stations (BSs) constitutes a significant share of infrastructure costs in future heterogeneous networks (HetNets). To address this issue, the emerging idea of flexible backhaul is proposed. Under this architecture, not all the pico BSs are connected to the backhaul, resulting in a significant reduction in the infrastructure costs. In this regard, pico BSs without backhaul connectivity need to communicate with their nearby BSs in order to have indirect accessibility to the backhaul. This makes the radio resource management (RRM) in such networks more complex and challenging. In this paper, we address the problem of cross-layer RRM in HetNets with flexible backhaul. We formulate this problem as a two-timescale non-convex stochastic optimization which jointly optimizes flow control, routing, interference mitigation and link scheduling in order to maximize a generic network utility. By exploiting a hidden convexity of this non-convex problem, we propose an iterative algorithm which converges to the global optimal solution. The proposed algorithm benefits from low complexity and low signalling, which makes it scalable. Moreover, due to the proposed two-timescale design, it is robust to the backhaul signalling latency as well. Simulation results demonstrate the significant performance gain of the proposed solution over various baselines. △ Less

Submitted 27 August, 2020; v1 submitted 15 April, 2015; originally announced April 2015.

Comments: We note that the definition of Subproblem 2 (named as Problem P2 in the paper) was missed in the published version of this paper in IEEE Transactions on Wireless Communications. This definition is denoted by equation (8) in this arXiv version

Journal ref: IEEE Transactions on Wireless Communications, 17(7), pp.4239-4255 (2018)

Energy-Efficient Delay-Constrained Transmission and Sensing for Cognitive Radio Systems

Authors: Yuan Wu, Vincent K. N. Lau, Danny H. K. Tsang, Li** Qian

Abstract: In this work we study energy-efficient transmission for Cognitive Radio (CR) which opportunistically operates on Primary User's (PU's) channel through spectrum sensing. Spectrum sensing and compulsory idling (for incumbent protection) introduce energy-overheads for Secondary User's (SU's) operations, and thus an appropriate balance between energy consumption in data transmission and energy-overhea… ▽ More In this work we study energy-efficient transmission for Cognitive Radio (CR) which opportunistically operates on Primary User's (PU's) channel through spectrum sensing. Spectrum sensing and compulsory idling (for incumbent protection) introduce energy-overheads for Secondary User's (SU's) operations, and thus an appropriate balance between energy consumption in data transmission and energy-overheads is required. We formulate this problem as a discrete-time Markov Decision Process (MDP) in which the SU aims at minimizing its average cost (including both energy consumption and delay cost) to finish a target traffic payload through an appropriate rate allocation. Based on Certainty Equivalent Control, we propose a low-complexity rate-adaptation policy that achieves comparable performance as the optimal policy. With the low-complexity policy, we quantify the impact of energy-overheads (including the power consumption for spectrum sensing and compulsory idling) on the SU transmission strategy. Specifically, the SU rate increases with the increase of energy-overheads, whose marginal impact, however, diminishes. Moreover, the marginal impact of energy-overheads is more significant for delay-insensitive traffic compared to that for delay-sensitive traffic. To mitigate the loss due to imperfect spectrum sensing, we quantify that the SU decreases (increases) its rate with a larger mis-detection probability (false alarm probability). △ Less

Submitted 8 May, 2012; v1 submitted 2 May, 2012; originally announced May 2012.

Comments: waiting for the agreements from all the coauthors