Showing 1–40 of 40 results for author: Eryilmaz, A

Non-Federated Multi-Task Split Learning for Heterogeneous Sources

Authors: Yilin Zheng, Atilla Eryilmaz

Abstract: With the development of edge networks and mobile computing, the need to serve heterogeneous data sources at the network edge requires the design of new distributed machine learning mechanisms. As a prevalent approach, Federated Learning (FL) employs parameter-sharing and gradient-averaging between clients and a server. Despite its many favorable qualities, such as convergence and data-privacy guar… ▽ More With the development of edge networks and mobile computing, the need to serve heterogeneous data sources at the network edge requires the design of new distributed machine learning mechanisms. As a prevalent approach, Federated Learning (FL) employs parameter-sharing and gradient-averaging between clients and a server. Despite its many favorable qualities, such as convergence and data-privacy guarantees, it is well-known that classic FL fails to address the challenge of data heterogeneity and computation heterogeneity across clients. Most existing works that aim to accommodate such sources of heterogeneity stay within the FL operation paradigm, with modifications to overcome the negative effect of heterogeneous data. In this work, as an alternative paradigm, we propose a Multi-Task Split Learning (MTSL) framework, which combines the advantages of Split Learning (SL) with the flexibility of distributed network architectures. In contrast to the FL counterpart, in this paradigm, heterogeneity is not an obstacle to overcome, but a useful property to take advantage of. As such, this work aims to introduce a new architecture and methodology to perform multi-task learning for heterogeneous data sources efficiently, with the hope of encouraging the community to further explore the potential advantages we reveal. To support this promise, we first show through theoretical analysis that MTSL can achieve fast convergence by tuning the learning rate of the server and clients. Then, we compare the performance of MTSL with existing multi-task FL methods numerically on several image classification datasets to show that MTSL has advantages over FL in training speed, communication cost, and robustness to heterogeneous data. △ Less

Submitted 31 May, 2024; originally announced June 2024.

Recurrent Natural Policy Gradient for POMDPs

Authors: Semih Cayci, Atilla Eryilmaz

Abstract: In this paper, we study a natural policy gradient method based on recurrent neural networks (RNNs) for partially-observable Markov decision processes, whereby RNNs are used for policy parameterization and policy evaluation to address curse of dimensionality in non-Markovian reinforcement learning. We present finite-time and finite-width analyses for both the critic (recurrent temporal difference l… ▽ More In this paper, we study a natural policy gradient method based on recurrent neural networks (RNNs) for partially-observable Markov decision processes, whereby RNNs are used for policy parameterization and policy evaluation to address curse of dimensionality in non-Markovian reinforcement learning. We present finite-time and finite-width analyses for both the critic (recurrent temporal difference learning), and correspondingly-operated recurrent natural policy gradient method in the near-initialization regime. Our analysis demonstrates the efficiency of RNNs for problems with short-term memory with explicit bounds on the required network widths and sample complexity, and points out the challenges in the case of long-term dependencies. △ Less

Submitted 28 May, 2024; originally announced May 2024.

SwiftCache: Model-Based Learning for Dynamic Content Caching in CDNs

Authors: Bahman Abolhassani, Atilla Eryilmaz, Tom Hou

Abstract: We introduce SwiftCache, a "fresh" learning-based caching framework designed for content distribution networks (CDNs) featuring distributed front-end local caches and a dynamic back-end database. Users prefer the most recent version of the dynamically updated content, while the local caches lack knowledge of item popularity and refresh rates. We first explore scenarios with requests arriving at a… ▽ More We introduce SwiftCache, a "fresh" learning-based caching framework designed for content distribution networks (CDNs) featuring distributed front-end local caches and a dynamic back-end database. Users prefer the most recent version of the dynamically updated content, while the local caches lack knowledge of item popularity and refresh rates. We first explore scenarios with requests arriving at a local cache following a Poisson process, whereby we prove that the optimal policy features a threshold-based structure with updates occurring solely at request arrivals. Leveraging these findings, SwiftCache is proposed as a model-based learning framework for dynamic content caching. The simulation demonstrates near-optimal cost for Poisson process arrivals and strong performance with limited cache sizes. For more general environments, we present a model-free Reinforcement Learning (RL) based caching policy without prior statistical assumptions. The model-based policy performs well compared to the model-free policy when the variance of interarrival times remains moderate. However, as the variance increases, RL slightly outperforms model-based learning at the cost of longer training times, and higher computational resource consumption. Model-based learning's adaptability to environmental changes without retraining positions it as a practical choice for dynamic network environments. Distributed edge caches can utilize this approach in a decentralized manner to effectively meet the evolving behaviors of users. △ Less

Submitted 26 February, 2024; originally announced February 2024.

Comments: arXiv admin note: text overlap with arXiv:2401.03613

Convergence of Gradient Descent for Recurrent Neural Networks: A Nonasymptotic Analysis

Authors: Semih Cayci, Atilla Eryilmaz

Abstract: We analyze recurrent neural networks trained with gradient descent in the supervised learning setting for dynamical systems, and prove that gradient descent can achieve optimality \emph{without} massive overparameterization. Our in-depth nonasymptotic analysis (i) provides sharp bounds on the network size $m$ and iteration complexity $τ$ in terms of the sequence length $T$, sample size $n$ and amb… ▽ More We analyze recurrent neural networks trained with gradient descent in the supervised learning setting for dynamical systems, and prove that gradient descent can achieve optimality \emph{without} massive overparameterization. Our in-depth nonasymptotic analysis (i) provides sharp bounds on the network size $m$ and iteration complexity $τ$ in terms of the sequence length $T$, sample size $n$ and ambient dimension $d$, and (ii) identifies the significant impact of long-term dependencies in the dynamical system on the convergence and network width bounds characterized by a cutoff point that depends on the Lipschitz continuity of the activation function. Remarkably, this analysis reveals that an appropriately-initialized recurrent neural network trained with $n$ samples can achieve optimality with a network size $m$ that scales only logarithmically with $n$. This sharply contrasts with the prior works that require high-order polynomial dependency of $m$ on $n$ to establish strong regularity conditions. Our results are based on an explicit characterization of the class of dynamical systems that can be approximated and learned by recurrent neural networks via norm-constrained transportation map**s, and establishing local smoothness properties of the hidden state with respect to the learnable parameters. △ Less

Submitted 19 February, 2024; originally announced February 2024.

Optimal Push and Pull-Based Edge Caching For Dynamic Content

Authors: Bahman Abolhassani, John Tadrous, Atilla Eryilmaz, Serdar Yüksel

Abstract: We introduce a framework and optimal `fresh' caching for a content distribution network (CDN) comprising a front-end local cache and a back-end database. The data content is dynamically updated at a back-end database and end-users are interested in the most-recent version of that content. We formulate the average cost minimization problem that captures the system's cost due to the service of aging… ▽ More We introduce a framework and optimal `fresh' caching for a content distribution network (CDN) comprising a front-end local cache and a back-end database. The data content is dynamically updated at a back-end database and end-users are interested in the most-recent version of that content. We formulate the average cost minimization problem that captures the system's cost due to the service of aging content as well as the regular cache update cost. We consider the cost minimization problem from two individual perspectives based on the available information to either side of the CDN: the back-end database perspective and the front-end local cache perspective. For the back-end database, the instantaneous version of content is observable but the exact demand is not. Caching decisions made by the back-end database are termed `push-based caching'. For the front-end local cache, the age of content version in the cache is not observable, yet the instantaneous demand is. Caching decisions made by the front-end local cache are termed `pull-based caching'. Our investigations reveal which type of information, updates, or demand dynamic, is of higher value towards achieving the minimum cost based on other network parameters including content popularity, update rate, and demand intensity. △ Less

Submitted 7 January, 2024; originally announced January 2024.

Comments: 14 pages, 7 figures

Optimal Edge Caching For Individualized Demand Dynamics

Authors: Guocong Quan, Atilla Eryilmaz, Ness Shroff

Abstract: The ever-growing end user data demands, and the simultaneous reductions in memory costs are fueling edge-caching deployments. Caching at the edge is substantially different from that at the core and needs to take into account the nature of individual data demands. For example, an individual user may not be interested in requesting the same data item again, if it has recently requested it. Such ind… ▽ More The ever-growing end user data demands, and the simultaneous reductions in memory costs are fueling edge-caching deployments. Caching at the edge is substantially different from that at the core and needs to take into account the nature of individual data demands. For example, an individual user may not be interested in requesting the same data item again, if it has recently requested it. Such individual dynamics are not apparent in the aggregated data requests at the core and have not been considered in popularity-driven caching designs for the core. Hence, these traditional caching policies could induce significant inefficiencies when applied at the edges. To address this issue, we develop new edge caching policies optimized for the individual demands that also leverage overhearing opportunities at the wireless edge. With the objective of maximizing the hit ratio, the proposed policies will actively evict the data items that are not likely to be requested in the near future, and strategically bring them back into the cache through overhearing when they are likely to be popular again. Both theoretical analysis and numerical simulations demonstrate that the proposed edge caching policies could outperform the popularity-driven policies that are optimal at the core. △ Less

Submitted 23 October, 2023; originally announced October 2023.

Provably Robust Temporal Difference Learning for Heavy-Tailed Rewards

Authors: Semih Cayci, Atilla Eryilmaz

Abstract: In a broad class of reinforcement learning applications, stochastic rewards have heavy-tailed distributions, which lead to infinite second-order moments for stochastic (semi)gradients in policy evaluation and direct policy optimization. In such instances, the existing RL methods may fail miserably due to frequent statistical outliers. In this work, we establish that temporal difference (TD) learni… ▽ More In a broad class of reinforcement learning applications, stochastic rewards have heavy-tailed distributions, which lead to infinite second-order moments for stochastic (semi)gradients in policy evaluation and direct policy optimization. In such instances, the existing RL methods may fail miserably due to frequent statistical outliers. In this work, we establish that temporal difference (TD) learning with a dynamic gradient clip** mechanism, and correspondingly operated natural actor-critic (NAC), can be provably robustified against heavy-tailed reward distributions. It is shown in the framework of linear function approximation that a favorable tradeoff between bias and variability of the stochastic gradients can be achieved with this dynamic gradient clip** mechanism. In particular, we prove that robust versions of TD learning achieve sample complexities of order $\mathcal{O}(\varepsilon^{-\frac{1}{p}})$ and $\mathcal{O}(\varepsilon^{-1-\frac{1}{p}})$ with and without the full-rank assumption on the feature matrix, respectively, under heavy-tailed rewards with finite moments of order $(1+p)$ for some $p\in(0,1]$, both in expectation and with high probability. We show that a robust variant of NAC based on Robust TD learning achieves $\tilde{\mathcal{O}}(\varepsilon^{-4-\frac{2}{p}})$ sample complexity. We corroborate our theoretical results with numerical experiments. △ Less

Submitted 20 June, 2023; originally announced June 2023.

Age-Optimal Multi-Channel-Scheduling under Energy and Tolerance Constraints

Authors: Xu** Zhou, Irem Koprulu, Atilla Eryilmaz

Abstract: We study the optimal scheduling problem where n source nodes attempt to transmit updates over L shared wireless on/off fading channels to optimize their age performance under energy and age-violation tolerance constraints. Specifically, we provide a generic formulation of age-optimization in the form of a constrained Markov Decision Processes (CMDP), and obtain the optimal scheduler as the solutio… ▽ More We study the optimal scheduling problem where n source nodes attempt to transmit updates over L shared wireless on/off fading channels to optimize their age performance under energy and age-violation tolerance constraints. Specifically, we provide a generic formulation of age-optimization in the form of a constrained Markov Decision Processes (CMDP), and obtain the optimal scheduler as the solution of an associated Linear Programming problem. We investigate the characteristics of the optimal single-user multi-channel scheduler for the important special cases of average-age and violation-rate minimization. This leads to several key insights on the nature of the optimal allocation of the limited energy, where a usual threshold-based policy does not apply and will be useful in guiding scheduler designers. We then investigate the stability region of the optimal scheduler for the multi-user case. We also develop an online scheduler using Lyapunov-drift-minimization methods that do not require the knowledge of channel statistics. Our numerical studies compare the stability region of our online scheduler to the optimal scheduler to reveal that it performs closely with unknown channel statistics. △ Less

Submitted 2 January, 2023; originally announced January 2023.

A Lyapunov-Based Methodology for Constrained Optimization with Bandit Feedback

Authors: Semih Cayci, Yilin Zheng, Atilla Eryilmaz

Abstract: In a wide variety of applications including online advertising, contractual hiring, and wireless scheduling, the controller is constrained by a stringent budget constraint on the available resources, which are consumed in a random amount by each action, and a stochastic feasibility constraint that may impose important operational limitations on decision-making. In this work, we consider a general… ▽ More In a wide variety of applications including online advertising, contractual hiring, and wireless scheduling, the controller is constrained by a stringent budget constraint on the available resources, which are consumed in a random amount by each action, and a stochastic feasibility constraint that may impose important operational limitations on decision-making. In this work, we consider a general model to address such problems, where each action returns a random reward, cost, and penalty from an unknown joint distribution, and the decision-maker aims to maximize the total reward under a budget constraint $B$ on the total cost and a stochastic constraint on the time-average penalty. We propose a novel low-complexity algorithm based on Lyapunov optimization methodology, named ${\tt LyOn}$, and prove that for $K$ arms it achieves $O(\sqrt{K B\log B})$ regret and zero constraint-violation when $B$ is sufficiently large. The low computational cost and sharp performance bounds of ${\tt LyOn}$ suggest that Lyapunov-based algorithm design methodology can be effective in solving constrained bandit optimization problems. △ Less

Submitted 23 January, 2022; v1 submitted 9 June, 2021; originally announced June 2021.

Continuous-Time Multi-Armed Bandits with Controlled Restarts

Authors: Semih Cayci, Atilla Eryilmaz, R. Srikant

Abstract: Time-constrained decision processes have been ubiquitous in many fundamental applications in physics, biology and computer science. Recently, restart strategies have gained significant attention for boosting the efficiency of time-constrained processes by expediting the completion times. In this work, we investigate the bandit problem with controlled restarts for time-constrained decision processe… ▽ More Time-constrained decision processes have been ubiquitous in many fundamental applications in physics, biology and computer science. Recently, restart strategies have gained significant attention for boosting the efficiency of time-constrained processes by expediting the completion times. In this work, we investigate the bandit problem with controlled restarts for time-constrained decision processes, and develop provably good learning algorithms. In particular, we consider a bandit setting where each decision takes a random completion time, and yields a random and correlated reward at the end, with unknown values at the time of decision. The goal of the decision-maker is to maximize the expected total reward subject to a time constraint $τ$. As an additional control, we allow the decision-maker to interrupt an ongoing task and forgo its reward for a potentially more rewarding alternative. For this problem, we develop efficient online learning algorithms with $O(\log(τ))$ and $O(\sqrt{τ\log(τ)})$ regret in a finite and continuous action space of restart strategies, respectively. We demonstrate an applicability of our algorithm by using it to boost the performance of SAT solvers. △ Less

Submitted 30 June, 2020; originally announced July 2020.

Group-Fair Online Allocation in Continuous Time

Authors: Semih Cayci, Swati Gupta, Atilla Eryilmaz

Abstract: The theory of discrete-time online learning has been successfully applied in many problems that involve sequential decision-making under uncertainty. However, in many applications including contractual hiring in online freelancing platforms and server allocation in cloud computing systems, the outcome of each action is observed only after a random and action-dependent time. Furthermore, as a conse… ▽ More The theory of discrete-time online learning has been successfully applied in many problems that involve sequential decision-making under uncertainty. However, in many applications including contractual hiring in online freelancing platforms and server allocation in cloud computing systems, the outcome of each action is observed only after a random and action-dependent time. Furthermore, as a consequence of certain ethical and economic concerns, the controller may impose deadlines on the completion of each task, and require fairness across different groups in the allocation of total time budget $B$. In order to address these applications, we consider continuous-time online learning problem with fairness considerations, and present a novel framework based on continuous-time utility maximization. We show that this formulation recovers reward-maximizing, max-min fair and proportionally fair allocation rules across different groups as special cases. We characterize the optimal offline policy, which allocates the total time between different actions in an optimally fair way (as defined by the utility function), and impose deadlines to maximize time-efficiency. In the absence of any statistical knowledge, we propose a novel online learning algorithm based on dual ascent optimization for time averages, and prove that it achieves $\tilde{O}(B^{-1/2})$ regret bound. △ Less

Submitted 23 July, 2020; v1 submitted 11 June, 2020; originally announced June 2020.

Comments: Corrected figure captions. Added references

Optimal Decisions of a Rational Agent in the Presence of Biased Information Providers

Authors: H. Kesavareddigari, A. Eryilmaz

Abstract: We consider information networks whereby multiple biased-information-providers (BIPs), e.g., media outlets/social network users/sensors, share reports of events with rational-information-consumers (RICs). Making the reasonable abstraction that an event can be reported as an answer to a logical statement, we model the input-output behavior of each BIP as a binary channel. For various reasons, some… ▽ More We consider information networks whereby multiple biased-information-providers (BIPs), e.g., media outlets/social network users/sensors, share reports of events with rational-information-consumers (RICs). Making the reasonable abstraction that an event can be reported as an answer to a logical statement, we model the input-output behavior of each BIP as a binary channel. For various reasons, some BIPs might share incorrect reports of the event. Moreover, each BIP is: 'biased' if it favors one of the two outcomes while reporting, or 'unbiased' if it favors neither outcome. Such biases occur in information/social networks due to differences in users' characteristics/worldviews. We study the impact of the BIPs' biases on an RIC's choices while deducing the true information. Our work reveals that a "graph-blind" RIC looking for $n$ BIPs among its neighbors, acts peculiarly in order to minimize its probability of making an error while deducing the true information. First, we establish the counter-intuitive fact that the RIC's expected error is minimized by choosing BIPs that are fully-biased against the a-priori likely event. Then, we study the gains that fully-biased BIPs provide over unbiased BIPs when the error rates of their binary channels are equalized, for fair comparison, at some $r>0$. Specifically, the unbiased-to-fully-biased ratio of the RIC's expected error probabilities grows exponentially with the exponent $\frac{n}{2}\ln\left(4ρ_0^2\left(\frac{1}{r}-1\right)\right)$, where $ρ_0$ is the event's prior probability of being $0$. This shows not only that fully-biased BIPs are preferable to unbiased or heterogeneously-biased BIPs, but also that the gains can be substantial for small $r$. △ Less

Submitted 27 April, 2020; originally announced April 2020.

Comments: To be published in WiOpt 2020

Budget-Constrained Bandits over General Cost and Reward Distributions

Authors: Semih Cayci, Atilla Eryilmaz, R. Srikant

Abstract: We consider a budget-constrained bandit problem where each arm pull incurs a random cost, and yields a random reward in return. The objective is to maximize the total expected reward under a budget constraint on the total cost. The model is general in the sense that it allows correlated and potentially heavy-tailed cost-reward pairs that can take on negative values as required by many applications… ▽ More We consider a budget-constrained bandit problem where each arm pull incurs a random cost, and yields a random reward in return. The objective is to maximize the total expected reward under a budget constraint on the total cost. The model is general in the sense that it allows correlated and potentially heavy-tailed cost-reward pairs that can take on negative values as required by many applications. We show that if moments of order $(2+γ)$ for some $γ> 0$ exist for all cost-reward pairs, $O(\log B)$ regret is achievable for a budget $B>0$. In order to achieve tight regret bounds, we propose algorithms that exploit the correlation between the cost and reward of each arm by extracting the common information via linear minimum mean-square error estimation. We prove a regret lower bound for this problem, and show that the proposed algorithms achieve tight problem-dependent regret bounds, which are optimal up to a universal constant factor in the case of jointly Gaussian cost and reward pairs. △ Less

Submitted 29 February, 2020; originally announced March 2020.

Predictive Scheduling for Virtual Reality

Authors: I-Hong Hou, Narges Zarnaghi Naghsh, Sibendu Paul, Y. Charlie Hu, Atilla Eryilmaz

Abstract: A significant challenge for future virtual reality (VR) applications is to deliver high quality-of-experience, both in terms of video quality and responsiveness, over wireless networks with limited bandwidth. This paper proposes to address this challenge by leveraging the predictability of user movements in the virtual world. We consider a wireless system where an access point (AP) serves multiple… ▽ More A significant challenge for future virtual reality (VR) applications is to deliver high quality-of-experience, both in terms of video quality and responsiveness, over wireless networks with limited bandwidth. This paper proposes to address this challenge by leveraging the predictability of user movements in the virtual world. We consider a wireless system where an access point (AP) serves multiple VR users. We show that the VR application process consists of two distinctive phases, whereby during the first (proactive scheduling) phase the controller has uncertain predictions of the demand that will arrive at the second (deadline scheduling) phase. We then develop a predictive scheduling policy for the AP that jointly optimizes the scheduling decisions in both phases. In addition to our theoretical study, we demonstrate the usefulness of our policy by building a prototype system. We show that our policy can be implemented under Furion, a Unity-based VR gaming software, with minor modifications. Experimental results clearly show visible difference between our policy and the default one. We also conduct extensive simulation studies, which show that our policy not only outperforms others, but also maintains excellent performance even when the prediction of future user movements is not accurate. △ Less

Submitted 29 December, 2019; originally announced December 2019.

Comments: Accepted for publication in Infocom 2020

On the Performance of Mobility-Aware D2D Caching Networks

Authors: Sameh Hosny, Atilla Eryilmaz, Alhussein A. Abouzeid, Hesham El Gamal

Abstract: The increase in demand for spectrum-based services forms a bottleneck in wireless networks. Device-to-Device (D2D) caching networks tackle this problem by exploiting user's behavior predictability and the possibility of sharing data between them to alleviate the network congestion. However, capturing mobility statistics allows Service Providers (SPs) to enhance their caching strategies. In this wo… ▽ More The increase in demand for spectrum-based services forms a bottleneck in wireless networks. Device-to-Device (D2D) caching networks tackle this problem by exploiting user's behavior predictability and the possibility of sharing data between them to alleviate the network congestion. However, capturing mobility statistics allows Service Providers (SPs) to enhance their caching strategies. In this work, we introduce a mobility-aware D2D caching network where SP harnesses user demand and mobility statistics to minimize the incurred service cost through an optimal caching policy. We investigate two caching schemes: centralized and decentralized caching schemes. In the centralized caching scheme, SP makes the caching decision towards its cost minimization to increase its profit. However, the complexity of optimal caching policy grows exponentially with the number of users. Therefore, we discuss a greedy caching algorithm which has a polynomial order complexity. We also use this greedy algorithm to establish upper and lower bounds on the proactive service gain achieved by the optimal caching policy. In the decentralized caching scheme, users take over and make their caching decisions, in a distributed fashion affected by the SP pricing policy, towards their payment minimization. We formulated the tension between the SP and users as a Stackelberg game. The Best response analysis was used to identify a sub-game perfect Nash equilibrium (SPNE) between users. The optimal solution of proposed model was found to depend on the SP reward preference, which affects the assigned memory in users' devices. We found some regimes for the reward value where the SPNE was non-unique. A fair allocation caching policy was adopted to choose one of these SPNEs. To understand the impact of user behavior, we investigated some special cases to explore how user's mobility statistics affect their caching decision. △ Less

Submitted 24 March, 2019; originally announced March 2019.

Optimal Learning for Dynamic Coding in Deadline-Constrained Multi-Channel Networks

Authors: Semih Cayci, Atilla Eryilmaz

Abstract: We study the problem of serving randomly arriving and delay-sensitive traffic over a multi-channel communication system with time-varying channel states and unknown statistics. This problem deviates from the classical exploration-exploitation setting in that the design and analysis must accommodate the dynamics of packet availability and urgency as well as the cost of each channel use at the time… ▽ More We study the problem of serving randomly arriving and delay-sensitive traffic over a multi-channel communication system with time-varying channel states and unknown statistics. This problem deviates from the classical exploration-exploitation setting in that the design and analysis must accommodate the dynamics of packet availability and urgency as well as the cost of each channel use at the time of decision. To that end, we have developed and investigated an index-based policy UCB-Deadline, which performs dynamic channel allocation decisions that incorporate these traffic requirements and costs. Under symmetric channel conditions, we have proved that the UCB-Deadline policy can achieve bounded regret in the likely case where the cost of using a channel is not too high to prevent all transmissions, and logarithmic regret otherwise. In this case, we show that UCB-Deadline is order-optimal. We also perform numerical investigations to validate the theoretical findings, and also compare the performance of the UCB-Deadline to another learning algorithm that we propose based on Thompson Sampling. △ Less

Submitted 27 November, 2018; originally announced November 2018.

Comments: Submitted to IEEE/ACM Transactions on Networking

Proactive Resource Allocation with Predictable Channel Statistics

Authors: L. Srikar Muppirisetty, John Tadrous, Atilla Eryilmaz, Henk Wymeersch

Abstract: The behavior of users in relatively predictable, both in terms of the data they request and the wireless channels they observe. In this paper, we consider the statistics of such predictable patterns of the demand and channel jointly across multiple users, and develop a novel predictive resource allocation method. This method is shown to provide performance benefits over a reactive approach, which… ▽ More The behavior of users in relatively predictable, both in terms of the data they request and the wireless channels they observe. In this paper, we consider the statistics of such predictable patterns of the demand and channel jointly across multiple users, and develop a novel predictive resource allocation method. This method is shown to provide performance benefits over a reactive approach, which ignores these patterns and instead aims to satisfy the instantaneous demands, irrespective of cost to the system. In particular, we show that our proposed method is able to attain a novel fundamental bound on the achievable cost, as the service window grows. Through numerical evaluation, we gain insights into how different uncertainty sources affect the decisions and the cost. △ Less

Submitted 13 June, 2018; originally announced June 2018.

Combinatorial Multi-Objective Multi-Armed Bandit Problem

Authors: Doruk Öner, Altuğ Karakurt, Atilla Eryılmaz, Cem Tekin

Abstract: In this paper, we introduce the COmbinatorial Multi-Objective Multi-Armed Bandit (COMO-MAB) problem that captures the challenges of combinatorial and multi-objective online learning simultaneously. In this setting, the goal of the learner is to choose an action at each time, whose reward vector is a linear combination of the reward vectors of the arms in the action, to learn the set of super Paret… ▽ More In this paper, we introduce the COmbinatorial Multi-Objective Multi-Armed Bandit (COMO-MAB) problem that captures the challenges of combinatorial and multi-objective online learning simultaneously. In this setting, the goal of the learner is to choose an action at each time, whose reward vector is a linear combination of the reward vectors of the arms in the action, to learn the set of super Pareto optimal actions, which includes the Pareto optimal actions and actions that become Pareto optimal after adding an arbitrary small positive number to their expected reward vectors. We define the Pareto regret performance metric and propose a fair learning algorithm whose Pareto regret is $O(N L^3 \log T)$, where $T$ is the time horizon, $N$ is the number of arms and $L$ is the maximum number of arms in an action. We show that COMO-MAB has a wide range of applications, including recommending bundles of items to users and network routing, and focus on a resource-allocation application for multi-user communication in the presence of multidimensional performance metrics, where we show that our algorithm outperforms existing MAB algorithms. △ Less

Submitted 11 March, 2018; originally announced March 2018.

Reward Maximization Under Uncertainty: Leveraging Side-Observations on Networks

Authors: Swapna Buccapatnam, Fang Liu, Atilla Eryilmaz, Ness B. Shroff

Abstract: We study the stochastic multi-armed bandit (MAB) problem in the presence of side-observations across actions that occur as a result of an underlying network structure. In our model, a bipartite graph captures the relationship between actions and a common set of unknowns such that choosing an action reveals observations for the unknowns that it is connected to. This models a common scenario in onli… ▽ More We study the stochastic multi-armed bandit (MAB) problem in the presence of side-observations across actions that occur as a result of an underlying network structure. In our model, a bipartite graph captures the relationship between actions and a common set of unknowns such that choosing an action reveals observations for the unknowns that it is connected to. This models a common scenario in online social networks where users respond to their friends' activity, thus providing side information about each other's preferences. Our contributions are as follows: 1) We derive an asymptotic lower bound (with respect to time) as a function of the bi-partite network structure on the regret of any uniformly good policy that achieves the maximum long-term average reward. 2) We propose two policies - a randomized policy; and a policy based on the well-known upper confidence bound (UCB) policies - both of which explore each action at a rate that is a function of its network position. We show, under mild assumptions, that these policies achieve the asymptotic lower bound on the regret up to a multiplicative factor, independent of the network structure. Finally, we use numerical examples on a real-world social network and a routing example network to demonstrate the benefits obtained by our policies over other existing policies. △ Less

Submitted 12 July, 2017; v1 submitted 25 April, 2017; originally announced April 2017.

Comments: minor revision

Towards A Marketplace for Mobile Content: Dynamic Pricing and Proactive Caching

Authors: F. Alotaibi, S. Hosny, J. Tadrous, H. El Gamal, A. Eryilmaz

Abstract: In this work, we investigate the profit maximization problem for a wireless network carrier and the payment minimization for end-users. Motivated by recent findings on proactive resource allocation, we focus on the scenario whereby end-users who are equipped with device-to-device (D2D)communication can harness predictable demand in proactive data contents caching and the possibility of trading the… ▽ More In this work, we investigate the profit maximization problem for a wireless network carrier and the payment minimization for end-users. Motivated by recent findings on proactive resource allocation, we focus on the scenario whereby end-users who are equipped with device-to-device (D2D)communication can harness predictable demand in proactive data contents caching and the possibility of trading their proactive downloads to minimize their expected payments. The carrier, on the other hand, utilizes a dynamic pricing scheme to differentiate between off-peak and peak time prices and applies commissions on each trading process to further maximize its profit. A novel marketplace that is based on risk sharing between end-users is proposed where the tension between carrier and end-users is formulated as a Stackelberg game. The existence and uniqueness of the non-cooperative sub-game Nash equilibrium is shown. Furthermore, we explore the equilibrium points for the case when the D2D is available and when it is not available, and study the impact of the uncertainty of users future demands on the system's performance. In particular, we compare the new equilibrium with the baseline scenario of flat pricing. Despite end-users connectivity with each other, the uncertainty of their future demands, and the freshness of the pre-cached contents, we characterize a new equilibrium region which yields to a win-win situation with respect to the baseline equilibrium. We show that end-users activity patterns can be harnessed to maximize the carrier's profit while minimizing the end-users expected payments. △ Less

Submitted 24 November, 2015; originally announced November 2015.

Comments: 31 pages

Throughput-Optimal Scheduling Design with Regular Service Guarantees in Wireless Networks

Authors: Bin Li, Ruogu Li, Atilla Eryilmaz

Abstract: Motivated by the regular service requirements of video applications for improving Quality-of-Experience (QoE) of users, we consider the design of scheduling strategies in multi-hop wireless networks that not only maximize system throughput but also provide regular inter-service times for all links. Since the service regularity of links is related to the higher-order statistics of the arrival proce… ▽ More Motivated by the regular service requirements of video applications for improving Quality-of-Experience (QoE) of users, we consider the design of scheduling strategies in multi-hop wireless networks that not only maximize system throughput but also provide regular inter-service times for all links. Since the service regularity of links is related to the higher-order statistics of the arrival process and the policy operation, it is highly challenging to characterize and analyze directly. We overcome this obstacle by introducing a new quantity, namely the time-since-last-service (TSLS), which tracks the time since the last service. By combining it with the queue-length in the weight, we propose a novel maximum-weight type scheduling policy, called Regular Service Guarantee (RSG) Algorithm. The unique evolution of the TSLS counter poses significant challenges for the analysis of the RSG Algorithm. To tackle these challenges, we first propose a novel Lyapunov function to show the throughput optimality of the RSG Algorithm. Then, we prove that the RSG Algorithm can provide service regularity guarantees by using the Lyapunov-drift based analysis of the steady-state behavior of the stochastic processes. In particular, our algorithm can achieve a degree of service regularity within a factor of a fundamental lower bound we derive. This factor is a function of the system statistics and design parameters and can be as low as two in some special networks. Our results, both analytical and numerical, exhibit significant service regularity improvements over the traditional throughput-optimal policies, which reveals the importance of incorporating the metric of time-since-last-service into the scheduling policy for providing regulated service. △ Less

Submitted 3 May, 2014; originally announced May 2014.

Comments: This work has been submitted to IEEE/ACM Transactions on Networking

Scheduling in Time-correlated Wireless Networks with Imperfect CSI and Stringent Constraint

Authors: Wenzhuo Ouyang, Atilla Eryilmaz, Ness B. Shroff

Abstract: In a wireless network, the efficiency of scheduling algorithms over time-varying channels depends heavily on the accuracy of the Channel State Information (CSI), which is usually quite ``costly'' in terms of consuming network resources. Scheduling in such systems is also subject to stringent constraints such as power and bandwidth, which limit the maximum number of simultaneous transmissions. In t… ▽ More In a wireless network, the efficiency of scheduling algorithms over time-varying channels depends heavily on the accuracy of the Channel State Information (CSI), which is usually quite ``costly'' in terms of consuming network resources. Scheduling in such systems is also subject to stringent constraints such as power and bandwidth, which limit the maximum number of simultaneous transmissions. In the meanwhile, communication channels in wireless systems typically fluctuate in a time-correlated manner. We hence design schedulers to exploit the temporal-correlation inherent in channels with memory and ARQ-styled feedback from the users for better channel state knowledge, under the assumption of Markovian channels and the stringent constraint on the maximum number of simultaneously active users. We model this problem under the framework of a Partially Observable Markov Decision Processes. In recent work, a low-complexity optimal solution was developed for this problem under a long-term time-average resource constraint. However, in real systems with instantaneous resource constraints, how to optimally exploit the temporal correlation and satisfy realistic stringent constraint on the instantaneous service remains elusive. In this work, we incorporate a stringent constraint on the simultaneously scheduled users and propose a low-complexity scheduling algorithm that dynamically implements user scheduling and dummy packet broadcasting. We show that the throughput region of the optimal policy under the long-term average resource constraint can be asymptotically achieved in the stringent constrained scenario by the proposed algorithm, in the many users limiting regime. △ Less

Submitted 30 March, 2014; originally announced March 2014.

Proactive Data Download and User Demand Sha** for Data Networks

Authors: John Tadrous, Atilla Eryilmaz, Hesham El Gamal

Abstract: In this work, we propose and study optimal proactive resource allocation and demand sha** for data networks. Motivated by the recent findings on the predictability of human behavior patterns in data networks, and the emergence of highly capable handheld devices, our design aims to smooth out the network traffic over time and minimize the data delivery costs. Our framework utilizes proactive da… ▽ More In this work, we propose and study optimal proactive resource allocation and demand sha** for data networks. Motivated by the recent findings on the predictability of human behavior patterns in data networks, and the emergence of highly capable handheld devices, our design aims to smooth out the network traffic over time and minimize the data delivery costs. Our framework utilizes proactive data services as well as smart content recommendation schemes for sha** the demand. Proactive data services take place during the off-peak hours based on a statistical prediction of a demand profile for each user, whereas smart content recommendation assigns modified valuations to data items so as to render the users' demand less uncertain. Hence, our recommendation scheme aims to boost the performance of proactive services within the allowed flexibility of user requirements. We conduct theoretical performance analysis that quantifies the leveraged cost reduction through the proposed framework. We show that the cost reduction scales at the same rate as the cost function scales with the number of users. Further, we prove that \emph{demand sha**} through smart recommendation strictly reduces the incurred cost even below that of proactive downloads without recommendation. △ Less

Submitted 28 December, 2014; v1 submitted 21 April, 2013; originally announced April 2013.

Low-complexity Optimal Scheduling over Correlated Fading Channels with ARQ Feedback

Authors: Wenzhuo Ouyang, Atilla Eryilmaz, Ness B. Shroff

Abstract: We investigate the downlink scheduling problem under Markovian ON/OFF fading channels, where the instantaneous channel state information is not directly accessible, but is revealed via ARQ-type feedback. The scheduler can exploit the temporal correlation/channel memory inherent in the Markovian channels to improve network performance. However, designing low-complexity and throughput-optimal algori… ▽ More We investigate the downlink scheduling problem under Markovian ON/OFF fading channels, where the instantaneous channel state information is not directly accessible, but is revealed via ARQ-type feedback. The scheduler can exploit the temporal correlation/channel memory inherent in the Markovian channels to improve network performance. However, designing low-complexity and throughput-optimal algorithms under temporal correlation is a challenging problem. In this paper, we find that under an average number of transmissions constraint, a low-complexity index policy is throughput-optimal. The policy uses Whittle's index value, which was previously used to capture opportunistic scheduling under temporally correlated channels. Our results build on the interesting finding that, under the intricate queue length and channel memory evolutions, the importance of scheduling a user is captured by a simple multiplication of its queue length and Whittle's index value. The proposed queue-based index policy has provably low complexity. Numerical results show that significant throughput gains can be realized by exploiting the channel memory using the proposed low-complexity policy. △ Less

Submitted 24 March, 2015; v1 submitted 7 June, 2012; originally announced June 2012.

Comments: A shorter version of this paper appeared in IEEE WiOpt 2012

A Fast-CSMA Algorithm for Deadline-Constrained Scheduling over Wireless Fading Channels

Authors: Bin Li, Atilla Eryilmaz

Abstract: Recently, low-complexity and distributed Carrier Sense Multiple Access (CSMA)-based scheduling algorithms have attracted extensive interest due to their throughput-optimal characteristics in general network topologies. However, these algorithms are not well-suited for serving real-time traffic under time-varying channel conditions for two reasons: (1) the mixing time of the underlying CSMA Markov… ▽ More Recently, low-complexity and distributed Carrier Sense Multiple Access (CSMA)-based scheduling algorithms have attracted extensive interest due to their throughput-optimal characteristics in general network topologies. However, these algorithms are not well-suited for serving real-time traffic under time-varying channel conditions for two reasons: (1) the mixing time of the underlying CSMA Markov Chain grows with the size of the network, which, for large networks, generates unacceptable delay for deadline-constrained traffic; (2) since the dynamic CSMA parameters are influenced by the arrival and channel state processes, the underlying CSMA Markov Chain may not converge to a steady-state under strict deadline constraints and fading channel conditions. In this paper, we attack the problem of distributed scheduling for serving real-time traffic over time-varying channels. Specifically, we consider fully-connected topologies with independently fading channels (which can model cellular networks) in which flows with short-term deadline constraints and long-term drop rate requirements are served. To that end, we first characterize the maximal set of satisfiable arrival processes for this system and, then, propose a Fast-CSMA (FCSMA) policy that is shown to be optimal in supporting any real-time traffic that is within the maximal satisfiable set. These theoretical results are further validated through simulations to demonstrate the relative efficiency of the FCSMA policy compared to some of the existing CSMA-based algorithms. △ Less

Submitted 13 March, 2012; originally announced March 2012.

Comments: This work appears in workshop on Resource Allocation and Cooperation in Wireless Networks (RAWNET), Princeton, NJ, May, 2011

Journal ref: The journal version of this paper is published in IEEE Transactions on Wireless Communications, 12(7): 3278-3288, 2013

Proactive Resource Allocation: Harnessing the Diversity and Multicast Gains

Authors: John Tadrous, Atilla Eryilmaz, Hesham El Gamal

Abstract: This paper introduces the novel concept of proactive resource allocation through which the predictability of user behavior is exploited to balance the wireless traffic over time, and hence, significantly reduce the bandwidth required to achieve a given blocking/outage probability. We start with a simple model in which the smart wireless devices are assumed to predict the arrival of new requests an… ▽ More This paper introduces the novel concept of proactive resource allocation through which the predictability of user behavior is exploited to balance the wireless traffic over time, and hence, significantly reduce the bandwidth required to achieve a given blocking/outage probability. We start with a simple model in which the smart wireless devices are assumed to predict the arrival of new requests and submit them to the network T time slots in advance. Using tools from large deviation theory, we quantify the resulting prediction diversity gain} to establish that the decay rate of the outage event probabilities increases with the prediction duration T. This model is then generalized to incorporate the effect of the randomness in the prediction look-ahead time T. Remarkably, we also show that, in the cognitive networking scenario, the appropriate use of proactive resource allocation by the primary users improves the diversity gain of the secondary network at no cost in the primary network diversity. We also shed lights on multicasting with predictable demands and show that the proactive multicast networks can achieve a significantly higher diversity gain that scales super-linearly with T. Finally, we conclude by a discussion of the new research questions posed under the umbrella of the proposed proactive (non-causal) wireless networking framework. △ Less

Submitted 21 October, 2011; originally announced October 2011.

Downlink Scheduling over Markovian Fading Channels

Authors: Wenzhuo Ouyang, Atilla Eryilmaz, Ness B. Shroff

Abstract: We consider the scheduling problem in downlink wireless networks with heterogeneous, Markov-modulated, ON/OFF channels. It is well-known that the performance of scheduling over fading channels relies heavily on the accuracy of the available Channel State Information (CSI), which is costly to acquire. Thus, we consider the CSI acquisition via a practical ARQ-based feedback mechanism whereby channel… ▽ More We consider the scheduling problem in downlink wireless networks with heterogeneous, Markov-modulated, ON/OFF channels. It is well-known that the performance of scheduling over fading channels relies heavily on the accuracy of the available Channel State Information (CSI), which is costly to acquire. Thus, we consider the CSI acquisition via a practical ARQ-based feedback mechanism whereby channel states are revealed at the end of only scheduled users' transmissions. In the assumed presence of temporally-correlated channel evolutions, the desired scheduler must optimally balance the exploitation-exploration trade-off, whereby it schedules transmissions both to exploit those channels with up-to-date CSI and to explore the current state of those with outdated CSI. In earlier works, Whittle's Index Policy had been suggested as a low-complexity and high-performance solution to this problem. However, analyzing its performance in the typical scenario of statistically heterogeneous channel state processes has remained elusive and challenging, mainly because of the highly-coupled and complex dynamics it possesses. In this work, we overcome these difficulties to rigorously establish the asymptotic optimality properties of Whittle's Index Policy in the limiting regime of many users. More specifically: (1) we prove the local optimality of Whittle's Index Policy, provided that the initial state of the system is within a certain neighborhood of a carefully selected state; (2) we then establish the global optimality of Whittle's Index Policy under a recurrence assumption that is verified numerically for our problem. These results establish that Whittle's Index Policy possesses analytically provable optimality characteristics for scheduling over heterogeneous and temporally-correlated channels. △ Less

Submitted 25 February, 2015; v1 submitted 18 August, 2011; originally announced August 2011.

Comments: A shorter version of this paper will appear in INFOCOM 2012, Orlando, FL

Asymptotically Tight Steady-State Queue Length Bounds Implied By Drift Conditions

Authors: Atilla Eryilmaz, R. Srikant

Abstract: The Foster-Lyapunov theorem and its variants serve as the primary tools for studying the stability of queueing systems. In addition, it is well known that setting the drift of the Lyapunov function equal to zero in steady-state provides bounds on the expected queue lengths. However, such bounds are often very loose due to the fact that they fail to capture resource pooling effects. The main contri… ▽ More The Foster-Lyapunov theorem and its variants serve as the primary tools for studying the stability of queueing systems. In addition, it is well known that setting the drift of the Lyapunov function equal to zero in steady-state provides bounds on the expected queue lengths. However, such bounds are often very loose due to the fact that they fail to capture resource pooling effects. The main contribution of this paper is to show that the approach of "setting the drift of a Lyapunov function equal to zero" can be used to obtain bounds on the steady-state queue lengths which are tight in the heavy-traffic limit. The key is to establish an appropriate notion of state-space collapse in terms of steady-state moments of weighted queue length differences, and use this state-space collapse result when setting the Lyapunov drift equal to zero. As an application of the methodology, we prove the steady-state equivalent of the heavy-traffic optimality result of Stolyar for wireless networks operating under the MaxWeight scheduling policy. △ Less

Submitted 27 June, 2013; v1 submitted 2 April, 2011; originally announced April 2011.

Throughput-Delay Analysis of Random Linear Network Coding for Wireless Broadcasting

Authors: B. T. Swapna, Atilla Eryilmaz, Ness B. Shroff

Abstract: In an unreliable single-hop broadcast network setting, we investigate the throughput and decoding-delay performance of random linear network coding as a function of the coding window size and the network size. Our model consists of a source transmitting packets of a single flow to a set of $n$ users over independent erasure channels. The source performs random linear network coding (RLNC) over… ▽ More In an unreliable single-hop broadcast network setting, we investigate the throughput and decoding-delay performance of random linear network coding as a function of the coding window size and the network size. Our model consists of a source transmitting packets of a single flow to a set of $n$ users over independent erasure channels. The source performs random linear network coding (RLNC) over $k$ (coding window size) packets and broadcasts them to the users. We note that the broadcast throughput of RLNC must vanish with increasing $n$, for any fixed $k.$ Hence, in contrast to other works in the literature, we investigate how the coding window size $k$ must scale for increasing $n$. Our analysis reveals that the coding window size of $Θ(\ln(n))$ represents a phase transition rate, below which the throughput converges to zero, and above which it converges to the broadcast capacity. Further, we characterize the asymptotic distribution of decoding delay and provide approximate expressions for the mean and variance of decoding delay for the scaling regime of $k=ω(\ln(n)).$ These asymptotic expressions reveal the impact of channel correlations on the throughput and delay performance of RLNC. We also show how our analysis can be extended to other rateless block coding schemes such as the LT codes. Finally, we comment on the extension of our results to the cases of dependent channels across users and asymmetric channel model. △ Less

Submitted 19 January, 2011; v1 submitted 26 December, 2010; originally announced January 2011.

Delay-Aware Cross-Layer Design for Network Utility Maximization in Multi-hop Networks

Authors: Haozhi Xiong, Ruogu Li, Atilla Eryilmaz, Eylem Ekici

Abstract: We investigate the problem of designing delay-aware joint flow control, routing, and scheduling algorithms in general multi-hop networks for maximizing network utilization. Since the end-to-end delay performance has a complex dependence on the high-order statistics of cross-layer algorithms, earlier optimization-based design methodologies that optimize the long term network utilization are not imm… ▽ More We investigate the problem of designing delay-aware joint flow control, routing, and scheduling algorithms in general multi-hop networks for maximizing network utilization. Since the end-to-end delay performance has a complex dependence on the high-order statistics of cross-layer algorithms, earlier optimization-based design methodologies that optimize the long term network utilization are not immediately well-suited for delay-aware design. This motivates us in this work to develop a novel design framework and alternative methods that take advantage of several unexploited design choices in the routing and the scheduling strategy spaces. In particular, we reveal and exploit a crucial characteristic of back pressure-type controllers that enables us to develop a novel link rate allocation strategy that not only optimizes long-term network utilization, but also yields loop free multi-path routes} between each source-destination pair. Moreover, we propose a regulated scheduling strategy, based on a token-based service discipline, for sha** the per-hop delay distribution to obtain highly desirable end-to-end delay performance. We establish that our joint flow control, routing, and scheduling algorithm achieves loop-free routes and optimal network utilization. Our extensive numerical studies support our theoretical results, and further show that our joint design leads to substantial end-to-end delay performance improvements in multi-hop networks compared to earlier solutions. △ Less

Submitted 7 December, 2010; originally announced December 2010.

Comments: 14 pages, JSAC

Proactive Resource Allocation: Turning Predictable Behavior into Spectral Gain

Authors: Hesham El-Gamal, John Tadrous, Atilla Eryilmaz

Abstract: This paper introduces the novel concept of proactive resource allocation in which the predictability of user behavior is exploited to balance the wireless traffic over time, and hence, significantly reduce the bandwidth required to achieve a given blocking/outage probability. We start with a simple model in which the smart wireless devices are assumed to predict the arrival of new requests and sub… ▽ More This paper introduces the novel concept of proactive resource allocation in which the predictability of user behavior is exploited to balance the wireless traffic over time, and hence, significantly reduce the bandwidth required to achieve a given blocking/outage probability. We start with a simple model in which the smart wireless devices are assumed to predict the arrival of new requests and submit them to the network T time slots in advance. Using tools from large deviation theory, we quantify the resulting prediction diversity gain to establish that the decay rate of the outage event probabilities increases linearly with the prediction duration T. This model is then generalized to incorporate the effect of prediction errors and the randomness in the prediction lookahead time T. Remarkably, we also show that, in the cognitive networking scenario, the appropriate use of proactive resource allocation by the primary users results in more spectral opportunities for the secondary users at a marginal, or no, cost in the primary network outage. Finally, we conclude by a discussion of the new research questions posed under the umbrella of the proposed proactive (non-causal) wireless networking framework. △ Less

Submitted 8 October, 2010; originally announced October 2010.

Exploiting Channel Memory for Joint Estimation and Scheduling in Downlink Networks

Authors: Wenzhuo Ouyang, Sugumar Murugesan, Atilla Eryilmaz, Ness B. Shroff

Abstract: We address the problem of opportunistic multiuser scheduling in downlink networks with Markov-modeled outage channels. We consider the scenario in which the scheduler does not have full knowledge of the channel state information, but instead estimates the channel state information by exploiting the memory inherent in the Markov channels along with ARQ-styled feedback from the scheduled users. Oppo… ▽ More We address the problem of opportunistic multiuser scheduling in downlink networks with Markov-modeled outage channels. We consider the scenario in which the scheduler does not have full knowledge of the channel state information, but instead estimates the channel state information by exploiting the memory inherent in the Markov channels along with ARQ-styled feedback from the scheduled users. Opportunistic scheduling is optimized in two stages: (1) Channel estimation and rate adaptation to maximize the expected immediate rate of the scheduled user; (2) User scheduling, based on the optimized immediate rate, to maximize the overall long term sum-throughput of the downlink. The scheduling problem is a partially observable Markov decision process with the classic 'exploitation vs exploration' trade-off that is difficult to quantify. We therefore study the problem in the framework of Restless Multi-armed Bandit Processes (RMBP) and perform a Whittle's indexability analysis. Whittle's indexability is traditionally known to be hard to establish and the index policy derived based on Whittle's indexability is known to have optimality properties in various settings. We show that the problem of downlink scheduling under imperfect channel state information is Whittle indexable and derive the Whittle's index policy in closed form. Via extensive numerical experiments, we show that the index policy has near-optimal performance. Our work reveals that, under incomplete channel state information, exploiting channel memory for opportunistic scheduling can result in significant performance gains and that almost all of these gains can be realized using an easy-to-implement index policy. △ Less

Submitted 6 December, 2011; v1 submitted 20 September, 2010; originally announced September 2010.

Comments: A shorter version of this report appeared in INFOCOM 2011, Shanghai, China

Scheduling with Rate Adaptation under Incomplete Knowledge of Channel/Estimator Statistics

Authors: Wenzhuo Ouyang, Sugumar Murugesan, Atilla Eryilmaz, Ness B. Shroff

Abstract: In time-varying wireless networks, the states of the communication channels are subject to random variations, and hence need to be estimated for efficient rate adaptation and scheduling. The estimation mechanism possesses inaccuracies that need to be tackled in a probabilistic framework. In this work, we study scheduling with rate adaptation in single-hop queueing networks under two levels of chan… ▽ More In time-varying wireless networks, the states of the communication channels are subject to random variations, and hence need to be estimated for efficient rate adaptation and scheduling. The estimation mechanism possesses inaccuracies that need to be tackled in a probabilistic framework. In this work, we study scheduling with rate adaptation in single-hop queueing networks under two levels of channel uncertainty: when the channel estimates are inaccurate but complete knowledge of the channel/estimator joint statistics is available at the scheduler; and when the knowledge of the joint statistics is incomplete. In the former case, we characterize the network stability region and show that a maximum-weight type scheduling policy is throughput-optimal. In the latter case, we propose a joint channel statistics learning - scheduling policy. With an associated trade-off in average packet delay and convergence time, the proposed policy has a stability region arbitrarily close to the stability region of the network under full knowledge of channel/estimator joint statistics. △ Less

Submitted 3 October, 2010; v1 submitted 10 August, 2010; originally announced August 2010.

Comments: 48th Allerton Conference on Communication, Control, and Computing, Monticello, IL, Sept. 2010

Network Coding in a Multicast Switch

Authors: MinJi Kim, Jay Kumar Sundararajan, Muriel Medard, Atilla Eryilmaz, Ralf Koetter

Abstract: The problem of serving multicast flows in a crossbar switch is considered. Intra-flow linear network coding is shown to achieve a larger rate region than the case without coding. A traffic pattern is presented which is achievable with coding but requires a switch speedup when coding is not allowed. The rate region with coding can be characterized in a simple graph-theoretic manner, in terms of t… ▽ More The problem of serving multicast flows in a crossbar switch is considered. Intra-flow linear network coding is shown to achieve a larger rate region than the case without coding. A traffic pattern is presented which is achievable with coding but requires a switch speedup when coding is not allowed. The rate region with coding can be characterized in a simple graph-theoretic manner, in terms of the stable set polytope of the "enhanced conflict graph". No such graph-theoretic characterization is known for the case of fanout splitting without coding. The minimum speedup needed to achieve 100% throughput with coding is shown to be upper bounded by the imperfection ratio of the enhanced conflict graph. When applied to KxN switches with unicasts and broadcasts only, this gives a bound of min{(2K-1)/K,2N/(N+1)} on the speedup. This shows that speedup, which is usually implemented in hardware, can often be substituted by network coding, which can be done in software. Computing an offline schedule (using prior knowledge of the flow rates) is reduced to fractional weighted graph coloring. A graph-theoretic online scheduling algorithm (using only queue occupancy information) is also proposed, that stabilizes the queues for all rates within the rate region. △ Less

Submitted 9 October, 2008; originally announced October 2008.

Comments: 26 pages, 27 figures, submitted to IEEE Transactions on Information Theory

Information Theory vs. Queueing Theory for Resource Allocation in Multiple Access Channels

Authors: Ali ParandehGheibi, Muriel Medard, Asuman Ozdaglar, Atilla Eryilmaz

Abstract: We consider the problem of rate allocation in a fading Gaussian multiple-access channel with fixed transmission powers. The goal is to maximize a general concave utility function of the expected achieved rates of the users. There are different approaches to this problem in the literature. From an information theoretic point of view, rates are allocated only by using the channel state information… ▽ More We consider the problem of rate allocation in a fading Gaussian multiple-access channel with fixed transmission powers. The goal is to maximize a general concave utility function of the expected achieved rates of the users. There are different approaches to this problem in the literature. From an information theoretic point of view, rates are allocated only by using the channel state information. The queueing theory approach utilizes the global queue-length information for rate allocation to guarantee throughput optimality as well as maximizing a utility function of the rates. In this work, we make a connection between these two approaches by showing that the information theoretic capacity region of a multiple-access channel and its stability region are equivalent. Moreover, our numerical results show that a simple greedy policy which does not use the queue-length information can outperform queue-length based policies in terms of convergence rate and fairness. △ Less

Submitted 7 October, 2008; originally announced October 2008.

Comments: 5 pages. In proc. of PIMRC 2008

Rate and Power Allocation in Fading Multiple Access Channels

Authors: Ali ParandehGheibi, Atilla Eryilmaz, Asuman Ozdaglar, Muriel Medard

Abstract: We consider the problem of rate and power allocation in a fading multiple-access channel. Our objective is to obtain rate and power allocation policies that maximize a utility function defined over average transmission rates. In contrast with the literature, which focuses on the linear case, we present results for general concave utility functions. We consider two cases. In the first case, we as… ▽ More We consider the problem of rate and power allocation in a fading multiple-access channel. Our objective is to obtain rate and power allocation policies that maximize a utility function defined over average transmission rates. In contrast with the literature, which focuses on the linear case, we present results for general concave utility functions. We consider two cases. In the first case, we assume that power control is possible and channel statistics are known. In this case, we show that the optimal policies can be obtained greedily by maximizing a linear utility function at each channel state. In the second case, we assume that power control is not possible and channel statistics are not available. In this case, we define a greedy rate allocation policy and provide upper bounds on the performance difference between the optimal and the greedy policy. Our bounds highlight the dependence of the performance difference on the channel variations and the structure of the utility function. △ Less

Submitted 7 October, 2008; originally announced October 2008.

Comments: 6 pages, In proc. of WiOpt 2008

Dynamic Rate Allocation in Fading Multiple-access Channels

Authors: Ali ParandehGheibi, Atilla Eryilmaz, Asuman Ozdaglar, Muriel Medard

Abstract: We consider the problem of rate allocation in a fading Gaussian multiple-access channel (MAC) with fixed transmission powers. Our goal is to maximize a general concave utility function of transmission rates over the throughput capacity region. In contrast to earlier works in this context that propose solutions where a potentially complex optimization problem must be solved in every decision inst… ▽ More We consider the problem of rate allocation in a fading Gaussian multiple-access channel (MAC) with fixed transmission powers. Our goal is to maximize a general concave utility function of transmission rates over the throughput capacity region. In contrast to earlier works in this context that propose solutions where a potentially complex optimization problem must be solved in every decision instant, we propose a low-complexity approximate rate allocation policy and analyze the effect of temporal channel variations on its utility performance. To the best of our knowledge, this is the first work that studies the tracking capabilities of an approximate rate allocation scheme under fading channel conditions. We build on an earlier work to present a new rate allocation policy for a fading MAC that implements a low-complexity approximate gradient projection iteration for each channel measurement, and explicitly characterize the effect of the speed of temporal channel variations on the tracking neighborhood of our policy. We further improve our results by proposing an alternative rate allocation policy for which tighter bounds on the size of the tracking neighborhood are derived. These proposed rate allocation policies are computationally efficient in our setting since they implement a single gradient projection iteration per channel measurement and each such iteration relies on approximate projections which has polynomial-complexity in the number of users. △ Less

Submitted 7 October, 2008; originally announced October 2008.

Comments: 9 pages, In proc. of ITA 2008

Resource Allocation in Multiple Access Channels

Authors: Ali ParandehGheibi, Atilla Eryilmaz, Asuman Ozdaglar, Muriel Medard

Abstract: We consider the problem of rate allocation in a Gaussian multiple-access channel, with the goal of maximizing a utility function over transmission rates. In contrast to the literature which focuses on linear utility functions, we study general concave utility functions. We present a gradient projection algorithm for this problem. Since the constraint set of the problem is described by exponentia… ▽ More We consider the problem of rate allocation in a Gaussian multiple-access channel, with the goal of maximizing a utility function over transmission rates. In contrast to the literature which focuses on linear utility functions, we study general concave utility functions. We present a gradient projection algorithm for this problem. Since the constraint set of the problem is described by exponentially many constraints, methods that use exact projections are computationally intractable. Therefore, we develop a new method that uses approximate projections. We use the polymatroid structure of the capacity region to show that the approximate projection can be implemented by a recursive algorithm in time polynomial in the number of users. We further propose another algorithm for implementing the approximate projections using rate-splitting and show improved bounds on its convergence time. △ Less

Submitted 7 October, 2008; originally announced October 2008.

Comments: 5 pages, In proc. of ACSSC 2007

On Resource Allocation in Fading Multiple Access Channels - An Efficient Approximate Projection Approach

Authors: Ali ParandehGheibi, Atilla Eryilmaz, Asuman Ozdaglar, Muriel Medard

Abstract: We consider the problem of rate and power allocation in a multiple-access channel. Our objective is to obtain rate and power allocation policies that maximize a general concave utility function of average transmission rates on the information theoretic capacity region of the multiple-access channel. Our policies does not require queue-length information. We consider several different scenarios.… ▽ More We consider the problem of rate and power allocation in a multiple-access channel. Our objective is to obtain rate and power allocation policies that maximize a general concave utility function of average transmission rates on the information theoretic capacity region of the multiple-access channel. Our policies does not require queue-length information. We consider several different scenarios. First, we address the utility maximization problem in a nonfading channel to obtain the optimal operating rates, and present an iterative gradient projection algorithm that uses approximate projection. By exploiting the polymatroid structure of the capacity region, we show that the approximate projection can be implemented in time polynomial in the number of users. Second, we consider resource allocation in a fading channel. Optimal rate and power allocation policies are presented for the case that power control is possible and channel statistics are available. For the case that transmission power is fixed and channel statistics are unknown, we propose a greedy rate allocation policy and provide bounds on the performance difference of this policy and the optimal policy in terms of channel variations and structure of the utility function. We present numerical results that demonstrate superior convergence rate performance for the greedy policy compared to queue-length based policies. In order to reduce the computational complexity of the greedy policy, we present approximate rate allocation policies which track the greedy policy within a certain neighborhood that is characterized in terms of the speed of fading. △ Less

Submitted 7 October, 2008; originally announced October 2008.

Comments: 32 pages, Submitted to IEEE Trans. on Information Theory

Report number: LIDS report 2787

Network Coding in a Multicast Switch

Authors: Jay Kumar Sundararajan, Muriel Medard, MinJi Kim, Atilla Eryilmaz, Devavrat Shah, Ralf Koetter

Abstract: We consider the problem of serving multicast flows in a crossbar switch. We show that linear network coding across packets of a flow can sustain traffic patterns that cannot be served if network coding were not allowed. Thus, network coding leads to a larger rate region in a multicast crossbar switch. We demonstrate a traffic pattern which requires a switch speedup if coding is not allowed, wher… ▽ More We consider the problem of serving multicast flows in a crossbar switch. We show that linear network coding across packets of a flow can sustain traffic patterns that cannot be served if network coding were not allowed. Thus, network coding leads to a larger rate region in a multicast crossbar switch. We demonstrate a traffic pattern which requires a switch speedup if coding is not allowed, whereas, with coding the speedup requirement is eliminated completely. In addition to throughput benefits, coding simplifies the characterization of the rate region. We give a graph-theoretic characterization of the rate region with fanout splitting and intra-flow coding, in terms of the stable set polytope of the 'enhanced conflict graph' of the traffic pattern. Such a formulation is not known in the case of fanout splitting without coding. We show that computing the offline schedule (i.e. using prior knowledge of the flow arrival rates) can be reduced to certain graph coloring problems. Finally, we propose online algorithms (i.e. using only the current queue occupancy information) for multicast scheduling based on our graph-theoretic formulation. In particular, we show that a maximum weighted stable set algorithm stabilizes the queues for all rates within the rate region. △ Less

Submitted 8 August, 2006; originally announced August 2006.

Comments: 9 pages, submitted to IEEE INFOCOM 2007