-
Exponentially Weighted Algorithm for Online Network Resource Allocation with Long-Term Constraints
Authors:
Ahmed Sid-Ali,
Ioannis Lambadaris,
Yiqiang Q. Zhao,
Gennady Shaikhet,
Amirhossein Asgharnia
Abstract:
This paper studies an online optimal resource reservation problem in communication networks with job transfers where the goal is to minimize the reservation cost while maintaining the blocking cost under a certain budget limit. To tackle this problem, we propose a novel algorithm based on a randomized exponentially weighted method that encompasses long-term constraints. We then analyze the perform…
▽ More
This paper studies an online optimal resource reservation problem in communication networks with job transfers where the goal is to minimize the reservation cost while maintaining the blocking cost under a certain budget limit. To tackle this problem, we propose a novel algorithm based on a randomized exponentially weighted method that encompasses long-term constraints. We then analyze the performance of our algorithm by establishing an upper bound for the associated regret and the cumulative constraint violations. Finally, we present numerical experiments where we compare the performance of our algorithm with those of reinforcement learning where we show that our algorithm surpasses it.
△ Less
Submitted 3 May, 2024;
originally announced May 2024.
-
Online Optimization for Network Resource Allocation and Comparison with Reinforcement Learning Techniques
Authors:
Ahmed Sid-Ali,
Ioannis Lambadaris,
Yiqiang Q. Zhao,
Gennady Shaikhet,
Amirhossein Asgharnia
Abstract:
We tackle in this paper an online network resource allocation problem with job transfers. The network is composed of many servers connected by communication links. The system operates in discrete time; at each time slot, the administrator reserves resources at servers for future job requests, and a cost is incurred for the reservations made. Then, after receptions, the jobs may be transferred betw…
▽ More
We tackle in this paper an online network resource allocation problem with job transfers. The network is composed of many servers connected by communication links. The system operates in discrete time; at each time slot, the administrator reserves resources at servers for future job requests, and a cost is incurred for the reservations made. Then, after receptions, the jobs may be transferred between the servers to best accommodate the demands. This incurs an additional transport cost. Finally, if a job request cannot be satisfied, there is a violation that engenders a cost to pay for the blocked job. We propose a randomized online algorithm based on the exponentially weighted method. We prove that our algorithm enjoys a sub-linear in time regret, which indicates that the algorithm is adapting and learning from its experiences and is becoming more efficient in its decision-making as it accumulates more data. Moreover, we test the performance of our algorithm on artificial data and compare it against a reinforcement learning method where we show that our proposed method outperforms the latter.
△ Less
Submitted 16 November, 2023;
originally announced November 2023.
-
Online Optimization for Randomized Network Resource Allocation with Long-Term Constraints
Authors:
Ahmed Sid-Ali,
Ioannis Lambadaris,
Yiqiang Q. Zhao,
Gennady Shaikhet,
Shima Kheradmand
Abstract:
In this paper, we study an optimal online resource reservation problem in a simple communication network. The network is composed of two compute nodes linked by a local communication link. The system operates in discrete time; at each time slot, the administrator reserves resources for servers before the actual job requests are known. A cost is incurred for the reservations made. Then, after the c…
▽ More
In this paper, we study an optimal online resource reservation problem in a simple communication network. The network is composed of two compute nodes linked by a local communication link. The system operates in discrete time; at each time slot, the administrator reserves resources for servers before the actual job requests are known. A cost is incurred for the reservations made. Then, after the client requests are observed, jobs may be transferred from one server to the other to best accommodate the demands by incurring an additional transport cost. If certain job requests cannot be satisfied, there is a violation that engenders a cost to pay for each of the blocked jobs. The goal is to minimize the overall reservation cost over finite horizons while maintaining the cumulative violation and transport costs under a certain budget limit. To study this problem, we first formalize it as a repeated game against nature where the reservations are drawn randomly according to a sequence of probability distributions that are derived from an online optimization problem over the space of allowable reservations. We then propose an online saddle-point algorithm for which we present an upper bound for the associated K-benchmark regret together with an upper bound for the cumulative constraint violations. Finally, we present numerical experiments where we compare the performance of our algorithm with those of simple deterministic resource allocation policies.
△ Less
Submitted 3 April, 2024; v1 submitted 24 May, 2023;
originally announced May 2023.
-
Necessary condition for null controllability in many-server heavy traffic
Authors:
Gennady Shaikhet
Abstract:
Throughput sub-optimality (TSO), introduced in Atar and Shaikhet [Ann. Appl. Probab. 19 (2009) 521-555] for static fluid models of parallel queueing networks, corresponds to the existence of a resource allocation, under which the total service rate becomes greater than the total arrival rate. As shown in Atar, Mandelbaum and Shaikhet [Ann. Appl. Probab. 16 (2006) 1764-1804] and Atar and Shaikhet (…
▽ More
Throughput sub-optimality (TSO), introduced in Atar and Shaikhet [Ann. Appl. Probab. 19 (2009) 521-555] for static fluid models of parallel queueing networks, corresponds to the existence of a resource allocation, under which the total service rate becomes greater than the total arrival rate. As shown in Atar, Mandelbaum and Shaikhet [Ann. Appl. Probab. 16 (2006) 1764-1804] and Atar and Shaikhet (2009), in the many server Halfin-Whitt regime, TSO implies null controllability (NC), the existence of a routing policy under which, for every finite $T$, the measure of the set of times prior to $T$, at which at least one customer is in the buffer, converges to zero in probability at the scaling limit. The present paper investigates the question whether the converse relation is also true and TSO is both sufficient and necessary for the NC behavior. In what follows we do get the affirmation for systems with either two customer classes (and possibly more service pools) or vice-versa and state a condition on the underlying static fluid model that allows the extension of the result to general structures.
△ Less
Submitted 26 January, 2015;
originally announced January 2015.
-
Power Strip Packing of Malleable Demands in Smart Grid
Authors:
Mohammad M. Karbasioun,
Gennady Shaikhet,
Evangelos Kranakis,
Ioannis Lambadaris
Abstract:
We consider a problem of supplying electricity to a set of $\mathcal{N}$ customers in a smart-grid framework. Each customer requires a certain amount of electrical energy which has to be supplied during the time interval $[0,1]$. We assume that each demand has to be supplied without interruption, with possible duration between $\ell$ and $r$, which are given system parameters ($\ell\le r$). At eac…
▽ More
We consider a problem of supplying electricity to a set of $\mathcal{N}$ customers in a smart-grid framework. Each customer requires a certain amount of electrical energy which has to be supplied during the time interval $[0,1]$. We assume that each demand has to be supplied without interruption, with possible duration between $\ell$ and $r$, which are given system parameters ($\ell\le r$). At each moment of time, the power of the grid is the sum of all the consumption rates for the demands being supplied at that moment. Our goal is to find an assignment that minimizes the {\it power peak} - maximal power over $[0,1]$ - while satisfying all the demands. To do this first we find the lower bound of optimal power peak. We show that the problem depends on whether or not the pair $\ell, r$ belongs to a "good" region $\mathcal{G}$. If it does - then an optimal assignment almost perfectly "fills" the rectangle $time \times power = [0,1] \times [0, A]$ with $A$ being the sum of all the energy demands - thus achieving an optimal power peak $A$. Conversely, if $\ell, r$ do not belong to $\mathcal{G}$, we identify the lower bound $\bar{A} >A$ on the optimal value of power peak and introduce a simple linear time algorithm that almost perfectly arranges all the demands in a rectangle $[0, A /\bar{A}] \times [0, \bar{A}]$ and show that it is asymptotically optimal.
△ Less
Submitted 15 February, 2013;
originally announced February 2013.
-
Critically loaded queueing models that are throughput suboptimal
Authors:
Rami Atar,
Gennady Shaikhet
Abstract:
This paper introduces and analyzes the notion of throughput suboptimality for many-server queueing systems in heavy traffic. The queueing model under consideration has multiple customer classes, indexed by a finite set $\mathcal{I}$, and heterogenous, exponential servers. Servers are dynamically chosen to serve customers, and buffers are available for customers waiting to be served. The arrival…
▽ More
This paper introduces and analyzes the notion of throughput suboptimality for many-server queueing systems in heavy traffic. The queueing model under consideration has multiple customer classes, indexed by a finite set $\mathcal{I}$, and heterogenous, exponential servers. Servers are dynamically chosen to serve customers, and buffers are available for customers waiting to be served. The arrival rates and the number of servers are scaled up in such a way that the processes representing the number of class-$i$ customers in the system, $i\in\mathcal{I}$, fluctuate about a static fluid model, that is assumed to be critically loaded in a standard sense. At the same time, the fluid model is assumed to be throughput suboptimal. Roughly, this means that the servers can be allocated so as to achieve a total processing rate that is greater than the total arrival rate. We show that there exists a dynamic control policy for the queueing model that is efficient in the following strong sense: Under this policy, for every finite $T$, the measure of the set of times prior to $T$, at which at least one customer is in the buffer, converges to zero in probability as the arrival rates and number of servers go to infinity. On the way to prove our main result, we provide a characterization of throughput suboptimality in terms of properties of the buffer-station graph.
△ Less
Submitted 12 June, 2009;
originally announced June 2009.
-
Queueing systems with many servers: Null controllability in heavy traffic
Authors:
Rami Atar,
Avi Mandelbaum,
Gennady Shaikhet
Abstract:
A queueing model has $J\ge2$ heterogeneous service stations, each consisting of many independent servers with identical capabilities. Customers of $I\ge2$ classes can be served at these stations at different rates, that depend on both the class and the station. A system administrator dynamically controls scheduling and routing. We study this model in the central limit theorem (or heavy traffic)…
▽ More
A queueing model has $J\ge2$ heterogeneous service stations, each consisting of many independent servers with identical capabilities. Customers of $I\ge2$ classes can be served at these stations at different rates, that depend on both the class and the station. A system administrator dynamically controls scheduling and routing. We study this model in the central limit theorem (or heavy traffic) regime proposed by Halfin and Whitt. We derive a diffusion model on $\mathbb {R}^I$ with a singular control term that describes the scaling limit of the queueing model. The singular term may be used to constrain the diffusion to lie in certain subsets of $\mathbb {R}^I$ at all times $t>0$. We say that the diffusion is null-controllable if it can be constrained to $\mathbb {X}_-$, the minimal closed subset of $\mathbb {R}^I$ containing all states of the prelimit queueing model for which all queues are empty. We give sufficient conditions for null controllability of the diffusion. Under these conditions we also show that an analogous, asymptotic result holds for the queueing model, by constructing control policies under which, for any given $0<ε<T<\infty$, all queues in the system are kept empty on the time interval $[ε, T]$, with probability approaching one. This introduces a new, unusual heavy traffic ``behavior'': On one hand, the system is critically loaded, in the sense that an increase in any of the external arrival rates at the ``fluid level'' results with an overloaded system. On the other hand, as far as queue lengths are concerned, the system behaves as if it is underloaded.
△ Less
Submitted 9 February, 2007; v1 submitted 17 July, 2005;
originally announced July 2005.
