Showing 1–50 of 88 results for author: Modiano, E

Power Failure Cascade Prediction using Graph Neural Networks

Authors: Sathwik Chadaga, Xinyu Wu, Eytan Modiano

Abstract: We consider the problem of predicting power failure cascades due to branch failures. We propose a flow-free model based on graph neural networks that predicts grid states at every generation of a cascade process given an initial contingency and power injection values. We train the proposed model using a cascade sequence data pool generated from simulations. We then evaluate our model at various le… ▽ More We consider the problem of predicting power failure cascades due to branch failures. We propose a flow-free model based on graph neural networks that predicts grid states at every generation of a cascade process given an initial contingency and power injection values. We train the proposed model using a cascade sequence data pool generated from simulations. We then evaluate our model at various levels of granularity. We present several error metrics that gauge the model's ability to predict the failure size, the final grid state, and the failure time steps of each branch within the cascade. We benchmark the graph neural network model against influence models. We show that, in addition to being generic over randomly scaled power injection values, the graph neural network model outperforms multiple influence models that are built specifically for their corresponding loading profiles. Finally, we show that the proposed model reduces the computational time by almost two orders of magnitude. △ Less

Submitted 24 April, 2024; originally announced April 2024.

Comments: 2023 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm). Oct. 31, 2023. See implementations at https://github.com/sathwikchadaga/failure-cascade

Optimal Slicing and Scheduling with Service Guarantees in Multi-Hop Wireless Networks

Authors: Nicholas Jones, Eytan Modiano

Abstract: We analyze the problem of scheduling in wireless networks to meet end-to-end service guarantees. Using network slicing to decouple the queueing dynamics between flows, we show that the network's ability to meet hard throughput and deadline requirements is largely influenced by the scheduling policy. We characterize the feasible throughput/deadline region for a flow under a fixed route and set of s… ▽ More We analyze the problem of scheduling in wireless networks to meet end-to-end service guarantees. Using network slicing to decouple the queueing dynamics between flows, we show that the network's ability to meet hard throughput and deadline requirements is largely influenced by the scheduling policy. We characterize the feasible throughput/deadline region for a flow under a fixed route and set of slices, and find throughput- and deadline-optimal policies for a solitary flow. We formulate the feasibility problem for multiple flows in a general topology as a mixed-integer program, and show that it grows exponentially in the size of the network. Drawing on results from the solitary flow setting, we show that scheduling links in a regular fashion leads to smaller delay, and we derive tighter upper bounds on end-to-end delay for regular schedules. Finally, we design a polynomial-time algorithm that returns an (almost) regular schedule, optimized to meet service guarantees for all flows. △ Less

Submitted 15 April, 2024; v1 submitted 12 April, 2024; originally announced April 2024.

Intervention-Assisted Policy Gradient Methods for Online Stochastic Queuing Network Optimization: Technical Report

Authors: Jerrod Wigmore, Brooke Shrader, Eytan Modiano

Abstract: Deep Reinforcement Learning (DRL) offers a powerful approach to training neural network control policies for stochastic queuing networks (SQN). However, traditional DRL methods rely on offline simulations or static datasets, limiting their real-world application in SQN control. This work proposes Online Deep Reinforcement Learning-based Controls (ODRLC) as an alternative, where an intelligent agen… ▽ More Deep Reinforcement Learning (DRL) offers a powerful approach to training neural network control policies for stochastic queuing networks (SQN). However, traditional DRL methods rely on offline simulations or static datasets, limiting their real-world application in SQN control. This work proposes Online Deep Reinforcement Learning-based Controls (ODRLC) as an alternative, where an intelligent agent interacts directly with a real environment and learns an optimal control policy from these online interactions. SQNs present a challenge for ODRLC due to the unbounded nature of the queues within the network resulting in an unbounded state-space. An unbounded state-space is particularly challenging for neural network policies as neural networks are notoriously poor at extrapolating to unseen states. To address this challenge, we propose an intervention-assisted framework that leverages strategic interventions from known stable policies to ensure the queue sizes remain bounded. This framework combines the learning power of neural networks with the guaranteed stability of classical control policies for SQNs. We introduce a method to design these intervention-assisted policies to ensure strong stability of the network. Furthermore, we extend foundational DRL theorems for intervention-assisted policies and develop two practical algorithms specifically for ODRLC of SQNs. Finally, we demonstrate through experiments that our proposed algorithms outperform both classical control approaches and prior ODRLC algorithms. △ Less

Submitted 5 April, 2024; originally announced April 2024.

Comments: 25 pages, 6 Figures

ACM Class: F.2.2; I.2.6

Monitoring Correlated Sources: AoI-based Scheduling is Nearly Optimal

Authors: R Vallabh Ramakanth, Vishrant Tripathi, Eytan Modiano

Abstract: We study the design of scheduling policies to minimize monitoring error for a collection of correlated sources, where only one source can be observed at any given time. We model correlated sources as a discrete-time Wiener process, where the increments are multivariate normal random variables, with a general covariance matrix that captures the correlation structure between the sources. Under a Kal… ▽ More We study the design of scheduling policies to minimize monitoring error for a collection of correlated sources, where only one source can be observed at any given time. We model correlated sources as a discrete-time Wiener process, where the increments are multivariate normal random variables, with a general covariance matrix that captures the correlation structure between the sources. Under a Kalman filter-based optimal estimation framework, we show that the performance of all scheduling policies oblivious to instantaneous error, can be lower and upper bounded by the weighted sum of Age of Information (AoI) across the sources for appropriately chosen weights. We use this insight to design scheduling policies that are only a constant factor away from optimality, and make the rather surprising observation that AoI-based scheduling that ignores correlation is sufficient to obtain performance guarantees. We also derive scaling results that show that the optimal error scales roughly as the square of the dimensionality of the system, even in the presence of correlation. Finally, we provide simulation results to verify our claims. △ Less

Submitted 29 December, 2023; v1 submitted 27 December, 2023; originally announced December 2023.

Comments: Accepted at IEEE INFOCOM 2024

Queueing Delay Minimization in Overloaded Networks via Rate Control

Authors: Xinyu Wu, Dan Wu, Eytan Modiano

Abstract: We develop link rate control policies to minimize the queueing delay of packets in overloaded networks. We show that increasing link rates does not guarantee delay reduction during overload. We consider a fluid queueing model that facilitates explicit characterization of the queueing delay of packets, and establish explicit conditions on link rates that can minimize the average and maximum queuein… ▽ More We develop link rate control policies to minimize the queueing delay of packets in overloaded networks. We show that increasing link rates does not guarantee delay reduction during overload. We consider a fluid queueing model that facilitates explicit characterization of the queueing delay of packets, and establish explicit conditions on link rates that can minimize the average and maximum queueing delay in both single-hop and multi-stage (switching) networks. These min-delay conditions require maintaining an identical ratio between the ingress and egress rates of different nodes at the same layer of the network. We term the policies that follow these conditions rate-proportional policies. We further generalize the rate-proportional policies to queue-proportional policies, which minimize the queueing delay asymptotically based on the time-varying queue length while remaining agnostic of packet arrival rates. We validate that the proposed policies lead to minimum queueing delay under various network topologies and settings, compared with benchmarks including the backpressure policy that maximizes network throughput and the max-link-rate policy that fully utilizes bandwidth. We further remark that the explicit min-delay policy design in multi-stage networks facilitates co-optimization with other metrics, such as minimizing total bandwidth, balancing link utilization and node buffer usage. This demonstrates the wider utility of our main results in data center network optimization in practice. △ Less

Submitted 7 December, 2023; originally announced December 2023.

Learning to Schedule in Non-Stationary Wireless Networks With Unknown Statistics

Authors: Quang Minh Nguyen, Eytan Modiano

Abstract: The emergence of large-scale wireless networks with partially-observable and time-varying dynamics has imposed new challenges on the design of optimal control policies. This paper studies efficient scheduling algorithms for wireless networks subject to generalized interference constraint, where mean arrival and mean service rates are unknown and non-stationary. This model exemplifies realistic edg… ▽ More The emergence of large-scale wireless networks with partially-observable and time-varying dynamics has imposed new challenges on the design of optimal control policies. This paper studies efficient scheduling algorithms for wireless networks subject to generalized interference constraint, where mean arrival and mean service rates are unknown and non-stationary. This model exemplifies realistic edge devices' characteristics of wireless communication in modern networks. We propose a novel algorithm termed MW-UCB for generalized wireless network scheduling, which is based on the Max-Weight policy and leverages the Sliding-Window Upper-Confidence Bound to learn the channels' statistics under non-stationarity. MW-UCB is provably throughput-optimal under mild assumptions on the variability of mean service rates. Specifically, as long as the total variation in mean service rates over any time period grows sub-linearly in time, we show that MW-UCB can achieve the stability region arbitrarily close to the stability region of the class of policies with full knowledge of the channel statistics. Extensive simulations validate our theoretical results and demonstrate the favorable performance of MW-UCB. △ Less

Submitted 4 August, 2023; originally announced August 2023.

Comments: Accepted to ACM MobiHoc '23

Age of Broadcast and Collection in Spatially Distributed Wireless Networks

Authors: Chirag Rao, Eytan Modiano

Abstract: We consider a wireless network with a base station broadcasting and collecting time-sensitive data to and from spatially distributed nodes in the presence of wireless interference. The Age of Information (AoI) is the time that has elapsed since the most-recently delivered packet was generated, and captures the freshness of information. In the context of broadcast and collection, we define the Age… ▽ More We consider a wireless network with a base station broadcasting and collecting time-sensitive data to and from spatially distributed nodes in the presence of wireless interference. The Age of Information (AoI) is the time that has elapsed since the most-recently delivered packet was generated, and captures the freshness of information. In the context of broadcast and collection, we define the Age of Broadcast (AoB) to be the amount of time elapsed until all nodes receive a fresh update, and the Age of Collection (AoC) as the amount of time that elapses until the base station receives an update from all nodes. We quantify the average broadcast and collection ages in two scenarios: 1) instance-dependent, in which the locations of all nodes and interferers are known, and 2) instance-independent, in which they are not known but are located randomly, and expected age is characterized with respect to node locations. In the instance-independent case, we show that AoB and AoC scale super-exponentially with respect to the radius of the region surrounding the base station. Simulation results highlight how expected AoB and AoC are affected by network parameters such as network density, medium access probability, and the size of the coverage region. △ Less

Submitted 9 December, 2022; originally announced December 2022.

Comments: To be presented at IEEE INFOCOM 2023

Minimizing Age of Information in Spatially Distributed Random Access Wireless Networks

Authors: Nicholas Jones, Eytan Modiano

Abstract: We analyze Age of Information (AoI) in wireless networks where nodes use a spatially adaptive random access scheme to send status updates to a central base station. We show that the set of achievable AoI in this setting is convex, and design policies to minimize weighted sum, min-max, and proportionally fair AoI by setting transmission probabilities as a function of node locations. We show that un… ▽ More We analyze Age of Information (AoI) in wireless networks where nodes use a spatially adaptive random access scheme to send status updates to a central base station. We show that the set of achievable AoI in this setting is convex, and design policies to minimize weighted sum, min-max, and proportionally fair AoI by setting transmission probabilities as a function of node locations. We show that under the capture model, when the spatial topology of the network is considered, AoI can be significantly improved, and we obtain tight performance bounds on weighted sum and min-max AoI. Finally, we design a policy where each node sets its transmission probability based only on its own distance from the base station, when it does not know the positions of other nodes, and show that it converges to the optimal proportionally fair policy as the size of the network goes to infinity. △ Less

Submitted 4 January, 2023; v1 submitted 7 December, 2022; originally announced December 2022.

Comments: Accepted to IEEE INFOCOM 2023

WiSwarm: Age-of-Information-based Wireless Networking for Collaborative Teams of UAVs

Authors: Vishrant Tripathi, Igor Kadota, Ezra Tal, Muhammad Shahir Rahman, Alexander Warren, Sertac Karaman, Eytan Modiano

Abstract: The Age-of-Information (AoI) metric has been widely studied in the theoretical communication networks and queuing systems literature. However, experimental evaluation of its applicability to complex real-world time-sensitive systems is largely lacking. In this work, we develop, implement, and evaluate an AoI-based application layer middleware that enables the customization of WiFi networks to the… ▽ More The Age-of-Information (AoI) metric has been widely studied in the theoretical communication networks and queuing systems literature. However, experimental evaluation of its applicability to complex real-world time-sensitive systems is largely lacking. In this work, we develop, implement, and evaluate an AoI-based application layer middleware that enables the customization of WiFi networks to the needs of time-sensitive applications. By controlling the storage and flow of information in the underlying WiFi network, our middleware can: (i) prevent packet collisions; (ii) discard stale packets that are no longer useful; and (iii) dynamically prioritize the transmission of the most relevant information. To demonstrate the benefits of our middleware, we implement a mobility tracking application using a swarm of UAVs communicating with a central controller via WiFi. Our experimental results show that, when compared to WiFi-UDP/WiFi-TCP, the middleware can improve information freshness by a factor of 109x/48x and tracking accuracy by a factor of 4x/6x, respectively. Most importantly, our results also show that the performance gains of our approach increase as the system scales and/or the traffic load increases. △ Less

Submitted 6 December, 2022; originally announced December 2022.

Comments: To be presented at IEEE INFOCOM 2023

Fresh-CSMA: A Distributed Protocol for Minimizing Age of Information

Authors: Vishrant Tripathi, Nicholas Jones, Eytan Modiano

Abstract: We consider the design of distributed scheduling algorithms that minimize age of information in single-hop wireless networks. The centralized max-weight policy is known to be nearly optimal in this setting; hence, our goal is to design a distributed CSMA scheme that can mimic its performance. To that end, we propose a distributed protocol called Fresh-CSMA and show that in an idealized setting, Fr… ▽ More We consider the design of distributed scheduling algorithms that minimize age of information in single-hop wireless networks. The centralized max-weight policy is known to be nearly optimal in this setting; hence, our goal is to design a distributed CSMA scheme that can mimic its performance. To that end, we propose a distributed protocol called Fresh-CSMA and show that in an idealized setting, Fresh-CSMA can match the scheduling decisions of the max-weight policy with high probability in each frame, and also match the theoretical performance guarantees of the max-weight policy over the entire time horizon. We then consider a more realistic setting and study the impact of protocol parameters on the probability of collisions and the overhead caused by the distributed nature of the protocol. We also consider the monitoring of Markov sources and extend our approach to CSMA protocols that incorporate Age of Incorrect Information (AoII) instead of AoI. Finally, we provide simulations that support our theoretical results and show that the performance gap between the ideal and realistic versions of Fresh-CSMA is small. △ Less

Submitted 12 July, 2023; v1 submitted 6 December, 2022; originally announced December 2022.

Comments: Accepted at IEEE INFOCOM 2023

Universal Policy Tracking: Scheduling for Wireless Networks with Delayed State Observation

Authors: Bai Liu, Eytan Modiano

Abstract: Numerous scheduling algorithms have been proposed to optimize various performance metrics like throughput, delay and utility in wireless networks. However, these algorithms often require instantaneous access to network state information, which is not always available. While network stability can sometimes be achieved with delayed state information, other performance metrics such as latency may deg… ▽ More Numerous scheduling algorithms have been proposed to optimize various performance metrics like throughput, delay and utility in wireless networks. However, these algorithms often require instantaneous access to network state information, which is not always available. While network stability can sometimes be achieved with delayed state information, other performance metrics such as latency may degrade. Thus, instead of simply stabilizing the system, our goal is to design a framework that can mimic arbitrary scheduling algorithms with performance guarantees. A naive approach is to make decisions directly with delayed information, but we show that such methods may lead to poor performance. Instead, we propose the Universal Tracking (UT) algorithm that can mimic the actions of arbitrary scheduling algorithms under observation delay. We rigorously show that the performance gap between UT and the scheduling algorithm being tracked is bounded by constants. Our numerical experiments show that UT significantly outperforms the naive approach in various applications. △ Less

Submitted 30 November, 2022; v1 submitted 27 September, 2022; originally announced September 2022.

Optimizing Age of Information with Correlated Sources

Authors: Vishrant Tripathi, Eytan Modiano

Abstract: We develop a simple model for the timely monitoring of correlated sources over a wireless network. Using this model, we study how to optimize weighted-sum average Age of Information (AoI) in the presence of correlation. First, we discuss how to find optimal stationary randomized policies and show that they are at-most a factor of two away from optimal policies in general. Then, we develop a Lyapun… ▽ More We develop a simple model for the timely monitoring of correlated sources over a wireless network. Using this model, we study how to optimize weighted-sum average Age of Information (AoI) in the presence of correlation. First, we discuss how to find optimal stationary randomized policies and show that they are at-most a factor of two away from optimal policies in general. Then, we develop a Lyapunov drift-based max-weight policy that performs better than randomized policies in practice and show that it is also at-most a factor of two away from optimal. Next, we derive scaling results that show how AoI improves in large networks in the presence of correlation. We also show that for stationary randomized policies, the expression for average AoI is robust to the way in which the correlation structure is modeled. Finally, for the setting where correlation parameters are unknown and time-varying, we develop a heuristic policy that adapts its scheduling decisions by learning the correlation parameters in an online manner. We also provide numerical simulations to support our theoretical results. △ Less

Submitted 2 August, 2022; originally announced August 2022.

Comments: To be presented at ACM MobiHoc 2022

Information Freshness in Multi-Hop Wireless Networks

Authors: Vishrant Tripathi, Rajat Talak, Eytan Modiano

Abstract: We consider the problem of minimizing age of information in multihop wireless networks and propose three classes of policies to solve the problem - stationary randomized, age difference, and age debt. For the unicast setting with fixed routes between each source-destination pair, we first develop a procedure to find age optimal Stationary Randomized policies. These policies are easy to implement a… ▽ More We consider the problem of minimizing age of information in multihop wireless networks and propose three classes of policies to solve the problem - stationary randomized, age difference, and age debt. For the unicast setting with fixed routes between each source-destination pair, we first develop a procedure to find age optimal Stationary Randomized policies. These policies are easy to implement and allow us to derive closed-form expression for average AoI. Next, for the same unicast setting, we develop a class of heuristic policies, called Age Difference, based on the idea that if neighboring nodes try to reduce their age differential then all nodes will have fresher updates. This approach is useful in practice since it relies only on the local age differential between nodes to make scheduling decisions. Finally, we propose the class of policies called Age Debt, which can handle 1) non-linear AoI cost functions; 2) unicast, multicast and broadcast flows; and 3) no fixed routes specified per flow beforehand. Here, we convert AoI optimization problems into equivalent network stability problems and use Lyapunov drift to find scheduling and routing schemes that stabilize the network. We also provide numerical results comparing our proposed classes of policies with the best known scheduling and routing schemes available in the literature for a wide variety of network settings. △ Less

Submitted 17 November, 2021; originally announced November 2021.

Comments: submitted to Transactions on Networking, conference versions published in Allerton 2017 and Infocom AoI workshop 2021. arXiv admin note: substantial text overlap with arXiv:2101.10225

Computation and Communication Co-Design for Real-Time Monitoring and Control in Multi-Agent Systems

Authors: Vishrant Tripathi, Luca Ballotta, Luca Carlone, Eytan Modiano

Abstract: We investigate the problem of co-designing computation and communication in a multi-agent system (e.g. a sensor network or a multi-robot team). We consider the realistic setting where each agent acquires sensor data and is capable of local processing before sending updates to a base station, which is in charge of making decisions or monitoring phenomena of interest in real time. Longer processing… ▽ More We investigate the problem of co-designing computation and communication in a multi-agent system (e.g. a sensor network or a multi-robot team). We consider the realistic setting where each agent acquires sensor data and is capable of local processing before sending updates to a base station, which is in charge of making decisions or monitoring phenomena of interest in real time. Longer processing at an agent leads to more informative updates but also larger delays, giving rise to a delay-accuracy-tradeoff in choosing the right amount of local processing at each agent. We assume that the available communication resources are limited due to interference, bandwidth, and power constraints. Thus, a scheduling policy needs to be designed to suitably share the communication channel among the agents. To that end, we develop a general formulation to jointly optimize the local processing at the agents and the scheduling of transmissions. Our novel formulation leverages the notion of Age of Information to quantify the freshness of data and capture the delays caused by computation and communication. We develop efficient resource allocation algorithms using the Whittle index approach and demonstrate our proposed algorithms in two practical applications: multi-agent occupancy grid map** in time-varying environments, and ride sharing in autonomous vehicle networks. Our experiments show that the proposed co-design approach leads to a substantial performance improvement (18-82% in our tests). △ Less

Submitted 8 August, 2021; v1 submitted 6 August, 2021; originally announced August 2021.

Comments: accepted at WiOpt 2021

An Online Learning Approach to Optimizing Time-Varying Costs of AoI

Authors: Vishrant Tripathi, Eytan Modiano

Abstract: We consider systems that require timely monitoring of sources over a communication network, where the cost of delayed information is unknown, time-varying and possibly adversarial. For the single source monitoring problem, we design algorithms that achieve sublinear regret compared to the best fixed policy in hindsight. For the multiple source scheduling problem, we design a new online learning al… ▽ More We consider systems that require timely monitoring of sources over a communication network, where the cost of delayed information is unknown, time-varying and possibly adversarial. For the single source monitoring problem, we design algorithms that achieve sublinear regret compared to the best fixed policy in hindsight. For the multiple source scheduling problem, we design a new online learning algorithm called Follow-the-Perturbed-Whittle-Leader and show that it has low regret compared to the best fixed scheduling policy in hindsight, while remaining computationally feasible. The algorithm and its regret analysis are novel and of independent interest to the study of online restless multi-armed bandit problems. We further design algorithms that achieve sublinear regret compared to the best dynamic policy when the environment is slowly varying. Finally, we apply our algorithms to a mobility tracking problem. We consider non-stationary and adversarial mobility models and illustrate the performance benefit of using our online learning algorithms compared to an oblivious scheduling policy. △ Less

Submitted 27 May, 2021; originally announced May 2021.

Comments: Accepted to ACM Mobihoc '21

Age Debt: A General Framework For Minimizing Age of Information

Authors: Vishrant Tripathi, Eytan Modiano

Abstract: We consider the problem of minimizing age of information in general single-hop and multihop wireless networks. First, we formulate a way to convert AoI optimization problems into equivalent network stability problems. Then, we propose a heuristic low complexity approach for achieving stability that can handle general network topologies; unicast, multicast and broadcast flows; interference constrai… ▽ More We consider the problem of minimizing age of information in general single-hop and multihop wireless networks. First, we formulate a way to convert AoI optimization problems into equivalent network stability problems. Then, we propose a heuristic low complexity approach for achieving stability that can handle general network topologies; unicast, multicast and broadcast flows; interference constraints; link reliabilities; and AoI cost functions. We provide numerical results to show that our proposed algorithms behave as well as the best known scheduling and routing schemes available in the literature for a wide variety of network settings. △ Less

Submitted 25 January, 2021; originally announced January 2021.

Comments: To be presented at the 4th AoI Workshop, IEEE INFOCOM 2021

WiFresh: Age-of-Information from Theory to Implementation

Authors: Igor Kadota, Muhammad Shahir Rahman, Eytan Modiano

Abstract: Emerging applications, such as smart factories and fleets of drones, increasingly rely on sharing time-sensitive information for monitoring and control. In such application domains, it is essential to keep information fresh, as outdated information loses its value and can lead to system failures and safety risks. The Age-of-Information is a performance metric that captures how fresh the informatio… ▽ More Emerging applications, such as smart factories and fleets of drones, increasingly rely on sharing time-sensitive information for monitoring and control. In such application domains, it is essential to keep information fresh, as outdated information loses its value and can lead to system failures and safety risks. The Age-of-Information is a performance metric that captures how fresh the information is from the perspective of the destination. In this paper, we show that as the congestion in the wireless network increases, the Age-of-Information degrades sharply, leading to outdated information at the destination. Leveraging years of theoretical research, we propose WiFresh: an unconventional architecture that achieves near optimal information freshness in wireless networks of any size, even when the network is overloaded. Our experimental results show that WiFresh can improve information freshness by two orders of magnitude when compared to an equivalent standard WiFi network. We propose and realize two strategies for implementing WiFresh: one at the MAC layer using hardware-level programming and another at the Application layer using Python. △ Less

Submitted 28 December, 2020; originally announced December 2020.

Learning-NUM: Network Utility Maximization with Unknown Utility Functions and Queueing Delay

Authors: Xinzhe Fu, Eytan Modiano

Abstract: Network Utility Maximization (NUM) studies the problems of allocating traffic rates to network users in order to maximize the users' total utility subject to network resource constraints. In this paper, we propose a new NUM framework, Learning-NUM, where the users' utility functions are unknown apriori and the utility function values of the traffic rates can be observed only after the correspondin… ▽ More Network Utility Maximization (NUM) studies the problems of allocating traffic rates to network users in order to maximize the users' total utility subject to network resource constraints. In this paper, we propose a new NUM framework, Learning-NUM, where the users' utility functions are unknown apriori and the utility function values of the traffic rates can be observed only after the corresponding traffic is delivered to the destination, which means that the utility feedback experiences \textit{queueing delay}. The goal is to design a policy that gradually learns the utility functions and makes rate allocation and network scheduling/routing decisions so as to maximize the total utility obtained over a finite time horizon $T$. In addition to unknown utility functions and stochastic constraints, a central challenge of our problem lies in the queueing delay of the observations, which may be unbounded and depends on the decisions of the policy. We first show that the expected total utility obtained by the best dynamic policy is upper bounded by the solution to a static optimization problem. Without the presence of feedback delay, we design an algorithm based on the ideas of gradient estimation and Max-Weight scheduling. To handle the feedback delay, we embed the algorithm in a parallel-instance paradigm to form a policy that achieves $\tilde{O}(T^{3/4})$-regret, i.e., the difference between the expected utility obtained by the best dynamic policy and our policy is in $\tilde{O}(T^{3/4})$. Finally, to demonstrate the practical applicability of the Learning-NUM framework, we apply it to three application scenarios including database query, job scheduling and video streaming. We further conduct simulations on the job scheduling application to evaluate the empirical performance of our policy. △ Less

Submitted 16 December, 2020; originally announced December 2020.

Aging Bandits: Regret Analysis and Order-Optimal Learning Algorithm for Wireless Networks with Stochastic Arrivals

Authors: Eray Unsal Atay, Igor Kadota, Eytan Modiano

Abstract: We consider a single-hop wireless network with sources transmitting time-sensitive information to the destination over multiple unreliable channels. Packets from each source are generated according to a stochastic process with known statistics and the state of each wireless channel (ON/OFF) varies according to a stochastic process with unknown statistics. The reliability of the wireless channels i… ▽ More We consider a single-hop wireless network with sources transmitting time-sensitive information to the destination over multiple unreliable channels. Packets from each source are generated according to a stochastic process with known statistics and the state of each wireless channel (ON/OFF) varies according to a stochastic process with unknown statistics. The reliability of the wireless channels is to be learned through observation. At every time slot, the learning algorithm selects a single pair (source, channel) and the selected source attempts to transmit its packet via the selected channel. The probability of a successful transmission to the destination depends on the reliability of the selected channel. The goal of the learning algorithm is to minimize the Age-of-Information (AoI) in the network over $T$ time slots. To analyze the performance of the learning algorithm, we introduce the notion of AoI regret, which is the difference between the expected cumulative AoI of the learning algorithm under consideration and the expected cumulative AoI of a genie algorithm that knows the reliability of the channels a priori. The AoI regret captures the penalty incurred by having to learn the statistics of the channels over the $T$ time slots. The results are two-fold: first, we consider learning algorithms that employ well-known solutions to the stochastic multi-armed bandit problem (such as $ε$-Greedy, Upper Confidence Bound, and Thompson Sampling) and show that their AoI regret scales as $Θ(\log T)$; second, we develop a novel learning algorithm and show that it has $O(1)$ regret. To the best of our knowledge, this is the first learning algorithm with bounded AoI regret. △ Less

Submitted 21 December, 2020; v1 submitted 15 December, 2020; originally announced December 2020.

RL-QN: A Reinforcement Learning Framework for Optimal Control of Queueing Systems

Authors: Bai Liu, Qiaomin Xie, Eytan Modiano

Abstract: With the rapid advance of information technology, network systems have become increasingly complex and hence the underlying system dynamics are often unknown or difficult to characterize. Finding a good network control policy is of significant importance to achieve desirable network performance (e.g., high throughput or low delay). In this work, we consider using model-based reinforcement learning… ▽ More With the rapid advance of information technology, network systems have become increasingly complex and hence the underlying system dynamics are often unknown or difficult to characterize. Finding a good network control policy is of significant importance to achieve desirable network performance (e.g., high throughput or low delay). In this work, we consider using model-based reinforcement learning (RL) to learn the optimal control policy for queueing networks so that the average job delay (or equivalently the average queue backlog) is minimized. Traditional approaches in RL, however, cannot handle the unbounded state spaces of the network control problem. To overcome this difficulty, we propose a new algorithm, called Reinforcement Learning for Queueing Networks (RL-QN), which applies model-based RL methods over a finite subset of the state space, while applying a known stabilizing policy for the rest of the states. We establish that the average queue backlog under RL-QN with an appropriately constructed subset can be arbitrarily close to the optimal result. We evaluate RL-QN in dynamic server allocation, routing and switching problems. Simulation results show that RL-QN minimizes the average queue backlog effectively. △ Less

Submitted 7 April, 2022; v1 submitted 14 November, 2020; originally announced November 2020.

Throughput Maximization in Uncooperative Spectrum Sharing Networks

Authors: Thomas Stahlbuhk, Brooke Shrader, Eytan Modiano

Abstract: Throughput-optimal transmission scheduling in wireless networks has been a well considered problem in the literature, and the method for achieving optimality, MaxWeight scheduling, has been known for several decades. This algorithm achieves optimality by adaptively scheduling transmissions relative to each user's stochastic traffic demands. To implement the method, users must report their queue ba… ▽ More Throughput-optimal transmission scheduling in wireless networks has been a well considered problem in the literature, and the method for achieving optimality, MaxWeight scheduling, has been known for several decades. This algorithm achieves optimality by adaptively scheduling transmissions relative to each user's stochastic traffic demands. To implement the method, users must report their queue backlogs to the network controller and must rapidly respond to the resulting resource allocations. However, many currently-deployed wireless systems are not able to perform these tasks and instead expect to occupy a fixed assignment of resources. To accommodate these limitations, adaptive scheduling algorithms need to interactively estimate these uncooperative users' queue backlogs and make scheduling decisions to account for their predicted behavior. In this work, we address the problem of scheduling with uncooperative legacy systems by develo** algorithms to accomplish these tasks. We begin by formulating the problem of inferring the uncooperative systems' queue backlogs as a partially observable Markov decision process and proceed to show how our resulting learning algorithms can be successfully used in a queue-length-based scheduling policy. Our theoretical analysis characterizes the throughput-stability region of the network and is verified using simulation results. △ Less

Submitted 4 August, 2020; originally announced August 2020.

Comments: 15 pages, 7 figures

Age of Information: An Introduction and Survey

Authors: Roy D. Yates, Yin Sun, D. Richard Brown III, Sanjit K. Kaul, Eytan Modiano, Sennur Ulukus

Abstract: We summarize recent contributions in the broad area of age of information (AoI). In particular, we describe the current state of the art in the design and optimization of low-latency cyberphysical systems and applications in which sources send time-stamped status updates to interested recipients. These applications desire status updates at the recipients to be as timely as possible; however, this… ▽ More We summarize recent contributions in the broad area of age of information (AoI). In particular, we describe the current state of the art in the design and optimization of low-latency cyberphysical systems and applications in which sources send time-stamped status updates to interested recipients. These applications desire status updates at the recipients to be as timely as possible; however, this is typically constrained by limited system resources. We describe AoI timeliness metrics and present general methods of AoI evaluation analysis that are applicable to a wide variety of sources and systems. Starting from elementary single-server queues, we apply these AoI methods to a range of increasingly complex systems, including energy harvesting sensors transmitting over noisy channels, parallel server systems, queueing networks, and various single-hop and multi-hop wireless networks. We also explore how update age is related to MMSE methods of sampling, estimation and control of stochastic processes. The paper concludes with a review of efforts to employ age optimization in cyberphysical applications. △ Less

Submitted 16 July, 2020; originally announced July 2020.

Learning Algorithms for Minimizing Queue Length Regret

Authors: Thomas Stahlbuhk, Brooke Shrader, Eytan Modiano

Abstract: We consider a system consisting of a single transmitter/receiver pair and $N$ channels over which they may communicate. Packets randomly arrive to the transmitter's queue and wait to be successfully sent to the receiver. The transmitter may attempt a frame transmission on one channel at a time, where each frame includes a packet if one is in the queue. For each channel, an attempted transmission i… ▽ More We consider a system consisting of a single transmitter/receiver pair and $N$ channels over which they may communicate. Packets randomly arrive to the transmitter's queue and wait to be successfully sent to the receiver. The transmitter may attempt a frame transmission on one channel at a time, where each frame includes a packet if one is in the queue. For each channel, an attempted transmission is successful with an unknown probability. The transmitter's objective is to quickly identify the best channel to minimize the number of packets in the queue over $T$ time slots. To analyze system performance, we introduce queue length regret, which is the expected difference between the total queue length of a learning policy and a controller that knows the rates, a priori. One approach to designing a transmission policy would be to apply algorithms from the literature that solve the closely-related stochastic multi-armed bandit problem. These policies would focus on maximizing the number of successful frame transmissions over time. However, we show that these methods have $Ω(\log{T})$ queue length regret. On the other hand, we show that there exists a set of queue-length based policies that can obtain order optimal $O(1)$ queue length regret. We use our theoretical analysis to devise heuristic methods that are shown to perform well in simulation. △ Less

Submitted 14 May, 2020; v1 submitted 11 May, 2020; originally announced May 2020.

Comments: 28 Pages, 11 figures

Age-Delay Tradeoffs in Queueing Systems

Authors: Rajat Talak, Eytan Modiano

Abstract: We consider an M server system in which each server can service at most one update packet at a time. The system designer controls (1) scheduling - the order in which the packets get serviced, (2) routing - the server that an arriving update packet joins for service, and (3) the service time distribution with fixed service rate. Given a fixed update generation process, we prove a strong age-delay a… ▽ More We consider an M server system in which each server can service at most one update packet at a time. The system designer controls (1) scheduling - the order in which the packets get serviced, (2) routing - the server that an arriving update packet joins for service, and (3) the service time distribution with fixed service rate. Given a fixed update generation process, we prove a strong age-delay and age-delay variance tradeoff, wherein, as the average AoI approaches its minimum, the packet delay and its variance approach infinity. In order to prove this result, we consider two special cases of the M server system, namely, a single server system with last come first server with preemptive service and an infinite server system. In both these cases, we derive sufficient conditions to show that three heavy tailed service time distributions, namely Pareto, log-normal, and Weibull, asymptotically minimize the average AoI as their tail gets heavier, and establish the age-delay tradeoff results. We provide an intuitive explanation as to why such a seemingly counter intuitive age-delay tradeoff is natural, and that it should exist in many systems. △ Less

Submitted 11 November, 2019; originally announced November 2019.

Comments: arXiv admin note: text overlap with arXiv:1810.04371

A Theory of Uncertainty Variables for State Estimation and Inference

Authors: Rajat Talak, Sertac Karaman, Eytan Modiano

Abstract: We develop a new framework of uncertainty variables to model uncertainty. An uncertainty variable is characterized by an uncertainty set, in which its realization is bound to lie, while the conditional uncertainty is characterized by a set map, from a given realization of a variable to a set of possible realizations of another variable. We prove Bayes' law and the law of total probability equivale… ▽ More We develop a new framework of uncertainty variables to model uncertainty. An uncertainty variable is characterized by an uncertainty set, in which its realization is bound to lie, while the conditional uncertainty is characterized by a set map, from a given realization of a variable to a set of possible realizations of another variable. We prove Bayes' law and the law of total probability equivalents for uncertainty variables. We define a notion of independence, conditional independence, and pairwise independence for a collection of uncertainty variables, and show that this new notion of independence preserves the properties of independence defined over random variables. We then develop a graphical model, namely Bayesian uncertainty network, a Bayesian network equivalent defined over a collection of uncertainty variables, and show that all the natural conditional independence properties, expected out of a Bayesian network, hold for the Bayesian uncertainty network. We also define the notion of point estimate, and show its relation with the maximum a posteriori estimate. Probability theory starts with a distribution function (equivalently a probability measure) as a primitive and builds all other useful concepts, such as law of total probability, Bayes' law, independence, graphical models, point estimate, on it. Our work shows that it is perfectly possible to start with a set, instead of a distribution function, and retain all the useful ideas needed for state estimation and inference. △ Less

Submitted 9 December, 2019; v1 submitted 23 September, 2019; originally announced September 2019.

A Whittle Index Approach to Minimizing Functions of Age of Information

Authors: Vishrant Tripathi, Eytan Modiano

Abstract: We consider a setting where multiple active sources send real-time updates over a single-hop wireless broadcast network to a monitoring station. Our goal is to design a scheduling policy that minimizes the time-average of general non-decreasing cost functions of Age of Information. We use a Whittle index based approach to find low complexity scheduling policies that have good performance. We prove… ▽ More We consider a setting where multiple active sources send real-time updates over a single-hop wireless broadcast network to a monitoring station. Our goal is to design a scheduling policy that minimizes the time-average of general non-decreasing cost functions of Age of Information. We use a Whittle index based approach to find low complexity scheduling policies that have good performance. We prove that for a system with two sources, having possibly different cost functions and reliable channels, the Whittle index policy is exactly optimal. We derive structural properties of an optimal policy, that suggest that the performance of the Whittle index policy may be close to optimal in general. These results might also be of independent interest in the study of restless multi-armed bandit problems with similar underlying structure. We further establish that minimizing monitoring error for linear time-invariant systems and symmetric Markov chains is equivalent to minimizing appropriately chosen monotone functions of Age of Information. Finally, we provide simulations comparing the Whittle index policy with optimal scheduling policies found using dynamic programming, which support our results. △ Less

Submitted 20 June, 2023; v1 submitted 27 August, 2019; originally announced August 2019.

Comments: Accepted for Allerton'19

Minimizing the Age of Information in Wireless Networks with Stochastic Arrivals

Authors: Igor Kadota, Eytan Modiano

Abstract: We consider a wireless network with a base station serving multiple traffic streams to different destinations. Packets from each stream arrive to the base station according to a stochastic process and are enqueued in a separate (per stream) queue. The queueing discipline controls which packet within each queue is available for transmission. The base station decides, at every time t, which stream t… ▽ More We consider a wireless network with a base station serving multiple traffic streams to different destinations. Packets from each stream arrive to the base station according to a stochastic process and are enqueued in a separate (per stream) queue. The queueing discipline controls which packet within each queue is available for transmission. The base station decides, at every time t, which stream to serve to the corresponding destination. The goal of scheduling decisions is to keep the information at the destinations fresh. Information freshness is captured by the Age of Information (AoI) metric. In this paper, we derive a lower bound on the AoI performance achievable by any given network operating under any queueing discipline. Then, we consider three common queueing disciplines and develop both an Optimal Stationary Randomized policy and a Max-Weight policy under each discipline. Our approach allows us to evaluate the combined impact of the stochastic arrivals, queueing discipline and scheduling policy on AoI. We evaluate the AoI performance both analytically and using simulations. Numerical results show that the performance of the Max-Weight policy is close to the analytical lower bound. △ Less

Submitted 16 May, 2019; originally announced May 2019.

Topology Discovery Using Path Interference

Authors: Anurag Rai, Eytan Modiano

Abstract: We consider the problem of inferring the topology of a network using the measurements available at the end nodes, without cooperation from the internal nodes. To this end, we provide a simple method to obtain path interference which identifies whether two paths in the network intersect with each other. Using this information, we formulate the topology inference problem as an integer program and de… ▽ More We consider the problem of inferring the topology of a network using the measurements available at the end nodes, without cooperation from the internal nodes. To this end, we provide a simple method to obtain path interference which identifies whether two paths in the network intersect with each other. Using this information, we formulate the topology inference problem as an integer program and develop polynomial time algorithms to solve it optimally for networks with tree and ring topologies. Finally, we use the insight developed from these algorithms to develop a heuristic for identifying general topologies. Simulation results show that our heuristic outperforms a recently proposed algorithm that uses distance measurements for topology discovery. △ Less

Submitted 17 March, 2019; originally announced March 2019.

Age of Information for Discrete Time Queues

Authors: Vishrant Tripathi, Rajat Talak, Eytan Modiano

Abstract: Age of information (AoI) is a time-evolving measure of information freshness, that tracks the time since the last received fresh update was generated. Analyzing peak and average AoI, two time average metrics of AoI, for various continuous time queueing systems has received considerable attention. We analyze peak and average age for various discrete time queueing systems. We first consider first co… ▽ More Age of information (AoI) is a time-evolving measure of information freshness, that tracks the time since the last received fresh update was generated. Analyzing peak and average AoI, two time average metrics of AoI, for various continuous time queueing systems has received considerable attention. We analyze peak and average age for various discrete time queueing systems. We first consider first come first serve (FCFS) Ber/G/1 and Ber/G/1 queue with vacations, and derive explicit expressions for peak and average age. We also obtain age expressions for the last come first serve (LCFS) queue and the $G/G/\infty$ queue. We build upon proof techniques from earlier results, and also present new techniques that might be of independent interest in analyzing age in discrete time queuing systems. △ Less

Submitted 28 January, 2019; originally announced January 2019.

Comments: arXiv admin note: text overlap with arXiv:1810.04371

A Hierarchical WDM-based Scalable Data Center Network Architecture

Authors: Maotong Xu, Jelena Diakonikolas, Eytan Modiano, Suresh Subramaniam

Abstract: Massive data centers are at the heart of the Internet. The rapid growth of Internet traffic and the abundance of rich data-driven applications have raised the need for enormous network bandwidth. Towards meeting this growing traffic demand, optical interconnects have gained significant attention, as they can provide high throughput, low latency, and scalability. In particular, optical Wavelength D… ▽ More Massive data centers are at the heart of the Internet. The rapid growth of Internet traffic and the abundance of rich data-driven applications have raised the need for enormous network bandwidth. Towards meeting this growing traffic demand, optical interconnects have gained significant attention, as they can provide high throughput, low latency, and scalability. In particular, optical Wavelength Division Multiplexing (WDM) provides the possibility to build data centers comprising of millions of servers, while providing hundreds of terabits per second bandwidth. In this paper, we propose a WDM-based Reconfigurable Hierarchical Optical Data Center Architecture (RHODA) that can satisfy future Internet traffic demands. To improve scalability, our DCN architecture is hierarchical, as it groups server racks into clusters. Cluster membership is reconfigurable through the use of optical switches. Each cluster enables heavy-traffic communication among the racks within. To support varying traffic patterns, the inter-cluster network topology and link capacities are also reconfigurable, which is achieved through the use of optical space switches and Wavelength Selective Switches (WSSs). Our simulation results demonstrate that in terms of average hop distance, RHODA outperforms OSA, FatTree and WaveCube by up to 81%, 66% and 60%, respectively. △ Less

Submitted 25 January, 2019; v1 submitted 18 January, 2019; originally announced January 2019.

Age-Delay Tradeoffs in Single Server Systems

Authors: Rajat Talak, Eytan Modiano

Abstract: Information freshness and low latency communication is important to many emerging applications. While Age of Information (AoI) serves as a metric of information freshness, packet delay is a traditional metric of communication latency. We prove that there is a natural tradeoff between the AoI and packet delay. We consider a single server system, in which at most one update packet can be serviced at… ▽ More Information freshness and low latency communication is important to many emerging applications. While Age of Information (AoI) serves as a metric of information freshness, packet delay is a traditional metric of communication latency. We prove that there is a natural tradeoff between the AoI and packet delay. We consider a single server system, in which at most one update packet can be serviced at a time. The system designer controls the order in which the packets get serviced and the service time distribution, with a given service rate. We analyze two tradeoff problems that minimize packet delay and the variance in packet delay, respectively, subject to an average age constraint. We prove a strong age-delay and age-delay variance tradeoff, wherein, as the average age approaches its minimum, the delay and its variance approach infinity. We show that the service time distribution that mininizes average age, must necessarily have an unbounded-second moment. △ Less

Submitted 14 January, 2019; originally announced January 2019.

On the Robustness of Distributed Computing Networks

Authors: Jianan Zhang, Hyang-Won Lee, Eytan Modiano

Abstract: Traffic flows in a distributed computing network require both transmission and processing, and can be interdicted by removing either communication or computation resources. We study the robustness of a distributed computing network under the failures of communication links and computation nodes. We define cut metrics that measure the connectivity, and show a non-zero gap between the maximum flow a… ▽ More Traffic flows in a distributed computing network require both transmission and processing, and can be interdicted by removing either communication or computation resources. We study the robustness of a distributed computing network under the failures of communication links and computation nodes. We define cut metrics that measure the connectivity, and show a non-zero gap between the maximum flow and the minimum cut. Moreover, we study a network flow interdiction problem that minimizes the maximum flow by removing communication and computation resources within a given budget. We develop mathematical programs to compute the optimal interdiction, and polynomial-time approximation algorithms that achieve near-optimal interdiction in simulation. △ Less

Submitted 26 November, 2021; v1 submitted 9 January, 2019; originally announced January 2019.

Comments: International Conference on the Design of Reliable Communication Networks (DRCN)

Network Interdiction Using Adversarial Traffic Flows

Authors: Xinzhe Fu, Eytan Modiano

Abstract: Traditional network interdiction refers to the problem of an interdictor trying to reduce the throughput of network users by removing network edges. In this paper, we propose a new paradigm for network interdiction that models scenarios, such as stealth DoS attack, where the interdiction is performed through injecting adversarial traffic flows. Under this paradigm, we first study the deterministic… ▽ More Traditional network interdiction refers to the problem of an interdictor trying to reduce the throughput of network users by removing network edges. In this paper, we propose a new paradigm for network interdiction that models scenarios, such as stealth DoS attack, where the interdiction is performed through injecting adversarial traffic flows. Under this paradigm, we first study the deterministic flow interdiction problem, where the interdictor has perfect knowledge of the operation of network users. We show that the problem is highly inapproximable on general networks and is NP-hard even when the network is acyclic. We then propose an algorithm that achieves a logarithmic approximation ratio and quasi-polynomial time complexity for acyclic networks through harnessing the submodularity of the problem. Next, we investigate the robust flow interdiction problem, which adopts the robust optimization framework to capture the case where definitive knowledge of the operation of network users is not available. We design an approximation framework that integrates the aforementioned algorithm, yielding a quasi-polynomial time procedure with poly-logarithmic approximation ratio for the more challenging robust flow interdiction. Finally, we evaluate the performance of the proposed algorithms through simulations, showing that they can be efficiently implemented and yield near-optimal solutions. △ Less

Submitted 8 January, 2019; originally announced January 2019.

Age Optimal Information Gathering and Dissemination on Graphs

Authors: Vishrant Tripathi, Rajat Talak, Eytan Modiano

Abstract: We consider the problem of timely exchange of updates between a central station and a set of ground terminals $V$, via a mobile agent that traverses across the ground terminals along a mobility graph $G = (V, E)$. We design the trajectory of the mobile agent to minimize peak and average age of information (AoI), two newly proposed metrics for measuring timeliness of information. We consider random… ▽ More We consider the problem of timely exchange of updates between a central station and a set of ground terminals $V$, via a mobile agent that traverses across the ground terminals along a mobility graph $G = (V, E)$. We design the trajectory of the mobile agent to minimize peak and average age of information (AoI), two newly proposed metrics for measuring timeliness of information. We consider randomized trajectories, in which the mobile agent travels from terminal $i$ to terminal $j$ with probability $P_{i,j}$. For the information gathering problem, we show that a randomized trajectory is peak age optimal and factor-$8\mathcal{H}$ average age optimal, where $\mathcal{H}$ is the mixing time of the randomized trajectory on the mobility graph $G$. We also show that the average age minimization problem is NP-hard. For the information dissemination problem, we prove that the same randomized trajectory is factor-$O(\mathcal{H})$ peak and average age optimal. Moreover, we propose an age-based trajectory, which utilizes information about current age at terminals, and show that it is factor-$2$ average age optimal in a symmetric setting. △ Less

Submitted 8 January, 2019; originally announced January 2019.

Optimal Network Control in Partially-Controllable Networks

Authors: Qingkai Liang, Eytan Modiano

Abstract: The effectiveness of many optimal network control algorithms (e.g., BackPressure) relies on the premise that all of the nodes are fully controllable. However, these algorithms may yield poor performance in a partially-controllable network where a subset of nodes are uncontrollable and use some unknown policy. Such a partially-controllable model is of increasing importance in real-world networked s… ▽ More The effectiveness of many optimal network control algorithms (e.g., BackPressure) relies on the premise that all of the nodes are fully controllable. However, these algorithms may yield poor performance in a partially-controllable network where a subset of nodes are uncontrollable and use some unknown policy. Such a partially-controllable model is of increasing importance in real-world networked systems such as overlay-underlay networks. In this paper, we design optimal network control algorithms that can stabilize a partially-controllable network. We first study the scenario where uncontrollable nodes use a queue-agnostic policy, and propose a low-complexity throughput-optimal algorithm, called Tracking-MaxWeight (TMW), which enhances the original MaxWeight algorithm with an explicit learning of the policy used by uncontrollable nodes. Next, we investigate the scenario where uncontrollable nodes use a queue-dependent policy and the problem is formulated as an MDP with unknown queueing dynamics. We propose a new reinforcement learning algorithm, called Truncated Upper Confidence Reinforcement Learning (TUCRL), and prove that TUCRL achieves tunable three-way tradeoffs between throughput, delay and convergence rate. △ Less

Submitted 6 January, 2019; originally announced January 2019.

Learning to Route Efficiently with End-to-End Feedback: The Value of Networked Structure

Authors: Ruihao Zhu, Eytan Modiano

Abstract: We introduce efficient algorithms which achieve nearly optimal regrets for the problem of stochastic online shortest path routing with end-to-end feedback. The setting is a natural application of the combinatorial stochastic bandits problem, a special case of the linear stochastic bandits problem. We show how the difficulties posed by the large scale action set can be overcome by the networked str… ▽ More We introduce efficient algorithms which achieve nearly optimal regrets for the problem of stochastic online shortest path routing with end-to-end feedback. The setting is a natural application of the combinatorial stochastic bandits problem, a special case of the linear stochastic bandits problem. We show how the difficulties posed by the large scale action set can be overcome by the networked structure of the action set. Our approach presents a novel connection between bandit learning and shortest path algorithms. Our main contribution is an adaptive exploration algorithm with nearly optimal instance-dependent regret for any directed acyclic network. We then modify it so that nearly optimal worst case regret is achieved simultaneously. Driven by the carefully designed Top-Two Comparison (TTC) technique, the algorithms are efficiently implementable. We further conduct extensive numerical experiments to show that our proposed algorithms not only achieve superior regret performances, but also reduce the runtime drastically. △ Less

Submitted 18 December, 2018; v1 submitted 24 October, 2018; originally announced October 2018.

Can Determinacy Minimize Age of Information?

Authors: Rajat Talak, Sertac Karaman, Eytan Modiano

Abstract: Age-of-information (AoI) is a newly proposed performance metric of information freshness. It differs from the traditional delay metric, because it is destination centric and measures the time that elapsed since the last received fresh information update was generated at the source. AoI has been analyzed for several queueing models, and the problem of optimizing AoI over arrival and service rates h… ▽ More Age-of-information (AoI) is a newly proposed performance metric of information freshness. It differs from the traditional delay metric, because it is destination centric and measures the time that elapsed since the last received fresh information update was generated at the source. AoI has been analyzed for several queueing models, and the problem of optimizing AoI over arrival and service rates has been studied in the literature. We consider the problem of minimizing AoI over the space of update generation and service time distributions. In particular, we ask whether determinacy, i.e. periodic generation of update packets and/or deterministic service, optimizes AoI. By considering several queueing systems, we show that in certain settings, deterministic service can in fact result in the worst case AoI, while a heavy-tailed distributed service can yield the minimum AoI. This leads to an interesting conclusion that, in some queueing systems, the service time distribution that minimizes expected packet delay, or variance in packet delay can, in fact, result in the worst case AoI. This exposes a fundamental difference between AoI metrics and packet delay. △ Less

Submitted 14 January, 2019; v1 submitted 10 October, 2018; originally announced October 2018.

Low-latency Networking: Where Latency Lurks and How to Tame It

Authors: Xiaolin Jiang, Hossein S. Ghadikolaei, Gabor Fodor, Eytan Modiano, Zhibo Pang, Michele Zorzi, Carlo Fischione

Abstract: While the current generation of mobile and fixed communication networks has been standardized for mobile broadband services, the next generation is driven by the vision of the Internet of Things and mission critical communication services requiring latency in the order of milliseconds or sub-milliseconds. However, these new stringent requirements have a large technical impact on the design of all… ▽ More While the current generation of mobile and fixed communication networks has been standardized for mobile broadband services, the next generation is driven by the vision of the Internet of Things and mission critical communication services requiring latency in the order of milliseconds or sub-milliseconds. However, these new stringent requirements have a large technical impact on the design of all layers of the communication protocol stack. The cross layer interactions are complex due to the multiple design principles and technologies that contribute to the layers' design and fundamental performance limitations. We will be able to develop low-latency networks only if we address the problem of these complex interactions from the new point of view of sub-milliseconds latency. In this article, we propose a holistic analysis and classification of the main design principles and enabling technologies that will make it possible to deploy low-latency wireless communication networks. We argue that these design principles and enabling technologies must be carefully orchestrated to meet the stringent requirements and to manage the inherent trade-offs between low latency and traditional performance metrics. We also review currently ongoing standardization activities in prominent standards associations, and discuss open problems for future research. △ Less

Submitted 6 August, 2018; originally announced August 2018.

Data-driven Localization and Estimation of Disturbance in the Interconnected Power System

Authors: Hyang-Won Lee, Jianan Zhang, Eytan Modiano

Abstract: Identifying the location of a disturbance and its magnitude is an important component for stable operation of power systems. We study the problem of localizing and estimating a disturbance in the interconnected power system. We take a model-free approach to this problem by using frequency data from generators. Specifically, we develop a logistic regression based method for localization and a linea… ▽ More Identifying the location of a disturbance and its magnitude is an important component for stable operation of power systems. We study the problem of localizing and estimating a disturbance in the interconnected power system. We take a model-free approach to this problem by using frequency data from generators. Specifically, we develop a logistic regression based method for localization and a linear regression based method for estimation of the magnitude of disturbance. Our model-free approach does not require the knowledge of system parameters such as inertia constants and topology, and is shown to achieve highly accurate localization and estimation performance even in the presence of measurement noise and missing data. △ Less

Submitted 4 June, 2018; originally announced June 2018.

Optimal Control of Distributed Computing Networks with Mixed-Cast Traffic Flows

Authors: Jianan Zhang, Abhishek Sinha, Jaime Llorca, Antonia Tulino, Eytan Modiano

Abstract: Distributed computing networks, tasked with both packet transmission and processing, require the joint optimization of communication and computation resources. We develop a dynamic control policy that determines both routes and processing locations for packets upon their arrival at a distributed computing network. The proposed policy, referred to as Universal Computing Network Control (UCNC), guar… ▽ More Distributed computing networks, tasked with both packet transmission and processing, require the joint optimization of communication and computation resources. We develop a dynamic control policy that determines both routes and processing locations for packets upon their arrival at a distributed computing network. The proposed policy, referred to as Universal Computing Network Control (UCNC), guarantees that packets i) are processed by a specified chain of service functions, ii) follow cycle-free routes between consecutive functions, and iii) are delivered to their corresponding set of destinations via proper packet duplications. UCNC is shown to be throughput-optimal for any mix of unicast and multicast traffic, and is the first throughput-optimal policy for non-unicast traffic in distributed computing networks with both communication and computation constraints. Moreover, simulation results suggest that UCNC yields substantially lower average packet delay compared with existing control policies for unicast traffic. △ Less

Submitted 25 June, 2019; v1 submitted 26 May, 2018; originally announced May 2018.

Joint Frequency Regulation and Economic Dispatch Using Limited Communication

Authors: Jianan Zhang, Eytan Modiano

Abstract: We study the performance of a decentralized integral control scheme for joint power grid frequency regulation and economic dispatch. We show that by properly designing the controller gains, after a power flow perturbation, the control achieves near-optimal economic dispatch while recovering the nominal frequency, without requiring any communication. We quantify the gap between the controllable pow… ▽ More We study the performance of a decentralized integral control scheme for joint power grid frequency regulation and economic dispatch. We show that by properly designing the controller gains, after a power flow perturbation, the control achieves near-optimal economic dispatch while recovering the nominal frequency, without requiring any communication. We quantify the gap between the controllable power generation cost under the decentralized control scheme and the optimal cost, based on the DC power flow model. Moreover, we study the tradeoff between the cost and the convergence time, by adjusting parameters of the control scheme. Communication between generators reduces the convergence time. We identify key communication links whose failures have more significant impacts on the performance of a distributed power grid control scheme that requires information exchange between neighbors. △ Less

Submitted 6 September, 2018; v1 submitted 26 May, 2018; originally announced May 2018.

Scheduling Policies for Age Minimization in Wireless Networks with Unknown Channel State

Authors: Rajat Talak, Igor Kadota, Sertac Karaman, Eytan Modiano

Abstract: Age of information (AoI) is a recently proposed metric that measures the time elapsed since the generation of the last received information update. We consider the problem of AoI minimization for a network under general interference constraints, and time varying channel. We study the case where the channel statistics are known, but the current channel state is unknown. We propose two scheduling po… ▽ More Age of information (AoI) is a recently proposed metric that measures the time elapsed since the generation of the last received information update. We consider the problem of AoI minimization for a network under general interference constraints, and time varying channel. We study the case where the channel statistics are known, but the current channel state is unknown. We propose two scheduling policies, namely, the virtual queue based policy and age-based policy. In the virtual queue based policy, the scheduler schedules links with maximum weighted sum of the virtual queue lengths, while in the age-based policy, the scheduler schedules links with maximum weighted sum of a function of link AoI. We prove that the virtual queue based policy is peak age optimal, up to an additive constant, while the age-based policy is at most factor 4 away from the optimal age. Numerical results suggest that both the proposed policies are, in fact, very close to the optimal. △ Less

Submitted 15 May, 2018; originally announced May 2018.

Comments: arXiv admin note: substantial text overlap with arXiv:1803.06471

Optimizing Age of Information in Wireless Networks with Perfect Channel State Information

Authors: Rajat Talak, Sertac Karaman, Eytan Modiano

Abstract: Age of information (AoI), defined as the time elapsed since the last received update was generated, is a newly proposed metric to measure the timeliness of information updates in a network. We consider AoI minimization problem for a network with general interference constraints, and time varying channels. We propose two policies, namely, virtual-queue based policy and age-based policy when the cha… ▽ More Age of information (AoI), defined as the time elapsed since the last received update was generated, is a newly proposed metric to measure the timeliness of information updates in a network. We consider AoI minimization problem for a network with general interference constraints, and time varying channels. We propose two policies, namely, virtual-queue based policy and age-based policy when the channel state is available to the network scheduler at each time step. We prove that the virtual-queue based policy is nearly optimal, up to a constant additive factor, and the age-based policy is at-most factor 4 away from optimality. Comparing with our previous work, which derived age optimal policies when channel state information is not available to the scheduler, we demonstrate a 4 fold improvement in age due to the availability of channel state information. △ Less

Submitted 17 March, 2018; originally announced March 2018.

Distributed Scheduling Algorithms for Optimizing Information Freshness in Wireless Networks

Authors: Rajat Talak, Sertac Karaman, Eytan Modiano

Abstract: Age of Information (AoI), measures the time elapsed since the last received information packet was generated at the source. We consider the problem of AoI minimization for single-hop flows in a wireless network, under pairwise interference constraints and time varying channel. We consider simple, yet broad, class of distributed scheduling policies, in which a transmission is attempted over each li… ▽ More Age of Information (AoI), measures the time elapsed since the last received information packet was generated at the source. We consider the problem of AoI minimization for single-hop flows in a wireless network, under pairwise interference constraints and time varying channel. We consider simple, yet broad, class of distributed scheduling policies, in which a transmission is attempted over each link with a certain attempt probability. We obtain an interesting relation between the optimal attempt probability and the optimal AoI of the link, and its neighboring links. We then show that the optimal attempt probabilities can be computed by solving a convex optimization problem, which can be done distributively. △ Less

Submitted 17 March, 2018; originally announced March 2018.

Optimizing Information Freshness in Wireless Networks under General Interference Constraints

Authors: Rajat Talak, Sertac Karaman, Eytan Modiano

Abstract: Age of information (AoI) is a recently proposed metric for measuring information freshness. AoI measures the time that elapsed since the last received update was generated. We consider the problem of minimizing average and peak AoI in a wireless networks, consisting of a set of source-destination links, under general interference constraints. When fresh information is always available for transmis… ▽ More Age of information (AoI) is a recently proposed metric for measuring information freshness. AoI measures the time that elapsed since the last received update was generated. We consider the problem of minimizing average and peak AoI in a wireless networks, consisting of a set of source-destination links, under general interference constraints. When fresh information is always available for transmission, we show that a stationary scheduling policy is peak age optimal. We also prove that this policy achieves average age that is within a factor of two of the optimal average age. In the case where fresh information is not always available, and packet/information generation rate has to be controlled along with scheduling links for transmission, we prove an important separation principle: the optimal scheduling policy can be designed assuming fresh information, and independently, the packet generation rate control can be done by ignoring interference. Peak and average AoI for discrete time G/Ber/1 queue is analyzed for the first time, which may be of independent interest. △ Less

Submitted 9 December, 2019; v1 submitted 17 March, 2018; originally announced March 2018.

Network Utility Maximization with Heterogeneous Traffic Flows

Authors: Abhishek Sinha, Eytan Modiano

Abstract: We consider the Network Utility Maximization (NUM) problem for wireless networks in the presence of arbitrary types of flows, including unicast, broadcast, multicast, and anycast traffic. Building upon the recent framework of a universal control policy (UMW), we design a utility optimal cross-layer admission control, routing and scheduling policy, called UMW+. The UMW+ policy takes packet level ac… ▽ More We consider the Network Utility Maximization (NUM) problem for wireless networks in the presence of arbitrary types of flows, including unicast, broadcast, multicast, and anycast traffic. Building upon the recent framework of a universal control policy (UMW), we design a utility optimal cross-layer admission control, routing and scheduling policy, called UMW+. The UMW+ policy takes packet level actions based on a precedence-relaxed virtual network. Using Lyapunov optimization techniques, we show that UMW+ maximizes network utility, while simultaneously kee** the physical queues in the network stable. Extensive simulation results validate the performance of UMW+; demonstrating both optimal utility performance and bounded average queue occupancy. Moreover, we establish a precise one-to-one correspondence between the dynamics of the virtual queues under the UMW+ policy, and the dynamics of the dual variables of an associated offline NUM program, under a subgradient algorithm. This correspondence sheds further insight into our understanding of UMW+. △ Less

Submitted 28 February, 2018; originally announced March 2018.

Comments: To appear in WiOpt 2018, Shanghai, China

Accelerated Primal-Dual Policy Optimization for Safe Reinforcement Learning

Authors: Qingkai Liang, Fanyu Que, Eytan Modiano

Abstract: Constrained Markov Decision Process (CMDP) is a natural framework for reinforcement learning tasks with safety constraints, where agents learn a policy that maximizes the long-term reward while satisfying the constraints on the long-term cost. A canonical approach for solving CMDPs is the primal-dual method which updates parameters in primal and dual spaces in turn. Existing methods for CMDPs only… ▽ More Constrained Markov Decision Process (CMDP) is a natural framework for reinforcement learning tasks with safety constraints, where agents learn a policy that maximizes the long-term reward while satisfying the constraints on the long-term cost. A canonical approach for solving CMDPs is the primal-dual method which updates parameters in primal and dual spaces in turn. Existing methods for CMDPs only use on-policy data for dual updates, which results in sample inefficiency and slow convergence. In this paper, we propose a policy search method for CMDPs called Accelerated Primal-Dual Optimization (APDO), which incorporates an off-policy trained dual variable in the dual update procedure while updating the policy in primal space with on-policy likelihood ratio gradient. Experimental results on a simulated robot locomotion task show that APDO achieves better sample efficiency and faster convergence than state-of-the-art approaches for CMDPs. △ Less

Submitted 18 February, 2018; originally announced February 2018.

Scheduling Policies for Minimizing Age of Information in Broadcast Wireless Networks

Authors: Igor Kadota, Abhishek Sinha, Elif Uysal-Biyikoglu, Rahul Singh, Eytan Modiano

Abstract: We consider a wireless broadcast network with a base station sending time-sensitive information to a number of clients through unreliable channels. The Age of Information (AoI), namely the amount of time that elapsed since the most recently delivered packet was generated, captures the freshness of the information. We formulate a discrete-time decision problem to find a transmission scheduling poli… ▽ More We consider a wireless broadcast network with a base station sending time-sensitive information to a number of clients through unreliable channels. The Age of Information (AoI), namely the amount of time that elapsed since the most recently delivered packet was generated, captures the freshness of the information. We formulate a discrete-time decision problem to find a transmission scheduling policy that minimizes the expected weighted sum AoI of the clients in the network. We first show that in symmetric networks a Greedy policy, which transmits the packet with highest current age, is optimal. For general networks, we develop three low-complexity scheduling policies: a randomized policy, a Max-Weight policy and a Whittle's Index policy, and derive performance guarantees as a function of the network configuration. To the best of our knowledge, this is the first work to derive performance guarantees for scheduling policies that attempt to minimize AoI in wireless networks with unreliable channels. Numerical results show that both Max-Weight and Whittle's Index policies outperform the other scheduling policies in every configuration simulated, and achieve near optimal performance. △ Less

Submitted 5 January, 2018; originally announced January 2018.

Network Utility Maximization in Adversarial Environments

Authors: Qingkai Liang, Eytan Modiano

Abstract: Stochastic models have been dominant in network optimization theory for over two decades, due to their analytical tractability. However, these models fail to capture non-stationary or even adversarial network dynamics which are of increasing importance for modeling the behavior of networks under malicious attacks or characterizing short-term transient behavior. In this paper, we consider the netwo… ▽ More Stochastic models have been dominant in network optimization theory for over two decades, due to their analytical tractability. However, these models fail to capture non-stationary or even adversarial network dynamics which are of increasing importance for modeling the behavior of networks under malicious attacks or characterizing short-term transient behavior. In this paper, we consider the network utility maximization problem in adversarial network settings. In particular, we focus on the tradeoffs between total queue length and utility regret which measures the difference in network utility between a causal policy and an "oracle" that knows the future within a finite time horizon. Two adversarial network models are developed to characterize the adversary's behavior. We provide lower bounds on the tradeoff between utility regret and queue length under these adversarial models, and analyze the performance of two control policies (i.e., the Drift-plus-Penalty algorithm and the Tracking Algorithm). △ Less

Submitted 27 December, 2017; v1 submitted 22 December, 2017; originally announced December 2017.

Scheduling Algorithms for Minimizing Age of Information in Wireless Broadcast Networks with Random Arrivals: The No-Buffer Case

Authors: Yu-Pin Hsu, Eytan Modiano, Lingjie Duan

Abstract: Age of information is a new network performance metric that captures the freshness of information at end-users. This paper studies the age of information from a scheduling perspective. To that end, we consider a wireless broadcast network where a base-station (BS) is updating many users on random information arrivals under a transmission capacity constraint. For the offline case when the arrival s… ▽ More Age of information is a new network performance metric that captures the freshness of information at end-users. This paper studies the age of information from a scheduling perspective. To that end, we consider a wireless broadcast network where a base-station (BS) is updating many users on random information arrivals under a transmission capacity constraint. For the offline case when the arrival statistics are known to the BS, we develop a structural MDP scheduling algorithm and an index scheduling algorithm, leveraging Markov decision process (MDP) techniques and the Whittle's methodology for restless bandits. By exploring optimal structural results, we not only reduce the computational complexity of the MDP-based algorithm, but also simplify deriving a closed form of the Whittle index. Moreover, for the online case, we develop an MDP-based online scheduling algorithm and an index-based online scheduling algorithm. Both the structural MDP scheduling algorithm and the MDP-based online scheduling algorithm asymptotically minimize the average age, while the index scheduling algorithm minimizes the average age when the information arrival rates for all users are the same. Finally, the algorithms are validated via extensive numerical studies. △ Less

Submitted 16 September, 2018; v1 submitted 20 December, 2017; originally announced December 2017.

Comments: 15 pages (double column), submitted for possible journal publication