-
Towards Deep Application-Network Integration: Architectures, Progress and Opportunities
Authors:
Berta Serracanta,
Kai Gao,
Jordi Ros-Giralt,
Alberto Rodriguez-Natal,
Luis M. Contreras,
Richard Yang,
Albert Cabellos
Abstract:
With the rise of a new generation of applications (e.g., virtual and augmented reality, artificial intelligence, etc) demanding stringent performance requirements, the need for networking solutions and architectures that can enable a higher Quality of Experience (QoE) is becoming increasingly important.
While jointly optimizing application and network may increase the applications' QoE and simul…
▽ More
With the rise of a new generation of applications (e.g., virtual and augmented reality, artificial intelligence, etc) demanding stringent performance requirements, the need for networking solutions and architectures that can enable a higher Quality of Experience (QoE) is becoming increasingly important.
While jointly optimizing application and network may increase the applications' QoE and simultaneously improve the utilization of network resources, such a paradigm has had limited success in real production networks. However, with the combination of revolutionary trends in (1) compute processing demands, (2) networking capabilities, and (3) sustainable business models, it is high time the community explores the full potential of deeper integration between application and network.
In this paper, recent trends observed over the past few years are systematically reviewed. These include the paradigm shift in modern communication services towards computing-driven applications, such as on-site AI training, advances in programmable network technologies like Software Defined Networking (SDN), and new business models incentivizing collaboration and cooperation between parties. Following this, successful scenarios that benefit from various forms of deeper network-application integration are reported, highlighting their considerable potential. A unified framework is then introduced, providing an overview of possible architecture paradigms for network-application integration and bringing awareness to existing abstractions, mechanisms, tools, and their potential combinations. The paper concludes with a discussion of several remaining challenges in building practical network-application integrated systems.
△ Less
Submitted 18 June, 2024;
originally announced June 2024.
-
A Quantitative Theory of Bottleneck Structures for Data Networks
Authors:
Jordi Ros-Giralt,
Noah Amsel,
Sruthi Yellamraju,
James Ezick,
Richard Lethin,
Yuang Jiang,
Aosong Feng,
Leandros Tassiulas
Abstract:
The conventional view of the congestion control problem in data networks is based on the principle that a flow's performance is uniquely determined by the state of its bottleneck link, regardless of the topological properties of the network. However, recent work has shown that the behavior of congestion-controlled networks is better explained by models that account for the interactions between bot…
▽ More
The conventional view of the congestion control problem in data networks is based on the principle that a flow's performance is uniquely determined by the state of its bottleneck link, regardless of the topological properties of the network. However, recent work has shown that the behavior of congestion-controlled networks is better explained by models that account for the interactions between bottleneck links. These interactions are captured by a latent \textit{bottleneck structure}, a model describing the complex ripple effects that changes in one part of the network exert on the other parts. In this paper, we present a \textit{quantitative} theory of bottleneck structures (QTBS), a mathematical and engineering framework comprising a family of polynomial-time algorithms that can be used to reason about a wide variety of network optimization problems, including routing, capacity planning and flow control. QTBS can contribute to traffic engineering by making clear predictions about the relative performance of alternative flow routes, and by providing numerical recommendations for the optimal rate settings of traffic shapers. A particularly novel result in the domain of capacity planning indicates that previously established rules for the design of folded-Clos networks are suboptimal when flows are congestion controlled. We show that QTBS can be used to derive the optimal rules for this important class of topologies, and empirically demonstrate the correctness and efficacy of these results using the BBR and Cubic congestion-control algorithms.
△ Less
Submitted 6 October, 2022;
originally announced October 2022.
-
Algorithms and Data Structures to Accelerate Network Analysis
Authors:
Jordi Ros-Giralt,
Alan Commike,
Peter Cullen,
Richard Lethin
Abstract:
As the sheer amount of computer generated data continues to grow exponentially, new bottlenecks are unveiled that require rethinking our traditional software and hardware architectures. In this paper we present five algorithms and data structures (long queue emulation, lockless bimodal queues, tail early drop**, LFN tables, and multiresolution priority queues) designed to optimize the process of…
▽ More
As the sheer amount of computer generated data continues to grow exponentially, new bottlenecks are unveiled that require rethinking our traditional software and hardware architectures. In this paper we present five algorithms and data structures (long queue emulation, lockless bimodal queues, tail early drop**, LFN tables, and multiresolution priority queues) designed to optimize the process of analyzing network traffic. We integrated these optimizations on R-Scope, a high performance network appliance that runs the Bro network analyzer, and present benchmarks showcasing performance speed ups of 5X at traffic rates of 10 Gbps.
△ Less
Submitted 17 November, 2017;
originally announced November 2017.
-
Multiresolution Priority Queues
Authors:
Jordi Ros-Giralt,
Alan Commike,
Peter Cullen,
Jeff Lucovsky,
Dilip Madathil,
Richard Lethin
Abstract:
Priority queues are container data structures essential to many high performance computing (HPC) applications. In this paper, we introduce multiresolution priority queues, a data structure that improves the performance of the standard heap based implementations by trading off a controllable amount of resolution in the space of priorities. The new data structure can reduce the worst case performanc…
▽ More
Priority queues are container data structures essential to many high performance computing (HPC) applications. In this paper, we introduce multiresolution priority queues, a data structure that improves the performance of the standard heap based implementations by trading off a controllable amount of resolution in the space of priorities. The new data structure can reduce the worst case performance of inserting an element from O(log(n)) to O(log(r)), where n is the number of elements in the queue and r is the number of resolution groups in the priority space. The worst case cost of removing the top element is O(1). When the number of elements in the table is high, the amortized cost to insert an element becomes O(1).
△ Less
Submitted 10 August, 2017; v1 submitted 26 May, 2017;
originally announced May 2017.
-
High Speed Elephant Flow Detection Under Partial Information
Authors:
Jordi Ros-Giralt,
Alan Commike,
Sourav Maji,
Malathi Veeraraghavan
Abstract:
In this paper we introduce a new framework to detect elephant flows at very high speed rates and under uncertainty. The framework provides exact mathematical formulas to compute the detection likelihood and introduces a new flow reconstruction lemma under partial information. These theoretical results lead to the design of BubbleCache, a new elephant flow detection algorithm designed to operate ne…
▽ More
In this paper we introduce a new framework to detect elephant flows at very high speed rates and under uncertainty. The framework provides exact mathematical formulas to compute the detection likelihood and introduces a new flow reconstruction lemma under partial information. These theoretical results lead to the design of BubbleCache, a new elephant flow detection algorithm designed to operate near the optimal tradeoff between computational scalability and accuracy by dynamically tracking the traffic's natural cutoff sampling rate. We demonstrate on a real world 100 Gbps network that the BubbleCache algorithm helps reduce the computational cost by a factor of 1000 and the memory requirements by a factor of 100 while detecting the top flows on the network with very high probability.
△ Less
Submitted 28 September, 2018; v1 submitted 6 January, 2017;
originally announced January 2017.
