Scission: Performance-driven and Context-aware Cloud-Edge Distribution of Deep Neural Networks
Authors:
Luke Lockhart,
Paul Harvey,
Pierre Imai,
Peter Willis,
Blesson Varghese
Abstract:
Partitioning and distributing deep neural networks (DNNs) across end-devices, edge resources and the cloud has a potential twofold advantage: preserving privacy of the input data, and reducing the ingress bandwidth demand beyond the edge. However, for a given DNN, identifying the optimal partition configuration for distributing the DNN that maximizes performance is a significant challenge. This is…
▽ More
Partitioning and distributing deep neural networks (DNNs) across end-devices, edge resources and the cloud has a potential twofold advantage: preserving privacy of the input data, and reducing the ingress bandwidth demand beyond the edge. However, for a given DNN, identifying the optimal partition configuration for distributing the DNN that maximizes performance is a significant challenge. This is because the combination of potential target hardware resources that maximizes performance and the sequence of layers of the DNN that should be distributed across the target resources needs to be determined, while accounting for user-defined objectives/constraints for partitioning. This paper presents Scission, a tool for automated benchmarking of DNNs on a given set of target device, edge and cloud resources for determining optimal partitions that maximize DNN performance. The decision-making approach is context-aware by capitalizing on hardware capabilities of the target resources, their locality, the characteristics of DNN layers, and the network condition. Experimental studies are carried out on 18 DNNs. The decisions made by Scission cannot be manually made by a human given the complexity and the number of dimensions affecting the search space. The benchmarking overheads of Scission allow for responding to operational changes periodically rather than in real-time. Scission is available for public download at https://github.com/qub-blesson/Scission.
△ Less
Submitted 16 December, 2020; v1 submitted 8 August, 2020;
originally announced August 2020.
Designing Run-Time Environments to Have Predefined Global Dynamics
Authors:
Massimo Monti,
Pierre Imai,
Christian Tschudin
Abstract:
The stability and the predictability of a computer network algorithm's performance are as important as the main functional purpose of networking software. However, asserting or deriving such properties from the finite state machine implementations of protocols is hard and, except for singular cases like TCP, is not done today. In this paper, we propose to design and study run-time environments for…
▽ More
The stability and the predictability of a computer network algorithm's performance are as important as the main functional purpose of networking software. However, asserting or deriving such properties from the finite state machine implementations of protocols is hard and, except for singular cases like TCP, is not done today. In this paper, we propose to design and study run-time environments for networking protocols which inherently enforce desirable, predictable global dynamics. To this end we merge two complementary design approaches: (i) A design-time and bottom up approach that enables us to engineer algorithms based on an analyzable (reaction) flow model. (ii) A run-time and top-down approach based on an autonomous stack composition framework, which switches among implementation alternatives to find optimal operation configurations. We demonstrate the feasibility of our self-optimizing system in both simulations and real-world Internet setups.
△ Less
Submitted 6 June, 2013;
originally announced June 2013.
