-
Distributed Reconfigurable Intelligent Surfaces for Energy Efficient Indoor Terahertz Wireless Communications
Authors:
Yiming Huo,
Xiaodai Dong,
Nuwan Ferdinand
Abstract:
With the fifth-generation (5G) networks widely commercialized and fast deployed, the sixth-generation (6G) wireless communication is envisioned to provide competitive quality of service (QoS) in multiple aspects to global users. The critical and underlying research of the 6G is, firstly, highly dependent on the precise modeling and characterization of the wireless propagation when the spectrum is…
▽ More
With the fifth-generation (5G) networks widely commercialized and fast deployed, the sixth-generation (6G) wireless communication is envisioned to provide competitive quality of service (QoS) in multiple aspects to global users. The critical and underlying research of the 6G is, firstly, highly dependent on the precise modeling and characterization of the wireless propagation when the spectrum is believed to expand to the terahertz (THz) domain. Moreover, future networks' power consumption and energy efficiency are critical factors to consider. In this research, based on a review of the fundamental mechanisms of reconfigurable intelligent surface (RIS) assisted wireless communications, we utilize the 3D ray-tracing method to analyze a realistic indoor THz propagation environment with the existence of human blockers. Furthermore, we propose a distributed RISs framework (DRF) to assist the indoor THz wireless communication to achieve overall energy efficiency. The numerical analysis of simulation results based on more than 2,900 indoor THz wireless communication sub-scenarios has demonstrated the significant efficacy of applying distributed RISs to overcome the mobile human blockage issue, improve the THz signal coverage, increase signal-to-noise ratios (SNRs), and QoS. With practical hardware design constraints investigated, we eventually envision how to utilize the existing integrated sensing and communication techniques to deploy and operate such a system in reality. Such a distributed RISs framework can also lay the foundation of efficient THz communications for Internet-of-Things (IoT) networks.
△ Less
Submitted 12 October, 2022;
originally announced October 2022.
-
Anytime MiniBatch: Exploiting Stragglers in Online Distributed Optimization
Authors:
Nuwan Ferdinand,
Haider Al-Lawati,
Stark C. Draper,
Matthew Nokleby
Abstract:
Distributed optimization is vital in solving large-scale machine learning problems. A widely-shared feature of distributed optimization techniques is the requirement that all nodes complete their assigned tasks in each computational epoch before the system can proceed to the next epoch. In such settings, slow nodes, called stragglers, can greatly slow progress. To mitigate the impact of stragglers…
▽ More
Distributed optimization is vital in solving large-scale machine learning problems. A widely-shared feature of distributed optimization techniques is the requirement that all nodes complete their assigned tasks in each computational epoch before the system can proceed to the next epoch. In such settings, slow nodes, called stragglers, can greatly slow progress. To mitigate the impact of stragglers, we propose an online distributed optimization method called Anytime Minibatch. In this approach, all nodes are given a fixed time to compute the gradients of as many data samples as possible. The result is a variable per-node minibatch size. Workers then get a fixed communication time to average their minibatch gradients via several rounds of consensus, which are then used to update primal variables via dual averaging. Anytime Minibatch prevents stragglers from holding up the system without wasting the work that stragglers can complete. We present a convergence analysis and analyze the wall time performance. Our numerical results show that our approach is up to 1.5 times faster in Amazon EC2 and it is up to five times faster when there is greater variability in compute node performance.
△ Less
Submitted 10 June, 2020;
originally announced June 2020.
-
Hierarchical Coded Matrix Multiplication
Authors:
Shahrzad Kiani,
Nuwan Ferdinand,
Stark C. Draper
Abstract:
In distributed computing systems slow working nodes, known as stragglers, can greatly extend finishing times. Coded computing is a technique that enables straggler-resistant computation. Most coded computing techniques presented to date provide robustness by ensuring that the time to finish depends only on a set of the fastest nodes. However, while stragglers do compute less work than non-straggle…
▽ More
In distributed computing systems slow working nodes, known as stragglers, can greatly extend finishing times. Coded computing is a technique that enables straggler-resistant computation. Most coded computing techniques presented to date provide robustness by ensuring that the time to finish depends only on a set of the fastest nodes. However, while stragglers do compute less work than non-stragglers, in real-world commercial cloud computing systems (e.g., Amazon's Elastic Compute Cloud (EC2)) the distinction is often a soft one. In this paper, we develop hierarchical coded computing that exploits the work completed by all nodes, both fast and slow, automatically integrating the potential contribution of each. We first present a conceptual framework to represent the division of work amongst nodes in coded matrix multiplication as a cuboid partitioning problem. This framework allows us to unify existing methods and motivates new techniques. We then develop three methods of hierarchical coded computing that we term bit-interleaved coded computation (BICC), multilevel coded computation (MLCC), and hybrid hierarchical coded computation (HHCC). In this paradigm, each worker is tasked with completing a sequence (a hierarchy) of ordered subtasks. The sequence of subtasks, and the complexity of each, is designed so that partial work completed by stragglers can be used, rather than ignored. We note that our methods can be used in conjunction with any coded computing method. We illustrate this by showing how we can use our methods to accelerate all previously developed coded computing techniques by enabling them to exploit stragglers. Under a widely studied statistical model of completion time, our approach realizes a $66\%$ improvement in the expected finishing time. On Amazon EC2, the gain was $27\%$ when stragglers are simulated.
△ Less
Submitted 30 January, 2021; v1 submitted 14 December, 2019;
originally announced December 2019.
-
Cuboid Partitioning for Hierarchical Coded Matrix Multiplication
Authors:
Shahrzad Kiani,
Nuwan Ferdinand,
Stark C. Draper
Abstract:
Coded matrix multiplication is a technique to enable straggler-resistant multiplication of large matrices in distributed computing systems. In this paper, we first present a conceptual framework to represent the division of work amongst processors in coded matrix multiplication as a cuboid partitioning problem. This framework allows us to unify existing methods and motivates new techniques. Buildi…
▽ More
Coded matrix multiplication is a technique to enable straggler-resistant multiplication of large matrices in distributed computing systems. In this paper, we first present a conceptual framework to represent the division of work amongst processors in coded matrix multiplication as a cuboid partitioning problem. This framework allows us to unify existing methods and motivates new techniques. Building on this framework, we apply the idea of hierarchical coding (Ferdinand & Draper, 2018) to coded matrix multiplication. The hierarchical scheme we develop is able to exploit the work completed by all processors (fast and slow), rather than ignoring the slow ones, even if the amount of work completed by stragglers is much less than that completed by the fastest workers. On Amazon EC2, we achieve a 37% improvement in average finishing time compared to non-hierarchical schemes.
△ Less
Submitted 20 July, 2019;
originally announced July 2019.
-
Hierarchical Coded Matrix Multiplication
Authors:
Shahrzad Kiani,
Nuwan Ferdinand,
Stark C. Draper
Abstract:
Slow working nodes, known as stragglers, can greatly reduce the speed of distributed computation. Coded matrix multiplication is a recently introduced technique that enables straggler-resistant distributed multiplication of large matrices. A key property is that the finishing time depends only on the work completed by a set of the fastest workers, while the work done by the slowest workers is igno…
▽ More
Slow working nodes, known as stragglers, can greatly reduce the speed of distributed computation. Coded matrix multiplication is a recently introduced technique that enables straggler-resistant distributed multiplication of large matrices. A key property is that the finishing time depends only on the work completed by a set of the fastest workers, while the work done by the slowest workers is ignored completely. This paper is motivated by the observation that in real-world commercial cloud computing systems such as Amazon's Elastic Compute Cloud (EC2) the distinction between fast and slow nodes is often a soft one. Thus, if we could also exploit the work completed by stragglers we may realize substantial performance gains. To realize such gains, in this paper we use the idea of hierarchical coding (Ferdinand and Draper, IEEE Int. Symp. Inf. Theory, 2018). We decompose the overall matrix multiplication task into a hierarchy of heterogeneously sized subtasks. The duty to complete each subtask is shared amongst all workers and each subtask is (generally) of a different complexity. The motivation for the hierarchical decomposition is the recognition that more workers will finish the first subtask than the second (or third, forth, etc.). Connecting to error correction coding, earlier subtasks can therefore be designed to be of a higher rate than later subtasks. Through this hierarchical design our scheme exploits the work completed by stragglers, rather than ignoring it, even if that amount is much less than that completed by the fastest workers. We numerically show that our method realizes a 60% improvement in the expected finishing time for a widely studied statistical model of the speed of computation and, on Amazon EC2, the gain is 35%.
△ Less
Submitted 20 July, 2019;
originally announced July 2019.
-
Anytime Stochastic Gradient Descent: A Time to Hear from all the Workers
Authors:
Nuwan Ferdinand,
Stark Draper
Abstract:
In this paper, we focus on approaches to parallelizing stochastic gradient descent (SGD) wherein data is farmed out to a set of workers, the results of which, after a number of updates, are then combined at a central master node. Although such synchronized SGD approaches parallelize well in idealized computing environments, they often fail to realize their promised computational acceleration in pr…
▽ More
In this paper, we focus on approaches to parallelizing stochastic gradient descent (SGD) wherein data is farmed out to a set of workers, the results of which, after a number of updates, are then combined at a central master node. Although such synchronized SGD approaches parallelize well in idealized computing environments, they often fail to realize their promised computational acceleration in practical settings. One cause is slow workers, termed stragglers, who can cause the fusion step at the master node to stall, which greatly slowing convergence. In many straggler mitigation approaches work completed by these nodes, while only partial, is discarded completely. In this paper, we propose an approach to parallelizing synchronous SGD that exploits the work completed by all workers. The central idea is to fix the computation time of each worker and then to combine distinct contributions of all workers. We provide a convergence analysis and optimize the combination function. Our numerical results demonstrate an improvement of several factors of magnitude in comparison to existing methods.
△ Less
Submitted 6 October, 2018;
originally announced October 2018.
-
Exploitation of Stragglers in Coded Computation
Authors:
Shahrzad Kiani,
Nuwan Ferdinand,
Stark C. Draper
Abstract:
In cloud computing systems slow processing nodes, often referred to as "stragglers", can significantly extend the computation time. Recent results have shown that error correction coding can be used to reduce the effect of stragglers. In this work we introduce a scheme that, in addition to using error correction to distribute mixed jobs across nodes, is also able to exploit the work completed by a…
▽ More
In cloud computing systems slow processing nodes, often referred to as "stragglers", can significantly extend the computation time. Recent results have shown that error correction coding can be used to reduce the effect of stragglers. In this work we introduce a scheme that, in addition to using error correction to distribute mixed jobs across nodes, is also able to exploit the work completed by all nodes, including stragglers. We first consider vector-matrix multiplication and apply maximum distance separable (MDS) codes to small blocks of sub-matrices. The worker nodes process blocks sequentially, working block-by-block, transmitting partial per-block results to the master as they are completed. Sub-blocking allows a more continuous completion process, which thereby allows us to exploit the work of a much broader spectrum of processors and reduces computation time. We then apply this technique to matrix-matrix multiplication using product code. In this case, we show that the order of computing sub-tasks is a new degree of design freedom that can be exploited to reduce computation time further. We propose a novel approach to analyze the finishing time, which is different from typical order statistics. Simulation results show that the expected computation time decreases by a factor of at least two in compared to previous methods.
△ Less
Submitted 26 June, 2018;
originally announced June 2018.
-
Hierarchical Coded Computation
Authors:
Nuwan Ferdinand,
Stark Draper
Abstract:
Coded computation is a method to mitigate "stragglers" in distributed computing systems through the use of error correction coding that has lately received significant attention. First used in vector-matrix multiplication, the range of application was later extended to include matrix-matrix multiplication, heterogeneous networks, convolution, and approximate computing. A drawback to previous resul…
▽ More
Coded computation is a method to mitigate "stragglers" in distributed computing systems through the use of error correction coding that has lately received significant attention. First used in vector-matrix multiplication, the range of application was later extended to include matrix-matrix multiplication, heterogeneous networks, convolution, and approximate computing. A drawback to previous results is they completely ignore work completed by stragglers. While stragglers are slower compute nodes, in many settings the amount of work completed by stragglers can be non-negligible. Thus, in this work, we propose a hierarchical coded computation method that exploits the work completed by all compute nodes. We partition each node's computation into layers of sub-computations such that each layer can be treated as (distinct) erasure channel. We then design different erasure codes for each layer so that all layers have the same failure exponent. We propose design guidelines to optimize parameters of such codes. Numerical results show the proposed scheme has an improvement of a factor of 1.5 in the expected finishing time compared to previous work.
△ Less
Submitted 26 June, 2018;
originally announced June 2018.
-
Low-Dimensional Sha** for High-Dimensional Lattice Codes
Authors:
Nuwan S. Ferdinand,
Brian M. Kurkoski,
Matthew Nokleby,
Behnaam Aazhang
Abstract:
We propose two low-complexity lattice code constructions that have competitive coding and sha** gains. The first construction, named systematic Voronoi sha**, maps short blocks of integers to the dithered Voronoi integers, which are dithered integers that are uniformly distributed over the Voronoi region of a low-dimensional sha** lattice. Then, these dithered Voronoi integers are encoded us…
▽ More
We propose two low-complexity lattice code constructions that have competitive coding and sha** gains. The first construction, named systematic Voronoi sha**, maps short blocks of integers to the dithered Voronoi integers, which are dithered integers that are uniformly distributed over the Voronoi region of a low-dimensional sha** lattice. Then, these dithered Voronoi integers are encoded using a high-dimensional lattice retaining the same sha** and coding gains of low and high-dimensional lattices. A drawback to this construction is that there is no isomorphism between the underlying message and the lattice code, preventing its use in applications such as compute-and- forward. Therefore we propose a second construction, called mixed nested lattice codes, in which a high-dimensional coding lattice is nested inside a concatenation of low-dimensional sha** lattices. This construction not only retains the same sha**/coding gains as first construction but also provides the desired algebraic structure. We numerically study these methods, for point-to-point channels as well as compute-and-forward using low-density lattice codes (LDLCs) as coding lattices and E8 and Barnes-Wall as sha** lattices. Numerical results indicate a sha** gain of up to 0.86 dB, compared to the state-of-the-art of 0.4 dB; furthermore, the proposed method has lower complexity than state-of-the-art approaches.
△ Less
Submitted 3 August, 2016;
originally announced August 2016.
