HammingMesh: A Network Topology for Large-Scale Deep Learning
Authors:
Torsten Hoefler,
Tommaso Bonato,
Daniele De Sensi,
Salvatore Di Girolamo,
Shigang Li,
Marco Heddes,
Jon Belk,
Deepak Goel,
Miguel Castro,
Steve Scott
Abstract:
Numerous microarchitectural optimizations unlocked tremendous processing power for deep neural networks that in turn fueled the AI revolution. With the exhaustion of such optimizations, the growth of modern AI is now gated by the performance of training systems, especially their data movement. Instead of focusing on single accelerators, we investigate data-movement characteristics of large-scale t…
▽ More
Numerous microarchitectural optimizations unlocked tremendous processing power for deep neural networks that in turn fueled the AI revolution. With the exhaustion of such optimizations, the growth of modern AI is now gated by the performance of training systems, especially their data movement. Instead of focusing on single accelerators, we investigate data-movement characteristics of large-scale training at full system scale. Based on our workload analysis, we design HammingMesh, a novel network topology that provides high bandwidth at low cost with high job scheduling flexibility. Specifically, HammingMesh can support full bandwidth and isolation to deep learning training jobs with two dimensions of parallelism. Furthermore, it also supports high global bandwidth for generic traffic. Thus, HammingMesh will power future large-scale deep learning systems with extreme bandwidth requirements.
△ Less
Submitted 21 October, 2022; v1 submitted 3 September, 2022;
originally announced September 2022.
Some undecidability results for asynchronous transducers and the Brin-Thompson group 2V
Authors:
James Belk,
Collin Bleak
Abstract:
Using a result of Kari and Ollinger, we prove that the torsion problem for elements of the Brin-Thompson group 2V is undecidable. As a result, we show that there does not exist an algorithm to determine whether an element of the rational group R of Grigorchuk, Nekrashevich, and Sushchanskii has finite order. A modification of the construction gives other undecidability results about the dynamics o…
▽ More
Using a result of Kari and Ollinger, we prove that the torsion problem for elements of the Brin-Thompson group 2V is undecidable. As a result, we show that there does not exist an algorithm to determine whether an element of the rational group R of Grigorchuk, Nekrashevich, and Sushchanskii has finite order. A modification of the construction gives other undecidability results about the dynamics of the action of elements of 2V on Cantor Space. Arzhantseva, Lafont, and Minasyanin prove in 2012 that there exists a finitely presented group with solvable word problem and unsolvable torsion problem. To our knowledge, 2V furnishes the first concrete example of such a group, and gives an example of a direct undecidability result in the extended family of R. Thompson type groups.
△ Less
Submitted 5 May, 2014;
originally announced May 2014.
