Swift for TensorFlow: A portable, flexible platform for deep learning
Authors:
Brennan Saeta,
Denys Shabalin,
Marc Rasi,
Brad Larson,
Xihui Wu,
Parker Schuh,
Michelle Casbon,
Daniel Zheng,
Saleem Abdulrasool,
Aleksandr Efremov,
Dave Abrahams,
Chris Lattner,
Richard Wei
Abstract:
Swift for TensorFlow is a deep learning platform that scales from mobile devices to clusters of hardware accelerators in data centers. It combines a language-integrated automatic differentiation system and multiple Tensor implementations within a modern ahead-of-time compiled language oriented around mutable value semantics. The resulting platform has been validated through use in over 30 deep lea…
▽ More
Swift for TensorFlow is a deep learning platform that scales from mobile devices to clusters of hardware accelerators in data centers. It combines a language-integrated automatic differentiation system and multiple Tensor implementations within a modern ahead-of-time compiled language oriented around mutable value semantics. The resulting platform has been validated through use in over 30 deep learning models and has been employed across data center and mobile applications.
△ Less
Submitted 25 February, 2021;
originally announced February 2021.
Glow: Graph Lowering Compiler Techniques for Neural Networks
Authors:
Nadav Rotem,
Jordan Fix,
Saleem Abdulrasool,
Garret Catron,
Summer Deng,
Roman Dzhabarov,
Nick Gibson,
James Hegeman,
Meghan Lele,
Roman Levenstein,
Jack Montgomery,
Bert Maher,
Satish Nadathur,
Jakob Olesen,
Jongsoo Park,
Artem Rakhov,
Misha Smelyanskiy,
Man Wang
Abstract:
This paper presents the design of Glow, a machine learning compiler for heterogeneous hardware. It is a pragmatic approach to compilation that enables the generation of highly optimized code for multiple targets. Glow lowers the traditional neural network dataflow graph into a two-phase strongly-typed intermediate representation. The high-level intermediate representation allows the optimizer to p…
▽ More
This paper presents the design of Glow, a machine learning compiler for heterogeneous hardware. It is a pragmatic approach to compilation that enables the generation of highly optimized code for multiple targets. Glow lowers the traditional neural network dataflow graph into a two-phase strongly-typed intermediate representation. The high-level intermediate representation allows the optimizer to perform domain-specific optimizations. The lower-level instruction-based address-only intermediate representation allows the compiler to perform memory-related optimizations, such as instruction scheduling, static memory allocation and copy elimination. At the lowest level, the optimizer performs machine-specific code generation to take advantage of specialized hardware features. Glow features a lowering phase which enables the compiler to support a high number of input operators as well as a large number of hardware targets by eliminating the need to implement all operators on all targets. The lowering phase is designed to reduce the input space and allow new hardware backends to focus on a small number of linear algebra primitives.
△ Less
Submitted 3 April, 2019; v1 submitted 2 May, 2018;
originally announced May 2018.
