-
arXiv:1805.02867 [pdf, ps, other]
Online normalizer calculation for softmax
Abstract: The Softmax function is ubiquitous in machine learning, multiple previous works suggested faster alternatives for it. In this paper we propose a way to compute classical Softmax with fewer memory accesses and hypothesize that this reduction in memory accesses should improve Softmax performance on actual hardware. The benchmarks confirm this hypothesis: Softmax accelerates by up to 1.3x and Softmax… ▽ More
Submitted 28 July, 2018; v1 submitted 8 May, 2018; originally announced May 2018.
Comments: 1) Added link to the benchmark code, 2) Benchmarked Safe Softmax + Top-K fused and attributed part of 5x explicitly to fusion in sections 5.2 and 6, 3) Stylistic changes, 4) Minor clarifications
-
Challenges in Representation Learning: A report on three machine learning contests
Abstract: The ICML 2013 Workshop on Challenges in Representation Learning focused on three challenges: the black box learning challenge, the facial expression recognition challenge, and the multimodal learning challenge. We describe the datasets created for these challenges and summarize the results of the competitions. We provide suggestions for organizers of future challenges and some comments on what kin… ▽ More
Submitted 1 July, 2013; originally announced July 2013.
Comments: 8 pages, 2 figures