Time-based Sequence Model for Personalization and Recommendation Systems
Authors:
Tigran Ishkhanov,
Maxim Naumov,
Xianjie Chen,
Yan Zhu,
Yuan Zhong,
Alisson Gusatti Azzolini,
Chonglin Sun,
Frank Jiang,
Andrey Malevich,
Liang Xiong
Abstract:
In this paper we develop a novel recommendation model that explicitly incorporates time information. The model relies on an embedding layer and TSL attention-like mechanism with inner products in different vector spaces, that can be thought of as a modification of multi-headed attention. This mechanism allows the model to efficiently treat sequences of user behavior of different length. We study t…
▽ More
In this paper we develop a novel recommendation model that explicitly incorporates time information. The model relies on an embedding layer and TSL attention-like mechanism with inner products in different vector spaces, that can be thought of as a modification of multi-headed attention. This mechanism allows the model to efficiently treat sequences of user behavior of different length. We study the properties of our state-of-the-art model on statistically designed data set. Also, we show that it outperforms more complex models with longer sequence length on the Taobao User Behavior dataset.
△ Less
Submitted 27 August, 2020;
originally announced August 2020.
Deep Learning Recommendation Model for Personalization and Recommendation Systems
Authors:
Maxim Naumov,
Dheevatsa Mudigere,
Hao-Jun Michael Shi,
Jianyu Huang,
Narayanan Sundaraman,
Jongsoo Park,
Xiaodong Wang,
Udit Gupta,
Carole-Jean Wu,
Alisson G. Azzolini,
Dmytro Dzhulgakov,
Andrey Mallevich,
Ilia Cherniavskii,
Yinghai Lu,
Raghuraman Krishnamoorthi,
Ansha Yu,
Volodymyr Kondratenko,
Stephanie Pereira,
Xianjie Chen,
Wenlin Chen,
Vijay Rao,
Bill Jia,
Liang Xiong,
Misha Smelyanskiy
Abstract:
With the advent of deep learning, neural network-based recommendation models have emerged as an important tool for tackling personalization and recommendation tasks. These networks differ significantly from other deep learning networks due to their need to handle categorical features and are not well studied or understood. In this paper, we develop a state-of-the-art deep learning recommendation m…
▽ More
With the advent of deep learning, neural network-based recommendation models have emerged as an important tool for tackling personalization and recommendation tasks. These networks differ significantly from other deep learning networks due to their need to handle categorical features and are not well studied or understood. In this paper, we develop a state-of-the-art deep learning recommendation model (DLRM) and provide its implementation in both PyTorch and Caffe2 frameworks. In addition, we design a specialized parallelization scheme utilizing model parallelism on the embedding tables to mitigate memory constraints while exploiting data parallelism to scale-out compute from the fully-connected layers. We compare DLRM against existing recommendation models and characterize its performance on the Big Basin AI platform, demonstrating its usefulness as a benchmark for future algorithmic experimentation and system co-design.
△ Less
Submitted 31 May, 2019;
originally announced June 2019.
