Showing 1–1 of 1 results for author: Sergienya, I

Distributional Models and Deep Learning Embeddings: Combining the Best of Both Worlds

Authors: Irina Sergienya, Hinrich Schütze

Abstract: There are two main approaches to the distributed representation of words: low-dimensional deep learning embeddings and high-dimensional distributional models, in which each dimension corresponds to a context word. In this paper, we combine these two approaches by learning embeddings based on distributional-model vectors - as opposed to one-hot vectors as is standardly done in deep learning. We sho… ▽ More There are two main approaches to the distributed representation of words: low-dimensional deep learning embeddings and high-dimensional distributional models, in which each dimension corresponds to a context word. In this paper, we combine these two approaches by learning embeddings based on distributional-model vectors - as opposed to one-hot vectors as is standardly done in deep learning. We show that the combined approach has better performance on a word relatedness judgment task. △ Less

Submitted 18 February, 2014; v1 submitted 19 December, 2013; originally announced December 2013.

Comments: 4 pages, 1 table, ICLR Workshop; main experimental table was extended with more experimental results; related word added

ACM Class: I.2.6; I.2.7