Showing 1–3 of 3 results for author: Sorokina, A

A Critique of the Smooth Inverse Frequency Sentence Embeddings

Authors: Aidana Karipbayeva, Alena Sorokina, Zhenisbek Assylbekov

Abstract: We critically review the smooth inverse frequency sentence embedding method of Arora, Liang, and Ma (2017), and show inconsistencies in its setup, derivation, and evaluation. We critically review the smooth inverse frequency sentence embedding method of Arora, Liang, and Ma (2017), and show inconsistencies in its setup, derivation, and evaluation. △ Less

Submitted 30 September, 2019; originally announced September 2019.

Comments: 2 pages, 2 figures, Abstract

Low-Rank Approximation of Matrices for PMI-based Word Embeddings

Authors: Alena Sorokina, Aidana Karipbayeva, Zhenisbek Assylbekov

Abstract: We perform an empirical evaluation of several methods of low-rank approximation in the problem of obtaining PMI-based word embeddings. All word vectors were trained on parts of a large corpus extracted from English Wikipedia (enwik9) which was divided into two equal-sized datasets, from which PMI matrices were obtained. A repeated measures design was used in assigning a method of low-rank approxim… ▽ More We perform an empirical evaluation of several methods of low-rank approximation in the problem of obtaining PMI-based word embeddings. All word vectors were trained on parts of a large corpus extracted from English Wikipedia (enwik9) which was divided into two equal-sized datasets, from which PMI matrices were obtained. A repeated measures design was used in assigning a method of low-rank approximation (SVD, NMF, QR) and dimensionality of the vectors (250, 500) to each of the PMI matrix replicates. Our experiments show that word vectors obtained from the truncated SVD achieve the best performance on two downstream tasks, similarity and analogy, compare to the other two low-rank approximation methods. △ Less

Submitted 21 September, 2019; originally announced September 2019.

Comments: 10 pages, 4 figures, CICLing 2019, Springer "Lecture Notes in Computer Science"

Shannon capacity of nonlinear regenerative channels

Authors: M. A. Sorokina, S. K. Turitsyn

Abstract: We compute Shannon capacity of nonlinear channels with regenerative elements. Conditions are found under which capacity of such nonlinear channels is higher than the Shannon capacity of the classical linear additive white Gaussian noise channel. We develop a general scheme for designing the proposed channels and apply it to the particular nonlinear sine-map**. The upper bound for regeneration ef… ▽ More We compute Shannon capacity of nonlinear channels with regenerative elements. Conditions are found under which capacity of such nonlinear channels is higher than the Shannon capacity of the classical linear additive white Gaussian noise channel. We develop a general scheme for designing the proposed channels and apply it to the particular nonlinear sine-map**. The upper bound for regeneration efficiency is found and the asymptotic behavior of the capacity in the saturation regime is derived. △ Less

Submitted 7 May, 2013; originally announced May 2013.

Comments: 5 pages, 4 figures