Showing 1–7 of 7 results for author: Petrov, A V

Shallow Cross-Encoders for Low-Latency Retrieval

Authors: Aleksandr V. Petrov, Sean MacAvaney, Craig Macdonald

Abstract: Transformer-based Cross-Encoders achieve state-of-the-art effectiveness in text retrieval. However, Cross-Encoders based on large transformer models (such as BERT or T5) are computationally expensive and allow for scoring only a small number of documents within a reasonably small latency window. However, kee** search latencies low is important for user satisfaction and energy usage. In this pape… ▽ More Transformer-based Cross-Encoders achieve state-of-the-art effectiveness in text retrieval. However, Cross-Encoders based on large transformer models (such as BERT or T5) are computationally expensive and allow for scoring only a small number of documents within a reasonably small latency window. However, kee** search latencies low is important for user satisfaction and energy usage. In this paper, we show that weaker shallow transformer models (i.e., transformers with a limited number of layers) actually perform better than full-scale models when constrained to these practical low-latency settings since they can estimate the relevance of more documents in the same time budget. We further show that shallow transformers may benefit from the generalized Binary Cross-Entropy (gBCE) training scheme, which has recently demonstrated success for recommendation tasks. Our experiments with TREC Deep Learning passage ranking query sets demonstrate significant improvements in shallow and full-scale models in low-latency scenarios. For example, when the latency limit is 25ms per query, MonoBERT-Large (a cross-encoder based on a full-scale BERT model) is only able to achieve NDCG@10 of 0.431 on TREC DL 2019, while TinyBERT-gBCE (a cross-encoder based on TinyBERT trained with gBCE) reaches NDCG@10 of 0.652, a +51% gain over MonoBERT-Large. We also show that shallow Cross-Encoders are effective even when used without a GPU (e.g., with CPU inference, NDCG@10 decreases only by 3% compared to GPU inference with 50ms latency), which makes Cross-Encoders practical to run even without specialized hardware acceleration. △ Less

Submitted 29 March, 2024; originally announced March 2024.

Comments: Accepted by ECIR2024

RecJPQ: Training Large-Catalogue Sequential Recommenders

Authors: Aleksandr V. Petrov, Craig Macdonald

Abstract: Sequential Recommendation is a popular recommendation task that uses the order of user-item interaction to model evolving users' interests and sequential patterns in their behaviour. Current state-of-the-art Transformer-based models for sequential recommendation, such as BERT4Rec and SASRec, generate sequence embeddings and compute scores for catalogue items, but the increasing catalogue size make… ▽ More Sequential Recommendation is a popular recommendation task that uses the order of user-item interaction to model evolving users' interests and sequential patterns in their behaviour. Current state-of-the-art Transformer-based models for sequential recommendation, such as BERT4Rec and SASRec, generate sequence embeddings and compute scores for catalogue items, but the increasing catalogue size makes training these models costly. The Joint Product Quantisation (JPQ) method, originally proposed for passage retrieval, markedly reduces the size of the retrieval index with minimal effect on model effectiveness, by replacing passage embeddings with a limited number of shared sub-embeddings. This paper introduces RecJPQ, a novel adaptation of JPQ for sequential recommendations, which takes the place of item embeddings tensor and replaces item embeddings with a concatenation of a limited number of shared sub-embeddings and, therefore, limits the number of learnable model parameters. The main idea of RecJPQ is to split items into sub-item entities before training the main recommendation model, which is inspired by splitting words into tokens and training tokenisers in language models. We apply RecJPQ to SASRec, BERT4Rec, and GRU4rec models on three large-scale sequential datasets. Our results showed that RecJPQ could notably reduce the model size (e.g., 48% reduction for the Gowalla dataset with no effectiveness degradation). RecJPQ can also improve model performance through a regularisation effect (e.g. +0.96% NDCG@10 improvement on the Booking.com dataset). Overall, RecJPQ allows the training of state-of-the-art transformer recommenders in industrial applications, where datasets with millions of items are common. △ Less

Submitted 18 December, 2023; v1 submitted 11 December, 2023; originally announced December 2023.

Comments: Accepted by ACM WSDM 2024

Generative Sequential Recommendation with GPTRec

Authors: Aleksandr V. Petrov, Craig Macdonald

Abstract: Sequential recommendation is an important recommendation task that aims to predict the next item in a sequence. Recently, adaptations of language models, particularly Transformer-based models such as SASRec and BERT4Rec, have achieved state-of-the-art results in sequential recommendation. In these models, item ids replace tokens in the original language models. However, this approach has limitatio… ▽ More Sequential recommendation is an important recommendation task that aims to predict the next item in a sequence. Recently, adaptations of language models, particularly Transformer-based models such as SASRec and BERT4Rec, have achieved state-of-the-art results in sequential recommendation. In these models, item ids replace tokens in the original language models. However, this approach has limitations. First, the vocabulary of item ids may be many times larger than in language models. Second, the classical Top-K recommendation approach used by these models may not be optimal for complex recommendation objectives, including auxiliary objectives such as diversity, coverage or coherence. Recent progress in generative language models inspires us to revisit generative approaches to address these challenges. This paper presents the GPTRec sequential recommendation model, which is based on the GPT-2 architecture. GPTRec can address large vocabulary issues by splitting item ids into sub-id tokens using a novel SVD Tokenisation algorithm based on quantised item embeddings from an SVD decomposition of the user-item interaction matrix. The paper also presents a novel Next-K recommendation strategy, which generates recommendations item-by-item, considering already recommended items. The Next-K strategy can be used for producing complex interdependent recommendation lists. We experiment with GPTRec on the MovieLens-1M dataset and show that using sub-item tokenisation GPTRec can match the quality of SASRec while reducing the embedding table by 40%. We also show that the recommendations generated by GPTRec on MovieLens-1M using the Next-K recommendation strategy match the quality of SASRec in terms of NDCG@10, meaning that the model can serve as a strong starting point for future research. △ Less

Submitted 19 June, 2023; originally announced June 2023.

Comments: Accepted at Gen-IR@SIGIR2023 workshop

Statistical parameters of femtosecond laser pulse post-filament propagation on 65m air path with localized optical turbulence

Authors: Dmitry V. Apeksimov, Andrey V. Bulygin, Yury E. Geints, Andrey M. Kabanov, Aleksey V. Petrov, Elena E. Khoroshaeva

Abstract: High-power femtosecond laser radiation propagates nonlinearly in air exhibiting pulse self-focusing and strong multiphoton medium ionization, which leads to the spatial fragmentation of laser pulse into highly-localized light channels usually called the filaments. The filaments are characterized by high optical intensity, reduced (even zero) angular spreading and can contain laser plasma or be pla… ▽ More High-power femtosecond laser radiation propagates nonlinearly in air exhibiting pulse self-focusing and strong multiphoton medium ionization, which leads to the spatial fragmentation of laser pulse into highly-localized light channels usually called the filaments. The filaments are characterized by high optical intensity, reduced (even zero) angular spreading and can contain laser plasma or be plasmaless (postfilaments). The presence of optical turbulence on the propagation path dramatically changes pulse filamentation dynamics and in some cases causes pulse fragmentation enhancement and collapse arrest. For the first time to our knowledge, we experimentally and theoretically investigate the transverse profile of Ti:sapphire femtosecond laser radiation nonlinearly propagating a 65 m air path to the region of postfilament evolution after passing through an artificial localized air turbulence. We show that when a turbulent layer is placed before the filamentation region, the average number of high-intensive local fluence maxima ("hot points") in pulse profile as well as their sizes grow as the turbulence strength increases, and then saturates at some levels. On the contrary, the deposition of a turbulent screen within the filamentation region has almost no effect on both the number and the average diameter of the postfilaments. △ Less

Submitted 14 August, 2022; originally announced August 2022.

FA2PbBr4: synthesis, structure and unusual optical properties of two polymorphs of formamidinium-based layered (110) hybrid perovskite

Authors: Sergey A. Fateev, Andrey A. Petrov, Ekaterina I. Marchenko, Yan V. Zubavichus, Victor N. Khrustalev, Andrey V. Petrov, Sergey M. Aksenov, Eugene A. Goodilin, Alexey B. Tarasov

Abstract: Small cations such as guanidinium and cesium can act as templating cations to form low dimensional phases (2D, 1D, 0D) in the case of excess of organic halides. However, such phases with the widely used formamidinium (FA+) cation have not been reported so far. In this study, we discovered two novel low dimensional phases with a composition of FA2PbBr4 and investigated the prerequisites of their fo… ▽ More Small cations such as guanidinium and cesium can act as templating cations to form low dimensional phases (2D, 1D, 0D) in the case of excess of organic halides. However, such phases with the widely used formamidinium (FA+) cation have not been reported so far. In this study, we discovered two novel low dimensional phases with a composition of FA2PbBr4 and investigated the prerequisites of their formation upon crystallization of FABr-excessive solutions of FAPbBr3. We found that both phases have the structure of (110) layered perovskite but are represented by two different polymorphs with eclipsed and staggered arrangement of adjacent layers. It was shown that FA2PbBr4 phases usually exist in a labile equilibrium with FAPbBr3 3D perovskite and can form composites with it. The optical properties of both polymorphs were comprehensively studied by means of absorption spectroscopy, diffuse reflection spectroscopy and photoluminescence spectroscopy. DFT calculations were applied to investigate the band structure of the FA2PbBr4 and to corroborate the conclusions on their optoelectronic properties. As a result, we found that FA2PbBr4 phases irradiated by UV can exhibit effective green photoluminescence due to a transfer of excitation energy to defective states or 3D perovskite inclusions. △ Less

Submitted 29 January, 2021; originally announced February 2021.

Comments: 24 pages, 19 figures

MSC Class: 0.0 ACM Class: A.0

Database of 2D hybrid perovskite materials: open-access collection of crystal structures, band gaps and atomic partial charges predicted by machine learning

Authors: Ekaterina I. Marchenko, Sergey A. Fateev, Andrey A. Petrov, Vadim V. Korolev, Artem A. Mi-trofanov, Andrey V. Petrov, Eugene A. Goodilin, Alexey B. Tarasov

Abstract: We describe a first open-access database of experimentally investigated hybrid organic-inorganic materials with two-dimensional (2D) perovskite-like crystal structure. The database includes 515 compounds, containing 180 different organic cations, 10 metals (Pb, Sn, Bi, Cd, Cu, Fe, Ge, Mn, Pd, Sb) and 3 halogens (I, Br, Cl) known so far and will be regularly updated. The database contains a geometr… ▽ More We describe a first open-access database of experimentally investigated hybrid organic-inorganic materials with two-dimensional (2D) perovskite-like crystal structure. The database includes 515 compounds, containing 180 different organic cations, 10 metals (Pb, Sn, Bi, Cd, Cu, Fe, Ge, Mn, Pd, Sb) and 3 halogens (I, Br, Cl) known so far and will be regularly updated. The database contains a geometrical and crystal chemical analysis of the structures, which are useful to reveal quantitative structure-property relationships for this class of compounds. We show that the penetration depth of spacer organic cation into the inorganic layer and M-X-M bond angles increase in the number of inorganic layers (n). The machine learning model is developed and trained on the database, for the prediction of a band gap with accuracy within 0.1 eV. Another machine learning model is trained for the prediction of atomic partial charges with accuracy within 0.01 e. We show that the predicted values of band gaps decrease with an increase of the n and with an increase of M-X-M angles for single-layered perovskites. In general, the proposed database and machine learning models are shown to be useful tools for the rational design of new 2D hybrid perovskite materials. △ Less

Submitted 25 June, 2020; originally announced June 2020.

Kosterlitz-Thouless phase in systems of one-dimensional strongly interacting fermions

Authors: V. V. Afonin V. Yu. Petrov

Abstract: We present the ground state wave functions for systems of one-dimensional interacting fermions. It is shown that these systems undergo phase transitions similar to the Kosterlitz-Thouless one independently of the interaction details. In the limit of an infinitely strong interaction the phase transition turns into the usual second order phase transition in a chiral phase. The temperature of the p… ▽ More We present the ground state wave functions for systems of one-dimensional interacting fermions. It is shown that these systems undergo phase transitions similar to the Kosterlitz-Thouless one independently of the interaction details. In the limit of an infinitely strong interaction the phase transition turns into the usual second order phase transition in a chiral phase. The temperature of the phase transition is calculated. △ Less

Submitted 18 October, 2005; v1 submitted 14 July, 2004; originally announced July 2004.

Comments: 24 pages