ReLLa: Retrieval-enhanced Large Language Models for Lifelong
Sequential Behavior Comprehension in Recommendation

Zero-shot recommendation implies that a model is directly employed for the target recommendation task without any tuning on the in-domain training data. Apparently, traditional recommendation models are incapable of accomplishing zero-shot recommendation tasks, since they are randomly initialized. However, LLMs possess a vast volume of open-world knowledge and logical reasoning abilities, which enable them to infer the user’s preference towards a certain target item based on the profile of user/item.

Few-shot recommendation refers to low-resource scenarios with $N$ training data samples. $N$ denotes the number of shots, which is a relatively small number. This highly requires the data efficiency characteristic of an algorithm to fully exploit the limited number of training samples to achieve better recommendation performance.

Extending from the definition of few-shot recommendation, we can therefore define full-shot recommendation as the setting where we train the model based on the entire training set.

For LLMs, we need to convert each data sample $x_{i}$ into textual sentences $x_{i}^{text}$ via hard prompt templates. Similarly, the binary label $y_{i}\in\{0,1\}$ is transformed into a pair of binary key answer words $y_{i}^{text}\in\{\text{``Yes''},\text{``No''}\}$. We give an illustrative example of the input-output pair $(x_{i}^{text},y_{i}^{text})$ in Figure 2, where $x_{i}^{text}$ contains the descriptive texts for user profile, user behavior sequence, target item and task description, respectively.

Notably, the predominant factor that determines the length of context is derived from the user behavior sequence, the length of which can varies from tens to hundreds. For each input $x_{i}$, we truncate the user behavior sequence to length $K$. For example, the length of behavior sequence in Figure 2 is $K=4$. While the common sequential CTR prediction settings usually truncate and adopt the most recent $K$ behaviors, ReLLa propose to conduct semantic user behavior retrieval to construct textual inputs with the most relevant $K$ behaviors towards the target item.

The large language model takes as input the discrete tokens of $x_{i}^{text}$, and generate the next token $\hat{y}_{i}^{text}$ as the output, the process of which can be formulated as follows:

where $V$ is the vocabulary size, and $\hat{y}_{i}^{text}$ is the next predicted token sampled from the probability distribution $p_{i}$.

However, CTR prediction requires the model to do pointwise scoring, and the output should be floating-point number $\hat{y}_{i}\in[0,1]$ instead of a discrete token $\hat{y}_{i}^{text}$. Therefore, following previous works (Bao et al., 2023; Zhang and Wang, 2023), we intercept the estimated scores $s_{i}\in\mathbb{R}^{V}$, and conduct a bidimensional softmax over the corresponding scores of the binary key answer words. Suppose the vocabulary indices for “Yes” and “No” are $a$ and $b$, respectively. The pointwise scoring of LLMs for CTR prediction can be written as:

It is worth noting that such an estimated click-through rate $\hat{y}_{i}$ is only leveraged for evaluation on the testing set. We preserve the common instruction tuning and causal language modeling paradigm for LLMs if training is involved.

In the ReLLa framework, we develop two key techniques for LLMs in zero-shot and few-shot recommendations, respectively.

For zero-shot recommendation, as illustrated in Figure 3, we propose to conduct semantic user behavior retrieval (SUBR) to improve the data quality of data samples. We first leverage the large language model to obtain the semantic vectors for each item. Then, for each textual data sample $x_{i}^{text}$, we retrieve the most semantically relevant $K$ behaviors, which can substitute the original most recent $K$ behaviors.

For few-shot recommendation, as shown in Figure 5, we propose to perform retrieval-enhanced instruction tuning (ReiT) to promote the ability of LLMs to extract useful information from long behavior sequences. Notably, the semantic user behavior retrieval (SUBR) is adopted as the data augmentation technique to form the mixed training dataset. The mixture of both original and retrieval-enhanced data samples introduces more variety and patterns in the training set, thus increasing the robustness and generalization ability of LLMs for lifelong sequential behavior comprehension.

Although ReLLa is tuned in few-shot settings, we would like to again emphasize that other recommendation baseline models are trained in full-shot settings with the entire training set.

In zero-shot settings, the parameters of LLMs cannot be tuned according to the in-domain training samples. Hence, as shown in Figure 3, semantic user behavior retrieval (SUBR) aims to improve the quality of each sample by replacing the simply truncated most recent $K$ behaviors with the most semantically relevant $K$ behaviors towards the target item. As suggested in previous works (Qin et al., 2020; Pi et al., 2020), the retrieved user behaviors can denoise the user history and convey more clear and essential user interests for the target item, while preserving the original length of user sequence as the model input.

Firstly, we conduct semantic item encoding to obtain the semantic vector for each item. For the $t$-th item in the pool, a descriptive text is constructed via hard prompt template (an example is given in Figure 4, and is then fed through LLM. We perform average pooling over all the hidden states from the last layer of LLM, resulting in a vector $u_{t}\in\mathbb{R}^{D}$, where $D$ is the hidden size of LLM (e.g., 4096 for Vicuna-7B, and 5120 for Vicuna-13B). A principal component analysis (PCA) (Shlens, 2014) module is further employed for both dimension reduction and denoising purposes, engendering the final semantic representation $v_{t}\in\mathbb{R}^{d}$, where we set $d=512$. Now we can measure the semantic relevance between each pair of items via the cosine similarity between their corresponding semantic representations.

Next, we can apply semantic user behavior retrieval on each testing sample to replace the original truncated top-$K$ recent behaviors with the top-$K$ semantically relevant behaviors towards the target item. In this way, we obtain a parallel retrieval-enhanced testing dataset with higher data quality, while kee** the length of input context roughly unchanged. Therefore, SUBR can improve the zero-shot recommendation performance, and mitigate the incomprehension problem on long user behavior sequences.

As for few-shot recommendation, we denote the training dataset as $\{(x_{i}^{text},y_{i}^{text})\}_{i=1}^{N}$, where $N$ is the number of shots (i.e., training samples). While previous works (Bao et al., 2023; Zhang et al., 2023b) directly employ instruction tuning for LLMs over the converted textual input-output pairs, we argue that simple instruction tuning could potentially expose large language models to risks of overfitting and catastrophic forgetting on limited number of training data (Korbak et al., 2022; Ramasesh et al., 2021).

To this end, we propose a novel retrieval-enhanced instruction tuning (ReiT), where semantic user behavior retrieval (SUBR) is adopted as the data augmentation technique to construct a mixed training dataset with enriched user behavior patterns. As shown in Figure 5, we apply SUBR on each training data to obtain its retrieval-enhanced counterpart $\tilde{x}_{i}^{text}$. Next, we merge the original and retrieval-enhanced data instances to construct a mixed training dataset with total $2N$ samples. Finally, we conduct instruction tuning for LLMs on the mixed training data. The pattern enrichment brought by SUBR can regularize and prevent the large language model from overfitting, thus promoting its robustness and generalization ability to effectively extract essential knowledge from a long user behavior sequence of length $K$.

We leverage the causal language modeling objective for instruction tuning to retain the original model structure:

where $\Theta$ is the parameter of LLM, $\mathcal{M}$ is the mixed training dataset with total $2N$ data samples, $y_{j}$ is the $j$-th token of the textual output $y$, and $y_{<j}$ denotes the tokens before $y_{j}$. There is no randomly initialized prediction layer appended upon LLM for CTR prediction with binary cross-entropy (BCE) loss. The CTR estimation method for pointwise scoring with LLMs discussed in Section 2.3 is only used for evaluation on the testing set.

While we maintain a mixed training dataset for instruction tuning, the testing set contains pure retrieval-enhanced data samples generated by SUBR, which is the same as zero-shot recommendation as described in Section 3.2. Moreover, we provide further discussion about ReiT to address readers’ possible concerns as follows:

• •

  Will ReiT cause the inconsistency between the training and testing data? Data augmentation is a common regularization technique, especially for low-resource few-shot settings in computer vision (CV) (Berthelot et al., 2019; Zhang et al., 2017) or natural language processing (NLP) (Li et al., 2022; Feng et al., 2021). The inconsistency would not exist, as long as the augmentation algorithm is sound and reasonable.

• •

  Which factor actually contribute to the performance improvement of ReiT? The doubled training samples, or the pattern enrichment? Both factors can lead to the final performance enhancement, but we argue that the pattern enrichment as regularization is a more important factor for model robustness. Empirical studies are provided in Section 4.5 to ablate and decouple these two factors.

We evaluate the performance of ReLLa in comparison to existing baseline models, and report the results in Table 2. Note that other recommendation baseline models are all trained in full-shot settings with the entire training set. We set the length of user behavior sequence $K$ to 60/30/30 for BookCrossing/MovieLens-1M/MovieLens-25M, respectively.

For zero-shot recommendation, we observe that:

• •

  The performance of Vicuna-7B is notably inferior to its 13B version on all three datasets. It demonstrates that a larger LLM possesses more excellent language comprehension and logical reasoning abilities, therefore leading to better zero-shot inference capability for user preference.

• •

  ReLLa significantly outperforms Vicuna-13B for all three metrics on BookCrossing and MovieLens-1M datasets. Although the AUC performance of ReLLa degenerates on MovieLens-25M, ReLLa attains significant improvements in terms of pointwise metrics (i.e., Log Loss and ACC). such phenomena validate the effectiveness of SUBR in reducing the difficulty for LLMs to extract useful information from user behavior sequences. Also, the AUC degeneration of AUC on MovieLens-25M reveals the potential instability of zero-shot LLMs for recommendation.

As for full-shot and few-shot settings, we can draw the following observations from Table 2:

• •

  SIM achieves the best performance among all the baseline models. SIM applies user behavior retrieval to reduce the noise of user sequences, which is essentially beneficial for CTR prediction. Besides, LM-based CTR models (i.e., CTR-BERT, PTab, P5) perform worse than most of the ID-based traditional CTR models, which is consistent with the results reported in (Li et al., 2023a; Rajput et al., 2023). These LM-based methods only incorporate small language models (e.g., BERT (Berthelot et al., 2019), T5 (Raffel et al., 2020)) for pure text-based recommendation, and therefore result in inferior performance.

• •

  ReLLa (few-shot) generally achieves significantly better performance over all the baseline models, except for few cases, which validates the effectiveness of our proposed retrieval-enhanced instruction tuning (ReiT). It is worth noting that ReLLa only utilizes less than 10% training samples for finetuning, while other baseline models are trained on the entire training set, e.g., $N=65,536$ for ReLLa and $N=19,349,912$ for SIM on MovieLens-25M dataset. This demonstrates the superior data efficiency of ReLLa for sequential recommendation tasks.

We vary the length of user behavior sequence $K$ to investigate its impact on CTR prediction performance, which can demonstrate the comprehension ability of a model towards user behavior sequences. Three different models, including SIM (full-shot), Vicuna-13B (zero-shot) and ReLLa (few-shot), are evaluated with different $K$s. On BookCrossing dataset, $K$ ranges in $\{10,20,30,40,50,60\}$, while on MovieLens-1M and MovieLens-25M datasets, $K$ ranges in $\{5,10,15,20,25,30\}$. The numbers of shots are set to $256$, $8192$, $8192$ for BookCrossing, MovieLens-1M, and MovieLens-25M, respectively (i.e., ¡1% few-shot setting). The results are shown in Figure 6, from which we obtain the following observations:

• •

  As a traditional CTR prediction model, SIM (full-shot) (Pi et al., 2020) enjoys steady performance improvement as the length $K$ grows. This is consistent with our common understanding, where longer user behavior sequences can introduce more useful information to better accomplish the recommendation tasks.

• •

  However, the performance of Vicuna-13B (zero-shot) only arrives at the peak with $K=30/15/15$ on BookCrossing/MovieLens-1M/MovieLens-25M datasets, and then starts to decrease with further longer sequence. It is worth noting that the number of involved tokens (i.e., around 500/700/700 for three datasets respectively) is actually far from reaching the context limitation of Vicuna-13B (i.e., 2048 tokens). This indicates that it is non-trivial for LLMs to comprehend the textual context of long behavior sequences for recommendation, where a certain amount of in-domain knowledge is required.

• •

  ReLLa mitigates the incomprehension problem of LLMs on long user behavior sequences for recommendation. Compared with Vicuna-13B (zero-shot), whose performance drops when $K>30$ on BookCrossing and $K>15$ on MovieLens-1M and MovieLens-25M, there are no performance turning points for ReLLa. Similar to SIM, the AUC performance of ReLLa achieves continuous improvement as $K$ grows, validating the comprehension ability of ReLLa for the textual contexts with longer behavior sequences.

Focusing on few-shot settings, we investigate the data efficiency property by varying the number of shots $N$. In Figure 7, we report the AUC performance of ReLLa and SIM (the best full-shot baseline) with different $N$s. For BookCrossing dataset, $N$ ranges in $\{128,256,512,1024,2048,4096\}$. For MovieLens-1M and MovieLens-25M datasets, $N$ ranges in $\{512,1024,2048,4096,8192,65536\}$. The length of user behavior sequence $K$ is set to $K=60/30/30$ for BookCrossing/MovieLens-1M/MovieLens-25M datasets, respectively.

As depicted in Figure 7, both ReLLa and SIM attain performance enhancement as the number of shots $N$ gradually grows. However, with the same number of shots $N$, ReLLa can outperform SIM significantly and consistently by a large margin. Moreover, when $N$ is extremely small (e.g., 128 and 256) on BookCrossing dataset, SIM even fails to accomplish the CTR prediction task where AUC is merely around 0.5. With limited number of training samples, ReLLa shows remarkable data efficiency property and display considerable few-shot inference ability due to the intrinsic logical reasoning abilities and possession of open-world knowledge of LLMs.

To analyze the efficacy of each component in our proposed ReLLa framework, we design the following model variants of ReLLa. We set $N=256/8192/8192$ (¡1% setting) and $K=60/30/30$ for BookCross-ing/MovieLens-1M/MovieLens-25M datasets, respectively.

• •

  ReLLa (Ours) is the complete version of our proposed method. The training data consists of both original and retrieval-enhanced samples, resulting in a mixed training dataset of $2N$ samples. The testing set only contains pure retrieval-enhanced samples.

• •

  ReLLa (w/o Mixture). We only maintain the retrieval-enhanced data instances to construct the training dataset of $N$ samples. The testing data is still all retrieval-enhanced samples.

• •

  ReLLa (w/o Retrieval). We remove the semantic user behavior retrieval for both training and testing samples. That is, training and testing data are all original samples without retrieval enhancements. The training set contains $N$ training samples. This variant indicates the vanilla instruction tuning version over Vicuna-13B, which is similar to TALLRec (Bao et al., 2023).

• •

  ReLLa ($\frac{1}{2}N$-shot). We halve the number of shots $N$ to $\frac{1}{2}N$, i.e., from 256 to 128 on BookCrossing and from 8192 to 4096 on MovieLens-1M and MovieLens-25M. Therefore, the constructed mixed training set contains $N$ training samples. This variant is intended to decouple and investigate the factors of doubled training samples and pattern enrichment.

• •

  ReLLa (w/o IT). We remove the instruction tuning, while preserving the retrieval-enhanced samples for testing data. This variant indicates the zero-shot version of our proposed ReLLa.

• •

  ReLLa (w/o IT & Retrieval). We remove both the instruction tuning and retrieval operation. Therefore, the testing data only contains original data samples. This variant indicates the zero-shot version of vanilla Vicuna-13B.

The performance of these variants are presented in Table 3, from which we can draw the following observations:

• •

  For ReLLa (w/o Mixture) and ReLLa (w/o Retrieval), their training and testing data comprise exactly the same type of samples, i.e., either pure original samples or retrieval-enhanced samples respectively, which indicates that there is no data inconsistency between the training and testing phases. Nevertheless, both of them significantly underperform our proposed ReLLa by at least 1.12%, 0.99% and 1.95% on BookCrossing, MovieLens-1M and MovieLens-25M in AUC respectively. This highlights the importance of the data mixture strategy, the benefits of which can be broken down into two prominent factors: doubled training samples and pattern enrichment. Doubled training samples lead to a more thorough training process, while pattern enrichment can prevent the model from overfitting and therefore increase the model robustness.

• •

  We introduce the variant ReLLa ($\frac{1}{2}N$-shot) to further decouple and analyze the two factors mentioned above, i.e., doubled training samples and pattern enrichment. Its total number of training samples is the same as those of ReLLa (w/o Mixture) and ReLLa (w/o Retrieval), except that ReLLa ($\frac{1}{2}N$-shot) loses the sight of half truly training instances. In this case, ReLLa ($\frac{1}{2}N$-shot) still outperforms ReLLa (w/o Mixture) and ReLLa (w/o Retrieval) with 0.21%, 0.16% and 0.48% relative AUC improvement, and achieves comparable or better performance in Log Loss and ACC. This indicates that pattern enrichment as regularization plays a more vital role that contributes to the performance improvement.

• •

  Finally, comparing ReLLa (w/o IT) and ReLLa (w/o IT & Retrieval), which fall back into zero-shot settings, we can observe that ReLLa (w/o IT) generally achieves significant improvements over ReLLa (w/o IT & Retrieval), except for the AUC metric on MovieLens-25M. This demonstrates that semantic user behavior retrieval (SUBR) improves the quality of data samples and makes the filtered behavior sequence more friendly for LLM to extract useful knowledge.

In this section, we conduct case study to further analyze how can ReLLa help LLM better understand the long user behavior sequence. As shown in Figure 8, we select a testing sample from MovieLens-25M dataset, and visualize the attention scores of target item over the user behavior sequence at the last hidden layer of three different models (i.e., Vicuna-13B, ReLLa (zero-shot), and ReLLa (few-shot)). The attention score for each historical item is computed by summing up the attention scores of every word token for the textual input of the corresponding item. In Figure 8, each historical item is represented as a rectangle with color ranging from yellow to green. The deeper green a rectangle possesses, the large attention score the corresponding historical item attains, thus contributing more to the final CTR estimation.

For Vicuna-13B (zero-shot), the largest attentions fall on the movie Roman Holiday and Warrior, which have little relationship with the target movie Thor: Ragnarok, and thus the model fails to correctly infer the user’s preference towards the target item. Equipped with semantic user behavior retrieval (SUBR), we can reduce the noise of user behavior sequence and bring in more relevant items. As shown in Figure 8, ReLLa (zero-shot) is able to put more attentions to superhero movies (e.g., Iron Man 3) that are semantically similar to the target item. However there are still outliers for ReLLa (zero-shot), e.g., the movie Kick-Ass 2 is generally non-correlated to Thor: Ragnarok produced the Marvel. Next, by further applying retrieval-enhanced instruction tuning (ReiT), we can observe that the large attention weights of ReLLa (few-shot) all fall onto relevant superhero movies that are also produced by the Marvel. Therefore, we can conclude that our proposed SUBR and ReiT can help LLM to correctly grasp the correlation between the target item and historical items, thus better comprehending the user behavior sequence.

CTR prediction serves as the key component for various online applications (e.g., recommender systems (Xi et al., 2023a), advertising (Ou et al., 2023), and web search (Lin et al., 2021; Fu et al., 2023a; Dai et al., 2021)). It aims to accurately estimate the user’s click probability towards a certain target item in a given context (Zhang et al., 2021b). Traditional CTR prediction models can be mainly classified into two categories: (1) feature interaction based models, and (2) sequential recommendation models.

The feature interaction based models generally derive from POLY2 (Chang et al., 2010) and FM (Rendle, 2010). Their core idea is to capture the second- or high-order feature interaction patterns across multiple feature fields with different operators (e.g., product (Qu et al., 2016; Wang et al., 2021; Guo et al., 2017; Chen et al., 2021), convolution (Xin et al., 2019; Liu et al., 2019), and attention (Song et al., 2019; Xiao et al., 2017)). For examples, DCN (Wang et al., 2017), xDeepFM (Lian et al., 2018), and DCNv2 (Wang et al., 2021) apply product-based feature crossing operation at each layer for explicit high-order feature interaction modeling. AutoInt (Song et al., 2019) and InterHAt (Li et al., 2020) adopt the attention mechanism for feature interactions, which provides additional explainable prediction via attention weights.

The sequential recommendation model (Zhou et al., 2019; Pi et al., 2019; Zhou et al., 2018) focuses on user behavior modeling and seeks to dynamically capture users’ interests towards a target item according to the given behavior history. They leverage different architectures (e.g., RNN (Hidasi and Karatzoglou, 2018; Hidasi et al., 2016), CNN (Tang and Wang, 2018), attention (Zhou et al., 2019, 2018), memory bank (Pi et al., 2019; Ren et al., 2019)) to handle the user behavior sequence for user preference modeling. For instances, GRU4Rec (Hidasi et al., 2016) adopt the gated recurrent unit (GRU) (Chung et al., 2014) to encode the user’s sequential behaviors. Caser (Tang and Wang, 2018) introduces the convolution neural network (CNN) to model the union-level patterns among user behavior sequences.

As suggested in previous work (Lin et al., 2023a), the adaption of language models to the field of recommender systems can be generally categorized according to the roles they serve in the recommendation pipeline, i.e., feature engineering (Liu et al., 2023a; Borisov et al., 2022; Li et al., 2023c; Mysore et al., 2023; Carranza et al., 2023; Christakopoulou et al., 2023), feature encoder (Muhamed et al., 2021b; Hou et al., 2022; Yu et al., 2021; Wang et al., 2022b; Hou et al., 2023a; Zhang et al., 2022; Fu et al., 2023b; Yuan et al., 2023; Qiu et al., 2021; Li et al., 2023b), scoring/ranking function (Liu et al., 2022; Kang et al., 2023; Zhang et al., 2021a; Li et al., 2023e; Bao et al., 2023; Li et al., 2023d; Zhang and Wang, 2023; Mao et al., 2023; Hua et al., 2023a; Geng et al., 2023; Hua et al., 2023b; Zhang et al., 2023a; Hou et al., 2023b; Chen, 2023; Petrov and Macdonald, 2023; Wang and Lim, 2023).

For feature engineering, large language models (LLMs) accept the raw data (e.g., user profiles and item descriptions) as input, and generate supplementary text-based attributes as data augmentations with delicately designed prompts and templates. For example, KAR (Xi et al., 2023b) utilizes the reasoning knowledge on user preferences and the factual knowledge on items by requesting LLMs with factorization prompting techniques. The obtained knowledge can serve as augmented features and promote the recommendation performance in a model-agnostic manner. GENRE (Liu et al., 2023a) employs LLMs to obtain news summarization, synthetic news pieces, and user profiles.

For feature encoder, LLMs are adopted as auxiliary textual feature encoders to (1) enrich the user/item representations with semantic information, and (2) enable cross-domain recommendation with the natural language interface. For instance, U-BERT (Qiu et al., 2021) enhances the user representation by encoding review texts into dense vectors via BERT. UniSRec (Hou et al., 2022) and VQ-Rec (Hou et al., 2023a) apply a fixed BERT as the encoder for item descriptive texts, in order to achieve unified cross-domain sequential recommendation.

For scoring/ranking function, researchers explore the potential of LLMs to directly serve as the core scoring or ranking module for recommendation, instead of an assistant role for conventional recommendation models (e.g., feature engineering or feature encoder). In this case, LLMs are employed to accomplish either the item scoring task (Liu et al., 2022; Kang et al., 2023; Zhang et al., 2021a; Li et al., 2023e; Bao et al., 2023; Li et al., 2023d; Zhang and Wang, 2023; Mao et al., 2023), or item generation task (Hua et al., 2023a; Geng et al., 2023; Hua et al., 2023b; Zhang et al., 2023a; Hou et al., 2023b; Chen, 2023; Petrov and Macdonald, 2023; Wang and Lim, 2023). Also, various works (Geng et al., 2022b; Cui et al., 2022; Zhang et al., 2023b; Liu et al., 2023b; Sun et al., 2023; Dai et al., 2023) attempt to utilize the multi-task capacity of LLMs, and instruct LLMs to solve the multiple tasks (e.g., both scoring and generation) through a unified language interface.

In this paper, we mainly focus on the utilization of LLMs as the scoring/ranking functions, where the pointwise scoring task is adopted for CTR prediction. To the best of our knowledge, we are the first to identify and well formulate the incomprehension problem of LLMs on lifelong user behavior sequences when adopting LLMs for scoring and ranking tasks. A novel ReLLa framework is proposed to mitigate such an issue by introducing the retrieval techniques to promote comprehension ability of LLMs and thus enhance their recommendation performance.