An empirical study of next-basket recommendations

Abstract

Next Basket Recommender Systems (NBRs) function to recommend the subsequent shop** baskets for users through the modeling of their preferences derived from purchase history, typically manifested as a sequence of historical baskets. Given their widespread applicability in the E-commerce industry, investigations into NBRs have garnered increased attention in recent years. Despite the proliferation of diverse NBR methodologies, a substantial challenge lies in the absence of a systematic and unified evaluation framework across these methodologies. Various studies frequently appraise NBR approaches using disparate datasets and diverse experimental settings, impeding a fair and effective comparative assessment of methodological performance. To bridge this gap, this study undertakes a systematic empirical inquiry into NBRs, reviewing seminal works within the domain and scrutinizing their respective merits and drawbacks. Subsequently, we implement designated NBR algorithms on uniform datasets, employing consistent experimental configurations, and assess their performances via identical metrics. This methodological rigor establishes a cohesive framework for the impartial evaluation of diverse NBR approaches. It is anticipated that this study will furnish a robust foundation and serve as a pivotal reference for forthcoming research endeavors in this dynamic field.

I Introduction

Over the years, Recommender Systems (RSs) have exhibited noteworthy advancements across diverse real-world applications, including but not limited to e-commerce[1, 2] and streaming media[3, 4, 5]. Their pivotal roles have assumed heightened significance within the contemporary Pan information age, wherein users are confronted with the imperative task of navigating through an exponentially expanding array of information resources to make informed choices. In response to this escalating need, Recommender Systems have emerged as indispensable instruments, enabling users to navigate efficiently and effectively through extensive and often redundant content, products, and services [6]. Notably, within the realm of Recommender Systems, a burgeoning area of interest in recent years pertains to next-basket recommender systems (NBRs), signifying a nuanced sub-domain that merits particular attention and investigation.

The examination of Next-Basket Recommender Systems (NBRs) is warranted due to their relevance in practical shop** scenarios. Consumers typically purchase a collection of items, known as a ”shop** basket,” during a single shop** visit, reflecting authentic shop** patterns. NBRs align with this behavior, aiming to provide recommendations for a carefully selected basket of items based on the user’s current needs and preferences. In contrast to prevalent next-item recommendation approaches that focus on the subsequent item within an ongoing basket [7, 8, 9, 10], NBRs address the more comprehensive next-basket recommendation task. This involves recommending a sequence of inter-correlated items across multiple baskets, necessitating novel theoretical frameworks. The emerging significance of Next-Basket Recommendation (NBR) underscores its importance as a research area. NBRs leverage the user’s purchasing history, analyzing sequential series of recently procured baskets to anticipate the next assortment of items. This approach models both enduring and ephemeral preferences, enhancing precision in the recommendation process.

Within the realm of user transaction records, typically manifested as a sequence of shop** baskets, the objective of Next-Basket Recommendation resides in prognosticating the subsequent basket of items a user is inclined to purchase. This is achieved through the modeling of sequential dependencies inherent in the sequence of baskets. A multitude of pertinent studies in the domain of next-basket recommendation has yielded noteworthy achievements. One such noteworthy endeavor, articulated by Rendle et al. [11], entails a Markov Chain (MC)-based approach, adept at capturing low-order dependencies across baskets to enhance next-basket recommendation. Additionally, embedding-based methodologies elucidated by Wan et al. [12] and Wang et al. [13], employ distributed representations for predicting the contents of the subsequent basket. Meanwhile, recurrent neural network (RNN) based approaches [14, 15, 16, 17], are designed to encapsulate higher-order dependencies spanning multiple baskets. Furthermore, intention-driven strategies, as proposed by Wang et al. [18, 19], seek to model the heterogeneous intentions inherent in historical sequences of purchased baskets, thereby tailoring next-basket recommendations to align with diverse user intentions. In an alternative domain, a method employing a K-nearest neighbor (KNN) framework [20], aimed to leverage personalized item frequency information for enhancing the efficacy of NBRs. Similarly, an alternative strategy [21], employed a combination of KNN and collaborative filtering (CF) to address the temporal dimension, specifically focusing on modeling the recency of items within the context of NBRs.

Despite notable successes in various studies, a discernible gap exists in articulating the latest advancements and objectively assessing the efficacy of diverse NBR methodologies. The lack of standardized problem formalization and experimental settings further complicates equitable comparisons, exacerbated by inconsistencies in dataset selection and baselines. This hampers the clarity and comparability of research outcomes in the NBR field. This research aims to address these gaps by defining the NBR problem, summarizing existing research, categorizing and comparing NBR approaches, establishing a standardized experimental setup, and conducting a systematic empirical study on selected methodologies. The study concludes by outlining directions for future NBR research, providing valuable insights for researchers in the field.

The main contributions of this work are summarized below:

• We present a novel taxonomy to categorize and organize diverse NBR approaches, offering a concise overview of current research progress in the NBR field.

• We systematically examine representative and state-of-the-art NBR approaches through a comprehensive empirical study, offering a unified evaluation for comparison of their performance. This aims to provide valuable insights for future research in this dynamic field.

• This work represents the inaugural systematic exploration of NBR approaches, incorporating comprehensive evaluation metrics in the existing literature.

II RELATED WORK

II-A Related Surveys

Numerous studies have explored next-basket recommendation, encompassing session-based and sequential recommendation. Wang et al. conducted a thorough survey [6, 22], defining research problems, highlighting challenges, and delineating progress and future directions in session-based recommendation. Ludewig et al. [23] performed an empirical study, comprehensively comparing session-based recommendation algorithms. In the realm of sequential recommendation, Wang et al. [24] outlined key challenges, progress, and future directions, while Fang et al. [25] summarized and compared deep learning-based approaches for sequential recommendation. While session-based recommendation and sequential recommendation share relevance, they diverge significantly from next-basket recommendation. Session-based and sequential recommendation operate on session and sequence data, respectively, where input comprises sets or sequences of items. Notably, there is an absence of a clear basket structure within these data types. They primarily focus on recommending the next item within the same basket by modeling intra-basket dependencies. In contrast, next-basket recommendation utilizes a sequence of baskets as input, aiming to recommend the subsequent basket by modeling inter-basket dependencies.

Extensive reviews and empirical studies have been conducted on session-based recommendation and sequential recommendation areas[26, 27, 28]. However, there is a notable absence of systematic investigations offering a comprehensive overview of studies in the NBR domain, as well as a lack of unified and rigorous evaluations of various NBR algorithms. Given the increasing popularity of NBR and the evolving research landscape, there is an urgent need for a systematic and thorough examination, summarization, and evaluation of NBR approaches. This paper represents the inaugural effort towards addressing this gap, focusing on delineating the problem statement, categorizing existing NBR approaches, providing benchmarking evaluations, and offering insights into future prospects in the field.

II-B Next Basket Recommendation vs. Bundle Recommendation

Numerous studies explore tasks related to NBR, with bundle recommendation being a prominent example [29]. Despite apparent similarities, distinctions in settings and assumptions exist between next-basket recommendation and bundle recommendation. The propensity for researchers to conflate these concepts underscores the importance of a comparative analysis to delineate the unique characteristics of next-basket recommendation as distinct from bundle recommendation. Next basket recommendation and bundle recommendation are contingent upon distinct datasets: basket data and bundle data. To establish a clear understanding, it is imperative to delineate the disparity between these datasets. A basket denotes a collection of items acquired by a user during a single shop** visit. Typically, there is no discernible order among the items within a basket, and their correlation or lack thereof depends on specific circumstances. In sequences of baskets, sorting commonly occurs in ascending order with respect to shop** visits. A bundle refers to the amalgamation of two or more closely related items [30]. These items, within a bundle, are typically unordered and exhibit similarities or complementarity [31]. When conceptualizing bundles as a sequence, bundle recommendations commonly involve organizing them in descending order based on their respective prices.

Recommendation systems often address the task of suggesting the next set of items for a user by framing it as a sequential prediction problem. This involves predicting a user’s future purchases based on their historical transaction data, leveraging the modeled sequential relationships among their past baskets. For instance, if a user buys a printer, recommendations for complementary items like printing paper and toners can be generated for their subsequent basket. In contrast, bundle recommendation, exemplified by [32], is posed as an optimization problem. It seeks to choose an optimal and correlated set of items from a diverse pool to meet a specific consumption goal. Unlike sequential prediction, bundle recommendation does not explicitly consider or model sequential dependencies. For example, a user with a preference for Apple products might receive a bundle recommendation for iPhones and AirPods, disregarding any sequential purchase patterns.

III PROBLEM STATEMENT

Diverse definitions of Next-Basket Recommendation (NBR) tasks emerge across various domains within the literature. For instance, in the realm of E-commerce, a basket is construed not only as a compilation of products procured in a single transaction but also as a collection of places visited during a trip [16, 33] within the tourism domain. To streamline terminology, we categorize both products and places as ”items” in this study. Consequently, we define a ”basket” as an assemblage of items collectively purchased in a single transaction event by a specific user. In the context of the next-basket recommendation task, the objective typically involves constructing and training a recommender system based on a user’s recently acquired sequence of historical baskets. This system aims to predict the subsequent basket of items that are likely to pique the user’s interest by modeling inter-basket dependencies. Typically, this prediction takes the form of generating a personalized, ranked list of items (refer to Figure 1), representing the anticipated contents of the next basket for the given user.

Figure 1: A running example for next basket recommendation.

The research problem addressed in this study, as delineated in [18], pertains to the formal definition of the next-basket recommendation task. The dataset under consideration, denoted as $D={s_{1},...,s_{|D|}}$ with $|D|$ representing the total number of sequences, comprises sequences of shop** baskets, hereafter referred to as ”baskets.” Each sequence $s={b_{1},...,b_{|s|}}(s\in D)$ within $D$ represents the historical purchasing behavior of a specific user, arranged chronologically. In this context, a basket $b={v_{1},...,v_{|b|}}(b\in s)$ contains items purchased in a single transaction event. The amalgamation of all items across the entire dataset forms the universal item set $V={v_{1},...,v_{|V|}}$, while the universal user set is denoted as $U={u_{1},...,u_{|U|}}$. Figure 1 provides an illustrative example for next basket recommendation. Given a sequence of baskets $s={b_{1},...,b_{t}}$, the target basket for prediction is typically the last one, $b_{t}$. The preceding baskets, denoted as the context $C_{t}={b_{1},...,b_{t-1}}$, serve as the historical information used in the prediction and recommendation stage. In the next-basket recommendation task, the objective is to predict the items most likely to appear in the user’s next basket, denoted as $b_{t}$, based on the context $C_{t}$ and the user’s purchasing history. Formally, this task is expressed as:

$$b_{t}=f(C_{t},u).$$ (1)

IV Classes and Comparison of NBR Approaches

Figure 2: The classes of NBR approaches.

IV-A Categorization of NBR Approaches.

The literature delineates three primary classes of NBR approaches, as illustrated in Figure 2. These classes are: (1) Conventional NBR approaches, grounded in traditional data mining techniques like pattern mining and K-nearest neighbor (KNN); (2) Latent representation approaches, primarily utilizing representation learning techniques, such as latent embedding; and (3) Deep neural network approaches, typically constructed on deep learning models like recurrent neural networks (RNN). Further granularity is achieved through subdivision, yielding four sub-classes for conventional NBR (pattern mining, KNN, Markov chain-based approaches), one sub-class for latent representation (distributed representation), and two sub-classes for deep neural network approaches (basic and advanced models). The basic deep neural network sub-class encompasses recurrent neural network-based NBR, while the advanced models sub-class further divides into attention models and contrastive learning. Notably, hybrid approaches amalgamate multiple algorithms/models; for instance, ANAM [17] integrates recurrent neural networks and attention models, and CLEA [1] combines a contrastive learning framework with a recurrent neural network.

IV-B A Comparison of Different Classes of NBR Approaches

Traditional NBR approaches, such as those employing pattern mining [34], KNN models [20], and Markov chain models [11], are generally straightforward and efficient due to their reliance on conventional models and algorithms. These methods are particularly suited for simple datasets with less complex dependencies within or between baskets. Despite their simplicity, some, like TIFUKNN [20], a KNN-based approach, can outperform deep learning-based methods such as Dream and Beacon across various metrics on experimental datasets (cf. TABLE I). In contrast, deep neural network-based approaches are inherently sophisticated, employing diverse neural architectures to capture intrinsic features and dependencies within and between baskets. While these approaches may be more time-consuming and less stable in certain cases, they excel at capturing complex intra- and inter-basket dependencies, leading to superior recommendation performance. For instance, recurrent neural networks (RNNs) have proven effective in handling basket sequences in recent years [14, 17, 15, 16]. Latent representation-based approaches, in contrast, tend to be more efficient and simpler than deep learning-based methods, as they do not employ complex neural architectures. However, due to the power of embedding techniques, they can effectively capture implicit and hidden patterns in data, resulting in enhanced performance. An illustrative example is HRM [13], a distributed representation-based NBR approach utilizing a three-layer structure to construct a user’s last basket representation for predicting their next basket.

V Datasets and Compared Approaches

V-A Datasets and Data Pre-processing

Upon analyzing the datasets in the collected papers, two significant issues have been identified. Firstly, certain datasets are not publicly available due to user privacy limitations. Secondly, there are instances where datasets with the same name exist in different versions. Considering the popularity and characteristics of these datasets, we have chosen three commonly utilized datasets for our experiments in evaluating next-basket prediction performance. They are TaFeng, Instacart and Dunnhumby. For data pre-processing, the removal of infrequent items and inactive users with minimal interactions constitutes a crucial step in this process. An examination of pertinent literature reveals a lack of uniformity in the pre-processing strategies employed by various studies. To ensure equitable performance assessment across diverse datasets, we uniformly pre-process by filtering out items purchased less than $n$ times, where $n$ assumes values of 10, 20, and 17 for the TaFeng, Instacart, and Dunnhumby datasets, respectively.

TABLE I: The statistics of experimental datasets.

Statistics	TaFeng	Instacart	Dunnhumby
#Users	20,212	19,982	22,530
#Baskets	105,140	280,941	214,861
#Items	10,411	13,400	3,920
#Basket/user	5.20	14.06	9.53
#Items/basket	6.32	9.61	7.45

The rows #Users, #Baskets, #Items, #Basket/user, #Items/basket correspond to the number of users, the number of baskets over all users, the number of items, the average number of baskets per user and the average number of items per basket, respectively.

V-B Compared Approaches and Evaluation Metrics

Within our paper compilation, diverse approaches are employed to ensure both reproducibility and popularity. Eight selected approaches for experimentation include TOP, FPMC [11], DREAM [14], Beacon [16], Sets2Sets [15], UP-CF@r [21], TIFUKNN [20], and CLEA [1]. For instance, Sets2Sets is tailored for sequential set prediction using an encoder-decoder framework, specifically adapted for NBR. In addition, to holistically assess various approaches, we employ seven metrics: Recall, Precision, F1-Score, Person-wise Hit Ratio (PHR), Normalized Discounted Cumulative Gain (NDCG), Mean Average Precision (MAP), and Mean Reciprocal Rank (MRR). It is noteworthy that the latter two metrics, namely MAP and MRR, are prevalent in top-$K$ recommendation despite their absence in the collected papers. Adhering to the conventions established in traditional NBR work [20, 1], each approach is evaluated by recommending a predicted basket (i.e., recommendation list) of a predetermined size $K$. Metrics are uniformly calculated across all predicted baskets, with a common attribute: superior performance is indicated by larger values.

TABLE II: The statistics of compared approaches.

VI Performance of Compared Approaches

Finding 1: Conventional approaches, such as TOP, FPMC, UP-CF@r, and TIFUKNN, exhibit remarkable performance. TOP, the simplest approach, often outperforms FPMC in terms of Recall, MAP, MRR, and NDCG across most datasets, emphasizing the significance of item frequency information in next-basket recommendation. This underscores the utility of TOP as a baseline for evaluating new models [35]. On Instacart and Dunnhumby datasets, UP-CF@r and TIFUKNN consistently demonstrate superior performance across all metrics, leveraging item attributes and a user-based nearest neighbor approach. Despite their simplicity, conventional approaches prove highly effective on suitable datasets.

Finding 2: Deep neural network approaches, such as DREAM, Beacon, Sets2Sets, and CLEA, lack consistent and stable performance. Notably, CLEA outperforms others on the TaFeng dataset by automatically capturing interactions between historical items and the target item. However, its superiority does not extend to the Instacart and Dunnhumby datasets, possibly due to dataset characteristics, specifically the lower average basket size in TaFeng. Sets2Sets, utilizing a repeated pattern based on an encoder-decoder architecture, surpasses DREAM, Beacon, and even CLEA in the Dunnhumby dataset, highlighting the prevalence of repeated purchases in the NBR task.

Finding 3: Conventional approaches generally outperform deep neural network-based methods across most datasets. However, the performance of different approaches varies significantly across datasets, with no single approach excelling on all of them. Analyzing performance on three datasets using F1-Score and NDCG, as presented in Table II, reveals Sets2Sets, UP-CF@r, TIFUKNN, and CLEA as notable performers. While Sets2Sets and CLEA exhibit superior performance on the Tafeng dataset compared to UP-CF@r and TIFUKNN, they are outperformed on Instacart and Dunnhumby datasets. This discrepancy suggests that Sets2Sets and CLEA may be more suitable for datasets with shorter baskets, whereas UP-CF@r and TIFUKNN excel in recommending next baskets for datasets with longer baskets.

VII Conclusion

This paper systematically reviews NBR task literature, delineating distinctions from traditional RSs and articulating the NBR problem statement. We categorize existing work into conventional, latent representation, and deep neural network approaches, analyzing their pros and cons. To enhance understanding and evaluation, we scrutinize popular NBR algorithms and metrics, conducting comparative performance assessments with unified experimental settings. Our unified framework offers a valuable reference for NBR research, aiming to provide clarity and fairness in the evaluation of NBRs for readers and researchers alike.