Previous work in transfer learning has focused on learning from data-rich domains and transferring knowledge to data-sparse regions or underrepresented classes (Gupta et al., 2016; Jean et al., 2016; Zhu et al., 2020).

In e-commerce, we aim to learn from popular products to improve the performance of new or less popular products. Multi-task learning has also been used to enhance model performance on data-sparse tasks (Bishop et al., 2014; Chang et al., 2019; Fiot and Dinuzzo, 2015; Pan and Yang, 2009). Conventional transfer learning methods learn transferable latent factors between one source domain and one target domain (Long et al., 2013; Gong et al., 2012; Tzeng et al., 2017; Long et al., 2015). In our work, we focus on adopting meta-learning techniques to learn from various tasks and then adapt them to unseen tasks in demand prediction.

Meta-learning has been applied to various retail and demand prediction tasks, with an emphasis on learning from diverse data sources and adapting to new tasks with limited data. For instance, Li et al. (2020) employed meta-learning to predict demand in retail settings, demonstrating the effectiveness of meta-learning in capturing complex patterns across diverse scenarios. Similarly, Wang et al. (2019) applied meta-learning to online retail data, highlighting the potential for meta-learning in e-commerce applications.

In the time series-related problems, Oreshkin et al. (Oreshkin et al., 2020) briefly discusses the relation between neural time series prediction and meta-learning (Oreshkin et al., 2020). Yao et al. incorporate the gradient-based meta-learning with a region functionality based memory (Yao et al., 2019) for spatiotemporal prediction. However, this method relies on the spatial semantic correlations between tasks, which limits its applicability to our problem.

Our work contributes to the problem of learning customer demand for new products with few historical data points. Previous works have suggested comparing the features of new products to existing ones (Ferreira et al., 2016; Baardman et al., 2017), or efficient methods for eliciting additional information (Cao and Zhang, 2021; Ma and Simchi-Levi, 2022). Our paper assumes that sales have already been observed at limited prices and leverages more information from other related products and environments as proxy data.

Meta-learning methods for few-shot learning can be broadly categorized into two main approaches: metric-learning-based and optimization-based. Metric-learning-based approaches focus on establishing similarity or dissimilarity between classes, as demonstrated by works such as Prototypical Networks (Snell et al., 2017), Matching Networks (Vinyals et al., 2016), and Relation Networks (Sung et al., 2017). These methods aim to learn representations that facilitate comparisons between few-shot examples and known classes. On the other hand, optimization-based approaches aim to learn a good initialization point that can quickly adapt to new tasks with minimal parameter updates. Prominent examples of this category include Model-Agnostic Meta-Learning (MAML) (Finn et al., 2017a), Reptile (Nichol et al., 2018), and Meta-SGD (Li et al., 2017b). These methods have been further extended by advanced techniques such as Task-Specific Adaptation (TSA) (Zhou et al., 2020) and Multi-Modal Model-Agnostic Meta-Learning (MUMOMAML) (Vuorio et al., 2019). These optimization-based approaches enhance the adaptability and robustness of the learned models across diverse tasks and domains.

Our method falls into the domain of optimization-based approaches.

Deep learning models have been extensively explored for time series forecasting especially those with the Seq2Seq structure (Sutskever et al., 2014), which involves learning an encoder to transform various inputs into fixed-length hidden states for producing forecasts. Recent developments include DeepAR (Salinas et al., 2020), TFT (Lim et al., 2019) and MQ-Forecasters (Wen et al., 2017; Eisenach et al., 2020). However, these methods do not account for cross-observation information, which becomes important in many practical applications. As a result, Graph Neural Networks (GNNs) have rapidly emerged as a promising framework to address this issue by combining temporal processing with graph convolution to augment the learning of individual time series (Kipf and Welling, 2017; Li et al., 2017a; Wu et al., 2020; Shang et al., 2021). A popular family of methods propose graph structure learning for the joint inference of a latent structure through GNN while forecasting (Kipf et al., 2018; Wu et al., 2020). However, they suffer limitations in scaling to large datasets. A scalable approach recently introduced by Yang et al. (Yang et al., 2023) uses predefined graphs as data augmentations rather than enabling graph structure learning, which demonstrates not only to scale to graphs over millions of nodes but also shows substantially improving model performance, especially for cold-start problems when data is scarce.

Our work builds upon these foundations by specifically applying meta-learning and few-shot learning techniques to the demand forecasting problem, with the goal of improving the adaptability and performance of models in this context. To the best of our knowledge, we are the first to study peak period demand prediction with limited records by borrowing relation-aware knowledge from other time periods. We focus on this domain, exploring the application of meta-learning for few-shot prediction and incorporating auxiliary information, such as proxy data from other related tasks, to improve model performance.

Suppose we have a set of tasks $\mathcal{T}$. For each task $t\in\mathcal{T}$, we observe training samples $\{(x_{k}^{(t)},y_{k}^{(t)})\}_{k=1}^{n_{t}}$, where $n_{t}$ is the sample size for the task $t$. In addition, for each task, we observe a task-specific feature $v_{t}\in\mathbb{R}^{r}$. For the simplicity of presentation, we assume $v_{t}\in[0,1]^{r}$. We denote the set of training domains by $\mathcal{D}^{tr}$ and assume there are ${T}=|\mathcal{T}|$ training tasks. Following equation (1), we assume that for each task $t$, the outcome prediction function $g$ takes the form of $y=g_{t}(x)+\epsilon:=h_{t}(f(x))+\epsilon$, where $h_{t}$ is the base-learner that depends on individual task $t$, $f$ is the meta-learner, and $\epsilon$ is a noise term which is assumed to be sub-Gaussian with mean 0 and variance $\sigma^{2}$.

Following Section 4, for each task $t\in\mathcal{T}$, we construct the proxy data $Z_{t}$ by including all similar tasks $t^{\prime}$ such that $\|v_{t^{\prime}}-v_{t}\|\leq h$ for some threshold parameter $h>0$. Then similarly, for the test task $\widetilde{t}$, the outcome prediction function $\widehat{g}_{\widetilde{t}}$ is computed as $\widehat{g}_{\widetilde{t}}(x)=\widehat{h}_{\widetilde{t}}(f(x)),$ where $\widehat{h}_{\widetilde{t}}(\cdot)=\frac{\sum_{t\in\mathcal{T}}w(\widetilde{t},t)\widehat{h}_{i}(\cdot)}{\sum_{t\in\mathcal{T}}w(\widetilde{t},t)}$, with the weight $w(\widetilde{t},t)=\bm{1}\{\|v_{\widetilde{t}}-v_{t}\|\leq h\}$. In the case where the denominator is $0$, we define $\widehat{h}_{\widetilde{t}}=0$.

To facilitate the theoretical analysis, we first assume that the distance between task-specific features indeed captures the similarity of tasks: there exists a universal constant $C$, such that $\|h_{t_{1}}-h_{t_{2}}\|_{\infty}\leq C\cdot\|v_{t_{1}}-v_{t_{2}}\|.$

In addition, we assume that for each training domain $t$, $\widehat{h}_{t}$ is well learned such that $\operatorname*{\mathbb{E}}\big{[}\big{(}\widehat{h}_{t}(f(x))-h_{t}(f(x))\big{)}^{2}\big{]}=O(\frac{C(\mathcal{H})}{n_{t}})$, where $C(\mathcal{H})$ is the Rademacher complexity of the function class $\mathcal{H}$. We further assume $v_{t}$ has a positive density over $[0,1]^{r}$. Then, we have:

Theorem 1 suggests that the superiority of our algorithm comes from a better bias-variance trade-off. More concretely, the threshold $h$ tunes the trade-off for the excess risk. On the one hand, when we do not use relational data at all (corresponding to the case where $h=0$), the first term in equation 8, bias, is negligible, while the second term, variance, becomes dominant as the data is limited. As the excess risk of a single task is of order $(C(\mathcal{H})/n)^{1/2}$, our result implies that when $T$ is sufficiently large such that $T>\big{(}\frac{n}{C(\mathcal{H})}\big{)}^{r/2}$, our proposed method will overcome the potential bias by incorporating similar (but still different) tasks and better than learning with only one single-source task. On the other hand, if we simply use the standard ERM to pool all the data together (corresponding to $h=\infty$), although the variance becomes small, the bias would dominate in this case. The second part of the theorem suggests that one can efficiently incorporate the proxy data with a carefully chosen threshold. The proof of Theorem 1 is deferred to Appendix B.

The theoretical perspective we discussed is particularly pertinent to the context of demand prediction. Given the limited data available from high-stakes sales events, relying solely on this data for prediction (akin to $h=0$) can result in outcomes with substantial variance. Conversely, utilizing the entire historical dataset (corresponding to $h=\infty$) can introduce significant bias, given the marked differences between regular and high-stakes sales events. Our method harnesses GNNs to understand the relationships within historical data. This approach strikes an optimal balance in the bias-variance trade-off, leading to improved prediction accuracy.